lab-03-2-minimizing_cost_gradient_update.ipynb

{
    "nbformat_minor": 2, 
    "cells": [
        {
            "source": "---\n* *** \uc6d0\ubcf8 \uc18c\uc2a4\ub294 \ud64d\ucf69\uacfc\uae30\ub300 \uae40\uc131\ud6c8 \uad50\uc218\ub2d8\uc758  [Github link](https://github.com/hunkim/DeepLearningZeroToAll)\ub97c \ucc38\uc870\ud558\uc138\uc694 ***     \n* *** Watson studio notebook \uc5d0\uc11c \uc791\uc5c5 \uac00\ub2a5\ud558\ub3c4\ub85d \uc218\uc815\ud558\uc600\uc73c\uba70***\n* *** \ud55c\uae00\ub85c \uc124\uba85\ub41c \ubd80\ubd84\uc740 \uc81c\uac00 \uc2a4\ud130\ub514 \ud558\uba74\uc11c \uc774\ud574\ud55c \ub0b4\uc6a9\uc744 \ucd94\uac00\ud55c \uac83\uc785\ub2c8\ub2e4. ***\n---", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "# Lab-03-2 Minimizing Cost Gradinet Update", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "* ***cost \uc218\uc2dd \ub9cc\ub4dc\ub294 \ub370 \uae4c\uc9c0\ub294 \ub3d9\uc77c\ud569\ub2c8\ub2e4\ub9cc, ***\n* ***gradient \ub9cc\ub4e4\uace0, learning rate \uacf1\ud558\ub294 \uacfc\uc815\uc744 \uc218\uc791\uc5c5\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \ubd05\ub2c8\ub2e4.***", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import tensorflow as tf\ntf.set_random_seed(777)  # for reproducibility"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "x_data = [1, 2, 3]\ny_data = [1, 2, 3]"
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Try to find values for W and b to compute y_data = W * x_data + b\n# We know that W should be 1 and b should be 0\n# But let's use TensorFlow to figure it out\nW = tf.Variable(tf.random_normal([1]), name='weight')\n\nX = tf.placeholder(tf.float32)\nY = tf.placeholder(tf.float32)"
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Our hypothesis for linear model X * W\nhypothesis = X * W\n\n# cost/loss function\ncost = tf.reduce_mean(tf.square(hypothesis - Y))"
        }, 
        {
            "source": "* ***gradient\ub97c manual\ub85c \uc218\uc2dd\ud654 \ud588\uc2b5\ub2c8\ub2e4. (tensorflow \uac00 \uc81c\uacf5\ud558\ub294 gradientdescent \ud568\uc218\ub864 \uc0ac\uc6a9\ud558\uc9c0 \uc54a\uace0)***   \n<img src='https://github.com/moreal70/TensorFlow-Lab-with-Watson-Studio/raw/master/images/gradient_manual.png' width='300')>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative\nlearning_rate = 0.3\ngradient = tf.reduce_mean((W * X - Y) * X)\ndescent = W - learning_rate * gradient\nupdate = W.assign(descent)"
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Launch the graph in a session.\nsess = tf.Session()\n# Initializes global variables in the graph.\nsess.run(tf.global_variables_initializer())"
        }, 
        {
            "source": "* ***update\ub85c \ucd5c\uc885 training \uc744 \uc218\ud589\ud569\ub2c8\ub2e4.***\n* ***\uadf8\ub9ac\uace0 cost\uc640 W \uac12\uc744 \ucd9c\ub825\ud574\ubd05\ub2c8\ub2e4.***", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "0 0.529374 [ 1.33680451]\n1 0.0847 [ 0.86527812]\n2 0.013552 [ 1.0538888]\n3 0.00216833 [ 0.97844446]\n4 0.00034694 [ 1.00862229]\n5 5.55116e-05 [ 0.99655104]\n6 8.8825e-06 [ 1.00137961]\n7 1.42126e-06 [ 0.99944812]\n8 2.27462e-07 [ 1.00022078]\n9 3.64346e-08 [ 0.99991167]\n10 5.85823e-09 [ 1.00003541]\n11 9.35742e-10 [ 0.99998581]\n12 1.45164e-10 [ 1.0000056]\n13 2.29612e-11 [ 0.99999779]\n14 4.24431e-12 [ 1.00000095]\n15 5.16328e-13 [ 0.99999964]\n16 9.9476e-14 [ 1.00000012]\n17 2.4869e-14 [ 0.99999994]\n18 0.0 [ 1.]\n19 0.0 [ 1.]\n20 0.0 [ 1.]\n"
                }
            ], 
            "source": "for step in range(21):\n    sess.run(update, feed_dict={X: x_data, Y: y_data})\n    print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))"
        }, 
        {
            "source": "* ***cost = 0, W=1\ub85c \uc218\ub834\ud558\ub294 \uac83\uc744 \ud655\uc778\ud558\uc168\ub2e4\uba74, \uc218\uc791\uc5c5\uc73c\ub85c \uacc4\uc0b0\ud558\uac8c \ub9de\ub2e4\ub294 \uc758\ubbf8\uc785\ub2c8\ub2e4.***\n* ***learning rate \ub97c \ud55c\ubc88 \uc218\uc815\ud574\uc11c \ub3cc\ub824\ubcf4\uc138\uc694 --> \ud558\uac15\uc18d\ub3c4\ub97c \uacb0\uc815\ud569\ub2c8\ub2e4. \ud06c\uae30\uc5d0 \ub530\ub77c\uc11c\ub294 +- \ub97c \uc654\ub2e4\uac14\ub2e4 \ud560\uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. ***", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}