Day_5_build_model_tutorial.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.12"
    },
    "colab": {
      "name": "buildmodel_tutorial.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Hemant-ML/Pytorch-Master/blob/main/Day_5_build_model_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GVSUKS7TZIOM"
      },
      "source": [
        "import os\n",
        "import torch\n",
        "from torch import nn\n",
        "from torch.utils.data import DataLoader\n",
        "from torchvision import datasets, transforms\n",
        "import numpy as np"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ImzAHomEZIOM"
      },
      "source": [
        "Get Device for Training\n",
        "-----------------------\n",
        "We want to be able to train our model on a hardware accelerator like the GPU, \n",
        "if it is available. Let's check to see if \n",
        "`torch.cuda.is available`, else we \n",
        "continue to use the CPU. \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pRvVJWfEZION",
        "outputId": "8732df08-ec12-484a-c8b5-caf1961f6f36"
      },
      "source": [
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "print(f'Using {device} device')"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using cuda device\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wu0BOQ1uZION"
      },
      "source": [
        "Define the Class\n",
        "-------------------------\n",
        "We define our neural network by subclassing ``nn.Module``, and \n",
        "initialize the neural network layers in ``__init__``. Every ``nn.Module`` subclass implements\n",
        "the operations on input data in the ``forward`` method. \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t249zLGDZIOO"
      },
      "source": [
        "class NeuralNetwork(nn.Module):\n",
        "  def __init__(self):\n",
        "    super(NeuralNetwork,self).__init__()\n",
        "    self.flatten=nn.Flatten()\n",
        "    self.linear_Relu = nn.Sequential(\n",
        "        nn.Linear(28*28,512),\n",
        "        nn.ReLU(),\n",
        "        nn.Linear(512,10),\n",
        "        nn.ReLU()\n",
        "    )\n",
        "  def forward(self,x):\n",
        "    x = self.flatten(x)\n",
        "    logits = self.linear_Relu(x)\n",
        "    return logits\n",
        "\n",
        "\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F1V3AiC-ZIOO"
      },
      "source": [
        "We create an instance of ``NeuralNetwork``, and move it to the ``device``, and print \n",
        "its structure.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gKUsBoSGZIOP",
        "outputId": "9ec0720b-02a7-4558-a7ee-43fc572b1733"
      },
      "source": [
        "model = NeuralNetwork().to(device)\n",
        "print(model)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "NeuralNetwork(\n",
            "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
            "  (linear_Relu): Sequential(\n",
            "    (0): Linear(in_features=784, out_features=512, bias=True)\n",
            "    (1): ReLU()\n",
            "    (2): Linear(in_features=512, out_features=10, bias=True)\n",
            "    (3): ReLU()\n",
            "  )\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0nFB3yleZIOP"
      },
      "source": [
        "To use the model, we pass it the input data. This executes the model's ``forward``,\n",
        "along with some `background operations`_. \n",
        "Do not call ``model.forward()`` directly!\n",
        "\n",
        "Calling the model on the input returns a 10-dimensional tensor with raw predicted values for each class.\n",
        "We get the prediction probabilities by passing it through an instance of the ``nn.Softmax`` module.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3V2i7mYEZIOQ",
        "outputId": "faa57832-5a34-42e1-b070-2a8038790952"
      },
      "source": [
        "X = torch.rand(1, 28, 28, device=device)\n",
        "logits = model(X) \n",
        "pred_probab = nn.Softmax(dim=1)(logits)\n",
        "y_pred = pred_probab.argmax(1)\n",
        "print(f\"Predicted class: {y_pred}\")"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Predicted class: tensor([3], device='cuda:0')\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xq8Dtcs_cOIW",
        "outputId": "ce4527b5-a444-4284-f8f7-aac40f27ea13"
      },
      "source": [
        "X.size()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 28, 28])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NIdIooyoZIOQ"
      },
      "source": [
        "--------------\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3sGhTb_-ZIOQ"
      },
      "source": [
        "Model Layers\n",
        "-------------------------\n",
        "\n",
        "Lets break down the layers in the FashionMNIST model. To illustrate it, we \n",
        "will take a sample minibatch of 3 images of size 28x28 and see what happens to it as \n",
        "we pass it through the network. \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w5qHR2-PZIOR",
        "outputId": "f48c825f-baee-4f21-d026-d6191d84e654"
      },
      "source": [
        "input_image = torch.rand(3,28,28)\n",
        "input_image.view(3,-1).size()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([3, 784])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qAWBqVTuZIOR"
      },
      "source": [
        "``nn.Flatten``\n",
        "\n",
        "We initialize the `nn.Flatten `_ \n",
        "layer to convert each 2D 28x28 image into a contiguous array of 784 pixel values (\n",
        "the minibatch dimension (at dim=0) is maintained).\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UB4_SRB4ZIOS",
        "outputId": "387d4521-6b8d-4ae0-fb80-9b8dd0ab9730"
      },
      "source": [
        "flatten = nn.Flatten()\n",
        "flat_image = flatten(input_image)\n",
        "print(flat_image.size())"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([3, 784])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2A20Iw2OZIOT"
      },
      "source": [
        "``nn.Linear``\n",
        "\n",
        "The `linear layer` \n",
        "is a module that applies a linear transformation on the input using its stored weights and biases.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ckw6yaKvZIOT",
        "outputId": "b8f81483-e786-4391-fa1a-aecd07fb4cf4"
      },
      "source": [
        "layer1 = nn.Linear(in_features=28*28, out_features=20)\n",
        "hidden1 = layer1(flat_image)\n",
        "print(hidden1.size())"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([3, 20])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-sWISBKsZIOT"
      },
      "source": [
        "``nn.ReLU``\n",
        "Non-linear activations are what create the complex mappings between the model's inputs and outputs.\n",
        "They are applied after linear transformations to introduce *nonlinearity*, helping neural networks\n",
        "learn a wide variety of phenomena.\n",
        "\n",
        "In this model, we use `nn.ReLU` between our\n",
        "linear layers, but there's other activations to introduce non-linearity in your model.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rl_HamZ_ZIOU"
      },
      "source": [
        "print(f\"Before ReLU: {hidden1}\\n\\n\")\n",
        "hidden1 = nn.ReLU()(hidden1)\n",
        "print(f\"After ReLU: {hidden1}\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lh5nG61OZIOV"
      },
      "source": [
        "``nn.Sequential``\n",
        "\n",
        "`nn.Sequential` is an ordered \n",
        "container of modules. The data is passed through all the modules in the same order as defined. You can use\n",
        "sequential containers to put together a quick network like ``seq_modules``.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Wh_m8GwZIOW"
      },
      "source": [
        "seq_modules = nn.Sequential(\n",
        "    flatten,\n",
        "    layer1,\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(20, 10)\n",
        ")\n",
        "input_image = torch.rand(3,28,28)\n",
        "logits = seq_modules(input_image)"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fAhgoxZOZIOX"
      },
      "source": [
        "nn.Softmax\n",
        "\n",
        "The last linear layer of the neural network returns `logits` - raw values in [-infty, infty] - which are passed to the\n",
        "`nn.Softmax >`_ module. The logits are scaled to values \n",
        "[0, 1] representing the model's predicted probabilities for each class. ``dim`` parameter indicates the dimension along \n",
        "which the values must sum to 1. \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lu6xoL1lZIOX"
      },
      "source": [
        "softmax = nn.Softmax(dim=1)\n",
        "pred_probab = softmax(logits)"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OpAYEcU_ZIOY"
      },
      "source": [
        "Model Parameters\n",
        "-------------------------\n",
        "Many layers inside a neural network are *parameterized*, i.e. have associated weights \n",
        "and biases that are optimized during training. Subclassing ``nn.Module`` automatically \n",
        "tracks all fields defined inside your model object, and makes all parameters \n",
        "accessible using your model's ``parameters()`` or ``named_parameters()`` methods. \n",
        "\n",
        "In this example, we iterate over each parameter, and print its size and a preview of its values. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oE-GVKHbZIOY",
        "outputId": "f4a25dab-5200-47b1-9c6e-ca8b03fbd2a7"
      },
      "source": [
        "print(\"Model structure: \", model, \"\\n\\n\")\n",
        "\n",
        "for name, param in model.named_parameters():\n",
        "    print(f\"Layer: {name} | Size: {param.size()} | Values : {param[:2]} \\n\")"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model structure:  NeuralNetwork(\n",
            "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
            "  (linear_Relu): Sequential(\n",
            "    (0): Linear(in_features=784, out_features=512, bias=True)\n",
            "    (1): ReLU()\n",
            "    (2): Linear(in_features=512, out_features=10, bias=True)\n",
            "    (3): ReLU()\n",
            "  )\n",
            ") \n",
            "\n",
            "\n",
            "Layer: linear_Relu.0.weight | Size: torch.Size([512, 784]) | Values : tensor([[-0.0335,  0.0250,  0.0209,  ...,  0.0153,  0.0081,  0.0167],\n",
            "        [ 0.0095, -0.0353,  0.0291,  ...,  0.0094,  0.0332, -0.0098]],\n",
            "       device='cuda:0', grad_fn=<SliceBackward>) \n",
            "\n",
            "Layer: linear_Relu.0.bias | Size: torch.Size([512]) | Values : tensor([ 0.0289, -0.0017], device='cuda:0', grad_fn=<SliceBackward>) \n",
            "\n",
            "Layer: linear_Relu.2.weight | Size: torch.Size([10, 512]) | Values : tensor([[ 0.0230, -0.0332, -0.0031,  ..., -0.0134,  0.0198,  0.0206],\n",
            "        [-0.0008,  0.0233, -0.0377,  ...,  0.0204, -0.0129, -0.0059]],\n",
            "       device='cuda:0', grad_fn=<SliceBackward>) \n",
            "\n",
            "Layer: linear_Relu.2.bias | Size: torch.Size([10]) | Values : tensor([ 0.0321, -0.0061], device='cuda:0', grad_fn=<SliceBackward>) \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y8Oh1MSNZIOY"
      },
      "source": [
        "--------------\n",
        "\n",
        "\n"
      ]
    }
  ]
}