Alpaca Image Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bfc25b03",
   "metadata": {},
   "source": [
    "# Image Classification for Alpaca Animal"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "94836eff",
   "metadata": {},
   "source": [
    "In this project, we will classify the animal as Alpaca or not Aplaca based on the features found in image using Custom CNN modek. For this task we will develop a novel approach where the conv2d layer of a Convolution Neural Network will be customised. The CNN built only have one layer.\n",
    "\n",
    "A **Convolutional Neural Network (CNN)** is a type of deep learning neural network designed for processing structured grid data, such as images and videos. It's particularly well-suited for tasks like image recognition and computer vision. CNNs are characterized by their use of convolutional layers, pooling layers, and fully connected layers. Here's a brief explanation of these key components:\n",
    "\n",
    "**Convolutional Layers:** These layers apply small filters (also known as kernels) to small regions of the input data. By sliding these filters across the input, the network can detect patterns and features, such as edges, corners, and textures, at various scales.\n",
    "\n",
    "**Pooling Layers:** Pooling layers reduce the spatial dimensions of the feature maps produced by convolutional layers. Max pooling, for example, selects the maximum value from a small region of the feature map. This helps reduce computational complexity and focus on the most important features.\n",
    "\n",
    "**Fully Connected Layers:** After multiple convolutional and pooling layers, CNNs typically have one or more fully connected layers. These layers process the high-level features extracted from the earlier layers and produce the final output, which might be class probabilities for image classification tasks.\n",
    "\n",
    "CNNs have revolutionized computer vision tasks by automatically learning hierarchical representations from raw pixel data, making them highly effective for tasks like image classification, object detection, and image segmentation. Their ability to capture local and global patterns within images has led to their widespread use in various applications beyond computer vision as well.\n",
    "\n",
    "## Traditional CNN vs Proposed CNN\n",
    "The key distinction between our proposed CNN and the traditional CNN lies in how we handle weight parameters. In a standard CNN, a fixed kernel is applied across the entire input data. However, in our novel approach, we use a weight matrix. This weight matrix is used dynamically, and for each operation, a locally positioned and orientation-specific kernel is employed to calculate the weighted sum. This approach allows for more flexibility and adaptability in capturing features from the input data.\n",
    "\n",
    "## Dataset\n",
    "\n",
    "We have 2 folders having different images of animals. One folder have all Alpaca's images and other folder have other animals' images. We will first load those images into our file, create a dataframe and label the images as 'alpaca' and 'not alpaca'.\n",
    "\n",
    "## Preprocess Data\n",
    "\n",
    "We will convert the images from RGB to Grayscale and normalize them using cv2 library build for computer vision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8c5b4743",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "# Step 1: Load and preprocess image data\n",
    "\n",
    "input1 = 'alpaca/'\n",
    "filename1 = []\n",
    "for filename in os.listdir(input1):                   # reading the directiry\n",
    "    filename1.append(input1 + filename)               # storing the file name data\n",
    "\n",
    "input2 = 'not alpaca/'\n",
    "filename2 = []\n",
    "for filename in os.listdir(input2):\n",
    "    filename2.append(input2 + filename) \n",
    "    \n",
    "\n",
    "# Load and preprocess images\n",
    "def load_and_preprocess_image(filename):\n",
    "    image = cv2.imread(filename)                                                #reading the image data\n",
    "    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)                              # Convert from BGR to RGB \n",
    "    image = cv2.resize(image, (32, 32))                                         # Resize to 32x32\n",
    "    image = image.astype('float32') / 255.0                                     # Normalize to [0, 1]\n",
    "    return image\n",
    "\n",
    "X1 = np.array([load_and_preprocess_image(filename) for filename in filename1])\n",
    "Y1 = np.ones(len(X1), dtype=np.int32)                                           # generate 1\n",
    "\n",
    "X2 = np.array([load_and_preprocess_image(filename) for filename in filename2])\n",
    "Y2 = np.zeros(len(X2), dtype=np.int32)                                          # generate 2\n",
    "\n",
    "# Concatenate and split data\n",
    "X = np.concatenate((X1, X2), axis=0)                                            # combining the training data\n",
    "Y = np.concatenate((Y1, Y2), axis=0)                                            # combining the testing data\n",
    "\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.20, random_state=42)\n",
    "X_test, X_val, Y_test, Y_val = train_test_split(X_test, Y_test, test_size=0.50, random_state=42)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48cd11e4",
   "metadata": {},
   "source": [
    "### Custom Layer Implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8f646b97",
   "metadata": {},
   "outputs": [],
   "source": [
    "# single layer custom model\n",
    "\n",
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras import layers, models\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import numpy as np\n",
    "import torch.nn.functional as F\n",
    "\n",
    "#labels = F.one_hot(labels, num_classes)\n",
    "\n",
    "\n",
    "class ConvolutionalLayer(nn.Module):\n",
    "    def __init__(self, input_size, num_channels, filter_size):\n",
    "        super(ConvolutionalLayer, self).__init__()\n",
    "        self.input_size = input_size\n",
    "        self.num_channels = num_channels\n",
    "        self.filter_size = filter_size\n",
    "        #make weights matrix the same size as the input\n",
    "        self.weight_matrix = nn.Parameter(torch.randn(batch_size, 32, 32, 3))\n",
    "        self.output_size = (input_size[0] - filter_size[0] + 1, input_size[1] - filter_size[1] + 1)\n",
    "        self.output_feature_map = torch.zeros((self.num_channels, self.output_size[0], self.output_size[1]))\n",
    "\n",
    "    def forward(self, input_feature_map):\n",
    "        batch_size = input_feature_map.size(0)\n",
    "        output_feature_maps = []\n",
    "        #in_channels = input_feature_map.shape[1]        # Check if the number of filters matches the number of input channels\n",
    "        #assert self.num_channels == in_channels, \"Number of filters must match the number of input channels\"\n",
    "    #following line to check if input_fm and weights same shape   \n",
    "    # print('input_f_m: ', np.shape(input_feature_map), 'weights: ', np.shape(self.weight_matrix))\n",
    "        for i in range(batch_size):\n",
    "            output_feature_map = torch.zeros((self.num_channels, self.output_size[0], self.output_size[1]))\n",
    "            for k in range(self.num_channels):\n",
    "                for j in range(self.output_size[0]):\n",
    "                    for l in range(self.output_size[1]):\n",
    "                        #the same receptive field is applied to the weights as the input\n",
    "                        receptive_field = input_feature_map[i, :, j:j+self.filter_size[0], l:l+self.filter_size[1]]\n",
    "                        receptive_field_weight = self.weight_matrix[i, :, j:j+self.filter_size[0], l:l+self.filter_size[1]]\n",
    "                       \n",
    "                       # print(l, 'rf:', np.shape(receptive_field), 'rfw: ', np.shape(receptive_field_weight))\n",
    "                        weighted_output = torch.sum(receptive_field * receptive_field_weight, dim=(1,2))\n",
    "                        output_feature_map[k, j, l] = weighted_output[k]\n",
    "            output_feature_maps.append(output_feature_map)\n",
    "        output_feature_maps = torch.stack(output_feature_maps, dim=0)\n",
    "        return output_feature_maps\n",
    "    \n",
    "    def backward(self, grad_output):\n",
    "        batch_size = grad_output.size(0)\n",
    "        grad_input = torch.zeros((batch_size, self.input_size[0], self.input_size[1], self.filter_size[0], self.filter_size[1]), device=self.weight_matrix.device)\n",
    "        grad_weight = torch.zeros_like(self.weight_matrix)\n",
    "        for i in range(batch_size):\n",
    "            for k in range(self.num_channels):\n",
    "                for j in range(self.output_size[0]):\n",
    "                    for l in range(self.output_size[1]):\n",
    "                        # compute the gradient of the output w.r.t. the receptive field\n",
    "                        grad_weight[k] += grad_output[i, k, j, l] * self.input_feature_map[i, :, j:j+self.filter_size[0], l:l+self.filter_size[1]]\n",
    "                        # compute the gradient of the output w.r.t. the input feature map\n",
    "                        grad_input[i, :, j:j+self.filter_size[0], l:l+self.filter_size[1]] += grad_output[i, k, j, l] * self.weight_matrix[k]\n",
    "        self.weight_matrix.grad = torch.sum(grad_weight, dim=0, keepdim=True)\n",
    "        return grad_input\n",
    "    \n",
    "\n",
    "# Define the CNN architecture\n",
    "class CNN(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(CNN, self).__init__()\n",
    "        self.conv1 = ConvolutionalLayer(input_size=(16,16), num_channels=16, filter_size=(3, 3))\n",
    "        #self.conv2 = ConvolutionalLayer(input_size=(32,32), num_channels=12, filter_size=(3, 3))\n",
    "        #self.conv3 = ConvolutionalLayer(input_size=(32,32), num_channels=8, filter_size=(3, 3))\n",
    "        self.pool = nn.MaxPool2d(2,2)\n",
    "        self.activation = nn.ReLU()\n",
    "        self.fc1 = nn.Linear(64 * 8 * 8, 128)\n",
    "        self.fc2 = nn.Linear(128, 2)\n",
    "        self.output_activation = nn.Softmax(dim=1)\n",
    "\n",
    "# leelakrishna's idea\n",
    "    def forward(self, x):\n",
    "        print('Input shape:', x.shape)\n",
    "        x = self.pool(nn.functional.relu(self.conv1(x)))\n",
    "       # print('Input shape:', x.shape)\n",
    "       # x = self.pool(nn.functional.relu(self.conv2(x)))\n",
    "       # print('Input shape:', x.shape)\n",
    "       # x = self.pool(nn.functional.relu(self.conv2(x)))\n",
    "       # print('Input shape:', x.shape)\n",
    "        #x = x.view(-1, 64 * 8 * 8)\n",
    "        x = x.view(x.size(0), -1)\n",
    "        print('Input shape:', x.shape)\n",
    "        x = nn.functional.relu((x))\n",
    "        #print('Input shape:', x.shape)\n",
    "        #x = self.fc2(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4aea88ad",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [1/10], Train Loss: 6.6210, Train Acc: 0.0115, Val Loss: 6.5668, Val Acc: 0.0000\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [2/10], Train Loss: 6.4112, Train Acc: 0.0268, Val Loss: 6.4547, Val Acc: 0.0000\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [3/10], Train Loss: 6.3420, Train Acc: 0.0383, Val Loss: 6.3528, Val Acc: 0.0303\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [4/10], Train Loss: 6.4029, Train Acc: 0.0307, Val Loss: 6.2602, Val Acc: 0.0303\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [5/10], Train Loss: 6.1638, Train Acc: 0.0536, Val Loss: 6.1681, Val Acc: 0.0303\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [6/10], Train Loss: 6.0489, Train Acc: 0.0766, Val Loss: 6.0808, Val Acc: 0.0606\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [7/10], Train Loss: 6.0542, Train Acc: 0.0651, Val Loss: 5.9848, Val Acc: 0.0606\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [8/10], Train Loss: 5.9138, Train Acc: 0.0613, Val Loss: 5.8961, Val Acc: 0.0606\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [9/10], Train Loss: 5.7591, Train Acc: 0.0843, Val Loss: 5.8120, Val Acc: 0.0606\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([1, 32, 32, 3])\n",
      "Input shape: torch.Size([1, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([5, 32, 32, 3])\n",
      "Input shape: torch.Size([5, 784])\n",
      "Input shape: torch.Size([3, 32, 32, 3])\n",
      "Input shape: torch.Size([3, 784])\n",
      "Epoch [10/10], Train Loss: 5.7304, Train Acc: 0.0920, Val Loss: 5.7341, Val Acc: 0.0606\n"
     ]
    }
   ],
   "source": [
    "# Single Layer implementation\n",
    "\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torch.utils.data import DataLoader, TensorDataset\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "# Convert the data to PyTorch tensors and create PyTorch datasets\n",
    "train_dataset = TensorDataset(torch.Tensor(X_train), torch.Tensor(Y_train))\n",
    "val_dataset = TensorDataset(torch.Tensor(X_val), torch.Tensor(Y_val))\n",
    "\n",
    "# Define the batch size for training and validation\n",
    "batch_size = 5 #16\n",
    "\n",
    "# Create data loaders for training and validation\n",
    "train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "val_loader = DataLoader(val_dataset, batch_size=batch_size)\n",
    "\n",
    "# Define the model\n",
    "model = CNN()\n",
    "\n",
    "# Define the loss function and optimizer\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
    "num_classes=2\n",
    "# Train the model\n",
    "num_epochs = 10\n",
    "for epoch in range(num_epochs):\n",
    "    # Training\n",
    "    train_loss = 0.0\n",
    "    train_acc = 0.0\n",
    "    model.train() # Set the model to training mode\n",
    "    for images, labels in train_loader:\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(images)\n",
    "        labels = labels.long()\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        train_loss += loss.item() * images.size(0)\n",
    "        _, preds = torch.max(outputs, 1)\n",
    "        train_acc += torch.sum(preds == labels.long())\n",
    "    train_loss /= len(train_loader.dataset)\n",
    "    train_acc = train_acc.float() / len(train_loader.dataset)\n",
    "\n",
    "    # Validation\n",
    "    val_loss = 0.0\n",
    "    val_acc = 0.0\n",
    "    model.eval() # Set the model to evaluation mode\n",
    "    with torch.no_grad():\n",
    "        for images, labels in val_loader:\n",
    "            outputs = model(images)\n",
    "            loss = criterion(outputs, labels.long())\n",
    "            val_loss += loss.item() * images.size(0)\n",
    "            _, preds = torch.max(outputs, 1)\n",
    "            val_acc += torch.sum(preds == labels.long())\n",
    "        val_loss /= len(val_loader.dataset)\n",
    "        val_acc = val_acc.float() / len(val_loader.dataset)\n",
    "\n",
    "    # Print the loss and accuracy for this epoch\n",
    "    print('Epoch [{}/{}], Train Loss: {:.4f}, Train Acc: {:.4f}, Val Loss: {:.4f}, Val Acc: {:.4f}'\n",
    "          .format(epoch+1, num_epochs, train_loss, train_acc, val_loss, val_acc))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eda207e3",
   "metadata": {},
   "source": [
    "The accuracy results obtained from this model are not good but we might be able to obtain better results further as we improvise the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b90a544e",
   "metadata": {},
   "source": [
    "# How to change the code to work over 1-Dimensional data? \n",
    "We would only need 1 for loop in our custom layer and that just goes across one row. The pooling layer would only be 1 \n",
    "dimension.\n",
    "\n",
    "# How to change the code to work over 3-Dimensional data?\n",
    "We would need another nested for loop in our custom layer to account for the extra dimension. The pooling layer would be 3 dimensions in the combined network."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dfc1d97",
   "metadata": {},
   "source": [
    "# References"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "990c05c8",
   "metadata": {},
   "source": [
    "https://towardsdatascience.com/building-a-convolutional-neural-network-from-scratch-using-numpy-a22808a00a40\n",
    "\n",
    "https://pyimagesearch.com/2021/07/19/pytorch-training-your-first-convolutional-neural-network-cnn/\n",
    "\n",
    "https://medium.com/thecyphy/train-cnn-model-with-pytorch-21dafb918f48\n",
    "\n",
    "https://www.analyticsvidhya.com/blog/2019/10/building-image-classification-models-cnn-pytorch/\n",
    "\n",
    "https://www.tensorflow.org/tutorials/images/cnn\n",
    "\n",
    "https://www.analyticsvidhya.com/blog/2020/09/overfitting-in-cnn-show-to-treat-overfitting-in-convolutional-neural-networks/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5446690",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}