WideResNet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wiTQibprRnlj"
   },
   "source": [
    "# **Cifar10 image classification**\n",
    "******************************************\n",
    "**점수관련**  \n",
    "본 프로젝트는 Accuracy 점수와 보고서를 성적에 반영할 예정입니다. 평가 항목은 아래와 같습니다. \n",
    "\n",
    "A. 결과 (40%)\n",
    "- Metric 성능 :(이미지 분류 - Accuracy)\n",
    "\n",
    "B. 신규성 (30%)\n",
    "- Network 변경 내용 - **필수**\n",
    "- 성능 개선 및 overfitting 방지 시도 (ex. train validation split)\n",
    "\n",
    "C. 이론적 근거 (20%)\n",
    "- 기존 baseline과의 차별점 \n",
    "- 성능 개선을 위한 시도와 이유\n",
    "\n",
    "D. 보고서 완성도 (10%)\n",
    "\n",
    "******************************************\n",
    "**보고서**\n",
    "1. 연구 목적\n",
    "2. 모델 구조\n",
    "3. 실험 내용\n",
    "4. 실험 결과\n",
    "5. 고찰 및 결론\n",
    "6. Colab 파일 (동작 가능여부)\n",
    "\n",
    "제출하실 파일은 **주피터 노트북 파일**('.ipynb',파일>다운로드)과 **결과 보고서**(pdf)입니다.\n",
    "******************************************\n",
    "**코드 검증**  \n",
    "- 코드 성능 평가(metric)는 torchvision.datasets에 있는 cifar10의 testset을 사용해 Accuracy로 평가합니다.\n",
    "- pretrain 모델 사용 불가능합니다.\n",
    "\n",
    "또한 Random 라이브러리 사용시에 seed 고정하는 등 재현을 고려하시고 코딩부탁드립니다. \n",
    "******************************************\n",
    "**GPU 사용 법**  \n",
    "런타임 > 런타임 유형 변경 > 하드웨어 가속기에서 GPU를 선택하면 GPU를 사용 할 수 있습니다.  \n",
    "******************************************\n",
    "**Colab 사용시 유의사항**  \n",
    "12시간 단위로 가상머신을 사용할 수 있고 12시간이 지나면 모든 파일과 작업로그들이 초기화 됩니다. 또한, 12시간 넘게 GPU를 사용하기는 어려우니 일찍 시작하셔서 틈틈히 실험해보시는 것을 추천드립니다.\n",
    "\n",
    "\n",
    "******************************************\n",
    "**Q?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CTVqQzTrIwTx"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "NtweS09sMWoh"
   },
   "outputs": [],
   "source": [
    "# Training\n",
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "from torch.utils.data import DataLoader\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from tqdm import tqdm as tqdm\n",
    "import time\n",
    "import os\n",
    "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\"\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NV_iVhYhJmMB",
    "outputId": "33447616-9cb4-4697-d852-37ca3e9f7b6b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sun Oct 31 13:40:49 2021       \r\n",
      "+-----------------------------------------------------------------------------+\r\n",
      "| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |\r\n",
      "|-------------------------------+----------------------+----------------------+\r\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\r\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\r\n",
      "|                               |                      |               MIG M. |\r\n",
      "|===============================+======================+======================|\r\n",
      "|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |\r\n",
      "| 35%   35C    P0    60W / 260W |     21MiB / 11016MiB |     25%      Default |\r\n",
      "|                               |                      |                  N/A |\r\n",
      "+-------------------------------+----------------------+----------------------+\r\n",
      "|   1  NVIDIA GeForce ...  Off  | 00000000:03:00.0 Off |                  N/A |\r\n",
      "|  0%   37C    P8    12W / 340W |   2019MiB / 10018MiB |      0%      Default |\r\n",
      "|                               |                      |                  N/A |\r\n",
      "+-------------------------------+----------------------+----------------------+\r\n",
      "                                                                               \r\n",
      "+-----------------------------------------------------------------------------+\r\n",
      "| Processes:                                                                  |\r\n",
      "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\r\n",
      "|        ID   ID                                                   Usage      |\r\n",
      "|=============================================================================|\r\n",
      "+-----------------------------------------------------------------------------+\r\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 120,
     "referenced_widgets": [
      "1996b2035d394feb8ddea161be66e787",
      "9acbedbeaa724ce08dffeddc71222505",
      "2d011007d89c4aad96b59ab60216ab9b",
      "3d976f276c2242bdb3dc84857838b627",
      "69306e2c35aa46e8b951227a9c3e9696",
      "a0b54dda21fc4fc8b8262d899d385dfe",
      "117786fb9ad04bbfba9000efc67417ca",
      "30bffec40690426489b084343473f2bb",
      "a7f8572bbbcb42c39c2bd7287d4ccdba",
      "665730bcefbd4ec79ff322d8fdcac2fe",
      "4c74f0cd1e2d461a8b64740fa8b61c29"
     ]
    },
    "id": "pmO0w741Ilao",
    "outputId": "a3c65b18-5958-4849-de9c-0872f093ecd3",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0 is available\n",
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "# 하이퍼 파라미터\n",
    "EPOCH = 200\n",
    "batch_size = 64\n",
    "learning_rate = 0.1\n",
    "momentum = 0.9\n",
    "weight_decay = 5e-4\n",
    "\n",
    "device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
    "print(f'{device} is available')\n",
    "\n",
    "# 분류 Class list\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n",
    "\n",
    "\n",
    "# 이미지 전처리\n",
    "transform = transforms.Compose([\n",
    "    transforms.RandomHorizontalFlip(),\n",
    "    transforms.RandomCrop(32, padding=4),\n",
    "    transforms.ToTensor(),\n",
    "    transforms.Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))\n",
    "])\n",
    "\n",
    "# Dataset. 변경 불가\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "Hhzrl2X5I938"
   },
   "outputs": [],
   "source": [
    "class WideModule(nn.Module):\n",
    "    def __init__(self, in_planes, planes, dropout_rate, stride=1):\n",
    "        super(WideModule, self).__init__()\n",
    "        self.bn1 = nn.BatchNorm2d(in_planes)\n",
    "        self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, padding=1, bias=True)\n",
    "        self.dropout = nn.Dropout(p=dropout_rate)\n",
    "\n",
    "        self.bn2 = nn.BatchNorm2d(planes)\n",
    "        self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=True)\n",
    "\n",
    "        self.shortcut = nn.Sequential()\n",
    "        if stride != 1 or in_planes != planes:  # Shortcut 부분에서 input과 output Feature Map이 달라진 경우 보정시켜줌.\n",
    "            self.shortcut = nn.Sequential(\n",
    "                nn.Conv2d(in_planes, planes, kernel_size=1, stride=stride, bias=True)\n",
    "            )\n",
    "\n",
    "    def forward(self, x):\n",
    "        out = self.bn1(x)\n",
    "        out = F.relu(out)\n",
    "        out = self.conv1(out)\n",
    "        out = self.dropout(out)\n",
    "\n",
    "        out = self.bn2(out)\n",
    "        out = F.relu(out)\n",
    "        out = self.conv2(out)\n",
    "\n",
    "        out += self.shortcut(x)\n",
    "        return out\n",
    "\n",
    "\n",
    "class WideResnet(nn.Module):\n",
    "    def __init__(self, depth, widen_factor, dropout_rate, num_classes=10):\n",
    "        super(WideResnet, self).__init__()\n",
    "        self.in_planes = 16\n",
    "\n",
    "        n = (depth - 4) / 6     # 최소 CONV : (conv1,2,3,4) + 6 * N\n",
    "        k = widen_factor\n",
    "\n",
    "        groups = [16, 16*k, 32*k, 64*k]\n",
    "\n",
    "        self.conv1 = nn.Conv2d(3, groups[0], kernel_size=3, stride=1, padding=1, bias=True)\n",
    "        self.layer1 = self._wide_layer(WideModule, groups[1], n, dropout_rate, stride=1)\n",
    "        self.layer2 = self._wide_layer(WideModule, groups[2], n, dropout_rate, stride=2)\n",
    "        self.layer3 = self._wide_layer(WideModule, groups[3], n, dropout_rate, stride=2)\n",
    "        self.bn1 = nn.BatchNorm2d(groups[3])\n",
    "        self.linear = nn.Linear(groups[3], num_classes)\n",
    "\n",
    "    def _wide_layer(self, block, planes, num_blocks, dropout_rate, stride):\n",
    "        strides = [stride] + [1]*(int(num_blocks)-1)\n",
    "        layers = []\n",
    "\n",
    "        for stride in strides:\n",
    "            layers.append(block(self.in_planes, planes, dropout_rate, stride))\n",
    "            self.in_planes = planes\n",
    "\n",
    "        return nn.Sequential(*layers)\n",
    "\n",
    "    def forward(self, x):\n",
    "        out = self.conv1(x)\n",
    "        out = self.layer1(out)\n",
    "        out = self.layer2(out)\n",
    "        out = self.layer3(out)\n",
    "        out = F.relu(self.bn1(out))\n",
    "        out = F.avg_pool2d(out, 8)\n",
    "        out = out.view(out.size(0), -1)\n",
    "        out = self.linear(out)\n",
    "\n",
    "        return out\n",
    "#\n",
    "# if __name__ == \"__main__\":\n",
    "#     net = WideResnet(28, 10, 0.3, 10)   # N=28, k=10, dropout rate = 0.3일 때 가장 결과가 좋았다고 함.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "N2M_5k1qI939"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WideResnet(\n",
       "  (conv1): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "  (layer1): Sequential(\n",
       "    (0): WideModule(\n",
       "      (bn1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(16, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential(\n",
       "        (0): Conv2d(16, 160, kernel_size=(1, 1), stride=(1, 1))\n",
       "      )\n",
       "    )\n",
       "    (1): WideModule(\n",
       "      (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (2): WideModule(\n",
       "      (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (3): WideModule(\n",
       "      (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "  )\n",
       "  (layer2): Sequential(\n",
       "    (0): WideModule(\n",
       "      (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
       "      (shortcut): Sequential(\n",
       "        (0): Conv2d(160, 320, kernel_size=(1, 1), stride=(2, 2))\n",
       "      )\n",
       "    )\n",
       "    (1): WideModule(\n",
       "      (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (2): WideModule(\n",
       "      (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (3): WideModule(\n",
       "      (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "  )\n",
       "  (layer3): Sequential(\n",
       "    (0): WideModule(\n",
       "      (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(320, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
       "      (shortcut): Sequential(\n",
       "        (0): Conv2d(320, 640, kernel_size=(1, 1), stride=(2, 2))\n",
       "      )\n",
       "    )\n",
       "    (1): WideModule(\n",
       "      (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (2): WideModule(\n",
       "      (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "    (3): WideModule(\n",
       "      (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (dropout): Dropout(p=0.0, inplace=False)\n",
       "      (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "      (shortcut): Sequential()\n",
       "    )\n",
       "  )\n",
       "  (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (linear): Linear(in_features=640, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = WideResnet(28, 10, 0.0, 10)   # N(number of layers) = 28,  k(widen_factor) = 10\n",
    "model.to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 276
    },
    "id": "RFG0KEDpIWRJ",
    "outputId": "9134cc75-5a4c-416a-d381-54489c973d45"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0/782 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adjusting learning rate of group 0 to 1.0000e-01.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:09<00:00,  6.02it/s]\n",
      "  0%|          | 1/782 [00:00<02:15,  5.78it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1] loss: 1.648\n",
      "epoch time : 129.99448919296265\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.96it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2] loss: 1.131\n",
      "epoch time : 131.28945207595825\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.94it/s]\n",
      "  0%|          | 1/782 [00:00<02:15,  5.78it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3] loss: 0.934\n",
      "epoch time : 131.7313494682312\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:18,  5.65it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4] loss: 0.836\n",
      "epoch time : 132.12741374969482\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.84it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5] loss: 0.752\n",
      "epoch time : 132.18334889411926\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6] loss: 0.699\n",
      "epoch time : 132.03012251853943\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:14,  5.82it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7] loss: 0.655\n",
      "epoch time : 132.18485975265503\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:20,  5.55it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8] loss: 0.626\n",
      "epoch time : 132.15264773368835\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:14,  5.81it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9] loss: 0.598\n",
      "epoch time : 131.9855616092682\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.91it/s]\n",
      "  0%|          | 1/782 [00:00<02:24,  5.42it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[10] loss: 0.579\n",
      "epoch time : 132.64748978614807\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:20,  5.55it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[11] loss: 0.562\n",
      "epoch time : 132.20654606819153\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[12] loss: 0.547\n",
      "epoch time : 132.0350320339203\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[13] loss: 0.532\n",
      "epoch time : 132.0732021331787\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.84it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[14] loss: 0.527\n",
      "epoch time : 132.04854202270508\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[15] loss: 0.511\n",
      "epoch time : 132.0241551399231\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:14,  5.79it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[16] loss: 0.504\n",
      "epoch time : 132.13472509384155\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:20,  5.54it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[17] loss: 0.502\n",
      "epoch time : 131.8320848941803\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:16,  5.73it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[18] loss: 0.492\n",
      "epoch time : 132.0796709060669\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.85it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[19] loss: 0.487\n",
      "epoch time : 131.78138613700867\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:23,  5.46it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[20] loss: 0.486\n",
      "epoch time : 132.43747854232788\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:12,  5.88it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21] loss: 0.480\n",
      "epoch time : 131.76719164848328\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.94it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[22] loss: 0.478\n",
      "epoch time : 131.7324960231781\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[23] loss: 0.472\n",
      "epoch time : 131.88351154327393\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.54it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24] loss: 0.473\n",
      "epoch time : 131.83243584632874\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.94it/s]\n",
      "  0%|          | 1/782 [00:00<02:20,  5.55it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[25] loss: 0.465\n",
      "epoch time : 131.7610981464386\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[26] loss: 0.468\n",
      "epoch time : 131.91111826896667\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:15,  5.77it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[27] loss: 0.460\n",
      "epoch time : 131.85298705101013\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.93it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.53it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[28] loss: 0.464\n",
      "epoch time : 131.95119261741638\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:11<00:00,  5.94it/s]\n",
      "  0%|          | 1/782 [00:00<02:13,  5.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[29] loss: 0.464\n",
      "epoch time : 131.67089581489563\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.91it/s]\n",
      "  0%|          | 1/782 [00:00<02:20,  5.55it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[30] loss: 0.459\n",
      "epoch time : 132.6028435230255\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 782/782 [02:12<00:00,  5.92it/s]\n",
      "  0%|          | 1/782 [00:00<02:21,  5.52it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[31] loss: 0.460\n",
      "epoch time : 132.1834487915039\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 40%|███▉      | 311/782 [00:52<01:19,  5.94it/s]"
     ]
    }
   ],
   "source": [
    "# criterion. 변경 '가능'\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "# Optimizer\n",
    "optimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=momentum, weight_decay=weight_decay)\n",
    "\n",
    "# Learning Rate Scheduler\n",
    "scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=60, gamma=0.2, verbose=True)\n",
    "\n",
    "loss_ = []\n",
    "n = len(trainloader)  # number of batches\n",
    "\n",
    "# Training\n",
    "for epoch in range(EPOCH): \n",
    "\n",
    "  running_loss = 0.0\n",
    "  start = time.time()\n",
    "  pbar = tqdm(trainloader)\n",
    "  for batch, (X, y) in enumerate(pbar):\n",
    "    \n",
    "    X, y = X.to(device), y.to(device)\n",
    "    optimizer.zero_grad() # 배치마다 optimizer 초기화\n",
    "\n",
    "    pred = model(X) \n",
    "    loss = criterion(pred, y) # 크로스 엔트로피 손실함수 계산 \n",
    "      \n",
    "\n",
    "    loss.backward() # backpropagation\n",
    "    optimizer.step() # 가중치 최적화\n",
    "\n",
    "    running_loss += loss.item()\n",
    "  \n",
    "  if (epoch + 1) % 10 == 0:\n",
    "    dir = './model_' + str(epoch) + '.pt'\n",
    "    torch.save(model.state_dict() , dir) # 모델 저장, path 수정\n",
    "  loss_.append(running_loss / n)\n",
    "  print('[%d] loss: %.3f' %(epoch + 1, running_loss / len(trainloader)))\n",
    "  print(\"epoch time :\", time.time()-start)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sf9UIvloTtt2"
   },
   "source": [
    "# Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "amoq6d5NMYvA"
   },
   "outputs": [],
   "source": [
    "# Evaluation\n",
    "\n",
    "# 수정\n",
    "# net = ResNet152().to(device)\n",
    "net = model\n",
    "# net.load_state_dict(torch.load('/content/gdrive/MyDrive/Baseline//model_ckpt.pt')) # 저장된 모델 불러오기\n",
    "\n",
    "\n",
    "# 이하 전체 수정 불가\n",
    "correct = 0\n",
    "total = 0\n",
    "correct_pred = {classname: 0 for classname in classes}\n",
    "total_pred = {classname: 0 for classname in classes}\n",
    "# since we're not training, we don't need to calculate the gradients for our outputs\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data[0].to(device), data[1].to(device)\n",
    "        # calculate outputs by running images through the network\n",
    "        outputs = net(images)\n",
    "        # the class with the highest energy is what we choose as prediction\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "        # collect the correct predictions for each class\n",
    "        for label, prediction in zip(labels, predicted):\n",
    "            if label == prediction:\n",
    "                correct_pred[classes[label]] += 1\n",
    "            total_pred[classes[label]] += 1\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))\n",
    "\n",
    "# print accuracy for each class\n",
    "for classname, correct_count in correct_pred.items():\n",
    "    accuracy = 100 * float(correct_count) / total_pred[classname]\n",
    "    print(\"Accuracy for class {:5s} is: {:.1f} %\".format(classname,\n",
    "                                                   accuracy))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1lUnuy-g4Aiq"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.figure(figsize=(20,10))\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(8)))\n",
    "\n",
    "net = model\n",
    "# net = Net()\n",
    "# net.to(device)\n",
    "# net.load_state_dict(torch.load('/content/gdrive/MyDrive/Baseline//model_ckpt.pt'))\n",
    "\n",
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(8)))"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "WideResNet.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "117786fb9ad04bbfba9000efc67417ca": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1996b2035d394feb8ddea161be66e787": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2d011007d89c4aad96b59ab60216ab9b",
       "IPY_MODEL_3d976f276c2242bdb3dc84857838b627",
       "IPY_MODEL_69306e2c35aa46e8b951227a9c3e9696"
      ],
      "layout": "IPY_MODEL_9acbedbeaa724ce08dffeddc71222505"
     }
    },
    "2d011007d89c4aad96b59ab60216ab9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_117786fb9ad04bbfba9000efc67417ca",
      "placeholder": "​",
      "style": "IPY_MODEL_a0b54dda21fc4fc8b8262d899d385dfe",
      "value": ""
     }
    },
    "30bffec40690426489b084343473f2bb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3d976f276c2242bdb3dc84857838b627": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a7f8572bbbcb42c39c2bd7287d4ccdba",
      "max": 170498071,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_30bffec40690426489b084343473f2bb",
      "value": 170498071
     }
    },
    "4c74f0cd1e2d461a8b64740fa8b61c29": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "665730bcefbd4ec79ff322d8fdcac2fe": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "69306e2c35aa46e8b951227a9c3e9696": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4c74f0cd1e2d461a8b64740fa8b61c29",
      "placeholder": "​",
      "style": "IPY_MODEL_665730bcefbd4ec79ff322d8fdcac2fe",
      "value": " 170499072/? [00:03&lt;00:00, 54078059.84it/s]"
     }
    },
    "9acbedbeaa724ce08dffeddc71222505": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a0b54dda21fc4fc8b8262d899d385dfe": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a7f8572bbbcb42c39c2bd7287d4ccdba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}