multiTextClassiffication.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/elifftosunn/Bert-Bank-Model/blob/main/multiTextClassiffication.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q9AF2cB5096o"
      },
      "source": [
        "#  Multi-Class Text Classification for make-up products using Doc2Vec \n",
        "Multi-Class Text Classification to make-up products based on their description and categories by using Doc2Vec vectors.\n",
        "\n",
        "Doc2vec is a method of vector representation of entire documents, not individual words. By document, you can mean a single sentence, paragraph, or an entire book. Doc2Vec architecture has two algorithms. One of the them is called Distributed Bag of Words (DBOW). The second algorithm is “distributed memory” (DM).\n",
        "\n",
        "In this project, we used Doc2Vec to get the document vectors and we used as input to the classification model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g8qHn-c21MQJ"
      },
      "source": [
        "# Importing packages and loading data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BmHGr08e0-9v",
        "outputId": "b907848d-d499-41ad-89f2-5e73c06cf076"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ]
        }
      ],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "import re\n",
        "from time import time\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem.wordnet import WordNetLemmatizer\n",
        "from nltk.tokenize import sent_tokenize, word_tokenize\n",
        "\n",
        "from gensim.models.doc2vec import Doc2Vec, TaggedDocument\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.svm import LinearSVC\n",
        "from sklearn.metrics import confusion_matrix, accuracy_score, classification_report, f1_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "wqJm6cju1OcG",
        "outputId": "7763a7cc-ed5a-46f0-a22f-bf048b1a7a39"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     Unnamed: 0       id           target username  \\\n",
              "0           0.0      0.0            Diğer     Elif   \n",
              "1           1.0      5.0            Diğer     Elif   \n",
              "2           2.0      6.0            Diğer     Elif   \n",
              "3           3.0      7.0            Diğer     Elif   \n",
              "4           4.0      8.0            Diğer     Elif   \n",
              "..          ...      ...              ...      ...   \n",
              "641       642.0   5623.0  Hesap İşlemleri     Mina   \n",
              "642       643.0   6923.0  Hesap İşlemleri   Zeycan   \n",
              "643       644.0  12652.0         KKB Skor     Mina   \n",
              "644       645.0  13077.0   Kart İşlemleri   Zeycan   \n",
              "645         NaN      NaN              NaN      NaN   \n",
              "\n",
              "                                                  text  \n",
              "0    vatan telefon kredi iç başvurmak kredi başvuru...  \n",
              "1    banka bulmak sicil kayıt 01102022 tarihî çıkma...  \n",
              "2    07102022 yürümek girmek kanun ilmek madde tari...  \n",
              "3    türkiye faaliyet göstermek devlet özel banka n...  \n",
              "4    vakıfbank swift ücret 2500 para kesmek insan ş...  \n",
              "..                                                 ...  \n",
              "641  ben bor bitmek 07102022 tarihî icra olmak bank...  \n",
              "642  ptt ara ilmek kredi çek 1 ay iç ödemek promosy...  \n",
              "643  yapı kredi world kart 05062022de yansımak 216 ...  \n",
              "644                                                NaN  \n",
              "645  halkbankndan 50000 tl kredi başvuru yapmak ona...  \n",
              "\n",
              "[646 rows x 5 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-73495e95-dde0-4e0a-874f-dffc8e1c5833\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>id</th>\n",
              "      <th>target</th>\n",
              "      <th>username</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>vatan telefon kredi iç başvurmak kredi başvuru...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>banka bulmak sicil kayıt 01102022 tarihî çıkma...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>07102022 yürümek girmek kanun ilmek madde tari...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>türkiye faaliyet göstermek devlet özel banka n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>vakıfbank swift ücret 2500 para kesmek insan ş...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>641</th>\n",
              "      <td>642.0</td>\n",
              "      <td>5623.0</td>\n",
              "      <td>Hesap İşlemleri</td>\n",
              "      <td>Mina</td>\n",
              "      <td>ben bor bitmek 07102022 tarihî icra olmak bank...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>642</th>\n",
              "      <td>643.0</td>\n",
              "      <td>6923.0</td>\n",
              "      <td>Hesap İşlemleri</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>ptt ara ilmek kredi çek 1 ay iç ödemek promosy...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>643</th>\n",
              "      <td>644.0</td>\n",
              "      <td>12652.0</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>Mina</td>\n",
              "      <td>yapı kredi world kart 05062022de yansımak 216 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>644</th>\n",
              "      <td>645.0</td>\n",
              "      <td>13077.0</td>\n",
              "      <td>Kart İşlemleri</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>645</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>halkbankndan 50000 tl kredi başvuru yapmak ona...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>646 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-73495e95-dde0-4e0a-874f-dffc8e1c5833')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-73495e95-dde0-4e0a-874f-dffc8e1c5833 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-73495e95-dde0-4e0a-874f-dffc8e1c5833');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "df = pd.read_csv(\"/content/drive/MyDrive/datas/cleanBankData.csv\")\n",
        "df"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# birey banka sicil af kanun dâhi 07102022de yürümek girmek kanun ilmek madde tarih 5411 sayılı kanun 1 madde uyarı karşı çek protesto senet ilmek kredi kredi kart borç ilişki kayıt dikkat alınmak hakkında kanun aşağı geçici mad\n",
        "df.loc[df.text == \"birey banka sicil af kanun dâhi 07102022de yürümek girmek kanun ilmek madde tarih 5411 sayılı kanun 1 madde uyarı karşı çek protesto senet ilmek kredi kredi kart borç ilişki kayıt dikkat alınmak hakkında kanun aşağı geçici mad\"]\n",
        "deleteRows = df.loc[(df.text == \"birey banka sicil af kanun dâhi 07102022de yürümek girmek kanun ilmek madde tarih 5411 sayılı kanun 1 madde uyarı karşı çek protesto senet ilmek kredi kredi kart borç ilişki kayıt dikkat alınmak hakkında kanun aşağı geçici mad\") & (df.target == \"Diğer\")]\n",
        "deleteRows"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 144
        },
        "id": "9JZeor5O7MlZ",
        "outputId": "27bddea0-e6ee-4ed7-f313-5f6cd5c0a973"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     Unnamed: 0     id target username  \\\n",
              "488       489.0  839.0  Diğer   Zeycan   \n",
              "494       495.0  848.0  Diğer   Zeycan   \n",
              "545       546.0  931.0  Diğer     Elif   \n",
              "\n",
              "                                                  text  \n",
              "488  birey banka sicil af kanun dâhi 07102022de yür...  \n",
              "494  birey banka sicil af kanun dâhi 07102022de yür...  \n",
              "545  birey banka sicil af kanun dâhi 07102022de yür...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-334928a5-9758-4723-9cd6-4f1f72da57a5\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>id</th>\n",
              "      <th>target</th>\n",
              "      <th>username</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>488</th>\n",
              "      <td>489.0</td>\n",
              "      <td>839.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>birey banka sicil af kanun dâhi 07102022de yür...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>494</th>\n",
              "      <td>495.0</td>\n",
              "      <td>848.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>birey banka sicil af kanun dâhi 07102022de yür...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>545</th>\n",
              "      <td>546.0</td>\n",
              "      <td>931.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>birey banka sicil af kanun dâhi 07102022de yür...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-334928a5-9758-4723-9cd6-4f1f72da57a5')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-334928a5-9758-4723-9cd6-4f1f72da57a5 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-334928a5-9758-4723-9cd6-4f1f72da57a5');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.drop([488, 494, 545], axis = 0)\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "j5HqZfQ676i-",
        "outputId": "3e819b83-733d-4218-df43-2ba7bdde07cd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     Unnamed: 0       id           target username  \\\n",
              "0           0.0      0.0            Diğer     Elif   \n",
              "1           1.0      5.0            Diğer     Elif   \n",
              "2           2.0      6.0            Diğer     Elif   \n",
              "3           3.0      7.0            Diğer     Elif   \n",
              "4           4.0      8.0            Diğer     Elif   \n",
              "..          ...      ...              ...      ...   \n",
              "641       642.0   5623.0  Hesap İşlemleri     Mina   \n",
              "642       643.0   6923.0  Hesap İşlemleri   Zeycan   \n",
              "643       644.0  12652.0         KKB Skor     Mina   \n",
              "644       645.0  13077.0   Kart İşlemleri   Zeycan   \n",
              "645         NaN      NaN              NaN      NaN   \n",
              "\n",
              "                                                  text  \n",
              "0    vatan telefon kredi iç başvurmak kredi başvuru...  \n",
              "1    banka bulmak sicil kayıt 01102022 tarihî çıkma...  \n",
              "2    07102022 yürümek girmek kanun ilmek madde tari...  \n",
              "3    türkiye faaliyet göstermek devlet özel banka n...  \n",
              "4    vakıfbank swift ücret 2500 para kesmek insan ş...  \n",
              "..                                                 ...  \n",
              "641  ben bor bitmek 07102022 tarihî icra olmak bank...  \n",
              "642  ptt ara ilmek kredi çek 1 ay iç ödemek promosy...  \n",
              "643  yapı kredi world kart 05062022de yansımak 216 ...  \n",
              "644                                                NaN  \n",
              "645  halkbankndan 50000 tl kredi başvuru yapmak ona...  \n",
              "\n",
              "[643 rows x 5 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-eec0391d-b0c0-4e24-a908-cb80bba86a51\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>id</th>\n",
              "      <th>target</th>\n",
              "      <th>username</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>vatan telefon kredi iç başvurmak kredi başvuru...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>banka bulmak sicil kayıt 01102022 tarihî çıkma...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>07102022 yürümek girmek kanun ilmek madde tari...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>türkiye faaliyet göstermek devlet özel banka n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>Diğer</td>\n",
              "      <td>Elif</td>\n",
              "      <td>vakıfbank swift ücret 2500 para kesmek insan ş...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>641</th>\n",
              "      <td>642.0</td>\n",
              "      <td>5623.0</td>\n",
              "      <td>Hesap İşlemleri</td>\n",
              "      <td>Mina</td>\n",
              "      <td>ben bor bitmek 07102022 tarihî icra olmak bank...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>642</th>\n",
              "      <td>643.0</td>\n",
              "      <td>6923.0</td>\n",
              "      <td>Hesap İşlemleri</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>ptt ara ilmek kredi çek 1 ay iç ödemek promosy...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>643</th>\n",
              "      <td>644.0</td>\n",
              "      <td>12652.0</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>Mina</td>\n",
              "      <td>yapı kredi world kart 05062022de yansımak 216 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>644</th>\n",
              "      <td>645.0</td>\n",
              "      <td>13077.0</td>\n",
              "      <td>Kart İşlemleri</td>\n",
              "      <td>Zeycan</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>645</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>halkbankndan 50000 tl kredi başvuru yapmak ona...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>643 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eec0391d-b0c0-4e24-a908-cb80bba86a51')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-eec0391d-b0c0-4e24-a908-cb80bba86a51 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-eec0391d-b0c0-4e24-a908-cb80bba86a51');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.to_csv(\"/content/drive/MyDrive/datas/cleanBankData.csv\", index= False)"
      ],
      "metadata": {
        "id": "p2L5jDWD8FIU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ymjVfTde1TVP",
        "outputId": "133fb3ee-2dd7-4220-a735-b083a76e9963"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Unnamed: 0    1\n",
              "id            1\n",
              "target        1\n",
              "username      1\n",
              "text          1\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "df.isnull().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Uciz1kVs1aPb",
        "outputId": "03bbd03a-f0e7-4e6b-871b-408122cdfa55"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Unnamed: 0    0\n",
              "id            0\n",
              "target        0\n",
              "username      0\n",
              "text          0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "df = df.dropna(axis = 0)\n",
        "df.isnull().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "-m3KpYi11brT",
        "outputId": "4e02feda-2819-4944-b879-2cd9b8acdc10"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  target                                               text\n",
              "0  Diğer  vatan telefon kredi iç başvurmak kredi başvuru...\n",
              "1  Diğer  banka bulmak sicil kayıt 01102022 tarihî çıkma...\n",
              "2  Diğer  07102022 yürümek girmek kanun ilmek madde tari...\n",
              "3  Diğer  türkiye faaliyet göstermek devlet özel banka n...\n",
              "4  Diğer  vakıfbank swift ücret 2500 para kesmek insan ş..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4d40f009-a270-4af1-8a2d-62446ccecd1c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>target</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Diğer</td>\n",
              "      <td>vatan telefon kredi iç başvurmak kredi başvuru...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Diğer</td>\n",
              "      <td>banka bulmak sicil kayıt 01102022 tarihî çıkma...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Diğer</td>\n",
              "      <td>07102022 yürümek girmek kanun ilmek madde tari...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Diğer</td>\n",
              "      <td>türkiye faaliyet göstermek devlet özel banka n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Diğer</td>\n",
              "      <td>vakıfbank swift ücret 2500 para kesmek insan ş...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4d40f009-a270-4af1-8a2d-62446ccecd1c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-4d40f009-a270-4af1-8a2d-62446ccecd1c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-4d40f009-a270-4af1-8a2d-62446ccecd1c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "df = df.drop([\"id\",\"username\",\"Unnamed: 0\"], axis = 1)\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NmjVZ6tx1exp",
        "outputId": "ebc10167-3234-45e2-905a-c192f6131bfd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Hesap İşlemleri                   137\n",
              "KKB Skor                          124\n",
              "Kart İşlemleri                    110\n",
              "Diğer                              73\n",
              "Kredi İşlemleri                    72\n",
              "Müşteri Temsilcisi                 55\n",
              "Kargo                              51\n",
              "Limit(Hesap, KMH, Kredi Kartı)     19\n",
              "Name: target, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "df.target.value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wo2HQz_41gKH"
      },
      "outputs": [],
      "source": [
        "train, test = train_test_split(df, test_size=0.3, random_state=42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RHT2k8qt1mft"
      },
      "outputs": [],
      "source": [
        "train_tag = train.apply(lambda x: TaggedDocument(words=word_tokenize(x['text']), tags=[x.target]), axis=1)\n",
        "test_tag = test.apply(lambda x: TaggedDocument(words=word_tokenize(x['text']), tags=[x.target]), axis=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RM1s116j1qOk",
        "outputId": "2bac8063-522f-49c2-a837-f41c60399870"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "208    ([iş, banka, bank, kart, kaymak, kaynak, yen, ...\n",
              "9      ([papara, hesap, para, yüklemek, iç, hesap, ba...\n",
              "526    ([1, yılmak, enparacomla, çalışıyorum, herhang...\n",
              "404    ([son, 3288, ilmek, bitmek, bankkart, gold, mc...\n",
              "350    ([enparada, kurmak, korumak, vade, mevduat, he...\n",
              "                             ...                        \n",
              "71     ([exxen, bilgi, dış, para, çekilmek, müşteri, ...\n",
              "106    ([bugün, iş, banka, ihtiyaç, kredi, kredi, baş...\n",
              "270    ([birey, banka, sicil, af, kanun, dâhi, 071020...\n",
              "435    ([müşteri, hizmet, f, kart, kapamak, havale, m...\n",
              "102    ([birey, banka, sicil, af, kanun, dâhi, 071020...\n",
              "Length: 448, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "train_tag"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u8HrgvRP19Tb",
        "outputId": "f4700668-33aa-4064-dbe5-7f096625f6c9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TaggedDocument(words=['birey', 'banka', 'sicil', 'af', 'kanun', 'dâhi', '07102022', 'yürümek', 'girmek', 'kanun', 'ilmek', 'madde', 'tarih', '5411', 'sayılı', 'kanun', '1', 'madde', 'uyarı', 'karşı', 'çek', 'protesto', 'senet', 'ilmek', 'kredi', 'kredi', 'kart', 'borç', 'ilişki', 'kayıt', 'dikkat', 'alınmak', 'hakkında', 'kanun', 'aşağı', 'geçici', 'mad'], tags=['Kart İşlemleri'])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "train_tag[400]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "luYG5Iol2FF2"
      },
      "source": [
        "#  Getting the feature vector from doc2vec model \n",
        "In this step we initialize the gensim doc2vec model. Doc2Vec architecture is also similar to word2vec and has two algorithms like word2vec and they are the corresponding algorithms for those two algorithms. One of the them is called Distributed Bag of Words (DBOW) which is similar to “Skip-gram” (SG) model in word2vec except that additional paragraph id vector is added. The second algorithm is “distributed memory” (DM) which is similar to “Continuous bag of words” CBOW in word vector.\n",
        "\n",
        "# Distributed Bag of Words (DBOW)\n",
        "First, we instantiate a doc2vec model — Distributed Bag of Words (DBOW). We set the following parameters:\n",
        "\n",
        "- dm=0 , distributed bag of words (DBOW) is used;\n",
        "- vector_size = 100, word embeddings will have shape of;\n",
        "- window = 2, model will try to predict every second word;\n",
        "- sample=0 , the threshold for configuring which higher-frequency words are - randomly down sampled;\n",
        "- min_count=2, ignores all words with total frequency lower than this."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DaJUzlmk15AI"
      },
      "outputs": [],
      "source": [
        "doc_model = Doc2Vec(dm=0, vector_size=100, min_count=2, window=2, sample = 0)\n",
        "               \n",
        "doc_model.build_vocab(train_tag)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qQe4HsxT2Up3",
        "outputId": "b127f99e-6d84-4390-d69c-701e1f47ead8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "15798"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "doc_model.corpus_total_words"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NVNhn8Ie2W2h",
        "outputId": "4b1bbdc8-3c1d-4f6d-f17e-ad7003617ef3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 1.29 s, sys: 61.9 ms, total: 1.36 s\n",
            "Wall time: 957 ms\n"
          ]
        }
      ],
      "source": [
        "%time doc_model.train(train_tag, total_examples=doc_model.corpus_count, epochs=30) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-DM63ThA2ZHL",
        "outputId": "6f1cb7b6-a7f9-4d83-ac12-530027970e7e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-26-0193b5a185c2>:1: DeprecationWarning: Call to deprecated `most_similar` (Method will be removed in 4.0.0, use self.wv.most_similar() instead).\n",
            "  doc_model.most_similar('kredi')\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('agt', 0.29740089178085327),\n",
              " ('findeks', 0.2900507152080536),\n",
              " ('sek', 0.28870928287506104),\n",
              " ('nfc', 0.2850169241428375),\n",
              " ('sorumak', 0.28180384635925293),\n",
              " ('kaza', 0.2769043445587158),\n",
              " ('uygun', 0.25099897384643555),\n",
              " ('elde', 0.23874403536319733),\n",
              " ('gönderilmedi', 0.23541449010372162),\n",
              " ('bilgilenmek', 0.23111969232559204)]"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "doc_model.most_similar('kredi')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lxobzEbG2clS"
      },
      "outputs": [],
      "source": [
        "#save model\n",
        "doc_model.save('/content/drive/MyDrive/models/model.doc2vec')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Z94Z1_WP201B"
      },
      "outputs": [],
      "source": [
        "def vector_for_learning(model, input_docs):\n",
        "    sents = input_docs\n",
        "    targets, feature_vectors = zip(*[(doc.tags[0], model.infer_vector(doc.words, steps=20)) for doc in sents])\n",
        "    return targets, feature_vectors"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7RrikdyM22Oj"
      },
      "outputs": [],
      "source": [
        "\n",
        "y_train, X_train = vector_for_learning(doc_model, train_tag)\n",
        "y_test, X_test = vector_for_learning(doc_model, test_tag)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GKPEmcoI27MN"
      },
      "source": [
        "# Training the Classifier\n",
        "We choose Logistic Regression Classifier and Linear Support Vector Machine.\n",
        "\n",
        "*Logistic Regression with DBOW*"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P2SCdr4e25CZ",
        "outputId": "3e883a0a-9a5b-4480-81ae-5525458c56d0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/sklearn/linear_model/_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  n_iter_i = _check_optimize_result(\n"
          ]
        }
      ],
      "source": [
        "log_reg = LogisticRegression(n_jobs=1, C=5)\n",
        "log_reg.fit(X_train, y_train)\n",
        "y_pred = log_reg.predict(X_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PVL_7axZ3BWV",
        "outputId": "d6b029fe-9cf9-4e60-f891-9f5fa6522d46"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Testing accuracy 0.19689119170984457\n",
            "Testing F1 score: 0.1884641133830912\n"
          ]
        }
      ],
      "source": [
        "print('Testing accuracy %s' % accuracy_score(y_pred, y_test))\n",
        "print('Testing F1 score: {}'.format(f1_score(y_test, y_pred, average='weighted')))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lmyE1GAa3KM5",
        "outputId": "84298e83-00b4-4442-f77c-a333b709ae6c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                                precision    recall  f1-score   support\n",
            "\n",
            "                         Diğer       0.14      0.17      0.15        24\n",
            "               Hesap İşlemleri       0.27      0.30      0.28        43\n",
            "                      KKB Skor       0.16      0.17      0.16        35\n",
            "                         Kargo       0.15      0.17      0.16        12\n",
            "                Kart İşlemleri       0.23      0.27      0.25        33\n",
            "               Kredi İşlemleri       0.19      0.16      0.17        19\n",
            "Limit(Hesap, KMH, Kredi Kartı)       0.00      0.00      0.00         8\n",
            "            Müşteri Temsilcisi       0.14      0.05      0.08        19\n",
            "\n",
            "                      accuracy                           0.20       193\n",
            "                     macro avg       0.16      0.16      0.16       193\n",
            "                  weighted avg       0.19      0.20      0.19       193\n",
            "\n"
          ]
        }
      ],
      "source": [
        "\n",
        "ytest = np.array(y_test)\n",
        "print(classification_report(ytest, y_pred))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z163MbhR3N3K",
        "outputId": "3c1bb9df-6861-409e-a8f4-389a57c84945"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearSVC()"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "svm = LinearSVC()\n",
        "svm.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fT35w5613SEC",
        "outputId": "cfc57bb4-e773-42fc-bf66-9114d64e28dc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Testing accuracy 0.20207253886010362\n",
            "Testing F1 score: 0.1915555867678797\n"
          ]
        }
      ],
      "source": [
        "preds = svm.predict(X_test)\n",
        "print('Testing accuracy %s' % accuracy_score(preds, y_test))\n",
        "print('Testing F1 score: {}'.format(f1_score(y_test, preds, average='weighted')))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8eQbur233UUJ",
        "outputId": "e474b7a2-58cf-4c65-daf5-6a86e4392863"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                                precision    recall  f1-score   support\n",
            "\n",
            "                         Diğer       0.18      0.21      0.19        24\n",
            "               Hesap İşlemleri       0.27      0.33      0.29        43\n",
            "                      KKB Skor       0.15      0.14      0.14        35\n",
            "                         Kargo       0.17      0.17      0.17        12\n",
            "                Kart İşlemleri       0.25      0.27      0.26        33\n",
            "               Kredi İşlemleri       0.21      0.21      0.21        19\n",
            "Limit(Hesap, KMH, Kredi Kartı)       0.00      0.00      0.00         8\n",
            "            Müşteri Temsilcisi       0.00      0.00      0.00        19\n",
            "\n",
            "                      accuracy                           0.20       193\n",
            "                     macro avg       0.15      0.17      0.16       193\n",
            "                  weighted avg       0.18      0.20      0.19       193\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(classification_report(ytest, preds))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S-ZhyBG73Wxj"
      },
      "outputs": [],
      "source": [
        "\n",
        "dm_model = Doc2Vec(dm=1, vector_size=100, min_count=2, window=2, sample = 0, negative=5, alpha=0.025, min_alpha=0.001)\n",
        "dm_model.build_vocab(train_tag)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kpTw-w8T3ZmB",
        "outputId": "ec68fcd0-94eb-478c-fe31-76d078fb7fd4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "15798"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "dm_model.corpus_total_words"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "79_hDxKF3b2E",
        "outputId": "cd702e1b-a7dd-4187-cf02-f87f8e0263e9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 2.22 s, sys: 207 ms, total: 2.43 s\n",
            "Wall time: 1.72 s\n"
          ]
        }
      ],
      "source": [
        "%time dm_model.train(train_tag, total_examples=dm_model.corpus_count, epochs=30) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5GXlELsZ3eRE"
      },
      "outputs": [],
      "source": [
        "dm_model.save('/content/drive/MyDrive/models/dm_model.doc2vec')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HOEdi1Fi3h-8"
      },
      "outputs": [],
      "source": [
        "y_train_dm, X_train_dm = vector_for_learning(dm_model, train_tag)\n",
        "y_test_dm, X_test_dm = vector_for_learning(dm_model, test_tag)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VNrEaWET3jnj",
        "outputId": "fe61c467-1612-49fb-e9e3-8215b3d0c903"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Testing accuracy 0.19170984455958548\n",
            "Testing F1 score: 0.1818533116364134\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/sklearn/linear_model/_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  n_iter_i = _check_optimize_result(\n"
          ]
        }
      ],
      "source": [
        "log_reg = LogisticRegression(n_jobs=1, C=5)\n",
        "log_reg.fit(X_train_dm, y_train_dm)\n",
        "pred = log_reg.predict(X_test_dm)\n",
        "\n",
        "print('Testing accuracy %s' % accuracy_score(y_test_dm, pred))\n",
        "print('Testing F1 score: {}'.format(f1_score(y_test_dm, pred, average='weighted')))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z-fHP6rG3k1L",
        "outputId": "807a88e6-d97b-4456-a820-c2140a95ce43"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                                precision    recall  f1-score   support\n",
            "\n",
            "                         Diğer       0.15      0.12      0.14        24\n",
            "               Hesap İşlemleri       0.26      0.23      0.24        43\n",
            "                      KKB Skor       0.20      0.26      0.22        35\n",
            "                         Kargo       0.14      0.17      0.15        12\n",
            "                Kart İşlemleri       0.21      0.27      0.24        33\n",
            "               Kredi İşlemleri       0.20      0.21      0.21        19\n",
            "Limit(Hesap, KMH, Kredi Kartı)       0.00      0.00      0.00         8\n",
            "            Müşteri Temsilcisi       0.00      0.00      0.00        19\n",
            "\n",
            "                      accuracy                           0.19       193\n",
            "                     macro avg       0.14      0.16      0.15       193\n",
            "                  weighted avg       0.18      0.19      0.18       193\n",
            "\n"
          ]
        }
      ],
      "source": [
        "ytest = np.array(y_test_dm)\n",
        "print(classification_report(ytest, pred))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TfwHTRqB3myy"
      },
      "outputs": [],
      "source": [
        "svm = LinearSVC()\n",
        "svm.fit(X_train_dm, y_train_dm)\n",
        "pred_y = svm.predict(X_test_dm)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kllCBxxz3pnZ",
        "outputId": "394f8c86-b9f3-4f25-c737-45c96f6c4981"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Testing accuracy 0.20207253886010362\n",
            "Testing F1 score: 0.1973015018968177\n"
          ]
        }
      ],
      "source": [
        "print('Testing accuracy %s' % accuracy_score(pred_y, y_test_dm))\n",
        "print('Testing F1 score: {}'.format(f1_score(y_test_dm, pred_y, average='weighted')))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "52aYOh0v3qzK",
        "outputId": "63b9e41e-b9dc-4e9b-cb55-44e025c94a15"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                                precision    recall  f1-score   support\n",
            "\n",
            "                         Diğer       0.12      0.12      0.12        24\n",
            "               Hesap İşlemleri       0.26      0.26      0.26        43\n",
            "                      KKB Skor       0.20      0.23      0.21        35\n",
            "                         Kargo       0.09      0.08      0.09        12\n",
            "                Kart İşlemleri       0.21      0.24      0.22        33\n",
            "               Kredi İşlemleri       0.24      0.26      0.25        19\n",
            "Limit(Hesap, KMH, Kredi Kartı)       0.00      0.00      0.00         8\n",
            "            Müşteri Temsilcisi       0.21      0.16      0.18        19\n",
            "\n",
            "                      accuracy                           0.20       193\n",
            "                     macro avg       0.17      0.17      0.17       193\n",
            "                  weighted avg       0.19      0.20      0.20       193\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(classification_report(ytest, pred_y))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NzbV2KSV4QZC"
      },
      "source": [
        "\n",
        "-------------------------------------------------------------------------------------------\n",
        "### SECOND METHOD -- BERT MODEL\n",
        "-------------------------------------------------------------------------------------------"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uaW12BnA4dYA",
        "outputId": "2de44177-cdec-4640-f740-cd9bfb67e37b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting transformers\n",
            "  Downloading transformers-4.26.0-py3-none-any.whl (6.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m43.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (21.3)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n",
            "  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m79.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n",
            "Collecting huggingface-hub<1.0,>=0.11.0\n",
            "  Downloading huggingface_hub-0.12.0-py3-none-any.whl (190 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m18.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.4.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n",
            "Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n",
            "Installing collected packages: tokenizers, huggingface-hub, transformers\n",
            "Successfully installed huggingface-hub-0.12.0 tokenizers-0.13.2 transformers-4.26.0\n"
          ]
        }
      ],
      "source": [
        "!pip install transformers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z54P-7aE6FM7"
      },
      "outputs": [],
      "source": [
        "import numpy as np \n",
        "import pandas as pd \n",
        "import tensorflow as tf\n",
        "import torch\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from transformers import BertTokenizer\n",
        "from torch.utils.data import TensorDataset, random_split\n",
        "from torch.utils.data import DataLoader, RandomSampler, SequentialSampler\n",
        "from transformers import BertForSequenceClassification, AdamW, BertConfig, Adafactor\n",
        "from transformers import get_linear_schedule_with_warmup\n",
        "import datetime\n",
        "import random\n",
        "import seaborn as sns\n",
        "import numpy as np\n",
        "import time\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.metrics import recall_score\n",
        "from sklearn.metrics import precision_score\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.metrics import accuracy_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113,
          "referenced_widgets": [
            "59b79c6f5c8b4c41a08de0b4127e46ae",
            "02b998e4ef91487da692ef4a2ea6e92f",
            "dcb6cbb68ede4561bef12163b30518ac",
            "ce11612771f145c6b75fbe0c16796bad",
            "07afbe66a72d44adafa8998447490f3a",
            "449ddbe2b069434ca69f21c1fc1e8883",
            "9a7a0c7334e24803ac0a2430094f768a",
            "9fc70fccbfa5424fb2ec5556326a6557",
            "f154edb856274f439ffd0eecb4a7afff",
            "15bc5d42978e4eceb6133b466c388e1b",
            "7d3be203dfd64cc6be1ad3d32224e922",
            "e36912ec0b33427c9bedbde8f0f20cfc",
            "2ea145eaaf4c4bb1a57448c9bcb01495",
            "0e0b79e1c53f4d608130fb54c0fcbe62",
            "9e2f36b69d9440f6af0539ec71eb0fe1",
            "185a6f8f1a894e9ab95b2d0e4f7bcd32",
            "ccc752ccb3fc451d8e29e4befda15bfd",
            "ce7eee9a75164ffda5d7c212392df8d3",
            "f7e7ea7ad453409ca98a5751323a12fe",
            "efaa62ab75e747159c31a78a18750fa0",
            "6d4fd21bbd1f4f93adaae9bf8506d3cc",
            "caa8fe34ff2143eaa076c111ac8180d8",
            "9f270f1ce6a64c4c9d44ddb195194de2",
            "0753a531a9154300a7c7bbcd496de476",
            "712e82a8d57b488cb59fd35acfcc1713",
            "3ac15f361f514a97b714d5be041514a8",
            "239927f1d3684534a0cc763bd3c7e270",
            "6e7264ba9a0b41e083eefcb607239764",
            "3ef5ae4f610b4d3e91160186acaae647",
            "cb25d88d560c41d994d64873bfc57f6f",
            "a6aed6cae73643c6bad28396c0394e80",
            "9c550d46640243ecb7ec5c804d9a8f74",
            "2f76cdfa80724c329d3b1143a24bc356"
          ]
        },
        "id": "g5YimjaM7o5H",
        "outputId": "24f74fac-daf7-4ca4-e03b-b06a37a63727"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/1.23M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "59b79c6f5c8b4c41a08de0b4127e46ae"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/59.0 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e36912ec0b33427c9bedbde8f0f20cfc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/386 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9f270f1ce6a64c4c9d44ddb195194de2"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "tokenizer = BertTokenizer.from_pretrained('dbmdz/bert-base-turkish-128k-uncased', do_lower_case=True)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv(\"/content/drive/MyDrive/datas/cleanBankData2.csv\")"
      ],
      "metadata": {
        "id": "VxYcHGseYKyN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xkfD0wcB7ugL",
        "outputId": "16914c82-c443-4fd1-a3f9-c6956e923cd8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "label\n",
              "Diğer                 36626\n",
              "Hesap İşlemleri       36504\n",
              "KKB Skor              12622\n",
              "Kargo                 35392\n",
              "Kart İşlemleri        37296\n",
              "Kredi İşlemleri       36904\n",
              "Limit                 30802\n",
              "Müşteri Temsilcisi    20266\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "df.groupby(\"label\").size()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "coDcHRxB72ca",
        "outputId": "b24ca4b6-0cde-4723-d930-88bd6fbe663f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                name  \\\n",
              "0          MOOV Mov Uyarı Vermeden Kısıtlama Getirdi   \n",
              "1  Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...   \n",
              "2  Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...   \n",
              "3                   Ziraat Bankası Borç Ödeme Sorunu   \n",
              "4                   Ziraat Bankası Bloke Açma Sorunu   \n",
              "\n",
              "                                                link  label  \\\n",
              "0  https://www.sikayetvar.com/moov/moov-mov-uyari...      5   \n",
              "1  https://www.sikayetvar.com/ziraat-bankasi/zira...      5   \n",
              "2  https://www.sikayetvar.com/hepsiburada/hepsibu...      5   \n",
              "3  https://www.sikayetvar.com/ziraat-bankasi/zira...      5   \n",
              "4  https://www.sikayetvar.com/ziraat-bankasi/zira...      5   \n",
              "\n",
              "                                                text  \n",
              "0  moov ad araç kiralamak şirket sürek araç kiral...  \n",
              "1  ziraat banka bankkart kart kredi kart alışveri...  \n",
              "2  22 ara 2022 tarih hepsiburadacomdan yapmak 475...  \n",
              "3  banka kart par olmak rağmen kredi kart bor öde...  \n",
              "4  ziraat banka kredi bor dolay hesap bloke koyma...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-31902afb-6aa3-4682-962c-5dadc926a426\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>name</th>\n",
              "      <th>link</th>\n",
              "      <th>label</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi</td>\n",
              "      <td>https://www.sikayetvar.com/moov/moov-mov-uyari...</td>\n",
              "      <td>5</td>\n",
              "      <td>moov ad araç kiralamak şirket sürek araç kiral...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>5</td>\n",
              "      <td>ziraat banka bankkart kart kredi kart alışveri...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "      <td>https://www.sikayetvar.com/hepsiburada/hepsibu...</td>\n",
              "      <td>5</td>\n",
              "      <td>22 ara 2022 tarih hepsiburadacomdan yapmak 475...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>5</td>\n",
              "      <td>banka kart par olmak rağmen kredi kart bor öde...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>5</td>\n",
              "      <td>ziraat banka kredi bor dolay hesap bloke koyma...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-31902afb-6aa3-4682-962c-5dadc926a426')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-31902afb-6aa3-4682-962c-5dadc926a426 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-31902afb-6aa3-4682-962c-5dadc926a426');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "le = LabelEncoder()\n",
        "df.label = le.fit_transform(df.label)\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df['text'] = df['name'] + \". \" + df['text']"
      ],
      "metadata": {
        "id": "ipUfrmNwYnQs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4G_oz39877JM",
        "outputId": "4ba43281-af4f-4f31-9436-9ca5f4fdc013"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "17115"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "training = df.groupby('label').apply(lambda x : x.sample(frac = 0.8))\n",
        "test = pd.concat([df,training]).drop_duplicates(keep=False)\n",
        "result = test.groupby('text')['label'].count().sum()\n",
        "result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 499
        },
        "id": "b-sYP1qF8Olv",
        "outputId": "5d90f93c-7a3a-4391-d155-8bbccae976cd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                           name  \\\n",
              "label                                                             \n",
              "0     164292  English Home İadede Haksızlığa Ve Aşağılamaya ...   \n",
              "      77727   T.C. Cumhurbaşkanlığı İmam Hatip Ortaokulu'na ...   \n",
              "      76376     A101  İndirimde Olan Şampuanı Haftaya Diyorlar!   \n",
              "      73760         Hepsiburada Puma Marka Ayakkabı Mağduriyeti   \n",
              "      72813           Ziraat Banka Kartı Kuryenet Sorumsuzluğu!   \n",
              "...                                                         ...   \n",
              "7     141187         Dyo Boya  Teknotex Boya 1 Yılda Küf Yaptı!   \n",
              "      39237         Çiçek Sepeti Hesabımdan Para Çekimi Yapıldı   \n",
              "      143163                     Garanti Bankası Hizmet Kusuru!   \n",
              "      37958   Türkiye Hayat Emeklilik Vefat Nedeniyle Kapatı...   \n",
              "      137701                        Enza Home Servis Bekliyoruz   \n",
              "\n",
              "                                                           link  label  \\\n",
              "label                                                                    \n",
              "0     164292  https://www.sikayetvar.com/english-home/englis...      0   \n",
              "      77727   https://www.sikayetvar.com/tc-cumhurbaskanligi...      0   \n",
              "      76376   https://www.sikayetvar.com/a101/a101-indirimde...      0   \n",
              "      73760   https://www.sikayetvar.com/hepsiburada/hepsibu...      0   \n",
              "      72813   https://www.sikayetvar.com/kuryenet/ziraat-ban...      0   \n",
              "...                                                         ...    ...   \n",
              "7     141187  https://www.sikayetvar.com/dyo-boya/dyo-boyate...      7   \n",
              "      39237   https://www.sikayetvar.com/ciceksepeti/cicek-s...      7   \n",
              "      143163  https://www.sikayetvar.com/garanti-bbva/garant...      7   \n",
              "      37958   https://www.sikayetvar.com/turkiye-hayat-emekl...      7   \n",
              "      137701  https://www.sikayetvar.com/enza-home/enza-home...      7   \n",
              "\n",
              "                                                           text  \n",
              "label                                                            \n",
              "0     164292  English Home İadede Haksızlığa Ve Aşağılamaya ...  \n",
              "      77727   T.C. Cumhurbaşkanlığı İmam Hatip Ortaokulu'na ...  \n",
              "      76376   A101  İndirimde Olan Şampuanı Haftaya Diyorlar...  \n",
              "      73760   Hepsiburada Puma Marka Ayakkabı Mağduriyeti. n...  \n",
              "      72813   Ziraat Banka Kartı Kuryenet Sorumsuzluğu!. yen...  \n",
              "...                                                         ...  \n",
              "7     141187  Dyo Boya  Teknotex Boya 1 Yılda Küf Yaptı!. uz...  \n",
              "      39237   Çiçek Sepeti Hesabımdan Para Çekimi Yapıldı. 0...  \n",
              "      143163  Garanti Bankası Hizmet Kusuru!. garanti banka ...  \n",
              "      37958   Türkiye Hayat Emeklilik Vefat Nedeniyle Kapatı...  \n",
              "      137701  Enza Home Servis Bekliyoruz. 12 ocak 2021de ec...  \n",
              "\n",
              "[197131 rows x 4 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-49a19099-41e8-4b81-8376-71a6a95223f2\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>name</th>\n",
              "      <th>link</th>\n",
              "      <th>label</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>label</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">0</th>\n",
              "      <th>164292</th>\n",
              "      <td>English Home İadede Haksızlığa Ve Aşağılamaya ...</td>\n",
              "      <td>https://www.sikayetvar.com/english-home/englis...</td>\n",
              "      <td>0</td>\n",
              "      <td>English Home İadede Haksızlığa Ve Aşağılamaya ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>77727</th>\n",
              "      <td>T.C. Cumhurbaşkanlığı İmam Hatip Ortaokulu'na ...</td>\n",
              "      <td>https://www.sikayetvar.com/tc-cumhurbaskanligi...</td>\n",
              "      <td>0</td>\n",
              "      <td>T.C. Cumhurbaşkanlığı İmam Hatip Ortaokulu'na ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>76376</th>\n",
              "      <td>A101  İndirimde Olan Şampuanı Haftaya Diyorlar!</td>\n",
              "      <td>https://www.sikayetvar.com/a101/a101-indirimde...</td>\n",
              "      <td>0</td>\n",
              "      <td>A101  İndirimde Olan Şampuanı Haftaya Diyorlar...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73760</th>\n",
              "      <td>Hepsiburada Puma Marka Ayakkabı Mağduriyeti</td>\n",
              "      <td>https://www.sikayetvar.com/hepsiburada/hepsibu...</td>\n",
              "      <td>0</td>\n",
              "      <td>Hepsiburada Puma Marka Ayakkabı Mağduriyeti. n...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>72813</th>\n",
              "      <td>Ziraat Banka Kartı Kuryenet Sorumsuzluğu!</td>\n",
              "      <td>https://www.sikayetvar.com/kuryenet/ziraat-ban...</td>\n",
              "      <td>0</td>\n",
              "      <td>Ziraat Banka Kartı Kuryenet Sorumsuzluğu!. yen...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">7</th>\n",
              "      <th>141187</th>\n",
              "      <td>Dyo Boya  Teknotex Boya 1 Yılda Küf Yaptı!</td>\n",
              "      <td>https://www.sikayetvar.com/dyo-boya/dyo-boyate...</td>\n",
              "      <td>7</td>\n",
              "      <td>Dyo Boya  Teknotex Boya 1 Yılda Küf Yaptı!. uz...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39237</th>\n",
              "      <td>Çiçek Sepeti Hesabımdan Para Çekimi Yapıldı</td>\n",
              "      <td>https://www.sikayetvar.com/ciceksepeti/cicek-s...</td>\n",
              "      <td>7</td>\n",
              "      <td>Çiçek Sepeti Hesabımdan Para Çekimi Yapıldı. 0...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>143163</th>\n",
              "      <td>Garanti Bankası Hizmet Kusuru!</td>\n",
              "      <td>https://www.sikayetvar.com/garanti-bbva/garant...</td>\n",
              "      <td>7</td>\n",
              "      <td>Garanti Bankası Hizmet Kusuru!. garanti banka ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37958</th>\n",
              "      <td>Türkiye Hayat Emeklilik Vefat Nedeniyle Kapatı...</td>\n",
              "      <td>https://www.sikayetvar.com/turkiye-hayat-emekl...</td>\n",
              "      <td>7</td>\n",
              "      <td>Türkiye Hayat Emeklilik Vefat Nedeniyle Kapatı...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>137701</th>\n",
              "      <td>Enza Home Servis Bekliyoruz</td>\n",
              "      <td>https://www.sikayetvar.com/enza-home/enza-home...</td>\n",
              "      <td>7</td>\n",
              "      <td>Enza Home Servis Bekliyoruz. 12 ocak 2021de ec...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>197131 rows × 4 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-49a19099-41e8-4b81-8376-71a6a95223f2')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-49a19099-41e8-4b81-8376-71a6a95223f2 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-49a19099-41e8-4b81-8376-71a6a95223f2');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "training"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cjwURJAL8RXC",
        "outputId": "c70f1c77-b4b8-464f-b318-f33cb717007c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training:  197131\n",
            "Test:  17115\n"
          ]
        }
      ],
      "source": [
        "print(\"Training: \", len(training))\n",
        "print(\"Test: \", len(test))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UdR7e0zM8XZo"
      },
      "outputs": [],
      "source": [
        "training_texts = training.text.values\n",
        "training_labels = training.label.values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dte-VyOH8kay",
        "outputId": "e3e932cd-aeca-4fbe-b872-9d6acde1c574"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 0, 0, ..., 7, 7, 7])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "training_labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wyv0Ekdn8lyi",
        "outputId": "01370de0-4770-4061-b4af-70c28b12ab80"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n",
            "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:2339: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original:  English Home İadede Haksızlığa Ve Aşağılamaya Uğradım. hediye ge bir ürün değişim i̇stanbul buyaka avm english home mağaza gittim nevresim yastık kılıf alıp ev geldiğimde yastık kılıf bir tane dikiş defo olduğunu görü vakit yaramak ( ) akşam değişim aynı mağaza gittim ürün değişim istedim k\n",
            "Token IDs: tensor([     2,  15445,  13100,   6761,   1942,   8980,   1945,   3287,   1946,\n",
            "         10892,   2893,   4108,  82365,   2257,     18,   5861,   5185,   1947,\n",
            "         36664, 119646,   6810,   5509,   5868,  12105,  15445,  13100, 108916,\n",
            "         10013,  37181,  14993,  15783,   5257,   2205,  92267, 109673,  14993,\n",
            "         15783,   1947,   4889,   2661,   1967,   4786,   1008,  15053,   8178,\n",
            "          1012,   5653,   3598,   2020,     12,     13,  16991, 119646,   2573,\n",
            "        108916,  10013,  36664, 119646,   7296,     53,      3,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0])\n"
          ]
        }
      ],
      "source": [
        "input_ids = []\n",
        "attention_masks = []\n",
        "max_len = 100\n",
        "\n",
        "\n",
        "for text in training_texts:\n",
        "    encoded_dict = tokenizer.encode_plus(\n",
        "                        str(text),                     \n",
        "                        add_special_tokens = True,\n",
        "                        max_length = max_len,      \n",
        "                        pad_to_max_length = True,\n",
        "                        return_attention_mask = True, \n",
        "                        return_tensors = 'pt',\n",
        "                   )\n",
        "    \n",
        "    input_ids.append(encoded_dict['input_ids'])\n",
        "    attention_masks.append(encoded_dict['attention_mask'])\n",
        "\n",
        "input_ids = torch.cat(input_ids, dim=0)\n",
        "attention_masks = torch.cat(attention_masks, dim=0)\n",
        "labels = torch.tensor(training_labels)\n",
        "\n",
        "print('Original: ', training_texts[0])\n",
        "print('Token IDs:', input_ids[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "40oGB5gj8uja"
      },
      "outputs": [],
      "source": [
        "train_dataset = TensorDataset(input_ids, attention_masks, labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P-OXjnn48xdB"
      },
      "outputs": [],
      "source": [
        "batch_size = 16\n",
        "\n",
        "train_dataloader = DataLoader(\n",
        "            train_dataset,  \n",
        "            sampler = RandomSampler(train_dataset), \n",
        "            batch_size = batch_size \n",
        "        )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yv6GJ4SG8zAR"
      },
      "outputs": [],
      "source": [
        "number_of_categories = len(df['label'].unique())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_FpeNMuN82B1",
        "outputId": "9057b035-d173-426c-f47c-137cc3c9ca76"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "number_of_categories"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156,
          "referenced_widgets": [
            "548f3cf1155c487b9fbae88878d6dbdd",
            "aa440a5a5bbf4d00aceff2449c7873d9",
            "8c58e7d59eea4fc9b4757a8a12c9ad2d",
            "9f3215b40d3f4e7584d13c328fbb14f1",
            "bb97d0d30ab049f2a29f9d835d09f008",
            "2959622874ba4e0d8b0bc23f89d0cafe",
            "02c09e9e4fba4312a21f8b56e3938226",
            "d14953dca6b3451cabc504c06cd9d342",
            "9f5081ca82994abd962bea5491d95323",
            "9443af5a2f02436cade8b5ea2b44294e",
            "a77f768e98664d10aefacd911daab57d"
          ]
        },
        "id": "QNvsMyuv82-Z",
        "outputId": "0297078d-e3c3-4fa2-c1ff-bd2b282e7f76"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/740M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "548f3cf1155c487b9fbae88878d6dbdd"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the model checkpoint at dbmdz/bert-base-turkish-128k-uncased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight']\n",
            "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at dbmdz/bert-base-turkish-128k-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "model = BertForSequenceClassification.from_pretrained(\n",
        "    \"dbmdz/bert-base-turkish-128k-uncased\",\n",
        "    num_labels = number_of_categories, \n",
        "    output_attentions = False,\n",
        "    output_hidden_states = False,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GKVQdhXc85CQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3971b08-5276-4eba-b990-94ee94eb4f1c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "epochs = 10\n",
        "\n",
        "optimizer = AdamW(model.parameters(),\n",
        "                      lr = 5e-5,\n",
        "                      eps = 1e-8 \n",
        "                    )\n",
        "total_steps = len(train_dataloader) * epochs\n",
        "scheduler = get_linear_schedule_with_warmup(optimizer, \n",
        "                                            num_warmup_steps = 0,\n",
        "                                            num_training_steps = total_steps)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7k1vCGeA9HTy"
      },
      "outputs": [],
      "source": [
        "def format_time(elapsed):\n",
        "    elapsed_rounded = int(round((elapsed)))\n",
        "    return str(datetime.timedelta(seconds=elapsed_rounded))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "id": "eZNkz-Lv9MHk",
        "outputId": "82b67e4d-a68c-4510-ffa6-505abdc134df"
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-30-43a88170ef82>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0mtotal_t0\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m \u001b[0;32mfor\u001b[0m \u001b[0mepoch_i\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     15\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'======== Epoch {:} / {:} ========'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mepoch_i\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m     \u001b[0mt0\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'epochs' is not defined"
          ]
        }
      ],
      "source": [
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "\n",
        "seed_val = 3000\n",
        "\n",
        "random.seed(seed_val)\n",
        "np.random.seed(seed_val)\n",
        "torch.manual_seed(seed_val)\n",
        "torch.cuda.manual_seed_all(seed_val)\n",
        "\n",
        "training_stats = []\n",
        "total_t0 = time.time()\n",
        "\n",
        "for epoch_i in range(0, epochs):\n",
        "    print('======== Epoch {:} / {:} ========'.format(epoch_i + 1, epochs))\n",
        "    t0 = time.time()\n",
        "    total_train_loss = 0\n",
        "    model.train()\n",
        "    \n",
        "    for step, batch in enumerate(train_dataloader):\n",
        "        if step % 10 == 0 and not step == 0:\n",
        "            elapsed = format_time(time.time() - t0)\n",
        "            print('Batch {:>5,}  of  {:>5,}.    Elapsed: {:}.'.format(step, len(train_dataloader), elapsed))\n",
        "\n",
        "        b_input_ids = batch[0].to(device)\n",
        "        b_input_mask = batch[1].to(device)\n",
        "        b_labels = batch[2].to(device)\n",
        "\n",
        "        model.zero_grad()        \n",
        "        output = model(b_input_ids, \n",
        "                             token_type_ids=None, \n",
        "                             attention_mask=b_input_mask, \n",
        "                             labels=b_labels)\n",
        "        loss = output['loss']\n",
        "        logits = output['logits']\n",
        "        total_train_loss += loss.item()\n",
        "        loss.backward()\n",
        "        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
        "        optimizer.step()\n",
        "        scheduler.step()\n",
        "\n",
        "    avg_train_loss = total_train_loss / len(train_dataloader)            \n",
        "    training_time = format_time(time.time() - t0)\n",
        "\n",
        "    print(\"Average training loss: {0:.2f}\".format(avg_train_loss))\n",
        "    print(\"Training epoch took: {:}\".format(training_time))\n",
        "\n",
        "    training_stats.append(\n",
        "        {\n",
        "            'epoch': epoch_i + 1,\n",
        "            'Training Loss': avg_train_loss,\n",
        "            'Training Time': training_time,\n",
        "        }\n",
        "    )\n",
        "\n",
        "print(\"Training completed in {:} (h:mm:ss)\".format(format_time(time.time()-total_t0)))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.makedirs(\"/content/drive/MyDrive/bert-bank\")\n",
        "tokenizer.save_pretrained(\"/content/drive/MyDrive/bert-bank/bert_bank_tokenizer\")\n",
        "model.save_pretrained(\"/content/drive/MyDrive/bert-bank/bert_bank_model\")"
      ],
      "metadata": {
        "id": "tCTtREyeZ_wo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IcTKi3oF9QDW"
      },
      "outputs": [],
      "source": [
        "tokenizer.save_pretrained(\"/content/drive/MyDrive/models/bank_tokenizer\")\n",
        "model.save_pretrained(\"/content/drive/MyDrive/models/bank_model\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UK1-gVtr-DV4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "d8ed7c6e-f9f8-4a17-86ad-8aa051f16f62"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5RV9b3//+f7nDnThzKFXoY6IAgoQ0tsMcm1xKi/xNgQLCCaRG+8aebe9TNNvUmMSdRYIiJiwXITvWkmuUajaCIgg4KFJiAICjIMbQpTmHl//5gDIlJmcM7Zp7wea83izD57znnNWnpes/fns/fH3B0REUlfoaADiIhIsFQEIiJpTkUgIpLmVAQiImlORSAikuZUBCIiaU5FIGnNzP5qZpd29L4iycR0HYEkGzOr2e/bXKABaI5+f5W7z41/qqNnZqcAj7h7n6CzSHrKCDqASHu5e/7ex2a2Dpju7s8euJ+ZZbj7nnhmE0lGOjUkKcPMTjGzjWZ2vZltBh4ws65m9mczqzSz7dHHffb7mRfMbHr08WVm9k8zuzW67ztmdsZR7jvAzF40s2oze9bM7jKzR47idxoefd8dZvaWmZ2933Nnmtmy6Hu8Z2bfjm4vjv6eO8xsm5m9ZGb6f10OSf9xSKrpARQC/YEZtP43/kD0+37AbuDOw/z8BGAlUAzcAtxvZnYU+z4KvAIUAT8EprT3FzGzCPAn4BmgG3AtMNfMyqK73E/rqbACYCTwj+j2bwEbgRKgO/BfgM4ByyGpCCTVtAA/cPcGd9/t7lXu/qS717l7NXAzcPJhfn69u9/n7s3Ag0BPWj9M27yvmfUDxgHfd/dGd/8n8Mej+F0mAvnAT6Ov8w/gz8BF0eebgGPMrJO7b3f3V/fb3hPo7+5N7v6SazBQDkNFIKmm0t3r935jZrlmdq+ZrTezXcCLQBczCx/i5zfvfeDuddGH+e3ctxewbb9tABva+XsQfZ0N7t6y37b1QO/o4y8DZwLrzWyemU2Kbv85sBp4xszWmtn3juK9JY2oCCTVHPiX77eAMmCCu3cCTopuP9Tpno6wCSg0s9z9tvU9itd5H+h7wPn9fsB7AO6+yN3PofW00e+B/4lur3b3b7n7QOBs4Jtm9tmjeH9JEyoCSXUFtI4L7DCzQuAHsX5Dd18PVAA/NLPM6F/qXzzSz5lZ9v5ftI4x1AHfNbNIdJrpF4HHo6872cw6u3sTsIvW02KY2VlmNjg6XrGT1qm1LQd9UxFUBJL6bgNygK3AAuBvcXrfycAkoAq4CXiC1usdDqU3rYW1/1dfWj/4z6A1/93AVHdfEf2ZKcC66Cmvq6PvCTAEeBaoAeYDd7v78x32m0nK0QVlInFgZk8AK9w95kckIu2lIwKRGDCzcWY2yMxCZnY6cA6t5/FFEo6uLBaJjR7AU7ReR7AR+Kq7vxZsJJGD06khEZE0p1NDIiJpLulODRUXF3tpaWnQMUREksrixYu3unvJwZ5LuiIoLS2loqIi6BgiIknFzNYf6jmdGhIRSXMqAhGRNKciEBFJcyoCEZE0pyIQEUlzKgIRkTSnIhARSXNJdx3B0VpTWcMfXnuPsh6dKOtRQGlRLhlh9aCISNoUwbL3d3Hn86tpid5aKTMjxJBu+Qzr0YlhPQoo61HAsB4FlBRkcei1ykVEUk/S3XSuvLzcj/bK4vqmZlZvqWHF5mpWbt4V/beaLdUfrhfSNTcSLYXWI4eyHgWUdS8gLyttOlNEUpCZLXb38oM9l1afbtmRMCN7d2Zk784f2b6ttpGV+5XDis3V/E/FBuoam/ft07cwh7LuHz16GFCcp9NLIpL00qoIDqUwL5NJg4qYNKho37aWFmfj9t2s2LyLlZurWfFB69HD8yu30Bw9v5QZDjGoWz7D9x45RI8kunfS6SURSR4qgkMIhYx+Rbn0K8rl30b02Le9vqmZNZU10SOI1qOHl9dU8dRr7+3bp3NOZN9Rw95/h3YvoCA7EsSvIiJyWCqCdsqOhBnRqzMjen309NKOusZ9Yw57xyCeevU9ahr27Nund5cchvUoYETvzpw0pJgxfbvo1JKIBC6tBovjzb319NLKzdWs/ODDgli9pYYWh07ZGZw4pISTh5ZwclkJ3TtlBx1ZRFJUIIPFZtYXeAjoDjgw091vP2AfA24HzgTqgMvc/dVYZYo3M6NvYS59C3P53DHd923fWdfEP1dv5YWVW5i3qpKn39gEwLAeBZxS1o2Th5Ywtn9XMjN0tCAisRezIwIz6wn0dPdXzawAWAyc6+7L9tvnTOBaWotgAnC7u0843Osm0xFBW7g7KzZX88LKSuat2kLFuu3saXHyszL41KAiTi5rPWLo0zU36KgiksQCOSJw903ApujjajNbDvQGlu232znAQ97aRgvMrIuZ9Yz+bFowM4b37MTwnp346imDqK5v4uU1VcxbVcm8lZU8s+wDAAZ3y+fkoSWcUlbCuNJCsiPhgJN3DHfXDCuRgMVlsNjMSoHjgIUHPNUb2LDf9xuj29KmCA5UkB3htBE9OG1ED9ydNZU10aOFSh6ev577//kO2ZEQkwYW7TuNVFqcF3Tsw3J3KqsbWLu1lneiX2sra3lnaw0btu3m8yO6c9sFY4ho4FwkEDEvAjPLB54ErnP3XUf5GjOAGQD9+vXrwHSJzcwY3K2Awd0KmH7iQOoa97BgbRXzosXw/Mq3ACgtyt034DxpYDE5mcEcLVTXNx3wQf/h1/6zpzIzQgwoymNItwKO69eV3y3eCA63XzhGs6hEAhDTIjCzCK0lMNfdnzrILu8Bfff7vk9020e4+0xgJrSOEcQgalLIzczg1GHdOXVY68Dzuq21raeQVlXyRMUGHpy/nsyMEBMGFO47jTSoJL9DT7007mnh3W11rK2s+fBDP/pv5X636jCDPl1zGFCcz9j+XRlQnMfAkjwGFOfRq3MOodCHmcq6F3DzX5aTFQlx63mjP/KciMReLAeLDXgQ2Obu1x1iny8A1/DhYPEd7j7+cK+baoPFHaW+qZlF67btO420eksN0Hrtwt4B508PLia/DfdMamlxNu+q3/chv/+H/oZtdftu3AdQlJe57wN+QHH+vg/8foW57RrHuOO5t/nl31dx8YR+3HzuSI0biHSwww0Wx7IITgBeAt4AWqKb/wvoB+Duv4mWxZ3A6bROH73c3Q/7Ka8iaJuN2+v2DTj/a/VWahubyQgZ5aVdOXloN04pK6FHp2zeqarlncq9f9nXsLaylnVVtdQ3tex7rZxIeN8H/MDiPAaURD/0i/LonNsxV0u7O7f830rueWENV3x6ADecNVxlINKBAimCWFERtF/jnhYWr9/OvFWVvLByCys2V39sn3DI6FeYG/3L/sPTOAOL8+N27yR350d/Wsacl9dxzWcG8+3TymL+niLpQncfTXOZGaF9N9X73hnD+GBXPfNWVbKzrmnfh37fwtzAZ+2YGT/44jE07GnmzudXkx0Jcc2pQwLNJJIOVARpqHunbM4v73vkHQNgZtx87rE0NLVw6zOryI6EmX7iwKBjiaQ0FYEknFDIuOW8UdTvaeamp5eTFQkzZWL/oGOJpCwVgSSkjHCI2y44joamxdzw+zfJiYQ5b2yfoGOJpCRdvSMJKzMjxF2Tj+eEwcV893dL+dPS94OOJJKSVASS0LIjYWZOHUt5/0L+44klPPPW5qAjiaQcFYEkvNzMDO6/rJyRvTtzzaOvMW9VZdCRRFKKikCSQkF2hAcvH8/gbvnMeKiC+Wuqgo4kkjJUBJI0OudGeHjaePoV5jLtwUUsXr896EgiKUFFIEmlKD+LudMn0K0gi8tmv8IbG3cGHUkk6akIJOl065TN3Csn0iknwpTZC1l5kFtmiEjbqQgkKfXuksOjV04gKyPE5FkLWVNZE3QkkaSlIpCk1b8oj7nTJwLO5PsWsmFbXdCRRJKSikCS2uBu+Tw8bQL1e5q56L4FvL9jd9CRRJKOikCS3vCenXjoivHsrGti8qyFbKmuDzqSSFJREUhKGNWnC3OuGMcHu+q5ZNZCttU2Bh1JJGmoCCRljO1fyKyp5ayvqmPK/QvZubsp6EgiSUFFICnlU4OL+c2Usaz6oJrLHniFmoY9QUcSSXgqAkk5nynrxq8vOp7XN+5k2pxF7G5sDjqSSEJTEUhKOn1kD351wRheWbeNGQ9XUN+kMhA5FBWBpKyzR/fiZ18exUtvb+WaR1+lqbkl6EgiCUlFICnt/PK+3HjOCJ5dvoXrHl/CHpWByMdoqUpJeVMmlVLf1MLNf1lOViTEreeNJhSyoGOJJAwVgaSFK08ayO6mZn7591VkR8LcfO5IzFQGIqAikDRy7amDqW9q5u4X1pCdEeaGs4arDERQEUgaMTO+c1oZu5uamf2vd8jNDPPt08qCjiUSOBWBpBUz4/tnHUN9Uwt3Pr+a7EiIa04dEnQskUCpCCTtmBk3nzuShqZmbn2mdcxg+okDg44lEhgVgaSlUMi45bxR1O9p5qanl5MVCTNlYv+gY4kEQkUgaSsjHOK2C46jcc9ibvj9m2SGjQvG9Qs6lkjcxeyCMjObbWZbzOzNQzzf2cz+ZGZLzewtM7s8VllEDiUzI8SdFx/PyUNLuP7JN3h4/rqgI4nEXSyvLJ4DnH6Y578OLHP30cApwC/MLDOGeUQOKjsSZubUsXxueDdu+MNbzHppbdCRROIqZkXg7i8C2w63C1BgrRO586P76p7BEoisjDB3Tx7Lmcf24Kanl3P3C6uDjiQSN0GOEdwJ/BF4HygALnD3g94IxsxmADMA+vXTOVyJjcyMEHdceByR8FJu+dtKGppauO5zQ3TRmaS8IIvgNGAJcCowCPi7mb3k7rsO3NHdZwIzAcrLyz2uKSWtZIRD/PL8MUTCIW5/7m0am1v47mllKgNJaUEWweXAT93dgdVm9g4wDHglwEwihEPGLV8eRWZGiHteWENDU4tuRyEpLcgieBf4LPCSmXUHygCN0klCCIVaLzrLyggx+1/v0NjczI/PHqm7lkpKilkRmNljtM4GKjazjcAPgAiAu/8GuBGYY2ZvAAZc7+5bY5VHpL323o4iMyPEvfPW0rinhZ98aRRhlYGkmJgVgbtfdITn3wf+LVbvL9IRzIzvnT6MrIwwdzz3No17Wrj1K6PJCGtNJ0kdurJY5AjMjG9+fihZGSF+/n8raWp2bruwdUBZJBWoCETa6OufGUxWRoibnl5OY3MLd158HFkZ4aBjiXxi+pNGpB2mnziQH58zgr8v+4CrHl5MfVNz0JFEPjEVgUg7TZ1Uyk++dCzzVlUy7cFF1DXqgnhJbioCkaNw0fh+3HreaOavqeKy2YuoaVAZSPJSEYgcpS+P7cPtFx7H4ne3M+X+hezc3RR0JJGjoiIQ+QS+OLoXd118PG++t5NLZi1kR11j0JFE2k1FIPIJnT6yB/dOGcvKD6q5cOYCqmoago4k0i4qApEOcOqw7syaWs66qlounLmALbvqg44k0mYqApEOctLQEh64bDzv7djNBTMXsGnn7qAjibSJikCkA00aVMTD08aztbqB8++dz4ZtdUFHEjkiFYFIBxvbv5BHpk9gZ10TF9w7n3Vba4OOJHJYKgKRGBjdtwuPzZjI7qZmzr93Pqu31AQdSeSQVAQiMTKiV2cenzGJFocLZ85n5ebqoCOJHJSKQCSGynoU8PiMiYRDxoUz5/PmezuDjiTyMSoCkRgb3C2f/7lqErmZGVx83wKWbNgRdCSRj1ARiMRB/6I8nrhqIl1yM7lk1kIq1m0LOpLIPioCkTjp0zWXJ66aSLeCLKbOfoWX12hlVkkMKgKROOrZOYfHr5pI7y45XP7AIuatqgw6koiKQCTeuhVk8/iMiQwsyefKByt4bvkHQUeSNKciEAlAUX4Wj105gbIeBVz18GL+9uamoCNJGlMRiASkS24mc6+cwKg+nfn6o6/xx6XvBx1J0pSKQCRAnbIjPDRtAmP7d+W6x1/jd4s3Bh1J0pCKQCRg+VkZPHj5eD41qJhv/3apykDiTkUgkgByMsPMurScEwYX893fLeXPr+s0kcSPikAkQWRHwsycOjZ6mmgJzy7TbCKJDxWBSALJzcxg9mXjOKZXJ74291X++bYuOpPYUxGIJJiC7AgPXTGegSV5XPlQBYt0OwqJMRWBSALqkpvJw9Mm0LNLNpc/sIilulGdxJCKQCRBlRRkMXf6BLrmRZg6+xWWb9oVdCRJUTErAjObbWZbzOzNw+xzipktMbO3zGxerLKIJKuenXN4dPpEciJhpty/UCudSUzE8ohgDnD6oZ40sy7A3cDZ7j4C+EoMs4gkrb6Fucy9cgIAl8xayIZtdQEnklQTsyJw9xeBw41yXQw85e7vRvffEqssIsluUEk+D0+bwO6mZi66bwGbdu4OOpKkkCDHCIYCXc3sBTNbbGZTD7Wjmc0wswozq6is1G17JT0N79mJh64Yz466Jibft5DK6oagI0mKCLIIMoCxwBeA04AbzGzowXZ095nuXu7u5SUlJfHMKJJQRvftwgOXj+P9nbuZcv9CdtQ1Bh1JUkCQRbAR+D93r3X3rcCLwOgA84gkhXGlhdw3tZy1lbVcOvsVquubgo4kSa5NRWBmeWYWij4eamZnm1nkE773H4ATzCzDzHKBCcDyT/iaImnhxCEl3D35eN56fxdXzFlEXeOeoCNJEmvrEcGLQLaZ9QaeAabQOivokMzsMWA+UGZmG81smpldbWZXA7j7cuBvwOvAK8Asdz/kVFMR+ajPHdOd2y4cw+L125nx0GLqm5qDjiRJKqON+5m715nZNOBud7/FzJYc7gfc/aIjvai7/xz4eRsziMgBzhrVi/qmFr7926Vc8+ir3HPJWCJhXScq7dPW/2LMzCYBk4Gno9vCsYkkIu1x3tg+3HjOCJ5dvoXrnlhCc4sHHUmSTFuPCK4D/hP4X3d/y8wGAs/HLpaItMeUSaXsbmrmv/+yguyMMD8/bxShkAUdS5JEm4rA3ecB8wCig8Zb3f3fYxlMRNpnxkmDqGts5rZn3yYnM8SN54zETGUgR9bWWUOPmlknM8sD3gSWmdl3YhtNRNrrG58dwlUnDeSRBe/yk7+uwF2nieTI2jpGcIy77wLOBf4KDKB15pCIJBAz43tnDGPqpP7MfHEttz37dtCRJAm0dYwgEr1u4FzgTndvMjP9qSGSgMyMH35xBHWNzdz+3NvkZoa56uRBQceSBNbWIrgXWAcsBV40s/6Abo4ukqBCIeNnXx5FfVMzP/nrCnIzw0yZVBp0LElQbR0svgO4Y79N683sM7GJJCIdIRwyfnXBGOqbWrjhD2+RHQnzlfK+QceSBNTWweLOZvbLvXcANbNfAHkxziYin1AkHOLOi4/jxCHFXP/k6/xp6ftBR5IE1NbB4tlANXB+9GsX8ECsQolIx8mOhJk5pZzy/oX8xxNLeHbZB0FHkgTT1iIY5O4/cPe10a8fAQNjGUxEOk5OZpj7LytnRO/OfG3uq7z0ttb1kA+1tQh2m9kJe78xs08DWiJJJIkUZEd48PJxDCzJ48qHKnjlncMtICjppK1FcDVwl5mtM7N1wJ3AVTFLJSIx0SU3k0emT6B3lxyumLOIJRt2BB1JEkCbisDdl7r7aGAUMMrdjwNOjWkyEYmJ4vws5k6fSGFeJpfOfoXlmzQTPN2163617r4reoUxwDdjkEdE4qBH52zmTp9AbmaYS2YtZPWWmqAjSYA+yY3LdTcrkSTWtzCXudMnYGZMnrWAd6vqgo4kAfkkRaBbTIgkuYEl+cydPoGGPS1cPGsB7+/QHJB0dNgiMLNqM9t1kK9qoFecMopIDJX1KODhKyaws66JS2YtpLK6IehIEmeHLQJ3L3D3Tgf5KnD3tt6nSEQS3LF9OvPA5ePYtLOeS2YtZHttY9CRJI60uKmIAFBeWsisS8t5p6qWSx94hbrGPUFHkjhREYjIPp8eXMzdFx/Pm+/t5Nu/XaqFbdKEikBEPuJzx3Tne2cM4y9vbOau51cHHUfiQEUgIh9z5YkDOXdML37x91W6SV0aUBGIyMeYGT/98ihG9urMdU8sYfWW6qAjSQypCETkoLIjYWZOHUt2JMyVDy1mZ11T0JEkRlQEInJIPTvn8JtLjmfj9jquffw1mls0eJyKVAQicljlpYX8+JyRvLiqklv+tiLoOBIDuihMRI7oovH9WL5pF/e+uJbhPTtx7nG9g44kHUhHBCLSJjecdQwTBhRy/ZOv8/pGrWOQSlQEItImkXCIuycfT3F+Flc9vJgt1fVBR5IOErMiMLPZZrbFzN48wn7jzGyPmZ0Xqywi0jGK8rOYOXUsO+qa+Oojr9KwpznoSNIBYnlEMAc4/XA7mFkY+BnwTAxziEgHGtGrM7d+ZTSL12/nB394S7ehSAExKwJ3fxE40urY1wJPAltilUNEOt4XRvXkms8M5vFFG3hkwfqg48gnFNgYgZn1Bv4/4J427DvDzCrMrKKysjL24UTkiL75+aF8bng3fvSnZcxfUxV0HPkEghwsvg243t1bjrSju89093J3Ly8pKYlDNBE5klDI+NUFYygtzuPrj77Khm1a6jJZBVkE5cDjZrYOOA+428zODTCPiLRTQXaE+6aWs6e5hRkPL9YaBkkqsCJw9wHuXurupcDvgK+5+++DyiMiR2dAcR6/vvh4Vm7exXd++7oGj5NQLKePPgbMB8rMbKOZTTOzq83s6li9p4gE4+ShJXzvjGE8/cYm7n5hTdBxpJ1idosJd7+oHfteFqscIhIfV544kGXv7+LWZ1ZS1r2Azx3TPehI0ka6slhEOsTeNQxG9OqkNQySjIpARDpMdiTMzCnlZEdCWsMgiagIRKRD9eqSwz2XjNUaBklERSAiHW6c1jBIKlqPQERi4qLx/Vj2vtYwSAY6IhCRmPn+F49hvNYwSHgqAhGJmUg4xD1awyDhqQhEJKb2rmGwva5RaxgkKBWBiMSc1jBIbBosFpG4OGtUL5Zv2sVdz69hRK9OTJlUGnQkidIRgYjEzbc+X8Znh2kNg0SjIhCRuAmFjF9dOIb+RblawyCBqAhEJK46RdcwaNIaBglDRSAicTewJJ9fX3Sc1jBIECoCEQnEKWXduP50rWGQCFQEIhKYGScN5Jwxvbj1mZU8u+yDoOOkLRWBiATGzPiZ1jAInIpARAKlNQyCpyIQkcBpDYNgqQhEJCFoDYPg6BYTIpIwtIZBMHREICIJRWsYxJ+KQEQSyv5rGMx4aDGbd2oNg1hTEYhIwinKz+K+qeVU1zcx7cFF1DboNhSxpCIQkYR0TK9O3Hnx8SzftIt/f0wziWJJRSAiCeszw7rxo7NH8NyKLdz452VBx0lZmjUkIgltyqRS1lXVcf8/36G0KJfLPj0g6EgpR0UgIgnvv84czrvb6vjxn5fRtzCXzw7vHnSklKJTQyKS8MIh4/YLxzCiV2eufew13nxvZ9CRUkrMisDMZpvZFjN78xDPTzaz183sDTN72cxGxyqLiCS/3MwMZl1aTpecCNMeXMSmnbuDjpQyYnlEMAc4/TDPvwOc7O7HAjcCM2OYRURSQPdO2dx/2ThqG5qZNqeCGk0r7RAxKwJ3fxHYdpjnX3b37dFvFwB9YpVFRFLH8J6duGvy8az8oJprH32VPc0tQUdKeokyRjAN+GvQIUQkOZw8tIQfnT2C51dW8uM/L9NSl59Q4LOGzOwztBbBCYfZZwYwA6Bfv35xSiYiieySif1ZX1XLfS+9Q2lRHlecoGmlRyvQIwIzGwXMAs5x96pD7efuM9293N3LS0pK4hdQRBLaf54xnNNGdOfGp5fxdy11edQCKwIz6wc8BUxx91VB5RCR5BUKGbddcByjenfm3x97jTc2alrp0Yjl9NHHgPlAmZltNLNpZna1mV0d3eX7QBFwt5ktMbOKWGURkdSVkxnmvkvLKczLZNqDi3h/h6aVtpcl2yBLeXm5V1SoM0Tko1Zurua8e16md9ccfvfVT5GfFfgQaEIxs8XuXn6w5xJl1pCIyCdS1qOAuyYfz9tbarhG00rbRUUgIinjpKEl3HTuSF5YWckP//SWppW2kY6dRCSlXDS+H+uqarl33lpKi/KYfuLAoCMlPBWBiKSc608bxrtVddz8l+X0LczltBE9go6U0HRqSERSTihk/PL8MYzq04XrHl/C6xt3BB0poakIRCQl5WSGmTV177TSCt7TtNJDUhGISMoqKchizuXjqG9q5ooHFlFd3xR0pISkIhCRlDakewH3TB7Lmsoavv7oa5pWehAqAhFJeScMKeamc0fy4qpKvv9HTSs9kGYNiUhauHB8P9Zvq+OeF9YwoCiPK0/StNK9VAQikja+829lvFtVx3//dTl9C3M4fWTPoCMlBJ0aEpG0EQoZvzh/NGP6duG6J5awdIOmlYKKQETSTHYkzH1TyynOz2LagxVs3F4XdKTAqQhEJO0U57dOK23Y08wVcxaxK82nlaoIRCQtDe5WwL2XjGVtZS1fn/sqTWk8rVRFICJp61ODi/nvLx3LS29v5Ybfv5m200o1a0hE0tr55X1ZX1XLXc+vobQ4j6tPHhR0pLhTEYhI2vvW58tYX1XHT/+6gn6FuZx5bHpNK1URiEjaC4WMW78ymk076/mPJ5bQs3M2x/XrGnSsuNEYgYgIrdNKZ04ZS/dO2Vz5UAUbtqXPtFIVgYhIVFF+FrMvG0fjnhYun7OInbvTY1qpikBEZD+Du+Vz75Ry1lfV8rW5i9NiWqmKQETkAJMGFfGTL43iX6ur+P//N/WnlWqwWETkIM4b24f1VbX8+h+r2VJdT2lxHkV5mRTlZ1GYl0lxfiZFeVkU5mdSkJWBmQUd+aipCEREDuGbnx9KXWMzzy7/gIp126lu2HPQ/TLDIQrzMinKby2KorxMivIyKczPpDgvi6L8zGh5tJZIbmY4oYrDku2Qp7y83CsqKoKOISJpqL6pme11jVTVNLK1poFttdHHtQ1sq2mkqjb6VdNAVU0ju5uaD/o62ZEQRdGCKMrLpDAvq/UII7/18d7te0slOxL+xNnNbLG7lx/sOR0RiIi0UXYkTM/OOfTsnNOm/esa91BV09haGLUNbFhT/UsAAASnSURBVN37OFoUVbWNVNY0sHJzNVtrG2ncc/CB6bzMMEX5WUyZ2D8mC+qoCEREYiQ3M4Pcwgz6FuYecV93p6ZhD9tqGz9aGNGjjqraBrp1yopJThWBiEgCMDMKsiMUZEfoX5QX1/fW9FERkTQXsyIws9lmtsXM3jzE82Zmd5jZajN73cyOj1UWERE5tFgeEcwBTj/M82cAQ6JfM4B7YphFREQOIWZF4O4vAtsOs8s5wEPeagHQxczS696vIiIJIMgxgt7Ahv2+3xjd9jFmNsPMKsysorKyMi7hRETSRVIMFrv7THcvd/fykpKSoOOIiKSUIIvgPaDvft/3iW4TEZE4CrII/ghMjc4emgjsdPdNAeYREUlLMbvXkJk9BpwCFAMfAD8AIgDu/htrvePSnbTOLKoDLnf3I95EyMwqgfUxCR0/xcDWoEOgHAdKlBwisdDf3Q96bj3pbjqXCsys4lA3f1IO5RCJt6QYLBYRkdhREYiIpDkVQTBmBh0gSjk+KlFyiMSVxghERNKcjghERNKcikBEJM2pCOLoSLfmjmOOvmb2vJktM7O3zOwbAWTINrNXzGxpNMOP4p3hgDxhM3vNzP4cZA6RIKgI4msOh781d7zsAb7l7scAE4Gvm9kxcc7QAJzq7qOBMcDp0SvMg/INYHmA7y8SGBVBHLXh1tzxyrHJ3V+NPq6m9QPwoHd+jWEGd/ea6LeR6FcgMxfMrA/wBWBWEO8vEjQVQZozs1LgOGBhAO8dNrMlwBbg7+4e9wxRtwHfBVoCen+RQKkI0piZ5QNPAte5+654v7+7N7v7GFrvPDvezEbGO4OZnQVscffF8X5vkUShIkhTZhahtQTmuvtTQWZx9x3A8wQzfvJp4GwzWwc8DpxqZo8EkEMkMCqCNBS98+v9wHJ3/2VAGUrMrEv0cQ7weWBFvHO4+3+6ex93LwUuBP7h7pfEO4dIkFQEcRS9Nfd8oMzMNprZtICifBqYQutfv0uiX2fGOUNP4Hkzex1YROsYgaZuigRAt5gQEUlzOiIQEUlzKgIRkTSnIhARSXMqAhGRNKciEBFJcyoCkQOYWfN+02qXmNn3OvC1S4O++6zIgTKCDiCSgHZHb30hkhZ0RCDSRma2zsxuMbM3omspDI5uLzWzf5jZ62b2nJn1i27vbmb/G11zYamZfSr6UmEzuy+6DsMz0SurRQKjIhD5uJwDTg1dsN9zO939WOBOWu9aCvBr4EF3HwXMBe6Ibr8DmBddc+F44K3o9iHAXe4+AtgBfDnGv4/IYenKYpEDmFmNu+cfZPs6WhfTWRu9ad9mdy8ys61AT3dvim7f5O7FZlYJ9HH3hv1eo5TW22kMiX5/PRBx95ti/5uJHJyOCETaxw/xuD0a9nvcjMbqJGAqApH2uWC/f+dHH79M651LASYDL0UfPwd8FfYtwtM5XiFF2kN/iYh8XE505bS9/ubue6eQdo3eMbUBuCi67VrgATP7DlAJXB7d/g1gZvQus820lsKmmKcXaSeNEYi0UXSMoNzdtwadRaQj6dSQiEia0xGBiEia0xGBiEiaUxGIiKQ5FYGISJpTEYiIpDkVgYhImvt/N9YY3VKHTMcAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "df_stats = pd.DataFrame(data=training_stats)\n",
        "plt.plot(df_stats['Training Loss'], label=\"Training\")\n",
        "plt.title(\"Training Loss\")\n",
        "plt.xlabel(\"Epoch\")\n",
        "plt.ylabel(\"Loss\")\n",
        "plt.xticks([1, 2, 3, 4])\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZXXZ6vTl-E-R",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "edddafcb-9e45-4db8-f075-4474f6ace268"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:2339: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "test_texts = test.CONTEXT.values\n",
        "test_labels = test.label.values\n",
        "\n",
        "input_ids = []\n",
        "attention_masks = []\n",
        "\n",
        "for text in test_texts:\n",
        "    encoded_dict = tokenizer.encode_plus(\n",
        "                        text,                     \n",
        "                        add_special_tokens = True, \n",
        "                        max_length = max_len,          \n",
        "                        pad_to_max_length = True,\n",
        "                        return_attention_mask = True,  \n",
        "                        return_tensors = 'pt',   \n",
        "                   )\n",
        "    \n",
        "    input_ids.append(encoded_dict['input_ids'])\n",
        "    attention_masks.append(encoded_dict['attention_mask'])\n",
        "\n",
        "input_ids = torch.cat(input_ids, dim=0)\n",
        "attention_masks = torch.cat(attention_masks, dim=0)\n",
        "labels = torch.tensor(test_labels)\n",
        "\n",
        "batch_size = 32  \n",
        "\n",
        "prediction_data = TensorDataset(input_ids, attention_masks, labels)\n",
        "prediction_sampler = SequentialSampler(prediction_data)\n",
        "prediction_dataloader = DataLoader(prediction_data, sampler=prediction_sampler, batch_size=batch_size)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "-----------------------------------------------------------------------------------------------------------------------------------\n",
        "- https://medium.com/@toprakucar/bert-modeli-ile-t%C3%BCrk%C3%A7e-metinlerde-s%C4%B1n%C4%B1fland%C4%B1rma-yapmak-260f15a65611"
      ],
      "metadata": {
        "id": "3aG3UbFpVnWw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install transformers"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gcXvWBj5Vs1m",
        "outputId": "54c4bb08-06e2-46e2-b660-786cf030b1b1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting transformers\n",
            "  Downloading transformers-4.26.0-py3-none-any.whl (6.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m96.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (21.3)\n",
            "Collecting huggingface-hub<1.0,>=0.11.0\n",
            "  Downloading huggingface_hub-0.12.0-py3-none-any.whl (190 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m26.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n",
            "  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m105.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.4.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n",
            "Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n",
            "Installing collected packages: tokenizers, huggingface-hub, transformers\n",
            "Successfully installed huggingface-hub-0.12.0 tokenizers-0.13.2 transformers-4.26.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np # linear algebra\n",
        "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
        "import tensorflow as tf\n",
        "import torch\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from transformers import BertTokenizer\n",
        "from torch.utils.data import TensorDataset, random_split\n",
        "from torch.utils.data import DataLoader, RandomSampler, SequentialSampler\n",
        "from transformers import BertForSequenceClassification, AdamW, BertConfig, Adafactor\n",
        "from transformers import get_linear_schedule_with_warmup\n",
        "import datetime\n",
        "import random\n",
        "import seaborn as sns\n",
        "import numpy as np\n",
        "import time\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.metrics import recall_score\n",
        "from sklearn.metrics import precision_score\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.metrics import accuracy_score"
      ],
      "metadata": {
        "id": "fCA3sx-69UI-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = BertTokenizer.from_pretrained('dbmdz/bert-base-turkish-128k-uncased', do_lower_case=True)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113,
          "referenced_widgets": [
            "4383c5fff8df47caaf5c9fb0b15c3c88",
            "44646114491a45648c3cbea07f611f1c",
            "aba28ef94f7f4a4dbacdb7de8948ccae",
            "2cb2a08ecb56486c8a2983f3818a1930",
            "af065f11114c494384a47d0036764c91",
            "80516465b6204c369aac4d578fda6407",
            "407fe6feca8049cbb1187219ff2efa55",
            "0f8f001d138a444eb9d527553408a95b",
            "0903440cbdd842bca13b5b341e74f7ec",
            "d9aec35cd54f4faab0936b9074698248",
            "adac947777c74e13bcd6d36564b40455",
            "611961907409400eaa6ec4cde684a6cf",
            "d5fa5f43cd0c46c79cb8c29bc8760de7",
            "9c3c5cfe7366405caf3c1f9ebb6e600e",
            "dad2b1f251b44f278c61d59f4fc4e847",
            "0418d2389c1e4f389fa51c3c411b043f",
            "0135c8422eb143529e44080261242ad7",
            "45845d7cd6294ba79cd021d7d4566eee",
            "cd32b213fece44419978083ebedaf4aa",
            "ed97ec00ba944ce2a76a2af5abf40ce1",
            "7d1dbc49343141f0baf6d705a9a8ed6c",
            "3342bddd4a5c453ba829e272c5973b16",
            "14d79a3bae574b9cb50eb0dfe72c06db",
            "2bac87d2a5234178ac89199c51e5dc32",
            "0b881ccdf17b440ca5d7cf88dd222a27",
            "6f7cad1a9419420781c461ef471464d2",
            "56b1d3c8055f46dda5141bf3ae2867bc",
            "1ec7178c6646451ba7836d0d4f5bee91",
            "2cf060543a08456984f1d71363b7b237",
            "a8b2c81316db4d8a89bfc34869db8d91",
            "c2caaa4f71ca49d58da7a15c1b75cbb7",
            "ca19bbd4b1334743aea25ef2e775334a",
            "b1c38a910a9542338310de61e71c16bc"
          ]
        },
        "id": "TQ5_DzduVpln",
        "outputId": "fcdfc8ab-4853-4e1c-ec08-0fe20e7969b3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/1.23M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4383c5fff8df47caaf5c9fb0b15c3c88"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/59.0 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "611961907409400eaa6ec4cde684a6cf"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/386 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "14d79a3bae574b9cb50eb0dfe72c06db"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv(\"/content/drive/MyDrive/datas/cleanBankData2.csv\")\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "MVF5kRkaVrAO",
        "outputId": "78bebc33-1074-4210-85f8-a8f9d0af85cc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                     name  \\\n",
              "0               MOOV Mov Uyarı Vermeden Kısıtlama Getirdi   \n",
              "1       Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...   \n",
              "2       Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...   \n",
              "3                        Ziraat Bankası Borç Ödeme Sorunu   \n",
              "4                        Ziraat Bankası Bloke Açma Sorunu   \n",
              "...                                                   ...   \n",
              "246407                KKB'de Borçlarım Ödenmedi Gözüküyor   \n",
              "246408                            İş Bankası İlgisizliği!   \n",
              "246409          Garanti Bankası Her Şey Gönlünüzce Olsun!   \n",
              "246410            ING Bank Ticari Kredi Kartı Alamıyorum!   \n",
              "246411             TEB İnsanları Mağdur Ediyor, Aldatıyor   \n",
              "\n",
              "                                                     link            label  \\\n",
              "0       https://www.sikayetvar.com/moov/moov-mov-uyari...  Kredi İşlemleri   \n",
              "1       https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "2       https://www.sikayetvar.com/hepsiburada/hepsibu...  Kredi İşlemleri   \n",
              "3       https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "4       https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "...                                                   ...              ...   \n",
              "246407  https://www.sikayetvar.com/kredi-kayit-burosu/...         KKB Skor   \n",
              "246408  https://www.sikayetvar.com/is-bankasi/is-banka...         KKB Skor   \n",
              "246409  https://www.sikayetvar.com/garanti-bbva/garant...         KKB Skor   \n",
              "246410  https://www.sikayetvar.com/ing/ing-bank-ticari...         KKB Skor   \n",
              "246411  https://www.sikayetvar.com/teb/teb-insanlari-m...         KKB Skor   \n",
              "\n",
              "                                                     text  \n",
              "0       moov ad araç kiralamak şirket sürek araç kiral...  \n",
              "1       ziraat banka bankkart kart kredi kart alışveri...  \n",
              "2       22 ara 2022 tarih hepsiburadacomdan yapmak 475...  \n",
              "3       banka kart par olmak rağmen kredi kart bor öde...  \n",
              "4       ziraat banka kredi bor dolay hesap bloke koyma...  \n",
              "...                                                   ...  \n",
              "246407  2011 yıl çeşit banka borç var var şirket aracı...  \n",
              "246408  2010 yıl açtırdığım 2015 yıl kapanan kredi kar...  \n",
              "246409  merhaba müşteri no 16790602 kkb 1589 3 ay şâhı...  \n",
              "246410  yaklaşık 1 yıldır pos hesap kul kkb puan yeter...  \n",
              "246411  teb kredi kart başvuru yaptım 12122018 9 bin t...  \n",
              "\n",
              "[246412 rows x 4 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0004ef40-567c-491f-bf77-ee7a4c7866c6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>name</th>\n",
              "      <th>link</th>\n",
              "      <th>label</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi</td>\n",
              "      <td>https://www.sikayetvar.com/moov/moov-mov-uyari...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>moov ad araç kiralamak şirket sürek araç kiral...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>ziraat banka bankkart kart kredi kart alışveri...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "      <td>https://www.sikayetvar.com/hepsiburada/hepsibu...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>22 ara 2022 tarih hepsiburadacomdan yapmak 475...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>banka kart par olmak rağmen kredi kart bor öde...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>ziraat banka kredi bor dolay hesap bloke koyma...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>246407</th>\n",
              "      <td>KKB'de Borçlarım Ödenmedi Gözüküyor</td>\n",
              "      <td>https://www.sikayetvar.com/kredi-kayit-burosu/...</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>2011 yıl çeşit banka borç var var şirket aracı...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>246408</th>\n",
              "      <td>İş Bankası İlgisizliği!</td>\n",
              "      <td>https://www.sikayetvar.com/is-bankasi/is-banka...</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>2010 yıl açtırdığım 2015 yıl kapanan kredi kar...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>246409</th>\n",
              "      <td>Garanti Bankası Her Şey Gönlünüzce Olsun!</td>\n",
              "      <td>https://www.sikayetvar.com/garanti-bbva/garant...</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>merhaba müşteri no 16790602 kkb 1589 3 ay şâhı...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>246410</th>\n",
              "      <td>ING Bank Ticari Kredi Kartı Alamıyorum!</td>\n",
              "      <td>https://www.sikayetvar.com/ing/ing-bank-ticari...</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>yaklaşık 1 yıldır pos hesap kul kkb puan yeter...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>246411</th>\n",
              "      <td>TEB İnsanları Mağdur Ediyor, Aldatıyor</td>\n",
              "      <td>https://www.sikayetvar.com/teb/teb-insanlari-m...</td>\n",
              "      <td>KKB Skor</td>\n",
              "      <td>teb kredi kart başvuru yaptım 12122018 9 bin t...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>246412 rows × 4 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0004ef40-567c-491f-bf77-ee7a4c7866c6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0004ef40-567c-491f-bf77-ee7a4c7866c6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0004ef40-567c-491f-bf77-ee7a4c7866c6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NE2vauWuWE8H",
        "outputId": "62ec7a0c-8687-4298-ecbd-80c33e17093c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 246412 entries, 0 to 246411\n",
            "Data columns (total 4 columns):\n",
            " #   Column  Non-Null Count   Dtype \n",
            "---  ------  --------------   ----- \n",
            " 0   name    246412 non-null  object\n",
            " 1   link    246412 non-null  object\n",
            " 2   label   246412 non-null  object\n",
            " 3   text    246412 non-null  object\n",
            "dtypes: object(4)\n",
            "memory usage: 7.5+ MB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df[\"CONTEXT\"] = df[\"name\"]+\". \"+df[\"text\"]\n",
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "Wl3OfuGHV2pP",
        "outputId": "d4532833-2389-43e2-c691-17baa0c3d699"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                name  \\\n",
              "0          MOOV Mov Uyarı Vermeden Kısıtlama Getirdi   \n",
              "1  Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...   \n",
              "2  Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...   \n",
              "3                   Ziraat Bankası Borç Ödeme Sorunu   \n",
              "4                   Ziraat Bankası Bloke Açma Sorunu   \n",
              "\n",
              "                                                link            label  \\\n",
              "0  https://www.sikayetvar.com/moov/moov-mov-uyari...  Kredi İşlemleri   \n",
              "1  https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "2  https://www.sikayetvar.com/hepsiburada/hepsibu...  Kredi İşlemleri   \n",
              "3  https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "4  https://www.sikayetvar.com/ziraat-bankasi/zira...  Kredi İşlemleri   \n",
              "\n",
              "                                                text  \\\n",
              "0  moov ad araç kiralamak şirket sürek araç kiral...   \n",
              "1  ziraat banka bankkart kart kredi kart alışveri...   \n",
              "2  22 ara 2022 tarih hepsiburadacomdan yapmak 475...   \n",
              "3  banka kart par olmak rağmen kredi kart bor öde...   \n",
              "4  ziraat banka kredi bor dolay hesap bloke koyma...   \n",
              "\n",
              "                                             CONTEXT  \n",
              "0  MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...  \n",
              "1  Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...  \n",
              "2  Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...  \n",
              "3  Ziraat Bankası Borç Ödeme Sorunu. banka kart p...  \n",
              "4  Ziraat Bankası Bloke Açma Sorunu. ziraat banka...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e8adbb6c-bd42-4faa-bc9d-d254434406b4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>name</th>\n",
              "      <th>link</th>\n",
              "      <th>label</th>\n",
              "      <th>text</th>\n",
              "      <th>CONTEXT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi</td>\n",
              "      <td>https://www.sikayetvar.com/moov/moov-mov-uyari...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>moov ad araç kiralamak şirket sürek araç kiral...</td>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>ziraat banka bankkart kart kredi kart alışveri...</td>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "      <td>https://www.sikayetvar.com/hepsiburada/hepsibu...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>22 ara 2022 tarih hepsiburadacomdan yapmak 475...</td>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>banka kart par olmak rağmen kredi kart bor öde...</td>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu. banka kart p...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu</td>\n",
              "      <td>https://www.sikayetvar.com/ziraat-bankasi/zira...</td>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>ziraat banka kredi bor dolay hesap bloke koyma...</td>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu. ziraat banka...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e8adbb6c-bd42-4faa-bc9d-d254434406b4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e8adbb6c-bd42-4faa-bc9d-d254434406b4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e8adbb6c-bd42-4faa-bc9d-d254434406b4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.drop([\"name\", \"link\", \"text\"], axis = 1)\n",
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "kSAvH7sWWBOg",
        "outputId": "00819689-779c-4387-80ed-85bf33e92862"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "             label                                            CONTEXT\n",
              "0  Kredi İşlemleri  MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...\n",
              "1  Kredi İşlemleri  Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...\n",
              "2  Kredi İşlemleri  Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...\n",
              "3  Kredi İşlemleri  Ziraat Bankası Borç Ödeme Sorunu. banka kart p...\n",
              "4  Kredi İşlemleri  Ziraat Bankası Bloke Açma Sorunu. ziraat banka..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-90f77db6-cee0-439d-88a2-726c8ad0ebb8\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>CONTEXT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu. banka kart p...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Kredi İşlemleri</td>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu. ziraat banka...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-90f77db6-cee0-439d-88a2-726c8ad0ebb8')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-90f77db6-cee0-439d-88a2-726c8ad0ebb8 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-90f77db6-cee0-439d-88a2-726c8ad0ebb8');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2HqhV_gAWNnG",
        "outputId": "266b3e59-13e7-44da-be4a-1a40b0a5b107"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Thu Feb  2 06:58:44 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   58C    P0    30W /  70W |   4632MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|    0   N/A  N/A      2151      C                                    4629MiB |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.label.value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bfmUJ7qyWS9p",
        "outputId": "7b41f837-7bd5-40cb-f444-ea957ecd2c65"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Kart İşlemleri        37296\n",
              "Kredi İşlemleri       36904\n",
              "Diğer                 36626\n",
              "Hesap İşlemleri       36504\n",
              "Kargo                 35392\n",
              "Limit                 30802\n",
              "Müşteri Temsilcisi    20266\n",
              "KKB Skor              12622\n",
              "Name: label, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "le = LabelEncoder()\n",
        "df.label = le.fit_transform(df.label)\n",
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "fuc9c1LuWs5T",
        "outputId": "eb43a12e-a75c-4d54-9c94-70c9efcbb8eb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   label                                            CONTEXT\n",
              "0      5  MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...\n",
              "1      5  Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...\n",
              "2      5  Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...\n",
              "3      5  Ziraat Bankası Borç Ödeme Sorunu. banka kart p...\n",
              "4      5  Ziraat Bankası Bloke Açma Sorunu. ziraat banka..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c5f2d3c1-6538-4c00-8390-f565a44c1c94\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>CONTEXT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>5</td>\n",
              "      <td>MOOV Mov Uyarı Vermeden Kısıtlama Getirdi. moo...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Bankkart Arka Yüzündeki Kredi K...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>5</td>\n",
              "      <td>Hepsiburada.com Sipariş İptaliyle 500 TL Zarar...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Borç Ödeme Sorunu. banka kart p...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Bloke Açma Sorunu. ziraat banka...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c5f2d3c1-6538-4c00-8390-f565a44c1c94')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c5f2d3c1-6538-4c00-8390-f565a44c1c94 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c5f2d3c1-6538-4c00-8390-f565a44c1c94');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.label.value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QsaKOJ0j695P",
        "outputId": "1fb38055-4d1a-4d13-eb91-0b1a06864a4c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "4    37296\n",
              "5    36904\n",
              "0    36626\n",
              "1    36504\n",
              "3    35392\n",
              "6    30802\n",
              "7    20266\n",
              "2    12622\n",
              "Name: label, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "training = df.groupby(\"label\").apply(lambda x: x.sample(frac = 0.8))\n",
        "test = pd.concat([df, training]).drop_duplicates(keep = False)\n",
        "test"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "hilkZoR5W2BH",
        "outputId": "5281f8a4-668a-422d-b4d0-6d7e15eec0c2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        label                                            CONTEXT\n",
              "7           5  Ziraat Bankası Onaylanan Kredi Tutarı. temel i...\n",
              "21          5  Akbank Axess Yanıltıcı Kampanya Yapıyor Ve Uym...\n",
              "28          5  VakıfBank'tan Kredi Çekenlere Durumum Örnek Ol...\n",
              "31          5  Ziraat Bankası Kredi Kartı Limit. merhaba bank...\n",
              "36          5  Ziraat Bankası Kk Tahsilatı Nedir. ben ziraat ...\n",
              "...       ...                                                ...\n",
              "231739      3  Passolig Sorumsuzluk. Başka Bir Şey Değil. pas...\n",
              "232107      3  Sürat Kargo Kargonun Teslim Edilmemesi. hatay ...\n",
              "232257      3  Sürat Kargo Kayıt Bulunama Di Hatası. surat ka...\n",
              "233322      3  Kuryenet Teslim Edilemeyen Ürün. kuryenet turk...\n",
              "233995      3  PTT Kargo Gönderi İle İlgili. trendyol kp04015...\n",
              "\n",
              "[16870 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c5ecbf4a-f21b-4801-8e78-42b02441a90c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>CONTEXT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Onaylanan Kredi Tutarı. temel i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>5</td>\n",
              "      <td>Akbank Axess Yanıltıcı Kampanya Yapıyor Ve Uym...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>5</td>\n",
              "      <td>VakıfBank'tan Kredi Çekenlere Durumum Örnek Ol...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Kredi Kartı Limit. merhaba bank...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36</th>\n",
              "      <td>5</td>\n",
              "      <td>Ziraat Bankası Kk Tahsilatı Nedir. ben ziraat ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>231739</th>\n",
              "      <td>3</td>\n",
              "      <td>Passolig Sorumsuzluk. Başka Bir Şey Değil. pas...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>232107</th>\n",
              "      <td>3</td>\n",
              "      <td>Sürat Kargo Kargonun Teslim Edilmemesi. hatay ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>232257</th>\n",
              "      <td>3</td>\n",
              "      <td>Sürat Kargo Kayıt Bulunama Di Hatası. surat ka...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233322</th>\n",
              "      <td>3</td>\n",
              "      <td>Kuryenet Teslim Edilemeyen Ürün. kuryenet turk...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>233995</th>\n",
              "      <td>3</td>\n",
              "      <td>PTT Kargo Gönderi İle İlgili. trendyol kp04015...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>16870 rows × 2 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c5ecbf4a-f21b-4801-8e78-42b02441a90c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c5ecbf4a-f21b-4801-8e78-42b02441a90c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c5ecbf4a-f21b-4801-8e78-42b02441a90c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "res = test.groupby(\"CONTEXT\")[\"label\"].count().sum()\n",
        "res"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RPwTh2BKXXVG",
        "outputId": "e1b6f0f0-8fee-4ebe-ff9b-e25bb3168b72"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "16870"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(training), len(test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Wg1mO87XoPd",
        "outputId": "8451ee0a-de5b-4442-8a3b-0cb75d33e1dd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(197131, 16870)"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_texts = training.CONTEXT.values\n",
        "train_labels = training.label.values \n",
        "train_labels"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h5Ju89HZXwDO",
        "outputId": "b8baeac6-2c8e-498b-e330-164d7085e6d7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 0, 0, ..., 7, 7, 7])"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "input_ids = []\n",
        "attention_masks = []\n",
        "max_len = 100\n",
        "\n",
        "\n",
        "for text in train_texts:\n",
        "    encoded_dict = tokenizer.encode_plus(\n",
        "                        str(text),                     \n",
        "                        add_special_tokens = True,\n",
        "                        max_length = max_len,      \n",
        "                        pad_to_max_length = True,\n",
        "                        return_attention_mask = True, \n",
        "                        return_tensors = 'pt',\n",
        "                   )\n",
        "    \n",
        "    input_ids.append(encoded_dict['input_ids'])\n",
        "    attention_masks.append(encoded_dict['attention_mask'])\n",
        "\n",
        "input_ids = torch.cat(input_ids, dim=0)\n",
        "attention_masks = torch.cat(attention_masks, dim=0)\n",
        "labels = torch.tensor(train_labels)\n",
        "\n",
        "print('Original: ', train_texts[0])\n",
        "print('Token IDs:', input_ids[0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W36yQXefYCxd",
        "outputId": "58013ced-d664-4293-8048-769074ed22ec"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original:  Kredi Kayıt Bürosu Sicil Affı Kredi Puanı. birey banka sicil af kanu dâhil 02112022de yürürlük giren kanu madde tarih 5411 sayıl kanu 1 madde uyarınca karşı çek protesto senet kredi kredi kart borç ilişkin kayıt dikkat alınmaması hak kanu aşağı geçici mad\n",
            "Token IDs: tensor([     2,   4139,   3621,   2312,   4086,  12094,  36238,   4139,  11056,\n",
            "            18,   4632,   3730,  12094,   3285,   2476,   1012,   3931,   5621,\n",
            "          8215, 115459,   1045,   1942,   6131,  30562,   5485,   2476,   1012,\n",
            "          3077,   2388, 127610,  23282,   2476,   1012,     21,   3077,   8697,\n",
            "         40209,  15012,   8306,  11194,   4139,   4139,   4219,   9271,  89805,\n",
            "          3621,   2868,  33468,   2254,   2476,   1012,  10892,   2893,   3538,\n",
            "          3213,   2690,      3,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0,      0,      0,      0,      0,      0,      0,      0,      0,\n",
            "             0])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset = TensorDataset(input_ids, attention_masks, labels)"
      ],
      "metadata": {
        "id": "JvieF_yLYPNW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "batch_size = 16\n",
        "\n",
        "train_dataloader = DataLoader(\n",
        "            train_dataset,  \n",
        "            sampler = RandomSampler(train_dataset), \n",
        "            batch_size = batch_size \n",
        "        )"
      ],
      "metadata": {
        "id": "0WaH_Ct6YZXG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "number_of_categories = len(df['label'].unique())\n",
        "number_of_categories"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mnp7WLcMYfue",
        "outputId": "09da981d-da3e-4192-dd59-00cab579edad"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = BertForSequenceClassification.from_pretrained(\n",
        "    \"dbmdz/bert-base-turkish-128k-uncased\",\n",
        "    num_labels = number_of_categories, \n",
        "    output_attentions = False,\n",
        "    output_hidden_states = False,\n",
        ")\n",
        "\n",
        "model.cuda()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "b7d8a9dee32b455ea4be76ca8eee877e",
            "1c5aac7c12964cb889ed2c93e743bf40",
            "1242cd42339d41a2b51ba3427aecbd03",
            "e94e228f50b94c3e94aa64d1cebf0a38",
            "a991ed66559b4640a2933aa3977e933b",
            "88cf19b41f9946739c0e5984cfb4f33b",
            "5b4ba0e17ea546359a2798ee56ccc855",
            "038e24a9d8b14ee997d8de44dc7bfad5",
            "0002d351601b4c76a318ebe611480624",
            "2736b3e029f54df4bf5dc3143804aa4c",
            "884bba6d9c3645a8a661e15085fe2a15"
          ]
        },
        "id": "z3FG1XKiYj5P",
        "outputId": "b54a2432-b36e-48ff-ca61-e4dd2ec5b9ae"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/740M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b7d8a9dee32b455ea4be76ca8eee877e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the model checkpoint at dbmdz/bert-base-turkish-128k-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.bias', 'cls.predictions.decoder.bias', 'cls.predictions.decoder.weight']\n",
            "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at dbmdz/bert-base-turkish-128k-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "BertForSequenceClassification(\n",
              "  (bert): BertModel(\n",
              "    (embeddings): BertEmbeddings(\n",
              "      (word_embeddings): Embedding(128000, 768, padding_idx=0)\n",
              "      (position_embeddings): Embedding(512, 768)\n",
              "      (token_type_embeddings): Embedding(2, 768)\n",
              "      (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "      (dropout): Dropout(p=0.1, inplace=False)\n",
              "    )\n",
              "    (encoder): BertEncoder(\n",
              "      (layer): ModuleList(\n",
              "        (0): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (1): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (2): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (3): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (4): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (5): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (6): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (7): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (8): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (9): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (10): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "        (11): BertLayer(\n",
              "          (attention): BertAttention(\n",
              "            (self): BertSelfAttention(\n",
              "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "            (output): BertSelfOutput(\n",
              "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "              (dropout): Dropout(p=0.1, inplace=False)\n",
              "            )\n",
              "          )\n",
              "          (intermediate): BertIntermediate(\n",
              "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
              "            (intermediate_act_fn): GELUActivation()\n",
              "          )\n",
              "          (output): BertOutput(\n",
              "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
              "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
              "            (dropout): Dropout(p=0.1, inplace=False)\n",
              "          )\n",
              "        )\n",
              "      )\n",
              "    )\n",
              "    (pooler): BertPooler(\n",
              "      (dense): Linear(in_features=768, out_features=768, bias=True)\n",
              "      (activation): Tanh()\n",
              "    )\n",
              "  )\n",
              "  (dropout): Dropout(p=0.1, inplace=False)\n",
              "  (classifier): Linear(in_features=768, out_features=8, bias=True)\n",
              ")"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "epochs = 8 \n",
        "\n",
        "optimizer = torch.optim.Adam(model.parameters(),\n",
        "                  lr = 5e-5,\n",
        "                  eps = 1e-8 \n",
        "                )\n",
        "\n",
        "total_steps = len(train_dataloader) * epochs\n",
        "scheduler = get_linear_schedule_with_warmup(optimizer, \n",
        "                                            num_warmup_steps = 0,\n",
        "                                            num_training_steps = total_steps)"
      ],
      "metadata": {
        "id": "3ugggpigYp5H"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def format_time(elapsed):\n",
        "    elapsed_rounded = int(round((elapsed)))\n",
        "    return str(datetime.timedelta(seconds=elapsed_rounded))"
      ],
      "metadata": {
        "id": "LO9omxcRY98Q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "seed_val = 3000\n",
        "\n",
        "random.seed(seed_val)\n",
        "np.random.seed(seed_val)\n",
        "torch.manual_seed(seed_val)\n",
        "torch.cuda.manual_seed_all(seed_val)\n",
        "\n",
        "training_stats = []\n",
        "total_t0 = time.time()\n",
        "\n",
        "for epoch_i in range(0, epochs):\n",
        "    print('======== Epoch {:} / {:} ========'.format(epoch_i + 1, epochs))\n",
        "    t0 = time.time()\n",
        "    total_train_loss = 0\n",
        "    model.train()\n",
        "    \n",
        "    for step, batch in enumerate(train_dataloader):\n",
        "        if step % 10 == 0 and not step == 0:\n",
        "            elapsed = format_time(time.time() - t0)\n",
        "            print('Batch {:>5,}  of  {:>5,}.    Elapsed: {:}.'.format(step, len(train_dataloader), elapsed))\n",
        "\n",
        "        b_input_ids = batch[0].to(device)\n",
        "        b_input_mask = batch[1].to(device)\n",
        "        b_labels = batch[2].to(device)\n",
        "\n",
        "        model.zero_grad()        \n",
        "        output = model(b_input_ids, \n",
        "                             token_type_ids=None, \n",
        "                             attention_mask=b_input_mask, \n",
        "                             labels=b_labels)\n",
        "        loss = output['loss']\n",
        "        logits = output['logits']\n",
        "        total_train_loss += loss.item()\n",
        "        loss.backward()\n",
        "        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
        "        optimizer.step()\n",
        "        scheduler.step()\n",
        "\n",
        "    avg_train_loss = total_train_loss / len(train_dataloader)            \n",
        "    training_time = format_time(time.time() - t0)\n",
        "\n",
        "    print(\"Average training loss: {0:.2f}\".format(avg_train_loss))\n",
        "    print(\"Training epoch took: {:}\".format(training_time))\n",
        "\n",
        "    training_stats.append(\n",
        "        {\n",
        "            'epoch': epoch_i + 1,\n",
        "            'Training Loss': avg_train_loss,\n",
        "            'Training Time': training_time,\n",
        "        }\n",
        "    )\n",
        "\n",
        "print(\"Training completed in {:} (h:mm:ss)\".format(format_time(time.time()-total_t0)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HQTTAohmZB8T",
        "outputId": "8c486b2c-fc59-4c6c-90ac-a772eec0ba48"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "======== Epoch 1 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:17.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:20.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:23.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:26.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:29.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:32.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:35.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:38.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:41.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:44.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:47.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:50.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:53.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:56.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:59.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:02.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:05.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:08.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:11.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:56.\n",
            "Batch   390  of  1,055.    Elapsed: 0:01:59.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:02.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:43.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:29.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:32.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:15.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:18.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:55.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:58.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:01.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:04.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:07.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.86\n",
            "Training epoch took: 0:05:24\n",
            "======== Epoch 2 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:10.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:56.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:45.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:31.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:17.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:20.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:03.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:06.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.64\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 3 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:11.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:57.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:32.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:17.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:04.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:07.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.47\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 4 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:11.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:56.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:32.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:18.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:04.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:07.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.32\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 5 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:24.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:11.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:57.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:31.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:18.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:03.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:07.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.23\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 6 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:10.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:56.\n",
            "Batch   390  of  1,055.    Elapsed: 0:01:59.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:31.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:17.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:03.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:07.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.17\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 7 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:10.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:56.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:45.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:32.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:17.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:03.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:06.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.12\n",
            "Training epoch took: 0:05:23\n",
            "======== Epoch 8 / 8 ========\n",
            "Batch    10  of  1,055.    Elapsed: 0:00:03.\n",
            "Batch    20  of  1,055.    Elapsed: 0:00:06.\n",
            "Batch    30  of  1,055.    Elapsed: 0:00:09.\n",
            "Batch    40  of  1,055.    Elapsed: 0:00:12.\n",
            "Batch    50  of  1,055.    Elapsed: 0:00:15.\n",
            "Batch    60  of  1,055.    Elapsed: 0:00:18.\n",
            "Batch    70  of  1,055.    Elapsed: 0:00:21.\n",
            "Batch    80  of  1,055.    Elapsed: 0:00:25.\n",
            "Batch    90  of  1,055.    Elapsed: 0:00:28.\n",
            "Batch   100  of  1,055.    Elapsed: 0:00:31.\n",
            "Batch   110  of  1,055.    Elapsed: 0:00:34.\n",
            "Batch   120  of  1,055.    Elapsed: 0:00:37.\n",
            "Batch   130  of  1,055.    Elapsed: 0:00:40.\n",
            "Batch   140  of  1,055.    Elapsed: 0:00:43.\n",
            "Batch   150  of  1,055.    Elapsed: 0:00:46.\n",
            "Batch   160  of  1,055.    Elapsed: 0:00:49.\n",
            "Batch   170  of  1,055.    Elapsed: 0:00:52.\n",
            "Batch   180  of  1,055.    Elapsed: 0:00:55.\n",
            "Batch   190  of  1,055.    Elapsed: 0:00:58.\n",
            "Batch   200  of  1,055.    Elapsed: 0:01:01.\n",
            "Batch   210  of  1,055.    Elapsed: 0:01:04.\n",
            "Batch   220  of  1,055.    Elapsed: 0:01:07.\n",
            "Batch   230  of  1,055.    Elapsed: 0:01:11.\n",
            "Batch   240  of  1,055.    Elapsed: 0:01:14.\n",
            "Batch   250  of  1,055.    Elapsed: 0:01:17.\n",
            "Batch   260  of  1,055.    Elapsed: 0:01:20.\n",
            "Batch   270  of  1,055.    Elapsed: 0:01:23.\n",
            "Batch   280  of  1,055.    Elapsed: 0:01:26.\n",
            "Batch   290  of  1,055.    Elapsed: 0:01:29.\n",
            "Batch   300  of  1,055.    Elapsed: 0:01:32.\n",
            "Batch   310  of  1,055.    Elapsed: 0:01:35.\n",
            "Batch   320  of  1,055.    Elapsed: 0:01:38.\n",
            "Batch   330  of  1,055.    Elapsed: 0:01:41.\n",
            "Batch   340  of  1,055.    Elapsed: 0:01:44.\n",
            "Batch   350  of  1,055.    Elapsed: 0:01:47.\n",
            "Batch   360  of  1,055.    Elapsed: 0:01:50.\n",
            "Batch   370  of  1,055.    Elapsed: 0:01:53.\n",
            "Batch   380  of  1,055.    Elapsed: 0:01:57.\n",
            "Batch   390  of  1,055.    Elapsed: 0:02:00.\n",
            "Batch   400  of  1,055.    Elapsed: 0:02:03.\n",
            "Batch   410  of  1,055.    Elapsed: 0:02:06.\n",
            "Batch   420  of  1,055.    Elapsed: 0:02:09.\n",
            "Batch   430  of  1,055.    Elapsed: 0:02:12.\n",
            "Batch   440  of  1,055.    Elapsed: 0:02:15.\n",
            "Batch   450  of  1,055.    Elapsed: 0:02:18.\n",
            "Batch   460  of  1,055.    Elapsed: 0:02:21.\n",
            "Batch   470  of  1,055.    Elapsed: 0:02:24.\n",
            "Batch   480  of  1,055.    Elapsed: 0:02:27.\n",
            "Batch   490  of  1,055.    Elapsed: 0:02:30.\n",
            "Batch   500  of  1,055.    Elapsed: 0:02:33.\n",
            "Batch   510  of  1,055.    Elapsed: 0:02:36.\n",
            "Batch   520  of  1,055.    Elapsed: 0:02:39.\n",
            "Batch   530  of  1,055.    Elapsed: 0:02:42.\n",
            "Batch   540  of  1,055.    Elapsed: 0:02:46.\n",
            "Batch   550  of  1,055.    Elapsed: 0:02:49.\n",
            "Batch   560  of  1,055.    Elapsed: 0:02:52.\n",
            "Batch   570  of  1,055.    Elapsed: 0:02:55.\n",
            "Batch   580  of  1,055.    Elapsed: 0:02:58.\n",
            "Batch   590  of  1,055.    Elapsed: 0:03:01.\n",
            "Batch   600  of  1,055.    Elapsed: 0:03:04.\n",
            "Batch   610  of  1,055.    Elapsed: 0:03:07.\n",
            "Batch   620  of  1,055.    Elapsed: 0:03:10.\n",
            "Batch   630  of  1,055.    Elapsed: 0:03:13.\n",
            "Batch   640  of  1,055.    Elapsed: 0:03:16.\n",
            "Batch   650  of  1,055.    Elapsed: 0:03:19.\n",
            "Batch   660  of  1,055.    Elapsed: 0:03:22.\n",
            "Batch   670  of  1,055.    Elapsed: 0:03:25.\n",
            "Batch   680  of  1,055.    Elapsed: 0:03:28.\n",
            "Batch   690  of  1,055.    Elapsed: 0:03:32.\n",
            "Batch   700  of  1,055.    Elapsed: 0:03:35.\n",
            "Batch   710  of  1,055.    Elapsed: 0:03:38.\n",
            "Batch   720  of  1,055.    Elapsed: 0:03:41.\n",
            "Batch   730  of  1,055.    Elapsed: 0:03:44.\n",
            "Batch   740  of  1,055.    Elapsed: 0:03:47.\n",
            "Batch   750  of  1,055.    Elapsed: 0:03:50.\n",
            "Batch   760  of  1,055.    Elapsed: 0:03:53.\n",
            "Batch   770  of  1,055.    Elapsed: 0:03:56.\n",
            "Batch   780  of  1,055.    Elapsed: 0:03:59.\n",
            "Batch   790  of  1,055.    Elapsed: 0:04:02.\n",
            "Batch   800  of  1,055.    Elapsed: 0:04:05.\n",
            "Batch   810  of  1,055.    Elapsed: 0:04:08.\n",
            "Batch   820  of  1,055.    Elapsed: 0:04:11.\n",
            "Batch   830  of  1,055.    Elapsed: 0:04:14.\n",
            "Batch   840  of  1,055.    Elapsed: 0:04:18.\n",
            "Batch   850  of  1,055.    Elapsed: 0:04:21.\n",
            "Batch   860  of  1,055.    Elapsed: 0:04:24.\n",
            "Batch   870  of  1,055.    Elapsed: 0:04:27.\n",
            "Batch   880  of  1,055.    Elapsed: 0:04:30.\n",
            "Batch   890  of  1,055.    Elapsed: 0:04:33.\n",
            "Batch   900  of  1,055.    Elapsed: 0:04:36.\n",
            "Batch   910  of  1,055.    Elapsed: 0:04:39.\n",
            "Batch   920  of  1,055.    Elapsed: 0:04:42.\n",
            "Batch   930  of  1,055.    Elapsed: 0:04:45.\n",
            "Batch   940  of  1,055.    Elapsed: 0:04:48.\n",
            "Batch   950  of  1,055.    Elapsed: 0:04:51.\n",
            "Batch   960  of  1,055.    Elapsed: 0:04:54.\n",
            "Batch   970  of  1,055.    Elapsed: 0:04:57.\n",
            "Batch   980  of  1,055.    Elapsed: 0:05:00.\n",
            "Batch   990  of  1,055.    Elapsed: 0:05:03.\n",
            "Batch 1,000  of  1,055.    Elapsed: 0:05:06.\n",
            "Batch 1,010  of  1,055.    Elapsed: 0:05:10.\n",
            "Batch 1,020  of  1,055.    Elapsed: 0:05:13.\n",
            "Batch 1,030  of  1,055.    Elapsed: 0:05:16.\n",
            "Batch 1,040  of  1,055.    Elapsed: 0:05:19.\n",
            "Batch 1,050  of  1,055.    Elapsed: 0:05:22.\n",
            "Average training loss: 0.09\n",
            "Training epoch took: 0:05:23\n",
            "Training completed in 0:43:06 (h:mm:ss)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.save_pretrained(\"/content/drive/MyDrive/models/bert-bank-tokenizer\")\n",
        "model.save_pretrained(\"/content/drive/MyDrive/models/bert-bank-model\")"
      ],
      "metadata": {
        "id": "cIeiw2hHZJlf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_stats = pd.DataFrame(data=training_stats)\n",
        "plt.plot(df_stats['Training Loss'], label=\"Training\")\n",
        "plt.title(\"Training Loss\")\n",
        "plt.xlabel(\"Epoch\")\n",
        "plt.ylabel(\"Loss\")\n",
        "plt.xticks([1, 2, 3, 4])\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "AV3gTI19ZTog",
        "outputId": "ad19c73f-c329-4e3b-f197-a5861bc8afc2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3hV5Zn+8e+TkBASQkJCOEgSDgoioIBGDqKgVgWtxdp2FESrrYr1MFNHa6sz/U07TjvTTqfWqmhLEWs9ofXQodYBq+IZlaCABAQRwQQNhEMA5QzP74+9glsMECArax/uz3Xlcq+11977yVWae7/redd6zd0REZH0lRF1ASIiEi0FgYhImlMQiIikOQWBiEiaUxCIiKQ5BYGISJpTEEjaMrP/M7NLm/tYkWRjuo5AkomZfRq3mQtsA3YF21e5+0MtX9WhM7NTgQfdvTTqWiR9tYq6AJGD4e5tGx6b2XLgCnd/bu/jzKyVu+9sydpEkpVODUlKMLNTzazGzH5kZrXAfWbW3syeNrM6M1sfPC6Ne82LZnZF8PgyM3vVzP4nOPZDMzv7EI/tYWYvm9kmM3vOzCaa2YOH8DsdE3xuvZlVmdmYuOfOMbOFwWesNLMfBPs7BL9nvZmtM7NXzEz/P5f90j8QSSWdgSKgGzCB2L/v+4LtcmALcNd+Xj8EWAx0AP4buNfM7BCOfRh4CygGfgpccrC/iJllAX8FngU6Av8IPGRmRweH3EvsVFg+0B94Idh/I1ADlACdgH8BdP5X9ktBIKlkN/ATd9/m7lvcfa27P+Hum919E/BzYOR+Xr/C3f/g7ruA+4EuxP6YNvlYMysHTgT+zd23u/urwLRD+F2GAm2BXwTv8wLwNDAueH4H0NfM2rn7end/O25/F6Cbu+9w91dcjUA5AAWBpJI6d9/asGFmuWb2ezNbYWYbgZeBQjPL3MfraxseuPvm4GHbgzz2CGBd3D6A6oP8PQjep9rdd8ftWwF0DR5/EzgHWGFmL5nZsGD/r4ClwLNmtszMbj6Ez5Y0oyCQVLL3N98bgaOBIe7eDhgR7N/X6Z7m8AlQZGa5cfvKDuF9PgbK9jq/Xw6sBHD32e5+HrHTRn8BHgv2b3L3G929JzAGuMHMvnIIny9pREEgqSyfWF+g3syKgJ+E/YHuvgKoBH5qZtnBN/WvHeh1ZpYT/0Osx7AZ+KGZZQXTTL8GTA3ed7yZFbj7DmAjsdNimNm5ZnZU0K/YQGxq7e5GP1QkoCCQVHY70AZYA7wBTG+hzx0PDAPWAj8DHiV2vcO+dCUWWPE/ZcT+8J9NrP67gW+7+3vBay4BlgenvL4XfCZAL+A54FNgFnC3u89stt9MUpIuKBMJmZk9Crzn7qGPSEQOhUYEIs3MzE40syPNLMPMRgPnETuPL5KQdGWxSPPrDDxJ7DqCGuBqd38n2pJE9i3UU0PBt6HfApnAZHf/xV7PdwOmELv4ZR1wsbvXhFaQiIh8SWhBEMzVXgKcSexb0WxgnLsvjDvmz8DT7n6/mZ0OfMfdD/oqTBEROXRhnhoaDCx192UAZjaV2LnShXHH9AVuCB7PpAnnUTt06ODdu3dv3kpFRFLcnDlz1rh7SWPPhRkEXfniFZU1xO7PEm8e8A1ip4/OB/LNrNjd18YfZGYTiN07hvLyciorK0MrWkQkFZnZin09F/WsoR8AI83sHWL3gFnJ5/eW38PdJ7l7hbtXlJQ0GmgiInKIwhwRrOSLl9aXBvv2cPePiY0IMLO2wDfdvT7EmkREZC9hjghmA72Ce7NnA2PZ6y6Mwb3TG2q4hdgMIhERaUGhBUGwOtR1wAxgEfCYu1eZ2a1xC2ycCiw2syXEbvf787DqERGRxiXdLSYqKipczWIRkYNjZnPcvaKx56JuFouISMQUBCIiaS5tguCDuk/55fT3SLZTYSIiYUubIHhh0WruefEDHnhjn9dUiIikpbQJgstP7sHpfTrys6cX8W7NhqjLERFJGGkTBBkZxq//YQDFbbO59uG32bh1R9QliYgkhLQJAoD2edncddEgVtZv4eYn5qtfICJCmgUBwAndivjhqKN55t1a9QtEREjDIAC48pSe6heIiATSMgjULxAR+VxaBgGoXyAi0iBtgwDULxARgTQPAlC/QEQk7YNA/QIRSXdpHwSgfoGIpDcFQUD9AhFJV6EGgZmNNrPFZrbUzG5u5PlyM5tpZu+Y2XwzOyfMeg4kvl+wYKX6BSKSHkILAjPLBCYCZwN9gXFm1nevw35MbAnLQcTWNL47rHqaQv0CEUlHYY4IBgNL3X2Zu28HpgLn7XWMA+2CxwXAxyHW0yQN/YKa9Vu45Yl31S8QkZQXZhB0BarjtmuCffF+ClxsZjXAM8A/NvZGZjbBzCrNrLKuri6MWr+goV/wt3c/4UH1C0QkxUXdLB4H/NHdS4FzgAfM7Es1ufskd69w94qSkpIWKayhX/Af6heISIoLMwhWAmVx26XBvniXA48BuPssIAfoEGJNTaZ+gYikizCDYDbQy8x6mFk2sWbwtL2O+Qj4CoCZHUMsCMI/99NE6heISDoILQjcfSdwHTADWERsdlCVmd1qZmOCw24ErjSzecAjwGWeYH9t1S8QkVTXKsw3d/dniDWB4/f9W9zjhcDwMGtoDlee0pM3P1zHfzy9iEHl7enftSDqkkREmk3UzeKkoH6BiKQyBUETqV8gIqlKQXAQTuhWxE3qF4hIilEQHKQJp/TktKNLdH2BiKQMBcFBysgwfn3BQPULRCRlKAgOQVFeNneOU79ARFKDguAQVXRXv0BEUoOC4DCoXyAiqUBBcBjULxCRVKAgOEzqF4hIslMQNAP1C0QkmSkImon6BSKSrBQEzUT9AhFJVgqCZqR+gYgkIwVBM1O/QESSjYIgBOoXiEgyURCEYO9+wSb1C0QkgYUaBGY22swWm9lSM7u5ked/Y2Zzg58lZlYfZj0tKb5fcPOT6heISOIKLQjMLBOYCJwN9AXGmVnf+GPc/Z/dfaC7DwTuBJ4Mq54o7OkXzP+EB9/8KOpyREQaFeaIYDCw1N2Xuft2YCpw3n6OH0dsAfuUsqdf8NeF6heISEIKMwi6AtVx2zXBvi8xs25AD+CFfTw/wcwqzayyrq6u2QsNk/oFIpLoEqVZPBZ43N13Nfaku09y9wp3rygpKWnh0g6f+gUiksjCDIKVQFncdmmwrzFjScHTQvHULxCRRBVmEMwGeplZDzPLJvbHftreB5lZH6A9MCvEWhKC+gUikohCCwJ33wlcB8wAFgGPuXuVmd1qZmPiDh0LTPU0OF/S0C8oylO/QEQShyXb39+KigqvrKyMuozDMnv5OsZOeoPR/Ttz17hBmFnUJYlIijOzOe5e0dhzidIsTisndi/iB2epXyAiiUFBEJGrRvTkVPULRCQBKAgikpFh3KZ+gYgkAAVBhIrysrnzIl1fICLRUhBETP0CEYmagiABqF8gIlFSECQA9QtEJEoKggShfoGIREVBkEDULxCRKCgIEkx8v2B+Tcos2CYiCUxBkGAa+gUl+a25dMpbLPx4Y9QliUiKUxAkoKK8bB6+cgg5WZmMn/yGwkBEQqUgSFDdivOYOmEobbIyuUhhICIhUhAksG7FeTwyYSi5QRhUfaxrDESk+SkIElxsZDCM3KxMxk9+U2EgIs1OQZAEyotzvxAGuvpYRJpTqEFgZqPNbLGZLTWzm/dxzAVmttDMqszs4TDrSWYNYZCX3YqL71UYiEjzCS0IzCwTmAicDfQFxplZ372O6QXcAgx3937A9WHVkwrKi3N55Mqh5GW30shARJpNmCOCwcBSd1/m7tuBqcB5ex1zJTDR3dcDuPvqEOtJCQ1h0La1wkBEmkeYQdAVqI7brgn2xesN9Daz18zsDTMb3dgbmdkEM6s0s8q6urqQyk0esdNECgMRaR5RN4tbAb2AU4FxwB/MrHDvg9x9krtXuHtFSUlJC5eYmMqKFAYi0jzCDIKVQFncdmmwL14NMM3dd7j7h8ASYsEgTaAwEJHmEGYQzAZ6mVkPM8sGxgLT9jrmL8RGA5hZB2KnipaFWFPKURiIyOEKLQjcfSdwHTADWAQ85u5VZnarmY0JDpsBrDWzhcBM4CZ3XxtWTalKYSAih8OSbQGUiooKr6ysjLqMhFS9bjNjJ73Bpq07eOiKoRxbWhB1SSKSIMxsjrtXNPZc1M1iaUYNI4N2bbIYP/kN3q3RyEBEDkxBkGLKimLXGTSEgRa3EZEDURCkoPiRwcWT31QYiMh+KQhSVGl7hYGINI2CIIU1hEFBbhbjFQYisg8KghRX2j7WMygMwmBetcJARL5IQZAGYiODYRTmZnHxvQoDEfkiBUGa6FrY5gthMFdhICIBBUEaaQiD9rnZXKIwEJGAgiDNdC1swyMThsbCYLLCQEQUBGkpNjIYSvs8hYGIKAjS1hF7hcE7H62PuiQRiYiCII01hEFR22y+fe9bCgORNKUgSHNHFLbhkSs/D4O3FQYiaUdBIF8YGVyqMBBJOwoCAaBLwRdPEykMRNKHgkD2aAiDDgoDkbQSahCY2WgzW2xmS83s5kaev8zM6sxsbvBzRZj1yIF1KYhdZ9AQBnNWKAxEUl2TgsDM8swsI3jc28zGmFnWAV6TCUwEzgb6AuPMrG8jhz7q7gODn8kHWb+EIDYyGEaHttlcOkVhIJLqmjoieBnIMbOuwLPAJcAfD/CawcBSd1/m7tuBqcB5h1qotKzOBTl7hcG6qEsSkZA0NQjM3TcD3wDudvd/APod4DVdgeq47Zpg396+aWbzzexxMytr9MPNJphZpZlV1tXVNbFkOVwNYVCS35pLp8xWGIikqCYHgZkNA8YDfwv2ZTbD5/8V6O7uxwF/B+5v7CB3n+TuFe5eUVJS0gwfK03VuSCHR64cSkl+66BnoDAQSTVNDYLrgVuAp9y9ysx6AjMP8JqVQPw3/NJg3x7uvtbdtwWbk4ETmliPtKCGMOjYLkdhIJKCmhQE7v6Su49x918GTeM17v5PB3jZbKCXmfUws2xgLDAt/gAz6xK3OQZYdBC1SwuKnSb6PAwqlysMRFJFU2cNPWxm7cwsD1gALDSzm/b3GnffCVwHzCD2B/6xYDRxq5mNCQ77JzOrMrN5wD8Blx3qLyLh69Tu8zC4dIrCQCRVmLsf+CCzue4+0MzGA8cDNwNzgnP7LaqiosIrKytb+mMlzqqNWxk36Q1WbdzK/d8dTEX3oqhLEpEDMLM57l7R2HNN7RFkBdcNfB2Y5u47gAMniKSkTu1yeGTCUDoFI4PZGhmIJLWmBsHvgeVAHvCymXUDNoZVlCS++DC4TGEgktSa2iy+w927uvs5HrMCOC3k2iTBNfQMOgUN5OcWroq6JBE5BE1tFheY2W0NF3WZ2a+JjQ4kzXVsl8PUq4bSq1NbrnygkimvfkhT+k4ikjiaempoCrAJuCD42QjcF1ZRklw65ufw6IRhnNW3E7c+vZCfTKti567dUZclIk3U1CA40t1/Etw3aJm7/zvQM8zCJLm0yc7knvEnMGFET/40awVX/KmSTVt3RF2WiDRBU4Ngi5md3LBhZsOBLeGUJMkqI8P4l3OO4T/PP5ZX3l/DP/xuFh/X65+JSKJrahB8D5hoZsvNbDlwF3BVaFVJUrtoSDn3XXYiK9dv4esTX+Pdmg1RlyQi+9HUWUPz3H0AcBxwnLsPAk4PtTJJaiN6l/D41SeRlZnBBb+fxbNVtVGXJCL7cFArlLn7RndvuH7ghhDqkRRydOd8nrr2JHp3zueqB+cw+ZVlmlEkkoAOZ6lKa7YqJGV1zM9h6pVDGd2vMz/72yJ+/JcFmlEkkmAOJwj01U6apE12JhMvOp7vjTySh978iO/erxlFIolkv0FgZpvMbGMjP5uAI1qoRkkBGRnGzWf34RffOJbXl67hW/fMomb95qjLEhEOEATunu/u7Rr5yXf3Vi1VpKSOsYPL+eN3BvPxhi18feLrzKuuj7okkbR3OKeGRA7Jyb068OTVJ5GTlcGFk2YxfcEnUZckktYUBBKJXp3yeeqa4fTp3I6rH3qbSS9/oBlFIhEJNQjMbLSZLTazpWZ2836O+6aZuZk1umiCpKaS/NZMnTCUc/p34T+feY9/eWoBOzSjSKTFhXae38wygYnAmUANMNvMprn7wr2Oywe+D7wZVi2SuHKyMrlz3CC6d8hl4swPqFm/mYnjj6ddTlbUpYmkjTBHBIOBpcFN6rYDU4HzGjnuP4BfAltDrEUSWEaGcdOoPvz3t45j1gdr+ebdr1O9TjOKRFpKmEHQFaiO264J9u1hZscDZe7+t/29kZlNaFgLoa6urvkrlYRwQUUZf/ruYFZt3Mr5d7/GOx+tj7okkbQQWbPYzDKA24AbD3Ssu09y9wp3rygpKQm/OInMSUd14MlrhtMmO5Oxk97gmXc1o0gkbGEGwUqgLG67NNjXIB/oD7wY3NF0KDBNDWM5qmNb/nLNcPod0Y5rHnqbe17UjCKRMIUZBLOBXmbWw8yygbHAtIYn3X2Du3dw9+7u3h14Axjj7pUh1iRJorhtax6+cijnHteFX05/j5ufeFczikRCEtqsIXffaWbXATOATGCKu1eZ2a1ApbtP2/87SLrLycrkjrGD6NEhjztfWEpN/WbuHn8CBW00o0ikOVmyDbkrKiq8slKDhnTz+JwabnlyPt2K87jvshMpK8qNuiSRpGJmc9y90VPvurJYksK3TijlT98dQt2mbXx94mvMWaEZRSLNRUEgSWPYkcU8ec1JtM1pxbg/vMHT8z+OuiSRlKAgkKRyZElbnrpmOMd1LeC6h99h4sylmlEkcpgUBJJ0ivKyefCKIZw38Ah+NWMxP3x8Ptt3akaRyKHSmgKSlHKyMrn9woF0K87jjuffp2b9Fn538QkU5GpGkcjB0ohAkpaZccOZvbntggFUrljH+fe8xoq1n0VdlkjSURBI0vvG8aU8ePkQ1n22nfPvfp3K5euiLkkkqSgIJCUM6VnMU9cMp11OKy6a/Cb/O3flgV8kIoCCQFJIjw55PHXNcAaWFvL9qXO58/n3NaNIpAkUBJJS2udl88AVgzl/UFd+/fcl/ODPmlEkciCaNSQpp3WrTG67YADdinO5/bn3qVm/md9fcgKFudlRlyaSkDQikJRkZlx/Rm9uv3Ag73xUz/l3v87yNZpRJNIYBYGktK8P6sqDVwyhfvN2zr/7NWZrRpHIlygIJOUN7lHEU9cMp31uNuP/8CZ/eUczikTiKQgkLXTvkMeT15zEoPJCrn90Lrc8OZ/Vm7ZGXZZIQlAQSNoozM3mgcuHcPnJPfhzZQ2n/upFbn9uCZ9t2xl1aSKRUhBIWslulcH/O7cvz90wklOPLuH2597n1P95kUfe+oidWgpT0lSoQWBmo81ssZktNbObG3n+e2b2rpnNNbNXzaxvmPWINOjeIY+7x5/AE1cPo6x9G2558l3O/u0rvPDeKl2EJmkntKUqzSwTWAKcCdQQW8x+nLsvjDumnbtvDB6PAa5x99H7e18tVSnNzd2ZvqCWX05/j+VrNzOsZzH/+tVj6N+1IOrSRJpNVEtVDgaWuvsyd98OTAXOiz+gIQQCeYC+ikmLMzPOPrYLz/7zSH76tb68V7uRc+98leunvkPN+s1RlycSujCvLO4KVMdt1wBD9j7IzK4FbgCygdMbeyMzmwBMACgvL2/2QkUg1j+4bHgPvnFCKfe8+AFTXv2QZxbU8p2TunPNaUdR0EZrHUhqirxZ7O4T3f1I4EfAj/dxzCR3r3D3ipKSkpYtUNJOu5wsfjS6DzN/cCrnHteFSa8sY+SvZjLl1Q913yJJSWEGwUqgLG67NNi3L1OBr4dYj8hBOaKwDbddMJC/Xncy/Y8o4NanF3Lmb17ib/M/UUNZUkqYQTAb6GVmPcwsGxgLTIs/wMx6xW1+FXg/xHpEDkn/rgU8cPlg/vidE8lplcm1D7/NN+7RAjiSOkLrEbj7TjO7DpgBZAJT3L3KzG4FKt19GnCdmZ0B7ADWA5eGVY/I4TAzTj26I6f0KuGJOTX8+u+L+dbvZjGqXyd+NLoPPUvaRl2iyCELbfpoWDR9VBLB5u07ufeVD/ndSx+wbeduLhpSzve/0ovitq2jLk2kUfubPqogEDkMdZu2cftzS5g6u5o2WZlcfeqRfHd4D9pkZ0ZdmsgXRHUdgUjKK8lvzc/PP5YZ149gaM9ifjVjMaf/+kX+XFnNrt3J9SVL0peCQKQZHNWxLZMvreDRCUPpmN+amx6fz7l3vsor79dFXZrIASkIRJrRkJ7FPHXNcO4YN4hNW3dwyb1v8e0pb7Hok40HfrFIRBQEIs0sI8MYM+AInr9xJD/+6jHMq67nnDte4aY/z6N2g9ZAkMSjZrFIyDZs3sFdM9/n/tdXkJEBV5zck6tG9iQ/R7eskJajWUMiCaB63WZ+NWMx0+Z9THFeNtef0Yuxg8vJytTAXMKnWUMiCaCsKJc7xg3if68dzlEd2/L//reKUb95mRlVtbplhURKQSDSwgaUFTJ1wlAmf7sCM7jqgTlc+Ps3eOej9VGXJmlKQSASATPjjL6dmHH9CH5+fn+WrfmM8+9+nWsffpuP1moNBGlZ6hGIJIBPt+1k0svL+MPLy9i5ezffHtad6047ivZ52VGXJilCzWKRJLFq41Z+8/clPFZZTdvWrbj2tKO49KTu5GTplhVyeBQEIklmce0m/uv/FvHi4jpK8lszul9nRvXrzJCeRZplJIdEQSCSpF5buoYH31jBi4vr2LJjFwVtsvjKMR0Z1a8zI3qV6OZ20mT7C4Iw1ywWkcM0/KgODD+qA1u27+KV9+uYXlXL84tW8+TbK2mTlcnI3iWM6t+J0/t00prKcsgUBCJJoE12Jmf168xZ/TqzY9du3vpwHdMX1PLswlqmV9XSKsMYdmQxo/p15qx+neiYnxN1yZJEQj01ZGajgd8SW6Fssrv/Yq/nbwCuAHYCdcB33X3F/t5Tp4ZEPrd7tzOvpp7pVbXMWFDL8rWbMYPjy9szql8nRvXrTLfivKjLlAQQSY/AzDKBJcCZQA2xNYzHufvCuGNOA950981mdjVwqrtfuL/3VRCINM7dWbLqU2ZU1TJ9QS0Lgzue9umcz+j+sWZzn875mFnElUoUogqCYcBP3X1UsH0LgLv/1z6OHwTc5e7D9/e+CgKRpqlet5kZVbXMqKqlcsV63KG8KDcIhU4MKmtPRoZCIV1E1SzuClTHbdcAQ/Zz/OXA/zX2hJlNACYAlJeXN1d9IimtrCiXK07pyRWn9KRu0zaeW7SK6Qtque+1D5n08jJK8ltzVt/Y6aOhPYvJbqVpqekqIZrFZnYxUAGMbOx5d58ETILYiKAFSxNJCSX5rRk3uJxxg8vZuHUHM99bzYyqWp56ZyUPvfkR7XJa8ZVjOjGqXydG9C4hNzsh/jRICwnzf+2VQFncdmmw7wvM7AzgX4GR7r4txHpEBGiXk8V5A7ty3sCubN2xi1ffX8P0qlqeW7SKp95ZSU5WBiN6lTCqX2fOOKYTBbmalprqwgyC2UAvM+tBLADGAhfFHxD0BX4PjHb31SHWIiKNyMnK5Iy+nTijbyd2BtNSY32FVTy7cBWtMoyhPYsZ1a8TZ/XrTKd2mpaaisKePnoOcDux6aNT3P3nZnYrUOnu08zsOeBY4JPgJR+5+5j9vaeaxSLh273bmb9yQywUFtSybM1nAAwqL2RUv86M7teZ7h00LTWZ6BYTInLI3J2lq4NpqVW1LFgZm5Z6dKd8RgUzkPp2aadpqQlOQSAizaZm/WaerVrF9KpaKpevY7dDafs2sZFC/84cX96eTE1LTTgKAhEJxdpPP5+W+trStWzftZuCNlmc3KsDI3uXMLJ3ifoKCUJBICKh27R1By8uruOlJXW8vKSO1ZtikwD7dM5nZO8SRvQuoaJ7e1q30h1To6AgEJEW5e68V7tpTyjMXr6OHbucNlmZnHRkMSOC0YIazi1HQSAikfps205mfbCWl9+PjRhWBOsylxfl7jmFNOzIYvJa60K2sCgIRCShLF/zWSwUFtcxa9laNm/fRVamUdGtaM9o4ZguukFec1IQiEjC2rZzF3OWr+elJbHRwnu1m4DYbTFG9Cph5NElnHJUB9rnZUdcaXJTEIhI0li1cSsvB6Hwyvtr2LBlB2ZwXGlhcBqpAwNKC2mltZsPioJARJLSrt3O/Jr6PaOFedX17HZol9NqzxTVEb1L6FLQJupSE56CQERSQv3m7by6dM2eEcOqjbEpqr07tQ1GCx2p6N6enCxNUd2bgkBEUo67s3jVpj2hMPvD9WzftZucrAyG9fx8imqPDnlqOhPdwjQiIqExM/p0bkefzu2YMOJINm/fyRvL1vJScFHbzMV1QOz2Fw1TVE86qgNtNUX1SzQiEJGUtGLtZ3tGC69/EJui2irDOKFbe0YeXcKIXiX07dIubZbr1KkhEUlr23fupnLFuuBK5zUs+iR2B9X2uVkMKCtkQGkhA8sKOa60gOK2rSOuNhwKAhGROKs3buWlJXW89eE65tdsYMnqTTT8KSxt34YBZYUMLC1kQFkh/bu2S4mlOxUEIiL78em2nSxYuYF51fXMq6lnXvUGVtZvASDDoHenfAYEwXBcaQFHd84nK8muY1AQiIgcpLpN25hfU8+86nrm1mxgfk099Zt3ANC6VQb9uxYE4VDAwLJCyotyE3p2UmRBYGajgd8SW6pysrv/Yq/nRxBbyvI4YKy7P36g91QQiEgU3J2P1m1mbnVsxDC/pp53V25g287dABTmZnFcaSEDSwuCkUMhJfmJ02+IZPqomWUCE4EzgRpgtplNc/eFcYd9BFwG/CCsOkREmoOZ0a04j27FeZw3sCsAO3btZsmqTXuCYW51PXfNrGN38P26a2EbBpQV7Dmt1L9rQUJOXw2zosHAUndfBmBmU4HzgD1B4O7Lg+d2h1iHiEgosjIz6HdEAf2OKOCiIeUAbN6+kwUrN37eb6ip55l3awEwg14d2+4JhoFlhQnRbwgzCLoC1XHbNcCQQ3kjM5sATAAoLy8//MpEREKSm92KwT2KGNyjaM++tZ9uY37NBuZW1zO/pp7n3yagkaEAAATbSURBVFvNn+fUAJDdKoN+R7T7whTW7sV5LXp9Q+KNURrh7pOASRDrEURcjojIQSlu25rT+nTktD4dgVi/oWb9lj3BMK96A4/OruaPry8HYjfVa7i+IfbfAjqGuPZzmEGwEiiL2y4N9omIpDUzo6wol7KiXL424AgAdu7azfurPw16DbGprPe89AG7goZDl4Icbj67z57+RHMKMwhmA73MrAexABgLXBTi54mIJK1WmRkc06Udx3Rpx4UnxvZt2b6LhZ9s2BMMYc1CCi0I3H2nmV0HzCA2fXSKu1eZ2a1ApbtPM7MTgaeA9sDXzOzf3b1fWDWJiCSTNtmZnNCtiBO6FR344MMQao/A3Z8Bntlr37/FPZ5N7JSRiIhEJLmukRYRkWanIBARSXMKAhGRNKcgEBFJcwoCEZE0pyAQEUlzCgIRkTSXdAvTmFkdsCLqOlpIB2BN1EU0UTLVCslVbzLVKomrm7uXNPZE0gVBOjGzyn0tJJFokqlWSK56k6lWSU46NSQikuYUBCIiaU5BkNgmRV3AQUimWiG56k2mWiUJqUcgIpLmNCIQEUlzCgIRkTSnIEhAZjbFzFab2YKoazkQMyszs5lmttDMqszs+1HXtC9mlmNmb5nZvKDWf4+6pqYws0wze8fMno66FklNCoLE9EdgdNRFNNFO4EZ37wsMBa41s74R17Qv24DT3X0AMBAYbWZDI66pKb4PLIq6CEldCoIE5O4vA+uirqMp3P0Td387eLyJ2B+s5l9duxl4zKfBZlbwk9CzJcysFPgqMDnqWiR1KQik2ZhZd2AQ8Ga0lexbcJplLrAa+Lu7J2ytgduBHwK7oy5EUpeCQJqFmbUFngCud/eNUdezL+6+y90HElsre7CZ9Y+6pn0xs3OB1e4+J+paJLUpCOSwmVkWsRB4yN2fjLqepnD3emAmid2LGQ6MMbPlwFTgdDN7MNqSJBUpCOSwmJkB9wKL3P22qOvZHzMrMbPC4HEb4EzgvWir2jd3v8XdS929OzAWeMHdL464LElBCoIEZGaPALOAo82sxswuj7qm/RgOXELs2+rc4OecqIvahy7ATDObD8wm1iPQlExJe7rFhIhImtOIQEQkzSkIRETSnIJARCTNKQhERNKcgkBEJM0pCET2Yma74qbCzjWzm5vxvbsnw11lJb20iroAkQS0JbgNhUha0IhApInMbLmZ/beZvRusa3BUsL+7mb1gZvPN7HkzKw/2dzKzp4L1D+aZ2UnBW2Wa2R+CNRGeDa5yFomMgkDky9rsdWrowrjnNrj7scBdxO4MCnAncL+7Hwc8BNwR7L8DeClY/+B4oCrY3wuY6O79gHrgmyH/PiL7pSuLRfZiZp+6e9tG9i8ntrDNsuBGe7XuXmxma4Au7r4j2P+Ju3cwszqg1N23xb1Hd2K3tugVbP8IyHL3n4X/m4k0TiMCkYPj+3h8MLbFPd6FenUSMQWByMG5MO6/s4LHrxO7OyjAeOCV4PHzwNWwZ0GcgpYqUuRg6JuIyJe1CVYxazDd3RumkLYP7l66DRgX7PtH4D4zuwmoA74T7P8+MCm4e+wuYqHwSejVixwk9QhEmijoEVS4+5qoaxFpTjo1JCKS5jQiEBFJcxoRiIikOQWBiEiaUxCIiKQ5BYGISJpTEIiIpLn/DzO4dSJ5yRnqAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_texts = test.CONTEXT.values\n",
        "test_labels = test.label.values\n",
        "\n",
        "input_ids = []\n",
        "attention_masks = []\n",
        "\n",
        "for text in test_texts:\n",
        "    encoded_dict = tokenizer.encode_plus(\n",
        "                        text,                     \n",
        "                        add_special_tokens = True, \n",
        "                        max_length = max_len,          \n",
        "                        pad_to_max_length = True,\n",
        "                        return_attention_mask = True,  \n",
        "                        return_tensors = 'pt',   \n",
        "                   )\n",
        "    \n",
        "    input_ids.append(encoded_dict['input_ids'])\n",
        "    attention_masks.append(encoded_dict['attention_mask'])\n",
        "\n",
        "input_ids = torch.cat(input_ids, dim=0)\n",
        "attention_masks = torch.cat(attention_masks, dim=0)\n",
        "labels = torch.tensor(test_labels)\n",
        "\n",
        "batch_size = 32  \n",
        "\n",
        "prediction_data = TensorDataset(input_ids, attention_masks, labels)\n",
        "prediction_sampler = SequentialSampler(prediction_data)\n",
        "prediction_dataloader = DataLoader(prediction_data, sampler=prediction_sampler, batch_size=batch_size)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z7iIHrQ9ZWTq",
        "outputId": "3ae5f452-bac4-4578-a0c9-7e72fd9576d4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:2339: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print('Prediction started on test data')\n",
        "model.eval()\n",
        "predictions , true_labels = [], []\n",
        "\n",
        "for batch in prediction_dataloader:\n",
        "  batch = tuple(t.to(device) for t in batch)\n",
        "  b_input_ids, b_input_mask, b_labels = batch\n",
        "\n",
        "  with torch.no_grad():\n",
        "      outputs = model(b_input_ids, token_type_ids=None, \n",
        "                      attention_mask=b_input_mask)\n",
        "\n",
        "  logits = outputs[0]\n",
        "  logits = logits.detach().cpu().numpy()\n",
        "  label_ids = b_labels.to('cpu').numpy()\n",
        "  \n",
        "  predictions.append(logits)\n",
        "  true_labels.append(label_ids)\n",
        "\n",
        "print('Prediction completed')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A4uHO1eMb8_t",
        "outputId": "51c2e239-0053-4fe1-8598-9414bcc8fa9f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Prediction started on test data\n",
            "Prediction completed\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "prediction_set = []\n",
        "\n",
        "for i in range(len(true_labels)):\n",
        "  pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n",
        "  prediction_set.append(pred_labels_i)\n",
        "\n",
        "prediction_scores = [item for sublist in prediction_set for item in sublist]"
      ],
      "metadata": {
        "id": "nkMEqne4cAGj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "f_score = f1_score(test_labels, prediction_scores, average='macro')\n",
        "precision = precision_score(test_labels, prediction_scores, average='macro')\n",
        "recall = recall_score(test_labels, prediction_scores, average='macro')\n",
        "accr = accuracy_score(test_labels, prediction_scores)"
      ],
      "metadata": {
        "id": "nOTzCpiPcCkl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"F-Score: \", f_score)\n",
        "print(\"Recall: \", recall)\n",
        "print(\"Precision: \", precision)\n",
        "print(\"Accuracy: \", accr)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4DzEBWXwcE1s",
        "outputId": "bc209669-7db4-4d24-eeb7-6b77dede0366"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F-Score:  0.9799727481797514\n",
            "Recall:  0.9805662733081533\n",
            "Precision:  0.9794795142965443\n",
            "Accuracy:  0.9823355068168346\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "report = pd.DataFrame(classification_report(test_labels, prediction_scores, output_dict=True))"
      ],
      "metadata": {
        "id": "L6elJTyTcFLL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "report = report.rename(columns={\n",
        "                                '0':'Kart İşlemleri',\n",
        "                                '1':'Kredi İşlemleri',\n",
        "                                '2':'Diğer',\n",
        "                                '3':'Hesap İşlemleri',\n",
        "                                '4': 'Kargo',\n",
        "                                '5': 'Limit',\n",
        "                                '6': 'Müşteri Temsilcisi',\n",
        "                                '7': 'KKB Skor'})\n",
        "\n",
        "report"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "J20vMamEcGY_",
        "outputId": "59e319ef-4efc-4a45-ca5c-33a66f7d86fc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "           Kart İşlemleri  Kredi İşlemleri  Hesap İşlemleri        Kargo  \\\n",
              "precision        0.977292         0.971119         0.985294     0.953096   \n",
              "recall           0.978114         0.960714         0.985294     0.986348   \n",
              "f1-score         0.977703         0.965889         0.985294     0.969437   \n",
              "support       1188.000000       280.000000        68.000000  2637.000000   \n",
              "\n",
              "                 Limit  Müşteri Temsilcisi     KKB Skor  accuracy  \\\n",
              "precision     0.988616            0.989115     0.991824  0.982336   \n",
              "recall        0.978590            0.982224     0.992679  0.982336   \n",
              "f1-score      0.983577            0.985657     0.992251  0.982336   \n",
              "support    6212.000000         4163.000000  2322.000000  0.982336   \n",
              "\n",
              "              macro avg  weighted avg  \n",
              "precision      0.979480      0.982527  \n",
              "recall         0.980566      0.982336  \n",
              "f1-score       0.979973      0.982374  \n",
              "support    16870.000000  16870.000000  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b5d49cf0-22e3-4230-9527-f8d866a69cb5\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Kart İşlemleri</th>\n",
              "      <th>Kredi İşlemleri</th>\n",
              "      <th>Hesap İşlemleri</th>\n",
              "      <th>Kargo</th>\n",
              "      <th>Limit</th>\n",
              "      <th>Müşteri Temsilcisi</th>\n",
              "      <th>KKB Skor</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>macro avg</th>\n",
              "      <th>weighted avg</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>precision</th>\n",
              "      <td>0.977292</td>\n",
              "      <td>0.971119</td>\n",
              "      <td>0.985294</td>\n",
              "      <td>0.953096</td>\n",
              "      <td>0.988616</td>\n",
              "      <td>0.989115</td>\n",
              "      <td>0.991824</td>\n",
              "      <td>0.982336</td>\n",
              "      <td>0.979480</td>\n",
              "      <td>0.982527</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>recall</th>\n",
              "      <td>0.978114</td>\n",
              "      <td>0.960714</td>\n",
              "      <td>0.985294</td>\n",
              "      <td>0.986348</td>\n",
              "      <td>0.978590</td>\n",
              "      <td>0.982224</td>\n",
              "      <td>0.992679</td>\n",
              "      <td>0.982336</td>\n",
              "      <td>0.980566</td>\n",
              "      <td>0.982336</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>f1-score</th>\n",
              "      <td>0.977703</td>\n",
              "      <td>0.965889</td>\n",
              "      <td>0.985294</td>\n",
              "      <td>0.969437</td>\n",
              "      <td>0.983577</td>\n",
              "      <td>0.985657</td>\n",
              "      <td>0.992251</td>\n",
              "      <td>0.982336</td>\n",
              "      <td>0.979973</td>\n",
              "      <td>0.982374</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>support</th>\n",
              "      <td>1188.000000</td>\n",
              "      <td>280.000000</td>\n",
              "      <td>68.000000</td>\n",
              "      <td>2637.000000</td>\n",
              "      <td>6212.000000</td>\n",
              "      <td>4163.000000</td>\n",
              "      <td>2322.000000</td>\n",
              "      <td>0.982336</td>\n",
              "      <td>16870.000000</td>\n",
              "      <td>16870.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b5d49cf0-22e3-4230-9527-f8d866a69cb5')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b5d49cf0-22e3-4230-9527-f8d866a69cb5 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b5d49cf0-22e3-4230-9527-f8d866a69cb5');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd /content/drive/MyDrive/models/\n",
        "!ls"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "85WVwiQA2iBo",
        "outputId": "f4b50171-72e0-4607-8f27-68420a3f3271"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "drive  sample_data\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import BertTokenizer,BertTokenizerFast, TFBertForSequenceClassification, BertConfig, TFBertModel\n",
        "config = BertConfig.from_json_file(\"/content/drive/MyDrive/models/bert-bank-model/config.json\")\n",
        "model_path = \"/content/drive/MyDrive/models/bert-bank-model\"\n",
        "tokenizer_path = \"/content/drive/MyDrive/models/bert-bank-model\"\n",
        "#model = TFBertModel.from_pretrained(model_path, from_pt=True, config=config)\n",
        "model = TFBertForSequenceClassification.from_pretrained(model_path, from_pt=True) # modify labels as needed.\n",
        "model"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LrA44pRmceWt",
        "outputId": "29e6e2cd-0bd2-44a0-92a1-49c1058e684f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']\n",
            "- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<transformers.models.bert.modeling_tf_bert.TFBertForSequenceClassification at 0x7f8bde446fa0>"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = BertTokenizer.from_pretrained(\"/content/drive/MyDrive/models/bert-bank-tokenizer\")\n",
        "model = TFBertForSequenceClassification.from_pretrained(model_path, from_pt=True) # modify labels as needed."
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tz37rxMi5uN7",
        "outputId": "96ff2d00-96a4-46bb-ab38-179dd2cf9b33"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']\n",
            "- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import TextClassificationPipeline\n",
        "\n",
        "text = [\"Kredimi ödemek için Akbank mobile girmeye çalışıyorum. Şifrem blokeli kart şifresi istiyor kartı kullanmadığım için şifrem yok müşteri hizmetlerine bağlanamıyorum. Akbank tarafından bu saçma duruma bir çözüm bulunmasını rica ediyorum illa bankaya gidip kart kullanmak zorunda değiliz geçin bunları bö...\",\n",
        "        \"VakıfBank hesabımda mevcut bir ihtiyaç kredim varken ikinci bir ihtiyaç kredisi çektim (çünkü ihtiyacım var). Paranın aktarılacağı hesabı seçtim ve para hesabıma aktarıldı. Ne olduysa ondan sonra oldu. Hesabımda şu anda \\\"para çekilemez blokesi bulunmaktadır\\\" Uyarısı alıyorum ve inanın nedenini bilmi..\",\n",
        "        \"28 Ocak günü yani cumartesi Akbank a üye oldum. Migros davet kodu ile 30 Ocak tarihinde de hesabımı onayladım diğer banka hesabımdan para atarak bunlar iyi güzel hoş Akbank a üye olurken Migros davet kodu girince 300 lira çek verilmesi gerekiyordu. Kodum hala SMS ile tarafıma iletilmedi. Akbank a \",\n",
        "        \"ING Bank tan kredi başvurusu yaptım. Kredim onaylanmadı tamam dedim tabi bu arada bana bilgim dışında hesap açılmış bana söylendiğinde ben de hesabı neden açtınız dedim oda hesap açmadan bakamadığını söyledi bende kapatın hesabı dedim ve kapattı. Ben öyle sanıyorum bugün bana ING den mesaj\",\n",
        "        \"Hiç bilmediğim bir kargo geldi ben istemedim çalışıyordum kargocu kapıya gelmiş ararken yoğunluktan cevap vermedim eşim kargoyu almış 2 tane Tozlu pis parfüm böyle kurumsal bir firma MNG Kargo şubeye gittim gönderici numarasını verdi ulaşılmıyor fatura yok\",\n",
        "       \"Sonu 5176 ile biten Maximum kredi kartım var. İş Bankası anlaşmalı olduğu kamu kurumu çalışanıyım. Kurum anlaşma yaptıktan sonra kartlarımızı dağıtırken bana 4.000 (dörtbin) tl limitli Maximum kredi kartı verdiler. Limitinin düşük olduğunu söyledim banka yetkilisine. Bana başka bankadaki limitimi dü...\",\n",
        "       '30.01.2023 tarihinde araç satış işlemleri için alıcı tarafın kredi işlemleri için Van İpekyolu Ziraat Bankası şubesine gittiğimizde Müşteri temsilcisinin alıcı tarafa öğleye kadar noter işlemlerinin gerçekleşecek şubeye gelinmesi halinde kredi işlemlerinin yapılacağını beyan etmesi üzerine noter işl...',\n",
        "       'Bireysel bankacılık ve sicil affı kanunu dahilinde 07.10.2022\\'de yürürlüğe giren kanun ile madde tarihli ve 5411 sayılı kanunun 1. Maddesi uyarınca karşılıksız çek ve protestolu senetler ile kredi ve kredi kartları borçlarına ilişkin kayıtların dikkate alınmaması hakkında kanuna aşağıdaki geçici mad...',\n",
        "]\n",
        "\n",
        "pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer)"
      ],
      "metadata": {
        "id": "Yd7MNNHh3Y80"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "[print(f\"{text[index]} - {i['label']}\") for index, i in enumerate(pipe(text))]\n",
        "'''\n",
        "Kart İşlemleri        4\n",
        "Kredi İşlemleri       5\n",
        "Diğer                 0\n",
        "Hesap İşlemleri       1\n",
        "Kargo                 3\n",
        "Limit                 6\n",
        "Müşteri Temsilcisi    7\n",
        "KKB Skor              2\n",
        "'''"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "id": "pQZPG6VX5BBS",
        "outputId": "8a3bf399-e6b8-47bd-e398-e0d4bb2c3de5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Kredimi ödemek için Akbank mobile girmeye çalışıyorum. Şifrem blokeli kart şifresi istiyor kartı kullanmadığım için şifrem yok müşteri hizmetlerine bağlanamıyorum. Akbank tarafından bu saçma duruma bir çözüm bulunmasını rica ediyorum illa bankaya gidip kart kullanmak zorunda değiliz geçin bunları bö... - LABEL_4\n",
            "VakıfBank hesabımda mevcut bir ihtiyaç kredim varken ikinci bir ihtiyaç kredisi çektim (çünkü ihtiyacım var). Paranın aktarılacağı hesabı seçtim ve para hesabıma aktarıldı. Ne olduysa ondan sonra oldu. Hesabımda şu anda \"para çekilemez blokesi bulunmaktadır\" Uyarısı alıyorum ve inanın nedenini bilmi.. - LABEL_5\n",
            "28 Ocak günü yani cumartesi Akbank a üye oldum. Migros davet kodu ile 30 Ocak tarihinde de hesabımı onayladım diğer banka hesabımdan para atarak bunlar iyi güzel hoş Akbank a üye olurken Migros davet kodu girince 300 lira çek verilmesi gerekiyordu. Kodum hala SMS ile tarafıma iletilmedi. Akbank a  - LABEL_0\n",
            "ING Bank tan kredi başvurusu yaptım. Kredim onaylanmadı tamam dedim tabi bu arada bana bilgim dışında hesap açılmış bana söylendiğinde ben de hesabı neden açtınız dedim oda hesap açmadan bakamadığını söyledi bende kapatın hesabı dedim ve kapattı. Ben öyle sanıyorum bugün bana ING den mesaj - LABEL_5\n",
            "Hiç bilmediğim bir kargo geldi ben istemedim çalışıyordum kargocu kapıya gelmiş ararken yoğunluktan cevap vermedim eşim kargoyu almış 2 tane Tozlu pis parfüm böyle kurumsal bir firma MNG Kargo şubeye gittim gönderici numarasını verdi ulaşılmıyor fatura yok - LABEL_3\n",
            "Sonu 5176 ile biten Maximum kredi kartım var. İş Bankası anlaşmalı olduğu kamu kurumu çalışanıyım. Kurum anlaşma yaptıktan sonra kartlarımızı dağıtırken bana 4.000 (dörtbin) tl limitli Maximum kredi kartı verdiler. Limitinin düşük olduğunu söyledim banka yetkilisine. Bana başka bankadaki limitimi dü... - LABEL_4\n",
            "30.01.2023 tarihinde araç satış işlemleri için alıcı tarafın kredi işlemleri için Van İpekyolu Ziraat Bankası şubesine gittiğimizde Müşteri temsilcisinin alıcı tarafa öğleye kadar noter işlemlerinin gerçekleşecek şubeye gelinmesi halinde kredi işlemlerinin yapılacağını beyan etmesi üzerine noter işl... - LABEL_7\n",
            "Bireysel bankacılık ve sicil affı kanunu dahilinde 07.10.2022'de yürürlüğe giren kanun ile madde tarihli ve 5411 sayılı kanunun 1. Maddesi uyarınca karşılıksız çek ve protestolu senetler ile kredi ve kredi kartları borçlarına ilişkin kayıtların dikkate alınmaması hakkında kanuna aşağıdaki geçici mad... - LABEL_5\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\nKart İşlemleri        4\\nKredi İşlemleri       5\\nDiğer                 0\\nHesap İşlemleri       1\\nKargo                 3\\nLimit                 6\\nMüşteri Temsilcisi    7\\nKKB Skor              2\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install huggingface_hub"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-mdj9pNu56QW",
        "outputId": "9681e209-01fc-4183-9a2d-6a313972949e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting huggingface_hub\n",
            "  Downloading huggingface_hub-0.12.0-py3-none-any.whl (190 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (2.25.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (4.4.0)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (4.64.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (3.9.0)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (21.3)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.9->huggingface_hub) (3.0.9)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (2022.12.7)\n",
            "Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (4.0.0)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (2.10)\n",
            "Installing collected packages: huggingface_hub\n",
            "Successfully installed huggingface_hub-0.12.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!huggingface-cli login"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JQehNXoJAtw5",
        "outputId": "5b4bf720-650e-4b60-dd38-b16479fe5582"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|\n",
            "    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|\n",
            "    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|\n",
            "    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|\n",
            "    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|\n",
            "    \n",
            "    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.\n",
            "    Setting a new token will erase the existing one.\n",
            "    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .\n",
            "Token: \n",
            "Add token as git credential? (Y/n) y\n",
            "Token is valid.\n",
            "\u001b[1m\u001b[31mCannot authenticate through git-credential as no helper is defined on your machine.\n",
            "You might have to re-authenticate when pushing to the Hugging Face Hub.\n",
            "Run the following command in your terminal in case you want to set the 'store' credential helper as default.\n",
            "\n",
            "git config --global credential.helper store\n",
            "\n",
            "Read https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage for more details.\u001b[0m\n",
            "Token has not been saved to git credential helper.\n",
            "Your token has been saved to /root/.cache/huggingface/token\n",
            "Login successful\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from huggingface_hub import create_repo\n",
        "create_repo(\"elifftosunn/Bert-Bank-Model\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "m_fiWH_qDfVr",
        "outputId": "f161250c-683c-4b20-acc3-e1bff8645e55"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RepoUrl('https://huggingface.co/elifftosunn/Bert-Bank-Model', endpoint='https://huggingface.co', repo_type='model', repo_id='elifftosunn/Bert-Bank-Model')"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import AutoTokenizer, TextClassificationPipeline, TFBertForSequenceClassification\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"elifftosunn/Bert-Bank-Model\")\n",
        "model = TFBertForSequenceClassification.from_pretrained(\"elifftosunn/Bert-Bank-Model\", from_pt=True)\n",
        "pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer)\n",
        "\n",
        "print(pipe('QNB Finansbank 1.39 oranlı 50.000 TL yeni müşterilere özel ihtiyaç kredisi 1.92 oranında veriyor amaç hesap açtırmak kampanyanın hiçbir gerçekçiliği yoktur. Resmen milletle dalga geçiyorsunuz. Ne demek oluyor bu. 1,39 dan kredi deyip içeriğine girince 2 katına çıkıyor. Böyle saçma bir banka'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281,
          "referenced_widgets": [
            "6d0636be08b44cc3b35754a42c1ba821",
            "b9af1f4852ce4318934df700c5d0e6b1",
            "002449f8919f46988898ce4f20a5dd5a",
            "053d12bcb58d4d18bfb58e4378e6c189",
            "6c51ad635caf4b97a8ca679837520f16",
            "0a9d1eef1feb4e7298940d26c3e6c49e",
            "59cbddd9910c4fba99e44a45b956f287",
            "6befb313e05940ffaa2763eb0160b52e",
            "7c230cd5592746a8b6e81349809cf21c",
            "eabd8e2bd8214b8dabe9b5eb9ca010ac",
            "3c7ea1ee672a4d71800799f71917605c",
            "0be6262d12174f9d97220824d141d139",
            "d75e33f6f1ea458488a4ef39f2fe241f",
            "35f9a783c46e43c9953defdad6fa4a65",
            "1ceb6c1312f94267a12a6f8712748e10",
            "904c6ed28b4d4a7e891b7839a7e1f48a",
            "02e78503d48a48f39883157fc162d410",
            "2bfcea3c1eb94d8e887d5f92ed81009d",
            "6c9ec21ed55f49429b72f52ef27f6568",
            "eab69d6179024f71a126e67a802b41d0",
            "a808cd7642974bc7b76454337f7e17af",
            "11a9bfb333f8419dbd4489964610d9b3",
            "2084218865534c4684dd33bf8192dc8a",
            "91a0d98708f244268d7d7bfc01953a84",
            "81631544977845cd94a4bfd98a367a47",
            "2525ec8a4ba3469aa5ddbbcbcf8cb844",
            "a0d9ce6eaee144f6ae11444bb68db783",
            "804b66bcf3a94a10be78453d9da56069",
            "5d22cf3f0d21418fb3b86767c6129f09",
            "153c872d546a40d8b8e08be7781ab739",
            "9f8fb890f5f94c57b9fe4744c3ea5355",
            "ea471130ded8449fbf1f60108d0eb113",
            "a5cf72a1d48a480eb8309d55f44c843a",
            "fcc33bcc0a2943e18967ce4753ed196a",
            "1508dffc4fc641c9b46a7e1d87c723b8",
            "456a81faf4db42949351a07167aeef55",
            "a49a78169c9c44e5a9756d1296438790",
            "fcd22a05105d43738fad771e000db933",
            "46d1bf0946174f53bcfc9c96c72943aa",
            "909065cda1724fe8b044062643f96caa",
            "dc32eae233d143bba39716dd50e3c64c",
            "016eeead878d4e7cbc6a23933c07ce02",
            "d77b4d29743a4148bc6f1401f2abeed6",
            "12502d37c8424e3290252fd663563dad",
            "885071f080024eadbb05fcc1967a5208",
            "2d7ea4cf29534d168955d5e002c68336",
            "20e87b06d2c74daf8651fce0f213ce9e",
            "c480ef6cc0c841ec9af012b38f481d5e",
            "9508c99692234e968e3c518e03898051",
            "c5bc5c5ff6d14664967a4afb3ee1e63f",
            "78e67d225af14bca8c27461d1f9bb86a",
            "1d14bc186a62421eaff44a26ac895b05",
            "a12ac54e3036483287412fdb159e9c75",
            "df09e71a48be473c9a490581c529cb2c",
            "01d4485af17f41f6b241fa9c74a2e475"
          ]
        },
        "id": "CCxIaW7hHqhB",
        "outputId": "32020c35-1f22-4915-d163-bbbd4cf68455"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/437 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6d0636be08b44cc3b35754a42c1ba821"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/1.23M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0be6262d12174f9d97220824d141d139"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)cial_tokens_map.json:   0%|          | 0.00/125 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2084218865534c4684dd33bf8192dc8a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/1.13k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "fcc33bcc0a2943e18967ce4753ed196a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/737M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "885071f080024eadbb05fcc1967a5208"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']\n",
            "- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[{'label': 'Kredi İşlemleri', 'score': 0.9589990377426147}]\n"
          ]
        }
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "mount_file_id": "19oOgJsPF7IMEttSOjk-zM1b5x9mkQopL",
      "authorship_tag": "ABX9TyOiNNXCN5ne3G6CEjFDDDay",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "59b79c6f5c8b4c41a08de0b4127e46ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_02b998e4ef91487da692ef4a2ea6e92f",
              "IPY_MODEL_dcb6cbb68ede4561bef12163b30518ac",
              "IPY_MODEL_ce11612771f145c6b75fbe0c16796bad"
            ],
            "layout": "IPY_MODEL_07afbe66a72d44adafa8998447490f3a"
          }
        },
        "02b998e4ef91487da692ef4a2ea6e92f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_449ddbe2b069434ca69f21c1fc1e8883",
            "placeholder": "​",
            "style": "IPY_MODEL_9a7a0c7334e24803ac0a2430094f768a",
            "value": "Downloading (…)solve/main/vocab.txt: 100%"
          }
        },
        "dcb6cbb68ede4561bef12163b30518ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9fc70fccbfa5424fb2ec5556326a6557",
            "max": 1233088,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f154edb856274f439ffd0eecb4a7afff",
            "value": 1233088
          }
        },
        "ce11612771f145c6b75fbe0c16796bad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_15bc5d42978e4eceb6133b466c388e1b",
            "placeholder": "​",
            "style": "IPY_MODEL_7d3be203dfd64cc6be1ad3d32224e922",
            "value": " 1.23M/1.23M [00:01&lt;00:00, 1.12MB/s]"
          }
        },
        "07afbe66a72d44adafa8998447490f3a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "449ddbe2b069434ca69f21c1fc1e8883": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9a7a0c7334e24803ac0a2430094f768a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9fc70fccbfa5424fb2ec5556326a6557": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f154edb856274f439ffd0eecb4a7afff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "15bc5d42978e4eceb6133b466c388e1b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7d3be203dfd64cc6be1ad3d32224e922": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e36912ec0b33427c9bedbde8f0f20cfc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ea145eaaf4c4bb1a57448c9bcb01495",
              "IPY_MODEL_0e0b79e1c53f4d608130fb54c0fcbe62",
              "IPY_MODEL_9e2f36b69d9440f6af0539ec71eb0fe1"
            ],
            "layout": "IPY_MODEL_185a6f8f1a894e9ab95b2d0e4f7bcd32"
          }
        },
        "2ea145eaaf4c4bb1a57448c9bcb01495": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ccc752ccb3fc451d8e29e4befda15bfd",
            "placeholder": "​",
            "style": "IPY_MODEL_ce7eee9a75164ffda5d7c212392df8d3",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "0e0b79e1c53f4d608130fb54c0fcbe62": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7e7ea7ad453409ca98a5751323a12fe",
            "max": 59,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_efaa62ab75e747159c31a78a18750fa0",
            "value": 59
          }
        },
        "9e2f36b69d9440f6af0539ec71eb0fe1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6d4fd21bbd1f4f93adaae9bf8506d3cc",
            "placeholder": "​",
            "style": "IPY_MODEL_caa8fe34ff2143eaa076c111ac8180d8",
            "value": " 59.0/59.0 [00:00&lt;00:00, 2.38kB/s]"
          }
        },
        "185a6f8f1a894e9ab95b2d0e4f7bcd32": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ccc752ccb3fc451d8e29e4befda15bfd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ce7eee9a75164ffda5d7c212392df8d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f7e7ea7ad453409ca98a5751323a12fe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "efaa62ab75e747159c31a78a18750fa0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6d4fd21bbd1f4f93adaae9bf8506d3cc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "caa8fe34ff2143eaa076c111ac8180d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9f270f1ce6a64c4c9d44ddb195194de2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0753a531a9154300a7c7bbcd496de476",
              "IPY_MODEL_712e82a8d57b488cb59fd35acfcc1713",
              "IPY_MODEL_3ac15f361f514a97b714d5be041514a8"
            ],
            "layout": "IPY_MODEL_239927f1d3684534a0cc763bd3c7e270"
          }
        },
        "0753a531a9154300a7c7bbcd496de476": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6e7264ba9a0b41e083eefcb607239764",
            "placeholder": "​",
            "style": "IPY_MODEL_3ef5ae4f610b4d3e91160186acaae647",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "712e82a8d57b488cb59fd35acfcc1713": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cb25d88d560c41d994d64873bfc57f6f",
            "max": 386,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a6aed6cae73643c6bad28396c0394e80",
            "value": 386
          }
        },
        "3ac15f361f514a97b714d5be041514a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c550d46640243ecb7ec5c804d9a8f74",
            "placeholder": "​",
            "style": "IPY_MODEL_2f76cdfa80724c329d3b1143a24bc356",
            "value": " 386/386 [00:00&lt;00:00, 2.81kB/s]"
          }
        },
        "239927f1d3684534a0cc763bd3c7e270": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e7264ba9a0b41e083eefcb607239764": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ef5ae4f610b4d3e91160186acaae647": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb25d88d560c41d994d64873bfc57f6f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a6aed6cae73643c6bad28396c0394e80": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9c550d46640243ecb7ec5c804d9a8f74": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2f76cdfa80724c329d3b1143a24bc356": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "548f3cf1155c487b9fbae88878d6dbdd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_aa440a5a5bbf4d00aceff2449c7873d9",
              "IPY_MODEL_8c58e7d59eea4fc9b4757a8a12c9ad2d",
              "IPY_MODEL_9f3215b40d3f4e7584d13c328fbb14f1"
            ],
            "layout": "IPY_MODEL_bb97d0d30ab049f2a29f9d835d09f008"
          }
        },
        "aa440a5a5bbf4d00aceff2449c7873d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2959622874ba4e0d8b0bc23f89d0cafe",
            "placeholder": "​",
            "style": "IPY_MODEL_02c09e9e4fba4312a21f8b56e3938226",
            "value": "Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"
          }
        },
        "8c58e7d59eea4fc9b4757a8a12c9ad2d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d14953dca6b3451cabc504c06cd9d342",
            "max": 740314769,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9f5081ca82994abd962bea5491d95323",
            "value": 740314769
          }
        },
        "9f3215b40d3f4e7584d13c328fbb14f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9443af5a2f02436cade8b5ea2b44294e",
            "placeholder": "​",
            "style": "IPY_MODEL_a77f768e98664d10aefacd911daab57d",
            "value": " 740M/740M [00:36&lt;00:00, 22.5MB/s]"
          }
        },
        "bb97d0d30ab049f2a29f9d835d09f008": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2959622874ba4e0d8b0bc23f89d0cafe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "02c09e9e4fba4312a21f8b56e3938226": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d14953dca6b3451cabc504c06cd9d342": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f5081ca82994abd962bea5491d95323": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9443af5a2f02436cade8b5ea2b44294e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a77f768e98664d10aefacd911daab57d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4383c5fff8df47caaf5c9fb0b15c3c88": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_44646114491a45648c3cbea07f611f1c",
              "IPY_MODEL_aba28ef94f7f4a4dbacdb7de8948ccae",
              "IPY_MODEL_2cb2a08ecb56486c8a2983f3818a1930"
            ],
            "layout": "IPY_MODEL_af065f11114c494384a47d0036764c91"
          }
        },
        "44646114491a45648c3cbea07f611f1c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_80516465b6204c369aac4d578fda6407",
            "placeholder": "​",
            "style": "IPY_MODEL_407fe6feca8049cbb1187219ff2efa55",
            "value": "Downloading (…)solve/main/vocab.txt: 100%"
          }
        },
        "aba28ef94f7f4a4dbacdb7de8948ccae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0f8f001d138a444eb9d527553408a95b",
            "max": 1233088,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0903440cbdd842bca13b5b341e74f7ec",
            "value": 1233088
          }
        },
        "2cb2a08ecb56486c8a2983f3818a1930": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d9aec35cd54f4faab0936b9074698248",
            "placeholder": "​",
            "style": "IPY_MODEL_adac947777c74e13bcd6d36564b40455",
            "value": " 1.23M/1.23M [00:01&lt;00:00, 923kB/s]"
          }
        },
        "af065f11114c494384a47d0036764c91": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80516465b6204c369aac4d578fda6407": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "407fe6feca8049cbb1187219ff2efa55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0f8f001d138a444eb9d527553408a95b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0903440cbdd842bca13b5b341e74f7ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d9aec35cd54f4faab0936b9074698248": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "adac947777c74e13bcd6d36564b40455": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "611961907409400eaa6ec4cde684a6cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d5fa5f43cd0c46c79cb8c29bc8760de7",
              "IPY_MODEL_9c3c5cfe7366405caf3c1f9ebb6e600e",
              "IPY_MODEL_dad2b1f251b44f278c61d59f4fc4e847"
            ],
            "layout": "IPY_MODEL_0418d2389c1e4f389fa51c3c411b043f"
          }
        },
        "d5fa5f43cd0c46c79cb8c29bc8760de7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0135c8422eb143529e44080261242ad7",
            "placeholder": "​",
            "style": "IPY_MODEL_45845d7cd6294ba79cd021d7d4566eee",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "9c3c5cfe7366405caf3c1f9ebb6e600e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cd32b213fece44419978083ebedaf4aa",
            "max": 59,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ed97ec00ba944ce2a76a2af5abf40ce1",
            "value": 59
          }
        },
        "dad2b1f251b44f278c61d59f4fc4e847": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7d1dbc49343141f0baf6d705a9a8ed6c",
            "placeholder": "​",
            "style": "IPY_MODEL_3342bddd4a5c453ba829e272c5973b16",
            "value": " 59.0/59.0 [00:00&lt;00:00, 2.69kB/s]"
          }
        },
        "0418d2389c1e4f389fa51c3c411b043f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0135c8422eb143529e44080261242ad7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "45845d7cd6294ba79cd021d7d4566eee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cd32b213fece44419978083ebedaf4aa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed97ec00ba944ce2a76a2af5abf40ce1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "7d1dbc49343141f0baf6d705a9a8ed6c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3342bddd4a5c453ba829e272c5973b16": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "14d79a3bae574b9cb50eb0dfe72c06db": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2bac87d2a5234178ac89199c51e5dc32",
              "IPY_MODEL_0b881ccdf17b440ca5d7cf88dd222a27",
              "IPY_MODEL_6f7cad1a9419420781c461ef471464d2"
            ],
            "layout": "IPY_MODEL_56b1d3c8055f46dda5141bf3ae2867bc"
          }
        },
        "2bac87d2a5234178ac89199c51e5dc32": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ec7178c6646451ba7836d0d4f5bee91",
            "placeholder": "​",
            "style": "IPY_MODEL_2cf060543a08456984f1d71363b7b237",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "0b881ccdf17b440ca5d7cf88dd222a27": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a8b2c81316db4d8a89bfc34869db8d91",
            "max": 386,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c2caaa4f71ca49d58da7a15c1b75cbb7",
            "value": 386
          }
        },
        "6f7cad1a9419420781c461ef471464d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ca19bbd4b1334743aea25ef2e775334a",
            "placeholder": "​",
            "style": "IPY_MODEL_b1c38a910a9542338310de61e71c16bc",
            "value": " 386/386 [00:00&lt;00:00, 5.49kB/s]"
          }
        },
        "56b1d3c8055f46dda5141bf3ae2867bc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1ec7178c6646451ba7836d0d4f5bee91": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2cf060543a08456984f1d71363b7b237": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a8b2c81316db4d8a89bfc34869db8d91": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c2caaa4f71ca49d58da7a15c1b75cbb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ca19bbd4b1334743aea25ef2e775334a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b1c38a910a9542338310de61e71c16bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b7d8a9dee32b455ea4be76ca8eee877e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1c5aac7c12964cb889ed2c93e743bf40",
              "IPY_MODEL_1242cd42339d41a2b51ba3427aecbd03",
              "IPY_MODEL_e94e228f50b94c3e94aa64d1cebf0a38"
            ],
            "layout": "IPY_MODEL_a991ed66559b4640a2933aa3977e933b"
          }
        },
        "1c5aac7c12964cb889ed2c93e743bf40": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_88cf19b41f9946739c0e5984cfb4f33b",
            "placeholder": "​",
            "style": "IPY_MODEL_5b4ba0e17ea546359a2798ee56ccc855",
            "value": "Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"
          }
        },
        "1242cd42339d41a2b51ba3427aecbd03": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_038e24a9d8b14ee997d8de44dc7bfad5",
            "max": 740314769,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0002d351601b4c76a318ebe611480624",
            "value": 740314769
          }
        },
        "e94e228f50b94c3e94aa64d1cebf0a38": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2736b3e029f54df4bf5dc3143804aa4c",
            "placeholder": "​",
            "style": "IPY_MODEL_884bba6d9c3645a8a661e15085fe2a15",
            "value": " 740M/740M [00:09&lt;00:00, 75.4MB/s]"
          }
        },
        "a991ed66559b4640a2933aa3977e933b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "88cf19b41f9946739c0e5984cfb4f33b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5b4ba0e17ea546359a2798ee56ccc855": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "038e24a9d8b14ee997d8de44dc7bfad5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0002d351601b4c76a318ebe611480624": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2736b3e029f54df4bf5dc3143804aa4c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "884bba6d9c3645a8a661e15085fe2a15": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6d0636be08b44cc3b35754a42c1ba821": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b9af1f4852ce4318934df700c5d0e6b1",
              "IPY_MODEL_002449f8919f46988898ce4f20a5dd5a",
              "IPY_MODEL_053d12bcb58d4d18bfb58e4378e6c189"
            ],
            "layout": "IPY_MODEL_6c51ad635caf4b97a8ca679837520f16"
          }
        },
        "b9af1f4852ce4318934df700c5d0e6b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0a9d1eef1feb4e7298940d26c3e6c49e",
            "placeholder": "​",
            "style": "IPY_MODEL_59cbddd9910c4fba99e44a45b956f287",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "002449f8919f46988898ce4f20a5dd5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6befb313e05940ffaa2763eb0160b52e",
            "max": 437,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7c230cd5592746a8b6e81349809cf21c",
            "value": 437
          }
        },
        "053d12bcb58d4d18bfb58e4378e6c189": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eabd8e2bd8214b8dabe9b5eb9ca010ac",
            "placeholder": "​",
            "style": "IPY_MODEL_3c7ea1ee672a4d71800799f71917605c",
            "value": " 437/437 [00:00&lt;00:00, 6.84kB/s]"
          }
        },
        "6c51ad635caf4b97a8ca679837520f16": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0a9d1eef1feb4e7298940d26c3e6c49e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "59cbddd9910c4fba99e44a45b956f287": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6befb313e05940ffaa2763eb0160b52e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7c230cd5592746a8b6e81349809cf21c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "eabd8e2bd8214b8dabe9b5eb9ca010ac": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3c7ea1ee672a4d71800799f71917605c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0be6262d12174f9d97220824d141d139": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d75e33f6f1ea458488a4ef39f2fe241f",
              "IPY_MODEL_35f9a783c46e43c9953defdad6fa4a65",
              "IPY_MODEL_1ceb6c1312f94267a12a6f8712748e10"
            ],
            "layout": "IPY_MODEL_904c6ed28b4d4a7e891b7839a7e1f48a"
          }
        },
        "d75e33f6f1ea458488a4ef39f2fe241f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_02e78503d48a48f39883157fc162d410",
            "placeholder": "​",
            "style": "IPY_MODEL_2bfcea3c1eb94d8e887d5f92ed81009d",
            "value": "Downloading (…)solve/main/vocab.txt: 100%"
          }
        },
        "35f9a783c46e43c9953defdad6fa4a65": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6c9ec21ed55f49429b72f52ef27f6568",
            "max": 1233088,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_eab69d6179024f71a126e67a802b41d0",
            "value": 1233088
          }
        },
        "1ceb6c1312f94267a12a6f8712748e10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a808cd7642974bc7b76454337f7e17af",
            "placeholder": "​",
            "style": "IPY_MODEL_11a9bfb333f8419dbd4489964610d9b3",
            "value": " 1.23M/1.23M [00:00&lt;00:00, 3.03MB/s]"
          }
        },
        "904c6ed28b4d4a7e891b7839a7e1f48a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "02e78503d48a48f39883157fc162d410": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2bfcea3c1eb94d8e887d5f92ed81009d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6c9ec21ed55f49429b72f52ef27f6568": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eab69d6179024f71a126e67a802b41d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a808cd7642974bc7b76454337f7e17af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "11a9bfb333f8419dbd4489964610d9b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2084218865534c4684dd33bf8192dc8a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_91a0d98708f244268d7d7bfc01953a84",
              "IPY_MODEL_81631544977845cd94a4bfd98a367a47",
              "IPY_MODEL_2525ec8a4ba3469aa5ddbbcbcf8cb844"
            ],
            "layout": "IPY_MODEL_a0d9ce6eaee144f6ae11444bb68db783"
          }
        },
        "91a0d98708f244268d7d7bfc01953a84": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_804b66bcf3a94a10be78453d9da56069",
            "placeholder": "​",
            "style": "IPY_MODEL_5d22cf3f0d21418fb3b86767c6129f09",
            "value": "Downloading (…)cial_tokens_map.json: 100%"
          }
        },
        "81631544977845cd94a4bfd98a367a47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_153c872d546a40d8b8e08be7781ab739",
            "max": 125,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9f8fb890f5f94c57b9fe4744c3ea5355",
            "value": 125
          }
        },
        "2525ec8a4ba3469aa5ddbbcbcf8cb844": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ea471130ded8449fbf1f60108d0eb113",
            "placeholder": "​",
            "style": "IPY_MODEL_a5cf72a1d48a480eb8309d55f44c843a",
            "value": " 125/125 [00:00&lt;00:00, 2.34kB/s]"
          }
        },
        "a0d9ce6eaee144f6ae11444bb68db783": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "804b66bcf3a94a10be78453d9da56069": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5d22cf3f0d21418fb3b86767c6129f09": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "153c872d546a40d8b8e08be7781ab739": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f8fb890f5f94c57b9fe4744c3ea5355": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ea471130ded8449fbf1f60108d0eb113": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a5cf72a1d48a480eb8309d55f44c843a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fcc33bcc0a2943e18967ce4753ed196a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1508dffc4fc641c9b46a7e1d87c723b8",
              "IPY_MODEL_456a81faf4db42949351a07167aeef55",
              "IPY_MODEL_a49a78169c9c44e5a9756d1296438790"
            ],
            "layout": "IPY_MODEL_fcd22a05105d43738fad771e000db933"
          }
        },
        "1508dffc4fc641c9b46a7e1d87c723b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_46d1bf0946174f53bcfc9c96c72943aa",
            "placeholder": "​",
            "style": "IPY_MODEL_909065cda1724fe8b044062643f96caa",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "456a81faf4db42949351a07167aeef55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dc32eae233d143bba39716dd50e3c64c",
            "max": 1132,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_016eeead878d4e7cbc6a23933c07ce02",
            "value": 1132
          }
        },
        "a49a78169c9c44e5a9756d1296438790": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d77b4d29743a4148bc6f1401f2abeed6",
            "placeholder": "​",
            "style": "IPY_MODEL_12502d37c8424e3290252fd663563dad",
            "value": " 1.13k/1.13k [00:00&lt;00:00, 11.9kB/s]"
          }
        },
        "fcd22a05105d43738fad771e000db933": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46d1bf0946174f53bcfc9c96c72943aa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "909065cda1724fe8b044062643f96caa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dc32eae233d143bba39716dd50e3c64c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "016eeead878d4e7cbc6a23933c07ce02": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d77b4d29743a4148bc6f1401f2abeed6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "12502d37c8424e3290252fd663563dad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "885071f080024eadbb05fcc1967a5208": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2d7ea4cf29534d168955d5e002c68336",
              "IPY_MODEL_20e87b06d2c74daf8651fce0f213ce9e",
              "IPY_MODEL_c480ef6cc0c841ec9af012b38f481d5e"
            ],
            "layout": "IPY_MODEL_9508c99692234e968e3c518e03898051"
          }
        },
        "2d7ea4cf29534d168955d5e002c68336": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c5bc5c5ff6d14664967a4afb3ee1e63f",
            "placeholder": "​",
            "style": "IPY_MODEL_78e67d225af14bca8c27461d1f9bb86a",
            "value": "Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"
          }
        },
        "20e87b06d2c74daf8651fce0f213ce9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1d14bc186a62421eaff44a26ac895b05",
            "max": 737478773,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a12ac54e3036483287412fdb159e9c75",
            "value": 737478773
          }
        },
        "c480ef6cc0c841ec9af012b38f481d5e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df09e71a48be473c9a490581c529cb2c",
            "placeholder": "​",
            "style": "IPY_MODEL_01d4485af17f41f6b241fa9c74a2e475",
            "value": " 737M/737M [00:14&lt;00:00, 49.7MB/s]"
          }
        },
        "9508c99692234e968e3c518e03898051": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c5bc5c5ff6d14664967a4afb3ee1e63f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "78e67d225af14bca8c27461d1f9bb86a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1d14bc186a62421eaff44a26ac895b05": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a12ac54e3036483287412fdb159e9c75": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "df09e71a48be473c9a490581c529cb2c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01d4485af17f41f6b241fa9c74a2e475": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}