Deep Learning Fundamentals 3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b841ab31",
   "metadata": {},
   "source": [
    "# Deep Learning Fundamentals 3 - Keras Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf2cd6c0",
   "metadata": {},
   "source": [
    "In the previous notebooks, I focused more on the ideas and theories behind basic deep neural networks. My aim is in this notebook to use Keras to practice the neural networks that I have implemented previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f57b02bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from sklearn.model_selection import train_test_split\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import seaborn as sns\n",
    "from sklearn.metrics import mean_squared_error\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.preprocessing import PolynomialFeatures"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76d75017",
   "metadata": {},
   "source": [
    "## Hello World of Deep Learning: Fashion MNIST dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "443a91a3",
   "metadata": {},
   "source": [
    "Firstly, let's load our dataset and split it into train and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d6eb2bfc",
   "metadata": {},
   "outputs": [],
   "source": [
    "fashion_mnist = keras.datasets.fashion_mnist\n",
    "(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b6ad613c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_valid, X_train = X_train_full[:5000] / 255., X_train_full[5000:] / 255.\n",
    "y_valid, y_train = y_train_full[:5000], y_train_full[5000:]\n",
    "X_test = X_test / 255."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7873cea",
   "metadata": {},
   "source": [
    "I'm planning to use two different models on that dataset. In the first one, I will be using Sequential API and in the second one, I will use Functional API. My aim is to have over %90 accuracy on the validation set for both of the models. I will not use any hyperparameter tuning and regularization in these models."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "558a9e35",
   "metadata": {},
   "source": [
    "### Sequential Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "badfc1ef",
   "metadata": {},
   "source": [
    "Firstly, I want to see whether or not the target variable is skewed. If It is I will use the `class_weight` argument while calling the `fit()` function which would give a larger priority (weight) to underrepresented classes and a lower priority to overrepresented ones. So we would get a higher weight while computing the cost function and this would help us to get a better classifier for all the labels. It may be also good to use the `sample_weight` argument in these cases which give more priority to some samples. There is a nice StackOverflow discussion about the difference between sample_weight and class_weight check it out: [link](https://stackoverflow.com/questions/32492550/what-is-the-difference-between-sample-weight-and-class-weight-options-in-scikit)\n",
    "\n",
    "Note: we can also add sample_weight to validation_data while compiling.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3c1204ff",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\gorke\\anaconda3\\lib\\site-packages\\seaborn\\_decorators.py:36: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:ylabel='count'>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD4CAYAAAAdIcpQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAASzElEQVR4nO3df8yd5X3f8fcHm/KrQYFimGPDzFQ3CrA1CZ7HykS70Ba3TQOLQuRIBKulcoVIRLZqHbTSmm6ylGlt1ZIlSFZIsJs0yIVQaBTSMKdJlo7EeUySGkMYXkjBxcVO0g7YDxLT7/44F8qJ/eDrITznPsf4/ZKOzn2+577O9bVl++P7vu5zP6kqJEk6kuOm3YAkafYZFpKkLsNCktRlWEiSugwLSVLX0mk3MClnnHFGrVq1atptSNJRZefOnd+sqmWH1l+2YbFq1Srm5uam3YYkHVWS/NV8dU9DSZK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSul623+Aed+G/3TrYXDv/89WDzSVJQzkmwkLfc/F7Lx5srr9451/MW//sJT85WA8/+bnPvuB7/+XX/nSwPt7xu784b33TVW8ZrIff/PDtL/jeQ5s+PVgfr/nNNww21w9i2x+vHWyut165Y7C5XirDQtJMePe73/2ynOsH8eO3/9lgc331LZctaD/DYkCP/Yd/PMg85/z7XYPMI+nY4QK3JKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSuiYaFkm+kWRXkq8kmWu105Pcm+SR9nza2P43JtmT5OEkl43VL2yfsyfJTUkyyb4lSd9viCOLf1lVr62qNe31DcD2qloNbG+vSXIesB44H1gHvD/JkjbmZmAjsLo91g3QtySpmcZpqMuBLW17C3DFWP22qnq2qh4F9gBrkywHTq2q+6qqgK1jYyRJA5h0WBTwqSQ7k2xstbOqah9Aez6z1VcAj4+N3dtqK9r2ofXDJNmYZC7J3IEDBxbxlyFJx7ZJ/6S8i6vqiSRnAvcm+doR9p1vHaKOUD+8WLUZ2AywZs2aefeRJL14Ez2yqKon2vN+4E5gLfBkO7VEe97fdt8LnD02fCXwRKuvnKcuSRrIxMIiySlJXvH8NvCzwAPA3cCGttsG4K62fTewPskJSc5ltJC9o52qejrJRe0qqKvHxkiSBjDJ01BnAXe2q1yXAn9UVZ9M8iVgW5JrgMeAKwGqaneSbcCDwEHguqp6rn3WtcCtwEnAPe0hSRrIxMKiqr4O/Pg89W8Bl77AmE3Apnnqc8AFi92jJGlh/Aa3JKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1DXxsEiyJMmXk3y8vT49yb1JHmnPp43te2OSPUkeTnLZWP3CJLvaezclyaT7liR9zxBHFtcDD429vgHYXlWrge3tNUnOA9YD5wPrgPcnWdLG3AxsBFa3x7oB+pYkNRMNiyQrgV8APjBWvhzY0ra3AFeM1W+rqmer6lFgD7A2yXLg1Kq6r6oK2Do2RpI0gEkfWfw+8OvA34/VzqqqfQDt+cxWXwE8Prbf3lZb0bYPrR8mycYkc0nmDhw4sCi/AEnSBMMiyRuB/VW1c6FD5qnVEeqHF6s2V9WaqlqzbNmyBU4rSepZOsHPvhh4U5KfB04ETk3yYeDJJMural87xbS/7b8XOHts/ErgiVZfOU9dkjSQiR1ZVNWNVbWyqlYxWrj+dFVdBdwNbGi7bQDuatt3A+uTnJDkXEYL2Tvaqaqnk1zUroK6emyMJGkAkzyyeCHvAbYluQZ4DLgSoKp2J9kGPAgcBK6rqufamGuBW4GTgHvaQ5I0kEHCoqo+A3ymbX8LuPQF9tsEbJqnPgdcMLkOJUlH4je4JUldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6lpQWCTZvpCaJOnlaemR3kxyInAycEaS04C0t04FXjXh3iRJM+KIYQH8KvAuRsGwk++FxVPA+ybXliRplhwxLKrqD4A/SPLOqnrvQD1JkmZM78gCgKp6b5KfAFaNj6mqrRPqS5I0Qxa6wP2HwO8A/wL4p+2xpjPmxCQ7knw1ye4kv93qpye5N8kj7fm0sTE3JtmT5OEkl43VL0yyq713U5LMN6ckaTIWdGTBKBjOq6p6EZ/9LPCGqnomyfHA55PcA7wZ2F5V70lyA3AD8O+SnAesB85ntEbyX5P8WFU9B9wMbAS+AHwCWAfc8yJ6kSS9BAv9nsUDwD94MR9cI8+0l8e3RwGXA1tafQtwRdu+HLitqp6tqkeBPcDaJMuBU6vqvhZWW8fGSJIGsNAjizOAB5PsYHTEAEBVvelIg5IsYXQV1Y8C76uqLyY5q6r2tfH7kpzZdl/B6MjheXtb7btt+9D6fPNtZHQEwjnnnLPAX5okqWehYfHuH+TD2ymk1yZ5JXBnkguOsPt86xB1hPp8820GNgOsWbPmxZwykyQdwUKvhvrsS5mkqv4uyWcYrTU8mWR5O6pYDuxvu+0Fzh4bthJ4otVXzlOXJA1koVdDPZ3kqfb4f0meS/JUZ8yydkRBkpOAnwa+BtwNbGi7bQDuatt3A+uTnJDkXGA1sKOdsno6yUXtKqirx8ZIkgaw0COLV4y/TnIFsLYzbDmwpa1bHAdsq6qPJ7kP2JbkGuAx4Mo2x+4k24AHgYPAde00FsC1wK3ASYyugvJKKEka0ELXLL5PVf1Ju+z1SPv8JfC6eerfAi59gTGbgE3z1OeAI613SJImaEFhkeTNYy+PY/S9CxeQJekYsdAji18c2z4IfIPR9yIkSceAha5Z/NKkG5Ekza6FXg21MsmdSfYneTLJHUlW9kdKkl4OFnq7jw8xurT1VYy+Pf2nrSZJOgYsNCyWVdWHqupge9wKLJtgX5KkGbLQsPhmkquSLGmPq4BvTbIxSdLsWGhY/DLwVuBvgH3AWwAXvSXpGLHQS2f/I7Chqv4WRj/AiNEPQ/rlSTUmSZodCz2y+CfPBwVAVX2beb6dLUl6eVpoWBx3yI8/PZ0f8FYhkqSjz0L/wf9d4L8nuZ3RbT7eyjz3cJIkvTwt9BvcW5PMAW9g9MOI3lxVD060M0nSzFjwqaQWDgaEJB2DFrpmIUk6hhkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6ppYWCQ5O8mfJ3koye4k17f66UnuTfJIex7/CXw3JtmT5OEkl43VL0yyq713U5JMqm9J0uEmeWRxEPi1qnoNcBFwXZLzgBuA7VW1GtjeXtPeWw+cD6wD3p9kSfusm4GNwOr2WDfBviVJh5hYWFTVvqq6v20/DTwErAAuB7a03bYAV7Tty4HbqurZqnoU2AOsTbIcOLWq7quqAraOjZEkDWCQNYskq4DXAV8EzqqqfTAKFODMttsK4PGxYXtbbUXbPrQ+3zwbk8wlmTtw4MCi/hok6Vg28bBI8sPAHcC7quqpI+06T62OUD+8WLW5qtZU1Zply5a9+GYlSfOaaFgkOZ5RUHykqj7Wyk+2U0u05/2tvhc4e2z4SuCJVl85T12SNJBJXg0V4Bbgoar6vbG37gY2tO0NwF1j9fVJTkhyLqOF7B3tVNXTSS5qn3n12BhJ0gCWTvCzLwbeDuxK8pVW+w3gPcC2JNcAjwFXAlTV7iTbgAcZXUl1XVU918ZdC9wKnATc0x6SpIFMLCyq6vPMv94AcOkLjNkEbJqnPgdcsHjdSZJeDL/BLUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHVNLCySfDDJ/iQPjNVOT3Jvkkfa82lj792YZE+Sh5NcNla/MMmu9t5NSTKpniVJ85vkkcWtwLpDajcA26tqNbC9vSbJecB64Pw25v1JlrQxNwMbgdXtcehnSpImbGJhUVWfA759SPlyYEvb3gJcMVa/raqerapHgT3A2iTLgVOr6r6qKmDr2BhJ0kCGXrM4q6r2AbTnM1t9BfD42H57W21F2z60Pq8kG5PMJZk7cODAojYuSceyWVngnm8doo5Qn1dVba6qNVW1ZtmyZYvWnCQd64YOiyfbqSXa8/5W3wucPbbfSuCJVl85T12SNKChw+JuYEPb3gDcNVZfn+SEJOcyWsje0U5VPZ3konYV1NVjYyRJA1k6qQ9O8lHgp4AzkuwFfgt4D7AtyTXAY8CVAFW1O8k24EHgIHBdVT3XPupaRldWnQTc0x6SpAFNLCyq6m0v8NalL7D/JmDTPPU54IJFbE2S9CLNygK3JGmGGRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lS11ETFknWJXk4yZ4kN0y7H0k6lhwVYZFkCfA+4OeA84C3JTlvul1J0rHjqAgLYC2wp6q+XlXfAW4DLp9yT5J0zEhVTbuHriRvAdZV1a+0128H/llVveOQ/TYCG9vLVwMPv4RpzwC++RLGL5ZZ6GMWeoDZ6GMWeoDZ6GMWeoDZ6GMWeoDF6eMfVtWyQ4tLX+KHDiXz1A5LuaraDGxelAmTuapasxifdbT3MQs9zEofs9DDrPQxCz3MSh+z0MOk+zhaTkPtBc4ee70SeGJKvUjSMedoCYsvAauTnJvkh4D1wN1T7kmSjhlHxWmoqjqY5B3AnwFLgA9W1e4JT7sop7MWwSz0MQs9wGz0MQs9wGz0MQs9wGz0MQs9wAT7OCoWuCVJ03W0nIaSJE2RYSFJ6jIs5jELtxZJ8sEk+5M8MI35Ww9nJ/nzJA8l2Z3k+in0cGKSHUm+2nr47aF7OKSfJUm+nOTjU5r/G0l2JflKkrlp9ND6eGWS25N8rf35+OcDz//q9nvw/OOpJO8asoexXv51+7P5QJKPJjlxCj1c3+bfPanfB9csDtFuLfI/gJ9hdMnul4C3VdWDA/dxCfAMsLWqLhhy7rEelgPLq+r+JK8AdgJXDPl7kSTAKVX1TJLjgc8D11fVF4bq4ZB+/g2wBji1qt44hfm/Aaypqql+ASzJFuC/VdUH2hWKJ1fV302plyXAXzP6ou5fDTz3CkZ/Js+rqv+bZBvwiaq6dcAeLmB0V4u1wHeATwLXVtUjizmPRxaHm4lbi1TV54BvDz3vIT3sq6r72/bTwEPAioF7qKp6pr08vj2m8j+cJCuBXwA+MI35Z0WSU4FLgFsAquo70wqK5lLgfw4dFGOWAiclWQqczPDfAXsN8IWq+j9VdRD4LPCvFnsSw+JwK4DHx17vZeB/IGdRklXA64AvTmHuJUm+AuwH7q2qwXtofh/4deDvpzQ/jILyU0l2ttvbTMM/Ag4AH2qn5D6Q5JQp9QKj7119dBoTV9VfA78DPAbsA/5XVX1q4DYeAC5J8iNJTgZ+nu//EvOiMCwOt6BbixxLkvwwcAfwrqp6auj5q+q5qnoto2/ur22H3YNK8kZgf1XtHHruQ1xcVa9ndAfm69rpyqEtBV4P3FxVrwP+NzCttb0fAt4E/PGU5j+N0ZmHc4FXAackuWrIHqrqIeA/AfcyOgX1VeDgYs9jWBzOW4uMaesEdwAfqaqPTbOXdqrjM8C6KUx/MfCmtmZwG/CGJB8euomqeqI97wfuZHTadGh7gb1jR3i3MwqPafg54P6qenJK8/808GhVHaiq7wIfA35i6Caq6paqen1VXcLo9PWirleAYTEfby3StMXlW4CHqur3ptTDsiSvbNsnMfrL+bWh+6iqG6tqZVWtYvRn4tNVNej/IJOc0i40oJ32+VlGpyAGVVV/Azye5NWtdCkw6AUgY97GlE5BNY8BFyU5uf19uZTR2t6gkpzZns8B3swEfk+Oitt9DGlKtxY5TJKPAj8FnJFkL/BbVXXLwG1cDLwd2NXWDAB+o6o+MWAPy4Et7YqX44BtVTWVy1ZnwFnAnaN/k1gK/FFVfXJKvbwT+Ej7D9XXgV8auoF2fv5ngF8deu7nVdUXk9wO3M/o1M+Xmc6tP+5I8iPAd4HrqupvF3sCL52VJHV5GkqS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHX9f/F8Vtp9S+BuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3de20c2f",
   "metadata": {},
   "source": [
    "Okay now, let's start constructing our model. I will start with a flattened layer, I guess I have talked about the function `Flatten()`  in my previous notebooks but to revise it, It converts each input into a one-dimensional array. It basically does `.reshape(-1,1)`. I will also set the `input_shape()`. Then in the following layers, I simply used the ReLU activation function and in the last node, since we have 10 different outputs, I will use the softmax activation function. \n",
    "\n",
    "Documentation for activation functions: [link](https://keras.io/initializers/.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8a1bbbf1",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential()\n",
    "model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "model.add(keras.layers.Dense(400, activation=\"relu\"))\n",
    "model.add(keras.layers.Dense(260, activation=\"relu\"))\n",
    "model.add(keras.layers.Dense(140, activation=\"relu\"))\n",
    "model.add(keras.layers.Dense(90, activation=\"relu\"))\n",
    "model.add(keras.layers.Dense(10, activation=\"softmax\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a3fad75d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAALhCAYAAABxKfI1AAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdW2wb150/8C9jOwncB7JOSyVRK2+AwIKTbmWkgCy3XQuWDRg2MGy2sAxJDu0XSiAfHLgwH9aKBEGQ4aYAiQ3sAhZIvhiELMIK8E852PjFIqA0iGkD7orbdQoLzYVq17timy2JbG9xnPk/2Gc0w4s0HF6Gl+8HIGyeGc05M5TOj3POmXNsiqIoICIiKtMTVheAiIiaEwMIERGZwgBCRESmMIAQEZEpW/MT/ud//gc/+clP8PDhQyvKQ0REDWbLli3413/9Vzz77LO69II7kEQigVgsVreCETWz1dVVLCwsWF2MpnDr1i3cunXL6mKQCbFYDIlEoiC94A5EuHbtWk0LRNQKrl69ihMnTvDvxYATJ04AAObm5iwuCZXLZrMVTWcfCBERmcIAQkREpjCAEBGRKQwgRERkCgMIERGZwgBC1CAmJycxOTlpdTEais1m072KyWQyCAaDdS5Z4wkGg8jlckW3GbmOZjCAEBEAIJfLVbVyqSZFUVBs4vBMJoOpqSlIkqSmxWIxuFwu2Gw2+Hw+ZDKZsvNbXV2Fz+dTj1HsGQgAkGVZzcvlcpl+hq4a+R06dAhut7vo+Za6fhVT8szNzSlFkomoiFb6e4nH4zU9l5GREWVkZKSsnwFQskzZbFaRJEm5efOmmhYKhZTFxUX1/fz8vCJJkrK8vGw4z2w2q8TjcfX/8/PzCgA1TQgEAgoA9djLy8sKACUQCBjOq9r53bx5U5EkSclms0Xz2uh6bgSAMjc3V5ien9BKfxBEtdYqfy+iMm6mABIIBJSJiYmC/efn5wvSJEkynGd+xV2qHKXSysmrFvl5vd6SQazaAYRNWEQNIJPJqE0vxd7Lsqw2W6yurqr7iCYNAAiHw2oTyMrKinrsYm3f+WmBQACyLOu2AY3bL5PJZOD3+3HgwAFdeigUwtWrVwv27+zsNHxsbXOYltfr1b0PBAIAgGQyCQDq5zIzM2M4r1rkNzg4CL/fb6rprmz5EaVVvlER1UO1/l7Et39xLO170USTTqcVAIrX61UUZf3bpHafbDareL1eBYBy7949RVEUZW1treCbpziWNi3/vaIoysTERMG3fLOqeQcimtvS6fSGP3/v3j1ds48Z2Wy2aJOSojy6PuL6z8/PK2tra6bzqVZ+4rM1emdjBHgHQtS44vF4yfd9fX0AgK6uLgDA7OwsAOg6RcU+drtd/eYq7iicTmdBfuJYm5mZmSn7G3U93L59G8Dm5xGNRrG8vIyenh7Ted25cweSJGH//v0F22ZmZuD1erFv3z7cvXsXTz31lOl8qpWf3W4HAN1daK0wgBC1GFFZ+v1+i0tSO+fPn990n0QigWPHjlUUPADgrbfewvj4uFoxawWDQfT39yObzQIA3G53yaG09cpP/Fw9Pn8GECJqSdu3b684eMRiMUiSpN7h5W/z+/04cuQI7HY73G43ZFmuaGbmeudXKQYQohaV3wnbTmKxWNFKuBypVAp3797F6Oho0e3Dw8MA1r/xd3R0AADGxsaaIr9qYAAhajGi7fvo0aMWl6R2xIikUs1FQ0NDFR0/k8ngxo0buv6fVCoFn8+nvs8fPSUq9lKjquqd38TERNnlKBcDCFED0A65zGQyuveiktRWlvlDNMUTyblcDtFoFJIk6SoWcTcigosYCgpAraTE/tqpQRp1GO+uXbsAlA4gpcodDAZhs9mQSqVKHjuTycDj8cDv9+uGO+/Zs0cXlM+cOQNg/dqLayrSrcgPWB/e29vbWzLPamEAIWoAojlC/F/73uFw6P7N3x8Adu/eDZfLBYfDga6uLkSjUd32c+fOQZIkdHd3Q5Zl9PX1QZIkzM/PY3p6GsD68wSXLl2C2+2u7glW2d69ewEA9+/fL+vnstksvF7vhkFxampKHcGWr7u7W/3/wMAAFhcXsbS0BJvNhitXrmBxcREDAwOW5QesXxNxjWrJpij6CVLEEp1KLeZNIWoxVv+9iAf+muHv1cySthudn7hLOnv2bNllcblcBUOna6me+U1OTsLhcBS9LmZ/X2w2G+bm5jAyMqJL5x0IETUlj8eDpaUlXXOcEclkEuPj4zUqlbX5pVIppFIpeDyeuuTHAELUpPL7TdqN3W5HJBLBhQsXNuxj0EokEtixY0fFI7SMqmd+KysrmJ2dRSQSKfoMSS1UNYAkk0l1SmIxJ4+Yp6dZNWonIlF+v0krK7WOhdPpRDQaxY0bNwwdZ2BgQO2Ar4d65ifLMqanp4vOPFDtdUCErdU6UCKRwMGDB5FOp3H58mX4fD51ygWjcrkcHA6Hrn2uWFo7MXP+pX5RrLiG+eVvpLI1u3a4ZkbO0W63m+oHaTUbXYNa/a5U7Q5kYWEBwPrcNJcvXy77GO+9956htHqyei4gM+evKIo61QHwaCSIVZVNfvkVRcHa2pr63sqyEVFlqhZAyr3byJfL5RAOhzdNayeVnL+2DbRe7aH5SpVfe4ttVdmIqHIVB5BS6wwUIyoUsc/k5KTa+VdsPYJSaxQA6w87iTUSxBKQRtZRMKrV1mholPKXo9TvjPjsxUu7JrZ2m/a8Sv2+iPPN5XLw+Xzs8yIyKn9+d7PrG8DAClpinYK1tbWCtQ2MHkNRHq1vIEmSuvLY4uKiOue/kXUUjGr2NRryf7ZRyr9Rer6Nfmdu3rxZ8nOVJEldK6Gc35fl5eWyfk+4fo5xZtYDocaAWi9pa6Tyn5iY2DBgGA0gYs3g/P1EpWr0OGbOy8ixi+1TbP1is8cyW/ZGKr/R89rsd0asE61dWGh5eVm3rKnR35dS60hvhAHEOAaQ5tUQAURIp9PqH76ZAKL91pj/Krcs5Z5XNSvNZgog1S5/uedV6ndGBLZQKKSmBQIBXUAx8/tilPh74YuvVn8VCyBVG8ZrVDgchizLCAQCphc8Ee3qCkfvtIWNfmd6enrg9XoxNjaG48ePAwB++9vf6laqq8fvi5VrMjSLixcvAgBef/11i0tC5RJ/W/nqGkBisRjGxsaQTqcNL6m5kZWVlbo+FFQNzb5GQ73K7/P5cPnyZUO/M16vF7Ozs7h+/Tq+9rWv4dSpU0X3q+Xvy+DgYE2O20reeecdALxWraSuU5mIBVEqDR6hUAjAo/WOxXTO2imoG1Gzr9FQz/Ink0n09/cDMPY7I+5ChoeHEQ6HC6aNaMbfF6JmUJUAop2HRlQ0xebpEesNrK6u6oaE5m/X/nEXS/vRj34E4NG6yA6HAzabDR0dHRgcHCx7HYWNNPMaDdpyaSvNRij/Rp9BMpnEvn37sHv3bt3Pl/qdEcRdR7HFdYz+vhBRmUp1ChoFgx0wirLe4TkxMaGsra2pI2xEh2f+9lJpivKoU3ViYkIBoDtGsXyLpVXj3DbLTztMNBQKFYzySafT6vZ4PK4oiqION93o/DcbxlvOZ1Lv8hstm8hrs98ZLUmS1GHG+Yz8vkiSVPKalsJRWMZxFFbzQolOdK4HUgPNtEZDMc1Y/lwuh3/5l38xNYVOJfj3YpyZ9UCoMXA9EGpp165dY+csUZ0xgFRZs6/R0Ezln5yc1E1Zkr+0JzU/7XQ1pabC4YCIR4LBYMk14o1cRzPaMoDkX8xSLzOafY2GZiq/GJkVCoUsnTHZSrlcribrPNTr+EYpjx56LkjPZDKYmprSDZ4Q872JOdzMfBFaXV1V1zby+Xzq3Gn5xDxqYo41MfDEivwOHToEt9td9HxLXb+K5XeKsFOQyDir/17i8XhN86/m8c10omODgS/ZbFaRJEmds01RFCUUCimLi4vq+/n5eUWSJGV5edlwntlsVh0Uks1m1alwRJogZkYQxy423U+987t586YiSVLJaXk2up4bQYlOdAYQogpY+fciKtBa5V/t41c7gAQCgYLRiAB086CJtHJG2OVX3KXKUSqt3NF81c7P6/WWDGLVDiBt2YRFZLVcLodYLKY2l4bDYV3Tg9np8ht5OYFqymQy8Pv9OHDggC49FArh6tWrBft3dnYaPnaxZ4mAwlkYAoEAgPXnn8TSAeU2p1Y7v8HBQfj9/rr0YTKAEFnA7Xbj888/V1dolGUZHo9H7QTVrtoopNNp3XttxaE8buPu6OiAy+WCLMtIJpMYHR1VV6fs7u5Wg4jZ4zeKW7duAQBefPFFXfro6Cji8bj6XpxvJVPwiM8kfxaGs2fPYmJiAvv27UMymcQHH3yAtbU19PT0mM6rGvmJayKuUS0xgBDVWSKRgCzL6hPyTqcT4+PjkGUZ169fV9PyGZkCSFvJiyld7Ha7WoGKOwqzxwesX+YZAG7fvg1g8zJHo1EsLy9XVKnfuXMHkiRh//79BdtmZmbg9Xqxb98+3L17F0899ZTpfKqVn1jlU3vHWSsMIER1trCwAEBfiYupW4o1v1SDqEDNzoDdaM6fP7/pPolEAseOHav4juCtt97C+Ph40eWXg8Eg+vv71bs8t9tdcihtvfITP1ePz5oBhKjOZmdnC9LEH724Q6DKbd++veLgEYvFIElSwQSdYpvf78eRI0dgt9vhdrshy3JFU/vXO79KMYAQ1Zl2gsl8tZ4uv9mXEzAqFosVrYTLkUqlcPfuXYyOjhbdLmaKFsFfPDc1NjbWFPlVAwMIUZ2J+YQ+/vhjNU00Q9RqOpZmX04gnxiRVKq5aGhoqKLjZzIZ3LhxQ9fXk0ql1FmmgcLRU6JiLzWqqt75TUxMlF2OcjGAENXZkSNHIEkSLly4oN6FXL9+HV6vVzcdi9np8gUrlxOoNbEwWKkAUqqMwWAQNptNtwRFvkwmA4/HA7/frxvavGfPHl0APnPmDID16yyun0i3Ij9gfXhvb29vyTyrhQGEqM7sdjsikQgkSUJHR4f6fMWbb76p2+/cuXOQJAnd3d2QZRl9fX2QJAnz8/OYnp4GsD7U9tKlS3C73bqf3717N1wuFxwOB7q6uhCNRqt6fCvt3bsXAHD//v2yfi6bzcLr9W4YAKempkr2RXV3d6v/HxgYwOLiIpaWlmCz2XDlyhUsLi7qvgTUOz9g/ZqIa1RLnM6dqAKN+PfSqNPxm5nOfaNzEXdEZ8+eLbssLpdL97xIrdUzv8nJSTgcjqLXxezvBqdzJ6KW4vF4sLS0pGt6MyKZTGJ8fLxGpbI2v1QqhVQqBY/HU5f8GECIWkgzTcdfKdEUeOHChQ37GLQSiQR27NhR8Qgto+qZ38rKCmZnZxGJRIo+Q1ILDCBELaSZpuMvR6klFpxOJ6LRKG7cuGHoOAMDA2oHfD3UMz9ZljE9PV10loFqrwMibK36EYnIMo3W71EpI+djt9tN9YO0mo2uQa1+L3gHQkREpjCAEBGRKQwgRERkCgMIERGZUrITXUw5TUSliUV7+PeyOTHFBq9V6yh4Ev327dt1eQSeiIiax61btwrm1yoIIET0iJmpN4jaCftAiIjIFAYQIiIyhQGEiIhMYQAhIiJTGECIiMgUBhAiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhIiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAhIiJTGECIiMgUBhAiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhIiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAhIiJTGECIiMgUBhAiIjKFAYSIiExhACEiIlMYQIiIyJStVheAqBH8+c9/xuXLl/Hw4UM17cMPPwQA/OxnP1PTtmzZgtOnT+Opp56qexmJGo1NURTF6kIQWe2Xv/wl9u/fDwAlg8Pf//53AMCtW7fQ29tbt7IRNSoGECIADx8+REdHBz777LMN93vmmWewtraGLVu21KlkRI2LfSBEeNQ09dprr+HJJ58suc+TTz6J1157jcGD6DEGEKLHRkZG8MUXX5Tc/sUXX2BkZKSOJSJqbGzCItLo6urC7373u6Lbvv3tb2N1dbXOJSJqXLwDIdI4efIktm3bVpC+bds2nDx50oISETUu3oEQaXz44Yd4+eWXi267e/cuXnrppTqXiKhx8Q6ESOOll17Cyy+/DJvNpqbZbDa8/PLLDB5EeRhAiPKcPHkSW7euP2O7detWNl8RFcEmLKI86XQaL7zwAsSfhs1mwyeffIKdO3daXDKixsI7EKI8O3fuRG9vL5544gk88cQT6O3tZfAgKoIBhKiIU6dO4auvvsJXX32FU6dOWV0coobEJiyiIv74xz/im9/8JgDgD3/4A77xjW9YXCKiBqS0gDfeeEMBwBdffPHVFK833njD6mqzKlpiOvdPPvkE27Ztw9zcnNVFoTo7fvw4Xn/9dfzwhz+s+rH/+te/wmaz4emnn676sevt/fffx8WLF3Ht2jWri9L2Tpw4gU8++cTqYlRFSwQQABgcHMTg4KDVxSAL7N27l5/9Jh48eAAAvE4N4J133rG6CFXDTnQiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhAjA5OQkJicnrS5Gw8pkMggGg1YXw3LBYBC5XM7qYjQMBhCiBpDL5XRTyDeSTCaDqakpSJKkpsViMbhcLthsNvh8PmQymbKPu7q6Cp/Ppx4jkUgU3U+WZTUvl8uFWCxm6jyqkd+hQ4fgdrtNnW9LsvpJxmoYGRlRRkZGrC4GWQCAMjc3Z3UxKhaPx5Va/jnOzc2ZOn42m1UkSVJu3ryppoVCIWVxcVF9Pz8/r0iSpCwvL5d13Hg8rv5/fn5eAaCmCYFAQAGgHnt5eVkBoAQCgbLPo1r53bx5U5EkSclms2WVQWil+ooBhJpaKwQQUUk3YgAJBALKxMSELg2AMj8/X5AmSZLh4+ZX3OIY+WUslVZOXrXIz+v1lh3EhFaqr9iERW0vk8moTTLF3suyrDZnrK6uqvuIpg4ACIfDatPIysqKemybzaa+SqUFAgHIsqzbBljfL5PJZOD3+3HgwAFdeigUwtWrVwv27+zsNHxsbXOYltfr1b0PBAIAgGQyCQDq9Z+ZmTGcVy3yGxwchN/vZ1OW1RGsGlopolN5UIU7EPHtX/w5aN+Lppt0Oq0AULxer5pv/j7ZbFbxer0KAOXevXuKoijK2tpawbdacSxtWv57RVGUiYmJgm//Zpm5AxHNaul0esP97t27p2v2MSObzRZtUlKUR9dBXOf5+XllbW3NdD7Vyk98hsV+fjOtVF/xDoTaXjweL/m+r68PANDV1QUAmJ2dBQB1tULtPna7Xf1GK+4onE5nQX7iWJuZmZkp+5t2Nd2+fRvA5uWNRqNYXl5GT0+P6bzu3LkDSZKwf//+gm0zMzPwer3Yt28f7t69i6eeesp0PtXKz263A4DubrMdMYAQVZGoRP1+v8Ulqdz58+c33SeRSODYsWMVBQ8AeOuttzA+Pq5WzFrBYBD9/f3IZrMAALfbXfFQ2krzEz/XCp9zJRhAiMi07du3Vxw8YrEYJElS7+Tyt/n9fhw5cgR2ux1utxuyLFc0LX2982tlDCBENZDfOduKYrFY0Uq4HKlUCnfv3sXo6GjR7cPDwwDWv/F3dHQAAMbGxpoiv1bHAEJURaJN/OjRoxaXpHJiRFKp5qKhoaGKjp/JZHDjxg1dP08qlYLP51Pf54+eEhV7qVFV9c5vYmKi7HK0EgYQanvaoZiZTEb3XlSe2ko0f+imeFI5l8shGo1CkiRdhSPuRkRwEUNEAaiVl9hfO2WI1cN4d+3aBaB0AClVvmAwCJvNhlQqVfLYmUwGHo8Hfr9fN6x5z549uuB75swZAOvXWFw7kW5FfsD68N7e3t6SebYDBhBqe6KZQvxf+97hcOj+zd8fAHbv3g2XywWHw4Guri5Eo1Hd9nPnzkGSJHR3d0OWZfT19UGSJMzPz2N6ehrA+nMGly5dgtvtru4JmrR3714AwP3798v6uWw2C6/Xu2Hwm5qaUkeq5evu7lb/PzAwgMXFRSwtLcFms+HKlStYXFzEwMCAZfkB69dEXKN2ZVO04xGb1IkTJwCAa6K3IZvNhrm5OYyMjFiSN6Af0tuorl69ihMnTpRdVnE3dPbs2bLzdLlcBUOka6me+U1OTsLhcJi6Lq1UX/EOhIhK8ng8WFpa0jW7GZFMJjE+Pl6jUlmbXyqVQiqVgsfjqUt+jaztA0gymVRn6BRTUYjpKVpZ/nQdVJ78fpNWZbfbEYlEcOHChQ37GLQSiQR27NhR8Qgto+qZ38rKCmZnZxGJRIo+Q9JutlpdACslEgkcPHgQ6XQaly9fhs/nU580NiqXy8HhcOiaBoql1YPR6cAVRcHU1FRTn6vV8vtNWvn8nU4notEoIpGIoWc+8vsLaq2e+cmyjOnp6aIzDLSjtr4DWVhYALA+VcPly5fLPsZ7771nKK0eFEVRn54V77WvxcVFdVuzn6vV8q9tq7Pb7aba+1vN2bNnGTw02jqAlPsNPF8ul0M4HN40rZ42uq2u5JtaI54rEVmrLQNIqem1ixGVpNhncnJSbfMuNg13qam5gfUx/mJqcLEimpHpw4HKngswMmKokc6ViJpAXef+rRGz0yPDwIIyYnrutbW1gim9jR5DUR5N6y1JkroQz+LiojoFtpHpwxXF+PTe+fmLY222XyOdq1FogQWl6sHsglJUfa00nXtL/EbVMoBMTExsWIkarVTFEpr5+4mAYPQ45ZxX/qvUfkKznisDyOYYQBpHKwWQth6FZYR4Qnh1dVXtdDdDrOCW31R2/vz5mq35oDxurlpdXcXOnTs33b9Zz/XWrVvYtm1b1Y/bSm7dugUAFX2uVB2rq6uG14RpeFZHsGqo5R2IoihKKBRSJElSV16DiW/lxdIq/Zlyj2Vkv2Y9V774aqYX70DaRCwWw9jYGNLpdFW+NaysrKiT1NWTYmCoabOeq1VTmTQTs1OZUPWJqUxaQVuOwiqHWB+g0go1FAoBeLT8p5jdVDvzaiNop3Mlosq1bQDRTssgptkuNj2FmGZ7dXVVt/5x/nZtBVks7Uc/+hGAR/0ADocDNpsNHR0dGBwcNDx9uJFhvNqf22jZz0Y/VyJqfG0ZQMQ6AEJ3d7dayQni/6LTNxwOw+FwYGJiAl6vF3/7299027XTcBdLczqdSKfT6gI0Xq9XbSoqd/rwjc5L+3Oi8i6m2c+ViKzH6dypqVk5nXszYR9I42il+qot70CIiKhyDCBERGQKAwgRVU0zj7YLBoMbDjyhQgwgRCblcjnDa7A04vGrLZPJYGpqSh2ZB0CdOFMs1lbuKDtxDYq9YrGYbl9ZltW8XC5XwfZ8YuJQ4dChQ3C73RwJWAYGECKTar0WSjOttZLL5eDxeHDq1Cn14dFwOAyn04l4PA5FUdDf3w+Px2N4ZUMA+M1vflNym3Z5gmAwCJfLhZmZGSiKgpmZGQwPD5e8G0qlUhgbG9Ol9fT0YHx8HB6Ph3ciBjGAEJlQ67VQmm2tFbFaoXZZ2bGxMd23+aGhIciyXNaSBJ9++inS6bRu8a61tTVMTEzoFnby+/0AoK6YKP5dWloqOGYul8Pbb79dNL++vj50dnYiEokYLmM7YwChtpPL5RCLxdSmkHA4rKvotM0kpdKKrYWSyWTUZhRgvYnE5/PpHsw0e3ygsjVhaiWTycDv9+PAgQO69FAopE6sqdXZ2Wn42AMDAwUzIyQSCRw7dkyXFggEAADJZBIA1LVlik3eGYlEcPr06ZJ5Dg4Owu/3synLAAYQajtutxuff/65+m1WlmVds8Xa2lrBz6TTad17bcUkvhl3dHTA5XJBlmUkk0mMjo6qSwx3d3erQcTs8RuVmOn3xRdf1KWPjo4iHo+r78X5e71ew8cutnzs0tJSwdrsZ8+excTEBPbt24dkMokPPvgAa2trBfslEgn84Ac/2HBZWnEe4ryoNAYQaiuJRAKyLKvTrTidToyPj0OWZVy/fl1Ny2dkfjBtJS+acux2u1phijsKs8cHHgWWWk3/b9bt27cBbH4O0WgUy8vLBZV6OVKpFPr7+4tum5mZgdfrxb59+3D37l089dRTuu2ZTAYfffSRrpmtGLEstPaukYpjAKG2ItbD0Fbiu3fvBoCizS3VICpM0U7fas6fP7/pPqLZqZLgAQBvv/22rvNcKxgMor+/X73rc7vdus7wX/ziFxgdHd00DxFAWvXzqiYGEGors7OzBWmiwhB3CFR927dvrzh4iD6JYndwsVgMfr8fR44cgd1uh9vthizLuHbtGoBHn+3hw4cryp8KMYBQW9HOHpyvnLZ5M2p9/EYVi8U2bTYyoljnuSCWIhBfBsSknGKorsvlws6dO0sOYCBzGECorYhJFz/++GM1TTRzDA4O1iRP0ZZ+9OjRmhzfamIEVKlnJ4aGhqqST7HOc0H78CKwHkhEunYYcP6ghFIDFMRs0lQaAwi1lSNHjkCSJFy4cEG9C7l+/Tq8Xq+ubV3cLYjKXwwPBQCfzweg+FoogngKOpfLIRqNQpIkXSVn9viNOIxXPDhYKoCUKnMwGITNZjP0YOFGnecAcObMGQDr111cT5FeDjEEuLe3t+yfbTcMINRW7HY7IpEIJElCR0eH2nzx5ptv6vY7d+4cJElCd3c3ZFlGX18fJEnC/Pw8pqenARRfC0XYvXs3XC4XHA4Hurq6EI1Gq3r8RrJ3714AwP3798v6uWw2C6/XayggbtR5Djx6XmRxcRFLS0uw2Wy4cuUKFhcXN/yZUsR5iPOi0rgeCDW1RlsPRASkRvuzqvV6IOIO6ezZs2X/rMvl0j0vYrXJyUk4HA5T52JEK9VXvAMhoop5PB4sLS3pmuKMSCaTGB8fr1GpypdKpZBKpeDxeKwuSlNgACGqkmLrzLcL0TR44cIFw5MlJhIJ7NixoyojtKphZWUFs7OziEQiaic8bYwBhKhKiq0z306cTiei0Shu3LhhaP+BgQG1A74RyLKM6enpDac5Ib2tVheAqFU0Wr+HFex2e836DmqtWcttJd6BEBGRKQwgRERkCgMIERGZwgBCRESmtEwn+tWrV/HgwQOri3HxPt4AACAASURBVEEWuHjxIt555x2ri9HQxPQcx48ft7gktLCw0DAPvlaqJZ5El2W5YKoIokr953/+JwDgO9/5jsUloVbjdrsLJoBsRi0RQIhqoZWmnCCqBfaBEBGRKQwgRERkCgMIERGZwgBCRESmMIAQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSkMIEREZAoDCBERmcIAQkREpjCAEBGRKQwgRERkCgMIERGZwgBCRESmMIAQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSkMIEREZAoDCBERmcIAQkREpjCAEBGRKQwgRERkCgMIERGZwgBCRESmMIAQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSk2RVEUqwtBZLXf/va36OnpwT/8wz/giScefa/67LPPAADPPPMMAOCrr77Cp59+io8++gjPPvusZWUlahRbrS4AUSN4+PAh/vKXv+DDDz8s2Pbf//3fuve5XI4BhAhswiICAHR3d+O73/0ubDZbyX1sNhu++93voru7u44lI2pcDCBEj506dQpbtmwpuX3Lli04depUHUtE1NjYB0L02P379/Gtb30Lpf4kbDYbfv/73+P555+vc8mIGhPvQIgee/755/H9739f7UTXeuKJJ/D973+fwYNIgwGESOPkyZNF+0FsNhtOnjxpQYmIGhebsIg0/vd//xcdHR348ssvdelbt27F2toaduzYYVHJiBoP70CINHbs2IHDhw9j69b1Ee5bt27F4cOHGTyI8jCAEOUZGRnBV199pb7/6quvMDIyYmGJiBoTm7CI8vz5z3/GN77xDfztb38DADz99NP44x//iK997WsWl4yosfAOhCjP1772Nbz66qvYtm0btm3bhldffZXBg6gIBhCiIl577TU8ePAADx48wGuvvWZ1cYgaUtvOhfXll18iHo/j4cOHVheFGpD29+Lzzz/HwsKChaWhRrVlyxa4XC7doIt20rZ9IO+88w7++Z//2epiEFGT+3//7//h1VdftboYlmjPsAngL3/5CwCUnLaCmtvVq1dx4sQJfr4GnDhxAgAwNzdncUmaj81mU+uSdsQ+ECIiMoUBhIiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAh2sTk5CQmJyetLkbDymQyCAaDVhfDlGAwiFwuZ3UxmhYDCFGDy+VyRRe5agSZTAZTU1OQJElNi8VicLlcsNls8Pl8yGQyZR1TnG+xVywW0+0ry7Kal8vlKtieLxwO667loUOH4Ha7yy4jPcIAQrSJmZkZzMzMWJb/e++9Z1neG8nlcvB4PDh16hR27doF4FEF7XQ6EY/HoSgK+vv74fF4kEqlDB/3N7/5TcltAwMD6v+DwSBcLhdmZmagKApmZmYwPDxc8m4olUphbGxMl9bT04Px8XF4PB7eiZjAAELUwHK5HMLhsNXFKCoSiaCnpwd9fX1q2tjYmO7b/NDQEGRZLqsJ8NNPP0U6nYaiKOprbW0NExMTcDqd6n5+vx/AoyCg/XdpaangmLlcDm+//XbR/Pr6+tDZ2YlIJGK4jPQIAwjRBjKZjNokU+y9LMtq88nq6qq6j2haAdabTXw+H1ZWVtRja5tmSqUFAgHIsqzbBljfL5PJZOD3+3HgwAFdeigUwtWrVwv27+zsNHzsgYEBdHV16dISiQSOHTumSwsEAgCAZDIJAOr1L3a3GIlEcPr06ZJ5Dg4Owu/3symrXEqbmpubU9r49FtetT5fSZIUAOqxtO9v3rypKIqipNNpBYDi9XoVRVHU7dp9stms4vV6FQDKvXv3FEVRlLW1Nd2xtcfSpuW/VxRFmZiYUCYmJio+P0VRlJGREWVkZKSsn4nH4woAJZ1Ob7jfvXv3FADK8vJyJUVUr22+iYkJ9TrPz88ra2trBfssLi6qn0Oxa6ko69c9Ho+XVS4AytzcXFk/00p4B0K0gXg8XvK9aLoR35ZnZ2cB6CfoFPvY7XZ4vV4AUO8otM0xQv4371Ks7pe5ffs2gM3LG41Gsby8rDYvmZFKpdDf319028zMDLxeL/bt24e7d+/iqaee0m3PZDL46KOPdM1sxdjtdgDQ3SHS5hhAiOpEVKKi7b6ZnT9/ftN9RLNTJcEDAN5++21d57lWMBhEf38/stksAMDtdus6w3/xi19gdHR00zxEAGmFz6aeGECIqCa2b99ecfAQfRLF7tZisRj8fj+OHDkCu90Ot9sNWZZx7do1AI/u9A4fPlxR/rQxBhCiOhNNWa0sFott2mxkRLHOc2F4eBjA+t1DR0cHAKhDdV0uF3bu3FlysAJVjgGEqE5E+/rRo0ctLknlxAioUs9ODA0NVSWfpaWlkncx2ocXgfVAItIVzTBg8RKUEguNTUxMVKPYbYMBhGgD2mGdmUxG915UntpKNH8YqHgyOpfLIRqNQpIkXcUn7kZEcBFDUgHA5/MBWK8QtVOGWD2MVzw4WCqAlCpfMBiEzWYz9GDhRp3nAHDmzBkA69dYXDuRXg4xBLi3t7fsn21nDCBEGxDNIuL/2vcOh0P3b/7+ALB79264XC44HA50dXUhGo3qtp87dw6SJKG7uxuyLKOvrw+SJGF+fh7T09MA1p9ruHTpEtxud3VP0KS9e/cCAO7fv1/Wz2WzWXi9XkPBb6POc+DR8yKLi4tYWlqCzWbDlStXsLi4uOHPlCLOQ5wXGWNTSt3LtTiumd3arP58RRt7M/x+mV0TXdwNnT17tuw8XS5XwRBpK01OTsLhcJR9LjabDXNzcxgZGalRyRob70CIyBSPx4OlpSVds5sRyWQS4+PjNSpV+VKpFFKpFDwej9VFaToMIBXKn9qCKL/fpFXZ7XZEIhFcuHDB8GSJiUQCO3bsqMoIrWpYWVnB7OwsIpGI2glPxm21ugDNbmpqSn0CuZlsNIwxEAhg165d2L9/P/+oTMjvN2mGZiyznE4notGoOrHiZsz0T9SSLMuYnp4u+pwJbY53IBW6fPmy1UUwRXk8w6mQzWbVoY6HDh1COBzmOgkmlRo62qrsdrupfpBGcPbsWQaPCjCAtDHtH472TqOnp0ed2prrJBBRKQwgZcrlcojFYuoU3qUmXxNj9sV+iURCTd9sOnBB/Hw4HEYmkylodiqVB1D5cwJOpxNnzpyBLMsFCxpZfW5E1CDqO/lv4zA73bckSYrX61Wy2ayiKIoyPz9fMEX02tqaIkmSMj8/ryjKo+mk8XhKayPTgSuKogQCAXWq7Gw2q05bbSQPRTE+3Xd+2bWy2WxBuRrh3IzgdP3GmZnOnR5Bm0/n3rZ/YWYqGLEGgljPQVHWK1ntsURQ0QKgVujFKu38NAC6tQ3E2hFG8zBqowBSbHuznBsDiHEMIOa1ewDhKKwyvPvuuwDWp3EAUHSUkliRLb9Z5vz584bXcPB6vejo6MD8/DyOHDkCp9Op65CtRh5mNNu5HT9+vKz929GtW7cA8FpR+dgHUgajw3XFgkHKBpO5beYnP/kJJEnC8PAwHA6H+tRvNfPYjOg8104w1yrnRkSV4x1IDa2srOjuVsqxa9cuxONxpFIpzM7Oqgvd5A+XrCSPzdy5cwcACta9rjTfep6bWBuCSjM7lQlxWnjegZQhFAoBwKZP3Yr9otGo+i1eO5OqETabDblcDj09Pbh8+TKWl5d1q6VVI4+NZDIZvPXWW5AkSffwVyucGxFVST07XBqJmU5WMaJIkiR1FJEYIQTNSCPRKZz/SqfTum1iJJe2I150LuNxp7HIJ51OK4FAQC3LRnkoirFRWNp8RVkURVFHVEmSpOvsbpRzM4Kd6MaxE908tHknOu9AytDV1YV0Oo3Ozk7s3LkTPp8P3/nOdwqm33Y6nUin02rfgdfrRTqdRldXV1nTgZ8+fRoLCwuw2WxYWFjQNfFslIcRNptNl6/D4VBXbbtx4wbGx8cRj8cLntJthnMjovrgdO7tefotj5+vcewDMY/TuRMREZnAAEJEFWnmAQ7BYJBzvVWAAYSoBnK5XE2HeNb6+EZlMhlMTU3p1nkX86HZbDb4fL6qzOgcDoeLnq8sy3C5XHC5XOrzQ+Xsc+jQIc46XQEGEKIayJ+AstmOb0Qul4PH48GpU6fU53XC4TCcTifi8TgURUF/fz88Ho/hBaeKSaVSGBsbK0iPxWIIh8OIRqOIRqN49913EQ6Hy9qnp6cH4+PjnHXaJD5ISFRluVyuoCJrpuMbJRaR0q4uODY2hvn5efX90NAQhoeHAcDUGui5XA5vv/12Qfrq6iqGh4dx8+ZNdTohr9eLPXv2oLe3Fz09PYb2AYC+vj50dnYiEok07bomVuEdCJGGdrp+7XTzgkjXNqfkpwUCAbWpRKRnMhm1KQVYb5Lx+Xy6JQHMHh+ofAr/cmQyGfj9/oJZCkKhkDqXmVZnZ6epfCKRCE6fPl2Q/sEHHwAAnn/+eTXtueeeAwDcvn3b8D7C4OAg/H4/m7LKxABCpOF2u/H555+rKzbKsqxr3tCu4iik02nde+2Ej8rjObw6OjrUNvhkMonR0VFks1kAQHd3txpEzB6/3sQEjC+++KIufXR0VHenIc7L6/WWnUcikcAPfvCDoisGLi0tAYDu2SCxnwiuRvYRxHmI8yKDrHqC0Wp8Urm1mfl8xawC2qfvb968qQBQ1yZRFONT1m+2j6I8euofgO5JfLPHN8vMk+j5a7hstF8567gIa2trSigUUt8bPX9tupF9BDFjgvZzMAJ8Ep2IAGBhYQGAfqnf3bt3A0DRZplqEO3w2rnAmsH58+c33SeRSODYsWPqOZbjF7/4BUZHR80UzRTRR9Jsn4PVGECIHis2Xb+oWEoNEaXStm/fbip4yLKMw4cPb7iPdthwPtFcZmQfqgwDCNFjosIp1pFa6wqn1Sq0WCymG51VDpfLhZ07d5YcUAAU/6xWV1cBAK+88orhfagyDCBEj4n5jD7++GM1TXSeDw4O1iRP0cl89OjRmhy/VgKBAACUfHZiaGjI9LGVDRYSE/8Xdyjaz+r+/fu6bUb2yaddPI02xwBC9NiRI0cgSRIuXLigfmu9fv06vF6vbk0UcbcgKv9kMqlu8/l8APTffvOn+YjFYgAeVb7RaBSSJOmaW8wev57DeMWDg6UCSKmyBINB2Gy2ih4sBB6NrAqFQrhy5QpyuRxyuRyuXLmCUCikjroyso8g7kx6e3srKle7YQAhesxutyMSiUCSJHR0dKjNJW+++aZuv3PnzkGSJHR3d0OWZfT19RVM6S+G2l66dAlut1v387t374bL5YLD4UBXVxei0WhVj18Pe/fuBbD+jd6obDYLr9dblUA3OjqKo0ePwuFwwO12Y3BwsKDj3cg+wPp5iPMiYzide3uefstrxM9XBKRGKhNgfjp3cedj5ultl8tl6sn0WpmcnITD4Sj7XDidOxGRCR6PB0tLS7omNiOSySTGx8drVKrypVIppFIpeDweq4vSdBhAiOpAOxKoVabLEE1+Fy5cMNynkUgksGPHDtMjtKptZWUFs7OziEQi6pBtMo4BhKgOtMv5av/f7JxOJ6LRKG7cuGFo/4GBAbUDvhHIsozp6emi06XQ5jgbL1EdNFq/RzXZ7famncW2WcvdKHgHQkREpjCAEBGRKQwgRERkCgMIERGZwgBCRESmtO0orO3btwOAbqZPaj38fI2r1ZonrU7UJe2obacy+fLLLxGPx/Hw4UOri0IN6uLFiwCA119/3eKSUKPasmULXC4Xtm5tz+/ibRtAiDZjdo4oonbBPhAiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhIiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAhIiJTGECIiMgUBhAiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhIiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAhIiJTGECIiMgUBhAiIjKFAYSIiExhACEiIlMYQIiIyBQGECIiMoUBhIiITGEAISIiUxhAiIjIlK1WF4CoUaTTaTx8+FB9/3//938AgI8//lhN27JlC3bu3Fn3shE1IpuiKIrVhSCy2vvvv49/+qd/MrTvv//7v2PPnj01LhFR42MAIQKQzWbx9a9/3dC+f/rTn+BwOGpcIqLGxz4QIgAOhwMulwtbt5Zu1d26dStcLheDB9FjDCBEj7ndbl0fSL6HDx/C7XbXsUREjY1NWESP/e1vf8MzzzyDv/zlL0W3b9++HZ999hmefvrpOpeMqDHxDoTosaeffho//vGPsW3btoJt27Ztw49//GMGDyINBhAijRMnTuDBgwcF6Q8ePMCJEycsKBFR42ITFpHGl19+CafTiT/96U+69K9//evIZDIbdrITtRvegRBpbN26FSMjI3jyySfVtCeffBIjIyMMHkR5GECI8gwNDeGLL75Q33/xxRcYGhqysEREjYlNWER5FEXBt771Ldy/fx8A8Pzzz+P3v/89bDabxSUjaiy8AyHKY7PZcPLkSWzbtg3btm3DyZMnGTyIiuAdCFERv/71r/Hd734XAPAf//Ef+Md//EeLS0TUeFqyV1CWZUSjUauLQS1iZmbG6iJQk3O73ZAkyepiVF1LNmHFYjEsLCxYXQxqAgsLC1hdXS267cCBAxgYGKhziRrT6uoq/6ZMWlhYQCwWs7oYNdGSTVjiga+5uTmLS0KNzmazYW5uDiMjI1YXpaFdvXoVJ06cQAtWFzXXyvVRS96BEBFR7TGAEBGRKQwgRERkCgMIERGZwgBCRESmMIAQVcHk5CQmJyetLkbDymQyCAaDVhfDlGAwiFwuZ3UxGhIDCFELyOVyDTvdSiaTwdTUlO5BulgsBpfLBZvNBp/Ph0wmU3E+4XC46DWQZRkulwsulwuyLBf92Y32OXToENxud1XK2GoYQIiqYGZmxtIn1t977z3L8t5ILpeDx+PBqVOnsGvXLgCPKnqn04l4PA5FUdDf3w+Px4NUKmU6n1QqhbGxsYL0WCyGcDiMaDSKaDSKd999F+FwuKx9enp6MD4+Do/HwzuRPC05lQlRO8nlcgWVYqOIRCLo6elBX1+fmjY2Nob5+Xn1/dDQEIaHhwEA8Xi87DxyuRzefvvtgvTV1VUMDw/j5s2bsNvtAACv14s9e/agt7cXPT09hvYBgL6+PnR2diISieDs2bNll7FV8Q6EqEKZTEZtkin2XpZl2Gw2uFwuddqUTCajNpsA680vPp8PKysr6rFtNpv6KpUWCATUZhdtutX9MplMBn6/HwcOHNClh0IhXL16tWD/zs5OU/lEIhGcPn26IP2DDz4A8Gg6fuG5554DANy+fdvwPsLg4CD8fj+bsjQYQIgq5PF4MDw8rFbi2vfJZBKSJCGdTkOWZfz0pz8FAHR0dKjt7clkEqOjo8hmswCA7u5uNYisra0V5JdOp3XvtU1niqI0zHQjt27dAgC8+OKLuvTR0VHdnYY4V6/XW3YeiUQCP/jBD+B0Ogu2LS0tAQC6urrUNLGf+KyM7COI8xDnRQwgRBXLb3bRvhdNN6KCmp2dBQBdJS/2sdvtaiUqKq9iFaO2stuI1f0y4hv8ZuWNRqNYXl5Wm4uMymQy+Oijj3TNY1riWhcjrq+RfQTRxKW9Q2x3DCBEDURUon6/3+KSVO78+fOb7pNIJHDs2LGygwcA/OIXv8Do6KiZopkiAkgrfDbVwgBCRJbZvn27qeAhyzIOHz684T4brb8h7vSM7EOlMYAQNaB2qLxisVjJ5qfNuFwu7Ny5s+QgA2A9OGg7vcUghldeecXwPlQaAwhRAxHt60ePHrW4JJULBAIAUPLZiaGhIdPHFoMFtC/tNgDqHcrHH3+sbrt//75um5F98k1MTJgud6thACGqkPbbayaT0b0Xlae2Es0fBipWq8vlcohGo5AkSde0Iu5GRHBJJpPqNp/PB0D/TVpMGWL1MF7x4GCpAFKqfMFgEDabraIHC4FHnfehUAhXrlxBLpdDLpfDlStXEAqF1I59I/sI4s6kt7e3onK1EgYQogp1dHTo/q9973A4dP/m7w8Au3fvhsvlgsPhQFdXF6LRqG77uXPnIEkSuru7Icsy+vr6IEkS5ufnMT09DWB9KO+lS5fgdrure4Im7d27F8D6N3qjstksvF5vVYLf6Ogojh49CofDAbfbjcHBwYKOdyP7AOvnIc6LuKQttTkrl7QVbfXN8CdodklbcTdk5ultl8tl6sn0WpmcnITD4Sj7XFq5PuIdCBHVjMfjwdLSkq7ZzYhkMonx8fEalap8qVQKqVQKHo/H6qI0FAYQIgvk95u0KrvdjkgkggsXLhju00gkEtixY4fpEVrVtrKygtnZWUQiEfVZEHqEAWQD+XMaEVVLfr9JK3M6nYhGo7hx44ah/QcGBtQO+EYgyzKmp6eLzgrQ7jgb7wampqY2nOqg0eVyOfzmN7/Br3/9a8iybKo9eaM1JgKBAHbt2oX9+/fzm1mZmqHfo5rsdnvTzmLbrOWuB96BbODy5ctWF6EigUAA//Zv/4axsbGSC+lsRlEU3YR+2WxWHXd/6NAhhMNhLrZD1KYYQFpYtSbT0966a+80enp6EIlEAICL7RC1IQYQjVwuh1gspq7dUGrWTfGwltgvkUio6ZutAyGInw+Hw8hkMgVNRaXyqLZKHzZzOp04c+YMZFkuWBWvla4TERViANFwu91YWlpCNptFPB7Hr371q4J9MpkMPB4POjs7oSgKzpw5g4MHD6pD/DZbBwJ4VCkODg5CURQcP34cly5dMpxHI/re974HAHj33XfVNF4nojagtKCRkRFlZGSkrJ+Jx+MKAOXevXtqWjabVQAo2ss0Pz+v5F82AMrExIT6/2LbtWkAlLW1NfX92tpaWXmUq1iZqn2MZr1OAJS5uTnD+7erubm5in+H2pWZ+qhZtORvhJkPzOv1Fv0Dya/UJElS0/JfxfYvlibymp+fV7LZbEGem+VRLisCSLNcp1I/zxdf1Xy1agDhMN7HjA7XFaOZlAqGYf7kJz/Bf/3Xf2F4eBjAo9FS2qGC1cijnkTnuXaW0ma6Tq+//jp++MMfVnSMVvf+++/j4sWLuHbtmtVFaToXL160ugg1wwBi0srKiumHnXbt2oV4PI5UKoXZ2Vl1hbP88eaV5FFPd+7cAQAcOHCgYFszXKe9e/dicHDQ9M+3gwcPHgAAr5MJ77zzjtVFqBl2oj8WCoUAYNMOWLFfNBpVv3lrp9A2wmazIZfLoaenB5cvX8by8rJumcxq5FEvmUwGb731FiRJwsDAgJrO60TUBqxtQasNM30g6XRaAaBIkqSk02lFURRlcXFRbcP0er2Koqx35Oa/0um0bptos9d2xIsOYeBRR6/IJ51OK4FAQC3LRnmUS5t/sX6EiYmJTTudSx1jeXlZkSRJkSRJ19ndTNcJYCe6EexEN6+VO9F5B/JYV1cX0uk0Ojs7sXPnTvh8PnznO98pWHfB6XQinU6r7f1erxfpdBpdXV1lrQNx+vRpLCwswGazYWFhQdcss1Ee5bDZbLr8HQ7HhlOTlHMMm82GGzduYHx8HPF4vGCeoGa6TkRkDtcDobZm5XogzcTseiDU2vUR70CIiMgUBhAiqrlmHtwQDAY5z1sJDCBNRvQ/bPaixpfL5Wr6WdX6+EZlMhlMTU1BkiQ1TcyFZrPZ4PP5TM3mvLq6Cp/Ppx5js3nQUqkUwuGwmq+WLMtwuVxwuVwFM1cfOnSIM06XwADSZJTHU6lv9qLGlz/5ZLMd34hcLgePx4NTp06pz+qEw2E4nU7E43EoioL+/n54PJ6y5jDL5XJIpVK4fPkystks+vv7cfDgwZLLFgSDQUxOTuLZZ5/Fz3/+c93fSCwWQzgcRjQaRTQaxbvvvotwOKxu7+npwfj4OGecLoIBhMgCuVxOV0k12/GNikQi6Onp0S1POzY2pvs2PzQ0BFmWy5oV+r333lPvaOx2O4aGhgCg6OqhPp8P2WwW0WgUkiTpRumtrq5ieHgY4+PjsNvtsNvt8Hq9GBsb0wW0vr4+dHZ2qssX0CMMIERl0k77r51qXijWlJifFggE1G/LIj2TyahNKcCjb+qieUa7tIDZ4wOVT99fjkwmA7/fXzBDQSgUwtWrVwv27+zsNHxsbXOYltfr1b0X5zozM1N01cwPPvgAAPD888+rac899xwA4Pbt27p9BwcH4ff72ZSlwQBCVCa3243PP/9cXa1RlmVd84Z2BUchnU7r3msX+hLNjh0dHWobfDKZxOjoKLLZLACgu7tbDSJmj19vt27dAgC8+OKLuvTR0VHd8srivPIr/3KIa3/06FE1LZVK4fz58zh69KgajPPXjFlaWgIA3V2JeKYpvzlMnIc4L2IAISpLIpGALMv40Y9+BOBRZTM+Pg5ZlnH9+nU1LZ+Rhxu1lbxo8hFNKsB6hWb2+ED1Vqk0QnyD36xs0WgUy8vL6OnpMZ3XnTt3IEkS9u/fr6bduHFDzV8E487OThw8eBDJZBLAxpOo5gcQcQdTaqG5dsQAQlSGhYUFAPpKfPfu3QBQtFmmGkTFqp0HrBmcP39+030SiQSOHTtWUfAAgLfeekvtxxDE9RLH1gbjK1eulJ2HOHazfQ61xABCVIZi31hFxVJqBBCVtn379oqDRywWgyRJuo76UkRe4nMs1ZcCVNak1i4YQIjKICqcYh2pta5wWq1Ci8Vihir9jaRSKdy9exejo6MF28T1Kjb0VnyOxT7P1dVVAMArr7xSUdnaAQMIURnEnFkff/yxmiYqqFqtlSHa3LUdxM0gEAgAKF6BA1CH3pqVyWRw48YNXZ9OKpWCz+cDsP55fPrpp+p2URbxOR4+fBiA/vO8f/++bls+7cJp7Y4BhKgMR44cgSRJuHDhgvqt9fr16/B6vbr1UMS3X1H5i05bAGoFp/32mz/NRywWA/CowhPPL2ibW8wev57DeMWDg6UCSKmyBINB2Gy2DR8szGQyCqusngAAIABJREFU8Hg88Pv9uiHMe/bsUQPtwMAAJiYmMDk5qX5W165dgyRJavDq6upCKBTClStXkMvlkMvlcOXKFYRCoYLOf3Fn0tvbW+aVaF0MIERlsNvtiEQikCQJHR0d6vMVb775pm6/c+fOQZIkdHd3Q5Zl9PX1FSwNIL45X7p0CW63W/fzu3fvhsvlgsPhQFdXF6LRaFWPXw979+4FsP6N3qhsNguv17thoJuamirZ59Td3a3+f2ZmpuCzyr+Wo6OjOHr0KBwOB9xuNwYHB4s2iYnzEOdFnM6d2lyjTecuKrlG+7M0O527uPPJX4bYCJfLpXtexGqTk5NwOBxln0sr10e8AyGimvF4PFhaWtI1sRmRTCYxPj5eo1KVL5VKIZVKwePxWF2UhsIAQtQgtCOBWmW6DNHkd+HCBcOTJSYSCezYsaPiEVrVsrKygtnZWUQikaLTobQzBhCiBqFdylf7/2bndDoRjUbVJ8M3MzAwoHbANwJZljE9PV10BoB2t9XqAhDRI43W71FNdrvdVD9II2jWctcD70CIiMgUBhAiIjKFAYSIiExhACEiIlNathN9YWEBr776qtXFoCZw69YtbNu2zepiNDSxiJKYzp6MW1hYqNk8aVZryQDywgsv4MGDBzh+/LjVRaEmcPHiRVy8eNHqYjQF/k2Z88ILL1hdhJpoyalMiKqhlaegIKoG9oEQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSkMIEREZAoDCBERmcIAQkREpjCAEBGRKQwgRERkCgMIERGZwgBCRESmMIAQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSkMIEREZAoDCBERmcIAQkREpjCAEBGRKQwgRERkCgMIERGZwgBCRESmMIAQEZEpDCBERGQKAwgREZnCAEJERKYwgBARkSkMIEREZAoDCBERmcIAQkREpjCAEBGRKVutLgBRI/jzn/+My5cv4+HDh2rahx9+CAD42c9+pqZt2bIFp0+fxlNPPVX3MhI1GpuiKIrVhSCy2i9/+Uvs378fAEoGh7///e8AgFu3bqG3t7duZSNqVAwgRAAePnyIjo4OfPbZZxvu98wzz2BtbQ1btmypU8mIGhf7QIjwqGnqtddew5NPPllynyeffBKvvfYagwfRYwwgRI+NjIzgiy++KLn9iy++wMjISB1LRNTY2IRFpNHV1YXf/e53Rbd9+9vfxurqap1LRNS4eAdCpHHy5Els27atIH3btm04efKkBSUialy8AyHS+PDDD/Hyyy8X3Xb37l289NJLdS4RUePiHQiRxksvvYSXX34ZNptNTbPZbHj55ZcZPIjyMIAQ5Tl58iS2bl1/xnbr1q1sviIqgk1YRHnS6TReeOEFiD8Nm82GTz75BDt37rS4ZESNhXcgRHl27tyJ3t5ePPHEE3jiiSfQ29vL4EFUBAMIURGnTp3CV199ha+++gqnTp2yujhEDYlNWERF/PGPf8Q3v/lNAMAf/vAHfOMb37C4REQNSGlBb7zxhgKAL7744qshXm+88YbV1WJNtOR07p988gm2bduGubk5q4tCDe748eN4/fXX8cMf/rBg21//+lfYbDY8/fTTFpSssbz//vu4ePEirl27ZnVRms6JEyfwySefWF2MmmjJAAIAg4ODGBwctLoY1AT27t3L35VNPHjwAAB4nUx45513rC5CzbATnYiITGEAISIiUxhAiIjIFAYQIiIyhQGEiIhMYQAhqoLJyUlMTk5aXYyGlclkEAwGrS6GKcFgELlczupiNCQGEKIWkMvldFPQN5JMJoOpqSlIkqSmxWIxuFwu2Gw2+Hw+ZDKZso+7uroKn8+nHiORSGy4fyqVQjgcVvPVkmUZLpcLLpcLsizrth06dAhut9tUGVsdAwhRFczMzGBmZsay/N977z3L8t5ILpeDx+PBqVOnsGvXLgBAOByG0+lEPB6Hoijo7++Hx+NBKpUq67ipVAqXL19GNptFf38/Dh48WFD5C8FgEJOTk3j22Wfx85//XJ1pGXgUzMLhMKLRKKLRKN59912Ew2F1e09PD8bHx+HxeHgnkocBhKjJ5XI5XYXXSCKRCHp6etDX16emjY2N6b7NDw0NQZblspoA33vvPfWOxm63Y2hoCADgcrkK9vX5fMhms4hGo5AkCV1dXeq21dVVDA8PY3x8HHa7HXa7HV6vF2NjY7qA1tfXh87OTkQiEeMn3wYYQIgqlMlk1CaZYu9lWYbNZoPL5cLq6qq6j2g2AR59KxdNMSsrK+qxbTab+iqVFggE1G/e2nSr+2UymQz8fj8OHDigSw+FQrh69WrB/p2dnYaPrW0O0/J6vbr34vxnZmZgt9sL9v/ggw8AAM8//7ya9txzzwEAbt++rdt3cHAQfr+fTVkaDCBEFfJ4PBgeHlYrce37ZDIJSZKQTqchyzJ++tOfAgA6OjrU9vZkMonR0VFks1kAQHd3txpE1tbWCvJLp9O699qmM0VRdM0zVrp16xYA4MUXX9Slj46OIh6Pq+/FueZX/uUQTUtHjx5V01KpFM6fP4+jR4+qAdrlcun6SpaWlgBAd1fidDoBoKA5TJyHOC9iACGqmLYyzH8vmm5EBTU7OwsAukpe7COaT4D1yktUZlraym4jVvfLiG/wm5U3Go1ieXkZPT09pvO6c+cOJEnC/v371bQbN26o+YsA3dnZiYMHDyKZTAJY/zyKyQ8g4g5Ge4fY7hhAiBqIqET9fr/FJanc+fPnN90nkUjg2LFjFQUPAHjrrbfUfgxBXENxbG2AvnLlStl5iGO3wmdTLQwgRGSZ7du3Vxw8YrEYJEnSddSXIvISdx6l+lKAyprU2gUDCFEDaofKKxaLGar0N5JKpXD37l2Mjo4WbBPXsNjQWxE4xL/ajnEx0OGVV16pqGztgAGEqIGI9nVtZ3CzCgQCAIpX4ADUobdmZTIZ3LhxQ9fPk0ql4PP5AKyvXfLpp5+q20VZRkZGAACHDx8GAHz88cfqPvfv39dtyzcxMVFRuVsJAwhRhbTfXjOZjO69qLC0lWj+MNBYLKbuI55V0DatiG/SIriIDmAAamWp/SYtpgyxehiveHCwVAApVb5gMAibzbbhg4WZTAYejwd+v183rHnPnj1q8B0YGMDExAQmJyfVa37t2jVIkqQGr66uLoRCIVy5cgW5XA65XA5XrlxBKBQq6PwXdya9vb1lXonWxQBCVKGOjg7d/7XvHQ6H7t/8/QFg9+7dcLlccDgc6OrqQjQa1W0/d+4cJElCd3c3ZFlGX18fJEnC/Pw8pqenAawP5b106RLcbnd1T9CkvXv3Alj/Rm9UNpuF1+vdMPhNTU2VfOq8u7tb/f/MzAwkSUJHR4f6fEz+9R0dHcXRo0fhcDjgdrsxODhYtElMnIc4LwJsSqMMGq+iEydOAADXRKdN2Ww2zM3NqU0a9c4bQMM8t7GRq1ev4sSJE2WXVdwNnT17tuw8XS5XwRBpK01OTsLhcJR9Lq1cH/EOhIhqxuPxYGlpSdfsZkQymcT4+HiNSlW+VCqFVCoFj8djdVEaCgPIBvKnpCCqlvx+k1Zlt9sRiURw4cIFw5MlJhIJ7Nixo+IRWtWysrKC2dlZRCKRotOhtDMGkA1MTU3ppqhoNuVOd12MtoMy/xUMBiHLMmcoNSG/36SVOZ1ORKNR9cnwzQwMDKgd8I1AlmVMT08XnRWg3TGAbODy5ctWF8G0cqe7LkVRFN18TNlsVp1v6dChQwiHw1wrwQRxDRtp7qpastvtpvpBGsHZs2cZPEpgAGlR5Ux3vRntH4/2Fr6np0ed3pprJRC1HwYQjVwuh1gsps7aWWrSNDHWPn92TyPTeAvi58PhMDKZTMEKaaXyMKqc6a4reVbA6XTizJkzkGW5YFGjZrhORGQeA4iG2+3G0tISstks4vE4fvWrXxXsIx5g6uzshKIoOHPmDA4ePKiO0NhsGm/gUaU4ODgIRVFw/PhxXLp0yXAeZhWb7rpavve97wEA3n33XTWtWa8TEZVBaUEjIyPKyMhIWT8Tj8cVAMq9e/fUtGw2qwBQtJdpfn5eyb9sAJSJiQn1/8W2a9MAKGtra+r7tbW1svIwY3FxUZEkSclms6Z+vth5bbS9Wa4TAGVubs7w/u1qbm5uw8+fSjNTHzWLrXWJUk1AfHvWjv4oNmRPrKSW35Ry/vx5w2sveL1edHR0YH5+HkeOHIHT6dR1pFYjj3zFpruupWa6Trdu3cK2bdsM79+OxCJKCwsLFpek+ayurhpew6XpWB3BasFMxEeJb9j56aX222h7ftq9e/cUSZLU9EAgYKgsZs3PzyuhUKiiY2xUJnGnpv3m3yzXSRyDL75q+WrVOxD2gZhUyapku3btQjwex/LyMrxeL/x+vzrlQ7XyEDaa7rpa7ty5AwAFa18DzXGd5ubmCobV8qV/iWk4rC5HM76smCanXhhAHguFQgCwaQes2C8ajaod09oZUI2w2WzI5f5/e3cX2saV9gH8P9uklPRCarrIm7hVWt5SE8hWSy4S03YpMYFQL6OWEhd/VAktkpEuUlrii41wCMEh2wuZluQiRhItRtgW61y0GtLcJAaXNlUCS63dZaFhG1Zuk10Pm62G7pa2aTvvhXMmM/qwpbHkkeT/D0yj0WjmaFyfZ+Z8PEeDz+fDuXPnsLCwYFnlrB7nEJ9ZKd11PaiqinfeeQeyLKOnp8fY3krXiYhs0tuQnSasfD6vA9BlWdbz+byu68sdz7j7CBoOh3Vdv9eRW/yTz+ct74nOanNHvOgQBpabe8R58vm8pXlmpXNUa2lpydL8Y/7JZDLGfqOjo6t2Opu/g7kTfmFhQZdlWZdl2dLZ3UrXCWAnejXYiW5fO3ei8wnkLq/Xi3w+j87OTuzYsQORSAS7du0qSZvt8XiQz+eNRWXC4TDy+Ty8Xm9NabyPHDmC2dlZSJKE2dlZyyzdlc5RrWrTXa9GkiTLd3C73UYqk0uXLiEajSKTyZTM1G2V60RE9jGdO21oTqZzbyV207lTe9dHfAIhIiJbGECIiMgWBpAWs1J6dfMPUTNp5dFx4+PjTBRaAQNIi9GrHHtOzU/TtIYG+0Yfv1qqquLEiROWBJ8imaZYq8bOcgCapiGbzSKRSFSdZTqRSJS9JoqiwO/3w+/3lww+2b9/P5csqIABhMghxdmLW+341dA0DcFgEIcPHzbSBCUSCXg8HmQyGei6jueeew7BYLDmJJixWAwXLlzA8PBwVevc5HI5DA8Pl2xPp9NIJBJIpVJIpVL48MMPkUgkjPd9Ph+i0SiXLCiDAYTIAZqmWSqpVjt+tZLJJHw+n2V52uHhYcvdfH9/PxRFqXlZgbGxsapznmmahvPnz5dsX1xcxMDAgJEnzuVyIRwOY3h42BLQuru70dnZaax/Q8sYQIhqZF43xrxWiVCuL6p4WywWM+6axXZVVY2mFOBec0skErGka7F7fGDt67/UQlVVjIyMlKS4icfjRiJMs87OzoaVJZlM4siRIyXbr1y5AgDYvn27sW3btm0AgGvXrln27evrw8jICJuyTBhAiGoUCATwzTffQNeXl/tVFMXSvGFeAljI5/OW1+Y7Z9Fv1dHRYbTBZ7NZhEIhFAoFAMuTP0UQsXv89SYy+D7xxBOW7aFQCJlMxngtvlfxYmf1Mjc3h2eeeabssrTz8/MAYJl8KvYrbhYT30N8L2IAIarJ3NwcFEXBCy+8AGC5solGo1AUBRcvXjS2Fatmdry5khdNPqJJBbhXodk9PlBbs89aiTv41cqWSqWwsLAAn89X9zKoqoovvvjC0oRmNjExUfGzxQFELIVQjySn7YIBhKgGYj0McyW+c+dOACjbLFMPomI1J5JsBadOnVp1n7m5ORw8eLAhwQMAPvjgg7plohYBpNV+D43EAEJUg3J3rKJiqWYkEFlt2bKlYcFDURQcOHBgxX3MQ4uLNapJrZ0wgBDVQFQ45TpSG13htFuFlk6nKzYt1YPf78eOHTsqDjoAyv8+FxcXAQC7d+9uWNnaBQMIUQ1E0sUbN24Y20TneV9fX0POKdrce3t7G3L8RonFYgBQce5Ef39/Q8+/0gRb8W/xhGL+fd66dcvyXjGR/ZkYQIhq8vzzz0OWZZw+fdq4a7148SLC4bBlQS3xtCAq/2w2a7wnFvQy3/0Wp/lIp9MAlivfVCoFWZYtzS12j7+ew3jFxMFKAaRSWcbHxyFJUlUTC83HtjPJz+v1Ih6PY3JyEpqmQdM0TE5OIh6Pl3T+iyeTPXv21HyedsUAQlQDl8uFZDIJWZbR0dFhNIW89dZblv2OHTsGWZbR1dUFRVHQ3d1dsraMGA119uxZBAIBy+d37twJv98Pt9sNr9eLVCpV1+Ovh7179wK4d0dfrUKhgHA4vGqgq7RWTa1CoRB6e3vhdrsRCATQ19dXtuNdfA/xvYjrgdAG12zrgYgKsNn+LO2uByKefMwLgVXL7/db5os47fjx43C73TV/l3auj/gEQkQNEwwGMT8/b2liq0Y2m0U0Gm1QqWqXy+WQy+UQDAadLkpTYQAhahLmkUDtki5DNPmdPn266mSJc3Nz2Lp1a0NHaNXi+vXrmJiYQDKZNIZs0zIGEKImYV4L3vzvVufxeJBKpXDp0qWq9u/p6TE64JuBoig4efJk2QwAG90mpwtARMuard+jnlwul61+kGbQquVeD3wCISIiWxhAiIjIFgYQIiKyhQGEiIhsadtO9Onpady5c8fpYlALOHPmDN5//32ni9HURBqPl19+2eGStJ7Z2dmmmahab205E11RlJLUD0S1+utf/woA2LVrl8MloVYXCARWTB3fqtoygBDVQzunoCCqB/aBEBGRLQwgRERkCwMIERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBARkS0MIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkCwMIERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBARkS0MIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkCwMIERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBARkS2Sruu604Ugctrf//53+Hw+PPbYY/jFL5bvq27fvg0AePjhhwEAP//8M/7xj3/giy++wK9+9SvHykrULDY5XQCiZvDTTz/h22+/xd/+9reS9/75z39aXmuaxgBCBDZhEQEAurq68NRTT0GSpIr7SJKEp556Cl1dXetYMqLmxQBCdNfhw4dx3333VXz/vvvuw+HDh9exRETNjX0gRHfdunULjzzyCCr9SUiShK+++grbt29f55IRNSc+gRDdtX37djz99NNGJ7rZL37xCzz99NMMHkQmDCBEJocOHSrbDyJJEg4dOuRAiYiaF5uwiEz+85//oKOjAz/++KNl+6ZNm7C0tIStW7c6VDKi5sMnECKTrVu34sCBA9i06d4I902bNuHAgQMMHkRFGECIigwODuLnn382Xv/8888YHBx0sEREzYlNWERF/ve//+GXv/wlvvvuOwDAAw88gH//+9948MEHHS4ZUXPhEwhRkQcffBAvvvgiNm/ejM2bN+PFF19k8CAqgwGEqIxXXnkFd+7cwZ07d/DKK684XRyiptSWubC+/PJLZLNZp4tBLeynn34y/v3NN99gdnbWwdJQq+vu7sajjz7qdDHqri37QF577TW89957TheDiAgA8Oqrr+Ldd991uhh115ZPIN9//z0GBwcxNTXldFGoyUmShKmpKY6yWsX09DSGhoYqpnmhyoaGhvD99987XYyGYB8IERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBDVwfHjx3H8+HGni9G0VFXF+Pi408WwZXx8HJqmOV2MpsQAQtQGNE0ruxBWM1BVFSdOnIAsy8a2dDoNv98PSZIQiUSgqmrNx9U0DdlsFolEAn6/v6rPJBKJstdJURT4/X74/X4oimJ5b//+/QgEArbK2O7aciIh0XobGxtz9PwfffSRo+evRNM0BINBRKNRPPnkkwCWK/H/+7//QyaTAbAcTILBIMbGxuDz+ao+diwWAwCcOnWqqv1zuRyGh4dLtqfTaUxPTyOVSgEAfv/73+Nf//oXQqEQAMDn8yEajSIYDCKVSsHlclVdxnbHJxCiFqdpGhKJhNPFKCuZTMLn86G7u9vYNjw8bLmb7+/vh6IoNTcBjo2NVR24NU3D+fPnS7YvLi5iYGAA0WgULpcLLpcL4XAYw8PDyOVyxn7d3d3o7OxEMpmsqYztjgGEaI1UVTWaZMq9VhQFkiTB7/djcXHR2Ec0mwD3mlYikQiuX79uHFuSJOOn0rZYLGY0u5i3O90vo6oqRkZGsG/fPsv2eDyO6enpkv07OzsbVpZkMokjR46UbL9y5QoAYPv27ca2bdu2AQCuXbtm2bevrw8jIyNsyjJhACFao2AwiIGBAaMSN7/OZrOQZRn5fB6KouAPf/gDAKCjo8Nob89mswiFQigUCgCArq4uI4gsLS2VnC+fz1tem+/CdV1vmnxVV69eBQA88cQTlu2hUMhovgJgfNdwONyQcszNzeGZZ56Bx+MpeW9+fh4A4PV6jW1iv+K+EPE9xPciBhCiNTNXhsWvRdONqKAmJiYAwFLJi31E8wlwr/IqV+mZK7uV1NLE0wjiDn618qZSKSwsLNTU/1EtVVXxxRdfWJrQzMTvo5ziACL6PsxPiBsdAwhRExGV6MjIiMMlWbtqOrfn5uZw8ODBhgQPAPjggw+MzvC1EgGkHX439cIAQkSO2bJlS8OCh6IoOHDgwIr7mIcWF2tUk1o7YQAhakIbofJKp9MVm5bqwe/3Y8eOHRUHIgD3Aoi5Y1wMdNi9e3fDytYuGECImohoX+/t7XW4JGsn5mlUmsXd39/f0POLAQXmH/N7AIwnlBs3bhjv3bp1y/JesdHR0UYVueUwgBCtkfnuVVVVy2tReZor0eJhoOl02tgnlUpBlmVL04p4GhHBJZvNGu9FIhEA1jtpkTLE6WG8YuJgpQBSqXzj4+OQJMkyD6MS87HtpBvxer2Ix+OYnJyEpmnQNA2Tk5OIx+Mlnf/iyWTPnj01n6ddMYAQrVFHR4fl3+bXbrfb8t/i/QFg586d8Pv9cLvd8Hq9xoxo4dixY5BlGV1dXVAUBd3d3ZBlGTMzMzh58iSAe0N5z549i0AgUN8vaNPevXsB3Lujr1ahUEA4HF41+EmSZLmubrfbVjqXUCiE3t5euN1uBAIB9PX1le14F99DfC9iKhOiNatm3sVK+/h8vpKhwGZer3fFocLiGMXncDq9isfjQSwWwyeffFK2r6NS+cT21fJb2ZnvUukzsiyverwLFy4gFouVHVq9UfEJhIgaJhgMYn5+3tLsVo1sNotoNNqgUtUul8shl8shGAw6XZSmwgCyguKUFET1Utxv0q5cLheSySROnz5dVZ8GsDw3ZOvWrQ0doVWL69evY2JiAslkkokUizCArODEiROWFBWtRlVVHD9+3BjCKDpra2EeAln8Mz4+DkVRuFaCDcX9Ju3M4/EglUrh0qVLVe3f09NjdMA3A0VRcPLkSTZdlcEAsoJz5845XQTbVFXFjRs3MDY2Bl3XMTMzg4GBgZoX9dF13ZKPqVAoGEMi9+/fj0QiwbUSbKg0vLRduVwuHD161Oli2HL06FEGjwoYQNrUjRs3LE0AYsy9nTQM5j8e8yO8z+cz0lsHg0E+iRBtMAwgJpqmIZ1OG6m3KyVNE2PtxX5zc3PG9tXSeAvi84lEAqqqlgw/rHSOahW3H4vKvXgS1FrnCng8HrzxxhtQFKVkUaNWuE5EtAZ6GxocHNQHBwdr/pwsy3o4HNYLhYKu67o+MzOjA9DNl2lpaUmXZVmfmZnRdV3XL1++rAPQFxYWdFmWjf0//fRTXdd1PZ/P6wD0cDhsHCMWi+n5fF7XdV0vFAr66Oho1eewI5/PG+f4/PPPLe+Njo7qo6Ojqx6j+DqYFQqFku/YKtcJgD41NVX1/hvV1NRUxd8/rcxufdQK2vL/CDu/sEwmU1LBiorR/IcjgooZAKMSLlfRFm8DoC8tLRmvl5aWajpHLUTFLH5isVjNxxDnX6kCadXrxABSHQYQ+9o5gEi63n49eENDQwCAqampqj8TiUQwMTFR0qEpmkzEdrEIUDm6rpfsX+4Y4lwzMzN4/vnnS4YGrnYOO3K5HM6fP49Tp04hHo/XnOK63Pda6f1WuU6SJGHv3r1Vr7GxUS0uLuLq1avo6+tzuigt5+rVq3j22Wdrqo9aBftA7lppYRkzUWHpKyRqW82bb74JWZYxMDAAt9tdMjKqHuco5vP5jBQXw8PDto9TTrn+lVa9TkRUg0Y/4jjBziMjKjTRFG8Xr4v7ElY6TqVjLyws6OFwuKRpabVzrEWlsqzlc6Lv4fLlyyX7N/t1ApuwqsImLPvauQmLTyB3xeNxAFh1tqzYL5VKGXfe5gyo1ZAkCZqmwefz4dy5c1hYWLAMr63HOcoRx5qZmVnTccxUVcU777wDWZbR09NjbG/l60REVXI6gjWCnYgvOptlWTZG/og7a5hGB4mO3OKffD5veU+M5DJ3xIsOYdzt6BXnyefzljvrlc5RLVmWy45iKu5grmYUlvk7iO+l67oxokqWZUtndytdJ/AJpCp8ArGPTyAbgNfrRT6fR2dnJ3bs2IFIJIJdu3aVpM32eDzI5/NGe384HEY+n4fX660pjfeRI0cwOzsLSZIwOztrmaW70jmqFQqFMDIyYqzIlkwm8bvf/a7mDK2VUmZLkoRLly4hGo0ik8mUzNRtletERPZxFBZtaJIkYWpqCoODg04XpalNT09jaGiIAxRsaOf6iE8gRERkCwMIEa27Zh3sMD4+zpxuNWAAaTErpVc3/1Dz0zStob+rRh/fLlVVceLECcu67yLJXzJdAAAWOElEQVQ3miRJiEQitrI7q6qKRCKx6vIFiqLA7/eXnYi6f/9+ZpeuAQNIi9HLTJor90PNrzj5ZKsd3w5N0xAMBnH48GFjzY9EIgGPx4NMJgNd1/Hcc88hGAxWvQCV+bgAjCUIpqenSxKFptNpJBIJpFIppFIpfPjhh0gkEsb7Pp8P0WiU2aWrxABC5ABN0ywVV6sd365kMgmfz2fJFj08PGy54+/v74eiKDVlib548SIURcHLL78MYHmE3tjYGE6dOmVkaF5cXMTAwACi0ShcLhdcLhfC4TCGh4ctwaq7uxudnZ3GUgVUGQMIUY3Maf/NqeaFck2JxdtisZjRfCK2q6pqNK8AMJpjIpGIZWkBu8cH1p6+fy1UVcXIyAj27dtn2R6PxzE9PV2yf2dnZ9XHFp8350t77LHHAACzs7MAgCtXrgAAtm/fbuyzbds2AMC1a9csx+vr68PIyAibslbBAEJUo0AggG+++cZoKlEUxdLkYV7BUcjn85bX5vk4otmxo6PDaJfPZrMIhUIoFAoAgK6uLiOI2D2+065evQoAeOKJJyzbQ6EQMpmM8Vp8z3A4XPWxyyXVFMFE5Lmbn58HAMs8ITF/qfjzooyizFQeAwhRDebm5qAoCl544QUAyxVQNBqFoii4ePGisa1YNZMbzZW8aOIRzSzAvUrO7vGB5cBS62TSehF3+auVNZVKYWFhAT6fr+pji2tUaRE4YOWEqcUBRASflY5HDCBENRHNIeZKfOfOnQBQthmmHkRFamc54mZy6tSpVfeZm5vDwYMHawoeAHD48GEAwNtvv208CYp+jVgsVmNJ7wWQVr/mjcYAQlSDcnexorKptDYJVW/Lli01Bw9g+Ynt8uXLuHnzJtxuNxKJBG7fvg1geWguAMuw4WK1NJfRPQwgRDUQlVC5ztVGV0LtXsml02nL6Kxa9fT0GEOBQ6EQPvvsM4yOjhoBqdzvbnFxEQCwe/fuNZR842IAIaqByJl148YNY5toMmnUan2iHb63t7chx18voimp0vyK/v7+up0rnU5jfn7e0gR14MABANbf3a1btyzvFTMvkkalGECIavD8889DlmWcPn3auJO9ePEiwuGwZT2U4k7dbDZrvBeJRABY74iL03qIWdSapiGVSkGWZUsTjN3jOzmMV0wcrBRAKpVtfHwckiStOrFQ0zTkcjlEIhHcvHkTmUzGMqzX6/UiHo9jcnISmqZB0zRMTk4iHo+XdOyLJ5M9e/bU9B03GgYQohq4XC4kk0nIsoyOjg5jfsVbb71l2e/YsWOQZRldXV1QFAXd3d0lSwOI0VBnz541lhsWdu7cCb/fD7fbDa/Xi1QqVdfjO2Hv3r0A7t31V6tQKCAcDq8Y+MSyA9euXUM4HLak/TcLhULo7e2F2+1GIBBAX18fQqFQyX6ijKLMVB7TudOG1mzp3EVAarY/y3qlcxdPQpUq+JX4/X7LfJFGOn78ONxut61yFmvn+ohPIES0boLBIObn5y1NbtXIZrOIRqMNKpVVLpdDLpczcmtRZQwgRE3CPDqoXVNoiCbA06dPV50scW5uDlu3bl3TCK1qXb9+HRMTE0gmk5b+EyqPAYSoSZiX8jX/u914PB6kUilcunSpqv17enqMDvhGUxQFJ0+eLDvbn0ptcroARLSs2fo9GsnlctWlf6HemrFMzYxPIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkS1vORH/ttdfw3nvvOV0MIiIAwKuvvop3333X6WLUXVsGkC+//LLmma5Exc6cOQMAeP311x0uCbW67u5uPProo04Xo+7aMoAQ1UM75zAiqgf2gRARkS0MIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkCwMIERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBARkS0MIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkCwMIERHZwgBCRES2MIAQEZEtDCBERGQLAwgREdnCAEJERLYwgBARkS0MIEREZAsDCBER2cIAQkREtjCAEBGRLQwgRERkCwMIERHZwgBCRES2bHK6AETNIp/P46effjJe//e//wUA3Lhxw9h23333YceOHeteNqJmJOm6rjtdCCKnffzxx/jtb39b1b6fffYZfvOb3zS4RETNjwGECEChUMBDDz1U1b5ff/013G53g0tE1PzYB0IEwO12w+/3Y9Omyq26mzZtgt/vZ/AguosBhOiuQCBg6QMp9tNPPyEQCKxjiYiaG5uwiO767rvv8PDDD+Pbb78t+/6WLVtw+/ZtPPDAA+tcMqLmxCcQorseeOABvPTSS9i8eXPJe5s3b8ZLL73E4EFkwgBCZDI0NIQ7d+6UbL9z5w6GhoYcKBFR82ITFpHJjz/+CI/Hg6+//tqy/aGHHoKqqit2shNtNHwCITLZtGkTBgcHcf/99xvb7r//fgwODjJ4EBVhACEq0t/fjx9++MF4/cMPP6C/v9/BEhE1JzZhERXRdR2PPPIIbt26BQDYvn07vvrqK0iS5HDJiJoLn0CIikiShEOHDmHz5s3YvHkzDh06xOBBVAafQIjK+Mtf/oKnnnoKAPDnP/8Zv/71rx0uEVHz2RC9goqiIJVKOV0MalFjY2NOF4FaTCAQgCzLThej4TZEE1Y6ncbs7KzTxaAWMDs7i8XFRQDAvn370NPT43CJmtPi4iL/piqYnZ1FOp12uhjrYkM0YYkJYFNTUw6XhJqdJEmYmprC4OCg00VpatPT0xgaGsIGqD5qtpHqmw3xBEJERPXHAEJERLYwgBARkS0MIEREZAsDCBER2cIAQtQAx48fx/Hjx50uRtNSVRXj4+NOF6PE+Pg4NE1zuhgtgwGEqA1pmta06VdUVcWJEycsE+3S6TT8fj8kSUIkEoGqqraOm0gkIEkSJEmqOBdDURT4/X74/X4oimJ5b//+/QgEArbOvxExgBA1wNjYmKMz2D/66CPHzr0STdMQDAZx+PBhPPnkkwCARCIBj8eDTCYDXdfx3HPPIRgMIpfL1XxcYDkZ5tLSEqanp0ueAtPpNBKJBFKpFFKpFD788EMkEgnjfZ/Ph2g0imAwyCeRKjCAELUZTdMslWIzSSaT8Pl86O7uNrYNDw9b7vj7+/uhKEpNTYAXL16Eoih4+eWXAQAejwdjY2M4deoU5ubmACzPnh8YGEA0GoXL5YLL5UI4HMbw8LAlWHV3d6OzsxPJZHKtX7ftMYAQ1ZmqqkaTTLnXiqJAkiT4/X4jbYqqqkbTCgCjKSYSieD69evGsUXzjLl5qnhbLBYzmmbM253ul1FVFSMjI9i3b59lezwex/T0dMn+nZ2dVR9bfN7lchnbHnvsMQAwUq5cuXIFwHJ6fmHbtm0AgGvXrlmO19fXh5GRETZlrUbfAAYHB/XBwUGni0EtAIA+NTW1pmPIsqwD0MWfl/n1p59+quu6rufzeR2AHg6HjfMW71MoFPRwOKwD0D///HNd13V9aWnJcmzzsczbil/ruq6Pjo7qo6Oja/puwtTUVMnxV5PJZHQAej6fX3G/zz//XAegLywsVH3sct+3eLu4luX2kWXZsk1c00wmU3UZhI1U3/AJhKjOMplMxdei6cbr9QIAJiYmAMCSU0rsI5pYABhPFB6Pp+R84lircbpfRtzlr1beVCqFhYUF+Hy+qo8trpP5aa2YuNblFHemiyeZlY5HbMIiamqiEh0ZGXG4JGt36tSpVfeZm5vDwYMHawoeAHD48GEAwNtvv210fot+jVgsVmNJ7wWQdrjujcQAQkRNY8uWLTUHD2D5qe3y5cu4efMm3G43EokEbt++DWB5aC6AFdfnEE8wVBsGEKIWsBEquHQ6bRmdVauenh5jKHAoFMJnn32G0dFRIyCJAGLuGBeDGHbv3r2Gkm9cDCBETUy0wff29jpckrUTTUmV5lf09/fX7VzpdBrz8/OWJqgDBw4AAG7cuGFsu3XrluW9YqOjo3UrUztiACGqM/Mdrqqqltei8jRXosVDRcUMak3TkEqlIMuypfmluMM4m80a70UiEQDWu22RMsTpYbxi4mClAFKpfOPj45AkadWJhZqmIZfLIRKJ4ObNm8hkMpZhvV6vF/F4HJOTk9A0DZqmYXJyEvF4vKRjXzyZ7Nmzp6bvuNEwgBDVWUdHh+Xf5tdut9vy3+L9AWDnzp3w+/1wu93wer1IpVKW948dOwZZltHV1QVFUdDd3Q1ZljEzM4OTJ08CuLeO+9mzZxEIBOr7BW3au3cvgHt3/dUqFAoIh8MrBj9JkuB2u3Ht2jWEw2EcPXq07H6hUAi9vb1wu90IBALo6+tDKBQq2U+UUZSZytvkdAGI2o1exTKvK+3j8/lKhgKbeb3eFYcKi2MUn8PJIbzA8hDkWCyGTz75pGxfR6Xyie1ikmU51VxzQZblVfe/cOECYrFY2WHTdA+fQIho3QSDQczPz1ua3aqRzWYRjUYbVCqrXC6HXC5n5NaiyhhAiJpAcb9Ju3K5XEgmkzh9+nTVyRLn5uawdevWNY3Qqtb169cxMTGBZDJp6T+h8hhAalCc04ioXor7TdqZx+NBKpXCpUuXqtq/p6fH6IBvNEVRcPLkSTZdVYkBpAYnTpzAwMBASdqDViUS9tXCnLiv+Gd8fByKojANtg26rlt+2p3L5arY0e2ko0ePMnjUgAGkBufOnXO6CHWTy+UwPDxc8+f0u2stCIVCwaj09u/fj0QiwQV5iDYIBpANSNM0nD9/3vbnzXdo5nZin89nrKHABXmI2h8DyAo0TUM6nTbWbqiUmVNM1hL7iQVsqlkHQhCfTyQSUFW1pGmp0jnsSCaTOHLkSNn31jrZzOPx4I033oCiKCWr4rXadSKiVTiRQ3692c3PL8uyHg6H9UKhoOu6rs/MzJSsO7C0tKTLsqzPzMzouq7rly9fNtYyqGYdCF3X9VgsZqyRUCgU9NHR0arPUavLly8bZSn+Lrpe/ZoR5T4rFAqFku/YKtcJdVgPZCOwsx7IRrGR1gPZEP8H2PmFisVvxEI+un6vYjT/4YigYgbAqITLVbTF2wDoS0tLxmuxaFC156jW0tKSHo/HK5ajFqt9tlWvEwNIdRhAKmMAaTN2fqErrV5m3m6+ey7+Kbd/uW3iXDMzM8bTjtlq56iWOXhUKlu1ag0grXKdKn2eP/yp5WejBBCmMqlgpdXLzMSQXn0NQy/ffPNN3Lx5EwMDAwCWs5aahzjW4xyKolTMOFpvovPcnMm0Va4TALz++ut49tln13SMdvfxxx/jzJkz+OMf/+h0UZrOmTNnnC7CumEAqZPr16/bnuz05JNPIpPJIJfLYWJiwkhBXTxOfi3nWGnyoyRJdZ178Kc//QkAsG/fvpL3mv06AcsJ9Pr6+mx/fiO4c+cOAPA6lfH+++87XYR1w1FYFcTjcQBYNd2C2C+VShl33uYU2tWQJAmapsHn8+HcuXNYWFiwrGNQj3PoRRPVzAGjnsFDVVW88847kGUZPT09xvZWuU5EVAOHms7WlZ0+EDEKSJZlY+SPGNUD3BsdJDpyi3/y+bzlPdFmb+6IFx3CwHJHrzhPPp/XY7GYUZaVzrEW4jhm1YzCMn8Hc1+EGFEly7Kls3u179BM1wlgJ3o12Ile2UbqROcTSAVerxf5fB6dnZ3YsWMHIpEIdu3aVbLugsfjQT6fN9r7w+Ew8vk8vF5vTetAHDlyBLOzs5AkCbOzs5ZmmZXOsd7EuguC2+02UplcunQJ0WgUmUymJB3ERrtORBuBpOvtn3hnaGgIADA1NeVwSajZSZKEqakpDA4OOl2UpjY9PY2hoaENkberVhupvuETCBER2cIAQkTrrlkHN4yPjzOHWw0YQFrcSunVzT/U/DRNa+jvqtHHr5aqqjhx4gRkWTa2iVxokiQhEonYyuasaRqy2SwSicSKw9YVRYHf74ff7y9ZmmH//v3MJl0DBpAWp5cZnlvuh5pfcfLJVjt+NTRNQzAYxOHDh425OolEAh6PB5lMBrqu47nnnkMwGKx6xUIhFovhwoULGB4errhmTzqdRiKRQCqVQiqVwocffohEImG87/P5EI1GmU26SgwgRE1A0zRLRdZqx69WMpmEz+ezLE87PDxsuePv7++Hoig1Z4UeGxvD2NhYxfcXFxcxMDCAaDQKl8sFl8uFcDiM4eFhS7Dq7u5GZ2ensTQBVcYAQrRG5rT/5lTzQrmmxOJtsVjMuGsW21VVNZpbgHsrSEYiEcvSAnaPD6w9fX8tVFXFyMhISYaCeDyO6enpkv07Ozvrev4rV64AALZv325s27ZtGwDg2rVrln37+vowMjLCpqxVMIAQrVEgEMA333xjrNaoKIqlCcS8gqOQz+ctr813zqLZsaOjw2inz2azCIVCKBQKAICuri4jiNg9/nq7evUqAOCJJ56wbA+FQshkMsZr8b3C4XBdzz8/Pw8AlnlBYr5ScZOXKKMoM5XHAEK0BnNzc1AUBS+88AKA5QopGo1CURRcvHjR2FasmsmN5kpeNPmIZhfgXqVn9/jA6s0+9STu8lcrWyqVwsLCAnw+X13Pv1KC1OIAIlbarLSIHC1jACFag9nZWQDWSnznzp0AULZZph5ExWrOA9YKTp06teo+c3NzOHjwYN2DR61EAGm1a7zeGECI1qDcXa2ofCqNBKLKtmzZ0rDgYR42XKzezWUbBQMI0RqISqlcZ2ujK6V2q/TS6bRldFa9lftdLS4uAgB2797dsPO2MwYQojUQObNu3LhhbBOd541aK0O0y/f29jbk+I0Si8UAoOL8iv7+/oaeXyyoZv5d3bp1y/JeMfOiaFSKAYRoDZ5//nnIsozTp08bd7YXL15EOBy2rIcinhZE5Z/NZo33IpEIAOsdcnGaj3Q6DWC58k2lUpBl2dIkY/f46zmMV0wcrBRAKpVlfHwckiRVNbHQfOzi83i9XsTjcUxOTkLTNGiahsnJScTj8ZKOffFksmfPnlXPuZExgBCtgcvlQjKZhCzL6OjoMOZXvPXWW5b9jh07BlmW0dXVBUVR0N3dXbI0gBgNdfbsWQQCAcvnd+7cCb/fD7fbDa/Xi1QqVdfjr4e9e/cCuHfXX61CoYBwOLxqoKu01IBZKBRCb28v3G43AoEA+vr6EAqFSo4lyijKTOUxnTuRSbOlcxcVYLP9mdpN5y6efIqXIa6G3++3zBdppOPHj8Ptdtsq50aqb/gEQkTrJhgMYn5+3tLEVo1sNotoNNqgUlnlcjnkcjkEg8F1OV8rYwAhalLm0ULtklJDNPmdPn266mSJc3Nz2Lp1a0NHaAnXr1/HxMQEksmkMRybKmMAIWpS5qV8zf9udR6PB6lUCpcuXapq/56eHqMDvtEURcHJkyfLzu6nUpucLgARldds/R715HK5bPUvNFozlqmZ8QmEiIhsYQAhIiJbGECIiMgWBhAiIrJlw3Siz87O4sUXX3S6GNQCrl69is2bNztdjKYmFloS6ezpntnZ2YblQWs2GyKAPP7447hz5w5efvllp4tCLeDMmTM4c+aM08VoCfybKu/xxx93ugjrYkOkMiEiovpjHwgREdnCAEJERLYwgBARkS0MIEREZMv/A5BEePYcoIkvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.utils.plot_model(model, show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b3f3cf6",
   "metadata": {},
   "source": [
    "In my previous notebook, I initialized the weights randomly in addition to initializing the bias as 0. In fact, the `Dense()` function does this automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "32506b4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "hdn_lyr1 = model.layers[1]\n",
    "weights, biases = hdn_lyr1.get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b72f6139",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.06273407,  0.04514893, -0.00901059, ...,  0.0638226 ,\n",
       "        -0.02724256,  0.01759659],\n",
       "       [ 0.05610733,  0.0282582 , -0.02378   , ...,  0.04329328,\n",
       "        -0.02276049,  0.06753001],\n",
       "       [-0.05292141, -0.01655687,  0.06412166, ..., -0.05307759,\n",
       "         0.06251279, -0.00323521],\n",
       "       ...,\n",
       "       [ 0.05283526,  0.0629485 , -0.04900521, ..., -0.01888835,\n",
       "         0.01201991, -0.06575525],\n",
       "       [ 0.06133854,  0.00502718,  0.01579804, ..., -0.04358329,\n",
       "        -0.06071779,  0.06265812],\n",
       "       [ 0.04947041,  0.04297568, -0.04578072, ..., -0.04002057,\n",
       "         0.03814631, -0.05621094]], dtype=float32)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4d4e22bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
       "       0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "biases"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aba43fe",
   "metadata": {},
   "source": [
    "The difference here is that in the previous notebook I didn't set the `input_shape` while constructing my model. The weight shape depends on the `input_shape` argument, that's why, If we don't set input_shape as above, the algorithm will wait until we compile the model to get the input_shape as well as weight_shape. Delete the input_shape argument above and notice that you will not be able to get the weights until you compile the model. Initialization is an important step to breaking symmetry and I will revise it again in the following notebooks. \n",
    "\n",
    "Documentation for initializers : [link](https://keras.io/initializers/.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "602fb819",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=keras.losses.sparse_categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.SGD(),# default learning_rate is 0.01\n",
    "              metrics=[keras.metrics.sparse_categorical_accuracy])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "316c9210",
   "metadata": {},
   "source": [
    "Here we will have sparse labels. For every instance, we will have an array that shows the probability of being one of the clothes which will look something like this [0, 0.3, 0, 0, 0, 0,2 ,0.9 ,0 ,0 ]. In other words, when we have more than one or two labels we can use ` loss=\"sparse_categorical_crossentropy\"`. If there was one target we could also use `categorical_crossentropy`. Documentation:\n",
    "1. Cost Functions: [link](https://keras.io/losses)\n",
    "2. Optimizers: [link](https://keras.io/optimizers)\n",
    "3. Metrics: [link](https://keras.io/metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1426481a",
   "metadata": {},
   "source": [
    "Here I will use three Callbacks. In fact, I was gonna write my own Callback but I felt a bit lazy for it today. Let's briefly explore the callbacks that I will be using.\n",
    "1. **Checkpoint:** This callback saves the model at the end of each epoch. Here I also used the `save_best_only()=True` argument which makes the function save the checkpoint when its performance on validation data is the best so far. Thanks to this callback, even if the model starts overfitting after some epoch we will still have the best model.\n",
    "2. **Early_stopping:** I won't give much detail about early_stopping since I also introduced this in my machine learning notebooks. `restore_best_weights` is similar to `save_best_only` It saves the weights from the epoch with best values during training. At the end of the training, the model will use these weights. I thought this parameter should be true by default but It is not, probably the reason is it may be costly for training sometimes when this parameter is on. A discussion about that can be found on GitHub [link](https://github.com/keras-team/keras/issues/11371)\n",
    "3. **Tensorboard:** Tensorboard is an awesome monitoring API in TensorFlow which is sort of similar to our history plot in which we see how the performance changed during the training. Tensorboard, however, provides us with much more information.  It has some other arguments that I really like such as `update_freq` which is batch by default and `write_images` which is also false by default. Check the arguments, the link is below.\n",
    "\n",
    "For other callbacks: [link](https://keras.io/api/callbacks/)\n",
    "\n",
    "For custom callbacks there is very good documentation here: [link](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/Callback)\n",
    "\n",
    "For more about Tensorboard : [link](https://www.tensorflow.org/tensorboard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93d8ac3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "checkpoint = keras.callbacks.ModelCheckpoint(\"my_mnist_model.h5\", save_best_only=True)\n",
    "early_stopping = keras.callbacks.EarlyStopping(patience=200,restore_best_weights=True)\n",
    "tensorboard = tf.keras.callbacks.TensorBoard(log_dir=\"./my_logs\")\n",
    "\n",
    "history = model.fit(X_train, y_train, epochs=400,batch_size=512,\n",
    "                    validation_data=(X_valid, y_valid),callbacks=[checkpoint, early_stopping,tensorboard])\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af432591",
   "metadata": {},
   "source": [
    "Here I directly passed the `validation_data` since we already split our dataset. However, we can also use the `validation_split` argument to use a ratio of our training data for validation. For instance, `validation_split=0.1` tells Keras to use the last 10% of the data for validation. Note: It doesn't shuffle the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "199bdfeb",
   "metadata": {},
   "source": [
    "Let's also evaluate model performance on the test set. Documentation for `evaluate()` method: [link](https://www.tensorflow.org/api_docs/python/tf/keras/Model#evaluate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0920df27",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 1s 2ms/step - loss: 0.3848 - sparse_categorical_accuracy: 0.8854\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.3848017752170563, 0.8853999972343445]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb2c2234",
   "metadata": {},
   "source": [
    "Let's look at how performance changed during the training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0b10a890",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ERROR: Timed out waiting for TensorBoard to start. It may still be running as pid 17620."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%load_ext tensorboard\n",
    "%tensorboard --logdir=./my_logs --port=6006"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e47fef02",
   "metadata": {},
   "source": [
    "Even if I have used tensorboard, let's also look at the classical performance plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "12414cd9",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAACuxUlEQVR4nOydd3gU1RqH39mW3SSb3hMIvfdeBAEVECwgKipiReWq2L3YsTfsDcWOFeyiyFW69N57T+9te5v7x2QmuymQSCAhzvs8PGR3p5wzOzu/85XzHUEURVRUVFRUVFQaDk1DN0BFRUVFReXfjirGKioqKioqDYwqxioqKioqKg2MKsYqKioqKioNjCrGKioqKioqDYwqxioqKioqKg3MScVYEIRPBEHIFQRhZw2fC4IgvCUIwkFBELYLgtCr/pupoqKioqLSdKmNZfwZMPoEn18ItC3/dysw69SbpaKioqKi8u/hpGIsiuIKoPAEm1wKzBEl1gIRgiAk1lcDVVRUVFRUmjr1ETNOBtL8XqeXv6eioqKioqJSC3T1cAyhmveqrbEpCMKtSK5sTCZT72bNmtXD6SVKPCWU+EoI8aYQbTq789J8Ph8azdndBxm1L40TtS+Nk6bSl6bSD6j/vuzfvz9fFMXYyu/XhxinA/6qmgJkVrehKIqzgdkAffr0ETdu3FgPp5d45JdHmF88nwtDP+PlCb3r7bgNwbJlyxg2bFhDN6NeUPvSOFH70jhpKn1pKv2A+u+LIAjHqnu/PuT+V+C68qzqAUCJKIpZ9XDcOiGUG+hen+9Mn1pFRUVFReWUOKllLAjCN8AwIEYQhHRgBqAHEEXxfWABMAY4CNiAG09XY0/YznIx9vi8DXF6FRUVFRWVf8xJxVgUxatP8rkI3FFvLfqHCIJqGauoqKionJ00jQg7/paxKsYqKioqKmcXTUaMNeVd8YqqGKuoqKionF00GTFWLGOvKsYqKioq/0rEamfVBuJxge1EdawAnxeKj9dPm2pJfUxtahTIMWMfagKXioqKSpNFFEGopryFzwdfjofgaBhyPzjLoPkA2LcQ0tYBorTvwcVgK4BL34G/XwOXBcwJYAyHlkMhpR9s/hzWvAujXwQ6nJFuNR0xVmPGKioqKo0bUYTja8AQAvFdQaOpENeyHECUhNHjBLcN8g+CKRK2zwWNFnL3SGLaYrD0eVk2pA6CwsPgKIGsbYAAh5eDLR86XAQH/gKvEzTlcufzSuf55ioIjoG4DlCaCRmbpPMIGulfcBT89TjGPm+dkUvT5MRYzaZWUVFR+Qd4PeBzg95U8Z7HKQleZGrFe9YCyN4uidWa92DYdGnf7O2SazeqFSabDdwOyNwCeXug/RhJBH+9Ew4tkY4T31US5eztkNIXjv4Nog9aDIHMreAq82ucAIhgioK2F8Dun8EQCqmDYdtcSbC9TkjqKe1ry4fuV8Oe+RAaBzctBGOEZBG7LJIQFx+HS9+GNudLp/D5oOAgLHkajq6E2/4GSw6OA6Wn86orqGKsoqKi0thw2wNF8WSIIjiKJYHS6mHzHEl8gqOk48R2lITG64LIFpCxWbIEI1MlQczYDBs/kf6+5G3Y+xs4LVB8TNqu0zhJWG0F0rlcZdK5XBbY/m2V5vQHWP+fijd+f0CyNrV6GP0S6I3S+QA6Xgw7f5AEO6EbrHxNEud2oyGiGRSnQbP+ENVKap/eCJl3Se7oiOaS6Gt0KFWYf5oqCfP49+HClyWBN0VInwWFSv+fNwOOrYLW51W0UaOB2HYw8UsprqwzQHgyHFhW++/hFGgyYqxkU6tirKKicrbgcUnu190/Q+sRRBVsgn0O+O4GaHMejHq+wiq1FkD6BihJg3ajoOBQudC2hznjIGsrGMyQ2B2OrURZNkAfDCHRFQlJBjN4HJIV7E/n8bD/T/jqcmkbjRZcVsn63Pu71B5zouQeFkXY9SNc9qHUjshUiO8i/V94mL3Lf6BDQrAkrhHNYfcv0jn73ATRraXz9b6h4txjX5XEXRBg0DSpX9XFhWWSelb8rTcGfnb5xxV/G8Oq37/r5dK/mtAZav7sNNFkxLgigUsVYxUVlTOM1wO5uyXLLWur5GoFKXYZ1RKWvSAlBrUbJVmnu38Br1uyDpv1l8QzOIZutnzYAYTGw/7/SRaqIVQSJ2texfk2fAz5+ySrT6OXhGvEY1CSAYcWQ5cJ0nk0eji2WrJyx38AQWGw+i3J/XzlHGkbS65ksab0gQOL4PBSGPoAaA2SJRzRXHLhVl4sYewrEGSuei2SepKdWEIH/3rOid1OfP38j2MIrsOFbzo0HTFWE7hUVFT+CfJ0GEeJZD1Gt5asSWeZJEZH/4bcvRCeApmbYfDdksjm7IK250tu0mOrYOtXgccVtCCWz+4whEqJR8ueL/9MIwlpZAtJiJN6QfYOMhNHkdSirWQ16k2w6ycozZLcwlGtJEHP2gb/e1jK/r3gael1h7EVsc/KlOVIYh0aJ73uMCYwI1m2VEHqT1u/4xhCpP+rW7WoOiFW+cc0OTH2qWKsovLvpPCwFCf1eSUhO7JcskD1Jsl167RIyUOh8ZKllr5Bsvoyt0hJSk4LlMkLzpUnDFVG0MCO76S/dSbY9nXFZz0nQ2wHSO4tTaWxF0FMW6kNo16QxOvo35JF3P5CSYyDzJLgtr8QBA37V28gyd+iHDStahuaD5Cs73ajJAv4ZJjjq+nHCVzAKg1CkxNjj1qBS0Xl7MNeBIeXSfHJ0LiKKTDHVkvZs/v+gOg2UpZreAokdJUSi7Z+TXchHHaLkLuLKiIaHC2J36bPpNeRLaVjbv8WwpvDrp+l5J/QcsG65B0pE9dlA12QNM0mOFqyYPP2Sec9sgKSekiWas5uab+cndDn5goLMnVgRRt6Xlvxd3Vxym5X1u1aabRw2ey67aPS6Gk6YqwuFKGicuYQRbDkSP/b8iGuM3jsksu0OE2yBi96HeyFkLZeik2KPin7Nm29JLzmREncSjMll6/HLsU44zqAPgTS1krnWvJMxXl1RikRSKblUIxZ+8HcQirQ4CyTXMyOEskqlS3HsmzJQg6Okv5226VYbtY2yZJO6Ao+z4kzmOM6Sv/7u3Vl0fUXXxWVf0DTEWPZTa1axioqtcdllf6XY4MgCVX2Tkm4tAYoSYfSDLDmS4KVswtK0yULUSaypWRFHl9T8d6hxYHCKRPeDHpMAmuuNMc0ojn0uVHK1j3yt3T8kjS44BnJFbvyDUlUNRppbqrog6IjENUaQmNZV5vF38OTK/42J1T8ndi94m+t/mRXS0XltNFkxFhdKELlX4mcfFQ5BliSDvZi2PCRJDJ9bpKsyuJjkLuHzjt/gT2PSQUZRB/EtJPcsqIobeMoAQRpX68r8NiCVtrnnPsgJEaK0y55VhLic+6T3MjRraVM4ZbnQuvhFVNltIaKuZ7VUV0S0vhZVd+rLg6qonIW02TEWC36oXLWU930EVuhJKzGMGkOaEi05IZN6QcbPoS/X5UycZN7SQlJQWbJyt37uySY2iBJqNd/iH8sNdQYB816StaozljurnVLYhzbATpdIsVkXTap9KA2SCqcUJoJzfpJLurk3hXtbHuBNLWn1bCK9/z/VlFROSFNR4wF1U2tchbg80HODsnla06A9I1SwtKSZ6UKSf1vk+rxOkshf78ketVSnqjU7kIpeShrqxT3tBVA/gHofaOUZNR8oJSAtO59KR6a3AeiWrJuy8GTu3Y7XlzzZ+Epga9D4yqmzqioqNSZpiPGqmWscroRRaliEkhWp6NYmk5jyZHENThaircGmWH/Qskt67bDwUVSOcKolpL1mr+/6rGjWkklAFe9CWHJkti1Pk9KGgpPgbIsaDVcOq+tQLJ8214g/asNwx+p9MbBU7kSKioq9UyTE2PVMlapM7LAeuzS32VZUmJR1nYpcQkgIpXeOxfA8iPlO8kx2hrWTw2Okdy+Lht0HicVkzi0RBLdwfdIU2kKDkjlAo+vgX63SIUh0tZLlZBOlkzU6txT67OKikqjosmJsZrA9S/GXizNGZUThLxuqfSf0yJl05akS3FMjV6Kt4bGS0KYta364+lDpFq7og8OLsKgCYGhD1YsxabRSTVyjeEQEitN4/F6pMIR7cdK23jsNVcqkisdtRhc8Z46RUZF5V9J0xFjNWbcNBFFKZYaliQlJnmc0pSamLaSCB5cLL2XsRE2fS6VH2w1XMratWRLQqvRSVNyAFa+Lv1vipQyhkPj4dyHJJeyzij9C46SprxEtZLeB/C6WbPib4YNH1FzW/2XmZPRqiUDVVRUTk6TEWN5apMqxmcRXk95oQWj5M4tPi5Zr1lb4dBSycK15ktCK2gkcXSWVZ/UpNFBz0kQEgfbvi1frUWQCk/0vE4SZkMIrHpLKjbR81ppjq0hOHCObU1o9VIbVOqE12LBumIF5gsvVAbMKlXxWixoQ08w5esk+BwOBJ0OQddkHunV4ikoQGM2ozGc+VWVTjdN5ptT3dSNCJ8PsreBx0WQI1+qtgRSTd+CQ1LFo4jm0hJsoihZoraCwGMkdgdniWT1nv+UtGxb7h7JUu08XlrBxl4kzWMNiQFzUsVSauc9XrVNcvbv+TMq3juF1WF8djs+hwNdZGTVz2w2XOnpGNu1q3l/pxPr6tWEDBiAxlS7dWvdWVnoEhIQBAFRFGslbp7CQjQhIWiCgpT3in/8CUNGBvhlU9u2bEHQ6TB17Rqwf+6rr+HJzSXppRdr1cbKFH7yKfnvvUeizUbE5SdYsq4cn8tF3ptvEjVpEpqwsBoFSvR6EbRa5bWnqAit2VyjGImiSNqUWzB26ULcvfco7wmCgOjzgdeLoK+I04seD/h8iIDPaq3+e3Y68eTlYUhJwVNYiPPAQUL698O2aRPZTz1N1E03Yj7/AlxHDiMYgjC2l+4H0eUCvR775s3omzXDV1rK4fGXkfTsMwRv3IinSxd0MTG4c3Mp+eUXgnv3IbhXT2ybN2No2RJvURH6xEQ0JhOO3bsp+u47iufOI/KqiSQ88QSOffvRhoagT06u0ma53aLbgzZUGoTaNm+m4KOPEb0ezMOGEXn11dXuY1m6DJ/NRvjFFynXyudwYN+8meCBA096P7qzs9HFxVE4Zw662FjCLrwQQaPBk5+PNjpa2V/0+ch88L+g1RDzn/8Q1LIl7pxcXIcPkX7HnYSefx7JL7+sHDfr8Sdw7N1LyIABRF41EW1UFJ78fDShofjKytCYzRR++hneslISZ1T8/kWfD8eePXjz8wlq2xZ9UpLyWdmSpRT/+AMJjz9xwj7VJ01OjH2+GhJqVOoXrxvy9kpu3aKjkuAaw6Xkp10/SUIJDEADa8sHSMYIiOskxVYzNkrTb0JipRhreHMp2zg8RaoD7F8l6XQ0v6QETXBwwAMYpIc8Ph/FP/yIddUqwi66iLBRI6vsn3Hf/Tj37aP1X39iWbaM4p9+ImTAQMwXnE/arbfh3LePDrt2BgiG6PPhLSlBGxFB1uOPU/rrfLRRUbT4+isMLVrgLSmh6Nu56KKjCB83LkBYrKtXc/zmKSQ+/zwhA/pzcPgIUt57D/OI4VXaJno80kNr+HDy3nqLsLFjSXi0Ips6/513CC8owDV+PIaUZDyFhRy7+hoAOu7dQ9HceTj378eQ2pyCDz8EQSDuoenoIiMp/PprLEuXkfjsMzj37yd0yBDluM7DR7AsXULIkCEYUlNx7tmD67i0hm7em29hvuACtOHh1X4f1nXrybj/fmL+M5XCjz+h8ONPQBBIfPZZIiZchnX9enQxsRyfcjOCVoc7O5vom28i7p57CH9vFge2byfymqtJeKLqw9OdnY1t3Tqsq1ZhXbWKuHvvIX/WLAo+/Iigjh1BFBEdDlp8+w2CwUDx99+T88KL6OLjCe7fj5IffyLxmacJv+QSRJcLT0EBotdH+n/+g/PIERKeeJzcl17GZ7XS/PPPyXvrLZz795P10MNk8bDSjqRXXyGoTVuOXn450bfcQv5774FGQ1C7duB2k/XY45jdbnI9XhKenMGRCRPw5uUDEHLuUKzLV6BLSMCTnU3E1VcRfcMNHL1mkjSg9fko+eVXoq67jiOXXoqhZUuirpuMJtQMgoB982Zc6WkIegOuQ4fwFBQQO+1OIq+6iqxHHpV+DyEhZK9chT6lGe70NAwtW1L4xZeEXXghnpwccmfOBKBg9mxCzz2X2PvvI+Pe+7AsXUr4+PEkPvcs7vR0XMeOEzP9IQr+M5XoG24AwLJ8OWm3TSXuwQfInfkKAD6LFWOnThy9+mrCx44l8cUXEAQBx/btlP7+OwCakBBi77qLwxdfjK+0FIDSPxYS3LsPhtRUjF26UPzTT+giIyn49FNsGzagCQ7Gunq1ct31zZvjLr8Po6+/HndWFiEDB5L39tsUzHpfOo/ZTNtlS9GEhGDftYuMe+9FdDo5tnsPmjvvqPaerW+ajhirMeNTQxSlaTh6k+Q69nml4vrZ26X4rL0I0jZIwut1SklRXmfF/tog8Dpx2U34mo/AeOE4cJSQvv1vmg0cL8Vnu02Uju/1SJauMQxRFPFkZqJPTpb+zstDb47DU1BAzksvEX7JpQT37lWj9eg8eBBtVBRHr76aiPGXETP1NuUzT1ERRy+/Ak1oKAkzZhDcS1qQ3LZlC2m33oahWTOafTgb5759aMxhaEKCOXbtZELOGYx19Rq8+fnYd+7EPPICcl95hfAtW/H164dj7z4sS5dKx1q/noLZH2LfuRPLosXkvf46PqtUYtJbVIQuJkZpT+lvv5H53+mEXXQRpb/9RsSVV1K6cCGZDz9C888+JePe+5SHiCYsDEGnI6R/f9BoyJrxJIgixd9/jztDyvAu+/NPjB07kHbHHcRMnUrYyJHK+5bly7H8/Tf4fJQtWoQ+KYmgtm0JPWcwnuJiNE4nWY89RvOPPyLnhQqr11NYSO4rr+ArKwMqHmTWVasJv2gsJd//gGP3bo5dMwl3RgbR/5mKedgwir//nrI//8JbUgJvvIn5vPMoW7gQTXg4uthYPMXFZNx7H0kzXyb72Wdx7ttP+KWXEn3rLfhKSzl+/fUAFH//g9IWjclE3jvvoIuLJe2WW0GvRxAEQs89F31iIgXvf0BQm7YYt29HExZG0dx5RF47maBWLXHn5FDwwWxCBg8i7403cB6QpnLpYmOl8/z8M9rYGOzbtoHXC6JI/geziZ12J6V/LMRnteI6fBjXsWMIOh2Z/52OY+8+fBYLJfPnE9yzp/Q9CALZT8zAkJqKLzSUzOnT8WRnE//Iw+ibNcOxYwdB7TtQMHs2uS++RMTVVyG63eS/9x6a8HCCWrXCvmULuvh4PDk5iDodJfPnE3HF5Xjz8ol78AE8ubkUzvkCU69eOPbuBcCydBmuo0cR9Hpazf8V+/btZNx1N4dGXwiA68gRcme+gi4+Hp/Viic3V/rbbkfQaDB27kzO8y9Q+NnnuDMzSXr5JYLatuXI+MtIu+UWAELPOw/L4sVYlizBPHIk2tgYEh59jMI5cyj8/HPsO3di37SJkEEDKfnpJ3xWK2V//gk6HRpBIPfFlwju05egtm3Ifu55pd3K7+GPPyj+7jsEQaDkl18oW7YMY8eOCEEG0OkIatMGx67dFHwwG5/FQvzDD2Fo1Zq0W24hu9zCDe7TBzwekl5+CdvmzeS/8y4IAubRownu3RtPQT4F73+gnDP7mWexrlpF4vPPY9+4iaB27Yi66UayHnqY0r/+ImLcOHJfnonGbCbpnbcp+XU+vrCwap899U3TEWPZMsbbwC1phPi80kLlWoNkdaavl+KlcZ1g3wKp/OHxtZLrNyROKubvtkn7hsYjbvkKBC1Cs75SNrLeiMehRYzuiFicjafUQfCkxyn6bi7Zz72EYNhH28mj0YaFcaisFQmxXdHHxWHfsZPShX8Q98ADCEbpBs9/7z3y334HY6dOaCMjsa5aRfLrryG6XJT+Op/SX+cjmEw0n/0BwX374ti7l5wXXiRhxgzsW7eS9cgjGLt0wX3sOHlvvIE+KZHwSy4BIO/1NyTXWHQ0WY8/Tqtffgatlox77kVrNuM8fJic556jdMEfgDQK91mtlP46HwBj1644duyg4IMPKPz4E4xA+r33gk9EGx2N6HRS8NHH2LdtI/buuwgdMYLj112vXHZPQWGAGNs2bAAkUTaPHk3CjCcI7tuHzAf/y8Fhw/EWFZHw5JPkvvoqOc+/gCc7G32zZpi6dsGdlkbo8OFYli7FdeyY0t6Me+7FuXsPua++ivm88xBdLvI/mC256CwW0GjwZGWR+9JLaEJDafnD94g2G57ERGxr13Jk3DicBw5i7NYNx/bt5L7yKr6yMhKffQZDixaYunfnwJChWP9eQciggTh2S6sUuTMyMLRoQcGs9yn48CM0BgOGVq1Ieedtjk+5hbKFC6Vbr6SEqDvuQBcTTfZTT5P2n9tx7NyJqWtX8l5/neA+vfHkV4QonHv2gE5H279X4Ni9m7Sbp5B+191oIyLw2WzE3HknMbfeguhysX/wORR9LS1hGHv3XeTOfIWiL78k7qHpHLv6GtyZmVjXrcN16BDamBgMKSk4du/G53TiTksnZupUgvv1w+ewU7pgAfmzZhHcty+uY8cIHTEC69q1iDYbyW+/hXXlSgo/+URpp3X1aiInT8Zns1Lyw4/EP/IwnsIish57DPMFFxBx5ZVojEbMwyXPhS46imPXTpYs/nLCxlxIzG23kfPCi8TeeQdlixaxT6sl6rXXKZkv3YP6lGZE33wzUTfcgC42Fp/dTvH3P5D70ksVop+QgNZPMMIvn0DJ9z9I4ZIj0lS8+EcfJfLaSfjKyhC9XrQREZT99RfZT8xAGx2NedQoBIMhwIq0LF8uHVAUsa5cibFjR8JGj8I88gKOXHop9k2bCD3vPFLefIODI0dS9uef6JOS0MXHk929G+GffY63IJ/ClX9Lx9RopMEPEHHFFRR/Jy1FmfTKKyCK2DZuxLJiBZ6sLEIGDSSofQeKvvoK5759hF96KVHlA7aY229HYzbj2LGD0gULEIKCMPXqhSbUTP7b74AoEj1lCqYunRFFEfvWbdjWrgWNRvJ8ATnPPw8aDWGjRxN+6aXkvzeLkh9+RJ+UhG3dOuIffojQIUMIHTKEA8uW1fRkrVeajBhrkdyBIj58PhGN5l+WLGLJlWKyOmO51RmOePhvhLzdeEuKyFtdRnCsC41WRGPwYQz3oNGLUuJTVCuyticT3PV8wlu6wRSBaAjj+OyNBHXti/PQHnRRUSQ/+S7urCwce/aQ9dIMBP1mtOYwnPv2EZUTRsmvv2Jo1hzXkSMUf/c9UZOvJWzOHA5u3kLSs8+QOf0hAIL79qX01/mYzz+Pgvc/wNS7N76yMqzr16NPbU7mw48QcdllACQ+/zz5s2aR+dhjNP/4E47fPAVvQQF5b7whWX6AY+dOdEmJaM1hFHz0MWEXX4y3uJji778n8pprCO7Xl4y77qbkt98IGTgIT04O8Y8+imPPHkp++km5hMauXQkbPYrsJ58CIO6+ezl+403kvfEmxk6dyGvXFn7+BYCYaXfiycuj+Nu5AJjPP5+gtm1JmTWL/PdnYV3xN96CfECKE/qcTtwZGegSE4mZOpWIy8YjaLWEX3wx3rIy8t9+h8TnnpNcsmvWUPa//6GLjwefj9IFfxB++QRip93F4c2b8ebnK/22b9tG8IAB2NauJf/dd7GuX49z/36S33oT68pVGDt2IPvJp9BGRuKz28l5WXI12oYPI8Viwbl3H3EPPkj4ZeM5MHAQJT/+iKFlS8InTFC8TSGDBmFZvZqQNWuU6+Q6fpwW8+ZS8NHHWFeuJOW9d9EnSKGFyKsmUvj5HPTJybgzMjB27kTokCHkvfceju3bCTnnHBJmPMGhC0biOnoUV3o6aLUEtWqJ88BBDMnJ6CIjCR08mLiHppP76mvEPfYoYWPGKB4SwWDA0LKl8nA3detOcO/e2DZtouSnn3FnZqJPbY7r0CHpPnrmaVxHjmLfuhXH9u3g8xHUpjUhA/pL92Sfvjh27CRrxhO4MzMJv+RitJERWFf8LT2Uhw3DU1CIbe1ajJ07S2J89dVoIyMIPeccQoYORRAEwseOQagmucjUuze6xEQ8WVkYu3TBV1ZG5FVXoU9IIOXNNwAIatuWnYsWAeA+ngaANkzKxtcnJkqvzWZCBg+S/g4PV+LwmuBgUt55G214OEEdOlDy409S7kY5wf36IQhCgGiHjRxJcN++iDabklMQOXEihXPm4MnJAY+HsDEXUrrgD3xWK/oW0mwBQaMh9t57yX3pZeIffhhBpyNq0iRyZ75C3EPTCRs5krSvvwHAdew4+e9/gHnUKFyHD+E8cBBNcDAREy6j+LvvMI8aRdjYMdK1u/gi3BkZpN99D5GTJuGzO6T4OhB59VVKu2PvktZ49uTnY1m+HFOPHmiCgjB27iTFnrVajJ07SW0VBJrNeo/8We9TMHs23tIS6fdosUjXvEN7BEEgcuJEcmfOJO3W29AlJRJxZR2XtawHmo4YC+WxOcGLVxTR0ETEWBRx7tpC7tuzSXr5JbSCFY6sQCzLwbH3IIKnjCBTIc5tGwgKtSkJvx6nhkO/x5MwtjmF6x04MkSK/As/aTQk3TGe8InX49HGUPzOIKw5RsL+t1ByG/38M7Yd32LbUVGpKWzRIjIffQxfSQnodODx4CGLoA4dFKsh6YXnyX/3PXJnziT/vfcwWa1oo6PJfKgidpb7yiu4Dh6idMECdOUPI210NHg82Ldt49i1kyn960+0sTFEXDYefWICx2+8ieM33YS3sJCgTh0ld5heT+zdd5H35luEXTCSoHZtyXr0MY5ecSWmbt3A5yP8orEYu3dHn5SEZclSdNGSpRrUrh26hHhKfvwRtFrarV2D1mzG53CQ8+JLaMPDCR4wAEOrVrgzMkh6+SWOHT5MxMpVeEtKiLzySjRmM6LdISXxtGkDQHCvnsQ/9DCHV/wtxRY9Hsn9vG6d5D4bPpzIiYE/9KhrriHy6qsV8QsdOpSy//2PyGuuIXLSNZT+voCwsWPRhobQbs1qRJeLtCm3YN++HYDom25EYzSS/94shOBgkl6ZSdgFFxB2gVSdy52ejrFrNwo+/hj7dkm8fCGhJFcTXwVIeOLxgGSc4H59KV2wgKKvv0ETHk7q55/hs1rRhoURd9+9cN+9AfvH3ncfYWPHSl6MZ57F2KULgl5PxPjLKJg9m/Bx4yRx0WpxpaXh3LuPoFYtCerQEeeBg+hTmyvHir7hBiKvvjogAU3G0CJVElZAn5KMqXcv8t9+h/z338fYrRuRV19N1sPSfWfs2BGfRQofWNeuk/ZvXbEUojY0hIgJl5H7yqvSZ6mpRN9yCz6bTckrSH5DCkGIDgeO3bsJatVS+l1ceKFynOqEGCRRMJ93HkVfSjHY6JtvqnY7dDqE4GBc6ZIYa8xVXaRBbdti6tED8+hRaIIrkhDN51csshF67rloo6Mo/e13NEFBBLVtU/3pIiPBLzkt+uabiL75Jg4MH4EnKwvz6NGU/rUI3G4MqRVT98wjRhA6fLhyn0RNnkxQu3aEnHMOAKJR+r7s27YhOhxEXn01hZ9+Kn2/yUkYu3cn+Y03CDlncMC9pk9OpuX3ksXsOnoUkL4LY6XEQgBdTAypX32J1iwNWASNhvhHHkbQ6QOOqTGZ0CdKA0VPVjamPr2xb9wEgLFDB6n9N96AOyOdsiVLaf7RR7VOqqxPmowY6wSpK4LgxesT0WtPskMD47VYcKelYexYvkaqyyotWxdkJjJ/M47PfkaX3BxdxhJyPtuFNduI5ZXrCNcuB0RsuQaOL5GERR+uwV0SQejgC4h/8B4MqS2x/foTvp9mkvO3HW++jYSnn0JjMqGNjEJ0Ocl7/Q0K/reD8DvaYl8ixT/dx49j37wZx86d5L72OkHt2uE6elRx36bfOQ1tTAzJb72JsUMHyUotLib1yy/JevRRKTFi8GB0cfGULV6EY9duCg8fovPXX3P8+htw7pdGA66Dh9DFxhJ7912StSM/UPR65QHpzcvH1F1a3i5k4ECCBw7AtmYtoeeeS9jYMWT+dzrhl15C1E034SkoJPLaSeji4ihbvATbxo04du5EExoqCYEgYOrdG+vaNcoxg9q1RWMwIOj1GLt1U37QGqORqBtvQBtqRhAEEp95Gp/DQVCbNpCeTtILz0vxt/LYY3VZxrqYaKA8S3X2bCVeCWBo3ara+8H/4RE2ehSuo0eIvPoqtKGhAeItaDQIRiPamBjFatCnpJAy6z0sS5Zg7NgxICsUIO6BBwAomf+rIl6+arKUk2a+jM9qI2RgYOGR4N7SghD2TZsIv/QSNMHBASJQGU1QEKZu3TB27UrosGHo46Sa1VE33oAmJISwkRcg6HTok5JwH0/DsXcvwX37YmjZQrpGzVOrHK86DC2k7X1GI9qICIJ79QZRxJOVRcJjjyqfayMi0MXHK9+ZtdxlKX8uY+rZs+LYqaloTKaAh7IgCFJ2d2gooUOH1tj/mgi/aCxF335LyMABJ9xOGx6OOzNL+ttc9XsSBIEW335zwmM0m/UeIGVua4KDESovQHISjJ07YcnKwtS9O0EtUiWPRWrg9+J/zwoGQ0Ayn1j+nXlycsr7FIY+RZrRoEtKQhAEwkaPOmEb9M2bE9SuHRFXXlljprYspjLhY8dWu52m/H73FhVhSGmGz2bDuXsPQe3aS+3XaEh44gniH3+8xnOdbpqMGAdYxo0go1r0+bAsW46g1RAyeDDu7Bx0MdFojNL0m9yXZ1L8448kThlN/rxF4LIREmcnoU8JKftDOLI1HEHnI76vF2u2tI9103ZCbriRoiPheONF0P1I7LRpFH7+OWFjB1K6cCGWcZMw9eypZPF6y1P8w8eNC5ib587KIueZZyn65hvsW7eCXo9GryfnxZdw7NxJyNAhJD7zDLZ169GGh+FKT8d9PI2oydcqUyaSX3sNn6UMbWgIKW++IU030Wgwtm+nTOM4tGwZushImn/yMc4DB0i/cxo+q5XQESOqneqii4xEGxWFt7AwYGpG7LRpHNu4iagbb8TUvRuR1+0k+uab0QQFkfDYo8p2zd57l4JPPiX35Zcl11x5RrKpZw9K58/HsnQputhYZapKwjNPY0gJXPQg7u67lb9lIZLxf+DUhMZsRtDrKf7mWwS9nuQ33yTnpRfxZGYR1Lp6CyVg/5AQ4u6//4Tb+Mei9eXTncznnXfCfWRRBBBDq86tDr+4+oUhDK1bow0Px1tSgnlk1czymhAEIeCcushIYm67teK4zVKw79yJJzsbY4cO6JOlQYShefMqx6qOoHIx9ZZPizF16yol/rRsSWh5rFYTEkJQxw4IgoAuThJj+6ZN6FObVxF5Y5cuoNeD242+kvDUB6YePWi/Yf1JrS5teDieLEmMNaeYPOQ/BaguhF90MYKgQRcXh6FV63IxblHr/cXy55w7LxeQxFD+PVceLNaEoNHQ6tdf6tbwGtCEVAxqNGFmIiZMwJa8VpnepZyzAefCNxkx1sldETx4zoAYi243CELA9BNXWho+ux1ju3Zkz3hSSVCImHgFxXO/QxMaTMKoeMzxJZT+XAQekezZ89EaISgpiuKDxYTffB85//sCU6dUnIePkL3GgS4xEWP79pStX497QTG2tQtBEDB1707MbbcqD7i4++6lZMECCj/+BG9xMdqICLzFxZjPP7/KJPmw0aPJeeFFsp96GgB9anOiJl1LzvPPowkPJ/nll9GGhxN+8UU1XgNT1y4Br/2n8VRGFxODLiYGQ6tWOHbswNihfY3bBrVqha2wUHk4AwT36kX7jRuUB2jCI5UXPqgg4vIJFH75BWEXjg7YH8C2caPiSgOIGDeuxuP8UwRBQBsdjSc7m6B27QgbNRLbxo0UffEFQTVYxnVFtr614eEntFID9vETxuos45oQNBpMffpgXbOGkMGDT75DLdGnNMO6ujwO3bkT+pRmUjJOj+612l+2bL3lAxNNcDCJTz1FULt2iiUY/+ij6JOkeKv/AMbYqVOV42mCgjB16oTzyBG0ERH/tFsnpDbuT/+47qkUAjkVwkaPUixXY6dOWJYtw9C8We0PoNUiGAx4cvzEOEUSY0MN859PJxo/0dWGmomaNImoSZPOeDtORNMRYz839ZmYa3z8pptxHjlC7N13EXnFFQBkTn8I16FDJD88heLvvkMXHoSnxEnxvHmYYlyAk8wfrIR3C8PnEtEE6fA5PURN+y/h48ZxYNBgsucsQyixEfPCNGzr1lL4+Rxi77wT0ePBsmwZtrVr0cbE4M3PJ7g8+URGn5xMzC23YEhOJuO++4m64QbcmZlEVnPT6aKjSf30EzxFReQ88yyRV15J5ORr8RYXYezcucb5oKdKULkYB1VyL/ljaN0a28aNVYoW1OSurIw2LIy2S5YEnrdtW7TR0XgLCgju27fuDa8jOj8xBoieMgVDs2boa2n1nQxttCTGulpaGQC6uHjlb19ILaqO+RH/3wdx5+Qonp36QH64a0JDCe7TB0Gno/3WLbW2TmS3qbf8WgBETLgsYJuIy8Yrf2vMFaVJ/eOr/kTfMgVXWnqDWkjyb0+oZh58QxB1/XWYL7igznFUTXAw3uJiALQhIQS1lGLslcMDZwL/QY3/fdCYaDJi7O+mrk/LWPR6Kfn5Z1xHjmAeNYqgtm1x7NqFbcMGdAkJZD/+BJ7jh4hsWYB9y2YQ4fjDr6A1iLS40M7BeRrwCUSOG0No3w4cfuJ7SrbnEDJoEKbevSj4+BPCx41DFxmJsVtXHNu2426WQuiwcwnu2wdDmzaEj7sU0eVCdLkwde2Ct7SUtFtvqzFuFTZmjDSvtFOnE5aNk0XJXJ7oIwgCsXfdVW/XrjqMXbtStmjRCatTyYkxNVUQ+icIWi2tfpuP6HIr7srTiTY6CkARY318HFHXTa6348tWnpxlW6t9yi1jTUiIlIBXBwypqVVihqeKtjxUYD5vhOJhqosIakJCSJo5k21OR6229z926LnVr3pVk0ifSbQRkhhrG4loaIxG5TdZp/1CQvAWFyMEBSEYDAS1bUuLeXOlcMAZRuMnxtXF4RsDTUaMZcsYwYNPrD8xTr/7biyLFgPg2L0Hn8OOffMWBL1Aq6H7yF6nJ/+jz6CTBUQzoc3c+MLaEP/Ec+i798Z08FrsW7YSOuVZtBERpLw1BOvq1UTffDNoNEROnKjEL0OHDsWxbTuWyy5D0GjQms2K1S2YTERNvlZpV5vly9DHx1dpr4ypR49a9/FMWgGRV00k7MLRkiDUQPDAgRhatqxIbqsnqitpeLqQs7bl2Hm9H/8fibE0CDldLti6Yr7gAuw7dwbE6OtK+MUX4avDPFBTr16IHk+DuX9rgxwnlqc1na3I4RN/ITR169YwbQmwjM9MEY+60uTEWKjHBC5vSQmWxUuInHwtYu5hSldsxmeXRuGxvUW0A64lfngryu5+h/xdZrQx0aT88D1CWEUpx9g778R54KDyADR1765k9EJgHCvq+hsw9ehBjtt90radSIgbM4JOh87PrVgdxnbtaP3HgjPUotODHNMNOoEH4JSOX54ZLMdDa4OcTKU9g4OSE6ENCwuoFXwmaPH1V4j1OFg/HWjDI4DGKxq1RR5wa6pJFjzTqJbxGUQu+oHgqbMYF875Aq+ljPCLL8bQrCJJwbZxPYgi5vxPcRVaKbZHANDssesJvfoB0EppY4kvNMO2dh2h540IEGKAkAEDCBlw4qkMSh9CQwgdPBjOUMUXldNH+KWXSlNq/AZb9Yk+IYHE554jtJra1DWhCQ9HCApqNJZxQ9HYV4+SY8aNxU39T5EtY21Iw4ufPI1RdLvVmPHpRiNopGUU62AZuzMy0ISHS6XRgIKPPqbZu+8QEl0GS57GvjgHNFpMfQejDesHG96RspgvvQO0FZcufOzYGue3qfw7CWrTRpqbfBqpnKx0MgRBwNCqVd2yYlXOONpwySI+1WlNDY0mpKqbuiHRhIbiLSpqNO2pTJNaoFUr6BFqmcBVtngxB887n/z3ZgEQ85+paIMECl+8H8e7V+MutmEtisTUJhXNNZ8TNOpWBKNRysptpCMrFZWTkfrpJ0oREJXGSdOxjMvd1I2kH7IIaxvpIKfJWMYAWo3+pAlcos+HaLeT8eB/AakAAECQdQ3m2ByKD4RgPRgPehHRUUbcf/8DgKDXEznpmnrN8FVROdP8213UZwOyGGvO9gSucsu4cmGNhkIW48YyOKhMkxJjnaCTpjZ5qxdjV3o6h86/gKibbkK0SasSudOPAqBNX0zoBZdTtH8Z6LQYW7cmuG9fom68Qdk//sEHT3MPVFRU/u1owmTLuHFacLVFSeBqBDFjkOY6CwbDCad7NiRNTIz1IHhrtIzz334bQFnUQBvkxVMgreKhvfBxDKOmov16GOaRI0l86skz0mYVFRUVf/QJ8YSNHUvIoIEn37gRU93UpoZEExraaK1iaGpirNEj1FAO07JqFSXl69QCaHQ+jMmhWA/bpX0HXI3GYKDVgt8b9RxEFRWVpo2g05H86isN3YxTpmJqU+N4nuqbN8NbUtLQzaiRJibGumqzqd05uWTcdz9BqUmERudTsMlJUKweXefhcFiazyrHac5kYQgVFRWVpooytamRzOuNf+ABRI+noZtRI00qm1p2U1cW49yXXkK02Ujpso3gJGnB7aBBY9DGSXOCNeWxBBUVFRWV+qGxWcaCwVDrRVUagqYlxuVuan8xtq5dS+mCBUT30mFIiMT03wVoY2MIOfd8dFFS/eDGUpFIRUVFpamgxIwbSQJXY6dJuan1Gj0INiWBS3S7yX7mGfRRRqKbHYIxX6CNTaHd338DUPzTz4A63UNFRUWlvjGkpiIEBTXIKk1nI7WyjAVBGC0Iwj5BEA4KgvBQNZ+HC4IwXxCEbYIg7BIE4cb6b+rJkcTYi8sruaJL5v+G69Bh4jtlohnxX+h0ScD2uijJIlbFWEVFRaV+MbRoQYdtW//Rik//Rk4qxoIgaIF3gQuBTsDVgiBUXpn7DmC3KIrdgWHAq4IgnPEgrEFrQBC8ON1eRJ+Pgo8+IihaQ2i3ZDh3epXttVHlC7SrbmoVFRUVlQakNpZxP+CgKIqHRVF0Ad8Cl1baRgTMglSBPRQoBM542lqQzgCCB4fbh2PXblyHDxPVpgDhvMcDaknL6MrXnFUtYxUVFRWVhqQ2MeNkIM3vdTrQv9I27wC/ApmAGZgoiqKv8oEEQbgVuBUgPj6eZfW4OpHFYqHMUgqCl607d5N4dA1hgNg8gWW54ZBXzblcLuK0Wo5bytjbiFZKslgs9XptGhK1L40TtS+Nk6bSl6bSDzhzfamNGFe33ljlqhqjgK3ACKA18JcgCH+LolgasJMozgZmA/Tp00ccNmxYXdtbI8uWLSM5JJGdZWmktmpDi23zKNL6iJj0AsM6jKhxP/s3X2No1apRFfpYtmwZ9XltGhK1L40TtS+Nk6bSl6bSDzhzfamNmzod8F9zLQXJAvbnRuBHUeIgcAToUD9NrD1GXZDkpna6cezeTlAkCG0vOOE+pm7dGpUQq6ioqKj8+6iNGG8A2gqC0LI8KesqJJe0P8eB8wAEQYgH2gOH67OhtcGo0xNX4mLY7ZdiO1xKUMtm1caKVVRUVFRUGhMnVSpRFD2CINwJ/A/QAp+IorhLEISp5Z+/DzwDfCYIwg4kt/Z0URTzT2O7q8WgNdA2213xulPvM90EFRUVFRWVOlMrs1EUxQXAgkrvve/3dyYwsn6bVnf0Gj0JJVISt87kJfTCCQ3cIhUVFRUVlZPTpHy4eo2ehCIfvmAdbSeJ0KNfQzdJRUVFRUXlpDQtMdbqiS/yoTOLEN+loZujoqKioqJSK5rUQhF6jZ7EIjCF2CBBFWMVFRUVlbODJiXGRqdIhBV0ZrdqGauoqKionDU0KTEOzbVIf5i9EKkWJ1dRUVFROTtoUmJsLLYDIIaIEJ7cwK1RUVFRUVGpHU1KjIPKHAC4jEBIbMM2RkVFRUVFpZY0KTE2lEpinG8MA422gVujoqKioqJSO5qUGOtL7bi0kG0Ib+imqKioqKio1JomJca6UhulwZAlqGKsoqKionL20KTEWFtipTQYckR1FSYVFRUVlbOHpiXGRcWUBgtki6H4fJWXXFZRUVFRUWmcNCkx1hSXUhYM+UIITo+voZujoqKioqJSK5qUGAvlbupiIQiH29vQzVFRUVFRUakVTUeM3W5wuCgNFrBqNNhVMVZRUVFROUtoMmKsKSsDoDQYbIKgWsYqKioqKmcNTU6MS4LBoRFxuNWYsYqKiorK2UHTEWOrDQCHETwaLw6PahmrqKioqJwdNBkxFuzSIhG+IBA0LtVNraKioqJy1qBr6AbUF4JDEmMMAoLgxO5SxVhFRUVF5eygyVjGmnLLGKMWNE4sTk/DNkhFRUVFRaWWNBkxlt3UWqMeQeOi1KGKsYqKiorK2UGTEmONAUK0QQgaJ6V2d0M3SUVFRUVFpVY0GTHW2B1o9D5CdEYErYtShyrGKioqKipnB01GjAW7Da3OS4guGI3WSZnqplZRUVFROUtoMmKstVnR6H2Y9MFSzFh1U6uoqKionCU0GTEWbFY0epEQgxlRcKoJXCoqKioqZw1NRow1DjtavY+QoHAQPJTK845VVFRUVFQaOU1GjAW7E41BJNhgBqDEYWngFqmoqKioqNSOJiPGOJySZawPAaDMZW3gBqmoqKioqNSOJiHGPqcTwetDoxcx6YIBsLhVMVZRUVFROTtoGmJcvnyiRu8jRC+JscvrwKmu3KSioqKichbQJMTYWy7GWr1IcLkYC+pcYxUVFRWVs4QmIcb+lnGcKQYAQVesirGKioqKyllBkxBjf8s4KSQRvWBAG5SjFv5QUVFRUTkraBJiHNyzJ+LtozBGudFqg0gIbo4mKFetT62ioqKiclbQJMRYExyMEBeGRieCRkuLsFZognIotqlirKKioqLS+GkSYgwgiOWZ0xod7aPaotGXkFFa1LCNUlFRUVFRqQVNUow7x7QF4HDx4QZskYqKisq/F7fPzWc7P8PtVT2UtaFJinGL8FQAsqyZJ9zH6/My8vuR/Hb4t9PdPBUVFZV/Fdtyt/HqplfZkruloZtyVtCExNgn/aHREmmMBKDQUXjCfVw+F1nWLNJK005381RUVFT+VXhEaWqpx9c4pph+vutzHvn7kYZuRo3oGroB9YW/ZRyhDwZRoNRVfMJ93D53wP8qKioqKvWDzycZSLIoNzQ783eyM39nQzejRpqQZVwhxlqNFr0QgtVTcsJ95BFbYxm5qaioqDQVvOXPZK+vcZQl9opepU2NkSZpGQOYtOEU+UoRRRFBEKrdR04sUC3j04/b7SY9PR2Hw9HQTfnHhIeHs2fPnoZuRr2g9qVxYDQaSUlJQa/XN3RT6h1Z+HxyCLGB8fpUMT4jKDFjQQtAqC6cQo2FMqeHMGP1N3pji2k0ZdLT0zGbzbRo0aLGwVFjp6ysDLPZ3NDNqBfUvjQ8oihSUFBAeno6LVu2bOjm1Duy8DUWN7VP9DUaK706auWmFgRhtCAI+wRBOCgIwkM1bDNMEIStgiDsEgRhef02sxZtFL0gaEAjdSkiKBJBZyW/zFnjPrJl3FhulqaMw+EgOjr6rBViFZX6RhAEoqOjz2pv0YmQLeJGYxmL3kbTluo4qRgLgqAF3gUuBDoBVwuC0KnSNhHAe8Aloih2Bq6o/6aepJ2iV3FRA0SZohC0VvItrhr3UWPGZxZViFVUAmnKvwnFMm4kz1ef6GvUhldtLON+wEFRFA+LougCvgUurbTNNcCPoigeBxBFMbd+m3lyKotxfEg0gtZGdqm1xn3UbOp/F6GhoQ3dBBWVfw1yNnVjsUY9oqfRtKU6aiPGyYD/RNz08vf8aQdECoKwTBCETYIgXFdfDawtlcW4eXgcgiByqCCvxn1Uy1hFRUXl9KBkUzeSpCmf6GvUYlybBK7q/ChiNcfpDZwHmIA1giCsFUVxf8CBBOFW4FaA+Ph4li1bVucG10Sqy4nbK7Kq/JhFVsk4X7t3B8s0pdXuc8hxCICsnKx6bcupYrFYGlV7TgW5L+Hh4ZSVL3XZkJSVlSGKIo8//jh//fUXgiDw4IMPMmHCBLKzs7nhhhsoKyvD4/Hw+uuv079/f+644w62bJGqCE2ePJk777yzgXtx6ni93kbxfdQHZ3tfHA6H8ntvKr99i8XCrj27ANizdw/LMpY1bIOAwqJC3B53na/vmfpOaiPG6UAzv9cpQOU6k+lAviiKVsAqCMIKoDsQIMaiKM4GZgP06dNHHDZs2D9sdlUy9s9CH2REPqYxy8hnf36GK0igpvMEZwXDnxAZHVnjNg3BsmXLGlV7TgW5L3v27GkUGa9ms5kffviB3bt3s2PHDvLz8+nbty+jRo3i119/ZcyYMTz66KN4vV5sNhv79+8nNzeX3bt3U1ZWhtfrbRT9OFXO1gzk6jjb+2I0GunZsyfQdH77y5Yto11yO1gNrdu2ZliHYQ3dJD5e8DFivljn63umvpPaiPEGoK0gCC2BDOAqpBixP78A7wiCoAMMQH/g9fps6Mmo7KaONkYDkG2tOXwtx4pVN/WZ5an5u9idWb234p/SKSmMGRd3rtW2K1eu5Oqrr0ar1RIfH8+5557Lhg0b6Nu3LzfddBNut5tx48bRo0cPWrVqxeHDh5k2bRrDhw9n3Lhx9dpuFZWmSmNzU8tFP05Ue6IhOWnMWBRFD3An8D9gDzBPFMVdgiBMFQRhavk2e4CFwHZgPfCRKIpntO5YlZhxWHMENBS50/D6KnvVJdSY8b8TUaz+fhg6dCgrVqwgOTmZyZMnM2fOHCIjI9m2bRvDhg3jww8/ZMqUKWe4tSoqZydyAldjmdvb2IqQVKZWRT9EUVwALKj03vuVXs8EZtZf0+qGIPpAo1VeB2mDiDIkkR2UTVaJnZTI4Cr7qNnUDUNtLdjTxdChQ/nggw+4/vrrKSwsZMWKFcycOZNjx46RnJzMLbfcgtVqZfPmzYwZMwaDwcCECRNISEjgjjvuaNC2q6icLTQ28ZPb4RW9aNGeZOszTxOqwBVoGQO0MLcmz7KD44W2asVYtYz/nYwfP541a9bQvXt3BEHg5ZdfJiEhgc8//5yZM2ei1+sJDQ1lzpw5ZGRkcOONN+Lz+fD5fLz00ksN3XwVlbMCWfway9zexuY2r0yTFuNOMe3ZVLCcfbkFDGodU2Uf1TL+d2GxWACp0MLMmTOZOTPQkXP99ddz/fXXV9lv8+bNwNmfKKSiciZpdJZxI3ObV6ZprdpUSYx7JnQEYGv23mr3USzjRjJyU1FRUWkqNMZVm/z/b2w0ITEOjBkDtI1sA8DBokPV7qNmU6uoqKicHvxjtI2BxmapV6YJiXFVyzgpNAkQyLZmVbuP4qb2qm5qFRUVlfpEtogbixg3tsFBZZq0GBu0BkK0UVh9eZQ5qgqu6qZWUVFROT00NvFrbG7zyjRpMQaIMyUg6Is4mGup8pnqplZRUWlsOL1Oxv44ltWZqxu6KadEYxO/xmapV6YJiXHVmDFAangKGn0Re7Or1q5VxVhFRaWxUeos5XjZcY6UHGnoppwSjXE9Y///GxtNSIyrt4zbRjVHoy9h87GCKp+p84xVVFQaG41tHeB/ihz+ayz9UNzmjcRSr0yTF+Pk0GQQfGzMqDrKVOcZq5xNbN26lQULFpx8w3pgypQp7N69u877LVu2jIsuuug0tOjfgyxejdWCqy2NbT1jNZv6DFGjGJulpZePl6ZTYg8UXdUyVvmneDxn/p45U2Ls9Xr56KOP6NSp02k/1+nE6z07xayxxVr/KY3NLSxfz8aasNuExLj6mHFKaAoAGkM+29KKAz6TpzTJK3moNF2sVitjx46le/fudOnShblz59KiRQumT59Ov3796NevHwcPHgRg/vz59O/fn549e3L++eeTk5MDwPPPP8+tt97KyJEjue6669i1axf9+vWjR48edOvWjQMHDgDw5ZdfKu/fdtttJxSFhQsX0qtXL7p37855550HwPr16xk0aBA9e/Zk0KBB7Nu3D5fLxRNPPMHcuXPp0aMHc+fOxWq1ctNNN9G3b1969uzJL7/8AoDNZuPKK6+kW7duTJw4kf79+7Nx40YAvvnmG7p27Ur//v2ZPn260o7Q0FCeeOIJ+vfvz5o1axg2bJiyT23bWBtq2s/r9fLAAw/QtWtXunXrxttvvw3Ahg0bGDRoEN27d6dfv36UlZXx2WefBawpfcUVVyjrzVbux9NPP03fvn3p0qULt956q/I7P3jwIOeffz7du3enV69eHDp0iMmTJyvXEGDSpEn8+uuvtepXfdLYRaO2KOUwG4mx09hi2JVp0uUwQXJTRwRFkGc6zubjRQxtF6t85n+ze3we9Fr9GWnrv54/HoLsHfV7zISucOGLNX68cOFCkpKS+P333wEoKSlh+vTphIWFsX79eubMmcM999zDb7/9xjnnnMPatWsRBIGPPvqIl19+mVdffRWATZs2sXLlSkwmE9OmTePuu+9m0qRJuFwuvF4ve/bsYe7cuaxatQq9Xs/tt9/OV199xXXXXVelTXl5edxyyy2sWLGCli1bUlhYCECHDh1YsWIFOp2ORYsW8cgjj/DDDz/w9NNPs3HjRt555x0AHnnkEUaMGMEnn3xCcXEx/fr14/zzz2fWrFlERkayfft2du7cSY8ePQDIzMxk+vTpbNq0CZ1Ox4QJE/j5558ZN24cVquVLl268PTTT59SG09GTfvNnj2bI0eOsGXLFnQ6HYWFhbhcLiZOnMjcuXPp27cvpaWlmEymEx6/cj86derEE088AcDkyZP57bffuPjii5k0aRIPPfQQ48ePx+Fw4PP5mDJlCq+//jqXXnopJSUlrF69ms8///ykfapv5OdSU7GMG4v4NTZLvTJNXowFQaBHbA9W2vaw+XhxwGf+xT7cPrcqxk2Yrl278sADDzB9+nQuuugihgwZAsDVV1+t/H/vvfcCkJ6ezsSJE8nKysLlctGyZUvlOJdccokiCAMHDuS5554jPT2dyy67jLZt27J48WI2bdpE3759AbDb7cTFxVXbprVr1zJ06FDl+FFRUYA0ULj++us5cOAAgiDgdlef0/Dnn3/y66+/8sorrwDgcDg4fvw4K1eu5O677wagS5cudOvWDZCszGHDhhEbG0tZWRmTJk1ixYoVjBs3Dq1Wy4QJE+q9jZWpab9FixYxdepUdDqdcp4dO3aQmJioXMuwsLCTHr9yP5YuXcrLL7+MzWajsLCQzp07M2zYMDIyMhg/fjwARqMRgHPPPZc77riD3NxcfvzxRyZMmKC050zS2Kfg1JbGtlBEY0/gavJiDNA9rjvL0pex5Vg6Pl9fNBppYekAy7iR3DD/Ck5gwZ4u2rVrx6ZNm1iwYAEPP/wwI0eOBAhYZFz+e9q0adx3331ccsklLFu2jCeffFLZJiQkRPn7mmuuoX///vz++++MGjWKjz76CFEUuf7663nhhRdO2qaaFjl//PHHGT58OD/99BNHjx5l2LBhNe7/ww8/0L59+yrv17R9TRiNRrTaqmGeU21jbfer7jw1nVun0ynJQQBOp7PafjgcDm6//XY2btxIs2bNePLJJ3E4HCe8DpMnT+arr77i22+/5ZNPPqlVn+qbphIzbkxuYVEUG52lXpkmFDP2VhszBuge2x0Am+Ywh/Mrin/4xzLUkphNm8zMTIKDg7n22mt54IEHlJWY5s6dq/w/cOBAQLLekpOlxL8TuSkPHz5Mq1atuOuuu7jkkkvYvn075513Ht9//z25ubkAFBYWcuzYsWr3HzhwIMuXL+fIkSPKtpXP/9lnnynbm81mysoq5suPGjWKt99+WxGXLVu2AHDOOecwb948AHbv3s2OHVJIoH///ixfvpz8/Hy8Xi/ffPMN55577gmvW13beDJq2m/kyJG8//77SmJcYWEhHTp0IDMzkw0bNgDSqlkej4cWLVqwdetWfD4faWlpbNq0qdpzORwOAGJiYrBYLHz//feAZGGnpKTw888/A5KY22w2AG644QbeeOMNADp3bph1t5tKZUAlK7wRDCr8BbixxLAr04TE2FejZdw5ujMaNGiN6aw/UqS87y/AjfULUqkfduzYoSRVPffcczz22GOA9CDu378/b775Jq+//joATz75JFdccQVDhgwhJqbq0psyc+fOpUuXLvTo0YO9e/dy3XXX0alTJ5599llGjhxJt27duOCCC8jKqr42emxsLLNnz+ayyy6je/fuTJw4EYD//ve/PPzwwwwePDgg+Wv48OHs3r1bSeB6/PHHcbvddOvWjS5duvD4448DcPvtt5OXl0e3bt146aWX6NatG+Hh4SQmJvLCCy8wfPhwBg0aRK9evbj00ktPeN3q2saTUdN+U6ZMoXnz5nTr1o3u3bvz9ddfYzAYmDt3LtOmTaN79+5ccMEFOBwOBg8eTMuWLZXQQ/fu3as9V0REBLfccgtdu3Zl3Lhxirsb4IsvvuCtt96iW7duDBo0iOzsbADi4+Pp2LEjN954Y637VN80JhE7FRpTOUx/MW6sljGiKDbIv969e4v1ifPZFFH87b4aP7/0p0vFHh9MFK/7eJ3y3u2Lbhe7fNZF7PJZFzG9LL1e23MqLF26tKGbUG/Ifdm9e3fDNqQaUlNTxby8vFpvX1paehpbU394PB7RbreLoiiKBw8eFFNTU0Wn0xmwzdnSl9pQn32xWq1iq1atxOLi4no75snw/20sXbpUXJu5VuzyWRfxmTXPnLE21DdLly4V/7v8v2KXz7qIty+6vaGbI9rdduVZvypjVZ32re/nMbBRrEYT/xUxY4BO0Z3ILFvJ6r35lNjchAfrVctYpUlis9kYPnw4brcbURSZNWsWBoOhoZvV6Fm0aBE33XQT9913H+Hh4Q3WDmVq01n+TFIt47rxrxLj+Yfn49HkMfLHETw/5MkqU5tU/l0cPXr0jJ2rf//+AYlGILlKu3btWu/nMpvNyhzhhuLTTz/lzTffDHhv8ODBvPvuuw3UopNz/vnnc/z48YZuRqMrI/lPaUyJaP7P+sbQnupoQmJcfdEPmY7RHQFIaradUm8x2/N2VpnapKJyuli3bl1DN+GMcuONNzZo3PVspqlNbWoMlqh/9n1jva5NKIHrxJZxx6iO6AQdvpD1AKxPOxow8jzbR6EqKipNg8ZkUZ4Kjcnd7i/AjWFwUB1NQ4xFEc1JxDhYH0ynmE5YPMUA7MtPx+1zE6QNAhrHDaOioqKiuKnP8qlNjWleb8DUpkZ6XZuIGJdf6BOIMUDf+IqpDQ5fESUOByadVE1JdVOrqKg0BhQ39VluGTemBK4Ay9jX8IOD6mgaYixbtSeIGQP0Seij/K3VlwaIsWoZq6ioNAYaew3l2tKY+uHfhsbQnupoYmJ8Ysu4V1wvusd2Z3DyYNDasXusGLVSXVpVjFX8CQ0NrfGzo0eP0qVLlzPYGpV/E02lAldjqgWtJnCdKWopxsH6YL4c8yUXtrhQekNrweuVFodQ3dQqKiqNgaZSgauyZWxz27h7yd1kWaqvSHcm2gKN97o2jalN8sU9iRjLxAZXLKOYXuiFINUyPpO8tP4l9hburddjdojqwPR+02v8fPr06aSmpnL77bcDUslLQRBYsWIFRUVFuN1unn322ZOWh6yMw+HgP//5Dxs3bkSn0/Haa68xfPhwdu3axY033ojL5cLn8/HDDz+QlJTElVdeSXp6Ol6vl8cff1wpL6miItOY3LunQuWY8bHSYyxJW8KFrS4kMTSxQdri357GRhMR49rFjGXig+OVv0WfZBm7yuccb8zeSIvwFsSYaq5JrHL2cdVVV3HPPfcoYjxv3jwWLlzIvffeS1hYGPn5+QwYMIBLLrmk2pWCakIuYrFjxw727t3LyJEj2b9/P++//36VtY4XLFhQZU1lFZXKNJUELrn9shDK3seGWJQnoOiHKsankVq6qWX8xbhTQgy7SmDFgRxGt/QwddFUxrQcw9ODn2bakmkMbzacy9pedjpa/a/lRBbs6aJnz57k5uaSmZlJXl4ekZGRJCYmcu+997JixQo0Gg0ZGRnk5OSQkJBQ6+OuXLmSadOmAdChQwdSU1PZv39/tWsd17SmsoqKP7JYnO0xY6Uf5c9nWYwbwgt5NpTD/FfFjGVCDaF0jJIqcrWKjgRgwc50tmYdxOl1sjZrLSXOEpalLWPG6hmno8UqDcDll1/O999/z9y5c7nqqqv46quvyMvLY9OmTWzdupX4+Hhl2b3aItawNu4111zDr7/+islkYtSoUSxZskRZU7lr1648/PDDPP300/XRLZUmRlOJGVeuwKVYxg2Qn3M2xIz/lWIMSBnVBGYuzlq9GoAsaxZ/HPkDAK1QO9e3SuPnqquu4ttvv+X777/n8ssvp6SkhLi4OPR6PUuXLq1x3eETMXToUL766isA9u/fz/Hjx2nfvn21ax3XtKbymSTbmk2xo/iMn7exkFaWRqYls6GbcUKaSsy4ciUx2T3dEGJ8NmRTNxE3dd0SuAAGJg7kox0fcaD4AABdkkNZm7YXTbT0+YfbPwQgOTS5Xpuq0nB07tyZsrIykpOTSUxMZNKkSVx88cX06dOHHj160KFDhzof8/bbb2fq1Kl07doVnU7HZ599RlBQEHPnzuXLL79Er9eTkJDAE088wYYNG3jwwQfRaDTo9XpmzZp1Gnp5YkqdpXj0HsJpuFWJGhKX14VGaNw2SGMqI3kqVE7gcvlcQMPEjM+GecZNRIzrlsAF0Cu+F2NbjeWSVpdw26Lb6J0axs68nWi94UQHm8m1pwMVN5BK02DHjh3K3zExMaxZs6ba7SwWS43HaNGiBTt37gTAaDTy2WefVdnm4Ycf5uGHHw54b9SoUYwaNeoftLr+8OGTXOu1z1FrUoiIjTZmKCPHihuraNSWyha+6qY+MY17iFhb/oGbWqfR8eKQF+kV3wuA2DAdCbGlaDzxkHutst2/2aWn0vQQRRGR6uPc/wbkhdwbM03FMq6cFd6gbuqzIIGriVnGde+OXiNNbXpzs7T2au/4i1m2JoJPx3zPjtL/8eGOD7F77Jh0JkqcJeg1eoL1wfXWdJXGy44dO5g8ebLy2ufzYTKZzurlEEX+5WJM4xfjppLAVdkyrpxV3RBtgcabpd5ExLjuMWMZrUbL9L7TOVRyiFhTLOenXMzq9Tv5Yb2VYX2SAMk6NoWauH3R7bSKaMUzg5+pz9arNFK6du3K1q1blddlZWWYzeaGa9ApIluF/2o39VngGWgqU5sqx4wbcp6xfwKXahmfTv5BzNifaztdG/D6zhEOXvtrP6YIJwBFziISQhLYX7Qfp9d5Sk1VUWkoZBFq7GJ0OlEt4zNHY4oZBxT9aKTX9V8bMz4Rdwxvw8hO8fy4oRiQLOMCRwEOr4PjZccb/Y9ZRaU6ZIvg33z/iqKUwNWYr0FTmdpUeaGIxhIz9ope0krTWJu19oy340SoYlwNWo3AB5N7c3UfaarL/J0HSC+TsqvtHjv59vx6OY+KyplEFqB/s2Us05ivQeX5uWcryuCvPIO9sWRT+0Qfn+z6hIf/fvgEe5x5moYYp/Rlfd+3IbF7vR1SEATuGd4DgF93HGBHzhHls2OldS8OoaJSn/hEH0dLjmJz22q9j+KmbsRW4elG7ntjjRtC01lC0T8b3Ct6G005TI/PQ6mzFKvbesbbcSKahhgbQrCFNAdDSL0eNsIYjoAAWisfrNqgvH+87HiVbX8++DPz9s2r1/OrNBwnWs+4MeD1ebG6rdg8tRdjf0sFYNmyZawurzp3uhkzZgzFxcV13u+zzz7jzjvvrJc2+CdvNWYxbmqWsfx3g1rGvkDL2Oq24vQ6G9XAtGmI8WlCp9ERFhTGoPY6ilw5+DwhIGo5WHQ0YDub28bjqx7nmbVVs6y35W3jl4O/nKEWq5yteDx1sxb+iZVbeZ8zIcaiKOLz+ViwYAERERGn9VwnbYufa7quYiz340ygzM9tJDHjTEsmPx34qc77VS60ocSMG0EFLqvbGjBAaAw0jWzq08g5yeew8MhCNOFeWoR25FBBAQsOLufe3tPQa6U5yguPLlS29/q8aP2yur/c/SUrM1ZyaZu6rZPblMl+/nmce+p3PeOgjh1IeOSRGj+vz/WMs7KymDhxIqWlpXg8HmbNmsWQIUMIDQ3ltttuY+nSpURGRvLtt98SGxvLhx9+yOzZs3G5XLRp04YvvviC4OBgbrjhBqKiotiyZQu9evXikksu4e677wZQ2mY2m5k5cybz5s3D6XQyfvx4nnrqqRrdrXPmzOGVV15BEAS6devGF198wfz583n22WdxOB0Ehwfzygev4PK4eP/999FqtXz55Ze8/fbbdOjQgalTp3L8uOT5eeONNxg8eDB5eXlcc801FBQU0LdvXxYuXMimTZuIiYnhtdde45NPPgFgypQp3HPPPRw9epQLL7yQ4cOHs2bNGn7++WfOPfdcNm7cSExMzAnb6HK5iI6O5quvviI+Pp6TMX/+fJ566im8Xm/AfhaLhWnTprFx40YEQWDGjBmMGz+OlYtX8uZzb6JDR2xsLIsXL+bJJ58kNDSUBx54AIAuXbrw22+/AVTpx4svvsiGDRuw2+1cfvnlPPXUUwBs2LCBu+++G6vVSlBQEIsXL2bMmDG8/fbb9OjRA4DBgwcza9YsunXrdsI++SdwiaJYpyU9Twc/H/yZWdtmMbbVWAxaQ63384k+tIIWr+gNcFM3hgQui1uqsOfwOurUp9OJahmfhEf7P0rL8JYA9EnqTCfjBArch5m28HkA1mWt49WNryrbZ9uyA/bPseVgcVuwuGouryizIn0Fo38YrZbgPA1cddVVzJ07V3k9b948brzxRn766Sc2b97M0qVLuf/++2tlaX799deMGjWKrVu3sm3bNuVha7Va6dWrF5s3b2bI0CE8NuMxAC677DI2bNjAtm3b6NixIx9//LFyrP3797No0SJeffVVXnnlFd599122bt3K33//jclk4s8//+TAgQOsX7+erVu3smnTJlasWIGP8hVxqHjI7Nq1i+eee44lS5awbds23nxTKmRzzjnnsHbtWlatX8XocaP56O2PSE1NZerUqdx7771s3bqVIUOGcPfdd3PvvfeyYcMGfvjhB6ZMmQLAU089xYgRI9i8eTPjx49XxHrTpk18+umnrFu3jrVr1/Lhhx+yZcsWAPbt28d1113Hli1bSE1NrXUbt2zZwlVXXcXLL79cq+/1nHPOYcmSJcp+jz/7OMWOYp555hnCw8PZsWMH27dvZ8SIEeTm5TLjvhm8/unrrNm0hu++++6kx6/cj+eee46NGzeyfft2li9fzvbt23G5XEycOJE333yTbdu2sWjRIkwmE1OmTFFKpe7fvx+n03lSIYaqsdaGRo6t2j32Ou3nFb2K0Hl93ora1HUQ4093fsqugl11Om9NbVH+9nmVXAuHp26rtJ1OamUZC4IwGngT0AIfiaL4Yg3b9QXWAhNFUfy+3lrZgJgNZr67+DuyrdnEBcfh66PlvC/XsTJ7If/bP54ZG+8mMSSR+3rfx5NrniS9LD1gcYkcaw4grZbTxtAm4NgF9gJ2F+xmSIq0ru3O/J1kWDLINeeeuQ42ACeyYE8X9bmecd++fbnppptwu92MGzdOEWONRsPEiRMBGD5uOLdNug2f6GPnzp089thjFBcXY7FYAupTX3HFFWi1kidl8ODB3HfffUyaNInLLruMlJQU/vzzT/7880969uwJSDWzDxw4QJ+BfYBAN/WSJUu4/PLLiYmJASAqKgqA9PR0Jk6cSEZmBjaHjZTUlGr7tWjRInbv3q28Li0tpaysjJUrV/LTT5KbcvTo0URGSsuOrly5kvHjxxMSIuVqXHbZZfz9999ccsklpKamMmDAgCrnqNzGiMgIylxlpKWlMXHiRLKysnC5XLRs2fKE34FMeno6d999N3l5ebhcLuJS4rB6rCxatIhvv/1W2S4yMpKffvmJPgP7kJKagk/0KdfnRFTux7x585g9ezYej4esrCx2796NIAgkJibSt29fAMLCwgDpu33mmWeYOXMmn3zyCTfccEOt+lRZjHUN7MCURbiuNRZ8og+D1oDdY5cs4zpObRJFkTc2v8HE9hPpHN25bo2uhOz6N2gMAZax09N46kac1DIWBEELvAtcCHQCrhYEoVMN270E/K++G9nQ6DQ6UswpGLQGjHotdw0Yj6CzcN+K23F5YFziE/jsktDKU6BAuhlz7ZKwyhaz3WNXlnD7Zu833LnkTuVml6dMFXgKzljf/k3U13rGQ4cOZcWKFSQnJzN58mTmzJlTZRuXx4UgCHh8Hm644QbeeecdduzYwYwZMwLOIQsZwEMPPcRHH32E3W5nwIAB7N27F1EUefjhh9m6dStbt27l4MGD3HzzzdUmItXk0pw2bRp33nknazat4YlXn8DpqP4B5PP5WLNmjXKujIwMzGZzjd6CE3kR/PtVeR+5jaIoklaWxvHS49wx7Q7uvPNOduzYwQcffFDrdaWn3jGVm265KWA/eR5x5WvhX3nMv+06nS4gHlzT93PkyBFeeeUVFi9ezPbt2xk7diwOh6PG6x4cHMwFF1zAL7/8wrx587jmmmtq1afGtqiBnCRYF+GS53PL5Yb/iZva7XPjE30U2E/9eShfU71Wj1essIzt3rpZ+6eT2rip+wEHRVE8LIqiC/gWqC6wNg34AWjaZh0wquUwNGjQ6EsoTb+YGT9m8vSPmWgFHemWCjEudBQqo9wsaxYAs7bN4vJfL8fldZFlzcIn+siz5QGqGJ9u6ms942PHjhEXF8ctt9zCzTffrKxL7PP5+P57ySH0+4+/06t/L7w+L2VlZSQmJuJ2u5W1j6vj0KFDdO3alenTp9OnTx/27t3LqFGj+OSTT5RVpDIyMsjNza2YM+wnKueddx7z5s2joEC6fwoLCwEoKSkhOTkZHz5+/fZXpQqV2WymrKxM2X/kyJG88847ymu5FOg555zDvHnSTIE///yToqIiQBqU/Pzzz9hsNqxWKz/99BNDhgw54bXzb6PD4yAjJwOA0pJSkpMlj9Lnn39+wmPIiKJIYVEhkQmSpf7Z558BkoBV7ktRURH9B/Rn4+qNpB9Lxyf6lOvTokUL5TtcvnY5R44coTpKS0sJCQkhPDycnJwc/vhDWvO8Q4cOZGZmsmGDNOOirKxMScibMmUKd911F3379q2VJS63X6YxTG/6J8Ilh08MGslNHZBNXcsELtlIKXCc+vNQHrQaNJKlLl9XeYCxOnM1T6x64pTPcyrUxv+RDKT5vU4H+vtvIAhCMjAeGAH0rbfWNVIijBEMbTYUgPO738DBXAtvLzlIhDeEj3Z8RJnLwv197iPHlqPsk22VLOPNOZspc5exM3+n4sLOseXQPKy5MgIs9BSe4R79O6iv9YyXLVvGzJkz0ev1hIaGKpZxSEgIu3btonfv3hhCDLw8+2U8oodnnnmG/v37k5qaSteuXQME0J833niDpUuXotVq6dSpExdeeCFBQUHs2bOHgQMHAtKUqy+//BJjuBEIjBl37tyZRx99lHPPPRetVkvPnj357LPPePLJJ7niiiuIT4ynQ88OpB+XBowXX3wxl19+Ob/88gtvv/02b731FnfccQfdunXD4/EwdOhQ3n//fWbMmMHVV1/N3LlzOffcc0lMTMRsNtOrVy9uuOEG+vXrB0jC07NnT44ePXrC70Buo6ARaN2pNc+98xzTH5vOFVdcQXJyMgMGDKhREP3xiT5u/+/t/OfG/5DaLJV+/fsp1+Sxxx7jjjvuoEuXLmi1WmbMmMHYS8by5KtPcs8N96AVtCTGJ/LXX38xYcIE5syZQ48ePejYvSMtWreo9nzdu3enZ8+edO7cmVatWjF48GAADAYDc+fOZdq0adjtdkwmE4sWLSI0NJTevXsTFhbGjTfeeNL+yDSG0o3FjmIyLBl0julc4aaug2Us35dykqvH56mzZayIcT1bxqXOUuV9h1fygtz2120A3N/nfsKDGmatb+FkCSuCIFwBjBJFcUr568lAP1EUp/lt8x3wqiiKawVB+Az4rbqYsSAItwK3AsTHx/f2j+mcKhaL5YzODZWvm+yemrnBzpGQD9CGHAJRoKOpA0PMQ5idNxuAfiH9uCb6Gh5MexC36GZs+Fg2WDeQ68nl+pjr6RPShxnpMyj0FtJe3547k+pnbmVDI38v4eHhtGnT5uQ7NGK8Xq8S362OxMREsrIkD0iOOwenz0mULopQbf3fl1avlQJPAUGaIOL1J886BijzllHkkazaJF0SOm3tYpFOpxOtVotOp2PdunXcd999rFq16h+3XUbuA/CPrpNX9JLhyiBEE0K0Plp5rRf0JBoSq2zv8rnIdkuD4ghdBGHasCrb5LpzcYkuUgzVx9XrSlZWFmPGjGHTpk1oNNU7Ig8ePEhJSQkg/V5mW2ZzxCkNRp5NfpZw3ZkXh9+KfmN52XJmNp/Jq1mvctR1lLvi76KtsW2t9i8sK2RG4QwS9YlkubN4IukJviv8jj2OPcTp4ng8+fGTHiPXncszmc9g0ph4uVntEvpqYmnpUn4s+pEYXQw6QafcB1PjptLZ1JlH0h6hzFfGfQn30TIoMF+hvrVl+PDhm0RR7FP5/dr8GtOBZn6vU4DMStv0Ab4tF6YYYIwgCB5RFH/230gUxdnAbIA+ffqIw4YNq237T8qyZcuoz+PVlf6DvKSXdea5hZtYk72cPTGLOaftOZAHyaHJCKECSd2ScB+XRoX5wfmUWSULKTo1mnM7n4vlS8kVWUIJw4YNI8+Wxxd7vmBaz2lK7AWkgcD+ov20i2zX4NMeTob8vezZswez2Sy5q7xugnRBDd20OlObVZvkzwtLC3G6nOgMOszB9b/Sk9fhBQsIGqHWK0k57U6QK8dqNLXeLzs7myuvvBKfz4fBYODjjz+ul9WrPA4PlE8yMAQZMJvqdkyX1wXlEw/MZrPyWhTEattn99ihWPpbb9BX+73kleSBh3rp35w5c3j00Ud57bXXCA+vWVCNRqOSoLds2TJChVAoN0L7D+xPQkjVhMI7Ft/BqBajuKT1JafczupYsWYFjlIHg4cM5s3f3gQXdOjSQUk2PRkLlyyEQogwR5BVmEXvfr3539r/QTbojfpaPav3Fu6FTLD77AwaMuiUpiAd3XkUNoE5xEyZqwzKjfN2ndoxLHUYyfOT2Vu4l+g20QxrHdi2M6UttRHjDUBbQRBaAhnAVUBAJoIoispQws8y/rn+mtn4MRm0tI1uzruXJ3HRLBf5LObznV+hFbS0i+jInqKdbMrZBMDQlKGszlytxJNzbbmUuctw+VzoNXoKPYWIosjvh3/n052fMrzZcHrG9VTO9eWeL3l5w8t8PPJj+iX2a5D+/lOKHEXk2HLoENUBjdC4Z9bVdT1jOa7rz+mI+RUUFDBsxDDcXjeCIBCklQY2ixcvJjo6usb9/L1gdanN3LZtW2XKUn0SkHx2kvY899xzVaYijZ8wnsv/c3nFNC8x8P/KBPS/Bo+gT/TVmABWV6677jquu+66Ou/nn01dU+nI1RmriTHFnDYxll3EVrf1H2VTKzHjcgH1+eoeM/afdlToKKx2UFJbZDe1QWMIdFOXnyMiKAKAoyVH//E5TpWTirEoih5BEO5EypLWAp+IorhLEISp5Z+/f5rbeFYREqRj7k2XMubH2TjJxeuMY8mmRIT4Rby04SWSQ5MZ32Y8K9JXKPvk2HKUuEjH6I5sz9tOvj2f3YXSNJN9hfsUMfaJPt7e8jYAaWVpZ50Ye3weRFHE6/Oi0dYsxj7R1+BifSrrGcsP+9NRhzc6Oppla5eRbc1Gr9HTLqpd7drEPxPj00Xlcokn4tFHH+XRRx8NeM/usXO4+HCVymI1ZlPXogKXf8lQoYEWfa5cLaoybq8bj+ih2FF82togJ21ZPRViXJd5xvJ3ocwz/gfZ1P6lXgvsBackxvL3qtfoA+o4yDFjl1d6b3fhbvYW7qVDVO3yR+qTWgWNRFFcACyo9F61IiyK4g2n3qyzm7gwIy8Of5jfDy6ng2ks3652kJZpJyo6jefPe5SWMRHKtiadiVxbrpJJPTBxINvztrO/aD97CvYAsK9oH5mWTGasnkGJs0T5UcgZ2pVxep3M3DCT6zpdR4GjgI5RHTHqjLVq+6+HfqW5uTk94nr88wtQA6IoBlQX0qOvdjub28bR0qO0jWirJICcbciWwelKwFGmNnFiEQtoUx0s0TOB3AZBEJTwxYHiA7QMb4lJZzrp/pVrbftfC7n609GSo5h0JuJD4gOs4Zqum791fSYGg9VZ6P73THX3j5zVXOwsPm3tkoXQ6rYqwlwXy9hLhSUK/0yM/S3j2mZUy1Z35eeGYhlXcnXLSWlyYZNVGatYlbGKzZM3B4QGzwSN2094FnN+6vm8ft4z3DZoAD/dPogZw2/AlX0lt31+iHUHncQbWwDQKboTuwp28e1eKZltcLKUobkpZ5OyOtT+wv2szFjJ2qy1lDhLeHzA4ySGJJJhyahyXpfXxff7v2fuvrm8teUtrv/jen46WLu6si6vi6fXPM0H2z+ohysQiNFopKCgoFZ1d+UC7mdzJbLTaRn7H79OtakbUVF8qBA8DRrl+xZFUbFSTkblaxAgtuWi6vQ6FREJ8AycwE3t///pRBRFCgoKMBoDB8pe0asIQXVhDrtbEuNSV2mVz2RmbZ3F3L1zFbd7XZHF2OKyKNZjXcRYvr6yKAbUpv4nYlxDRvW8ffNYnrZcef3gigd5bNVjVbZTsqkrCazct8orOJU4S2rVxvpErU19BogINjB5QCr9W0Yx8YM1TP1yE0HxyQRFpdHC3I5NOZv489ifALQKb0WUNopfDv6CiEhzc3MOFB9gf9F+QvQhLJywEEEQ+OPIH0rxEJlMSyYX/XSRcrMX2AsQETleWnWVKX8WHVtEm4g2FDmLcHqdUuJEPZOSkkJ6ejoZRRk4vU5cRleN1rrFbaHUWXrCbRoCh8NR5cFZE7m2XDw+DxpBgzuk/mvxlrnKpEQUqPXM/mJnccWcUa2dHFPOSfY4vRQ7i3F4HAgIFOmKMGqNFDoKcQQ5CNYHn3R/h8dBoaMQraDFk+tRXgP4cnzoNDqyLFnotXqsJmvA58W6YsqMgVPMRFFUvE3y/qcbo9FISkpg5rbb5yZIG4Tb567eMvac3DJecGQBCSEJrMteR4g+hGcGV13E5kTI94m/RVoXN7Uytcmv6Edty2GuzFhJ5+jOilBWboeMXKGrZ1xPzm12LgAHiw9i1Fb9jXp9XjSCJmDdAKgQfJvHxvBmwwnVhzL/8HyKHEXEmGJq2916QRXjM0i7eDNf3zKAdYcLOJAfw7dbu/JDejAXnTOB8e3GoNPoCA8KJ8WQwnb7dvQaPRPaTeD1Ta+zLG0ZrcNbK3Gw5NBk1matDTj+msw1ATf6nkLJzV2TOxukVaXuXXYvveJ6KZmS+fZ88u359Xoz6vV6WrZsyZN7n2Rz7maeO+e5GpNP3tnyDh/s/oAXhrzARa0uqrc2nCrLli1Tsl5Pxj3f30OmVRosbb52c72721/b9Bqf7v4UgI3XblSSuE7EI38/wvzD8wF4MOFBrhtV++SihUcXsrdgL/f0vucftbem9mzO3YxOo6NTdCfOb34+92++n4f6PcSkjpNOuv9vh3/j4c0PE6eLY/Gkxfx++Hce2vwQAN+M/YZWka248ssr6RjVkXkXz+N/R//HA5sfwGww0zGqIx+P+jjgeCXOEiZ+K5UznXvRXDpGd6y3vtYFpaazu3oPkr8Y15RoVuoqJVgfTK4tt1Yu/5rOIRckglNL4PJ3U/tEH6WuUm5aeBOPD3yc7rHSOvQlzhI0gobbF93OHT3uINQgTScSEJQwnj/Z1mzKXGUBVmyBvaDa/ipemPLQg4CU+CiLsdVtpUVYC4akDGH+4fmnNQRQE6qb+gzTMTGMGwa35LlL+/PrLdfidSayf/dIXJZW9IrvBUCkTqooNKnjJEa1kOoY59hyaBXRSjlOcmgyubZc7B47C48uxO6xsyFnAzGmGH4f/zs9YnsorpfKFrTMpzs/5YHl0ko1bp+bzTmb0QnS+Ox0WMdQ4Q5SrLpqkN1vtVlco64cKz2mjPpPJw6vQ7EK5IIv9Yl/Rmpti937P0zlmF5t+fPon3y7r/7qAoB0jYxao/JQlF2jtbXA5HifW5SuhX//rG6rkjUr33OyGEQGRSq1if3xP+/pWkCgNklXXp9XGVxVF+aQ2+nxeaq9VqIoUuosVYTqnwhL5RK9UMeiH2Ily9jPTQ1S6G1f0T5WZqwEpN/8yO9H8uH2DxERKXAUKN9BbHBstW5q+Rkli7HT68TitlDkKKrimveJPnSCTnm+hehDMOqMOLwOPD4PTq+TYH2wklUte1DOJKoYNyCdk8J5dlwXdmSUcO3H63h6/m4W7c6hn+kcJrSdwH+6/4fk0GTaRUrZsm0iKopmJIUmISLy2sbXeHD5g8zcMJMN2RvoE9+H5mHNiQuOU7atTgwsLguvbXqNYF0wcaY4ihxFbM/fzvDmw4HTJ8byQ9B/ekFlFDGu5oF5Kjy5+kku+ukiPtzxYb0etzpcXhetI1oDkGZJQxTFeo0f+wuP3WNnX+G+k4qYfyy2rlOuihxFWN3Weh0g2T12TDqT9FD0OJRBUm0HS7IbUxZjfwG1uC3KgE++j2QxiDRGVtsP/7hhXVcoqg27C3YzdO5Q9hXuO+F2XrFCjKuzjP37WZ3QyuUeS12llLpKKXIU1bmt8nfgL8Z1yqYuj8/LCVwe0RPgtZPLBh8qPgRI18bmsbEsfRkgDVrk8yWFJAW4qbflbSPTksm+Iuk6yteg0F6onMt/sL/wyEK+2PNFgGUcZghT7jv5ew/RhxBpjFTOf6ZRxbiBuaxXCttnjOTy3il8suoIU+Zs5KkVZrZuuYCtx20czLUwrNkwAOXhDtAmUhLmb/d9i0ln4rv935Fry6VvglSN1F+Mi5xFVR5w+4v2A3Bf7/u4pM0lpFvSKXGW0Du+NymhKezKP/Vly/zJs+Wx9PjSCjE+QfKJLNT1+eAvcZbww4EfAAJi6P7Z6fWJw+tQvq8MSwZ/HfuLoXOH1ptV7i+sxc5irv79ar7Z+80J9wmwjOu4NJ/8wMu1nTxAvTZrLdf8fs1J55PaPXaMOiMmrQmHt+6WsbydS5SuhX+M0eq2KveY/LCVByCRxuotY//v5nRYxnsK9iAiLY5xIjw+T8DSg5Xxvz7VibHc7xJnCW6fG5vHVuukOPmc8rXMs9fOTe30Oll4ZGHF9LJyN7UsbnJb5EGGfA0OFh8EUGaOHCmRKo8VOYtweByYdCZiTDEBlvF9S+/jrS1vKYOaUleptKCEn2DLf4uiyFtb3sLj86AVtIoY94jrgVFrxOl1Kt97sC5YKYVZ5Kz7AOZUUcW4EWDUa3nliu6sffg8vrllAKNb6MkucXDNh+s4/7XlpBqG0ydmOJ2juiv7dI7uzAtDXqB/Qn/mXDiHqd2n8kCfB5QYq78YQ1XrWI4nd4jqQFJokvJ+u8h2dI/rzta8rfWafTt7+2zuXnq3MmI9oRjXg2W8PW97wMNVHnxAoIt8yp9TeH3T6//4PNXh9Xnx+Dw0MzdDp9GRUZbB9rztlLnK6qXOLgSKcXpZOm6f+6QFC5xeJ8E6KTGqrpax7LarvF53dWzJ3cKO/B0BD/LqcHgcGHXGKpZxbcVYFky36EYUxQA3qtVtVb5np9eJ2+tWBgdRxihKXaVV7m//ea2nYzUfWYBO9qD3t4yr+5782+kfLz1cfJg8W161IaC6uKr9r7///eo/QPH4PIpwAiw+tpgHVzyorD0su4mbhzUHpL67fW7l/pNXtzteehy3183ugoqlO+X2OrySGEebohVxdXld5NpzOVx8WLGMfaKvym9LtpKPlh5VrruIyOGSwwAMTBpYrWWs1+gxG8z/yJtwqqhi3IhICDcysHU0V7Q3sPCeIbx2ZXfizEE88HUaS/8exYWvr2fj0YpYxkWtLuKjUR/RIaoDd/S4g+s7X69kocYGxwIoSViVk7j2Fe4jMiiSuOC4gPWXW0e0pmdsT/Lt+QErUAHKClP/ZO7sxpyNiIjKj7SuYlzmKuORvx+pVSwnz5bH5D8m89WeihWSZDHuFN1JeTCJosjRkqOKq6y+kC0Ik85EUkgS6ZZ05VrWV2KI/7QveYpbdVPdAvbxugjRS0sC1iVm7BN9ykNfXtzkRMjbVu5rjjWHR1c+qjzU/d3Udo+9zsUl5OOISCEAh9ehFOrwt4zl1/4xYzlO6I+/m/p0WMayKJzMBer1eWttGfuL8d1L72bmxpnV/rbqIi7+Yi8PqDSCJsDz8Nvh37jsl8sUN7Y8SJN/S7JlHKILIdYUy/HS43h8HuX5JP8evKKXo6VHq4hxkaNI8pxojUSbohXLWvbMHC45TFpZGi3CWijXoTrL2L+4ksVtqRDjxIFSroLXgdUjfe9y2yKDIlXLWKWCiGADl/VK4f6R7XB5fUw9tzUmg5ZrPlrH+8sPcSCnDKen5gdqnEmyjHvE9gAkd5D/lKcd+TvoENUBQRAUMY42RhNljKJnvJQxvDV3K3cvuZuXN7yM1+fllj9vYcR3I7hh4Q14fB4WHVvEvH3zqpz7h/0/MPWvqcrDrthRrLijZEqcJTUKa3Vu6g3ZG5h/eD5rMtec9Nqty16HT/QFWMN7C/cSZYxSpnCB9PB1eB0Bq2vVB3K/g7RBJIcmk1GWoQhliat+5i/6W8YnE+MV6Ss4UnIEp89ZIcbVuKlli74yZa4yZfvaXCv5wV9ZjFdnrubXQ78q34vshjRqjf/ITe0vDg6vA4fHQag+FJ1Gh8VlCbAQLW6L0jfZdVrZ8+IvQqdVjE8yIPOIHmV6zomyqf2PJU/LOlh8sF4tY1lsI4IiAgYv+4v24xE9Sp9kkZStZXnQrRE0NDM340ip9L6c6Zxelk6YQVqoY0vuFo6XHSfWFBvQXjmMEW2USrwW2guV+09uS//E/sr2/s+Tp1Y/xcsbXubv9L8Dirfc3OVm4kxxxIfEV2sZg7QqnxozVqnCxL7N2frEBTx0YQd++M8ghrSJ4cU/9nLB6yvoOuNPZvyyk5+3ZLD+SGGA2022jLvGdqVbbDfe3/Y+g78ZzOOrHufCHy/kYPFBusR0ASAxJBEBQYlDt4log1lv5ovdX7AkbQk/H/yZT3d9yvrs9VzY4kK25m3l6z1fM3v7bF7b9Boen1SaT3aFv7f1PVZlruKtzW8BsDl3c5V+bcndwgXfXVBlDrQoisrI3v+hIhdAOV4mbe/yupi1bRbpZelcMf8KtudtV7bdkC2tK+s/ANhXuI/2ke2JMkYpPzT5QZNjzanRJe/2uesUb4OKB4VRayTZnCxZxuVuufoqJuDyuhRXpizC2dbsasX04b8f5tOdn+LyugjVS9NFqnvIP7LyEf674r9V3ve3qmojxvKDv3JfZStLdifWlMBVV8sYKop7GHVGQvWhAQlcEGgZRxmldYUrW5D+YY0TtcHmtgUM9GqDKIrKPVDsLK52qo7MySzj6hK4LG4LTq+TYyXHqhXe2lp6m3M2886Wd6q8H2WMCjiv/LvNskgeN3+LFSosY61GS4o5hcPF0vuym7rQUUj/xP6YdCbm7JaWIL2w5YXK8Z1eJ8XOYkmMTZIYFzgKquQs9Evop1wH/2lNZe4yvt37LZtyN3FuyrnK9vf0vofFVy4GqBIzlsU4KiiKImcRfxz5g0m/T6LUW7MXrz5RxfgsICJY+mFGhRj46Po+/DbtHN6Y2INLeiQxZ+0x7pm7lSs/WMOX6yqErbm5OZM6TuL85ufz3ODnADDrzfx88GcMWgNvj3ibKV2nANJcwO6x3RmYKK2ZqxE0XNPxGiWuXOYq463NbzGs2TBeGvoS/RP68/nuzzlQdACr28o7W95hyNwhTFowCVEUlYfenN1zWGtZy5rMNQHzYOWHocvn4qMdHwX01eF1KPv7uw1lMU4rlUbi3+z9hve2vsez655lb+Fe1mevB6Qf8dpMaf710dKjeHwevD4vh4oP0S6yHRFBETi8Duweu/JAdHgdNYrko38/yl1L76rtVyUdr/yhZdAaSDWnUuwsVqyw+nBT29w2XD4X4QYp2SSjTBJjr+itIpZur5tSVyn59nwsLgthQZI14qGqaB8qPsTmnKoDJ/8HeW3c1DWKcfmcVdmCkac2mXQmHB5H3d3U/pZx+f5GnZH44HiyrFlVLGO3z42AoCTpVE4QrCzGn+38jJv+d1OV836550uu+f2aOs27LXGWUOaW2rMxeyMj5o1ga+7WKtv5RB8iomIZy5ni/tg9dnQaHSH6EOVaywMcl89VbbZ2TZae2+sOuN4/HPiBhUcXVtku0hgZcL3lQbE8l/5klrF8/5v0FXOAU8wp9IjtwbHSY+g0OkWMZcHOtGRKbupyy/jHAz+yLqtikZbwoHDaR7UHyt3U9oIA69rtc+Pxebiy/ZXV9t2oM2Lz2BQjIkRXYRkXOYrYlLOJg8UHCdWcmaV51aIfZxmCINAlOZwuyeGM65nMExd3Iq/MyUM/bOetxQcIDdJic3np0SyC6X2nKwUBlk9cjlf08vSapxnbaiznJJ8TcNwvxnwR8Pr2Hrfj9DqJD47n9U2v4xW93Nf7PgRBYGSLkTyztqKiz8c7peIJubZc1mSuochZxIN9HmRFxgrmZs3FbDEzNGUoG7M3UuQsIjwonEJHIcG6YOYfms89ve+psFb8pjz5uxHlH/+xMkmUFxyRSqXLSUsZlgxsbhtXzL+CTGsmAxMHsiZrDcfLjmPUGnH5XLQIb6G4rIodxeQ7KqyTHFsOEcYIAN7d+i4huhBu6HIDuwt3k16WXmNm94bsDewv2h9QpEKxjHVGzm12Lq9uelX57ERTumpDvj2fMT+Owe6x0yaiDbn2XOWBCNIDzD8HQBbSHFsOhY5CEkOkdX6rs4yLnEUUOAoocZYELLAuW8ZxwXG1s4zLH/yVBx7y4KfQUYhP9EmWcfnD2eGtcBeeimUcpA0iNSyVfUX7Ah7MVrcVj8+jJOhAVTGWY4cmnZTdfTjvMFtyt1QprHGs9BhOr5NsazapYam1aqvsztUIGuX72l+0v0oNeKWm80lixiadiYSQBGUg5h8v3Z6/vco+lS3jxccXk2vL5ffDv7OrYBdbJkurcvknesoeBoPGQLghXBF8r8+rWPny9vJAK60sjYVHF/J6jpQUqRW0NDNXrMArCy1AfHA8Zr2ZNVlr6BzdmdYRrQnRh9A7vjcr0leQZc2iRVgLxTKeu2+ucgwRkfaR7ZV5wcXOYrKsWUSbopVnRZQxCrfPzYDEAVWuh8yx0mO8tOEl6bh+MeNiZzE78nfQNabrGVuwRrWMz3LCjHpax4by39EdyCtzcu/cbTz6007GvrWSq2av5WCuhcN5FrSCHpPOxAtDXqgixNWhETTc3+d+ru10Ldd2upbbut9Gy3BppUy5fjZAQkgCGkHDu+e9C8Anuz4BJPf4kwOfxIePImcRo1qMUlZdkX/UN3e9GY/oUaxaqHAdRhmjqnVTp5WmsSF7g5LwIbtoMy2Zivg+O/hZ7u51NyBlmMoPwmbmZkQGSfHCImdRQPalv8j8uP9HFhxZgE/0kWnJxCt6A9roz0c7PuLNzW8GuLn9Y8byNZM5Vct4W942RazkmJvdY1c8DweLD3L9H9fz51GpvKospIdLpNWNUsxS6cXKWbqiKCoi6p8l63+MDlEd6uSmzrRkBlhZ/mLs78qXrUB5v8piXFOBGH8xdngdSvZtalgq6WXpFDoKlSIPFpdkGeu1esVVL1uqMja3DZPOpFjqefY8PD5PFXe2LEAnS5jzR74H/WsFVDf/X7YoTzTPWBbjVHMqR0uPAoHzgbfnbVcS2TSChlB9aBXL+OMdH/PKhlfYlrcNj8+jnNc/0VNO/uwW2w2TzqR8Z1nWLMV7lWXNwif6yLXnkhCSgFf08ujfFatraQUtLcJbKK/9y5ymhqUqUzF7xffCpDPx+/jfuanLTcq18I8Zy0QaI7m5y81c1eEqzAYzAgIf7fiIrXlbGZg0kDYRbYgIkoyR+3vfX2NZU3kOtIzspm4W1gyn18megj10je1a7b6nA1WMmwh9W0Sx7IFhLH1gGH//dzhPXNSJ/TlljH5jBSNeXc4LC/b842Pf2/te/tP9P8rr5NBkWoS1IM4Ux9ODnubpQU8zNGUoLcJasC5rHQIC7SPbk2JOYVDoIMIMYQxNGUp8cDwAjw94nJGpI7mu03WE6kMDXE/ygzcpJAmr24ooitjcNnJtuYQZwihyFvHYysdIDEkMWOM505LJsrRlmPVmxrQaQ8vwlggI7C/ar4yUm5ubK8k7RY6igAeY/GAsdZWSa88lrSyNXFuu8tBZnbkakBLB5CQyj8/D1tyt2D32gAe7vxgD3N79dkBKkKttApc8+KiMf9ZpbHCsYhm0j2pPqD6U1ze9zubczYoVIbuE5VhyTZax3WNXMrSriHG5VdU+sj0lzhIsLkuVwvoyTq9TSYT66eBPPLj8QSWu6B8zlgVXjhn7t9VfjL/e8zWDvhnEwiNVXacOr0OxWhweKYFLtoy9opd9hfuUKX4WtwW3141Oo1Ms48oib/PYCNYFK2Kcb5Puj8rxXfleybJkkW3NZu7euSedBihnD8t5GkCAR0OmShnJE1jGqWGppFvS8fg8ysBSFh45Z8RsMBMeFM68/fPYmrsVt9eNxWVhb+HegIz8w8WHSS9LDwhDyPHX3vG9lbg+VHipwoPCybRkUuwsxuPzcG3Ha7mu03W0jmhNlFbydLl9bjpEVixH6G8Zd4vtRpeYLtzY+UauaHsFANGm6ADxNelMVeqUZ1gyuK37bVyQegEaQYOISKGjkMHJg7m1663Mu2gei69YzJhWY5jQbkKV6ydzb+97mXPhHOW1/Hsd3WI0Jp0JEZFuMd1q3L++UcW4CdEiJoSWMSE0iwrmpnNasuDuIVzcPYn+LaP4dPVRrpq9hm5P/o87vtrMrswSvlh7jIO5/2wu74N9H+TBfg8yMGkgl7a5FICLW1+MSWfivObnKT+gCVET+GXcL8oydgCDkgfx6rBXMeqM9Invw5rMNUpCiPzATgpNwit6sXlsijANTpIs8kxrJv/t+19ahVeUB023pLM8bTnnJJ+DXqMnWB9M64jW7MjfQVppGgaNgfiQeEW8ipxFSv1traBVHrByoonFbVEKEZj1Zv5O/xtRFHl+3fM8sfoJQEoKk4Un11qRWFJZjKd2n8rGazeSEJJQbWw615bLYysfU9ymO/J2cNFPF7Ho2CIeXfmoEieHiuII8vH7xPcBJJffq8NeVR7essBVntIiu7Arx4z93ZhyEo5MsaMYk86kWDjPrH2GCb9OQBRFrG5rwNSw6mKTufZcRFEMsIzlB7ucTS2/DxVinGPN4YX1LwCwPH155cPi8DgU74DT48TpcRKkC1Jcx5nWTMUbY3Vb8YgndlMX2AsI1ktibPPYqiScZVgyWJu1VrlXMq2ZfL7rc55d9yy/Hvq1Svv8SStLI9YUG7Amb5Yli1xbLufNO48deTuAqpZxdfOM5Sz01LBUPD4PWZYsChwFaASNkvcRZgjDrDcTZggjWB+Mx+fh/mX388jKR7j050urLNbwwPIHuGL+FdVOmesd31uZBgRwsEhKjOyX0I9sa7YSL04OTebBvg8y7+J5TE+czo1dbqRXfC+0Gq0ycNYKFQs1hBnC0Gl03NfnPpqFVbiy5QEzVNzHP1/6M0uvXKqcpzqeGvgUWo0WvVZfZanEpVcuZcH4gFWAiQ2OpWdcTz4c+SFTuk5RQhFmg5lLWl+CgKBaxir1Q2K4idcn9uD9a3vTIjoYq9PLeR3jWbQnh7FvreTxn3dy42fr+Wt3DvmW2iejAAxNGcroFqMD3ru1262sn7Se14dXFNHQCTrF3dU9tjtJIUlKogRIwpxhyWDsT2O5f9n9vLbxNeKD4+kWK41IB3w9gOfWPYdOo+P6LtfTMaojzwx+hvNTz1dcriBZfkXOIkY0H6G81y22G9vztnO87DjJ5mQ0giag3F2+PZ+44Dhig2PZnr8dn+gLEKJ12ZLFfnn7y8m0ZnLAeYBtedvItmZjc9vYmLNR2dbffSsXn5AfJIIgFaUPDwqvVox/P/w7vxz6RREc2fp9Yf0L/HroV6b/PR23Typs4W8ZB2mD6JMgibHT62RQ0iCWXLmEoSlDFQuncqxQLvBS2TL2F9HKYlzoKCQiKELxbCxPX06GJYNcWy7vb3ufK+dfWeN0JpAsyzJ3mTJIKXBUbxnL2D12RFFUBmEh+hA25Wzilj9vYVPOJmU7h9ehDK4UN7XWpMw9BUgMlWYKWN1W3F63NFDTBaMRNIo345Odn3D9H9ezNG0pQ1OGYtQaA7wi+fZ8dhfsZvQPo7nlz1sUwcqyZCnf2asbX63iXvd/nVaWFhAmAUnMV2WsIteeq8w4kGPGipv6BJaxPDg6WnqUAnsBkUGRjGszDpDCFWaDJMbPDH6GXnG9yLXnsixtGbl2STxv7HyjEnI6VHKoylQv+TfYPbY7Rp1Rua9XpK+gdXhrusd2x+K2sOT4EqDCGgcI1gZzX+/7FOtabpfsyZBzRKrDbDArzwxZvFtHtCbGFMPnoz/nk1GfBGz/6rmv8sH5HyiD/eqIMcUECL4/AxIHKGEtmXt63cOHIz88oys3qWL8LyAyxMDi+4cxf9o5vD6xB39PH87z47vy4mVdySx2cMucjYx7dxUHcy0s35/H+iOFrDqYT3pR/S6ocHHri/nf5f8LWMbs8raX88EFH3Bzl5tZnbkau8fO68NeD0gg2pa3jQltJ9A5ujPzLp6n/LBTQiUxbh1eUSZUrq0N0C2mG6WuUlZmrKS5WaoEZDaY0QpaJWYcY4phUodJrMtax4fbPwyYDiW73K/pcA0aQcO8gnmK5XKs9BibczYrDxvZOvCJPsWCqLySUrihejGWV9+S/5fbkGvLJUgbxI78HXy3Typ3WuAoUOKBBq1BsYzlB2V4UDiJIYnK4MB/7qVW0BJtjEYn6ALEWBRFRbSTQ5OrZONmWbNIDElUxFh2UR8oPsCazDW4fC4lNiyLsX/yVIG9QLGKQ/WhHCw+qCQAyhW4/NvoE324fC7FjTum5RiyrFmszVrLpzs/Vba1e+zKfeL0OiU3tS5IScQDmNxpMqH6UGVqk06jQxAEQvQhipv6i91fsDl3Mx2jOnJPr3sw6oyKKxYkb011a+SuyVpDWlka5zc/nyJnkRLKACms0e+rfkz9ayoOj4O0sjRSzClK20L0IeTacpUBnzzwWFkmLZzQLqqd0i+Pz8NTa55iXdY63t/2PptyNimWMUhu4wJ7AdGmaKV0bqg+lEhjJJHGSDpHd+be3vcCFRnoEUER3Nv7Xl4fVnP1uZlDZ/LLpb8QrA/GqDVK84pL09iYs5ERzUcwJGUI4UHhzNo2i+TQZNpGtK3xWOPbjOf989/nsraXASjetOrQCBqeGCB5nypb8L3iewVUDAQY2WIkg5IH1Xi8f0KoIVSZw3ymULOp/4XEmY1c018Spw6JYaQX2Xjohx2c/1qgK7B5VDAL7h7C4TwLRr2WdvHmem+LXqtnUNIgBiUN4p7e9yiZq7JgP9b/MXbk72Bq96lV9pXdVX0S+nCo5BBtI9sGCKDsYnJ6nUpGpzzV4rt931HiKqFvQl+u73w9uwt28/7290kJTSE1LJVjpcc4WHyQuOA4EkIS6JfQL2DJysMlh9mcu5lhzYbxx5E/yLHl8OrGV/ly95eKa7GKGAeFc7zsOHcsvoNnBz9LpDESp9epWHtrs9YiimKAZXpp60s5WHyQj3d8TJRJsia6xnRle/52DBoD7SLbcVfPu5TVvUDKei52FvProV/ZkrtFeT/aFK248bx4KXIUceeSO2kX2Y5ecdKKYYOTBjNv/7yAJTQzLZn0iu9VpcTqxuyNSknCXw7+wsT2ExVRNxvMips3356vxF/bRbZjc+5mpc9B2qCAeGukMZJ8ez52t12JNY9uMZrv9n8HBGbYOzx+lrHHIc0zLnd5v3fee8QFx9E+qj0hhhCl6Ie8ilCYIQyLy0K2NZt8ez7T+07nmo7SoMuoMwYMmj7Z+QmFjkJeGPICD//9MCCtOy5/T/f2vpf12etZcHgB0cZousd2V/IKVmWukqxRWy7NzM2UeGjfhL4sS1vGH0f+AKTM6tv+uo01JWsY03IMI5qNICkkiY3ZG4k2RfP9/u+Zf2h+QOJbZFAkYYYwdhfspsAhDSwNWgNfj/maUEMoNo+NII10D3aK7oRBY8Dlc/FY/8fQaDQIgoBJZyLKGBUwaDuv+XksPr6YUEOosozh4OTBvL/tfa7941q8opcRzUfQKrwV31/8PUuOL+HSNpeecA1qQRAUK3zuRXPpENWhxm1BGlS/NfwtOsd0PuF2TQlVjP/l9GgWQY9mEfRqHsmCHVkkhpsQBMi3OHny113cOmcjm48XYdRrWXj3UBLCqy7cXZ/IcZtO0Z3YOnkrWo2WiUysdtsOUR2Y2H4iN3e5meTQ5CprH7cOb02vuF6YDWau73y98v5rw17j9sW3MzhmMHf0uANBEHio/0OszlrN8bLjTO87XYlVtgyTsqFfHPIir/75Kn079WXG6hksOb6EYmcxAxMHsi5rHT8c+IFsazYtwlooGa6VxVie47sifQWvbHyF5855jk3Zm3B6nZybci7L05fzxe4vOFh8kE7RndhbuJfRLUfjE31M+XMK72x5B42gYXDyYLbnb5fmzQoCt3S7JeA8sgX76EopqzXOFEeuPVepyqbT6CjxlnDLn7ewr2gfewv2Kp+dk3wO8/bPY1f+LtpFtqPUVUqOLYek0CSC9cH8v737Do+yShs//j3TUie9hwSSEAKhEwhIbxYERBRcZEV0seuqP3d1dXdt+7quZfd9V9e1i13Bgg0FVAQVlSYlBCEEAklIIZDeJ8mc3x8zGRJIICgwSbw/15UrM888M3NOTpJ7znnOuY/VYnX1KJs3prgo4SI+3vcxD/7woOufbMst7I7UHnEtt2kOxs2OvY4e7BnsCMaNteRX5xPqFcrQ8KHMSphFTmUOu0t2U2GrwGq2Ut9U3yqxf4WtwjUjtnlvbnB8aFudsxovk1er4P1J1ieu/Z1TwlNcQ6gt98NVKErqSujl14vpcdNdwTgpKIms8iyu7n81sX6xjO8xnuVZy/k8+3NGRo6kpqGGvkF9ySrL4qN9HwGO2fzDwobxt9F/I8QrhLW5a11b+20/vB2AidaJ3DvqXpRSTIqdxJu73mRP6R5CvUJb5fr2NHmilGJqz6ms2L8CkzIxpecUgDavc1qMFoaFD6PSVslv+rb+e4r2jaakroTEwEQOVh5ss7c8IGQA951zH4vTFzOv7zz6B/d3td/8fvOPO/9EkoOTO3Rey1GuXwMJxgKAqAAvrhkX3+qYUor7P0on0NtCja2Juc99zw0TEpg9NJqSahsmgwFfTxO1tiZCrSff3P5UtRzObovZaOavoxzDh1cPuLrN57867dXjjicGJrLykpUYlMEV/IM8g3hj2hsYlIFYv1je3PUmOZU53DniTsDRqzzP/zwmJk7kubTn+DzbsXQoJTyFcO9wdpXsoqdfT1447wXOfe9cgOOuhTZvVNDLrxcf7/uYeUnzeGv3WwR6BPLg6Af587o/8/jmxwH43YDf8fy5z+Pv4Y/WmjCvMLIrsknwT3BNAmovv/ex184ifSMpqy9z9WzNBjObqzdjqbVwy5BbeGrbU7yX+R5GZSQ1MhWDMrC+YD23fHWL6zWaRyHCvcOptFViUiZqGmuwWqw8cM4DBHsF83L6y4R5hxHuHc7wiOEcqDiAl8mLzNJMVuxfwdjosa5gfd8593Fh3IX4mH1aZTgL9gqGUscQdEFVAZG+kZgNZh4a+xDLMpdx//f3M+btMcztM5f6pnpX7/3Nn96kvqn+uHkMAA+PfZgFKxZQVFPkGj4fGDLQtV0f4NqmtPl3oVmUbxR5VXmMjR6LUop7R93L0oyl3D7sdkZEjODSRMds3cv7Xk5hdSGDQgexON1xTfO3/X6Lr9nXNXwd5x+H0WBkduJsKm2VTIqZhNVixcvkxdKMpXgaPbko8CJXb3RizETe3PUm+dX5PDnpSawWK7tLdvPopkddS6Xm9pnLssxlmJSJK/pd0ebvQ7NHxz963D6/4Ljcc6D8AA+NeYis8qxWa6pbmp04m9mJs0/4HuLnk2As2rVgVE8GRvvj62GiqKKOR1bu5i8fpHPvh+nYNQR6m/H3MlNSbePNa0YxsIf/yV+0k2gr0LdcD/nsuc8CtEpY0KynX0/yqvKI9o0mxhrjSrhwed/LW82WPXZG5/x+8/Hz8GNun7lcuOxC/rb+b+wu2c3NQ24m2CuYZ6c+y02rb2Jd3joSAhJcvT6lFBNiJvDunndJDk52zSBuNxh7tw7GByoOMDJyJMPCHUPRzUOSNw65kUUDFvHB3g/Iq8ojyDMIH7MPvfx68cauN1q9RvN1unCfcPaW7eXK/lfyys5X+M/k/2A2mrllyC2s2r+K/Op87k69m0sTL2VWwiye3vY0PxQ4hmz/OPyPxPrFEuIVwvge413/9C1GC8PChrGl6Og1+Ad/eJAdR3YwJXaKqwwthzbf3fMuJmXiksRLWJy+mKLaIkZGjGyzVxjlG8XU2Km8tfstV+/37+P+jtaal9NfxqAMrX4fFiYvPG5pWHPiiMuSLnNldJrbZ67rOYNCB/HyBS+jtWZd3jr2lO5hYMhAfM2+bD60mVGRo+gX1M91vtVi5cnJjnSxn2Z9ytKMpQwNG4pZmV3npISnMD1+OhNjJrp6ibF+sTy66VHXtfn+wf2ZET+DASEDXNmo2tPepKkbhtzA7MTZJAcnd7jXKk4/mcAlTmhITAC9w3wZ3TuEj24ew5LrRvH7yYnceX4SBqUorKjD18PElYs3sGRjDm+sz6a6/tS25+uMYqwxbQZigGsHXsv1g67nhXNfQCnlSuQwM2Em4MheFuAR4Lo+2SzCJ4JrBl6Dv4c/VyRfwe6S3fQO6M3lfS8HHEH3yUlP8u9J/3YtUWnWPDEnOTjZFaTby+Z1bDAO9w7n6alPtxqqB5gZPxOlFAuSFwBHA8/C/guZGjuV6wdd7zo32sfRM+7h24NAj0BuG3YbWxdsJSU8BXAE1D+l/snVW/Q0eTIkbIir5xrnH0dCQAJmg5kJMROO6301D7U3/yy3FG2hwd7gWhcNkBiQyMjIkdw27DYUipkJM10T8wDuSj0+r3az5kxXzdd5/Sx++Hv4c3vK7dw6rHW60xi/GB4c/SBX9LvC9eGgedb6ySil+N2A32E2mBkWNowpsVOI8IngrhF3tdvjbJ6IdeyEIbPBzCPjHmnV2w/zDuPu1LtdQ8lKKf4x7h+tMsCdqnj/eM6JOufkJ4ozSnrGosOUUoyKD2ZUvGMSyqwhUdTYmrAYDcx++jvuXuZYK/mfrzJ5ePZAjAbFqgMNjGmyU1HbQLDv6R/KdofhEcNb/XN+dNyjFNcVu3qsNw6+sVWSlLZcM/AaBocOZkTEiFZB22w0t+oNNjsn6hxuHXorM+JnYDaaCfAI4PrB1x93HjiyHPUL6ufqMbW3LrN5OHtOnzk8svERV2aqSxIv4ZLES2i0N/LaT69R31Tv6vHfNOQmLku6rM0UgZNjJ7daWga4PjiMjDjxzNSx0WNZN28d+8v381zac63q0sxsNPPieY5c5iMiRtA7oDdKKeYGzWX2qNmthpqP1bzOtb1kJcdqnvU7p88cMssyXdeiO2J6/HQm9JiAr8WXSN9IvpjzxQnPb57FfWnipWwr3nbS1/8lgVd0XhKMxc/WI/DoP8ol153D/iNVBPt6cM+yHSx69ega3LWPraGspoGPbhlDn3ArdrumpMZGSDcJzgGeAa2W03SE2eCYRX4q57ecqPXtvG9PeP47M4/f2rLZiktWsHXj0VnWHkYPR0KEYzpuJoOJQaGDyK7Ixmx0fGAI8gw64RrRYzUvserIrFh/D39X0L879W4OVh50LWM71uDQwa7b463jT5qcoeXlg1OREJBAQkDCyU88RvN1344wGowsGrjolN9DdC8SjMVpkRRhJSnCsfTp01vHsnx7ARV1DazYlEF6SQOeZgM3vP4jj88dxNNr9vFN5mH+OXcwFbUNXJ4ai8koV0zOlh7WHuw1tt5fur2ECHcOv/MXbbR+7cBrKa4tbrO335YInwi2XLHFFfxPp1cueKVVOkYhOhMJxuK08zAZuTTFkZCjl+0AI0aPY2deOTe88SOXPuOYzBPgbea2JdsA2JJTxl+n9+s2w9jdyckmBZ1Mv+B+bc5oP5EzEYgB1/VtITojCcbijFJK4ethYmR8MKv+33h+2FdMQqgvWsNbG3Pw8zTx3DdZLE/LZ1xiKOF+HpiNBm6a2PuMr2kWQojOQoKxOGvCrJ7MGnJ0MtE/ejiu810yrAfvbM5lze4i0g6WU1HXwAdb8ogN9qZ/lB93nt8Xb4uRz38qZHLfcPy9zkzPSQgh3EWCsXC7pAgr985I5t4ZjjWOmYcqef6bLIoq6/l4ez7f7S2m0W7nUEU9wT4WnluQQmK4lcOVdcSH+GIwtL1kRAghugoJxqLTSQy38vhcx2zZzQdKePizXQR4W/jzhVE88WUm81/YgK3JkUloWGwAsUHejEsM5eKh0RglMAshuiAJxqJTG94riGU3jXHdHxkXzBOr9xDp74WX2cjL3+0np6SGD7flsyajiCExAXySVsC43iEE+lg4v394qyVYQgjRGUkwFl1KhL8n/7hkkOv+tePj0Vrz3zV7+efne1ieVkBciA9PrXEs3Vmzu4jXF6WilEJrzWs/ZNMn3Mo5CcHuqoIQQhxHgrHo8pRS3DI5kWGxgUT4exIf6suuggpWphfyxOpMrnl1MweKqwm1erA+qwSDgnum9eOz9ALG9g7hD+f9suU7QgjxS0kwFt3G6N4hrtv9Iv1ICPUlo7CSjEOVRAd4sT6rmMl9w1DA3z/bBUDawXJSegYyMi4Ys1E591KW685CiLNLgrHotiwmA88uOJro4UhVPQFeZuwaHl+1m14hPjy2MoOrXt5EqNUDW6Mdb4uRvhFWArwtLBzdiyExAQDYGu3YmrSbaiKE6O4kGItfjZa5sP8y3bGMalJSGD9ml7J0Uy6eZgNV9Y0UlNexNbeMT9MKGBEXiEEp9hZVYbbbGDm6ger6RqICvNp7GyGEOGUSjMWvWlSAF1EBXswcHNXqeHlNA7ct3UpOcQ1KgdloIKfczoTHHZteTEwK5f6Z/YkL6fhuPkII0R4JxkK0wd/bzCtXp7ruN9k1Y/++guK6JhaNjeOdzbnMePJbLh4aTZNd0yPQi2pbEwBbc0p5+apUzEZFSY2NMKuk9RRCnJgEYyE6wGhQ3JHiSf8hw0mO8uPacfFc//pmlm7KxcNkoNrWhEGB3XlZ+YGPd5JZVMmOvHKemDeUqrpGfiqo4P9N7YO/t6TzFEK0JsFYiA4K9TaQHOUHONY7f3jzGOob7TTZNZV1jZTW2Nh3uIrVu4pYujkXb4uR6AAvbnpzi+s11mYUcfOk3kxMCqOoso5ewT6szypmUlKYpPUU4ldMgrEQP5NSCk+zEQAfDxMR/p70i/Rjar9wrhrdi7hQHxqbNGt2FxEf6oOt0c5fP0znzvfSXK8R6G2mtKaB/5nVn3MSgrln2Q4uGdaDuSk9ZI9nIX5FJBgLcZp5mo0Mdi6JAlx7OwN87txGctOBUhrtdj7YmkeEvxePrcwgLtSHHXnlbDpQyuJ1+/nzhf2YmBSKUoqGJjtmCc5CdFsSjIU4i5RSjO4d4kpQ8ofzksgpruG3L60n7WA5f5vVnzCrB4+s2M3Vr2zinPhgkiKsvLUxh1sn9ybS34vUuCCKKusI9/OUvNtCdBMSjIVws9hgb1bdPp4NWSVM6BOKwaCY3DectzZk88zX+/ghq5iewd788/M9xz33kmHRoCEhzJf5qbHsyCuntMbGucnheFvkz1uIrkL+WoXoBLwtJib1DXPdt5gMXDUmjoWje1FV34iHycjbG3NIDPNl/f4SEkJ9SDtYzkvr9uNjMVK9tYn/+2IPjc7p3Bf0j2iVfWxlegF+XmZGJ4Qc995CCPeTYCxEJ6aUwurpWAq1cHQv4GgO7llDopk7vAexQd5kF9fw+vpsBkb7k11cw7Nf7+PWt7cyOiEYL4uR25ZsA2BCn1BunZKI1pLaU4jOpEPBWCl1AfAEYARe1Fo/cszjvwX+5LxbBdyotd5+OgsqhDhe3wjHUqt+kX48PHsgANX1jXyyPZ+vdhfx8fZ8AAZG+zNzcCRPfbWXS5/5Hn8PxYB967lseAwXDY7i9fXZeJmNJEf50dikGRwTQEVdA7W2JsL9JGmJEGfaSYOxUsoI/Bc4FzgIbFJKfay1/qnFafuBCVrrUqXUNOB5YOSZKLAQ4sR8PEx8e9cklILVu4qotjUyqW8Yfp5mLk+NZWV6Icu+30lBWR13vLOdpZty+X5fMQAW54ztp+YP5cmvMtlTWMW4xBCSIqzceX4SDU2ajMJKfioo5zcjYlu9b5Ndy45XQvxMHekZpwJ7tdZZAEqpJcAswBWMtdbftzh/PdADIYTbNCcQmZoc3uq41dPM3OExhFbtI2XUGGY//T078yv40wV9+X7fEeoamqhtaOK6138EYGzvEHYXVrJ6dxEr0wupa2jCaFTkltSS0jOQ3mFWAJ5cncmSjTmsuH08/l6SYUyIU6VOdu1IKTUHuEBrfY3z/gJgpNb6lnbO/yPQt/n8Yx67DrgOIDw8PGXJkiW/sPhHVVVV4evre9pez52kLp1Td6yLrUk7NsIwKNd1ZFsTvJ9pw6BgXl8P7Frz8IY6sivsGBTUN4FRwcAQIzPizaDg4Q112DXMS7JwQdzRYKy1Rqkz21vuju3S1XWXesDpr8ukSZN+1FoPP/Z4R3rGbf0ltRnBlVKTgEXA2LYe11o/j2MIm+HDh+uJEyd24O07Zu3atZzO13MnqUvn9Guqy/lTW98fNaaRitpGcktr2HOoksxDVbzy/QG2HW7Cy2wk0t+LEF8Ln2ZXU2ryZUq/cB5ZsRsFPHNFCqlxQW6rS1fSXerSXeoBZ68uHQnGB4GYFvd7APnHnqSUGgS8CEzTWhefnuIJIToDb4sJb4sj5eeIXkE0NtlZcE5PXv8hm437S3huQQqFFXU88WUmq3cVsWrnIZIj/ahrbOLKxRv4zfAYTEYDg3r4sy23jNumJGL1NLM+qxgPk4GUnoFnvActRGfWkWC8CUhUSsUBecA8YH7LE5RSscAyYIHW+vjMBEKIbsVkNJAQ6ssDF/V3HYsJ8uaNa0byzqZcFn+3nxcXDsdsNHDfR+m8tj4bs8GArckOwJbsUnw9TXy31/G5ffqgSO6fkcxL6/ZTVFnPiF5BXDw0Cm+Lie25ZSRFWGm0a3w9ZDWm6J5O+puttW5USt0CrMKxtGmx1nqnUuoG5+PPAvcBwcDTzk+3jW2NiQshur/LRsRw2Yijg2nPXJFCY5OdalsTaQfLKKm2cc+yHWgND8xMptrWxD8/z+CLnYdo0ppgHwsfbM3j31/uYeHoXjy+KoPBMQHsyq/gtqmJXDw0mhe+yeKCARGMig9ma1EjSeW1RPp7ubHWQvwyHfqYqbX+DPjsmGPPtrh9DXDchC0hhABHT9rfy8C4xFAALhochdZHZ333jbDyyIrd/OmCvkzpF8amA6Xc8c42Hl+VgZfZyPbcMsxGxRNfZvLk6kzqG+28tTGHmYOieH9LPasPbebDm8bITleiy5IxHyHEWaeUouUl4in9wpnS7+gyrNS4IJ5fMJzrXt/MfTOS8TA79oa+6uWNDIkJ4MaJCTy2MoP3txwkwkeRnlfB7Uu3YfU00y/SSpjVk36RVnoG+5zxutjtmpIaGyG+Hmf8vUT3JcFYCNEpJUf5OZOXHI3a6/402XX71d+lkl1cTeb2jaQ1RvHM1/tQSmFrdFyXDvfz4PE5gzEZFL6eJoqrbMQEebM1pxRbk515I2LZklOKrdHOmN4/P2f30s25PPjJTn64ewqBPpafX+FTcKSqHl8Pk2s/bdH1STAWQnRaJ5th3TPYh/0GxR3nJbFoXDweJgObD5RSXF3PXz5I58rFG1udbzQompybabz2fTZZR6oc21omBAPw7BUprM8qxstsZGR8cKvn2u3aNaze0updRdQ12NmRV874PqG/pLodorVm+pPf8pvhMdxxXtIZfz9xdkgwFkJ0C82Zv8YmOnq5qXFB7D9cjcGgKK9twMdi4qNteQR4mxkQ7c//fbGH5Eg/ckpqWJtxGIAxj3xFcbUNcFzXntw3jHA/T77NPMxnOwp478bRVNU10ivEh1pbE3uLqtiw3zEjPD3/7ATj8toGDlXUs6uw8oy/lzh7JBgLIbqlSH+v42ZYNwdqODqJbFdhBZV1jfx3zV525JXz3/nD2FVQwUvr9rs22lAKtIaJj6+lqr6RlJ6BAPyYXep6vZ15FceVQWvNzvwKBkT7n7Z6HSytBSC3pOa0vaZwPwnGQohfpeZJZP2jHIFyaGwATXaNt8XE9EGRXD8hnvyyOp7/JosdeWUM7hHA8rQCbpiQwPtbDnK4sp5AbzOlNQ2k9gpiw/4SjlTV42k28tmOAvy9zByqqOO+j3ay5LpRjIoPpqKuAV+Lqc3h7o5qDsZ5pbVnJd2oODskGAshBOBhaj0ZyuppJinCzL8uG4zWGq3h/ov64+th4uZJCRwsraV3mC+F5XV8kpbPxpUlDH/oS+JCfNh/pBoAkzPovrM5l0/TCnh9fTb+Xmaemj+Usb1DsGv4Zs9hxvQOoaTa1qF9pvPKHMG4sr6R8toGArzPzqQxcWZJMBZCiJNo7kU3ZwBzLKFyXKOOCfJm3ohYLEbH5LHPfyrkX3MHszW3lDfW55AQ6sOyLXkAXJ4ay9acUq56eRMeJgOpcUGszTjMtAERrNpZyJXJFhJLa4j096KuwZH32+CcdNa8PeXB0qPD07kltVhMBixGg6yx7uIkGAshxC8U5GPhmnHxLBqrKatpINDHwuyh0Vw/3tGDXrh4I/fNTOaKUT0pqqzjmbX7SDtYztqMwxgUrEgvBGBZpo1XH1vDHVP78Nw3WXhbjPQK9mFbbhkDov148KIB5JXWYjYqGpo0K3cW8MZLOUwfFMnDswe6+acgfgkJxkIIcZoopVxrjQ0GRUyQNzFB3ux48DzXMHiY1ZP7Z/anxtbIWxtyCPfz5Pdvb2VAtB/pzklgT36VSUOTZmRcEGW1DcweGs2ajCJmP/0djXZNSs9Afswu5b9r9gGwbMtB7pnWF6unmazDVfh6mgjx8UApWLYlD6NBcfHQaPf8UESHSDAWQogz7Njr0eDYCeuacfEAhFo9GBjtz52vfIXBGsLytAJig7x5ceFw1wSt8poG/vDudr7cdYjYIG8OltZQXtvAY3MGc+vbW1m6KZep/cKZ+Z919A634u9lxqBgfVYxJoOBKf3CXDtlVdU1MjU5/Lgyvbkhm5Xphbz2u1SZGHaWSTAWQgg3G+VMMHJZkoWQxASWpxUwbUBEq4Do723muQUpvPL9Acb2DuGBmf3xshgxGxUvf7efhz7dxaMrd9PQpNmeW3bMO9i5fck2kqP8eOHbLBSK9fdMwd/b3Oqs13/IZndhJZsOlJ7RPajF8SQYCyFEJ9I/yo8n5g1hQhsJRIwGxaKxcccdf/vaUbyzOZfckhrG9wnl2tc24+thYkhMAME+Hhwsq2FNRhGrdxdh9TRRWdfI75dsxc/ThNloICHUh+mDotjtTCTy2g8H2JpTyp5DVZzfP5ylm3J5bM4ggp35t/PKann/x4PcPKm3a2JZV/L6Dwf4es9hXlw4wt1FcZFgLIQQnYhSillDTu36rqfZyJXn9HLdf/TSQfh5mZmUFIbWGluTncYmTUm1DaNB8Yd3tvPNnsP0DPamodHOB1vzeGtDDgDjEh3D5MvTCgD4aFsejXbN5S+s57zkCPYcqqTRrvlqdxEhvh7kltZw3bj44/JyV9Y1YPU0d8q10N/tLWZNxuFWs9TdTYKxEEJ0My2DuVIKD5MRDxP4OJdmPXPFMCpqG4kN9gbgxW+zeO/Hg4zpHcIDF/Vnw/5ifCwmnlqzl28zj3DXBUm89n02T63Z68pGBvDnD3YA8P6PB0mKsHLfjGQ2Z5fy5U/1XLXyc2YMiuTrPYcJ8fVgUA9/7pnWjwh/z+PKe7iynmAfyy9KhtJsZ345WnPCrGeHKutosmuKKus6zT7YEoyFEOJXJsDb0ipZyDXj4l2TyQAm93VM7kqKsJJRWMnI+GAWjY2juMrG2ozDvLQui76RfnyaVsCclB6UVttYn1XMtCe+pdG5EUdCqA/L0wpIjQsiyNvCqp2FFJbX8da1o7BrzTubc5nQJ5TGJs15//6Gu85P4uoxcTy5OpNzk8PZmV/OqPjgU94G888fpNNkt7P89+PaPedQeR0A+WUSjIUQQnRyAd4W1+5VHiYjUQFezB8Zy/yRseSX1RIX7MOtUxKxmAx8sj2fv3ywg/+5eAB1eRnMmTaBrTmlDI0NxGhQvLs5lzvfS+Pvn+6irMbGsq15WD1NJIVbsTXaefm7AwA8sTqTF77NosbWxPCegby0cAS+nqZWw8mrdx0i0t+L5Cg/17HtuWVYTAb2FVVha7Rja7SzPC2fugY780fGus6z2zVFlfUAFJTXAoFn/gfZARKMhRBCnLKoAC/+eP7RLRxnDo5i+sBIDAbF2rJMjAbF8F5HZ2TPSenBttwyFn+3H4BFY+PYklPK5uxSEkJ92He4moc+3UX/KD92F1YSZvVgc3YpKQ99QWpcEP972RC8LEaq6hu54Y0fGdQjgHevP4e/fpROj0AvFq87gJfFQFV9I+BIQfrXD9MBuDw1xnXduqTG5uq9F5TVnZWfVUdIMBZCCHFanOiar1KKBy/qz+CYAJIj/RgQ7U9Dk51VOwsZ2zuEhz/bRYivB9eOi6estoFQqwfXvroZb4uRtXsOM+ofqwFHStKGJs2P2aXc//FO18SzYzUHYoB9h6vpHeYLQGH50QCcX1573PM+21HAS+v28/yCFOwnTxV+2kgwFkIIcVaYjAYuGx7jum82GpgxKAqAx+YMdh1vnpn99nWjAMekrM0HSqmsa+CN9TmMSwxhRXohr6/PZmJSKN/vLcbWZD/u/W6YkMCzX+9j1c5CLMYo/L3NpB0sBxzbYuaXHQ3Gn6YVsHRzLgdLa8g6XM0Dn/zE8rR8/jXh7FxTlmAshBCiU+sf5e/a6vLmSb0BuPv9HXiaDfx1RjIr0gupqW/k7mWOY3UNjsB8+9REnv16H4+vyuCJ1Zn4eZo4UmUDICHUlwJnL7nW1sQDn+zksPNaMsAn2/NJCrcS5Hl8kD8TJBgLIYToMpqv/T46Z5Dr2EWDHb3r//tyD4HeFv5z+VAa7RpPs5GLh0Tx4bZ8UnsFcaC42vWcoTEBfLA1j3P/92vyymqpsTUxf2Qs+4qqOFxZT9aRas7rHw4UnJV6STAWQgjRLdw2pQ9moyIx3Oo69silg3jk0kF4mo1orYm75zMAbp2SiJfFSF5pLWN6hxAf6uNKnHL/R+lkHanm/P4RHMmUYCyEEEJ0WMslTM08zUc36VBK8d3dkzlcWU9MkDd/mzWgzde5ekwckQFe9I/y4+vMM1bcViQYCyGE+NWIDvAiOuDEk7J6hfhww4SEs1QiB8NZfTchhBBCHEeCsRBCCOFmEoyFEEIIN5NgLIQQQriZBGMhhBDCzSQYCyGEEG4mwVgIIYRwMwnGQgghhJtJMBZCCCHcTIKxEEII4WYSjIUQQgg3k2AshBBCuJkEYyGEEMLNJBgLIYQQbibBWAghhHAzCcZCCCGEm0kwFkIIIdxMgrEQQgjhZhKMhRBCCDeTYCyEEEK4WYeCsVLqAqVUhlJqr1Lq7jYeV0qpJ52Ppymlhp3+ogohhBDd00mDsVLKCPwXmAYkA5crpZKPOW0akOj8ug545jSXUwghhOi2OtIzTgX2aq2ztNY2YAkw65hzZgGvaYf1QIBSKvI0l1UIIYToljoSjKOB3Bb3DzqPneo5QgghhGiDqQPnqDaO6Z9xDkqp63AMYwNUKaUyOvD+HRUCHDmNr+dOUpfOSerSOUldOp/uUg84/XXp2dbBjgTjg0BMi/s9gPyfcQ5a6+eB5zvwnqdMKbVZaz38TLz22SZ16ZykLp2T1KXz6S71gLNXl44MU28CEpVScUopCzAP+PiYcz4GrnTOqh4FlGutC05zWYUQQohu6aQ9Y611o1LqFmAVYAQWa613KqVucD7+LPAZcCGwF6gBrj5zRRZCCCG6l44MU6O1/gxHwG157NkWtzVw8+kt2ik7I8PfbiJ16ZykLp2T1KXz6S71gLNUF+WIo0IIIYRwF0mHKYQQQrhZtwjGJ0vX2dkppQ4opXYopbYppTY7jwUppb5QSmU6vwe6u5xtUUotVkoVKaXSWxxrt+xKqXuc7ZShlDrfPaVuWzt1eUApledsm21KqQtbPNYp66KUilFKrVFK7VJK7VRK3eY83uXa5QR16Yrt4qmU2qiU2u6sy4PO412xXdqrS5drF3BkmlRKbVVKLXfeP/ttorXu0l84JpXtA+IBC7AdSHZ3uU6xDgeAkGOOPQbc7bx9N/Cou8vZTtnHA8OA9JOVHUc61e2ABxDnbDeju+twkro8APyxjXM7bV2ASGCY87YV2OMsb5drlxPUpSu2iwJ8nbfNwAZgVBdtl/bq0uXaxVm+O4C3gOXO+2e9TbpDz7gj6Tq7olnAq87brwIXu68o7dNafwOUHHO4vbLPApZoreu11vtxzL5PPRvl7Ih26tKeTlsXrXWB1nqL83YlsAtHRrwu1y4nqEt7OnNdtNa6ynnX7PzSdM12aa8u7em0dVFK9QCmAy+2OHzW26Q7BOPukIpTA58rpX5UjixlAOHauVbb+T3MbaU7de2Vvau21S3KsRvZ4hbDVV2iLkqpXsBQHD2XLt0ux9QFumC7OIdDtwFFwBda6y7bLu3UBbpeu/wbuAuwtzh21tukOwTjDqXi7OTGaK2H4dj96mal1Hh3F+gM6Ypt9QyQAAwBCoB/OY93+roopXyB94HbtdYVJzq1jWOdvS5dsl201k1a6yE4shSmKqUGnOD0rliXLtUuSqkZQJHW+seOPqWNY6elHt0hGHcoFWdnprXOd34vAj7AMexxSDl3vnJ+L3JfCU9Ze2Xvcm2ltT7k/KdjB17g6JBUp66LUsqMI3i9qbVe5jzcJdulrbp01XZpprUuA9YCF9BF26VZy7p0wXYZA1yklDqA4xLnZKXUG7ihTbpDMO5Ius5OSynlo5SyNt8GzgPScdRhofO0hcBH7inhz9Je2T8G5imlPJRScTj2v97ohvJ1mGq9FehsHG0DnbguSikFvATs0lr/b4uHuly7tFeXLtouoUqpAOdtL2AqsJuu2S5t1qWrtYvW+h6tdQ+tdS8cseMrrfUVuKNN3D2L7XR84UjFuQfHzLa/uLs8p1j2eByz87YDO5vLDwQDq4FM5/cgd5e1nfK/jWM4qgHHp8ZFJyo78BdnO2UA09xd/g7U5XVgB5Dm/EOM7Ox1AcbiGDpLA7Y5vy7siu1ygrp0xXYZBGx1ljkduM95vCu2S3t16XLt0qJ8Ezk6m/qst4lk4BJCCCHcrDsMUwshhBBdmgRjIYQQws0kGAshhBBuJsFYCCGEcDMJxkIIIYSbSTAWQggh3EyCsRBCCOFmEoyFEEIIN/v/9CB4srLZ5jMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "pd.DataFrame(history.history).plot(figsize=(8, 5))\n",
    "plt.grid(True)\n",
    "plt.gca().set_ylim(0, 1)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0c61959",
   "metadata": {},
   "source": [
    "Note: The validation error is computed at the end of each epoch, while the training error is computed using a running mean during each epoch. Therefore, sometimes the performance blot can look like the model performed better on the validation set than on the training set at the beginning of training. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe091488",
   "metadata": {},
   "source": [
    "Okay, one thing that I didn't show before is using the `predict()` method. Firstly, I will use `predict()` on one instance and then show that instance with `imshow()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "80f99d69",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]], dtype=float32)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_new = X_test[:1] # getting the first row\n",
    "y_proba = model.predict(X_new)\n",
    "y_proba.round(3) # To get rid of scientific notation, I will use round() function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04d99655",
   "metadata": {},
   "source": [
    "Now the model provided us with a sparse array that contains the probabilities of each class. I will convert these values to one-hot vector labels. There is a couple of ways of doing that, the one I generally prefer using `keras.utils.to_categorical() ` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9f232dc1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]], dtype=float32)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.utils.to_categorical(np.argmax(y_proba.round(3), axis=1), 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7159e6d7",
   "metadata": {},
   "source": [
    "The algorithm assigns the class to ankle boat which is the 10th class in our target. Let's use `imshow()` function to show the instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9e19cc45",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAFUCAYAAAB7ksS1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAKEUlEQVR4nO3dMW+VdR/G8ftgoeARFJpAFA0sJA7G4OIrcNHBxDfi6pvwBbjoymLYUTdFIUwE1EAHxcSK1hpoC1ih9JmfaOG+nuc6tpDPZ4Rf/jk9PXy9B3/nP9na2hoA+P/t2ekXAPC0EFSAEkEFKBFUgBJBBSgRVICSucf8vf+nCuDvJv/0h55QAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBkbqdfAPDfNjc3o/k9e8Y/F00mk/TlRDY2NkbPzs/PR2cvLi5G86dOnYrmGzyhApQIKkCJoAKUCCpAiaAClAgqQImgApQIKkCJoAKUCCpAidVTdszW1tZMZochW8cchmH4+eefR89+88030dlvv/12ND+dTqP53SRdJ02cPXs2mv/ggw9m9Eq25wkVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgxC4/T4R0Nz/15Zdfjp69ePFidPbS0lI0//7770fzu8lvv/02evbcuXPR2QcPHkxfzr/OEypAiaAClAgqQImgApQIKkCJoAKUCCpAiaAClAgqQImgApQIKkCJXX52zObm5ujZubnso3rp0qVo/vvvvx89e+zYsejsxcXFaP69996L5g8fPjx69s8//4zOPnHiRDS/srIyenZ1dTU6+/jx49H8TvCEClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJVZPqXn48GE0n6yT3rlzJzr7008/jebn5+dHz6brm2tra9H81tbWzObTs7/99tto/uWXXx49m6zMDkO2qrxTPKEClAgqQImgApQIKkCJoAKUCCpAiaAClAgqQImgApQIKkCJoAKU2OX/H6T70JPJZPRsug+fnJ3Op7vTzzzzTDSf+Oijj6L59Krn/fv3j569ceNGdHa6+5++9gcPHoyeTT8v0+k0mk++E+H27dvR2RsbG9F88v0P6c+5HU+oACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJU/tLn+ybz/LffjUnj2z/W9csp8/y938YRiGM2fOjJ69efNmdPYbb7wRzSf78Ldu3YrOPnLkSDS/sLAQzf/++++jZ9fX16Ozk/cllX4nxt27d6P5xcXF0bOnT5+Ozt6OJ1SAEkEFKBFUgBJBBSgRVIASQQUoEVSAEkEFKBFUgBJBBSh5aldPZ7keml71nMyn657pzznLddJPPvkkmr9+/fro2VdeeSU6e2VlJZpP1iDv3bsXnX38+PFofm1tLZpPPgPPPvtsdHZ6BfYsV75T586dGz1r9RRglxFUgBJBBSgRVIASQQUoEVSAEkEFKBFUgBJBBSgRVIASQQUo2bFd/nQfPpXsCafX2aZXPc/6aujE0tLS6NmzZ89GZ6c77qdOnRo9m15/vLGxEc0nu/979+6Nzk531tPrkhPpZ3F+fn5m50+n0+js9H08f/58NN+we/6lAzzhBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoeucu/ubkZHZbc+b6b9ttnfT/48vLy6Nkff/wxOvvatWvR/C+//DJ6dt++fdHZhw4diuZv3bo1enZ1dTU6+/79+9F8svuffM6HIf+dPnjwIJp/4YUXRs+mv9O0Acn3Yhw4cGCmr+W5554bPXv16tXo7Ndee+0f/3z3VA3gCSeoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQ8shd/nRnOfHrr79G8zdu3Ijm79y5M5PZYcjvn//hhx9Gz6Z3ss/NPfJX+DcHDx4cPfvw4cPo7Nu3b0fzyfuY/pzp+5jslad31f/111/R/IsvvhjNJ99zkL4vhw8fjubX19dHz/7xxx/R2clu/jAMw82bN2f2WrbjCRWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUqyfb7H+OKLL0bPLi0tRWenq4fJ1c2zvC57GLLXnqyGDkO26jcM2TpeciXwMGRXMQ9DttaYrsGm70vyGZhOp9HZ6cpkci30MGSf9VlLfqfpVfLpyney8pv2ZTueUAFKBBWgRFABSgQVoERQAUoEFaBEUAFKBBWgRFABSgQVoERQAUoeucD62WefRYd9/PHHo2dfffXV6Oz0at1ZXpe8b9++aD7ZE0/359Pd/2S/Od21Xltbi+aTnzXd455MJtF88jtKvg9hGPIr07/77rtoPvmdpt9bkUq+tyC9vn3//v0zey1Hjx6Nzt6OJ1SAEkEFKBFUgBJBBSgRVIASQQUoEVSAEkEFKBFUgBJBBSgRVICSR+7yv/nmm9FhFy5cGD175cqV6Oyvvvoqmk/s3bs3mk/3548cOTKT2WEYhueffz6aT/a+0+8VWFlZieavXbs2evbu3bvR2aurq9F8svt/+fLl6OzXX389mj958mQ0//nnn4+e3djYiM5Ov88hMTf3yPz8zUsvvRTNHzp0aPRs+j0U2/GEClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJZPHrBdmu4cztL6+Hs1fvHhx9GyyAjkMw/D1119H88vLy6Nn05XJ9CreZJ00vYo5XVNM1mzTa8ffeuutaP6dd94ZPZteZzxr77777ujZn376KTp7YWEhmk/WPdMV7nRVdX5+fvTshx9+GJ09nU7/8R+HJ1SAEkEFKBFUgBJBBSgRVIASQQUoEVSAEkEFKBFUgBJBBSgRVICSJ2aXH2AXscsPMEuCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlggpQIqgAJYIKUCKoACWCClAiqAAlc4/5+8m/8ioAngKeUAFKBBWgRFABSgQVoERQAUoEFaDkPzS22+k0uC7uAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 6))\n",
    "for index, image in enumerate(X_new):\n",
    "    plt.imshow(image, cmap=\"binary\", interpolation=\"nearest\")\n",
    "    plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f330dae5",
   "metadata": {},
   "source": [
    "Instead of one-hot encoding, we could also directly see the class by using the strategy below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "ec8a3639",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9], dtype=int64)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = np.argmax(model.predict(X_new), axis=-1)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c9ff2fa",
   "metadata": {},
   "source": [
    "# Functional API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4034a13",
   "metadata": {},
   "source": [
    "Now we will be implementing a Deep Learning architecture named Wide and Deep Neural Network. This one was a very interesting architecture for me when I first read about it. I am actually a molecular biology student who is interested in neuroscience, therefore, I have a genuine interest in network architectures (I recently see much more interesting architectures in papers). I will be again using the Fashion MNIST dataset (The architecture was originally used for recommender systems).\n",
    "\n",
    "One of the general ideas behind this model is to have different inputs. We will connect some of our input directly to output layers. The sequential API is not suitable for constructing a model like that since we have different inputs connecting to other layers in a non-sequential way. To construct such a model, we can use the Functional API of TensorFlow.\n",
    "\n",
    "You can find more about the architecture in the paper. By the way, the paper is really easy to read, I recommend everybody to read it. [The paper](https://arxiv.org/abs/1606.07792)\n",
    "\n",
    "Additional Note: The name Functional API actually comes from the fact that we use the layers here like functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7435c89c",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_layer = keras.layers.Input(shape=[28,28])\n",
    "flatten1=keras.layers.Flatten(input_shape=[28, 28])(input_layer)\n",
    "hidden_layer1 = keras.layers.Dense(120, activation=\"relu\")(flatten1)\n",
    "hidden_layer2 = keras.layers.Dense(60, activation=\"relu\")(hidden_layer1)\n",
    "\n",
    "concatenated_layer = keras.layers.concatenate([flatten1, hidden_layer2])\n",
    "output = keras.layers.Dense(10,activation=\"softmax\")(concatenated_layer)\n",
    "model = keras.models.Model(inputs=[input_layer], outputs=[output])\n",
    "\n",
    "model.compile(loss=keras.losses.sparse_categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.SGD(),\n",
    "              metrics=[keras.metrics.sparse_categorical_accuracy])\n",
    "\n",
    "\n",
    "history = model.fit(X_train, y_train, epochs=100,validation_data=(X_valid, y_valid))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56bea0a7",
   "metadata": {},
   "source": [
    "Let's use `evaluate()` function to look at the model's performance on validation data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "e81ac800",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "157/157 [==============================] - 0s 847us/step - loss: 0.3731 - sparse_categorical_accuracy: 0.8886\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.37307849526405334, 0.8885999917984009]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_valid, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "398254d0",
   "metadata": {},
   "source": [
    "Okay, now we are done with an example using functional API. Let's now implement a similar structure of the Wide and Deep Neural Network. To do that I will split my training and validation instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "63d9c676",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_A, X_train_B = X_train[:, :5], X_train[:, 1:]\n",
    "X_valid_A, X_valid_B = X_valid[:, :5], X_valid[:, 1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "7848e456",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(55000, 5, 28)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_A.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5164195",
   "metadata": {},
   "source": [
    "I will code a callback which stops the training if the validation accuracy exceeds %90."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ec51cbec",
   "metadata": {},
   "outputs": [],
   "source": [
    "class terminate(tf.keras.callbacks.Callback):\n",
    "     def on_epoch_end(self, epoch, logs={}):\n",
    "        if logs.get('val_sparse_categorical_accuracy') is not None and logs.get('val_sparse_categorical_accuracy') > 0.90:\n",
    "            print(\"\\n \\n Validation accuracy is reached termination in process...\")\n",
    "            self.model.stop_training = True "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "638e5206",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_wide = keras.layers.Input(shape=X_train_A.shape[1:], name=\"wide_input\")\n",
    "input_deep = keras.layers.Input(shape=X_train_B.shape[1:], name=\"deep_input\")\n",
    "\n",
    "flatten_deep=keras.layers.Flatten(input_shape=[26, 28])(input_deep)\n",
    "hidden_layer1 = keras.layers.Dense(128, activation=\"relu\")(flatten_deep)\n",
    "hidden_layer2 = keras.layers.Dense(256, activation=\"relu\")(hidden_layer1)\n",
    "hidden_layer3 = keras.layers.Dense(512, activation=\"relu\")(hidden_layer2)\n",
    "\n",
    "flatten_wide=keras.layers.Flatten(input_shape=[5, 28])(input_wide)\n",
    "concat = keras.layers.concatenate([flatten_wide, hidden_layer3])\n",
    "\n",
    "output = keras.layers.Dense(10, activation=\"softmax\", name=\"output\")(concat)\n",
    "\n",
    "model = keras.models.Model(inputs=[input_wide, input_deep], outputs=[output])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60ee9674",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=keras.losses.sparse_categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.SGD(),# default learning_rate is 0.01\n",
    "              metrics=[keras.metrics.sparse_categorical_accuracy])\n",
    "\n",
    "callbacks = terminate()\n",
    "history = model.fit((X_train_A, X_train_B), y_train, epochs=50,validation_data=((X_valid_A, X_valid_B), y_valid), callbacks=callbacks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "51147f91",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "157/157 [==============================] - 0s 2ms/step - loss: 0.3251 - sparse_categorical_accuracy: 0.8912\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.3250572383403778, 0.8912000060081482]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate((X_valid_A, X_valid_B), y_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "544b4dd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a0554ab",
   "metadata": {},
   "source": [
    "## MNIST NUMBERS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f23102c",
   "metadata": {},
   "source": [
    "Let's also implement one model one MNIST Number dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "23c1afa3",
   "metadata": {},
   "outputs": [],
   "source": [
    "(X_train_full, y_train_full), (X_test, y_test) = keras.datasets.mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "648b6d99",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_valid, X_train = X_train_full[:5000] / 255., X_train_full[5000:] / 255.\n",
    "y_valid, y_train = y_train_full[:5000], y_train_full[5000:]\n",
    "X_test = X_test / 255."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "febb4c5f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x2b3b2bfb670>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAOXUlEQVR4nO3df4xU9bnH8c9zlfoDGsNeVtwIcaHhD/yRC3UgN2IIN9VGUQP9w5vyR10JKcRg0kZMLnoTyh/ehJhLm5rcEOnVsNz0ijVtAxpQCGmUJoYwEkRg06uXrC1dZJcYUzDGKj73jz3cbHHmO7tzzswZeN6vZDMz55kz5+Gwnz0z8z0zX3N3Abjy/V3ZDQBoD8IOBEHYgSAIOxAEYQeCuLqdG5s2bZr39va2c5NAKIODgzp79qzVquUKu5ndJ+nnkq6S9J/uvil1/97eXlWr1TybBJBQqVTq1pp+Gm9mV0n6D0n3S7pV0gozu7XZxwPQWnlesy+U9IG7n3T3v0raIWlZMW0BKFqesN8s6U9jbp/Klv0NM1ttZlUzq46MjOTYHIA88oS91psAXzv31t23unvF3Svd3d05NgcgjzxhPyVp5pjbMyQN5WsHQKvkCfshSXPMbJaZfUPS9yXtKqYtAEVreujN3b80s8clvaHRobcX3f14YZ0BKFSucXZ33y1pd0G9AGghTpcFgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgiFyzuKIYH330UbK+Z8+eZH1gYKBu7cSJE8l1d+9OT8K7bt26ZH3p0qXJ+ty5c+vWrrvuuuS6N9xwQ7J+4cKFZH379u11a59++mly3TVr1iTrkyZNStY7Ua6wm9mgpHOSLkj60t0rRTQFoHhFHNn/yd3PFvA4AFqI1+xAEHnD7pL2mtk7Zra61h3MbLWZVc2sOjIyknNzAJqVN+yL3P3bku6XtNbMFl96B3ff6u4Vd690d3fn3ByAZuUKu7sPZZfDkn4raWERTQEoXtNhN7PJZvbNi9clfVfSsaIaA1Asc/fmVjSbrdGjuTT6rv5/u/u/pdapVCperVab2t7lrL+/P1lfuXJlsm5mRbYzIY1+P/L0Nnv27GQ9NU4uSQcOHEjWn3rqqQn3dNGRI0eS9TvuuKPpx26lSqWiarVa8z+l6aE3dz8p6R+a7gpAWzH0BgRB2IEgCDsQBGEHgiDsQBB8xLUAQ0NDyfoTTzzRpk4m7pZbbknWBwcHW7btkydPJut33313sp5nWHDatGnJdadMmZKsX444sgNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzF2D//v3J+ieffJLr8ZcvX56sb9y4senHbjTe3OirxBr92x599NG6tQ8//DC5bl5dXV11ay+//HJy3VmzZhXdTuk4sgNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzF2DLli251p88eXKyPn/+/GT9jTfeaHrbCxYsSNYPHTqUrL/66qvJeqvH0lNmzJhRt7ZkyZL2NdIhOLIDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCMsxdg8eLFyXqjserz588n6xs2bJhwT0Vp5ZTN1157bbK+fv36ZP3ZZ59N1t999926tX379iXXvffee5P1y1HDI7uZvWhmw2Z2bMyyLjPbZ2bvZ5dTW9smgLzG8zR+m6T7Llm2XtJ+d58jaX92G0AHaxh2d39L0seXLF4mqT+73i9pebFtAShas2/QTXf305KUXd5Y745mttrMqmZWbfR9ZgBap+Xvxrv7VnevuHulu7u71ZsDUEezYT9jZj2SlF0OF9cSgFZoNuy7JPVl1/sk7SymHQCt0nCc3cxekrRE0jQzOyXpJ5I2SfqVma2S9EdJD7eyyU63adOmZH14OP3EZ9u2bcl6nrHsVms0v3vqs/hPPvlkct277rorWW90fsLmzZvr1vbs2ZNc90ocZ28YdndfUaf0nYJ7AdBCnC4LBEHYgSAIOxAEYQeCIOxAEHzEtQ0afUR1xYp6Ax6d7/bbb0/We3p62tTJxAwMDJTdQttxZAeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIBhnb4Pe3t5cddR24sSJptedO3dugZ1cHjiyA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQjLOjYzWa6nr37t3J+vTp0+vWHnvssaZ6upxxZAeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIBhnR2k+++yzZP3BBx9M1t09Wb/pppvq1ubMmZNc90rU8MhuZi+a2bCZHRuzbKOZ/dnMjmQ/S1vbJoC8xvM0fpuk+2os/5m7z8t+0qcyAShdw7C7+1uSPm5DLwBaKM8bdI+b2dHsaf7Uencys9VmVjWz6sjISI7NAcij2bBvkfQtSfMknZa0ud4d3X2ru1fcvdLd3d3k5gDk1VTY3f2Mu19w968k/ULSwmLbAlC0psJuZmPn4f2epGP17gugMzQcZzezlyQtkTTNzE5J+omkJWY2T5JLGpS0pnUt4nJ27ty5urW+vr7kumfPnk3WzSxZf+CBB5L1aBqG3d1X1Fj8Qgt6AdBCnC4LBEHYgSAIOxAEYQeCIOxAEHzEFS21Y8eOurWdO3fmeux58+Yl62vXrs31+FcajuxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EATj7Ehq9HXPq1atStYbTauccttttyXrzzzzTLLe09OTrEfDkR0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgmCcvQD9/f3J+p49e5L1N998s8h2JmTBggXJ+sGDB5P1Rl/3nMeuXbuS9d7e3pZt+0rEkR0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgmCcfZz27t1bt7Zy5cpcj+3uyXqjqYnzeO2115L1vL1dc801dWvPP/98cl3G0YvV8MhuZjPN7HdmNmBmx83sR9nyLjPbZ2bvZ5dTW98ugGaN52n8l5LWuftcSf8oaa2Z3SppvaT97j5H0v7sNoAO1TDs7n7a3Q9n189JGpB0s6Rlki6eJ9ovaXmLegRQgAm9QWdmvZLmSzooabq7n5ZG/yBIurHOOqvNrGpm1ZGRkZztAmjWuMNuZlMk/VrSj939L+Ndz923unvF3Svd3d3N9AigAOMKu5lN0mjQf+nuv8kWnzGznqzeI2m4NS0CKELDoTcbHVt5QdKAu/90TGmXpD5Jm7LLfPPvdrjXX3+9bq2VQ2PtePw8GvV255131q0tWbKk4G6QMp5x9kWSfiDpPTM7ki17WqMh/5WZrZL0R0kPt6RDAIVoGHZ3/72ken++v1NsOwBahdNlgSAIOxAEYQeCIOxAEIQdCIKPuGbOnTuXrB84cKBNnVxZ3n777bq1RYsWJdc9evRosj51Kh+0nAiO7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBOPsmZMnTybrhw8fblMnxXrooYeS9XvuuSdZb/RV0s8991yyntqvQ0NDyXW/+OKLZB0Tw5EdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4JgnD3TaLaahx+u/03Zr7zySq5tT548OVnfsGFDst7X11e31tXVlVz36qvz/Qo88sgjyfrnn39et3b8+PHkutdff31TPaE2juxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EIQ1+ryymc2UtF3STZK+krTV3X9uZhsl/VDSSHbXp919d+qxKpWKV6vV3E0DqK1SqahardacdXk8Z1R8KWmdux82s29KesfM9mW1n7n7vxfVKIDWGc/87Kclnc6unzOzAUk3t7oxAMWa0Gt2M+uVNF/SwWzR42Z21MxeNLOac/GY2Wozq5pZdWRkpNZdALTBuMNuZlMk/VrSj939L5K2SPqWpHkaPfJvrrWeu29194q7Vxqdfw6gdcYVdjObpNGg/9LdfyNJ7n7G3S+4+1eSfiFpYevaBJBXw7CbmUl6QdKAu/90zPKeMXf7nqRjxbcHoCjjeTd+kaQfSHrPzI5ky56WtMLM5klySYOS1rSgPwAFGc+78b+XVGvcLjmmDqCzcAYdEARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgiIZfJV3oxsxGJH04ZtE0SWfb1sDEdGpvndqXRG/NKrK3W9y95ve/tTXsX9u4WdXdK6U1kNCpvXVqXxK9NatdvfE0HgiCsANBlB32rSVvP6VTe+vUviR6a1Zbeiv1NTuA9in7yA6gTQg7EEQpYTez+8zsD2b2gZmtL6OHesxs0MzeM7MjZlbq/NLZHHrDZnZszLIuM9tnZu9nlzXn2Cupt41m9uds3x0xs6Ul9TbTzH5nZgNmdtzMfpQtL3XfJfpqy35r+2t2M7tK0v9IulfSKUmHJK1w9xNtbaQOMxuUVHH30k/AMLPFks5L2u7ut2fLnpX0sbtvyv5QTnX3f+mQ3jZKOl/2NN7ZbEU9Y6cZl7Rc0qMqcd8l+vpntWG/lXFkXyjpA3c/6e5/lbRD0rIS+uh47v6WpI8vWbxMUn92vV+jvyxtV6e3juDup939cHb9nKSL04yXuu8SfbVFGWG/WdKfxtw+pc6a790l7TWzd8xsddnN1DDd3U9Lo788km4suZ9LNZzGu50umWa8Y/ZdM9Of51VG2GtNJdVJ43+L3P3bku6XtDZ7uorxGdc03u1SY5rxjtDs9Od5lRH2U5Jmjrk9Q9JQCX3U5O5D2eWwpN+q86aiPnNxBt3scrjkfv5fJ03jXWuacXXAvitz+vMywn5I0hwzm2Vm35D0fUm7Sujja8xscvbGicxssqTvqvOmot4lqS+73idpZ4m9/I1Omca73jTjKnnflT79ubu3/UfSUo2+I/+/kv61jB7q9DVb0rvZz/Gye5P0kkaf1n2h0WdEqyT9vaT9kt7PLrs6qLf/kvSepKMaDVZPSb3drdGXhkclHcl+lpa97xJ9tWW/cbosEARn0AFBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEP8HcVZOwGrt+WgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X_train[13], cmap=\"binary\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a976adf7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Flatten(input_shape=[28, 28]),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(128, activation=\"relu\"),\n",
    "    keras.layers.Dense(10, activation=\"softmax\")\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "3f7f28a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"sparse_categorical_crossentropy\",\n",
    "              optimizer=keras.optimizers.SGD(lr=3e-1),\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "a3354405",
   "metadata": {},
   "outputs": [],
   "source": [
    "class terminate(tf.keras.callbacks.Callback):\n",
    "     def on_epoch_end(self, epoch, logs={}):\n",
    "        if logs.get('val_accuracy') is not None and logs.get('val_accuracy') > 0.9860:\n",
    "            print(\"\\n \\n Validation accuracy is reached termination in process...!\")\n",
    "            self.model.stop_training = True \n",
    "callbacks = terminate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57c31ffb",
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model.fit(X_train, y_train, epochs=100,\n",
    "                    validation_data=(X_valid, y_valid),callbacks=callbacks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b0cbece",
   "metadata": {},
   "source": [
    "## Time to deal with Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0379e6fe",
   "metadata": {},
   "source": [
    "Now time to deal with some regression problems. Actually, I don't like doing regression stuff with neural networks but I want to do some practice on the Boston dataset and the covid dataset that I previously used in my machine-learning notes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5921de22",
   "metadata": {},
   "source": [
    "### Boston Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c537a00",
   "metadata": {},
   "source": [
    "Let's load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "c3b7b0c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_valid, y_valid)=tf.keras.datasets.boston_housing.load_data(\n",
    "      path=\"boston_housing.npz\", test_split=0.3, seed=85)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5afd066",
   "metadata": {},
   "source": [
    "Let's scale our predictors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "3a7ddc45",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(x_train)\n",
    "X_valid = scaler.transform(x_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "abefa9d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "class terminate(tf.keras.callbacks.Callback):\n",
    "     def on_epoch_end(self, epoch, logs={}):\n",
    "        if logs.get('val_loss') is not None and logs.get('val_loss') < 10:\n",
    "            print(\"\\n \\n Validation accuracy is reached termination in process...!\")\n",
    "            self.model.stop_training = True \n",
    "callbacks = terminate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b92f863b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(100, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(30, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.SGD(lr=1e-3))\n",
    "history = model.fit(X_train, y_train, epochs=1000, validation_data=(X_valid, y_valid),callbacks=callbacks)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "0bd4f55d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 1ms/step - loss: 9.9964\n"
     ]
    }
   ],
   "source": [
    "mse = model.evaluate(X_valid, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d80960b0",
   "metadata": {},
   "source": [
    "We can also calculate MSE as we did in machine learning notes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "0dc08bbb",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9.996418144370738"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = model.predict(X_valid)\n",
    "\n",
    "mean_squared_error(y_valid, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "e6c3e175",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEyCAYAAAAvPHP2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA+30lEQVR4nO3deXxcdb3/8dd3JpN9a7qkS0oX2gKllQKlLEIpoIAoiyhQBOQqwlUQl3svCnr1ov64V+Vel98V9YdrFRQqoFZEEIGwKFsLhdJ9b9O0TdI2abZJZvn+/vieSdI2bSd7zpn38/HIYyYnZ858v9+cmc93O99jrLWIiIjI8BAa6gSIiIhIJwVmERGRYUSBWUREZBhRYBYRERlGFJhFRESGEQVmERGRYSStwGyM2WKMWWGMWW6MWeptKzPGPG2MWe89juiy/13GmA3GmLXGmIsGKvEiIiJB05MW83nW2jnW2rne73cCz1hrpwPPeL9jjJkJLAROBC4GfmiMCfdjmkVERAKrL13ZlwOLvOeLgCu6bH/IWttmrd0MbADm9eF9REREMkZWmvtZ4K/GGAv8P2vt/UC5tXYngLV2pzFmjLfvBOCVLq+t8rYdwBhzC3ALQF5e3qkTJ07sZRa6l0wmCYUydwg90/MPKoNMzz+oDEBlMJzzv27dujpr7eiDt6cbmN9tra32gu/Txpg1R9jXdLPtkHU/veB+P8DcuXPt0qVL00xKeiorK1mwYEG/HtNPMj3/oDLI9PyDygBUBsM5/8aYrd1tT6saYa2t9h5rgN/juqZ3G2PGeQcfB9R4u1cBXZu/FUB175ItIiKSWY4amI0xBcaYotRz4ELgHWAJcKO3243AH73nS4CFxpgcY8wUYDrwWn8nXEREJIjS6couB35vjEnt/xtr7ZPGmNeBxcaYm4BtwFUA1tqVxpjFwCogDtxmrU0MSOpFREQC5qiB2Vq7CTipm+17gAsO85p7gHv6nDoRERmWYrEYVVVVRKPRoU7KEZWUlLB69eohTUNubi4VFRVEIpG09k938peIiEiHqqoqioqKmDx5Ml6P6rDU2NhIUVHRkL2/tZY9e/ZQVVXFlClT0nrN8JxDLiIiw1o0GmXkyJHDOigPB8YYRo4c2aOeBQVmERHpFQXl9PS0nBSYRUREhhEFZhER8aXCwsKhTsKACNzkr/3RGI+/tZNQc3KokyIiItJjgWsx72tu50u/X8HGel06LSKSCay13HHHHcyaNYvZs2fz8MMPA7Bz504uvvhi5syZw6xZs3jxxRdJJBL80z/9U8e+3/3ud4c49YcKXIs55A2yH7I4t4iIDIiv/Wklq6r39+sxZ44v5j8uPTGtfR977DGWL1/OW2+9RV1dHaeddhrz58/nN7/5DRdccAFf//rXSSQStLS0sHz5cnbs2ME777wDQH19fb+muz8ErsWcYhWZRUQywksvvcS1115LOBymvLycc889l9dff53TTjuNBx54gLvvvpsVK1ZQVFTE1KlT2bRpE7fffjtPPvkkxcXFQ538QwSvxRxSi1lEZDCl27IdKPYwLbH58+fz5JNP8vzzz3PDDTdwxx138NGPfpS33nqLp556ivvuu4/Fixfz85//fJBTfGSBazF7cVktZhGRDDF//nwefvhhEokEtbW1vPDCC8ybN4+tW7cyevRobr75Zm666SbeeOMN6urqSCaTfOhDH+Ib3/gGb7zxxlAn/xDBazFrjFlEJKN88IMf5OWXX+akk07CGMO3v/1txo4dy6JFi/jWt75FTk4OhYWF/OpXv2LHjh187GMfI5l0V+7813/91xCn/lCBC8yp9VWSiswiIoHW1NQEuJW17r33Xu69994D/n7jjTdy5ZVXHrJW9nBsJXcVuK5sLREnIiJ+FrjAnBpjVotZRET8KICBWWPMIiLiX4ELzEazskVExMcCGJjVYhYREf8KXGDWdcwiIuJngQvMajGLiIifBS4wd7SYFZpFRKSLI92/ecuWLcyaNWsQU3N4AQzMXotZcVlERHwoeCt/aYxZRGRw/eVO2LWif485dja875tH3OWLX/wikyZN4tZbbwXg7rvvxhjDCy+8wL59+4jFYnz5y19m4cKFPXrraDTKpz71KZYuXUpWVhbf+c53OO+881i5ciUf+9jHaG9vJ5lM8uijjzJ+/HiuvvpqqqqqSCQSfOUrX+Gaa67pdbYhiIHZW5QzOcTpEBGRgbVw4UI+97nPdQTmxYsX8+STT/L5z3+e4uJi6urqmDdvHtdcc02PVoW87777AFixYgVr1qzhwgsvZN26dfz4xz/ms5/9LNdddx3t7e0kEgmeeOIJxo8fz5///GcAGhoa+pyvwAXmkFbkFBEZXEdp2Q6Uk08+mZqaGqqrq6mtrWXEiBGMGzeOz3/+87zwwguEQiF27tzJ7t27GTt2bNrHfemll7j99tsBOP7445k0aRLr1q3jzDPP5J577qGqqoorr7yS6dOnM3v2bP7t3/6NL37xi3zgAx/gnHPO6XO+NMYsIiK+9eEPf5hHHnmEhx9+mIULF/Lggw9SW1vLsmXLWL58OWPGjCEajfbomIe7v/NHPvIRlixZQl5eHhdddBHPPvssM2bMYNmyZcyePZu77rqLr3/9633OU+BazEZrZYuIZIyFCxdy8803U1dXx/PPP8/ixYsZM2YMkUiE5557jm3btvX4mPPnz+fBBx/k/PPPZ926dWzbto3jjjuOTZs2MXXqVD7zmc+wadMm3n77bY4//njKysq4/vrrKSws5Je//GWf8xTAwKzrmEVEMsWJJ55IY2MjEyZMYNy4cVx33XVceumlzJ07lzlz5jBjxoweH/PWW2/lk5/8JLNnzyYrK4tf/vKX5OTk8PDDD/PAAw8QiUQYO3YsX/3qV3n99de54447CIVCRCIRfvSjH/U5T4ELzODGmRWYRUQyw4oVnTPCR40axcsvv9zxe2NjY8f9mFP3b+7O5MmTeeeddwDIzc3ttuV71113cddddx2w7aKLLuKiiy7qS/IPEbgxZnDjzBpjFhERPwpki9kYTf4SEZFDrVixghtuuOGAbTk5Obz66qtDlKJDBTQwG3Vli4gMMGttj64PHg5mz57N8uXLB/U9DzfL+3AC2pWtMWYRkYGUm5vLnj17ehx0Mo21lj179pCbm5v2awLZYnZjzDpZREQGSkVFBVVVVdTW1g51Uo4oGo32KCgOhNzcXCoqKtLeP5CB2aAxZhGRgRSJRJgyZcpQJ+OoKisrOfnkk4c6GT0S0K5sjTGLiIg/BTIwG40xi4iITwUyMIdCRktyioiILwUyMBvUYhYREX8KZGAOqS9bRER8KpCB2RhDcqgTISIi0guBDMwhLckpIiI+FcjArJ5sERHxq7QDszEmbIx50xjzuPd7mTHmaWPMeu9xRJd97zLGbDDGrDXG9O/9sNKgu0uJiIhf9aTF/FlgdZff7wSesdZOB57xfscYMxNYCJwIXAz80BgT7p/kpkcLjIiIiF+lFZiNMRXA+4Gfdtl8ObDIe74IuKLL9oestW3W2s3ABmBev6S2B9RiFhERP0q3xfw94AtwwGTncmvtTgDvcYy3fQKwvct+Vd62QRMKQVJtZhER8aGj3sTCGPMBoMZau8wYsyCNY3Z3c85DoqQx5hbgFoDy8nIqKyvTOHR62qJR4uFkvx7Tb5qamjI6/6AyyPT8g8oAVAZ+zH86d5d6N3CZMeYSIBcoNsY8AOw2xoyz1u40xowDarz9q4CJXV5fAVQffFBr7f3A/QBz5861CxYs6H0uDlKwtJJwVpT+PKbfVFZWZnT+QWWQ6fkHlQGoDPyY/6N2ZVtr77LWVlhrJ+MmdT1rrb0eWALc6O12I/BH7/kSYKExJscYMwWYDrzW7yk/AgNaK1tERHypL/dj/iaw2BhzE7ANuArAWrvSGLMYWAXEgdustYk+p7QHTHed6SIiIj7Qo8Bsra0EKr3ne4ALDrPfPcA9fUxbr4WM7i4lIiL+FMiVv3Qds4iI+FUgA7PRWtkiIuJTAQ3MajGLiIg/BTIw6+5SIiLiVwENzGoxi4iIPwUyMGuMWURE/CqggVktZhER8adABmaNMYuIiF8FMjAbwKrNLCIiPhTIwKyVv0RExK8CG5hFRET8KJCB2RjdXUpERPwpsIFZcVlERPwokIFZXdkiIuJXgQ3M6soWERE/CmRg1spfIiLiVwENzFr5S0RE/CmQgTmkyV8iIuJTAQ3MRl3ZIiLiSwENzGoxi4iIPwUyMINmZYuIiD8FMjCHdBmziIj4VEADs8FqkFlERHwokIHZGEgOdSJERER6IZCBOaTFskVExKcCGZjVYhYREb8KZGDWdcwiIuJXgQzM6skWERG/CmRgVotZRET8KpCBWS1mERHxq0AGZrWYRUTErwIZmA1qMYuIiD8FMjCrxSwiIn4VzMAcUotZRET8KZCB2RjdXUpERPwpmIEZsGozi4iIDwUyMGutbBER8auABmatlS0iIv4UyMBsNCtbRER8KqCBWT3ZIiLiT4EMzLqOWURE/CqggVktZhER8adABmaNMYuIiF8FNDCrxSwiIv501MBsjMk1xrxmjHnLGLPSGPM1b3uZMeZpY8x673FEl9fcZYzZYIxZa4y5aCAz0B2NMYuIiF+l02JuA8631p4EzAEuNsacAdwJPGOtnQ484/2OMWYmsBA4EbgY+KExJjwAaT8sjTGLiIhfHTUwW6fJ+zXi/VjgcmCRt30RcIX3/HLgIWttm7V2M7ABmNefiT4ag9bKFhERf8pKZyevxbsMmAbcZ6191RhTbq3dCWCt3WmMGePtPgF4pcvLq7xtBx/zFuAWgPLyciorK3udiYNt29YO2H49pt80NTVldP5BZZDp+QeVAagM/Jj/tAKztTYBzDHGlAK/N8bMOsLuprtDdHPM+4H7AebOnWsXLFiQTlLS8mZsHXbjes4991yM6S45wVdZWUl/lqkfZXoZZHr+QWUAKgM/5r9Hs7KttfVAJW7seLcxZhyA91jj7VYFTOzysgqguq8J7YmQF4w1AUxERPwmnVnZo72WMsaYPOA9wBpgCXCjt9uNwB+950uAhcaYHGPMFGA68Fo/p/soaXaPSUVmERHxmXS6sscBi7xx5hCw2Fr7uDHmZWCxMeYmYBtwFYC1dqUxZjGwCogDt3ld4YMm5AVmhWUREfGbowZma+3bwMndbN8DXHCY19wD3NPn1PVSalxZLWYREfGbwK78BRpjFhER/wlkYNbkLxER8auABmb3qK5sERHxm4AGZo0xi4iIPwUyMKdoWU4REfGbQAbmkNH1UiIi4k8BDczuUV3ZIiLiN8ELzMkE+bG95NCuwCwiIr4TvMC8bwtXV57PJaFXNcYsIiK+E7zAHAoDEDZJrAaZRUTEZ4IXmI0LzCGSWmBERER8J4CB2WUphNUYs4iI+E7wAnOqK5ukxphFRMR3gheYD+jKVmQWERF/CV5g7tJiVlwWERG/CV5g9saYXVe2IrOIiPhL8AJzSLOyRUTEv4IXmE3XyV+KzCIi4i/BC8xdWsyalS0iIn4TvMBsuk7+UmQWERF/CV5gPmBJThEREX8JXmA2BovxurIVmkVExF+CF5gBa8Ju8ldyqFMiIiLSMwENzCGtlS0iIr4UyMCMCRNCzWUREfGfQAZma0K6jllERHwp4IF5qFMiIiLSM4EMzKmubLWYRUTEbwIZmFMtZsVlERHxm0AG5lSLWSt/iYiI3wQyMLvrmK3GmEVExHcCGZgJhdySnGoxi4iIzwQyMNuOyV9DnRIREZGeCWRgpmPylyKziIj4SyADc8da2YrLIiLiM4EMzIS8Wdm68aOIiPhMIAOzG2PWrGwREfGfQAZmtFa2iIj4VDADc0gLjIiIiD8FMzBrSU4REfGpgAZmXccsIiL+FMjArPsxi4iIXwUyMBMKa0lOERHxpaMGZmPMRGPMc8aY1caYlcaYz3rby4wxTxtj1nuPI7q85i5jzAZjzFpjzEUDmYHuE52a/DXo7ywiItIn6bSY48C/WmtPAM4AbjPGzATuBJ6x1k4HnvF+x/vbQuBE4GLgh8aY8EAk/rBCWvlLRET86aiB2Vq701r7hve8EVgNTAAuBxZ5uy0CrvCeXw48ZK1ts9ZuBjYA8/o53UemMWYREfGprJ7sbIyZDJwMvAqUW2t3ggvexpgx3m4TgFe6vKzK23bwsW4BbgEoLy+nsrKyp2k/rOn7mwiRZOWqVRTtW9dvx/WTpqamfi1TP8r0Msj0/IPKAFQGfsx/2oHZGFMIPAp8zlq73xhz2F272XZI09Vaez9wP8DcuXPtggUL0k3KUTVv+QH1++o44YQTWDDnkDpBRqisrKQ/y9SPMr0MMj3/oDIAlYEf85/WrGxjTAQXlB+01j7mbd5tjBnn/X0cUONtrwImdnl5BVDdP8lNUyh1HbO6skVExF/SmZVtgJ8Bq6213+nypyXAjd7zG4E/dtm+0BiTY4yZAkwHXuu/JKchdROL5KC+q4iISJ+l05X9buAGYIUxZrm37UvAN4HFxpibgG3AVQDW2pXGmMXAKtyM7tustYn+TviRGG9WttrLIiLiN0cNzNbal+h+3BjggsO85h7gnj6kq2+MurJFRMSfgrvyl+4uJSIiPhTMwGwMYaMFRkRExH8CGZhNSEtyioiIPwUyMHcuyanILCIi/hLMwNxxEwsFZhER8ZdABmajm1iIiIhPBTIwa1a2iIj4VSADs0mt/KW4LCIiPhPIwKzJXyIi4leBDcwhtFC2iIj4T4/ux+wXJhTGqMUsIiI+FNjAHNKsbBER8aFAdmWbUJiwsSR130cREfGZQAZmQmH3aBWYRUTEXwIZmE0qMCcH9TbQIiIifRbMwGxcYLYKzCIi4jOBDMydXdkKzOIzm1+E/5wArfuGOiUiMkQCGZhNyMuWJn+J3+zbAu1N0LxnqFMiIkMkoIHZXQVmkvEhTolIDyVj3qPOXZFMFcjATGqMWbOyxW9S8yIUmEUyVjADc0dXtsaYxWcSqRZzbGjTISJDJpiB2WsxgwKz+EyqpaxKpUjGCmZgDqUul1JXtvhMKjAn1GIWyVTBDMxei9nocinxm44Ws8aYRTJVMAOzVv4Sv+oIzGoxi2SqYAZmo7Wyxac0xiyS8YIZmDUrW/wqNbasMWaRjBXMwKwxZvErXccskvECGpjVYhaf0hizSMYLZmDW/ZjFrzqW5FSlUiRTBTMwG91dSnxK1zGLZLxgBmZdLiV+ldB1zCKZLpiB2RtjDmlJTvEbLTAikvGCGZg7WswaYxafUWAWyXjBDMy67aP4le7HLJLxghmYvRazUYtZ/CY1L0KTv0QyVjADs277KH6lrmyRjBfMwBzSyl/iUwl1ZYtkumAGZm9WtgKz+I5azCIZL5iBWWPM4lcaYxbJeMEMzB2zstViFp/RrGyRjBfMwOy1mG1CgVl8Rl3ZvbdqCTx03VCnQqTPghmYvTHmREJfbuIzWpKz97a/CmufGOpUiPTZUQOzMebnxpgaY8w7XbaVGWOeNsas9x5HdPnbXcaYDcaYtcaYiwYq4UdOtGsxKzCL7+gmFr0Xj7o7yulzLz6XTov5l8DFB227E3jGWjsdeMb7HWPMTGAhcKL3mh8a03FR8eAJpVrM6soWn+noyta522PxqHtMtA1tOkT66KiB2Vr7ArD3oM2XA4u854uAK7psf8ha22at3QxsAOb1T1J7wKsLJFVzFr/R5K/ei6UCc/vQpkOkj7J6+bpya+1OAGvtTmPMGG/7BOCVLvtVedsOYYy5BbgFoLy8nMrKyl4m5VA50VrOBKKtLf16XD9pamrK2Lyn+LEMzmhpJhfYvWsHq/uYdj/mvy9O3FXFaOAfL1TSnuNG1zKtDLqT6WXgx/z3NjAfjulmm+1uR2vt/cD9AHPnzrULFizov1Ts3wmvQDicRb8e10cqKyszNu8pviyDZVnQBuWjyijvY9p9mf++qPoB1MFZp8+F0olABpZBNzK9DPyY/97Oyt5tjBkH4D3WeNurgIld9qsAqnufvF4KqStbfCo16Uvnbs/Fe9iV/fZiqH5z4NIj0ku9DcxLgBu95zcCf+yyfaExJscYMwWYDrzWtyT2QmqMWRNoxG90HXPvxb1JX+kG5qe+DK//bODSI9JLR+3KNsb8FlgAjDLGVAH/AXwTWGyMuQnYBlwFYK1daYxZDKwC4sBtdiiW3/JmZVt9uYnfpCqTSV0u1WOpFnM8zVnZ8bb09xUZREcNzNbaaw/zpwsOs/89wD19SVSfpZbkTCZIJC3hUHdD3yLDkGZl915PW8zxaGcwFxlGgrnylzfGHCZJW1zd2eIjHQuMKDD3WE/GmK111zurxSzDUDADs0kFZktbTHeYEp+wVmPMfdGTruxU8NZiJDIMBTMwey3mEEmiajGLX3SdrKgx5p7raDGnUXY9HY8WGUTBDMymS1e2WsziF11byWox91zHGHMawTa1r8aYZRgKaGB2k73CRi1m8ZGurWSNMfeMtV1awWmMMXcEZi3fKcNPYANzkhAhtZjFT9Ri7r2uE77SmfylFrMMY8EMzIA1IW9WtgKz+MQBY8wKzD3SNcCm1ZWtMWYZvoIbmAkRwhKNqStbfKLrpCUF5p7pGmDTmfyV6MF4tMggC25gNl5XtlrM4hcdwdgoMPdU1xZzOq3gjq5sBWYZfgIdmLXAiPhKKhhH8tNr9UmnWNeu7HTGmKMHPooMI8ENzN7kr6gmf4lfdATmXLWYeyre08DcZflO2+2daUWGTGADM2oxi9+kgnGWAnOPde2S7klXdrr7iwyiwAbmVFe2WsziGwrMvXdAizmdlb+6BmZ1Z8vwEtjATMfkL7WYxSdSASWSpzHmnjpgVnYPLpeC9O9GJTJIAh2YtSSn+ErqOuasHLAJjX32RE8Dbdd91GKWYSawgdmaEJGQ7dmSnEt/ATuWDVyiRI4ktSRnVp73u7qz03bA5VI9mJWd7v4igyiwgRlCREI9bDH/9Svw2k8HLkkiR9J1VnbX3+XoUl3Z4Zyed2WrxSzDTGADszUhIsamv8BIIg7tjdBcM7AJEzmcrpO/QOPMPRFvdY85RWlO/urala1Z2TK8BDYwt2eXMtbspS3dJTnb9rvHpt0DlyiRI0ndUSqiruweSwXX3OI0L5fq4draIoMosIG5qXAK05NbaI+l+eUWbfBeWDtwiRI5koNbzEldUZC2VKDNKe7ZAiNdXysyTAQ4ME8ln1aKWren94JUYG6u1ReiDI3kwS1mdWWnLRVoc4rSnJWtBUZk+Ap0YAYY37o+vRekArNNQMveAUqVyBF0zMrW5K8ei0fdxK+snF60mBWYZXgJbGBuLphInCwq2noYmEHjzDI0Oq5j1uSvHou3uXILZ6d/uZQJd75WZBgJbGC2oQg7IpM4pn1Dei9QYJahdsjlUhpSSVs86sotnJ3m5VLtbqJY6rUiw0hgAzPAjtxpTI1vTG8FpdSsbIAmXTIlQyBx8AIjajGnLd7murHT7sqOQm6Je64lOWWYCXRg3pU3gxG2ARp3HX1ntZhlqGmBkd6LtXpd2ZE0u7Lb3AxuUIvZL6yF2nVDnYpBEejAvLvgOPdk14qj7xxtgJwSiBS4mdkig63jcimvxZxQYE5bqsUcTrPFnGjrbDErMPvD2r/AffNgz8ahTsmAC3Rg3lc0wz3Z9dbRd442uA9q4Ri1mGVoqMXce/Fo5+SvdLuyswsBo7Wy/aJuLWBh7+ahTsmAC3RgLi0byeZkOdHty4++c0dgLldglqFxcItZY8zpS83KzspOc+WvLmPSajH7Q723JsX+HUObjkEQ6MC8YMYYVtrJxHcsP/rO0QY3S7NwTHpj0iL9rWPyV457HE4t5kdvhteH8Q1e4tEDu7KPNuGzI5Dn6HIpv2hQYA6EE8YVsT17GoUtVZ2Tu+rWwx8/DbGDasmpFnP5iW6f5j2Dn+Ch9Pf/C7+6fKhTkdlSl0cNt7WykwlY+XtY99ehTsnhxVo6u7KxRy+7eJtrXWflaq1sv+jvFvOON6C5rn+O1c8CHZiNMeRMmgtA/PVfuI3P/Se8+WtY87j7PRF362NH97vAPP1CwMKGvw1OIpNJePPBQysKg235g7Cp0s1ulaGRjIMJuZnFMHwmfzXudN3qDWkubzvYrIWGKiipcMEWjj7O3DEmrRazL1jbpcVc3ffjJeLwy/fDC//d92MNgEAHZoDJp72PJxOnEX7mbvj792HVH90flj/oHl/4NvzvKW4mdm4JjJvjxpnXPTk4Cdz6d/jjrbDqD4Pzft2p3w61a9zzDJhYMWwlYxDKcj8wfFrM+7a6x/rt6a0JMNia66C9CUZM8VrMHD3YaozZX1r3uf8xQEM/tJj3bXG9LHs2uFnej35iWDVKAh+Y588o578L/5WVkVnw9FcBCyddCxufc7Xs5b91i4vEWyG3hC17W6kZdy5seGZw7jSVupQrnUu6BkrX3oE9aa6UJv0vGfcCs9diHi6Tv/ZtcY/tje4LcrjZ51Umy7oE5iMtZ2qt674O57hWs2ZlD38NVe6xeEL/tJhrV7vHfZth9Z9gxe+g+s2+H7efBD4wZ4VD3Dj/BC5v/AJbTvkCvPfrcO4X3B8fuQkatnXunFvC1/60ki+sOQ7bth++czy8/rPuD7zjDfif4+Gn73XPe2v3O+5x19u9P0ZfbfgbFIx2z/cG/xrBYSuZcEG5o8U8TJbkrN/a+Xygu7OjDW54pydSFYcRkzsnzh1p3DgZB5vsMotbLeZhL3XeTTzdVRCj+4+8/9Gkegj3bYWa1QduGwYCH5gBrpo7kTElBVz2xlyeLbsaO2IKvPszsP0VV2s+9nwAYpEiXt60h8rYCTxxzu9hyrnwxL/Byz+E134Ciy6FzS+6g1Z+03WF7N0Iz3yt54lqb4b2li4t5ndg2SL4w239kuesWJOb5FZ/hC/SWKub5LbhGTjhMigYkxEX7w9biRiEwiSMC8y76xuHOEGefV0C85HOp75qrYfvzoLXf9Kz16WGX0ondenKPkIrOBWIU5O/+nuMeffK/rlDXTLR80pKX1g7PHtEoPO8O+YM99jXCWC1a91jMubm1gDUKDAPqtxImMX/fCZjS3L5+C+Xcu69lZz+8hnsKHoXdvZVMOvDAGzcHyYaS5IdDvGTNdlw9a/cmPNTd7kAXbUUfnMNPP9tWP8UnHk7nHYzbHq+s6slHe0t8ONz4NdXuFpabim07oWnvwLLH4Dq5ekfa+dbsPHZQzaX737eTXJ75OPdTyJ66Xtw7zSXr3grzLsZRh7bv4F5z8a+1WzT+VJqb+n98dNRtdR90Q6GZBzCEbbWu0DxxuZ+GkpZ8Qi8+v96//p9W2D08e75wS1ma/vvCoaNz7phpXce7dnr9m2BovGdN7EAN/lr5R+o2P6HQ/dPBe3U5VKJNlcpWvxRN8TVF+3Nrhftybv6dhyAn18Mj3+278dJ17JfuF7ArhWxgbb1H+ndm6Bhu7u+f+y73O99Dcw1a7wFZoAm7/LYVPf2MJARgRlgYlk+Sz59Nt+4YhYzyguZOXEU7679Igt3X88f2ueytPxqfl49kUjY8Onzp7F8ez1L1uyHm56G216HT/0DPrMcRkyC5+5xy3fOuxlOWghYF6z/8b/w22vdpUdNNe5DuvUfsO1V1xoAVwt+5uuupb39VfcFMttVDDou6Vr2y/Qy1dYED14Fv76yc1KbZ0zNiy6NVa/BM3e7jdbCO4+5k/Kl77rJFCsfczPRx5zgAvPejVC3oXMiRDLh9u9pzX1/Nfz4bPjVZYeO9+3b4nogDj6mtfDyfa4XYflv4bsnHrmFtvw38O0proy7EWnff2hQTSZh7ZPwt7sPHavat+XAQN/WCA98yJVxuhWAI5XTrhWw5DPuvDjY/p2uzENZrK91Lbpd+5rSf8/DzYeIt8Ffvgh//ffeB9D6rTDhVIjkH/r/+Pv34bszoX4bPHy9K9d0HVxpW+9djrX9tZ5dxrJvsxtfhs7A3LoXHv8cUzf9yuW7ZrU7jxPxLi1m77rneBRWL3GfoZe+k/77dmfD3yDWDOv+0nnex9O4rvpgNavdZ/ft37nzcKAlE+5/GY/CWw8ded/ofvjJ+bDmz717r03PwzPfgMbd8MsPwGM3H718tv4dxhwPJRPc732ZAJZMQN06OPa8zm2hSGcrehjIGuoEDKbcSJgbzpjEDWdMwlrL4qXb+eZf1vC53+8FrgAaOX1KGTefM5WXNtTx+YeX89gbo3hXRSmXzJ7IpJwCuOkFchu3YsIRyCt1P5PPgTcWuTcpmQhrn3Ct31BW58xaE3KXc7TWu1bBnOvdh7hpF7xroVu8oWA0TD0P3n7YzRIfO9sdY88GGH+ye55dCIWj3Ym06o9ulbLRx8PvPgazr4Jp74GisZTsXw3nf8UtlvKP/3VrgEfr4dUfu7TYJFz2v24MfcGdLo1lx0LTA/CDue69j7vEpaulDmZf7a6v3fwCnPcl94U/djaMn+NeG4u6XoQVv3NfqtkFbp/qN11gOOdfXa9C8Th48Gq3vF7rvs7xfnAz5Z/6kpvgEWt1X64vfRcuuRee/T+w5SWX5nVPwqgZroITj8IfPuUqUFv/7t7v3DvBhDjprX+HV6rhhsdgynz3Hk/d5coA3L4X/SfsXuUuUXr0E+4L/poHYPRxrmyi9e6n8j/hpI+4ill2Qfcn2Ia/wcM3wDFnujKqmNv5N2vh8c9D1esuIFxyb+ffatfB/ee6oZGSY1hX18pFQLzBqzjs3QR//Yp731HT3bH274D5X3BXEDzyMfclee1v3flXMMotlANuYkuLF+TefhjO+BS88kNXOTjlo1A01gWraAMYA3kj3GNKLOoulyqd5I7ddU5Ga70LZPEoLLnddQmaMMy5zqXzSFY84r6Q59/hyiqZhPVPw5iZULMK1j0FJ1935GOk7NsCx14AwM7mJOOApsrvU9i6z7U8/vYf8OYDgHVjlJd+370unNO5wMgrP3LbNr/oenqshVHT0nt/cBWTuvWwaon7PdoAr90PK/8AO5bC6BPc52zmZYe+1lpXQR5/SmcF453H3GO8FVY/DnOu7f59l//WfY5O+SjkFKaf3q5q17rP1L4trvdu+YPu/9JY7c691GcnZenPYMcy+Mud7vsmNa6fjmQC/vwv7jutZhXYhDtv1j0Fx13cud/uVbDtH3DyR92+1W/Cxd90PSMFo10aT/nogedqV/u2QNG47tO27WXXSzLtPW797WTcDWeuf8oNQeSXpZ+fAWLsMLj8Ye7cuXbp0qX9eszKykoWLFhw1P0aWmNsrmumvDiH//vMBi6cWc55x4+huS3ONx5fxdtVDazetf+ACl15cQ5nHTuKkrwIz62tgZa9nD2igfefM5djJk8jq3YVRVXPk5/YT7zidGJJyNu1DFO/DXKKYNJZMPNykm/+BvvyDwjd+g/qfnEtoUlnMnL2Ra77Ges+6FjIHwkth2ntzL4KLvlveP5bbow61qU19pk3oeQY+N2Nnddtz7keala6L9qrFx14rHVPwW+udh/EqmXuWDPe52qpqVWfSiYe2J05+ngX5Ou3uS/ogjEQCrsv87Nudy3NpQdPoDMueG172bXU80a4O/1s/bs7ft1ad8zJ57h9xr7LfbmFcw6d1HPBV13tmy7/oAmnQl4ZbHjaBflog9tWNsX1Rsz9OIw6Dp78oktL6rVjTnQVpWgDzLgYtrzovixzS7pczmbcJKOSCvehL53kAmFbk+sKLBrneiKa6+C497mK0czLXMD665ddedWucf+HEZNcPlf/ydXgvWtvP37MU/zzls9yemgNrTOvIq/a63HJKYb9VS4N4Yj7giosh+o33BdW0y53vOwiOP0WNm+vZkrLWy49+aNcvqYu6Px/hLJg0rvd5MV2r1WWUwKn/7OrCBrjKkMv/wCu/IkL7PXb4P3/487NdU+6Vm75LDeJMbfUfcmVVLhyP/GDrru4fosr4vyyzpX1/voVN8YbbYDjP+Be8+qP4YP3uzkbuaVuoma81QXLnW+5cyqUBaXHuHxUzHW9Ho/dDOf9O5x7Bw8v/g3XrPqUy8u099JcvYaClu3uvDzr0+7KjIp5rjX64V+w4aVHmLzrKbKIw5mfdnnNLnLlMftqKCp3lRgTcnmZdBaMnO7Ow6YaF2g2P+8qcfFWt9+sD7mKUqzF/X9mX+W6yGvXwId/5j7Po2a4z0si5s7xJbe7c+fKn7hjLPm0K8P6rVBcAZd+zw1HhCOu8n7s+e78fugjLq8Fo+ED33OV6ZpVLlgfcyZEG3jhlWXMX3Ceqzi+8iNXqRs5HU6+3gXG3/2TO29KJ8G5X3SXbx7/AZfmWDNc8WP3fjbp8vjT97pKYv1WVwmbeh5Mf487P03I/Z9e/bErg2nvcd8D0Qb3f1v3FDx6k2tYJGPu8xVtcD11pcfARf/lfn/iDvfe5bPdubHhafjXte6z9savXfnMud79f0650X2WUtb+BR66zo1HX/a/vF35B971/pvd5/XJu9z5n1MCn3zRLaq0bzNc8SNXwS+d5I71wfvd91jTbtdDUDjm6JXNXjDGLLPWzj1ke6YH5nTsaojyj4117NofdZXb6gbe2FpPXVMbZ08fxcQR+fx11S527z/8JJKygmymjiogZExHJW/d7kYaWmNMHlXAptpmcrJCnD1tFOtrmrhiznjmlMVJ2iTR7DJyozWMKc6lLKudgng9OaEkOVkhYhNOZ1dTgrqmNo4fk0th01bY/ipr165l6jX3sGR5NYU5Yc4fuZdIa62b0GaMq6Ubg7UWYwwbappY8mYV7FjK5e+/lGNDu92X04RTXWKXLXIfimnvdQGreAKs/bP7UjchdzIfe4ELprFmN074rmtc9+fO5bDl7y6g7XwLyqa6YFX5X27iTms9tDW43oArfgg733YfzmPPgx+dDcXjXbCYdJabdHfydW54INEO7/1a5zh70Xj3Zb/ks5CMsXnC5Uy54ksucO9Z77qSR06HW55zQf73t7jH2R9y6TvzNtd6evF/XNApneRatmVT3fFjLa41VbPSfSnHWtx4XLTefWmPP8VVdrJy4PF/cRWNwnKXf3CVgU/8Df7yBVjzhMszuDRc9QvXs5FdyJnbP0UuMT7W/FOuzX+NSDgM1z8KE05xFYBk3AXIRz/hxlVP/Zj7Qn7yThes1j3lggW4QPa+b7my/f0/u22nfswFoaU/d/k85kwY9y533K3/6KzEpcy+2vVUPPsNF7hSsotg3ifc//yBK12vSP5I1yUaye+8jKk7I6fBjY+7Ct+yX7hzYMp8uObXLoA8/vnOiUiRfDfXo6TCfZnXbYDdXS4vjBTA9Y9ijzmDy/7nz9zW8D2aiqfz4Vu/waZHvsrUzQ/C5fe5QPT8vW4oyhjsx//KokX3c23sMdqOu4ziD98HD1/nzqcTLnWtbBN2lWmbdAG6u9neJuz2L6lwvRE3/snla/3T8PEnXc9SWyP87CJ37nSnYp5rGbZ2mTR22Q+8uSdf7TxPTMgFx1Rv3PiT4cL/4/73u1Z0BrzU/77jWnivAjpiMow7yX1+UmOrE051laARU1wlefFH3RDQuHe5823rSwdn2OXr1R+7FeG6yitz5WQT7rjbXz00r6OOg5OucT1el37fBfaVv4e3F3eWz/hTYN4trqdsfxUc93649jfub8kk/OJ9bvKuCbt8FU/wJtQ2ucrSiEnepEDb+T8KZ7uyO+NW1yOQXwa/usJ9Pj/5khs6yy113yuxboaupr3XfZZGHtv9/7AXFJgHQDJpCYVclG1tT/DypjpqG9sIGUNTW5x9ze3kRMJkhQzra5rYsa8Vi8VaFxcrRuQxujiH1zfv5ZLZ43h54x7eqW5g6qhCXt7U8/FAY6A4N0JRbhYm3kYynMOOejdWnBcJM6O8kEg4xPqaJkIGyotz2VjbxJiiXKobWjG47v6crBAXzxpLOGTIzQqTEwnR2p6kMDeLscW5RMKGrLAhOxwmNxIiJ8s95kbC7G1up7axreM4a3c3sj8a410TSplYlkdbPEkk7KY2/H1DHTPHFXP8uCLyI1nkRELEk5aXN+6hpT1OxYh8lm7ew9wpIznlmFKSFupb2inOi3QcIxpLsGJHA5NG5jOmyN2VqSkaoyAni+eff57588+lLZ4kLzvsvhxNGLLzicYS1Da2UVaQTUHOkUd0rLXEEpbsrMNMyUgm3Bfm4brV6ta7wDPmePcln3pN6rOXaIfsfLCW+pZ25nzjb9x23rHc99xG7rhwOredOxXCPRx1am/h+Zde5tzzL+jc1lTj3quk4siv3bfVC4rWfdF6rZGNu/bQsHEpp4zLccMeJRWdlbw1f3atqux8dwxrXUswku+6p41xvQjNNa4MRkzpLK9EzCuDLkME0QY38S631C2Tm7rjVsqeja7FVjDG9UKEs1i3u5ELv/sC5cU57GlqZ/l/XMibLz7NOeXNcOKVEPL+f7FWSCZYuSfB+//vi2QT59b3zORz75nROZcgO989z8rtfF283Q1FNO50ranCsW57+czOW0i2t7jXtjW6QFE0tkv517ru0qKxrvs4K9cF/Orl8J67XQu6+g3X8swvc61FY1w5bHnBTVItGgebnnOVvpwi11osHOPS9s4jbmx6xCRXLttfheLxbF67gimTjnHldMKlrtWdiLsK0JYX4ex/gYKR3Z8L0QY3HySnqHPxmwmnuMqGta4sa1bDpmfdOb2/2gXAs2536dj8gvt7wSg3PyEZd71RI491FZ9TPtrZ3RyLwtsPucrD5HNcGSfisLnS9coUjcVay2Nv7ODMyUWML8pyn6s3f+2CcCTP3e/AJl2eNr8Adet4a08WJ42Iuv/J1PNgxoWd+Vv3V9cDeNpNrkJbPssNNa7+k6ukFI5x/9uq12HpL+DmZw/8n/aRArPP7KhvpaElRigEYWNIWqiub2VfSzst7Qla2xO0tCeIZBlGFeZQlp/Nqp372dPURmM0zuYduygZUcZ1p08iHIIX19exoaaJ9niSKaMKSFrLrv1tTB9TyK6GKMeOLuCjZ02mKRrnsw+9SXVDlETSEo0liMYS5Gdn0dIeJ9mL0yUSNsQSfTvPsrNCxBNJkq6hT25WmKywoS2epD3uJlyNK8kllkhS19ROca5rLTTHDYmkpSgni/GleZQVZNPSHued6v0kkpbscIjp5YWEQwcGVWMMhTlhCnOyWLVzP7saopw6aQRlBdlkhUJkhQ1ZIUNWOETYGMIh1xMST1hW7dxPdjjExLI8srNCZIVCXmUmRENrjKVb9lKSF2HyyAJGF+UQT1pK8yO8tnkv2/a2sKm2mUUfn8e//2EFbbEkZ0wdSXFeFu3xJK2xJGEDxXkRqutbGV+ax+SRBbTGEmzb00LFiDyKcrOIJSxbN29k1gnHkRMJsbm2mfU1TZw+pYwxxbms293I7v1tnFRRQlFuhHDIEDIu34bOmBkOGbKzQmyua+Y//7ya5vYEnzz3WN47sxxrLa2xBNX1rby5rZ4TxhXzVlU9VXtbuemcKUwozaMkL0JuJExLe5yQMUTCruzCxv3vNtc1E40nKMmLMKE0j/qWGI8s244xhmtOm0hpXoSsUIhw2EsfnT1OHY8YWmMJ/uXh5Ty7tobvXTOHzz60nH9//wms37CRaN4orjylgr3NbVgLp00uY3RRDj98bgM/eG4DM8qLSFrLQ7ecyaPLqjAGrjt9kqvM9VJTW5yW9nhHZbG5LU5tYxtJa5kyqgBzuErcQRpaY+xrbmfyqMPMa0hDT78Lo7EEOVmhtNM4mH5YuYFvP7mWqaMKeOzWsyjNzz7qa46U//Z4EoslJyvMxtomKkbkkZN1mP97MuEqC/1o0AOzMeZi4PtAGPiptfabh9tXgbn/DUT+2+IJ9ja3E09Y4klLezzZEbjbvOdFuRHGleTSnkjS2p5gghco1u9uYkd9K3mRMNGYq1ScNW0kq3c2Ul3fSku7O44xMHtCCSV5EbbuaeHkY0p5fl0t2/a2kB0OUVaQzb6WGK3tcWIJSyRsOHXSCDbUNLFlTwsGmDyqgJ0NrVTtqGbWtMnk54Sp2d/GjvpW6lvaCYfca44py2djbTPrdx866zVh3ZdpYzTG+NI8po4qZOnWvbS0J4gnksQSlkTSEk8mSSQtSQtJazHAjPIi4klLdX0r8aQllkgST7jHnKwQp00po6U9wea6ZvY2u/S0x5NMKM1jbEkuuxqi/PkzZ/P8uloeWVbF1j0tNLXFyQ6HyM8O055I0tAaY1xJLlX7XNmBGy7Z29z99bupHpKdDW5GsjFQmJNFYzT9ZT9nTyhhenkhj71x6IzYguwwze0JciMhRhZ09tT0RpZXSYr3sBYYMvClS07ghjMnceo3/kZTm8tbYU5Wx/OujIHTp5Rx8YljuftPqw74W14k3BGYTZf96bKla9wyQCyRpLk9QUVpHlX7WmlPJBlTlOMF6c7FYsqLcxhf6m5UkrSQHwmTnx2mNZYgaS352VkkrTtflm3dRzSW5PixRYwvzaO2sY3mtjhFeRFK8yKU5EXYH42xqyFKOGQYW5xLfk4W1tqOWRc1NTWMHu0WEEr11u2ob6W5Pc6J40vIDodYWd1AXZPr7atpbKNiRB6zJ5RgLWysbWJ2RQll+dlYoCQvQiyR5K2qBgwwc3wxkY4KqqvctceTbKxtojEapyAnTMWIfLLDIcIhQ9Ja9jS3U13fyrTRhazd3cju/VHGluQxviSX8aV55GeHSVr3PZNMusf1NU08sWInp08p442t9YwvzeWKkyfQHk+Snx2mJC9CYW6Wq8iFTMfnr37HZiZPmwFAJBwinnSV9x31rTyxYifJpOWsY0fx5MpdHFOWzzWnTWRkQbbLZ9LS2h53DaFYgmvmTmRkYQ8mux3FoAZmY0wYWAe8F6gCXgeutdau6m5/Beb+l+n5B/+UgbWWprY4BdlZHUMj6Yonki5oZ4XIz3YBqD2eJBI2PPfCS8yddwbt8SSl+RFK87PZUd9KYzTG2OJcinMjbN/XQlvcVRyS1htm8YZbwAXHtliC0vxsZng9C6t3NrK7MUrYGPKzw4zw5k9sqmumNC9CUW6Ev2+oI5ZIUt8SIxpPUJCdRcJa4glLwqvMZIVDTB5ZQEFOmD1N7exsaCU3Eubs6aOIJywvrq8jnkx6r7EkuqTPlVtn+VkLZ00bxamTRgB4PQJRdq5fwWXvPZenvC/d7KwQK6oa2NPczr7mdj5w0niOKy/it69toy2e5Jzpo2hui/OXd3aRSNpD36vjf3bAfxBwvQt5kTDb9rZQMSKfscW5rNnVSGl+hNFFOYwqzKE9nuTVzXs6KlDGGFrbXeDOi4QxBlraE2SFDKGQ4YRxxUwdVcCza2poaI1RVpBNcV6E/a0xGryf/OwsJpTmkUgm2bW/jbaYVwkwrsLQ0tJCQUHBARWMMUW55EbCrNnleo6mjSmkYkQesYRlQmkea3btZ0NNE/Gka+G/tb2eNq9nKlXJmFFeiMGwvqax2560Y8ryGVGQzf7WGDvqW93/0NuxJM9V4DfWNjG+NI8Z5UXs3h+lur6VuqbuK5eji3J436yx3PW+E1i6dS/3PrWWt6saCIdMx3F7oigni3dPG0VDa4xXNu9h4WnH8Oa2fazZdfjL0x6//WxmTSjp8XsdzuEC80BdLjUP2GCt3eS9+UPA5UC3gVkkkxljKMqN9Oq1WeHQAd15hTlZ4FXoi7NNR8ssZUJpHtC5bdLInneRzhxfzEyKD9l+7OjOy3XOO35Mj497sI+cfkyvXzujvIgZ5UVU7nBzHy6fM6HjbyeOP/SL9eNnTzng99OnHmbMtR/0Jl+fOGdqr9/PVVDP7fXrDxZLJAl5wzcp1nb2GrmeI9PtnIxUQzDVTR5PJL1hoM5jpXrgskLGG2IxHRWVlHOmj+bsaaNoaU8c0IvU3OZ6tOJJS3lxLuGQ4fFnXuS8s8/03s8SCsGowhxyI+GONO1tbu9oCbe2J9jb0s7+1hiRcIi87DD5Xg9KzuHmmfSzgWoxfxi42Fr7Ce/3G4DTrbWf7rLPLcAtAOXl5ac+9NBRLmrvoaamJgoLe3ldXwBkev5BZZDp+QeVAagMhnP+zzvvvEFtMXfXH3dg54+19wP3g+vK7u8uR790Yw6UTM8/qAwyPf+gMgCVgR/zP1Dt8ipgYpffK4B+uFeXiIhIsA1UYH4dmG6MmWKMyQYWAksG6L1EREQCY0C6sq21cWPMp4GncJdL/dxaO0i36BEREfGvAbuJhbX2CeCJgTq+iIhIEGXMbR9FRET8QIFZRERkGFFgFhERGUYUmEVERIYRBWYREZFhZFjc9tEYUwts7efDjgLq+vmYfpLp+QeVQabnH1QGoDIYzvmfZK0dffDGYRGYB4IxZml3a5BmikzPP6gMMj3/oDIAlYEf86+ubBERkWFEgVlERGQYCXJgvn+oEzDEMj3/oDLI9PyDygBUBr7Lf2DHmEVERPwoyC1mERER31FgFhERGUYCF5iNMRcbY9YaYzYYY+4c6vQMFmPMFmPMCmPMcmPMUm9bmTHmaWPMeu9xxFCns78YY35ujKkxxrzTZdth82uMucs7J9YaYy4amlT3r8OUwd3GmB3eebDcGHNJl78FqgyMMRONMc8ZY1YbY1YaYz7rbc+Y8+AIZZBJ50GuMeY1Y8xbXhl8zdvu3/PAWhuYH9y9nzcCU4Fs4C1g5lCna5DyvgUYddC2bwN3es/vBL411Onsx/zOB04B3jlafoGZ3rmQA0zxzpHwUOdhgMrgbuDfutk3cGUAjANO8Z4XAeu8fGbMeXCEMsik88AAhd7zCPAqcIafz4OgtZjnARustZuste3AQ8DlQ5ymoXQ5sMh7vgi4YuiS0r+stS8Aew/afLj8Xg48ZK1ts9ZuBjbgzhVfO0wZHE7gysBau9Na+4b3vBFYDUwgg86DI5TB4QSxDKy1tsn7NeL9WHx8HgQtME8Atnf5vYojn6RBYoG/GmOWGWNu8baVW2t3gvsAA2OGLHWD43D5zbTz4tPGmLe9ru5U912gy8AYMxk4Gddaysjz4KAygAw6D4wxYWPMcqAGeNpa6+vzIGiB2XSzLVOuB3u3tfYU4H3AbcaY+UOdoGEkk86LHwHHAnOAncD/eNsDWwbGmELgUeBz1tr9R9q1m21BLYOMOg+stQlr7RygAphnjJl1hN2HfRkELTBXARO7/F4BVA9RWgaVtbbae6wBfo/rmtltjBkH4D3WDF0KB8Xh8psx54W1drf3JZUEfkJnF10gy8AYE8EFpAettY95mzPqPOiuDDLtPEix1tYDlcDF+Pg8CFpgfh2YboyZYozJBhYCS4Y4TQPOGFNgjClKPQcuBN7B5f1Gb7cbgT8OTQoHzeHyuwRYaIzJMcZMAaYDrw1B+gZc6ovI80HceQABLANjjAF+Bqy21n6ny58y5jw4XBlk2Hkw2hhT6j3PA94DrMHP58FQzz7r7x/gEtzMxI3Al4c6PYOU56m4WYZvAStT+QZGAs8A673HsqFOaz/m+be4LroYrgZ805HyC3zZOyfWAu8b6vQPYBn8GlgBvI37AhoX1DIAzsZ1Qb4NLPd+Lsmk8+AIZZBJ58G7gDe9vL4DfNXb7tvzQEtyioiIDCNB68oWERHxNQVmERGRYUSBWUREZBhRYBYRERlGFJhFRESGEQVmERGRYUSBWUREZBj5/2ju0xwRmyQ4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history).plot(figsize=(8, 5))\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4ab0311",
   "metadata": {},
   "source": [
    "### Covid Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0a395fc",
   "metadata": {},
   "source": [
    "I previously worked on this data in my machine-learning regression exercises. That's why I won't describe the data processing steps that I did to get data ready but you can find them in my machine learning repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "7edfdc69",
   "metadata": {},
   "outputs": [],
   "source": [
    "data=pd.read_csv(\"dataset/covidtenmost.csv\")\n",
    "data = data.rename(columns={'Sum_Confirmed': 'sum_Confirmed'})\n",
    "\n",
    "country_list = data['Country'].unique().tolist()\n",
    "population_list=[\"66727461\",\"64842509\",\"46647428\",\"17021347\",\"207833823\",\"11419748\",\"9904896\",\"8829628\",\"60673701\",\n",
    "                \"145530082\"]\n",
    "\n",
    "hospital_list=[\"1257\",\"3042\",\"782\",\"549\",\"6738\",\"165\",\"100\",\"350\",\"1048\",\"5300\",\n",
    "                ]\n",
    "df_population=pd.DataFrame(population_list,columns=[\"Population\"]) # Data frame of population\n",
    "df_hospital=pd.DataFrame(hospital_list,columns=[\"Number of Hospitals\"]) # Data frame of Hospitals\n",
    "df_country=pd.DataFrame(country_list,columns=[\"Country\"])\n",
    "df_GDP=pd.concat([df_country,df_population[\"Population\"].apply(pd.to_numeric,errors='coerce'),\n",
    "                  df_hospital[\"Number of Hospitals\"].apply(pd.to_numeric,errors='coerce')],axis = 1) # joining data frames\n",
    "df_GDP\n",
    "\n",
    "data_final = data.merge(df_GDP, how='left', on = 'Country')\n",
    "\n",
    "target=pd.DataFrame(data_final[\"sum_Deaths\"])\n",
    "\n",
    "data_final.drop([\"Date\",\"Lat\",\"Median_CaseFatalityRatio\",\"Median_IncidenceRate\",\"Lon\",\"sum_Deaths\"],axis=1,inplace=True)\n",
    "\n",
    "data_final_ohc=pd.get_dummies(data_final, columns=[\"Country\"], drop_first=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "dbc1ad5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_final.drop(\"Country\",axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "1cf3e1a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_valid, y_train, y_valid = train_test_split(data_final, target, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "0c2f8f5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X_train_s = scaler.fit_transform(X_train)\n",
    "X_valid_s = scaler.transform(X_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "282b5899",
   "metadata": {},
   "source": [
    "Time to construct our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94f0716b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(480, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(300, activation=\"relu\"),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(120, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.SGD())\n",
    "history = model.fit(X_train_s, y_train, epochs=50, validation_data=(X_valid_s, y_valid))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acc56aa0",
   "metadata": {},
   "source": [
    "Interesting, I don't think I have a problem with implementation but loss return NaN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc9a140f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(480, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(300, activation=\"relu\"),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(120, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.Adam())\n",
    "history = model.fit(X_train_s, y_train, epochs=50, validation_data=(X_valid_s, y_valid))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a38c52b0",
   "metadata": {},
   "source": [
    "I think there is a problem with the gradient, when I used Adam optimization, even though the loss is so high, it is calculated. I look for this nan loss problem on the internet. I found a StackOverflow post about that problem and using gradient_clipping solved the problem so it is probably about that gradients are exploding during the training. [stackoverflow_post](https://stackoverflow.com/questions/37232782/nan-loss-when-training-regression-network)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cdd3d81",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(600, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(480, activation=\"relu\"),\n",
    "    keras.layers.Dense(375, activation=\"relu\"),\n",
    "    keras.layers.Dense(300, activation=\"relu\"),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(128, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.SGD(clipnorm=1))\n",
    "history = model.fit(X_train_s, y_train, epochs=100, validation_data=(X_valid_s, y_valid), batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7117003",
   "metadata": {},
   "source": [
    "Let's analyze the situation a bit. When we use SGD, we can handle the exploding problem using `clipnorm` and we are dealing with a regression problem. The problem is probably caused by the different scales in our predictors and target. Let's scale our target variable and run the model again. \n",
    "\n",
    "You can find more about scaling in this article: [link](https://machinelearningmastery.com/how-to-improve-neural-network-stability-and-modeling-performance-with-data-scaling/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "14620741",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler2 = StandardScaler()\n",
    "y_train_s = scaler2.fit_transform(y_train)\n",
    "y_valid_s = scaler2.transform(y_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8e598e7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(600, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(480, activation=\"relu\"),\n",
    "    keras.layers.Dense(375, activation=\"relu\"),\n",
    "    keras.layers.Dense(300, activation=\"relu\"),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(128, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.SGD())\n",
    "history = model.fit(X_train_s, y_train_s, epochs=100, validation_data=(X_valid_s, y_valid_s), batch_size=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "8540d9f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24/24 [==============================] - 0s 3ms/step - loss: 0.0075\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.007515334989875555"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_valid_s, y_valid_s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "dae71e11",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(X_valid_s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "ab360dc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "inversed = scaler2.inverse_transform(y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "9b55a958",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12544317.143281946"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "mean_squared_error(y_valid,inversed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f085935",
   "metadata": {},
   "source": [
    "Well looks like my model doesn't perform well on this dataset. I will revisit this model at the end of this notebook to improve it a bit more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1e82572",
   "metadata": {},
   "source": [
    "Another option that we can use for scaling the data is the normalization layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "2f19455f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import layers\n",
    "\n",
    "predictor_normalizer = layers.experimental.preprocessing.Normalization(input_shape=data_final.shape[1:], axis=None)\n",
    "\n",
    "predictor_normalizer.adapt(np.array(data_final))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50381b2e",
   "metadata": {},
   "source": [
    "Let's also scale our target feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "b02bb8e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "scaler_norm = MinMaxScaler()\n",
    "y_train_norm = scaler_norm.fit_transform(target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "1b610c14",
   "metadata": {},
   "outputs": [],
   "source": [
    "horsepower_model = keras.Sequential([\n",
    "    predictor_normalizer,\n",
    "    keras.layers.Dense(300, activation=\"relu\",input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(256, activation=\"relu\"),\n",
    "    keras.layers.Dense(128, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c35f584c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.Adam())\n",
    "history = model.fit(data_final, y_train_norm, epochs=100, batch_size=128,validation_split = 0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "dc0df481",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.3762266049635104e+17"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ypredict=model.predict(data_final)\n",
    "inversed = scaler_norm.inverse_transform(ypredict)\n",
    "mean_squared_error(target,inversed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7104461d",
   "metadata": {},
   "source": [
    "There is very good documentation about this normalization layer : [link](https://www.tensorflow.org/tutorials/keras/regression#regression_using_a_dnn_and_multiple_inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76829909",
   "metadata": {},
   "source": [
    "## Deep and Wide Neural Network for Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5f3b66a",
   "metadata": {},
   "source": [
    "Okay now, Let's also use this deep and wide architecture for a regression problem. Here, I will fit the model on the Boston dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "a7f7ddf4",
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_valid, y_valid)=tf.keras.datasets.boston_housing.load_data(\n",
    "      path=\"boston_housing.npz\", test_split=0.3, seed=85)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c85904a",
   "metadata": {},
   "source": [
    "Splitting the training instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "7a24d3f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(x_train)\n",
    "X_valid = scaler.transform(x_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "d87f9bfe",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_wide, X_train_deep = X_train[:, :5], X_train[:, 2:]\n",
    "X_valid_wide, X_valid_deep = X_valid[:, :5], X_valid[:, 2:]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51be563c",
   "metadata": {},
   "source": [
    "Building the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "50a687cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_wide = keras.layers.Input(shape=X_train_wide.shape[1:], name=\"wide_model\")\n",
    "input_deep= keras.layers.Input(shape=X_train_deep.shape[1:], name=\"deep_model\")\n",
    "hidden_layer1 = keras.layers.Dense(30, activation=\"relu\")(input_deep)\n",
    "hidden_layer2 = keras.layers.Dense(30, activation=\"relu\")(hidden_layer1)\n",
    "concat_layer = keras.layers.concatenate([input_wide, hidden_layer2])\n",
    "output = keras.layers.Dense(1, name=\"output\")(concat_layer)\n",
    "model = keras.models.Model(inputs=[input_wide, input_deep], outputs=[output])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "883abf08",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"mean_squared_error\", optimizer=keras.optimizers.SGD(lr=1e-3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "f48efd72",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(354, 11)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_deep.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34b7d185",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "history = model.fit((X_train_wide, X_train_deep), y_train, epochs=30,validation_data=((X_valid_wide, X_valid_deep), y_valid))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "e3854886",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 799us/step - loss: 15.7981\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "15.798072814941406"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate((X_valid_wide, X_valid_deep), y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18ff0c6c",
   "metadata": {},
   "source": [
    "## Sub Class API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15296042",
   "metadata": {},
   "source": [
    "Up to now, we only dealt with Sequential and Functional API, Now It's time to go to the third one which provides us the most flexibility. The functional API is declarative, in other words, it is static. We cannot add loops, or basic dynamic behaviors using Sequential and Functional API. On the other hand, Sub Class API provides us with that freedom."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "51832cb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class WideAndDeepModel(keras.models.Model):\n",
    "    def __init__(self, units1=60,units2=30, activation=\"relu\", **kwargs):\n",
    "        super().__init__(**kwargs)\n",
    "        self.hidden_layer1 = keras.layers.Dense(units1, activation=activation)\n",
    "        self.hidden_layer2 = keras.layers.Dense(units2, activation=activation)\n",
    "        self.model_output = keras.layers.Dense(1)\n",
    "        \n",
    "    def call(self, inputs):\n",
    "        input_wide, input_deep = inputs\n",
    "        hidden_layer1 = self.hidden_layer1(input_deep)\n",
    "        hidden_layer2 = self.hidden_layer2(hidden_layer1)\n",
    "        concat_layer = keras.layers.concatenate([input_wide, hidden_layer2])\n",
    "        output = self.model_output(concat_layer)\n",
    "        return output\n",
    "\n",
    "model = WideAndDeepModel(60,30,activation=\"relu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "581fee31",
   "metadata": {},
   "source": [
    "The basic steps to do\n",
    "1. Create the layers that you are going to use in your model in the constructor.\n",
    "2. By using the method `call()` perform the computations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33933a60",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model.compile(loss=\"mse\", optimizer=keras.optimizers.SGD(lr=1e-3))\n",
    "\n",
    "history = model.fit((X_train_wide, X_train_deep), y_train, epochs=30,validation_data=((X_valid_wide, X_valid_deep), y_valid))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "99ed64d8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 1ms/step - loss: 14.7139\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "14.713883399963379"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate((X_valid_wide, X_valid_deep), y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d788b66",
   "metadata": {},
   "source": [
    "Subclass API looks very much like the Functional API (actually it looks more like PyTorch). Using SubClass API you can do pretty much anything you want in the `call()` method: for loops, if statements. etc..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa2d0d70",
   "metadata": {},
   "source": [
    "Now Let's also add an additional output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "e65589fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_wide = keras.layers.Input(shape=X_train_wide.shape[1:], name=\"wide_model\")\n",
    "input_deep= keras.layers.Input(shape=X_train_deep.shape[1:], name=\"deep_model\")\n",
    "hidden_layer1 = keras.layers.Dense(30, activation=\"relu\")(input_deep)\n",
    "hidden_layer2 = keras.layers.Dense(30, activation=\"relu\")(hidden_layer1)\n",
    "concat_layer = keras.layers.concatenate([input_wide, hidden_layer2])\n",
    "main_output = keras.layers.Dense(1, name=\"output\")(concat_layer)\n",
    "additional_output = keras.layers.Dense(1, name=\"additional_output\")(hidden_layer2)\n",
    "model = keras.models.Model(inputs=[input_wide, input_deep], outputs=[main_output, additional_output])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "7f88c47b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=[\"mse\", \"mse\"], loss_weights=[0.9, 0.1], optimizer=keras.optimizers.SGD(lr=1e-3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cceb915b",
   "metadata": {},
   "source": [
    "Each output needs its own loss function. Therefore, when we compile the model, we should pass a list of losses (if we pass a single loss, Keras will assume that the same loss must be used for all outputs). By default, Keras will compute all these losses and simply add them up to get the final loss used for training. We care much more about the main output than about the auxiliary output (as it is just used for regularization), so we want to give the main output’s loss a much greater weight. Fortunately, it is possible to set all the loss weights when compiling the model: - [Géron, A. (2019)](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cbaa765",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "history = model.fit((X_train_wide, X_train_deep),[y_train, y_train], epochs=30,\n",
    "                    validation_data=((X_valid_wide, X_valid_deep), [y_valid, y_valid]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "7ffc3259",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 1ms/step - loss: 14.9746 - output_loss: 14.9761 - additional_output_loss: 14.9613\n"
     ]
    }
   ],
   "source": [
    "total_loss, main_loss, aux_loss = model.evaluate(\n",
    "    [X_valid_wide, X_valid_deep], [y_valid, y_valid])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41020217",
   "metadata": {},
   "source": [
    "Now when we train the model, we need to provide labels for each output. In this example, since we are trying to predict the same thing, we need to pass (y_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "8c49029d",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_main, y_pred_aux = model.predict([X_train_wide, X_train_deep])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d3580ef",
   "metadata": {},
   "source": [
    "## Regression for Covid Dataset Revisited"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27b7f7c3",
   "metadata": {},
   "source": [
    "Okay, the regression performance for Covid Dataset was awful. I thought I could perhaps improve it a bit and built the model below. I didn't improve much, in fact, I could improve it more if I used hyperparameter tuning but I keep this subject for another day. On the other hand, I don't think spending time on this regression problem is important since deep learning models are rarely used for regression.\n",
    "\n",
    "A good discussion about Regression with MLP [link](https://stats.stackexchange.com/questions/319349/why-doesnt-deep-learning-work-as-well-in-regression-as-in-classification)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3ff0ad3",
   "metadata": {},
   "source": [
    "To omit the Country column while scaling I will use `ColumnTransformer()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "40096bb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_pipeline = Pipeline([\n",
    "         ('polynomial_features',PolynomialFeatures(degree=3)),#4\n",
    "        ('minmax-scaler', MinMaxScaler()),\n",
    "        \n",
    "    ])\n",
    "\n",
    "preprocessor = ColumnTransformer(\n",
    "        remainder='passthrough', #passthough features not listed\n",
    "        transformers=[\n",
    "            ('mm', num_pipeline , ['sum_Confirmed','sum_Recovered','sum_Active','Population','Number of Hospitals'])\n",
    "        ])\n",
    "covid_prepared = preprocessor.fit_transform(data_final_ohc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "47f2b9e2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3011, 65)"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "covid_prepared.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2a0b5a1",
   "metadata": {},
   "source": [
    "Let's split our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "3e6f9992",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_valid, y_train, y_valid = train_test_split(covid_prepared, target, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "922a8e8b",
   "metadata": {},
   "source": [
    "Normalizing the target:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "153d7fc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "normalizer=MinMaxScaler()\n",
    "y_train_norm=normalizer.fit_transform(y_train)\n",
    "y_valid_norm=normalizer.transform(y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad4207f0",
   "metadata": {},
   "source": [
    "Implementing the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bba57f4e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.Dense(420, activation=\"relu\",input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(300, activation=\"relu\"),\n",
    "    keras.layers.Dense(150, activation=\"relu\"),\n",
    "    keras.layers.Dense(60, activation=\"relu\"),\n",
    "    keras.layers.Dense(30, activation=\"relu\"),\n",
    "    keras.layers.Dense(1)\n",
    "])\n",
    "model.compile(loss=\"mean_absolute_error\", optimizer=keras.optimizers.Adam(lr=3e-4))\n",
    "history = model.fit(X_train, y_train_norm, epochs=4000, validation_data=(X_valid, y_valid_norm),batch_size=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "fbd0aab4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "194023.6878192994"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict_adam=model.predict(X_valid)\n",
    "\n",
    "mean_squared_error(y_valid,normalizer.inverse_transform(predict_adam))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "b1233d19",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAEvCAYAAACKSII9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtM0lEQVR4nO3deXhU5d3G8e9vkkCQTdawFlCpiCCIqFQtxqWAuOD2Krhv9XXXui/VYqtVS2tbW9Ti8ipuaJUqKu6SAorKIqssshNAlrCGEJLMPO8fZyDbJJmEwMw5uT/XlYuZc55z5vlxIPec52zmnENERESSUyjRHRAREZGKKahFRESSmIJaREQkiSmoRUREkpiCWkREJIkpqEVERJJYaqI7EEvLli1d586da219O3bsoGHDhrW2vkQKSi1BqQNUS7IKSi1BqQNUS2WmT5++0TnXKta8pAzqzp07M23atFpbX1ZWFpmZmbW2vkQKSi1BqQNUS7IKSi1BqQNUS2XMbEVF8zT0LSIiksQU1CIiIklMQS0iIpLEkvIYtYiI+EthYSHZ2dnk5+dX2KZp06bMnz9/P/Zq36lpLenp6XTo0IG0tLS4l1FQi4jIXsvOzqZx48Z07twZM4vZZvv27TRu3Hg/92zfqEktzjlycnLIzs6mS5cucS+noW8REdlr+fn5tGjRosKQFjAzWrRoUemoQywKahERqRUK6arV5O9IQS0iIoHQqFGjRHdhn1BQi4iIJLHAB/V7M1ezcFM40d0QEZH9xDnHXXfdRY8ePejZsydvvvkmAGvXrqV///707t2bHj16MGnSJMLhMFdcccWetn/9618T3PvyAn/W9+MfLeCQRkX8b6I7IiIi+8XYsWOZOXMms2bNYuPGjRx99NH079+f119/nYEDB/LAAw8QDofJy8tj5syZrF69mrlz5wKwZcuWxHY+hsAHtYiI7F8Pvz+PH9ZsKzc9HA6TkpJSo3V2b9eE3515eFxtJ0+ezLBhw0hJSSEjI4MTTzyRqVOncvTRR3PVVVdRWFjI2WefTe/evTnooINYunQpN998M6effjoDBgyoUf/2pcAPfYuISN3inIs5vX///kycOJH27dtz6aWXMnr0aJo1a8asWbPIzMxk5MiRXHPNNfu5t1XTHrWIiNSqivZ899cNT/r378+//vUvLr/8cjZt2sTEiRMZMWIEK1asoH379vz6179mx44dzJgxg8GDB1OvXj3OO+88Dj74YK644op93r/qCnxQ66o+EZG65ZxzzmHKlCn06tULM+NPf/oTbdq04eWXX2bEiBGkpaXRqFEjRo8ezerVq7nyyiuJRCIAPPbYYwnufXmBD2oREakbcnNzAe+mIiNGjGDEiBGl5l9++eVcfvnl5ZabMWPGfulfTdWJY9Sxj1aIiIgkv8AHtW5pJyIifhb4oBYREfEzBbWIiEgSiyuozWyQmS00s8Vmdm+M+WZmT0XnzzazPmXmp5jZ92b2QW11vDoquKROREQk6VUZ1GaWAowETgO6A8PMrHuZZqcBXaM/1wLPlJl/KzB/r3srIiJSx8SzR30MsNg5t9Q5VwCMAYaUaTMEGO083wAHmllbADPrAJwOPF+L/RYREakT4gnq9sCqEu+zo9PibfM34G4gUrMuioiI1K7Knl29fPlyevTosR97U7l4bngS6/qmskd9Y7YxszOA9c656WaWWemHmF2LN2xORkYGWVlZcXStart25VNYL1Jr60u03NzcQNQSlDpAtSSroNTilzqaNm3K9u3bK20TDoerbLM/VdSX3NxcIpFIpX3dm1ry8/OrtU3jCepsoGOJ9x2ANXG2OR84y8wGA+lAEzN71Tl3SdkPcc6NAkYB9O3b12VmZsZbQ6XSv/2StNRCamt9iZaVlRWIWoJSB6iWZBWUWvxSx/z586u8j/e+vNf3PffcQ6dOnbjhhhsAGD58OGbGxIkT2bx5M4WFhTzyyCMMGVJ85LaivjRq1IhQKETjxo3Jz8/n+uuvZ9q0aaSmpvLkk09y0kkn8d1333HTTTdRUFBAJBLhnXfeoV27dlxwwQVkZ2cTDod58MEHufDCC8utPz09nSOPPDLu2uIJ6qlAVzPrAqwGhgIXlWkzDrjJzMYAxwJbnXNrgfuiP0T3qO+MFdIiIhIgH90LP80pN7lBuAhSanjn6jY94bTHK5w9dOhQbrvttj1B/dZbb/Hxxx/zm9/8hiZNmrBx40b69evHWWedVa0bYY0cORKAOXPmsGDBAgYMGMCiRYt44YUXuPXWW7n44ospKCggHA4zfvx42rVrx4cffgjA1q1ba1ZrGVUeo3bOFQE3AZ/gnbn9lnNunpldZ2bXRZuNB5YCi4HngBtqpXe1RFdniYgE25FHHsn69etZs2YNs2bNolmzZrRt25b777+fI444glNPPZXVq1ezbt26aq138uTJXHrppQB069aNTp06sWjRIo455hj++Mc/8sQTT7BixQoaNGhAz549+fzzz7nnnnuYNGkSTZs2rZXa4vpq45wbjxfGJac9W+K1A26sYh1ZQFa1e7iXdAdREZH9rII93537+DGX559/Pm+//TY//fQTQ4cO5bXXXmPDhg1Mnz6dtLQ0OnfuTH5+frXWWdGzrS+44AIyMzP58MMPGThwIM8//zwnn3wy06dPZ/z48dx3330MGDCAhx56aK/r0p3JREQkEIYOHcqYMWN4++23Of/889m6dSutW7cmLS2NCRMmsGLFimqvs3///rz22msALFq0iJUrV3LooYeybNkyDjroIG655RbOOussZs+ezZo1azjggAO45JJLuPPOO2vtqVx14jGXToPfIiKBd/jhh7N9+3bat29P27ZtufjiiznzzDPp27cvvXv3plu3btVe5w033MB1111Hz549SU1N5aWXXqJ+/fqMHTuWYcOGkZaWRps2bXjooYeYOnUqd911F6FQiLS0NJ55puy9v2om8EFtMa8cExGRIJozp/gktpYtWzJlypSY7XY/uzqWzp07M3fuXMA7Q/ull14q1+aOO+5g+PDhpaYNHDiQgQMHVr/TVdDQt4iISBIL/B61iIhILHPmzNlzRvdu9evX59tvv01Qj2KrG0GtQ9QiIlJGz549mTlzZqK7UaXAD33r8iwRkf2jokuZpFhN/o4CH9QiIrLvpaenk5OTo7CuhHOOnJwc0tPTq7VcnRj61j8bEZF9q0OHDmRnZ7Nhw4YK2+Tn51c7pJJVTWtJT0+nQ4cO1Vom8EGtkW8RkX0vLS2NLl26VNomKyurWg+jSGb7sxYNfYuIiCQxBbWIiEgSU1CLiIgkscAHdXWeOyoiIpJsAh/UIiIiflYnglqX9YmIiF8FPqg18C0iIn4W+KAWERHxMwW1iIhIEqsTQa1D1CIi4lfBD2odpBYRER8LflCLiIj4mIJaREQkiQU+qDXyLSIifhb4oBYREfEzBbWIiEgSqxNBrcuzRETErwIf1Hp6loiI+Fngg1pERMTP6kRQ6+lZIiLiV4EPag18i4iInwU+qEVERPxMQS0iIpLEFNQiIiJJLPBBrauzRETEzwIf1CIiIn5WJ4JaV2eJiIhfBT6oTRdoiYiIjwU+qEVERPxMQS0iIpLEFNQiIiJJLPBBrcuzRETEzwIf1CIiIn5WJ4JaT88SERG/qhNBLSIi4lcKahERkSSmoBYREUlidSKodYhaRET8KvBBbbo+S0REfCzwQS0iIuJncQW1mQ0ys4VmttjM7o0x38zsqej82WbWJzo93cy+M7NZZjbPzB6u7QJERESCrMqgNrMUYCRwGtAdGGZm3cs0Ow3oGv25FngmOn0XcLJzrhfQGxhkZv1qp+vx0cC3iIj4WTx71McAi51zS51zBcAYYEiZNkOA0c7zDXCgmbWNvs+NtkmL/ujcLhERkTjFE9TtgVUl3mdHp8XVxsxSzGwmsB74zDn3bY17KyIiUsekxtEm1uhx2b3iCts458JAbzM7EPiPmfVwzs0t9yFm1+INm5ORkUFWVlYcXatabu5OQmnhWltfouXm5gailqDUAaolWQWllqDUAaqlpuIJ6mygY4n3HYA11W3jnNtiZlnAIKBcUDvnRgGjAPr27esyMzPj6FrVGs+eRGrhDmprfYmWlZUViFqCUgeolmQVlFqCUgeolpqKZ+h7KtDVzLqYWT1gKDCuTJtxwGXRs7/7AVudc2vNrFV0TxozawCcCiyove6LiIgEW5V71M65IjO7CfgESAFedM7NM7ProvOfBcYDg4HFQB5wZXTxtsDL0TPHQ8BbzrkPar+MKmrY3x8oIiJSS+IZ+sY5Nx4vjEtOe7bEawfcGGO52cCRe9nHvaIbk4mIiJ/pzmQiIiJJTEEtIiKSxBTUIiIiSSzwQW26iaiIiPhY4INaRETEz+pEUOvyLBER8avAB7UuzxIRET8LfFCLiIj4mYJaREQkidWNoNZBahER8anAB7UOUYuIiJ8FPqhFRET8TEEtIiKSxOpEUOsQtYiI+FXwg1oXUouIiI8FP6hFRER8rE4EtYa+RUTErwIf1Br4FhERPwt8UIuIiPiZglpERCSJ1Y2g1kFqERHxqcAHta7OEhERPwt8UIuIiPhZnQhqp7FvERHxqcAHtUa+RUTEzwIf1CIiIn6moBYREUliCmoREZEkFvigNl2fJSIiPhb4oBYREfGzOhHUujhLRET8KvBBrYFvERHxs8AHtYiIiJ8pqEVERJJYnQhqp4PUIiLiU4EPal2dJSIifhb4oBYREfEzBbWIiEgSC3xQmy7QEhERHwt8UIuIiPiZglpERCSJ1Ymg1tVZIiLiV8EPah2iFhERHwt+UIuIiPhYnQhq3ZlMRET8KvBBrZFvERHxs8AHtYiIiJ8pqEVERJKYglpERCSJBT6o9fQsERHxs7iC2swGmdlCM1tsZvfGmG9m9lR0/mwz6xOd3tHMJpjZfDObZ2a31nYBIiIiQVZlUJtZCjASOA3oDgwzs+5lmp0GdI3+XAs8E51eBNzhnDsM6AfcGGPZfU5XZ4mIiF/Fs0d9DLDYObfUOVcAjAGGlGkzBBjtPN8AB5pZW+fcWufcDADn3HZgPtC+FvtfJT09S0RE/CyeoG4PrCrxPpvyYVtlGzPrDBwJfFvtXoqIiNRRqXG0ibVLWnY0udI2ZtYIeAe4zTm3LeaHmF2LN2xORkYGWVlZcXStalu27CQcDtfa+hItNzc3ELUEpQ5QLckqKLUEpQ5QLTUVT1BnAx1LvO8ArIm3jZml4YX0a865sRV9iHNuFDAKoG/fvi4zMzOOrlXtX4u+IWfzZmprfYmWlZUViFqCUgeolmQVlFqCUgeolpqKZ+h7KtDVzLqYWT1gKDCuTJtxwGXRs7/7AVudc2vNzIAXgPnOuSdrtedx0uVZIiLiZ1XuUTvniszsJuATIAV40Tk3z8yui85/FhgPDAYWA3nAldHFjwcuBeaY2czotPudc+NrtQoREZGAimfom2iwji8z7dkSrx1wY4zlJpMEz8XQ07NERMSvdGcyERGRJBb4oBYREfEzBbWIiEgSqxNBrUPUIiLiV4EPat1CVERE/CzwQS0iIuJnCmoREZEkFvig1uVZIiLiZ4EPahERET9TUIuIiCSxOhHUuoWoiIj4VZ0IahEREb9SUIuIiCSxOhHUGvkWERG/CnxQm67PEhERHwt8UIuIiPiZglpERCSJKahFRESSWOCDWkeoRUTEzwIf1CIiIn5WJ4Jal2eJiIhfBT6odXWWiIj4WeCDWkRExM8U1CIiIkmsbgS1DlKLiIhPBT6odYhaRET8LPBBLSIi4md1Iqg18i0iIn4V+KDW07NERMTPAh/UIiIifqagFhERSWKBD+qG4a00cHmJ7oaIiEiNpCa6A/vaw2uuY4o7HDgv0V0RERGptsDvUTuMEJFEd0NERKRGAh/UEQsR0gVaIiLiU4EPakdIe9QiIuJbgQ9q7VGLiIifBT6otUctIiJ+FvigjhDCnIJaRET8KfBB7Z31raFvERHxp8AHdcRSNPQtIiK+Ffig1nXUIiLiZ4EP6oiFMA19i4iITwU+qB0hUrRHLSIiPhX4oI6gPWoREfGvwAe1Mx2jFhER/wp8UEfQWd8iIuJfgQ9qnfUtIiJ+FvigjlgIczpGLSIi/hT4oEZnfYuIiI8FPqidpYDO+hYREZ+KK6jNbJCZLTSzxWZ2b4z5ZmZPRefPNrM+Jea9aGbrzWxubXY8bmbaoxYREd+qMqjNLAUYCZwGdAeGmVn3Ms1OA7pGf64Fnikx7yVgUG10tiacpWAKahER8al49qiPARY755Y65wqAMcCQMm2GAKOd5xvgQDNrC+Ccmwhsqs1OV4uFiEQiOJ1QJiIiPpQaR5v2wKoS77OBY+No0x5YG29HzOxavL1xMjIyyMrKinfRyuXuoi2Oh175nFN+llY760yg3Nzc2vu7SaCg1AGqJVkFpZag1AGqpabiCWqLMa3s7mk8bSrlnBsFjALo27evy8zMrM7iFfp4wmOEcOyo34rMzN61ss5EysrKorb+bhIpKHWAaklWQaklKHWAaqmpeIa+s4GOJd53ANbUoE1CRKI3PEkJxfouISIiktziCeqpQFcz62Jm9YChwLgybcYBl0XP/u4HbHXOxT3svS9FCBEiQmqKglpERPynyqB2zhUBNwGfAPOBt5xz88zsOjO7LtpsPLAUWAw8B9ywe3kzewOYAhxqZtlmdnUt11CpCCG6hNZx4apH9ufHioiI1Ip4jlHjnBuPF8Ylpz1b4rUDbqxg2WF708G9FXEGBr03f5LIboiIiNRI4O9M1tK2JLoLIiIiNRb4oD7BZie6CyIiIjUW+KAWERHxs7oV1HmJu0GaiIhITdStoH4xYbccFxERqZHAB/WuY28ufrNxYeI6IiIiUgOBD+rwEQm9OkxERGSvBD6oQ6G4LhUXERFJSoEPaksJfIkiIhJggU+xFO1Ri4iIjwU+qEMpKYnugoiISI0FP6hDCmoREfGvwAc1CmoREfGx4Ae1xQjqGaNh9fT93xcREZFqCv6ZVmX3qLf/BOOiN0EZvnX/90dERKQagr9HfUDz0u/DhYnph4iISA0EP6jLihQlugciIiJxq3NBnbMtN9FdEBERiVudCOr53W7b89revCRxHREREammOhHU69qctOd1853LE9cRERGRaqoTQV2h4U0hEkl0L0RERCpUt4MaIH8LjOwHP81NdE9ERETKqTtBPfjPsad/Phw2zIcJj+7X7oiIiMSj7gT1Mb+OPX3Gy96fkfD+64uIiEic6k5QV0XXV4uISBJSUEdFCvNg2xooKkh0V0RERPZQUEeFVk6BJw8rvg+4iIhIElBQlxH+YVzxm63ZsGxi4jojIiJ1noK6jJSiPNzGxd6bkf3g5TMT2yEREanT6lZQnzo8rmZr3rrde1Gwfd/1RUREJA51K6iP/jX0vhjuXlZps815ehSmiIgkh7oV1PUbwdlPl39GdRk/bd1JYcGuPe9zd5W4dOvV82DB+H3VQxERkVLqVlDHqXtoBQv+dMqe9zlLZsCWVeAcLP4cxgwrbrzme3jzEgjrOmwREal9qYnuQMKccDs0bOUF75IvSs1qZ5toV7Rpz/tOb/3Ke/FgTvn1vHMN5CyGzcugZdfYn7V9HaSkVbknLyIiUlbdDepTf+f92ah1uaCukCvzpK1wIYTSoq8ruVHKX37u/Tn0dWhzBBzYsXp9FRGROktD3z3Pj7vpypxtxW82LYU/tPQe6AFs2p4LBTugcKc3vzDfu3/4unnFy4y5CJ4/BRERkXgpqAEufieuZhOev3/P6x9nfVVq3vRlOfDHdt7dzQAezYC3r4Ki/NIryV23V10VEZG6RUEN0PVUuHUWtKjgGHPU5YVv7nk9+Yv3S81L2ejtWbNzc/HEH95lwvP3lV/R08dB3qby0/el9fPhx8/372eKiMheU1Dv1qxztYbBr0z9pNT7kxf+fs/rTbnFl3ad5L4tv/D6efDR3aWnzXwdHm3rBX0kArnrYXEcx87zt8L6BbBkAmvffYgV45+M3e7pfvDaeVWvryKbl8O0F2u+fCKtmwe5GxLdCxGRGlFQl9T/brh3FfS5bK9Wk/P3E6tuNOffMLwp7r3oQ0A+uR8K8+CJzsz8v1vhpTPg1XO90C7BOYdb+BFMeMyb8MJAePpYeOVs2s78O52+exiKdhGvcMSxYXsc7V88DT74TXI/XSxcBHPf8S6jK+mZ4+CfRyWmTyJSPUv/CzPfSHQvkoqCuqRQCNKbQOb90O0MOOdfNVpN18L5cbe170fD8Kalhsw7rvwPbuMi701hnvfnp79l9ofPcP8nq7A3hsJ/H2frzsI9J7OV8khriua9X356DH/6ZAFHP/o5m3ZUEsDrfoDtawCILJtc+Qrzt8KT3WFljJGEfW3KP73zAsqOVuzuVzIq2FHzZWf/27u8UCRIRp8F716X6F4kFQV1LE3awtDX4IgLoV2f/f7xB7ALw9srHPvIhV6Qf/0Pjph6L1PSix/D2evhTypaBWvf/wN8/U8YdVKpR3feOfINlm3IhbWzYXhTls31AjUnt4K96kgEnvnFnreh187xXmxb612ettvKb72z3LOnwbbVkPXH6hW97gfvpjKxPv+HceVGFkrZuRl25XrPEwf4bhR89xy8f2v1+rC/Lf/KOwFxyZc1W37sNd6d8kT2h83LdQgpQRTUlTGDaz6Hs5/Zrx/bwIr3bs9NqXgPdnn6xRXO65i/ED59ANbMgBmj90z/84br6DKyPfzrlwCM2nEry9MvYmteiaDekQMFO4hEHG7FV2VX7V2C9mQ3+HNXeOZ4mPM2vDgAvvob7PIeZBJxxhMfL2B7fiX3TXcOZrxCUf4O78vA33p40/O3gXOM+GQBm16+CN661PupyBOdify9F1BiyHv8nTD9pVLN8gqK+GD2muLPLnnp3M7N8O6N3mfvLyu/9v5cXsUoRXWsm+d9scueVnvrFAH4ey/48yGJ7kWdpKCuSigFel8E962GU35XPP3EexPXp33gnlFj2fje/fwwYxKMOAj+2I4bfjsce/mMcm3/8uq73oudm2HdXHjnau/9F7+Hf18OQGjZBC6bchr3vRLdW5z6PF9N+pLJDx4Hw5uSmTWEon/2g3E38d+R1xevfHhTeLwjbvxdfJg1meYrPvKmL/iAb//zTwBW5kQPB0QiMHes93l5G8nZnleur6teKV732NeeYfA73Zn14zL4+D7v2PWqqQAUTf4HzHyV7ZOfrfova/tPsafvyIFFnxRfSx/L+7d6Jw6WtHNL6ePq+dtgx8bK+1D2OPxuu/fO5/2n8uVLyN1VxHfLKrkKIVzkbZevnvLeL/2vdy6AX333nHcCppQXLtq7wzHVtX5B6StlJKa6e2ey6qrfCH55O/T8H8DBgT+Dk0pcepWzxAupn+YkrIt747m0v9Dy+59o+f3IPdOerfe3mG3vWBHf8aO2tolHs69g+uxPOOrDOzgeIKV4fmqO98vylO3vlVvWpj5HVv3S046d9QDvb83jzOWPku1a0qTbSTRZ+O8981vMf7XcejouKQ7FS1b8Fgx6vdZ7zzS3dRXzNodInTWZbkDjyY/ivvkLq/93Ph1albnla0Eea2eMp+3HV7N+0HO07tYPDvwZkZwfYfiQ4nZ9LmNB81MIN2lPhw8vY8eQF1nbsBv3vjObz7a95O3p976ouP20F7xzIxq2JmfLJlp8O8KbPjzGcfU138OPnxNudWjJv0pP0S52Th9DAyDy7ShCmfd5/27B+wVs5n2JGH8nDHgUGrYA4LYxM/l8/jq+f/BXhGLdYW/3414njoDjb/GOIQIcfq63TvC+GHx0D9y+wDvXY9lE71DGkWVGfYp2wTfPQL8boHAHLPoUel1YPH/jYmjaAdLSy/djt0WfQkEu9Di39PSNi72TBg8+BY65lqO/uwMa3g5HX1263fg7vTsKPlTFl6FEy98Gc9+mydZdED4BUqK/rnOWeCNX7XqXbv+3ntCoDVzzWfG0nCXeIarW3eL7zP/8L8x9O/a/vX3h6WO9y2Jv3g8jQKunw8YfodfQmi0/7UXvhNp7lkODZrXataooqKurott/tjgYrpsMrw+F3sOg5aHw7vXe0PMBLSEv+kvh9Cfhw9v3X3/jdFCogr3EvdTU8jhq7C9rbX1nLn8UgA62EUqEdE2Ne/M5hqR8XWqaFeUz/6nz2GHrOTSUDcCSUBdah3+irXl7y60//jV8DF/0+QenzLmz9EpnjKYbxYcbmr41iJ8ih9Cw8DKIfvlYO+p/aH1gw+KwnfxXAFqUXM/wpnD+i2wrDNG4UWOsxOV1JUM6vzBMesFmGHEwDaLTQpEC3Ke/paDLKWzb5Wj1/qXws1+w89BzaDDrDZZtN7oMugWe7kevorN5Pv1dGAH9AXp8DRmHw/CmZHc4nQ5nPgBAxFJKD8Ht2u59wQD49xXen2u+h1aHwstneu8PORWmPge/uNH75TZlJHzxMGzNJn/ZFNI3zmVNw8Nod3BPyN/iBW3HfnDKg/D9a3DczTB7DBx/m3ev/DEXw4IPvHUX7IBOx3n/94CiCY97v9CWfAFLvqAheP/XmneBg0/2zoN46XRv2UiJQzKF+V5/Bz0GHfpSzpPdvfMu7l1VXG9ZO7d4txFu1Lp4tMOMbfmFLF6fS5+fxfjF/lhH6HY6/PKOUs8J2JpXSMP6KYTG301o9hv0AUjPhoNO8ur4R/S8mbJhumWl91NSybarvoOx13qP+j3xLm+RvALS1kyl4auDCV87iZS5b0f/fiLwRGfoeR6c8dfS6yzaBbPGwJGXel/Kdlv4sXeDp8PPjl55YpBaz3u9aSm0Pqy4bbgI/u8073XOj942KPPlzDmH7f4iuDecg2kv4sbfhbkw9LygdL/jXc2kv2CAy9vExqID9r5f1WCuoiG0BOrbt6+bNq32vmFlZWWRmZlZa+uL267t3rHCnZu8s5G7nw3nveDtfTRqDS4MnfvDH1pUuaoqpdSHcPyXZUmwbOtwIk2y/1tr68s95xUa/aeS8wKAH496kCUphzBowGB4pFXVKx2+lZ3/OI4GOfPKzYq0PpzQ+vLTd9vU5Qyan3wbvHBq+Zn3raZw0WekvXNFxZ/9YA6M/TXMG7tnkjugJXbyA+S3OoL0/zsFl3E4uzr2pygSoeGcV7ELX2Fz21/SbESJ2pp2hK2rCLfoSsrZT3t7/03a4f7QCgsXsKJVJp02ZMGJ97D12Lvo9ftPOcSyee7cTnRpXp9vVuXR6OB+9OjQHB4+sFw3wxlHcOrKyzm2d28en1/+dsPb+g+nycThABTct56Nz59L06bNaHjICfDxPXv+PqjfiEhhAaFHvb7vbNSRBrklTtb83Ra2zv6QoWNWMrzeKxxrP/B+y2s4c+Pz3vyrP4MXvIcRDT94DMO7r4f3b/H+3rqfjf3wLvS7EX4+0LvnwxH/432xBLh7GfypC4UHZJB29yIirw8ltOgjuHMxWdPmkdmvD6z4Gt4oMZIC8NAm71BjdD2R5gcTan0YXPAK4YXjCbXpSaRJR/Imj6T+pCewq8aTtmU5dPdGeNZvz2fyjxs5t08HL5yd8wJ50afw+v/s+ZiCO5ZSr3GM37krv4XGbaBZp+J/I84RGX83KfUOoOjrp0l1BbzTeTiFi7MI/eJmLjg9xr/HGjKz6c65GN8UFdT7z46cPUON5WxY5A0vTnrS21voe7U3HHrwyXDRvwEHoVRY8RXfzVvKMadf5h3DzJ4GGd29s53b94VV33h7DE1/BltLf7N2aQ2xwr089pTRwzsmLSJx2eYa0MQqOWdhH5k+6D2O+nhI1Q1rySlFf+eL1PJXWUw87kX6f30VAF81OwvaHc3x8x6MuY73GpzDkJ3xn1ux208XfUmb109mTqQzfy66kJfrPbFn3rCCB3jqV41o9d/iw5Tb+g/HrV9Ao5Wfk5K3kV29r6Cg04k0fu9KAJylsOqK6TRq2Z6xTz/ANTtGxfzcl3/+DJdfdFHMeTWhoE6GoI5HUYF33XR6U29vPK2B93jMEqqsZctK7/i5c94wZMufe8PuzTrDrDe941zLJnnDU6OHwPG3wqZlMH8c3PK9d6KUi3gnDv34CbTp6R13b9IebpsL71zljQj0OB9w0KC5N7QJ3mdsXg6H/Arycrzjo8sm7ulaYbujSRvwsPc873iva+52RvFQp4hIkng64xFuuP7mqhvGSUHtl6COQ63Wsms7pDWMfbzGOVjxFXQ63jsZxazcl4a4bFjkHXvM6AH1io/rTHv/Bfp+f493j/Um7bxRgabtvZkbF8PXT8Fxt0DLQ7zjWZFCb1ShaBf88K53SdOPn3pfNOa8DT/N9pa9bjLMfstbHrzjUT3O84YFNy/3jqv1ux6+fAQWjvfaXDLWW/fm5bBhIXTp731ZGTOsuI5BT3hfeCZGT/Q65lrvUqhYl69VwoVSsUhR3O13hhqxNaMfbdbGd2OTrY0Oomnu0mr1qaTX7XQuch/WeHmRumJhgyM59J6sWlvfXge1mQ0C/o53DsvzzrnHy8y36PzBQB5whXNuRjzLxqKgrlhQavFFHc4Vn9W82+bl0LgtpBafkp6VlUVm//5eWzPvi8WmaFg261Sq7R67cr229Rp671dP976gHHam195CsGubN7oSq1/hAq/d9nXeF6jCnd7JVa2izz7f+CMF435DvUMHeF8qCnK9LyGp6d7Ix7bV3s/PB3ntd26Geg3JmjyFzBOO8w6jHNDC+5w2R3ijMwW5RNr2IbRiEmDeiVW7T84a9Lj3pe7fV8DJD3if8+Wj0LYXzHqdcK+LKDygLZGV33DAjmxvROXIS6DXMO+Jcm17wcQ/Q+MM74Szc5/Dff8KhFIIb1yGy7yPbf8dSYvVX5B/5FXUXzkJO/gk+NXvcR/fR2FKA9KatMF1HYjVa4D9/Qg2H9iTZh0OJbfrEFK3LKb+ri0YEfj6H+R3+RVpJ9xCyhcPebX97DjcgR2wvteQP+52Qp1+Qdra6diaGV59GT3IswakNM6gXngHX7a5mlO+vgSAotSGFA0aQXreWtzU5wl3O5PUqaPIO+QMDlj8AeE2vdl8+KW0/OIOwm16sW3QP0if8RwbWv2ClJ2b2Z7WgozITzSbNByAwsFPkja+9Amna5sfQ9vrxxF54yIizpHbshcNDxsAP+tHyiOtCBHe07YgpSH1wt5hrl3H30XK+jmkZhzGrrU/sGvjCmY0P40Tdk0i1DiD0MIPiaQ2IFTkDc//0HEY6RldabziU1pt+IbpPX7LUXMf2bPumUM+o/d7vyr/bxKY3u0ODmINhcu+pvkBqaRuXhKzXVmzO17CEavKX7FRUs5JIyhY+Clt13xWabuilAakhksfashPacSCbjfTe95j5dqvbt6P9pu+iaufZU064XV+eerpNVo2lsqC2rt3dCU/eAG7BDgIqAfMArqXaTMY+AgwoB/wbbzLxvo56qijXG2aMGFCra4vkYJSS1DqcE61JKug1OLbOsJFpd9HIhXXUlTo3I6c0u93btlnXauRSMT7KSpwLhyu9e0CTHMVZGI856gfAyx2zi11zhUAY4CyZykMAUZHP+8b4EAzaxvnsiIiEjShMlf6V3apVUqqd/ldyfexRpMSafeIWUpajS7v2hvxfFp7oORNmLOj0+JpE8+yIiIiUoF4bngS62tQ2QPbFbWJZ1lvBWbXAtcCZGRkkJWVFUfX4pObm1ur60ukoNQSlDpAtSSroNQSlDpAtdRUPEGdDZS8HVcHYE2cberFsSwAzrlRwCjwTiarzRONfHHiUpyCUktQ6gDVkqyCUktQ6gDVUlPxDH1PBbqaWRczqwcMBcaVaTMOuMw8/YCtzrm1cS4rIiIiFahyj9o5V2RmNwGf4J3F/aJzbp6ZXRed/ywwHu/M78V4l2ddWdmy+6QSERGRAIrroRzOufF4YVxy2rMlXjvgxniXFRERkfjoedQiIiJJTEEtIiKSxBTUIiIiSUxBLSIiksSS8ulZZrYBWFGLq2wJbKzF9SVSUGoJSh2gWpJVUGoJSh2gWirTyTnXKtaMpAzq2mZm01xFTyXxmaDUEpQ6QLUkq6DUEpQ6QLXUlIa+RUREkpiCWkREJInVlaAelegO1KKg1BKUOkC1JKug1BKUOkC11EidOEYtIiLiV3Vlj1pERMSXAh3UZjbIzBaa2WIzuzfR/YmHmS03szlmNtPMpkWnNTezz8zsx+ifzUq0vy9a30IzG5i4noOZvWhm681sbolp1e67mR0V/TtYbGZPmVms55rv7zqGm9nq6HaZaWaDk72OaB86mtkEM5tvZvPM7NbodD9ul4pq8dW2MbN0M/vOzGZF63g4Ot2P26SiWny1TUr0IcXMvjezD6Lvk2ObOOcC+YP3tK4lwEF4z8WeBXRPdL/i6PdyoGWZaX8C7o2+vhd4Ivq6e7Su+kCXaL0pCex7f6APMHdv+g58B/wCMOAj4LQkqGM4cGeMtklbR7QPbYE+0deNgUXRPvtxu1RUi6+2TfQzG0VfpwHfAv18uk0qqsVX26RE/24HXgc+iL5Pim0S5D3qY4DFzrmlzrkCYAwwJMF9qqkhwMvR1y8DZ5eYPsY5t8s5twzvMaPH7P/ueZxzE4FNZSZXq+9m1hZo4pyb4rx/9aNLLLNfVFBHRZK2DgDn3Frn3Izo6+3AfKA9/twuFdVSkaSsxXlyo2/Toj8Of26TimqpSNLWYmYdgNOB58v0N+HbJMhB3R5YVeJ9NpX/p04WDvjUzKab2bXRaRnOubXg/bICWken+6HG6va9ffR12enJ4CYzm23e0PjuITDf1GFmnYEj8fZ6fL1dytQCPts20SHWmcB64DPnnG+3SQW1gM+2CfA34G4gUmJaUmyTIAd1rOMCfjjF/XjnXB/gNOBGM+tfSVu/1ggV9z1Za3oGOBjoDawF/hKd7os6zKwR8A5wm3NuW2VNY0xLqnpi1OK7beOcCzvnegMd8PbEelTSPGnrgApr8dU2MbMzgPXOuenxLhJj2j6rI8hBnQ10LPG+A7AmQX2Jm3NuTfTP9cB/8Iay10WHVIj+uT7a3A81Vrfv2dHXZacnlHNuXfQXUgR4juJDDElfh5ml4QXba865sdHJvtwusWrx87Zxzm0BsoBB+HSb7FayFh9uk+OBs8xsOd5h0pPN7FWSZJsEOainAl3NrIuZ1QOGAuMS3KdKmVlDM2u8+zUwAJiL1+/Lo80uB96Lvh4HDDWz+mbWBeiKdyJDMqlW36PDS9vNrF/0bMnLSiyTMLv/s0adg7ddIMnriH72C8B859yTJWb5brtUVIvfto2ZtTKzA6OvGwCnAgvw5zaJWYvftolz7j7nXAfnXGe8rPjSOXcJybJN9vZstGT+AQbjnRm6BHgg0f2Jo78H4Z1JOAuYt7vPQAvgC+DH6J/NSyzzQLS+hSTgLMky/X8Db5irEO+b5dU16TvQF+8/9hLgn0RvzJPgOl4B5gCzo/9J2yZ7HdE+nIA39DYbmBn9GezT7VJRLb7aNsARwPfR/s4FHopO9+M2qagWX22TMjVlUnzWd1JsE92ZTEREJIkFeehbRETE9xTUIiIiSUxBLSIiksQU1CIiIklMQS0iIpLEFNQiIiJJTEEtIiKSxBTUIiIiSez/ASZcCqmNGS33AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history).plot(figsize=(8, 5))\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3c9bcd1",
   "metadata": {},
   "source": [
    "## Notes:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "135d29b1",
   "metadata": {},
   "source": [
    "Actually, there is a function called `KerasRegressor()` which I didn't use here but it can be used for implementing neural networks for a regression problem. You can find very good examples of that function by clicking the [link](https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7decfd3",
   "metadata": {},
   "source": [
    "I found a discussion about setting `clipnorm=00.1` while using Adam optimization and tried it. This didn't yield a much better performance but actually improved the covid model a bit. I also add the link about this discussion here [link](https://github.com/fizyr/keras-retinanet/issues/942)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}