t81_558_class_09_4_transfer_nlp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 9: Regularization: L1, L2 and Dropout**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 9 Material\n",
    "\n",
    "* Part 9.1: Introduction to Keras Transfer Learning [[Video]](https://www.youtube.com/watch?v=WLlP6S-Z8Xs&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_1_keras_transfer.ipynb)\n",
    "* Part 9.2: Popular Pretrained Neural Networks for Keras [[Video]](https://www.youtube.com/watch?v=ctVA1_46YEE&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_2_popular_transfer.ipynb)\n",
    "* Part 9.3: Transfer Learning for Computer Vision and Keras [[Video]](https://www.youtube.com/watch?v=61vMUm_XBMI&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_3_transfer_cv.ipynb)\n",
    "* **Part 9.4: Transfer Learning for Languages and Keras** [[Video]](https://www.youtube.com/watch?v=ajmAAg9FxXA&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_4_transfer_nlp.ipynb)\n",
    "* Part 9.5: Transfer Learning for Keras Feature Engineering [[Video]](https://www.youtube.com/watch?v=Dttxsm8zpL8&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_09_5_transfer_feature_eng.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 9.4: Transfer Learning for Languages and Keras\n",
    "\n",
    "Transfer learning is commonly used with Natural Language Processing (NLP).  This course has an entire module that covers NLP.  However, for now we will look how a NLP network can be loaded into Keras for transfer learning.  The following three sources were helpful for the creation of this section.\n",
    "\n",
    "* Cer, D., Yang, Y., Kong, S. Y., Hua, N., Limtiaco, N., John, R. S., ... & Sung, Y. H. (2018). [Universal sentence encoder](https://arxiv.org/abs/1803.11175). arXiv preprint arXiv:1803.11175.\n",
    "* [Deep Transfer Learning for Natural Language Processing — Text Classification with Universal Embeddings](https://towardsdatascience.com/deep-transfer-learning-for-natural-language-processing-text-classification-with-universal-1a2c69e5baa9)\n",
    "* [Keras Tutorial - How to Use Google's Universal Sentence Encoder for Spam Classification](http://hunterheidenreich.com/blog/google-universal-sentence-encoder-in-keras/)\n",
    "\n",
    "These examples make use of TensorFlow Hub, which allows pretrained models to easily be loaded into TensorFlow.  To install TensorHub use the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow_hub in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (0.5.0)\n",
      "Requirement already satisfied: numpy>=1.12.0 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_hub) (1.16.4)\n",
      "Requirement already satisfied: six>=1.10.0 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_hub) (1.12.0)\n",
      "Requirement already satisfied: protobuf>=3.4.0 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_hub) (3.9.1)\n",
      "Requirement already satisfied: setuptools in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from protobuf>=3.4.0->tensorflow_hub) (41.0.1)\n"
     ]
    }
   ],
   "source": [
    "!pip install tensorflow_hub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also necessary to install TensorFlow Datasets.  This can be done with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow_datasets in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (1.1.0)\n",
      "Requirement already satisfied: tensorflow-metadata in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (0.14.0)\n",
      "Requirement already satisfied: requests>=2.19.0 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (2.22.0)\n",
      "Requirement already satisfied: promise in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (2.2.1)\n",
      "Requirement already satisfied: dill in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (0.3.0)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (3.9.1)\n",
      "Requirement already satisfied: tqdm in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (4.33.0)\n",
      "Requirement already satisfied: six in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (1.12.0)\n",
      "Requirement already satisfied: wrapt in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (1.11.2)\n",
      "Requirement already satisfied: numpy in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (1.16.4)\n",
      "Requirement already satisfied: psutil in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (5.6.3)\n",
      "Requirement already satisfied: termcolor in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (1.1.0)\n",
      "Requirement already satisfied: future in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (0.17.1)\n",
      "Requirement already satisfied: attrs in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (19.1.0)\n",
      "Requirement already satisfied: absl-py in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow_datasets) (0.7.1)\n",
      "Requirement already satisfied: googleapis-common-protos in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from tensorflow-metadata->tensorflow_datasets) (1.6.0)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.19.0->tensorflow_datasets) (2.8)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.19.0->tensorflow_datasets) (2019.6.16)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.19.0->tensorflow_datasets) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.19.0->tensorflow_datasets) (1.25.3)\n",
      "Requirement already satisfied: setuptools in c:\\users\\jheaton\\appdata\\local\\continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages (from protobuf>=3.6.1->tensorflow_datasets) (41.0.1)\n"
     ]
    }
   ],
   "source": [
    "!pip install tensorflow_datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb#scrollTo=2ew7HTbPpCJH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the Internet Movie DataBase (IMDB) reviews data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W0819 16:45:54.504038 19696 dataset_builder.py:439] Warning: Setting shuffle_files=True because split=TRAIN and shuffle_files=None. This behavior will be deprecated on 2019-08-06, at which point shuffle_files=False will be the default for all splits.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "import tensorflow_datasets as tfds\n",
    "\n",
    "train_data, test_data = tfds.load(name=\"imdb_reviews\", split=[\"train\", \"test\"], \n",
    "                                  batch_size=-1, as_supervised=True)\n",
    "\n",
    "train_examples, train_labels = tfds.as_numpy(train_data)\n",
    "test_examples, test_labels = tfds.as_numpy(test_data)\n",
    "\n",
    "# /Users/jheaton/tensorflow_datasets/imdb_reviews/plain_text/0.1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load a pretrained embedding model called [gnews-swivel-20dim](https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1).  This was trained by Google on gnews data and can convert RAW text into vectors.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = \"https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1\"\n",
    "hub_layer = hub.KerasLayer(model, output_shape=[20], input_shape=[], \n",
    "                           dtype=tf.string, trainable=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the following 3 movie reviews."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([b\"Oh yeah! Jenna Jameson did it again! Yeah Baby! This movie rocks. It was one of the 1st movies i saw of her. And i have to say i feel in love with her, she was great in this move.<br /><br />Her performance was outstanding and what i liked the most was the scenery and the wardrobe it was amazing you can tell that they put a lot into the movie the girls cloth were amazing.<br /><br />I hope this comment helps and u can buy the movie, the storyline is awesome is very unique and i'm sure u are going to like it. Jenna amazed us once more and no wonder the movie won so many awards. Her make-up and wardrobe is very very sexy and the girls on girls scene is amazing. specially the one where she looks like an angel. It's a must see and i hope u share my interests\",\n",
       "       b\"This is the most depressing film I have ever seen. I first saw it as a child and even thinking about it now really upsets me. I know it was set in a time when life was hard and I know these people were poor and the crops were vital. Yes, I get all that. What I find hard to take is I can't remember one single light moment in the entire film. Maybe it was true to life, I don't know. I'm quite sure the acting was top notch and the direction and quality of filming etc etc was wonderful and I know that every film can't have a happy ending but as a family film it is dire in my opinion.<br /><br />I wouldn't recommend it to anyone who wants to be entertained by a film. I can't stress enough how this film affected me as a child. I was talking about it recently and all the sad memories came flooding back. I think it would have all but the heartless reaching for the Prozac.\",\n",
       "       b\"As a lifelong fan of Dickens, I have invariably been disappointed by adaptations of his novels.<br /><br />Although his works presented an extremely accurate re-telling of human life at every level in Victorian Britain, throughout them all was a pervasive thread of humour that could be both playful or sarcastic as the narrative dictated. In a way, he was a literary caricaturist and cartoonist. He could be serious and hilarious in the same sentence. He pricked pride, lampooned arrogance, celebrated modesty, and empathised with loneliness and poverty. It may be a clich\\xc3\\xa9, but he was a people's writer.<br /><br />And it is the comedy that is so often missing from his interpretations. At the time of writing, Oliver Twist is being dramatised in serial form on BBC television. All of the misery and cruelty is their, but non of the humour, irony, and savage lampoonery. The result is just a dark, dismal experience: the story penned by a journalist rather than a novelist. It's not really Dickens at all.<br /><br />'Oliver!', on the other hand, is much closer to the mark. The mockery of officialdom is perfectly interpreted, from the blustering beadle to the drunken magistrate. The classic stand-off between the beadle and Mr Brownlow, in which the law is described as 'a ass, a idiot' couldn't have been better done. Harry Secombe is an ideal choice.<br /><br />But the blinding cruelty is also there, the callous indifference of the state, the cold, hunger, poverty and loneliness are all presented just as surely as The Master would have wished.<br /><br />And then there is crime. Ron Moody is a treasure as the sleazy Jewish fence, whilst Oliver Reid has Bill Sykes to perfection.<br /><br />Perhaps not surprisingly, Lionel Bart - himself a Jew from London's east-end - takes a liberty with Fagin by re-interpreting him as a much more benign fellow than was Dicken's original. In the novel, he was utterly ruthless, sending some of his own boys to the gallows in order to protect himself (though he was also caught and hanged). Whereas in the movie, he is presented as something of a wayward father-figure, a sort of charitable thief rather than a corrupter of children, the latter being a long-standing anti-semitic sentiment. Otherwise, very few liberties are taken with Dickens's original. All of the most memorable elements are included. Just enough menace and violence is retained to ensure narrative fidelity whilst at the same time allowing for children' sensibilities. Nancy is still beaten to death, Bullseye narrowly escapes drowning, and Bill Sykes gets a faithfully graphic come-uppance.<br /><br />Every song is excellent, though they do incline towards schmaltz. Mark Lester mimes his wonderfully. Both his and my favourite scene is the one in which the world comes alive to 'who will buy'. It's schmaltzy, but it's Dickens through and through.<br /><br />I could go on. I could commend the wonderful set-pieces, the contrast of the rich and poor. There is top-quality acting from more British regulars than you could shake a stick at.<br /><br />I ought to give it 10 points, but I'm feeling more like Scrooge today. Soak it up with your Christmas dinner. No original has been better realised.\"],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_examples[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The embedding layer can convert each to 20-number vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: id=311, shape=(3, 20), dtype=float32, numpy=\n",
       "array([[ 3.4232912 , -4.230874  ,  4.1488533 , -0.29553518, -6.802391  ,\n",
       "        -2.5163853 , -4.4002395 ,  1.905792  ,  4.7512794 , -0.40538004,\n",
       "        -4.3401685 ,  1.0361497 ,  0.9744097 ,  0.71507156, -6.2657013 ,\n",
       "         0.16533905,  4.560262  , -1.3106939 , -3.1121316 , -2.1338716 ],\n",
       "       [ 3.041689  , -4.7446456 ,  4.733248  ,  0.30785134, -6.9846196 ,\n",
       "        -5.9006767 , -4.4500394 ,  2.3718796 ,  5.0403566 ,  0.4119934 ,\n",
       "        -2.8661985 ,  0.6986558 ,  0.5313177 ,  1.4525845 , -7.200312  ,\n",
       "         2.9011002 ,  6.392641  , -2.6039283 , -2.9440727 , -2.4246173 ],\n",
       "       [ 3.9819887 , -4.4838037 ,  5.177359  , -2.3643482 , -3.2938678 ,\n",
       "        -3.5364532 , -2.4786978 ,  2.5525482 ,  6.688532  , -2.3076782 ,\n",
       "        -1.9807833 ,  1.1315885 , -3.0339816 , -0.7604128 , -5.743445  ,\n",
       "         3.4242578 ,  4.790099  , -4.03061   , -5.992149  , -1.7297493 ]],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hub_layer(train_examples[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We add addition layers to attempt to classify the movie reviews as either positive or negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "keras_layer (KerasLayer)     (None, 20)                400020    \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 16)                336       \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 17        \n",
      "=================================================================\n",
      "Total params: 400,373\n",
      "Trainable params: 400,373\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.Sequential()\n",
    "model.add(hub_layer)\n",
    "model.add(tf.keras.layers.Dense(16, activation='relu'))\n",
    "model.add(tf.keras.layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compile the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split and train the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_val = train_examples[:10000]\n",
    "partial_x_train = train_examples[10000:]\n",
    "\n",
    "y_val = train_labels[:10000]\n",
    "partial_y_train = train_labels[10000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0819 16:45:57.484114 19696 training_utils.py:1210] When passing input data as arrays, do not specify `steps_per_epoch`/`steps` argument. Please use `batch_size` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0819 16:45:57.599118 19696 deprecation.py:323] From C:\\Users\\jheaton\\AppData\\Local\\Continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages\\tensorflow_core\\python\\ops\\nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "W0819 16:45:57.879124 19696 deprecation.py:323] From C:\\Users\\jheaton\\AppData\\Local\\Continuum\\miniconda3\\envs\\tensorflow\\lib\\site-packages\\tensorflow_core\\python\\keras\\optimizer_v2\\optimizer_v2.py:455: BaseResourceVariable.constraint (from tensorflow.python.ops.resource_variable_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Apply a constraint manually following the optimizer update step.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15000/15000 [==============================] - ETA: 29s - loss: 1.3981 - accuracy: 0.503 - ETA: 15s - loss: 1.3916 - accuracy: 0.504 - ETA: 10s - loss: 1.3421 - accuracy: 0.504 - ETA: 8s - loss: 1.2883 - accuracy: 0.504 - ETA: 6s - loss: 1.2361 - accuracy: 0.50 - ETA: 5s - loss: 1.1960 - accuracy: 0.51 - ETA: 4s - loss: 1.1522 - accuracy: 0.51 - ETA: 4s - loss: 1.1093 - accuracy: 0.52 - ETA: 3s - loss: 1.0793 - accuracy: 0.52 - ETA: 3s - loss: 1.0528 - accuracy: 0.52 - ETA: 3s - loss: 1.0294 - accuracy: 0.52 - ETA: 2s - loss: 1.0098 - accuracy: 0.52 - ETA: 2s - loss: 0.9874 - accuracy: 0.53 - ETA: 2s - loss: 0.9731 - accuracy: 0.53 - ETA: 2s - loss: 0.9601 - accuracy: 0.53 - ETA: 1s - loss: 0.9487 - accuracy: 0.53 - ETA: 1s - loss: 0.9386 - accuracy: 0.53 - ETA: 1s - loss: 0.9308 - accuracy: 0.53 - ETA: 1s - loss: 0.9260 - accuracy: 0.53 - ETA: 1s - loss: 0.9189 - accuracy: 0.53 - ETA: 1s - loss: 0.9121 - accuracy: 0.53 - ETA: 0s - loss: 0.9048 - accuracy: 0.53 - ETA: 0s - loss: 0.8970 - accuracy: 0.53 - ETA: 0s - loss: 0.8912 - accuracy: 0.53 - ETA: 0s - loss: 0.8852 - accuracy: 0.53 - ETA: 0s - loss: 0.8783 - accuracy: 0.53 - ETA: 0s - loss: 0.8711 - accuracy: 0.54 - ETA: 0s - loss: 0.8659 - accuracy: 0.54 - ETA: 0s - loss: 0.8612 - accuracy: 0.54 - 5s 316us/sample - loss: 0.8595 - accuracy: 0.5457 - val_loss: 0.7026 - val_accuracy: 0.5903\n",
      "Epoch 2/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.6935 - accuracy: 0.59 - ETA: 2s - loss: 0.6805 - accuracy: 0.61 - ETA: 2s - loss: 0.6851 - accuracy: 0.61 - ETA: 2s - loss: 0.6858 - accuracy: 0.61 - ETA: 2s - loss: 0.6898 - accuracy: 0.61 - ETA: 2s - loss: 0.6858 - accuracy: 0.61 - ETA: 1s - loss: 0.6812 - accuracy: 0.61 - ETA: 1s - loss: 0.6793 - accuracy: 0.61 - ETA: 1s - loss: 0.6812 - accuracy: 0.61 - ETA: 1s - loss: 0.6800 - accuracy: 0.61 - ETA: 1s - loss: 0.6768 - accuracy: 0.61 - ETA: 1s - loss: 0.6796 - accuracy: 0.61 - ETA: 1s - loss: 0.6771 - accuracy: 0.61 - ETA: 1s - loss: 0.6781 - accuracy: 0.61 - ETA: 1s - loss: 0.6754 - accuracy: 0.61 - ETA: 1s - loss: 0.6730 - accuracy: 0.62 - ETA: 0s - loss: 0.6720 - accuracy: 0.62 - ETA: 0s - loss: 0.6712 - accuracy: 0.62 - ETA: 0s - loss: 0.6694 - accuracy: 0.62 - ETA: 0s - loss: 0.6684 - accuracy: 0.62 - ETA: 0s - loss: 0.6683 - accuracy: 0.62 - ETA: 0s - loss: 0.6674 - accuracy: 0.62 - ETA: 0s - loss: 0.6679 - accuracy: 0.62 - ETA: 0s - loss: 0.6668 - accuracy: 0.62 - ETA: 0s - loss: 0.6657 - accuracy: 0.62 - ETA: 0s - loss: 0.6643 - accuracy: 0.62 - ETA: 0s - loss: 0.6628 - accuracy: 0.62 - ETA: 0s - loss: 0.6612 - accuracy: 0.62 - ETA: 0s - loss: 0.6598 - accuracy: 0.62 - 3s 231us/sample - loss: 0.6596 - accuracy: 0.6292 - val_loss: 0.6306 - val_accuracy: 0.6541\n",
      "Epoch 3/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.6417 - accuracy: 0.62 - ETA: 2s - loss: 0.6378 - accuracy: 0.65 - ETA: 2s - loss: 0.6269 - accuracy: 0.64 - ETA: 2s - loss: 0.6293 - accuracy: 0.65 - ETA: 2s - loss: 0.6213 - accuracy: 0.66 - ETA: 2s - loss: 0.6235 - accuracy: 0.66 - ETA: 1s - loss: 0.6206 - accuracy: 0.66 - ETA: 1s - loss: 0.6163 - accuracy: 0.66 - ETA: 1s - loss: 0.6162 - accuracy: 0.66 - ETA: 1s - loss: 0.6172 - accuracy: 0.66 - ETA: 1s - loss: 0.6112 - accuracy: 0.67 - ETA: 1s - loss: 0.6115 - accuracy: 0.67 - ETA: 1s - loss: 0.6083 - accuracy: 0.67 - ETA: 1s - loss: 0.6058 - accuracy: 0.67 - ETA: 1s - loss: 0.6075 - accuracy: 0.67 - ETA: 1s - loss: 0.6065 - accuracy: 0.67 - ETA: 0s - loss: 0.6060 - accuracy: 0.67 - ETA: 0s - loss: 0.6048 - accuracy: 0.67 - ETA: 0s - loss: 0.6042 - accuracy: 0.67 - ETA: 0s - loss: 0.6039 - accuracy: 0.67 - ETA: 0s - loss: 0.6035 - accuracy: 0.67 - ETA: 0s - loss: 0.6016 - accuracy: 0.68 - ETA: 0s - loss: 0.5997 - accuracy: 0.68 - ETA: 0s - loss: 0.5993 - accuracy: 0.68 - ETA: 0s - loss: 0.5989 - accuracy: 0.68 - ETA: 0s - loss: 0.5971 - accuracy: 0.68 - ETA: 0s - loss: 0.5968 - accuracy: 0.68 - ETA: 0s - loss: 0.5961 - accuracy: 0.68 - ETA: 0s - loss: 0.5946 - accuracy: 0.68 - 4s 234us/sample - loss: 0.5950 - accuracy: 0.6857 - val_loss: 0.5819 - val_accuracy: 0.6952\n",
      "Epoch 4/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.5840 - accuracy: 0.71 - ETA: 2s - loss: 0.5800 - accuracy: 0.71 - ETA: 2s - loss: 0.5802 - accuracy: 0.70 - ETA: 2s - loss: 0.5744 - accuracy: 0.70 - ETA: 2s - loss: 0.5768 - accuracy: 0.70 - ETA: 1s - loss: 0.5667 - accuracy: 0.71 - ETA: 1s - loss: 0.5670 - accuracy: 0.71 - ETA: 1s - loss: 0.5615 - accuracy: 0.71 - ETA: 1s - loss: 0.5604 - accuracy: 0.71 - ETA: 1s - loss: 0.5589 - accuracy: 0.71 - ETA: 1s - loss: 0.5596 - accuracy: 0.71 - ETA: 1s - loss: 0.5602 - accuracy: 0.71 - ETA: 1s - loss: 0.5599 - accuracy: 0.71 - ETA: 1s - loss: 0.5587 - accuracy: 0.71 - ETA: 1s - loss: 0.5590 - accuracy: 0.71 - ETA: 1s - loss: 0.5612 - accuracy: 0.71 - ETA: 0s - loss: 0.5619 - accuracy: 0.71 - ETA: 0s - loss: 0.5593 - accuracy: 0.71 - ETA: 0s - loss: 0.5577 - accuracy: 0.71 - ETA: 0s - loss: 0.5562 - accuracy: 0.71 - ETA: 0s - loss: 0.5563 - accuracy: 0.71 - ETA: 0s - loss: 0.5565 - accuracy: 0.71 - ETA: 0s - loss: 0.5548 - accuracy: 0.71 - ETA: 0s - loss: 0.5544 - accuracy: 0.71 - ETA: 0s - loss: 0.5540 - accuracy: 0.71 - ETA: 0s - loss: 0.5531 - accuracy: 0.71 - ETA: 0s - loss: 0.5515 - accuracy: 0.71 - ETA: 0s - loss: 0.5512 - accuracy: 0.71 - ETA: 0s - loss: 0.5495 - accuracy: 0.72 - 4s 237us/sample - loss: 0.5494 - accuracy: 0.7213 - val_loss: 0.5460 - val_accuracy: 0.7271\n",
      "Epoch 5/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.5312 - accuracy: 0.74 - ETA: 2s - loss: 0.5356 - accuracy: 0.74 - ETA: 2s - loss: 0.5325 - accuracy: 0.74 - ETA: 2s - loss: 0.5392 - accuracy: 0.73 - ETA: 1s - loss: 0.5412 - accuracy: 0.73 - ETA: 1s - loss: 0.5399 - accuracy: 0.73 - ETA: 1s - loss: 0.5377 - accuracy: 0.73 - ETA: 1s - loss: 0.5344 - accuracy: 0.74 - ETA: 1s - loss: 0.5320 - accuracy: 0.74 - ETA: 1s - loss: 0.5281 - accuracy: 0.74 - ETA: 1s - loss: 0.5264 - accuracy: 0.74 - ETA: 1s - loss: 0.5266 - accuracy: 0.74 - ETA: 1s - loss: 0.5261 - accuracy: 0.74 - ETA: 1s - loss: 0.5248 - accuracy: 0.74 - ETA: 1s - loss: 0.5241 - accuracy: 0.74 - ETA: 1s - loss: 0.5202 - accuracy: 0.74 - ETA: 0s - loss: 0.5203 - accuracy: 0.74 - ETA: 0s - loss: 0.5201 - accuracy: 0.74 - ETA: 0s - loss: 0.5206 - accuracy: 0.74 - ETA: 0s - loss: 0.5210 - accuracy: 0.74 - ETA: 0s - loss: 0.5194 - accuracy: 0.74 - ETA: 0s - loss: 0.5182 - accuracy: 0.74 - ETA: 0s - loss: 0.5178 - accuracy: 0.74 - ETA: 0s - loss: 0.5172 - accuracy: 0.74 - ETA: 0s - loss: 0.5163 - accuracy: 0.75 - ETA: 0s - loss: 0.5152 - accuracy: 0.75 - ETA: 0s - loss: 0.5149 - accuracy: 0.75 - ETA: 0s - loss: 0.5126 - accuracy: 0.75 - ETA: 0s - loss: 0.5129 - accuracy: 0.75 - 3s 222us/sample - loss: 0.5126 - accuracy: 0.7521 - val_loss: 0.5164 - val_accuracy: 0.7513\n",
      "Epoch 6/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.5271 - accuracy: 0.71 - ETA: 2s - loss: 0.5020 - accuracy: 0.74 - ETA: 2s - loss: 0.5156 - accuracy: 0.73 - ETA: 2s - loss: 0.5089 - accuracy: 0.75 - ETA: 1s - loss: 0.5047 - accuracy: 0.75 - ETA: 1s - loss: 0.5014 - accuracy: 0.75 - ETA: 1s - loss: 0.4985 - accuracy: 0.75 - ETA: 1s - loss: 0.4977 - accuracy: 0.75 - ETA: 1s - loss: 0.4991 - accuracy: 0.75 - ETA: 1s - loss: 0.4974 - accuracy: 0.75 - ETA: 1s - loss: 0.4962 - accuracy: 0.75 - ETA: 1s - loss: 0.4987 - accuracy: 0.75 - ETA: 1s - loss: 0.4967 - accuracy: 0.75 - ETA: 1s - loss: 0.4940 - accuracy: 0.76 - ETA: 1s - loss: 0.4949 - accuracy: 0.76 - ETA: 1s - loss: 0.4946 - accuracy: 0.76 - ETA: 0s - loss: 0.4928 - accuracy: 0.76 - ETA: 0s - loss: 0.4931 - accuracy: 0.76 - ETA: 0s - loss: 0.4916 - accuracy: 0.76 - ETA: 0s - loss: 0.4880 - accuracy: 0.76 - ETA: 0s - loss: 0.4864 - accuracy: 0.76 - ETA: 0s - loss: 0.4846 - accuracy: 0.76 - ETA: 0s - loss: 0.4847 - accuracy: 0.76 - ETA: 0s - loss: 0.4847 - accuracy: 0.77 - ETA: 0s - loss: 0.4840 - accuracy: 0.77 - ETA: 0s - loss: 0.4839 - accuracy: 0.77 - ETA: 0s - loss: 0.4820 - accuracy: 0.77 - ETA: 0s - loss: 0.4812 - accuracy: 0.77 - ETA: 0s - loss: 0.4800 - accuracy: 0.77 - 3s 225us/sample - loss: 0.4799 - accuracy: 0.7745 - val_loss: 0.4902 - val_accuracy: 0.7719\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 7/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.4613 - accuracy: 0.78 - ETA: 2s - loss: 0.4542 - accuracy: 0.79 - ETA: 2s - loss: 0.4608 - accuracy: 0.78 - ETA: 2s - loss: 0.4526 - accuracy: 0.78 - ETA: 1s - loss: 0.4525 - accuracy: 0.78 - ETA: 1s - loss: 0.4513 - accuracy: 0.79 - ETA: 1s - loss: 0.4539 - accuracy: 0.79 - ETA: 1s - loss: 0.4534 - accuracy: 0.79 - ETA: 1s - loss: 0.4551 - accuracy: 0.79 - ETA: 1s - loss: 0.4539 - accuracy: 0.79 - ETA: 1s - loss: 0.4541 - accuracy: 0.79 - ETA: 1s - loss: 0.4541 - accuracy: 0.79 - ETA: 1s - loss: 0.4563 - accuracy: 0.79 - ETA: 1s - loss: 0.4551 - accuracy: 0.79 - ETA: 1s - loss: 0.4541 - accuracy: 0.79 - ETA: 1s - loss: 0.4538 - accuracy: 0.79 - ETA: 0s - loss: 0.4533 - accuracy: 0.79 - ETA: 0s - loss: 0.4529 - accuracy: 0.79 - ETA: 0s - loss: 0.4519 - accuracy: 0.79 - ETA: 0s - loss: 0.4527 - accuracy: 0.79 - ETA: 0s - loss: 0.4516 - accuracy: 0.79 - ETA: 0s - loss: 0.4531 - accuracy: 0.79 - ETA: 0s - loss: 0.4528 - accuracy: 0.79 - ETA: 0s - loss: 0.4538 - accuracy: 0.79 - ETA: 0s - loss: 0.4530 - accuracy: 0.79 - ETA: 0s - loss: 0.4523 - accuracy: 0.79 - ETA: 0s - loss: 0.4519 - accuracy: 0.79 - ETA: 0s - loss: 0.4499 - accuracy: 0.79 - ETA: 0s - loss: 0.4493 - accuracy: 0.79 - 4s 243us/sample - loss: 0.4495 - accuracy: 0.7958 - val_loss: 0.4666 - val_accuracy: 0.7848\n",
      "Epoch 8/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.4003 - accuracy: 0.82 - ETA: 2s - loss: 0.4286 - accuracy: 0.80 - ETA: 2s - loss: 0.4188 - accuracy: 0.81 - ETA: 2s - loss: 0.4201 - accuracy: 0.81 - ETA: 2s - loss: 0.4237 - accuracy: 0.81 - ETA: 2s - loss: 0.4270 - accuracy: 0.80 - ETA: 1s - loss: 0.4340 - accuracy: 0.80 - ETA: 1s - loss: 0.4357 - accuracy: 0.80 - ETA: 1s - loss: 0.4370 - accuracy: 0.80 - ETA: 1s - loss: 0.4359 - accuracy: 0.80 - ETA: 1s - loss: 0.4317 - accuracy: 0.81 - ETA: 1s - loss: 0.4317 - accuracy: 0.80 - ETA: 1s - loss: 0.4307 - accuracy: 0.81 - ETA: 1s - loss: 0.4289 - accuracy: 0.81 - ETA: 1s - loss: 0.4296 - accuracy: 0.81 - ETA: 1s - loss: 0.4284 - accuracy: 0.81 - ETA: 1s - loss: 0.4284 - accuracy: 0.81 - ETA: 0s - loss: 0.4272 - accuracy: 0.81 - ETA: 0s - loss: 0.4260 - accuracy: 0.81 - ETA: 0s - loss: 0.4250 - accuracy: 0.81 - ETA: 0s - loss: 0.4265 - accuracy: 0.81 - ETA: 0s - loss: 0.4255 - accuracy: 0.81 - ETA: 0s - loss: 0.4254 - accuracy: 0.81 - ETA: 0s - loss: 0.4244 - accuracy: 0.81 - ETA: 0s - loss: 0.4242 - accuracy: 0.81 - ETA: 0s - loss: 0.4231 - accuracy: 0.81 - ETA: 0s - loss: 0.4225 - accuracy: 0.81 - ETA: 0s - loss: 0.4218 - accuracy: 0.81 - ETA: 0s - loss: 0.4220 - accuracy: 0.81 - 4s 247us/sample - loss: 0.4217 - accuracy: 0.8138 - val_loss: 0.4449 - val_accuracy: 0.7975\n",
      "Epoch 9/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.4125 - accuracy: 0.82 - ETA: 2s - loss: 0.3981 - accuracy: 0.83 - ETA: 2s - loss: 0.4097 - accuracy: 0.82 - ETA: 2s - loss: 0.4046 - accuracy: 0.83 - ETA: 2s - loss: 0.4036 - accuracy: 0.82 - ETA: 2s - loss: 0.4035 - accuracy: 0.82 - ETA: 2s - loss: 0.4008 - accuracy: 0.82 - ETA: 1s - loss: 0.4012 - accuracy: 0.82 - ETA: 1s - loss: 0.4029 - accuracy: 0.82 - ETA: 1s - loss: 0.4035 - accuracy: 0.82 - ETA: 1s - loss: 0.4009 - accuracy: 0.82 - ETA: 1s - loss: 0.4017 - accuracy: 0.82 - ETA: 1s - loss: 0.4020 - accuracy: 0.82 - ETA: 1s - loss: 0.4025 - accuracy: 0.82 - ETA: 1s - loss: 0.4041 - accuracy: 0.82 - ETA: 1s - loss: 0.4042 - accuracy: 0.82 - ETA: 1s - loss: 0.4034 - accuracy: 0.82 - ETA: 0s - loss: 0.4028 - accuracy: 0.82 - ETA: 0s - loss: 0.4017 - accuracy: 0.82 - ETA: 0s - loss: 0.4001 - accuracy: 0.82 - ETA: 0s - loss: 0.3991 - accuracy: 0.82 - ETA: 0s - loss: 0.3982 - accuracy: 0.83 - ETA: 0s - loss: 0.3974 - accuracy: 0.83 - ETA: 0s - loss: 0.3971 - accuracy: 0.83 - ETA: 0s - loss: 0.3962 - accuracy: 0.83 - ETA: 0s - loss: 0.3964 - accuracy: 0.83 - ETA: 0s - loss: 0.3958 - accuracy: 0.83 - ETA: 0s - loss: 0.3949 - accuracy: 0.83 - ETA: 0s - loss: 0.3943 - accuracy: 0.83 - 4s 251us/sample - loss: 0.3943 - accuracy: 0.8340 - val_loss: 0.4251 - val_accuracy: 0.8105\n",
      "Epoch 10/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.3864 - accuracy: 0.86 - ETA: 2s - loss: 0.3693 - accuracy: 0.86 - ETA: 2s - loss: 0.3714 - accuracy: 0.85 - ETA: 2s - loss: 0.3711 - accuracy: 0.85 - ETA: 2s - loss: 0.3758 - accuracy: 0.84 - ETA: 2s - loss: 0.3759 - accuracy: 0.84 - ETA: 1s - loss: 0.3762 - accuracy: 0.84 - ETA: 1s - loss: 0.3726 - accuracy: 0.85 - ETA: 1s - loss: 0.3766 - accuracy: 0.84 - ETA: 1s - loss: 0.3777 - accuracy: 0.84 - ETA: 1s - loss: 0.3810 - accuracy: 0.84 - ETA: 1s - loss: 0.3799 - accuracy: 0.84 - ETA: 1s - loss: 0.3794 - accuracy: 0.84 - ETA: 1s - loss: 0.3778 - accuracy: 0.84 - ETA: 1s - loss: 0.3768 - accuracy: 0.84 - ETA: 1s - loss: 0.3771 - accuracy: 0.84 - ETA: 1s - loss: 0.3769 - accuracy: 0.84 - ETA: 0s - loss: 0.3766 - accuracy: 0.84 - ETA: 0s - loss: 0.3756 - accuracy: 0.84 - ETA: 0s - loss: 0.3766 - accuracy: 0.84 - ETA: 0s - loss: 0.3757 - accuracy: 0.84 - ETA: 0s - loss: 0.3728 - accuracy: 0.84 - ETA: 0s - loss: 0.3720 - accuracy: 0.84 - ETA: 0s - loss: 0.3702 - accuracy: 0.84 - ETA: 0s - loss: 0.3698 - accuracy: 0.84 - ETA: 0s - loss: 0.3701 - accuracy: 0.84 - ETA: 0s - loss: 0.3704 - accuracy: 0.84 - ETA: 0s - loss: 0.3702 - accuracy: 0.84 - ETA: 0s - loss: 0.3688 - accuracy: 0.84 - 4s 260us/sample - loss: 0.3693 - accuracy: 0.8477 - val_loss: 0.4069 - val_accuracy: 0.8195\n",
      "Epoch 11/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.3461 - accuracy: 0.86 - ETA: 2s - loss: 0.3697 - accuracy: 0.84 - ETA: 2s - loss: 0.3710 - accuracy: 0.84 - ETA: 2s - loss: 0.3679 - accuracy: 0.85 - ETA: 2s - loss: 0.3624 - accuracy: 0.85 - ETA: 2s - loss: 0.3608 - accuracy: 0.85 - ETA: 1s - loss: 0.3605 - accuracy: 0.85 - ETA: 1s - loss: 0.3602 - accuracy: 0.85 - ETA: 1s - loss: 0.3602 - accuracy: 0.85 - ETA: 1s - loss: 0.3595 - accuracy: 0.85 - ETA: 1s - loss: 0.3579 - accuracy: 0.85 - ETA: 1s - loss: 0.3552 - accuracy: 0.85 - ETA: 1s - loss: 0.3534 - accuracy: 0.85 - ETA: 1s - loss: 0.3539 - accuracy: 0.85 - ETA: 1s - loss: 0.3528 - accuracy: 0.85 - ETA: 1s - loss: 0.3526 - accuracy: 0.85 - ETA: 1s - loss: 0.3513 - accuracy: 0.85 - ETA: 0s - loss: 0.3482 - accuracy: 0.85 - ETA: 0s - loss: 0.3469 - accuracy: 0.85 - ETA: 0s - loss: 0.3493 - accuracy: 0.85 - ETA: 0s - loss: 0.3479 - accuracy: 0.85 - ETA: 0s - loss: 0.3476 - accuracy: 0.85 - ETA: 0s - loss: 0.3475 - accuracy: 0.86 - ETA: 0s - loss: 0.3481 - accuracy: 0.85 - ETA: 0s - loss: 0.3474 - accuracy: 0.86 - ETA: 0s - loss: 0.3475 - accuracy: 0.86 - ETA: 0s - loss: 0.3479 - accuracy: 0.85 - ETA: 0s - loss: 0.3465 - accuracy: 0.86 - ETA: 0s - loss: 0.3470 - accuracy: 0.85 - 4s 251us/sample - loss: 0.3463 - accuracy: 0.8601 - val_loss: 0.3907 - val_accuracy: 0.8305\n",
      "Epoch 12/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.3317 - accuracy: 0.87 - ETA: 2s - loss: 0.3303 - accuracy: 0.87 - ETA: 2s - loss: 0.3377 - accuracy: 0.87 - ETA: 2s - loss: 0.3367 - accuracy: 0.86 - ETA: 2s - loss: 0.3367 - accuracy: 0.86 - ETA: 2s - loss: 0.3357 - accuracy: 0.87 - ETA: 1s - loss: 0.3342 - accuracy: 0.87 - ETA: 1s - loss: 0.3357 - accuracy: 0.86 - ETA: 1s - loss: 0.3314 - accuracy: 0.87 - ETA: 1s - loss: 0.3301 - accuracy: 0.87 - ETA: 1s - loss: 0.3293 - accuracy: 0.87 - ETA: 1s - loss: 0.3269 - accuracy: 0.87 - ETA: 1s - loss: 0.3266 - accuracy: 0.87 - ETA: 1s - loss: 0.3279 - accuracy: 0.87 - ETA: 1s - loss: 0.3274 - accuracy: 0.86 - ETA: 1s - loss: 0.3295 - accuracy: 0.86 - ETA: 1s - loss: 0.3278 - accuracy: 0.86 - ETA: 0s - loss: 0.3289 - accuracy: 0.86 - ETA: 0s - loss: 0.3284 - accuracy: 0.86 - ETA: 0s - loss: 0.3281 - accuracy: 0.86 - ETA: 0s - loss: 0.3273 - accuracy: 0.86 - ETA: 0s - loss: 0.3277 - accuracy: 0.86 - ETA: 0s - loss: 0.3269 - accuracy: 0.86 - ETA: 0s - loss: 0.3271 - accuracy: 0.86 - ETA: 0s - loss: 0.3264 - accuracy: 0.86 - ETA: 0s - loss: 0.3265 - accuracy: 0.86 - ETA: 0s - loss: 0.3257 - accuracy: 0.87 - ETA: 0s - loss: 0.3239 - accuracy: 0.87 - ETA: 0s - loss: 0.3234 - accuracy: 0.87 - 4s 242us/sample - loss: 0.3237 - accuracy: 0.8711 - val_loss: 0.3765 - val_accuracy: 0.8368\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 13/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.3146 - accuracy: 0.88 - ETA: 2s - loss: 0.3101 - accuracy: 0.88 - ETA: 2s - loss: 0.3155 - accuracy: 0.87 - ETA: 2s - loss: 0.3138 - accuracy: 0.87 - ETA: 2s - loss: 0.3152 - accuracy: 0.87 - ETA: 1s - loss: 0.3130 - accuracy: 0.87 - ETA: 1s - loss: 0.3092 - accuracy: 0.88 - ETA: 1s - loss: 0.3102 - accuracy: 0.87 - ETA: 1s - loss: 0.3121 - accuracy: 0.87 - ETA: 1s - loss: 0.3097 - accuracy: 0.87 - ETA: 1s - loss: 0.3097 - accuracy: 0.87 - ETA: 1s - loss: 0.3118 - accuracy: 0.87 - ETA: 1s - loss: 0.3110 - accuracy: 0.87 - ETA: 1s - loss: 0.3119 - accuracy: 0.87 - ETA: 1s - loss: 0.3114 - accuracy: 0.87 - ETA: 1s - loss: 0.3089 - accuracy: 0.87 - ETA: 0s - loss: 0.3058 - accuracy: 0.88 - ETA: 0s - loss: 0.3042 - accuracy: 0.88 - ETA: 0s - loss: 0.3060 - accuracy: 0.88 - ETA: 0s - loss: 0.3060 - accuracy: 0.88 - ETA: 0s - loss: 0.3054 - accuracy: 0.88 - ETA: 0s - loss: 0.3059 - accuracy: 0.87 - ETA: 0s - loss: 0.3038 - accuracy: 0.88 - ETA: 0s - loss: 0.3029 - accuracy: 0.88 - ETA: 0s - loss: 0.3034 - accuracy: 0.88 - ETA: 0s - loss: 0.3051 - accuracy: 0.88 - ETA: 0s - loss: 0.3054 - accuracy: 0.87 - ETA: 0s - loss: 0.3043 - accuracy: 0.88 - ETA: 0s - loss: 0.3040 - accuracy: 0.88 - 4s 234us/sample - loss: 0.3035 - accuracy: 0.8816 - val_loss: 0.3641 - val_accuracy: 0.8406\n",
      "Epoch 14/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.2847 - accuracy: 0.89 - ETA: 2s - loss: 0.2823 - accuracy: 0.89 - ETA: 2s - loss: 0.2882 - accuracy: 0.88 - ETA: 2s - loss: 0.2884 - accuracy: 0.88 - ETA: 2s - loss: 0.2880 - accuracy: 0.89 - ETA: 1s - loss: 0.2889 - accuracy: 0.88 - ETA: 1s - loss: 0.2904 - accuracy: 0.88 - ETA: 1s - loss: 0.2914 - accuracy: 0.88 - ETA: 1s - loss: 0.2942 - accuracy: 0.88 - ETA: 1s - loss: 0.2914 - accuracy: 0.88 - ETA: 1s - loss: 0.2882 - accuracy: 0.89 - ETA: 1s - loss: 0.2886 - accuracy: 0.89 - ETA: 1s - loss: 0.2886 - accuracy: 0.89 - ETA: 1s - loss: 0.2894 - accuracy: 0.89 - ETA: 1s - loss: 0.2884 - accuracy: 0.89 - ETA: 1s - loss: 0.2879 - accuracy: 0.89 - ETA: 0s - loss: 0.2870 - accuracy: 0.89 - ETA: 0s - loss: 0.2875 - accuracy: 0.89 - ETA: 0s - loss: 0.2875 - accuracy: 0.88 - ETA: 0s - loss: 0.2874 - accuracy: 0.88 - ETA: 0s - loss: 0.2871 - accuracy: 0.88 - ETA: 0s - loss: 0.2872 - accuracy: 0.88 - ETA: 0s - loss: 0.2870 - accuracy: 0.88 - ETA: 0s - loss: 0.2867 - accuracy: 0.89 - ETA: 0s - loss: 0.2867 - accuracy: 0.89 - ETA: 0s - loss: 0.2864 - accuracy: 0.88 - ETA: 0s - loss: 0.2856 - accuracy: 0.89 - ETA: 0s - loss: 0.2846 - accuracy: 0.89 - ETA: 0s - loss: 0.2846 - accuracy: 0.89 - 3s 233us/sample - loss: 0.2848 - accuracy: 0.8911 - val_loss: 0.3547 - val_accuracy: 0.8408\n",
      "Epoch 15/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.2742 - accuracy: 0.90 - ETA: 2s - loss: 0.2861 - accuracy: 0.89 - ETA: 2s - loss: 0.2938 - accuracy: 0.88 - ETA: 2s - loss: 0.2847 - accuracy: 0.88 - ETA: 2s - loss: 0.2832 - accuracy: 0.89 - ETA: 1s - loss: 0.2887 - accuracy: 0.88 - ETA: 1s - loss: 0.2803 - accuracy: 0.89 - ETA: 1s - loss: 0.2813 - accuracy: 0.89 - ETA: 1s - loss: 0.2791 - accuracy: 0.89 - ETA: 1s - loss: 0.2779 - accuracy: 0.89 - ETA: 1s - loss: 0.2770 - accuracy: 0.89 - ETA: 1s - loss: 0.2776 - accuracy: 0.89 - ETA: 1s - loss: 0.2783 - accuracy: 0.89 - ETA: 1s - loss: 0.2777 - accuracy: 0.89 - ETA: 1s - loss: 0.2769 - accuracy: 0.89 - ETA: 1s - loss: 0.2757 - accuracy: 0.89 - ETA: 1s - loss: 0.2769 - accuracy: 0.89 - ETA: 0s - loss: 0.2753 - accuracy: 0.89 - ETA: 0s - loss: 0.2740 - accuracy: 0.89 - ETA: 0s - loss: 0.2743 - accuracy: 0.89 - ETA: 0s - loss: 0.2744 - accuracy: 0.89 - ETA: 0s - loss: 0.2734 - accuracy: 0.89 - ETA: 0s - loss: 0.2735 - accuracy: 0.89 - ETA: 0s - loss: 0.2714 - accuracy: 0.89 - ETA: 0s - loss: 0.2712 - accuracy: 0.89 - ETA: 0s - loss: 0.2696 - accuracy: 0.89 - ETA: 0s - loss: 0.2694 - accuracy: 0.89 - ETA: 0s - loss: 0.2679 - accuracy: 0.89 - ETA: 0s - loss: 0.2673 - accuracy: 0.89 - 4s 239us/sample - loss: 0.2677 - accuracy: 0.8987 - val_loss: 0.3452 - val_accuracy: 0.8456\n",
      "Epoch 16/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.2348 - accuracy: 0.92 - ETA: 2s - loss: 0.2418 - accuracy: 0.91 - ETA: 2s - loss: 0.2504 - accuracy: 0.90 - ETA: 2s - loss: 0.2509 - accuracy: 0.90 - ETA: 2s - loss: 0.2540 - accuracy: 0.90 - ETA: 1s - loss: 0.2599 - accuracy: 0.89 - ETA: 1s - loss: 0.2618 - accuracy: 0.89 - ETA: 1s - loss: 0.2613 - accuracy: 0.89 - ETA: 1s - loss: 0.2600 - accuracy: 0.89 - ETA: 1s - loss: 0.2598 - accuracy: 0.89 - ETA: 1s - loss: 0.2578 - accuracy: 0.90 - ETA: 1s - loss: 0.2559 - accuracy: 0.90 - ETA: 1s - loss: 0.2557 - accuracy: 0.90 - ETA: 1s - loss: 0.2541 - accuracy: 0.90 - ETA: 1s - loss: 0.2535 - accuracy: 0.90 - ETA: 1s - loss: 0.2539 - accuracy: 0.90 - ETA: 0s - loss: 0.2531 - accuracy: 0.90 - ETA: 0s - loss: 0.2544 - accuracy: 0.90 - ETA: 0s - loss: 0.2551 - accuracy: 0.90 - ETA: 0s - loss: 0.2549 - accuracy: 0.90 - ETA: 0s - loss: 0.2543 - accuracy: 0.90 - ETA: 0s - loss: 0.2533 - accuracy: 0.90 - ETA: 0s - loss: 0.2524 - accuracy: 0.90 - ETA: 0s - loss: 0.2526 - accuracy: 0.90 - ETA: 0s - loss: 0.2530 - accuracy: 0.90 - ETA: 0s - loss: 0.2525 - accuracy: 0.90 - ETA: 0s - loss: 0.2522 - accuracy: 0.90 - ETA: 0s - loss: 0.2518 - accuracy: 0.90 - ETA: 0s - loss: 0.2520 - accuracy: 0.90 - 3s 224us/sample - loss: 0.2525 - accuracy: 0.9061 - val_loss: 0.3360 - val_accuracy: 0.8531\n",
      "Epoch 17/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.2248 - accuracy: 0.91 - ETA: 2s - loss: 0.2376 - accuracy: 0.91 - ETA: 2s - loss: 0.2348 - accuracy: 0.90 - ETA: 2s - loss: 0.2386 - accuracy: 0.90 - ETA: 1s - loss: 0.2351 - accuracy: 0.91 - ETA: 1s - loss: 0.2353 - accuracy: 0.91 - ETA: 1s - loss: 0.2372 - accuracy: 0.91 - ETA: 1s - loss: 0.2369 - accuracy: 0.91 - ETA: 1s - loss: 0.2407 - accuracy: 0.91 - ETA: 1s - loss: 0.2386 - accuracy: 0.91 - ETA: 1s - loss: 0.2397 - accuracy: 0.90 - ETA: 1s - loss: 0.2390 - accuracy: 0.91 - ETA: 1s - loss: 0.2375 - accuracy: 0.91 - ETA: 1s - loss: 0.2374 - accuracy: 0.91 - ETA: 1s - loss: 0.2384 - accuracy: 0.91 - ETA: 0s - loss: 0.2375 - accuracy: 0.91 - ETA: 0s - loss: 0.2371 - accuracy: 0.91 - ETA: 0s - loss: 0.2375 - accuracy: 0.91 - ETA: 0s - loss: 0.2366 - accuracy: 0.91 - ETA: 0s - loss: 0.2366 - accuracy: 0.91 - ETA: 0s - loss: 0.2365 - accuracy: 0.91 - ETA: 0s - loss: 0.2390 - accuracy: 0.91 - ETA: 0s - loss: 0.2389 - accuracy: 0.91 - ETA: 0s - loss: 0.2390 - accuracy: 0.91 - ETA: 0s - loss: 0.2382 - accuracy: 0.91 - ETA: 0s - loss: 0.2378 - accuracy: 0.91 - ETA: 0s - loss: 0.2379 - accuracy: 0.91 - ETA: 0s - loss: 0.2372 - accuracy: 0.91 - ETA: 0s - loss: 0.2372 - accuracy: 0.91 - 3s 225us/sample - loss: 0.2374 - accuracy: 0.9129 - val_loss: 0.3296 - val_accuracy: 0.8560\n",
      "Epoch 18/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.1968 - accuracy: 0.93 - ETA: 2s - loss: 0.2153 - accuracy: 0.91 - ETA: 2s - loss: 0.2124 - accuracy: 0.92 - ETA: 2s - loss: 0.2205 - accuracy: 0.92 - ETA: 1s - loss: 0.2237 - accuracy: 0.91 - ETA: 1s - loss: 0.2207 - accuracy: 0.92 - ETA: 1s - loss: 0.2224 - accuracy: 0.92 - ETA: 1s - loss: 0.2238 - accuracy: 0.91 - ETA: 1s - loss: 0.2236 - accuracy: 0.91 - ETA: 1s - loss: 0.2218 - accuracy: 0.92 - ETA: 1s - loss: 0.2241 - accuracy: 0.92 - ETA: 1s - loss: 0.2237 - accuracy: 0.92 - ETA: 1s - loss: 0.2231 - accuracy: 0.92 - ETA: 1s - loss: 0.2243 - accuracy: 0.92 - ETA: 1s - loss: 0.2247 - accuracy: 0.92 - ETA: 0s - loss: 0.2260 - accuracy: 0.91 - ETA: 0s - loss: 0.2259 - accuracy: 0.91 - ETA: 0s - loss: 0.2272 - accuracy: 0.91 - ETA: 0s - loss: 0.2268 - accuracy: 0.91 - ETA: 0s - loss: 0.2250 - accuracy: 0.91 - ETA: 0s - loss: 0.2262 - accuracy: 0.91 - ETA: 0s - loss: 0.2249 - accuracy: 0.91 - ETA: 0s - loss: 0.2239 - accuracy: 0.91 - ETA: 0s - loss: 0.2250 - accuracy: 0.91 - ETA: 0s - loss: 0.2249 - accuracy: 0.91 - ETA: 0s - loss: 0.2240 - accuracy: 0.91 - ETA: 0s - loss: 0.2230 - accuracy: 0.92 - ETA: 0s - loss: 0.2231 - accuracy: 0.92 - ETA: 0s - loss: 0.2229 - accuracy: 0.92 - 3s 233us/sample - loss: 0.2231 - accuracy: 0.9197 - val_loss: 0.3243 - val_accuracy: 0.8582\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 19/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.2219 - accuracy: 0.92 - ETA: 2s - loss: 0.2193 - accuracy: 0.92 - ETA: 2s - loss: 0.2170 - accuracy: 0.92 - ETA: 2s - loss: 0.2161 - accuracy: 0.92 - ETA: 2s - loss: 0.2229 - accuracy: 0.91 - ETA: 2s - loss: 0.2190 - accuracy: 0.91 - ETA: 1s - loss: 0.2189 - accuracy: 0.91 - ETA: 1s - loss: 0.2225 - accuracy: 0.91 - ETA: 1s - loss: 0.2202 - accuracy: 0.91 - ETA: 1s - loss: 0.2190 - accuracy: 0.91 - ETA: 1s - loss: 0.2183 - accuracy: 0.91 - ETA: 1s - loss: 0.2196 - accuracy: 0.91 - ETA: 1s - loss: 0.2191 - accuracy: 0.91 - ETA: 1s - loss: 0.2181 - accuracy: 0.91 - ETA: 1s - loss: 0.2169 - accuracy: 0.92 - ETA: 1s - loss: 0.2160 - accuracy: 0.92 - ETA: 1s - loss: 0.2141 - accuracy: 0.92 - ETA: 0s - loss: 0.2115 - accuracy: 0.92 - ETA: 0s - loss: 0.2109 - accuracy: 0.92 - ETA: 0s - loss: 0.2109 - accuracy: 0.92 - ETA: 0s - loss: 0.2110 - accuracy: 0.92 - ETA: 0s - loss: 0.2103 - accuracy: 0.92 - ETA: 0s - loss: 0.2110 - accuracy: 0.92 - ETA: 0s - loss: 0.2109 - accuracy: 0.92 - ETA: 0s - loss: 0.2108 - accuracy: 0.92 - ETA: 0s - loss: 0.2113 - accuracy: 0.92 - ETA: 0s - loss: 0.2111 - accuracy: 0.92 - ETA: 0s - loss: 0.2104 - accuracy: 0.92 - ETA: 0s - loss: 0.2113 - accuracy: 0.92 - 4s 259us/sample - loss: 0.2109 - accuracy: 0.9247 - val_loss: 0.3199 - val_accuracy: 0.8625\n",
      "Epoch 20/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.2157 - accuracy: 0.91 - ETA: 2s - loss: 0.2231 - accuracy: 0.91 - ETA: 2s - loss: 0.2091 - accuracy: 0.92 - ETA: 2s - loss: 0.2077 - accuracy: 0.92 - ETA: 2s - loss: 0.2027 - accuracy: 0.92 - ETA: 2s - loss: 0.2041 - accuracy: 0.92 - ETA: 2s - loss: 0.1996 - accuracy: 0.92 - ETA: 1s - loss: 0.1955 - accuracy: 0.93 - ETA: 1s - loss: 0.1930 - accuracy: 0.93 - ETA: 1s - loss: 0.1975 - accuracy: 0.92 - ETA: 1s - loss: 0.1986 - accuracy: 0.92 - ETA: 1s - loss: 0.1974 - accuracy: 0.92 - ETA: 1s - loss: 0.1984 - accuracy: 0.92 - ETA: 1s - loss: 0.1988 - accuracy: 0.92 - ETA: 1s - loss: 0.2000 - accuracy: 0.92 - ETA: 1s - loss: 0.2017 - accuracy: 0.92 - ETA: 1s - loss: 0.2014 - accuracy: 0.92 - ETA: 0s - loss: 0.2005 - accuracy: 0.92 - ETA: 0s - loss: 0.1988 - accuracy: 0.92 - ETA: 0s - loss: 0.1991 - accuracy: 0.93 - ETA: 0s - loss: 0.1984 - accuracy: 0.93 - ETA: 0s - loss: 0.1988 - accuracy: 0.93 - ETA: 0s - loss: 0.1997 - accuracy: 0.92 - ETA: 0s - loss: 0.2003 - accuracy: 0.92 - ETA: 0s - loss: 0.2002 - accuracy: 0.92 - ETA: 0s - loss: 0.1990 - accuracy: 0.93 - ETA: 0s - loss: 0.1999 - accuracy: 0.93 - ETA: 0s - loss: 0.1995 - accuracy: 0.93 - ETA: 0s - loss: 0.1992 - accuracy: 0.93 - 4s 255us/sample - loss: 0.1991 - accuracy: 0.9303 - val_loss: 0.3167 - val_accuracy: 0.8650\n",
      "Epoch 21/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1764 - accuracy: 0.94 - ETA: 2s - loss: 0.1837 - accuracy: 0.93 - ETA: 2s - loss: 0.1899 - accuracy: 0.93 - ETA: 2s - loss: 0.1831 - accuracy: 0.93 - ETA: 2s - loss: 0.1852 - accuracy: 0.93 - ETA: 2s - loss: 0.1821 - accuracy: 0.93 - ETA: 2s - loss: 0.1831 - accuracy: 0.93 - ETA: 1s - loss: 0.1884 - accuracy: 0.93 - ETA: 1s - loss: 0.1883 - accuracy: 0.93 - ETA: 1s - loss: 0.1886 - accuracy: 0.93 - ETA: 1s - loss: 0.1889 - accuracy: 0.93 - ETA: 1s - loss: 0.1897 - accuracy: 0.93 - ETA: 1s - loss: 0.1890 - accuracy: 0.93 - ETA: 1s - loss: 0.1898 - accuracy: 0.93 - ETA: 1s - loss: 0.1895 - accuracy: 0.93 - ETA: 1s - loss: 0.1883 - accuracy: 0.93 - ETA: 1s - loss: 0.1881 - accuracy: 0.93 - ETA: 0s - loss: 0.1882 - accuracy: 0.93 - ETA: 0s - loss: 0.1887 - accuracy: 0.93 - ETA: 0s - loss: 0.1884 - accuracy: 0.93 - ETA: 0s - loss: 0.1870 - accuracy: 0.93 - ETA: 0s - loss: 0.1859 - accuracy: 0.93 - ETA: 0s - loss: 0.1858 - accuracy: 0.93 - ETA: 0s - loss: 0.1873 - accuracy: 0.93 - ETA: 0s - loss: 0.1873 - accuracy: 0.93 - ETA: 0s - loss: 0.1871 - accuracy: 0.93 - ETA: 0s - loss: 0.1870 - accuracy: 0.93 - ETA: 0s - loss: 0.1878 - accuracy: 0.93 - ETA: 0s - loss: 0.1889 - accuracy: 0.93 - 4s 257us/sample - loss: 0.1884 - accuracy: 0.9355 - val_loss: 0.3146 - val_accuracy: 0.8686\n",
      "Epoch 22/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1677 - accuracy: 0.95 - ETA: 2s - loss: 0.1700 - accuracy: 0.95 - ETA: 2s - loss: 0.1645 - accuracy: 0.95 - ETA: 2s - loss: 0.1723 - accuracy: 0.94 - ETA: 2s - loss: 0.1713 - accuracy: 0.94 - ETA: 2s - loss: 0.1744 - accuracy: 0.94 - ETA: 1s - loss: 0.1722 - accuracy: 0.94 - ETA: 1s - loss: 0.1729 - accuracy: 0.94 - ETA: 1s - loss: 0.1773 - accuracy: 0.94 - ETA: 1s - loss: 0.1771 - accuracy: 0.94 - ETA: 1s - loss: 0.1766 - accuracy: 0.94 - ETA: 1s - loss: 0.1748 - accuracy: 0.94 - ETA: 1s - loss: 0.1765 - accuracy: 0.94 - ETA: 1s - loss: 0.1782 - accuracy: 0.94 - ETA: 1s - loss: 0.1770 - accuracy: 0.94 - ETA: 1s - loss: 0.1784 - accuracy: 0.94 - ETA: 1s - loss: 0.1786 - accuracy: 0.94 - ETA: 0s - loss: 0.1769 - accuracy: 0.94 - ETA: 0s - loss: 0.1778 - accuracy: 0.94 - ETA: 0s - loss: 0.1784 - accuracy: 0.94 - ETA: 0s - loss: 0.1770 - accuracy: 0.94 - ETA: 0s - loss: 0.1767 - accuracy: 0.94 - ETA: 0s - loss: 0.1761 - accuracy: 0.94 - ETA: 0s - loss: 0.1757 - accuracy: 0.94 - ETA: 0s - loss: 0.1756 - accuracy: 0.94 - ETA: 0s - loss: 0.1756 - accuracy: 0.94 - ETA: 0s - loss: 0.1769 - accuracy: 0.94 - ETA: 0s - loss: 0.1779 - accuracy: 0.94 - ETA: 0s - loss: 0.1782 - accuracy: 0.94 - 4s 260us/sample - loss: 0.1781 - accuracy: 0.9401 - val_loss: 0.3126 - val_accuracy: 0.8700\n",
      "Epoch 23/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1836 - accuracy: 0.92 - ETA: 2s - loss: 0.1710 - accuracy: 0.93 - ETA: 2s - loss: 0.1731 - accuracy: 0.93 - ETA: 2s - loss: 0.1771 - accuracy: 0.93 - ETA: 2s - loss: 0.1758 - accuracy: 0.94 - ETA: 2s - loss: 0.1774 - accuracy: 0.94 - ETA: 2s - loss: 0.1782 - accuracy: 0.94 - ETA: 1s - loss: 0.1734 - accuracy: 0.94 - ETA: 1s - loss: 0.1754 - accuracy: 0.94 - ETA: 1s - loss: 0.1751 - accuracy: 0.94 - ETA: 1s - loss: 0.1740 - accuracy: 0.94 - ETA: 1s - loss: 0.1762 - accuracy: 0.93 - ETA: 1s - loss: 0.1768 - accuracy: 0.93 - ETA: 1s - loss: 0.1763 - accuracy: 0.93 - ETA: 1s - loss: 0.1748 - accuracy: 0.93 - ETA: 1s - loss: 0.1744 - accuracy: 0.93 - ETA: 1s - loss: 0.1740 - accuracy: 0.93 - ETA: 1s - loss: 0.1732 - accuracy: 0.94 - ETA: 0s - loss: 0.1724 - accuracy: 0.94 - ETA: 0s - loss: 0.1714 - accuracy: 0.94 - ETA: 0s - loss: 0.1712 - accuracy: 0.94 - ETA: 0s - loss: 0.1707 - accuracy: 0.94 - ETA: 0s - loss: 0.1702 - accuracy: 0.94 - ETA: 0s - loss: 0.1690 - accuracy: 0.94 - ETA: 0s - loss: 0.1696 - accuracy: 0.94 - ETA: 0s - loss: 0.1691 - accuracy: 0.94 - ETA: 0s - loss: 0.1687 - accuracy: 0.94 - ETA: 0s - loss: 0.1691 - accuracy: 0.94 - ETA: 0s - loss: 0.1695 - accuracy: 0.94 - 4s 259us/sample - loss: 0.1690 - accuracy: 0.9432 - val_loss: 0.3118 - val_accuracy: 0.8714\n",
      "Epoch 24/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1451 - accuracy: 0.95 - ETA: 2s - loss: 0.1466 - accuracy: 0.95 - ETA: 2s - loss: 0.1485 - accuracy: 0.95 - ETA: 2s - loss: 0.1544 - accuracy: 0.94 - ETA: 2s - loss: 0.1573 - accuracy: 0.94 - ETA: 2s - loss: 0.1558 - accuracy: 0.94 - ETA: 1s - loss: 0.1575 - accuracy: 0.94 - ETA: 1s - loss: 0.1579 - accuracy: 0.94 - ETA: 1s - loss: 0.1587 - accuracy: 0.94 - ETA: 1s - loss: 0.1587 - accuracy: 0.94 - ETA: 1s - loss: 0.1582 - accuracy: 0.94 - ETA: 1s - loss: 0.1591 - accuracy: 0.94 - ETA: 1s - loss: 0.1607 - accuracy: 0.94 - ETA: 1s - loss: 0.1602 - accuracy: 0.94 - ETA: 1s - loss: 0.1596 - accuracy: 0.94 - ETA: 1s - loss: 0.1585 - accuracy: 0.94 - ETA: 1s - loss: 0.1594 - accuracy: 0.94 - ETA: 0s - loss: 0.1595 - accuracy: 0.94 - ETA: 0s - loss: 0.1602 - accuracy: 0.94 - ETA: 0s - loss: 0.1607 - accuracy: 0.94 - ETA: 0s - loss: 0.1596 - accuracy: 0.94 - ETA: 0s - loss: 0.1592 - accuracy: 0.94 - ETA: 0s - loss: 0.1592 - accuracy: 0.94 - ETA: 0s - loss: 0.1590 - accuracy: 0.94 - ETA: 0s - loss: 0.1596 - accuracy: 0.94 - ETA: 0s - loss: 0.1597 - accuracy: 0.94 - ETA: 0s - loss: 0.1599 - accuracy: 0.94 - ETA: 0s - loss: 0.1597 - accuracy: 0.94 - ETA: 0s - loss: 0.1594 - accuracy: 0.94 - 4s 258us/sample - loss: 0.1593 - accuracy: 0.9478 - val_loss: 0.3132 - val_accuracy: 0.8712\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 25/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1835 - accuracy: 0.93 - ETA: 2s - loss: 0.1621 - accuracy: 0.94 - ETA: 2s - loss: 0.1642 - accuracy: 0.94 - ETA: 2s - loss: 0.1606 - accuracy: 0.94 - ETA: 2s - loss: 0.1560 - accuracy: 0.94 - ETA: 2s - loss: 0.1544 - accuracy: 0.94 - ETA: 2s - loss: 0.1560 - accuracy: 0.94 - ETA: 1s - loss: 0.1586 - accuracy: 0.94 - ETA: 1s - loss: 0.1583 - accuracy: 0.94 - ETA: 1s - loss: 0.1545 - accuracy: 0.95 - ETA: 1s - loss: 0.1538 - accuracy: 0.95 - ETA: 1s - loss: 0.1535 - accuracy: 0.95 - ETA: 1s - loss: 0.1523 - accuracy: 0.95 - ETA: 1s - loss: 0.1532 - accuracy: 0.95 - ETA: 1s - loss: 0.1541 - accuracy: 0.95 - ETA: 1s - loss: 0.1531 - accuracy: 0.95 - ETA: 1s - loss: 0.1525 - accuracy: 0.95 - ETA: 1s - loss: 0.1531 - accuracy: 0.95 - ETA: 0s - loss: 0.1535 - accuracy: 0.95 - ETA: 0s - loss: 0.1535 - accuracy: 0.95 - ETA: 0s - loss: 0.1529 - accuracy: 0.95 - ETA: 0s - loss: 0.1530 - accuracy: 0.95 - ETA: 0s - loss: 0.1526 - accuracy: 0.95 - ETA: 0s - loss: 0.1531 - accuracy: 0.95 - ETA: 0s - loss: 0.1530 - accuracy: 0.95 - ETA: 0s - loss: 0.1526 - accuracy: 0.95 - ETA: 0s - loss: 0.1517 - accuracy: 0.95 - ETA: 0s - loss: 0.1509 - accuracy: 0.95 - ETA: 0s - loss: 0.1507 - accuracy: 0.95 - 4s 260us/sample - loss: 0.1510 - accuracy: 0.9522 - val_loss: 0.3124 - val_accuracy: 0.8725\n",
      "Epoch 26/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1405 - accuracy: 0.94 - ETA: 2s - loss: 0.1424 - accuracy: 0.95 - ETA: 2s - loss: 0.1398 - accuracy: 0.95 - ETA: 2s - loss: 0.1444 - accuracy: 0.95 - ETA: 2s - loss: 0.1449 - accuracy: 0.95 - ETA: 2s - loss: 0.1417 - accuracy: 0.95 - ETA: 2s - loss: 0.1402 - accuracy: 0.95 - ETA: 1s - loss: 0.1434 - accuracy: 0.95 - ETA: 1s - loss: 0.1411 - accuracy: 0.95 - ETA: 1s - loss: 0.1440 - accuracy: 0.95 - ETA: 1s - loss: 0.1431 - accuracy: 0.95 - ETA: 1s - loss: 0.1426 - accuracy: 0.95 - ETA: 1s - loss: 0.1436 - accuracy: 0.95 - ETA: 1s - loss: 0.1439 - accuracy: 0.95 - ETA: 1s - loss: 0.1436 - accuracy: 0.95 - ETA: 1s - loss: 0.1448 - accuracy: 0.95 - ETA: 1s - loss: 0.1430 - accuracy: 0.95 - ETA: 0s - loss: 0.1430 - accuracy: 0.95 - ETA: 0s - loss: 0.1419 - accuracy: 0.95 - ETA: 0s - loss: 0.1427 - accuracy: 0.95 - ETA: 0s - loss: 0.1424 - accuracy: 0.95 - ETA: 0s - loss: 0.1417 - accuracy: 0.95 - ETA: 0s - loss: 0.1421 - accuracy: 0.95 - ETA: 0s - loss: 0.1422 - accuracy: 0.95 - ETA: 0s - loss: 0.1417 - accuracy: 0.95 - ETA: 0s - loss: 0.1409 - accuracy: 0.95 - ETA: 0s - loss: 0.1418 - accuracy: 0.95 - ETA: 0s - loss: 0.1421 - accuracy: 0.95 - ETA: 0s - loss: 0.1432 - accuracy: 0.95 - 4s 260us/sample - loss: 0.1430 - accuracy: 0.9553 - val_loss: 0.3135 - val_accuracy: 0.8727\n",
      "Epoch 27/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1580 - accuracy: 0.94 - ETA: 2s - loss: 0.1430 - accuracy: 0.94 - ETA: 2s - loss: 0.1400 - accuracy: 0.94 - ETA: 2s - loss: 0.1402 - accuracy: 0.95 - ETA: 2s - loss: 0.1385 - accuracy: 0.95 - ETA: 2s - loss: 0.1362 - accuracy: 0.95 - ETA: 1s - loss: 0.1363 - accuracy: 0.95 - ETA: 1s - loss: 0.1378 - accuracy: 0.95 - ETA: 1s - loss: 0.1357 - accuracy: 0.95 - ETA: 1s - loss: 0.1357 - accuracy: 0.95 - ETA: 1s - loss: 0.1356 - accuracy: 0.95 - ETA: 1s - loss: 0.1370 - accuracy: 0.95 - ETA: 1s - loss: 0.1384 - accuracy: 0.95 - ETA: 1s - loss: 0.1382 - accuracy: 0.95 - ETA: 1s - loss: 0.1366 - accuracy: 0.95 - ETA: 1s - loss: 0.1355 - accuracy: 0.95 - ETA: 1s - loss: 0.1359 - accuracy: 0.95 - ETA: 0s - loss: 0.1350 - accuracy: 0.95 - ETA: 0s - loss: 0.1359 - accuracy: 0.95 - ETA: 0s - loss: 0.1350 - accuracy: 0.95 - ETA: 0s - loss: 0.1349 - accuracy: 0.95 - ETA: 0s - loss: 0.1353 - accuracy: 0.95 - ETA: 0s - loss: 0.1354 - accuracy: 0.95 - ETA: 0s - loss: 0.1356 - accuracy: 0.95 - ETA: 0s - loss: 0.1352 - accuracy: 0.95 - ETA: 0s - loss: 0.1348 - accuracy: 0.95 - ETA: 0s - loss: 0.1352 - accuracy: 0.95 - ETA: 0s - loss: 0.1353 - accuracy: 0.95 - ETA: 0s - loss: 0.1356 - accuracy: 0.95 - 4s 257us/sample - loss: 0.1354 - accuracy: 0.9592 - val_loss: 0.3147 - val_accuracy: 0.8747\n",
      "Epoch 28/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1404 - accuracy: 0.95 - ETA: 2s - loss: 0.1217 - accuracy: 0.96 - ETA: 2s - loss: 0.1247 - accuracy: 0.96 - ETA: 2s - loss: 0.1206 - accuracy: 0.96 - ETA: 2s - loss: 0.1227 - accuracy: 0.96 - ETA: 2s - loss: 0.1274 - accuracy: 0.96 - ETA: 2s - loss: 0.1296 - accuracy: 0.96 - ETA: 1s - loss: 0.1302 - accuracy: 0.96 - ETA: 1s - loss: 0.1309 - accuracy: 0.96 - ETA: 1s - loss: 0.1322 - accuracy: 0.96 - ETA: 1s - loss: 0.1312 - accuracy: 0.96 - ETA: 1s - loss: 0.1309 - accuracy: 0.96 - ETA: 1s - loss: 0.1299 - accuracy: 0.96 - ETA: 1s - loss: 0.1299 - accuracy: 0.96 - ETA: 1s - loss: 0.1309 - accuracy: 0.96 - ETA: 1s - loss: 0.1309 - accuracy: 0.96 - ETA: 1s - loss: 0.1308 - accuracy: 0.96 - ETA: 1s - loss: 0.1311 - accuracy: 0.96 - ETA: 0s - loss: 0.1309 - accuracy: 0.96 - ETA: 0s - loss: 0.1304 - accuracy: 0.96 - ETA: 0s - loss: 0.1305 - accuracy: 0.96 - ETA: 0s - loss: 0.1289 - accuracy: 0.96 - ETA: 0s - loss: 0.1298 - accuracy: 0.96 - ETA: 0s - loss: 0.1295 - accuracy: 0.96 - ETA: 0s - loss: 0.1287 - accuracy: 0.96 - ETA: 0s - loss: 0.1290 - accuracy: 0.96 - ETA: 0s - loss: 0.1288 - accuracy: 0.96 - ETA: 0s - loss: 0.1282 - accuracy: 0.96 - ETA: 0s - loss: 0.1281 - accuracy: 0.96 - 4s 261us/sample - loss: 0.1279 - accuracy: 0.9627 - val_loss: 0.3158 - val_accuracy: 0.8744\n",
      "Epoch 29/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1081 - accuracy: 0.97 - ETA: 2s - loss: 0.1112 - accuracy: 0.97 - ETA: 2s - loss: 0.1147 - accuracy: 0.96 - ETA: 2s - loss: 0.1242 - accuracy: 0.96 - ETA: 2s - loss: 0.1261 - accuracy: 0.96 - ETA: 2s - loss: 0.1263 - accuracy: 0.96 - ETA: 2s - loss: 0.1236 - accuracy: 0.96 - ETA: 1s - loss: 0.1236 - accuracy: 0.96 - ETA: 1s - loss: 0.1236 - accuracy: 0.96 - ETA: 1s - loss: 0.1243 - accuracy: 0.96 - ETA: 1s - loss: 0.1243 - accuracy: 0.96 - ETA: 1s - loss: 0.1257 - accuracy: 0.96 - ETA: 1s - loss: 0.1242 - accuracy: 0.96 - ETA: 1s - loss: 0.1248 - accuracy: 0.96 - ETA: 1s - loss: 0.1246 - accuracy: 0.96 - ETA: 1s - loss: 0.1238 - accuracy: 0.96 - ETA: 1s - loss: 0.1233 - accuracy: 0.96 - ETA: 0s - loss: 0.1228 - accuracy: 0.96 - ETA: 0s - loss: 0.1218 - accuracy: 0.96 - ETA: 0s - loss: 0.1211 - accuracy: 0.96 - ETA: 0s - loss: 0.1212 - accuracy: 0.96 - ETA: 0s - loss: 0.1204 - accuracy: 0.96 - ETA: 0s - loss: 0.1213 - accuracy: 0.96 - ETA: 0s - loss: 0.1218 - accuracy: 0.96 - ETA: 0s - loss: 0.1210 - accuracy: 0.96 - ETA: 0s - loss: 0.1204 - accuracy: 0.96 - ETA: 0s - loss: 0.1204 - accuracy: 0.96 - ETA: 0s - loss: 0.1203 - accuracy: 0.96 - ETA: 0s - loss: 0.1212 - accuracy: 0.96 - 4s 265us/sample - loss: 0.1211 - accuracy: 0.9655 - val_loss: 0.3186 - val_accuracy: 0.8730\n",
      "Epoch 30/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1003 - accuracy: 0.97 - ETA: 3s - loss: 0.1080 - accuracy: 0.96 - ETA: 2s - loss: 0.1099 - accuracy: 0.96 - ETA: 2s - loss: 0.1105 - accuracy: 0.96 - ETA: 2s - loss: 0.1162 - accuracy: 0.96 - ETA: 2s - loss: 0.1186 - accuracy: 0.96 - ETA: 2s - loss: 0.1191 - accuracy: 0.96 - ETA: 2s - loss: 0.1178 - accuracy: 0.96 - ETA: 1s - loss: 0.1150 - accuracy: 0.96 - ETA: 1s - loss: 0.1136 - accuracy: 0.96 - ETA: 1s - loss: 0.1124 - accuracy: 0.96 - ETA: 1s - loss: 0.1123 - accuracy: 0.96 - ETA: 1s - loss: 0.1118 - accuracy: 0.96 - ETA: 1s - loss: 0.1112 - accuracy: 0.96 - ETA: 1s - loss: 0.1112 - accuracy: 0.96 - ETA: 1s - loss: 0.1123 - accuracy: 0.96 - ETA: 1s - loss: 0.1117 - accuracy: 0.96 - ETA: 1s - loss: 0.1112 - accuracy: 0.96 - ETA: 0s - loss: 0.1117 - accuracy: 0.96 - ETA: 0s - loss: 0.1131 - accuracy: 0.96 - ETA: 0s - loss: 0.1132 - accuracy: 0.96 - ETA: 0s - loss: 0.1136 - accuracy: 0.96 - ETA: 0s - loss: 0.1139 - accuracy: 0.96 - ETA: 0s - loss: 0.1140 - accuracy: 0.96 - ETA: 0s - loss: 0.1145 - accuracy: 0.96 - ETA: 0s - loss: 0.1140 - accuracy: 0.96 - ETA: 0s - loss: 0.1145 - accuracy: 0.96 - ETA: 0s - loss: 0.1151 - accuracy: 0.96 - ETA: 0s - loss: 0.1146 - accuracy: 0.96 - 4s 270us/sample - loss: 0.1145 - accuracy: 0.9675 - val_loss: 0.3193 - val_accuracy: 0.8747\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 31/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1360 - accuracy: 0.96 - ETA: 2s - loss: 0.1346 - accuracy: 0.96 - ETA: 2s - loss: 0.1234 - accuracy: 0.96 - ETA: 2s - loss: 0.1206 - accuracy: 0.96 - ETA: 2s - loss: 0.1175 - accuracy: 0.96 - ETA: 2s - loss: 0.1175 - accuracy: 0.96 - ETA: 2s - loss: 0.1197 - accuracy: 0.96 - ETA: 1s - loss: 0.1187 - accuracy: 0.96 - ETA: 1s - loss: 0.1169 - accuracy: 0.96 - ETA: 1s - loss: 0.1162 - accuracy: 0.96 - ETA: 1s - loss: 0.1141 - accuracy: 0.96 - ETA: 1s - loss: 0.1131 - accuracy: 0.96 - ETA: 1s - loss: 0.1112 - accuracy: 0.96 - ETA: 1s - loss: 0.1111 - accuracy: 0.96 - ETA: 1s - loss: 0.1116 - accuracy: 0.96 - ETA: 1s - loss: 0.1117 - accuracy: 0.96 - ETA: 1s - loss: 0.1109 - accuracy: 0.96 - ETA: 1s - loss: 0.1108 - accuracy: 0.96 - ETA: 0s - loss: 0.1116 - accuracy: 0.96 - ETA: 0s - loss: 0.1110 - accuracy: 0.96 - ETA: 0s - loss: 0.1105 - accuracy: 0.96 - ETA: 0s - loss: 0.1101 - accuracy: 0.96 - ETA: 0s - loss: 0.1092 - accuracy: 0.96 - ETA: 0s - loss: 0.1093 - accuracy: 0.96 - ETA: 0s - loss: 0.1107 - accuracy: 0.96 - ETA: 0s - loss: 0.1107 - accuracy: 0.96 - ETA: 0s - loss: 0.1097 - accuracy: 0.96 - ETA: 0s - loss: 0.1092 - accuracy: 0.96 - ETA: 0s - loss: 0.1090 - accuracy: 0.96 - 4s 265us/sample - loss: 0.1085 - accuracy: 0.9698 - val_loss: 0.3223 - val_accuracy: 0.8739\n",
      "Epoch 32/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0822 - accuracy: 0.98 - ETA: 2s - loss: 0.0840 - accuracy: 0.98 - ETA: 2s - loss: 0.0940 - accuracy: 0.97 - ETA: 2s - loss: 0.0950 - accuracy: 0.97 - ETA: 2s - loss: 0.0964 - accuracy: 0.97 - ETA: 2s - loss: 0.0977 - accuracy: 0.97 - ETA: 2s - loss: 0.1000 - accuracy: 0.97 - ETA: 1s - loss: 0.1023 - accuracy: 0.97 - ETA: 1s - loss: 0.1040 - accuracy: 0.97 - ETA: 1s - loss: 0.1043 - accuracy: 0.96 - ETA: 1s - loss: 0.1036 - accuracy: 0.97 - ETA: 1s - loss: 0.1017 - accuracy: 0.97 - ETA: 1s - loss: 0.1033 - accuracy: 0.97 - ETA: 1s - loss: 0.1008 - accuracy: 0.97 - ETA: 1s - loss: 0.1005 - accuracy: 0.97 - ETA: 1s - loss: 0.1001 - accuracy: 0.97 - ETA: 1s - loss: 0.0992 - accuracy: 0.97 - ETA: 1s - loss: 0.0994 - accuracy: 0.97 - ETA: 0s - loss: 0.1003 - accuracy: 0.97 - ETA: 0s - loss: 0.1004 - accuracy: 0.97 - ETA: 0s - loss: 0.1006 - accuracy: 0.97 - ETA: 0s - loss: 0.1007 - accuracy: 0.97 - ETA: 0s - loss: 0.1016 - accuracy: 0.97 - ETA: 0s - loss: 0.1021 - accuracy: 0.97 - ETA: 0s - loss: 0.1024 - accuracy: 0.97 - ETA: 0s - loss: 0.1019 - accuracy: 0.97 - ETA: 0s - loss: 0.1023 - accuracy: 0.97 - ETA: 0s - loss: 0.1023 - accuracy: 0.97 - ETA: 0s - loss: 0.1023 - accuracy: 0.97 - 4s 261us/sample - loss: 0.1024 - accuracy: 0.9724 - val_loss: 0.3263 - val_accuracy: 0.8726\n",
      "Epoch 33/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0999 - accuracy: 0.98 - ETA: 2s - loss: 0.0942 - accuracy: 0.97 - ETA: 2s - loss: 0.0927 - accuracy: 0.98 - ETA: 2s - loss: 0.0940 - accuracy: 0.98 - ETA: 2s - loss: 0.0924 - accuracy: 0.98 - ETA: 2s - loss: 0.0930 - accuracy: 0.97 - ETA: 2s - loss: 0.0938 - accuracy: 0.97 - ETA: 1s - loss: 0.0955 - accuracy: 0.97 - ETA: 1s - loss: 0.0952 - accuracy: 0.97 - ETA: 1s - loss: 0.0958 - accuracy: 0.97 - ETA: 1s - loss: 0.0958 - accuracy: 0.97 - ETA: 1s - loss: 0.0983 - accuracy: 0.97 - ETA: 1s - loss: 0.0981 - accuracy: 0.97 - ETA: 1s - loss: 0.0979 - accuracy: 0.97 - ETA: 1s - loss: 0.0968 - accuracy: 0.97 - ETA: 1s - loss: 0.0977 - accuracy: 0.97 - ETA: 1s - loss: 0.0967 - accuracy: 0.97 - ETA: 1s - loss: 0.0972 - accuracy: 0.97 - ETA: 0s - loss: 0.0972 - accuracy: 0.97 - ETA: 0s - loss: 0.0968 - accuracy: 0.97 - ETA: 0s - loss: 0.0973 - accuracy: 0.97 - ETA: 0s - loss: 0.0973 - accuracy: 0.97 - ETA: 0s - loss: 0.0971 - accuracy: 0.97 - ETA: 0s - loss: 0.0969 - accuracy: 0.97 - ETA: 0s - loss: 0.0978 - accuracy: 0.97 - ETA: 0s - loss: 0.0972 - accuracy: 0.97 - ETA: 0s - loss: 0.0967 - accuracy: 0.97 - ETA: 0s - loss: 0.0971 - accuracy: 0.97 - ETA: 0s - loss: 0.0970 - accuracy: 0.97 - 4s 264us/sample - loss: 0.0968 - accuracy: 0.9759 - val_loss: 0.3289 - val_accuracy: 0.8726\n",
      "Epoch 34/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.1060 - accuracy: 0.98 - ETA: 2s - loss: 0.0995 - accuracy: 0.98 - ETA: 2s - loss: 0.0964 - accuracy: 0.97 - ETA: 2s - loss: 0.0914 - accuracy: 0.97 - ETA: 2s - loss: 0.0935 - accuracy: 0.97 - ETA: 2s - loss: 0.0914 - accuracy: 0.97 - ETA: 2s - loss: 0.0942 - accuracy: 0.97 - ETA: 2s - loss: 0.0923 - accuracy: 0.97 - ETA: 1s - loss: 0.0933 - accuracy: 0.97 - ETA: 1s - loss: 0.0915 - accuracy: 0.97 - ETA: 1s - loss: 0.0915 - accuracy: 0.97 - ETA: 1s - loss: 0.0897 - accuracy: 0.97 - ETA: 1s - loss: 0.0895 - accuracy: 0.97 - ETA: 1s - loss: 0.0900 - accuracy: 0.97 - ETA: 1s - loss: 0.0907 - accuracy: 0.97 - ETA: 1s - loss: 0.0907 - accuracy: 0.97 - ETA: 1s - loss: 0.0908 - accuracy: 0.97 - ETA: 1s - loss: 0.0909 - accuracy: 0.97 - ETA: 0s - loss: 0.0910 - accuracy: 0.97 - ETA: 0s - loss: 0.0908 - accuracy: 0.97 - ETA: 0s - loss: 0.0899 - accuracy: 0.97 - ETA: 0s - loss: 0.0902 - accuracy: 0.97 - ETA: 0s - loss: 0.0901 - accuracy: 0.97 - ETA: 0s - loss: 0.0907 - accuracy: 0.97 - ETA: 0s - loss: 0.0903 - accuracy: 0.97 - ETA: 0s - loss: 0.0910 - accuracy: 0.97 - ETA: 0s - loss: 0.0912 - accuracy: 0.97 - ETA: 0s - loss: 0.0916 - accuracy: 0.97 - ETA: 0s - loss: 0.0914 - accuracy: 0.97 - 4s 261us/sample - loss: 0.0915 - accuracy: 0.9771 - val_loss: 0.3333 - val_accuracy: 0.8735\n",
      "Epoch 35/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0843 - accuracy: 0.97 - ETA: 2s - loss: 0.0993 - accuracy: 0.97 - ETA: 2s - loss: 0.0994 - accuracy: 0.97 - ETA: 2s - loss: 0.1003 - accuracy: 0.97 - ETA: 2s - loss: 0.1003 - accuracy: 0.97 - ETA: 2s - loss: 0.0952 - accuracy: 0.97 - ETA: 2s - loss: 0.0956 - accuracy: 0.97 - ETA: 1s - loss: 0.0939 - accuracy: 0.97 - ETA: 1s - loss: 0.0927 - accuracy: 0.97 - ETA: 1s - loss: 0.0917 - accuracy: 0.97 - ETA: 1s - loss: 0.0919 - accuracy: 0.97 - ETA: 1s - loss: 0.0918 - accuracy: 0.97 - ETA: 1s - loss: 0.0910 - accuracy: 0.97 - ETA: 1s - loss: 0.0903 - accuracy: 0.97 - ETA: 1s - loss: 0.0900 - accuracy: 0.97 - ETA: 1s - loss: 0.0895 - accuracy: 0.97 - ETA: 1s - loss: 0.0887 - accuracy: 0.97 - ETA: 1s - loss: 0.0901 - accuracy: 0.97 - ETA: 0s - loss: 0.0898 - accuracy: 0.97 - ETA: 0s - loss: 0.0884 - accuracy: 0.97 - ETA: 0s - loss: 0.0882 - accuracy: 0.97 - ETA: 0s - loss: 0.0880 - accuracy: 0.97 - ETA: 0s - loss: 0.0878 - accuracy: 0.97 - ETA: 0s - loss: 0.0870 - accuracy: 0.97 - ETA: 0s - loss: 0.0865 - accuracy: 0.97 - ETA: 0s - loss: 0.0862 - accuracy: 0.97 - ETA: 0s - loss: 0.0861 - accuracy: 0.97 - ETA: 0s - loss: 0.0863 - accuracy: 0.97 - ETA: 0s - loss: 0.0865 - accuracy: 0.97 - 4s 246us/sample - loss: 0.0863 - accuracy: 0.9791 - val_loss: 0.3367 - val_accuracy: 0.8741\n",
      "Epoch 36/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0756 - accuracy: 0.98 - ETA: 2s - loss: 0.0861 - accuracy: 0.97 - ETA: 2s - loss: 0.0873 - accuracy: 0.97 - ETA: 2s - loss: 0.0812 - accuracy: 0.98 - ETA: 2s - loss: 0.0832 - accuracy: 0.98 - ETA: 1s - loss: 0.0835 - accuracy: 0.98 - ETA: 1s - loss: 0.0884 - accuracy: 0.97 - ETA: 1s - loss: 0.0875 - accuracy: 0.97 - ETA: 1s - loss: 0.0866 - accuracy: 0.97 - ETA: 1s - loss: 0.0860 - accuracy: 0.98 - ETA: 1s - loss: 0.0858 - accuracy: 0.98 - ETA: 1s - loss: 0.0841 - accuracy: 0.98 - ETA: 1s - loss: 0.0856 - accuracy: 0.98 - ETA: 1s - loss: 0.0846 - accuracy: 0.98 - ETA: 1s - loss: 0.0845 - accuracy: 0.98 - ETA: 1s - loss: 0.0839 - accuracy: 0.98 - ETA: 0s - loss: 0.0828 - accuracy: 0.98 - ETA: 0s - loss: 0.0817 - accuracy: 0.98 - ETA: 0s - loss: 0.0818 - accuracy: 0.98 - ETA: 0s - loss: 0.0817 - accuracy: 0.98 - ETA: 0s - loss: 0.0813 - accuracy: 0.98 - ETA: 0s - loss: 0.0818 - accuracy: 0.98 - ETA: 0s - loss: 0.0816 - accuracy: 0.98 - ETA: 0s - loss: 0.0811 - accuracy: 0.98 - ETA: 0s - loss: 0.0816 - accuracy: 0.98 - ETA: 0s - loss: 0.0819 - accuracy: 0.98 - ETA: 0s - loss: 0.0814 - accuracy: 0.98 - ETA: 0s - loss: 0.0816 - accuracy: 0.98 - ETA: 0s - loss: 0.0820 - accuracy: 0.98 - 4s 234us/sample - loss: 0.0818 - accuracy: 0.9810 - val_loss: 0.3419 - val_accuracy: 0.8732\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 37/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0741 - accuracy: 0.98 - ETA: 2s - loss: 0.0781 - accuracy: 0.98 - ETA: 2s - loss: 0.0811 - accuracy: 0.98 - ETA: 2s - loss: 0.0812 - accuracy: 0.98 - ETA: 2s - loss: 0.0789 - accuracy: 0.98 - ETA: 1s - loss: 0.0797 - accuracy: 0.98 - ETA: 1s - loss: 0.0767 - accuracy: 0.98 - ETA: 1s - loss: 0.0753 - accuracy: 0.98 - ETA: 1s - loss: 0.0756 - accuracy: 0.98 - ETA: 1s - loss: 0.0762 - accuracy: 0.98 - ETA: 1s - loss: 0.0777 - accuracy: 0.98 - ETA: 1s - loss: 0.0795 - accuracy: 0.98 - ETA: 1s - loss: 0.0788 - accuracy: 0.98 - ETA: 1s - loss: 0.0789 - accuracy: 0.98 - ETA: 1s - loss: 0.0780 - accuracy: 0.98 - ETA: 1s - loss: 0.0778 - accuracy: 0.98 - ETA: 0s - loss: 0.0771 - accuracy: 0.98 - ETA: 0s - loss: 0.0769 - accuracy: 0.98 - ETA: 0s - loss: 0.0767 - accuracy: 0.98 - ETA: 0s - loss: 0.0764 - accuracy: 0.98 - ETA: 0s - loss: 0.0764 - accuracy: 0.98 - ETA: 0s - loss: 0.0767 - accuracy: 0.98 - ETA: 0s - loss: 0.0766 - accuracy: 0.98 - ETA: 0s - loss: 0.0761 - accuracy: 0.98 - ETA: 0s - loss: 0.0759 - accuracy: 0.98 - ETA: 0s - loss: 0.0759 - accuracy: 0.98 - ETA: 0s - loss: 0.0766 - accuracy: 0.98 - ETA: 0s - loss: 0.0768 - accuracy: 0.98 - ETA: 0s - loss: 0.0770 - accuracy: 0.98 - 4s 242us/sample - loss: 0.0769 - accuracy: 0.9839 - val_loss: 0.3457 - val_accuracy: 0.8732\n",
      "Epoch 38/40\n",
      "15000/15000 [==============================] - ETA: 3s - loss: 0.0768 - accuracy: 0.99 - ETA: 2s - loss: 0.0721 - accuracy: 0.98 - ETA: 2s - loss: 0.0670 - accuracy: 0.98 - ETA: 2s - loss: 0.0680 - accuracy: 0.98 - ETA: 2s - loss: 0.0670 - accuracy: 0.98 - ETA: 1s - loss: 0.0677 - accuracy: 0.98 - ETA: 1s - loss: 0.0682 - accuracy: 0.98 - ETA: 1s - loss: 0.0676 - accuracy: 0.98 - ETA: 1s - loss: 0.0700 - accuracy: 0.98 - ETA: 1s - loss: 0.0695 - accuracy: 0.98 - ETA: 1s - loss: 0.0712 - accuracy: 0.98 - ETA: 1s - loss: 0.0714 - accuracy: 0.98 - ETA: 1s - loss: 0.0716 - accuracy: 0.98 - ETA: 1s - loss: 0.0712 - accuracy: 0.98 - ETA: 1s - loss: 0.0710 - accuracy: 0.98 - ETA: 1s - loss: 0.0717 - accuracy: 0.98 - ETA: 0s - loss: 0.0726 - accuracy: 0.98 - ETA: 0s - loss: 0.0739 - accuracy: 0.98 - ETA: 0s - loss: 0.0734 - accuracy: 0.98 - ETA: 0s - loss: 0.0727 - accuracy: 0.98 - ETA: 0s - loss: 0.0720 - accuracy: 0.98 - ETA: 0s - loss: 0.0720 - accuracy: 0.98 - ETA: 0s - loss: 0.0717 - accuracy: 0.98 - ETA: 0s - loss: 0.0718 - accuracy: 0.98 - ETA: 0s - loss: 0.0715 - accuracy: 0.98 - ETA: 0s - loss: 0.0723 - accuracy: 0.98 - ETA: 0s - loss: 0.0727 - accuracy: 0.98 - ETA: 0s - loss: 0.0724 - accuracy: 0.98 - ETA: 0s - loss: 0.0726 - accuracy: 0.98 - 3s 225us/sample - loss: 0.0725 - accuracy: 0.9849 - val_loss: 0.3533 - val_accuracy: 0.8719\n",
      "Epoch 39/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.0815 - accuracy: 0.98 - ETA: 2s - loss: 0.0748 - accuracy: 0.98 - ETA: 2s - loss: 0.0728 - accuracy: 0.98 - ETA: 1s - loss: 0.0708 - accuracy: 0.98 - ETA: 1s - loss: 0.0692 - accuracy: 0.98 - ETA: 1s - loss: 0.0668 - accuracy: 0.98 - ETA: 1s - loss: 0.0677 - accuracy: 0.98 - ETA: 1s - loss: 0.0670 - accuracy: 0.98 - ETA: 1s - loss: 0.0681 - accuracy: 0.98 - ETA: 1s - loss: 0.0666 - accuracy: 0.98 - ETA: 1s - loss: 0.0663 - accuracy: 0.98 - ETA: 1s - loss: 0.0677 - accuracy: 0.98 - ETA: 1s - loss: 0.0681 - accuracy: 0.98 - ETA: 1s - loss: 0.0682 - accuracy: 0.98 - ETA: 1s - loss: 0.0685 - accuracy: 0.98 - ETA: 0s - loss: 0.0679 - accuracy: 0.98 - ETA: 0s - loss: 0.0688 - accuracy: 0.98 - ETA: 0s - loss: 0.0681 - accuracy: 0.98 - ETA: 0s - loss: 0.0685 - accuracy: 0.98 - ETA: 0s - loss: 0.0678 - accuracy: 0.98 - ETA: 0s - loss: 0.0676 - accuracy: 0.98 - ETA: 0s - loss: 0.0673 - accuracy: 0.98 - ETA: 0s - loss: 0.0676 - accuracy: 0.98 - ETA: 0s - loss: 0.0691 - accuracy: 0.98 - ETA: 0s - loss: 0.0691 - accuracy: 0.98 - ETA: 0s - loss: 0.0686 - accuracy: 0.98 - ETA: 0s - loss: 0.0685 - accuracy: 0.98 - ETA: 0s - loss: 0.0681 - accuracy: 0.98 - ETA: 0s - loss: 0.0685 - accuracy: 0.98 - 3s 232us/sample - loss: 0.0684 - accuracy: 0.9861 - val_loss: 0.3570 - val_accuracy: 0.8716\n",
      "Epoch 40/40\n",
      "15000/15000 [==============================] - ETA: 2s - loss: 0.0760 - accuracy: 0.98 - ETA: 2s - loss: 0.0726 - accuracy: 0.98 - ETA: 2s - loss: 0.0692 - accuracy: 0.98 - ETA: 2s - loss: 0.0652 - accuracy: 0.98 - ETA: 2s - loss: 0.0667 - accuracy: 0.98 - ETA: 1s - loss: 0.0670 - accuracy: 0.98 - ETA: 1s - loss: 0.0689 - accuracy: 0.98 - ETA: 1s - loss: 0.0688 - accuracy: 0.98 - ETA: 1s - loss: 0.0671 - accuracy: 0.98 - ETA: 1s - loss: 0.0668 - accuracy: 0.98 - ETA: 1s - loss: 0.0661 - accuracy: 0.98 - ETA: 1s - loss: 0.0662 - accuracy: 0.98 - ETA: 1s - loss: 0.0673 - accuracy: 0.98 - ETA: 1s - loss: 0.0676 - accuracy: 0.98 - ETA: 1s - loss: 0.0675 - accuracy: 0.98 - ETA: 1s - loss: 0.0666 - accuracy: 0.98 - ETA: 1s - loss: 0.0658 - accuracy: 0.98 - ETA: 0s - loss: 0.0657 - accuracy: 0.98 - ETA: 0s - loss: 0.0667 - accuracy: 0.98 - ETA: 0s - loss: 0.0670 - accuracy: 0.98 - ETA: 0s - loss: 0.0670 - accuracy: 0.98 - ETA: 0s - loss: 0.0664 - accuracy: 0.98 - ETA: 0s - loss: 0.0660 - accuracy: 0.98 - ETA: 0s - loss: 0.0658 - accuracy: 0.98 - ETA: 0s - loss: 0.0652 - accuracy: 0.98 - ETA: 0s - loss: 0.0649 - accuracy: 0.98 - ETA: 0s - loss: 0.0651 - accuracy: 0.98 - ETA: 0s - loss: 0.0645 - accuracy: 0.98 - ETA: 0s - loss: 0.0647 - accuracy: 0.98 - 4s 246us/sample - loss: 0.0647 - accuracy: 0.9869 - val_loss: 0.3654 - val_accuracy: 0.8693\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(partial_x_train,\n",
    "                    partial_y_train,\n",
    "                    epochs=40,\n",
    "                    batch_size=512,\n",
    "                    validation_data=(x_val, y_val),\n",
    "                    verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " - 4s 141us/sample - loss: 0.3145 - accuracy: 0.8585\n",
      "[0.3835735505437851, 0.85852]\n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(test_data, test_labels)\n",
    "\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_dict = history.history\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZhU5Zn38e8NNCA7AirSQgNiFBABW8QRRY1xQCMuMQriuERFnRA1TjISNUZJuMaoowRlTDCjJooiY14jGpdoxBizKCCLAhIQWVpQFkVREGi43z+eU91FU1291emq7vp9rquuqrPUqbtOw7nrWc7zmLsjIiL5q0m2AxARkexSIhARyXNKBCIieU6JQEQkzykRiIjkOSUCEZE8p0QgGWVmTc3sCzPrnsl9s8nMDjWzjPezNrNTzWxV0vIyMzuhOvvW4rN+bWY31fb9aY77MzN7JNPHlfrVLNsBSHaZ2RdJi62AHcDuaPkqd59ek+O5+26gTab3zQfu/rVMHMfMrgAucveTko59RSaOLY2TEkGec/eyC3H0i/MKd3+lsv3NrJm7l9ZHbCJSP1Q1JGlFRf8nzewJM9sKXGRmx5nZP8xsi5mtN7MpZlYQ7d/MzNzMiqLlx6LtL5jZVjP7u5n1rOm+0faRZvZPM/vMzO4zs7+a2aWVxF2dGK8ysxVm9qmZTUl6b1Mzu9fMNpvZ+8CINOfnFjObUWHdVDO7J3p9hZktjb7P+9Gv9cqOVWJmJ0WvW5nZo1Fsi4GjU3zuyui4i81sVLT+SOB+4ISo2m1T0rm9Len9V0fffbOZ/d7Mulbn3FTFzM6O4tliZq+a2deStt1kZuvM7HMzey/puw41s7ej9R+b2V3V/TzJEHfXQw/cHWAVcGqFdT8DdgJnEn447AccAxxLKFH2Av4JjI/2bwY4UBQtPwZsAoqBAuBJ4LFa7HsAsBU4K9p2A7ALuLSS71KdGJ8B2gNFwCeJ7w6MBxYDhUAn4PXwXyXl5/QCvgBaJx17A1AcLZ8Z7WPAKcB2YEC07VRgVdKxSoCTotd3A68BHYEewJIK+54PdI3+JhdGMRwYbbsCeK1CnI8Bt0WvT4tiHAi0BP4HeLU65ybF9/8Z8Ej0+ogojlOiv9FN0XkvAPoBq4GDon17Ar2i13OAMdHrtsCx2f6/kG8PlQikOt5w92fdfY+7b3f3Oe7+pruXuvtKYBowPM37n3L3ue6+C5hOuADVdN9vAgvc/Zlo272EpJFSNWP8L3f/zN1XES66ic86H7jX3UvcfTNwR5rPWQm8S0hQAN8Atrj73Gj7s+6+0oNXgT8BKRuEKzgf+Jm7f+ruqwm/8pM/d6a7r4/+Jo8TknhxNY4LMBb4tbsvcPevgAnAcDMrTNqnsnOTzmhglru/Gv2N7gDaERJyKSHp9IuqFz+Izh2EhN7HzDq5+1Z3f7Oa30MyRIlAqmNt8oKZHW5mfzCzj8zsc2Ai0DnN+z9Ker2N9A3Ele17cHIc7u6EX9ApVTPGan0W4ZdsOo8DY6LXFxISWCKOb5rZm2b2iZltIfwaT3euErqmi8HMLjWzhVEVzBbg8GoeF8L3Kzueu38OfAp0S9qnJn+zyo67h/A36ubuy4D/IPwdNkRVjQdFu14G9AWWmdlbZnZ6Nb+HZIgSgVRHxa6TvyL8Cj7U3dsBtxKqPuK0nlBVA4CZGXtfuCqqS4zrgUOSlqvq3vokcGr0i/osQmLAzPYDngL+i1Bt0wH4YzXj+KiyGMysF/AAcA3QKTrue0nHraqr6zpCdVPieG0JVVAfViOumhy3CeFv9iGAuz/m7scTqoWaEs4L7r7M3UcTqv/+G/idmbWsYyxSA0oEUhttgc+AL83sCOCqevjM54DBZnammTUDrgO6xBTjTOB6M+tmZp2AG9Pt7O4fA28ADwPL3H15tKkF0BzYCOw2s28CX69BDDeZWQcL91mMT9rWhnCx30jIiVcQSgQJHwOFicbxFJ4ALjezAWbWgnBB/ou7V1rCqkHMo8zspOizf0ho13nTzI4ws5Ojz9sePXYTvsC/mVnnqATxWfTd9tQxFqkBJQKpjf8ALiH8J/8V4RdxrKKL7QXAPcBmoDcwn3DfQ6ZjfIBQl/8OoSHzqWq853FC4+/jSTFvAb4PPE1ocD2PkNCq4yeEkskq4AXgt0nHXQRMAd6K9jkcSK5XfxlYDnxsZslVPIn3v0ioonk6en93QrtBnbj7YsI5f4CQpEYAo6L2ghbAnYR2nY8IJZBboreeDiy10CvtbuACd99Z13ik+ixUtYo0LGbWlFAVcZ67/yXb8Yg0ZCoRSINhZiPMrH1UvfBjQk+Ut7IclkiDp0QgDckwYCWhemEEcLa7V1Y1JCLVpKohEZE8pxKBiEiea3CDznXu3NmLioqyHYaISIMyb968Te6esst1g0sERUVFzJ07N9thiIg0KGZW6R3yqhoSEclzSgQiInlOiUBEJM81uDYCEalfu3btoqSkhK+++irboUg1tGzZksLCQgoKKhtqal9KBCKSVklJCW3btqWoqIgw6KvkKndn8+bNlJSU0LNnz6rfEMmLqqHp06GoCJo0Cc/TazQdu0h+++qrr+jUqZOSQANgZnTq1KnGpbdGXyKYPh3GjYNt28Ly6tVhGWBsncdbFMkPSgINR23+Vo2+RHDzzeVJIGHbtrBeRETyIBGsWVOz9SKSWzZv3szAgQMZOHAgBx10EN26dStb3rmzetMWXHbZZSxbtiztPlOnTmV6huqNhw0bxoIFCzJyrPrQ6KuGuncP1UGp1otI5k2fHkrca9aE/2eTJtWtGrZTp05lF9XbbruNNm3a8IMf/GCvfdwdd6dJk9S/bR9++OEqP+e73/1u7YNs4Bp9iWDSJGjVau91rVqF9SKSWYk2udWrwb28TS6ODhorVqygf//+XH311QwePJj169czbtw4iouL6devHxMnTizbN/ELvbS0lA4dOjBhwgSOOuoojjvuODZs2ADALbfcwuTJk8v2nzBhAkOGDOFrX/saf/vb3wD48ssv+da3vsVRRx3FmDFjKC4urvKX/2OPPcaRRx5J//79uemmmwAoLS3l3/7t38rWT5kyBYB7772Xvn37ctRRR3HRRRdl/JxVptEngrFjYdo06NEDzMLztGlqKBaJQ323yS1ZsoTLL7+c+fPn061bN+644w7mzp3LwoULefnll1myZMk+7/nss88YPnw4Cxcu5LjjjuOhhx5KeWx356233uKuu+4qSyr33XcfBx10EAsXLmTChAnMnz8/bXwlJSXccsstzJ49m/nz5/PXv/6V5557jnnz5rFp0ybeeecd3n33XS6++GIA7rzzThYsWMDChQu5//7763h2qq/RJwIIF/1Vq2DPnvCsJCASj/puk+vduzfHHHNM2fITTzzB4MGDGTx4MEuXLk2ZCPbbbz9GjhwJwNFHH82qVatSHvvcc8/dZ5833niD0aNHA3DUUUfRr1+/tPG9+eabnHLKKXTu3JmCggIuvPBCXn/9dQ499FCWLVvGddddx0svvUT79u0B6NevHxdddBHTp0+v0Q1hdZUXiUBE6kdlbW9xtcm1bt267PXy5cv5xS9+wauvvsqiRYsYMWJEyv70zZs3L3vdtGlTSktLUx67RYsW++xT04m8Ktu/U6dOLFq0iGHDhjFlyhSuuuoqAF566SWuvvpq3nrrLYqLi9m9e3eNPq+2lAhEJGOy2Sb3+eef07ZtW9q1a8f69et56aWXMv4Zw4YNY+bMmQC88847KUscyYYOHcrs2bPZvHkzpaWlzJgxg+HDh7Nx40bcnW9/+9vcfvvtvP322+zevZuSkhJOOeUU7rrrLjZu3Mi2ivVsMWn0vYZEpP4kql0z2WuougYPHkzfvn3p378/vXr14vjjj8/4Z3zve9/j4osvZsCAAQwePJj+/fuXVeukUlhYyMSJEznppJNwd84880zOOOMM3n77bS6//HLcHTPj5z//OaWlpVx44YVs3bqVPXv2cOONN9K2bduMf4dUGtycxcXFxa6JaUTqz9KlSzniiCOyHUZOKC0tpbS0lJYtW7J8+XJOO+00li9fTrNmufWbOtXfzMzmuXtxqv1zK3oRkRz2xRdf8PWvf53S0lLcnV/96lc5lwRqo+F/AxGRetKhQwfmzZuX7TAyTo3FIiJ5LtZEYGYjzGyZma0wswkptnc3s9lmNt/MFpnZ6XHGIyIi+4otEZhZU2AqMBLoC4wxs74VdrsFmOnug4DRwP/EFY+IiKQWZ4lgCLDC3Ve6+05gBnBWhX0caBe9bg+sizEeERFJIc5E0A1Ym7RcEq1LdhtwkZmVAM8D30t1IDMbZ2ZzzWzuxo0b44hVRHLUSSedtM/NYZMnT+bf//3f076vTZs2AKxbt47zzjuv0mNX1R198uTJe93Ydfrpp7Nly5bqhJ7Wbbfdxt13313n42RCnIkg1TQ5FW9aGAM84u6FwOnAo2a2T0zuPs3di929uEuXLjGEKiK5asyYMcyYMWOvdTNmzGDMmDHVev/BBx/MU089VevPr5gInn/+eTp06FDr4+WiOBNBCXBI0nIh+1b9XA7MBHD3vwMtgc4xxiQiDcx5553Hc889x44dOwBYtWoV69atY9iwYWX9+gcPHsyRRx7JM888s8/7V61aRf/+/QHYvn07o0ePZsCAAVxwwQVs3769bL9rrrmmbAjrn/zkJwBMmTKFdevWcfLJJ3PyyScDUFRUxKZNmwC455576N+/P/379y8bwnrVqlUcccQRXHnllfTr14/TTjttr89JZcGCBQwdOpQBAwZwzjnn8Omnn5Z9ft++fRkwYEDZYHd//vOfyybmGTRoEFu3bq31uU2I8z6COUAfM+sJfEhoDL6wwj5rgK8Dj5jZEYREoLofkRx1/fWQ6Ym3Bg6E6BqaUqdOnRgyZAgvvvgiZ511FjNmzOCCCy7AzGjZsiVPP/007dq1Y9OmTQwdOpRRo0ZVOm/vAw88QKtWrVi0aBGLFi1i8ODBZdsmTZrE/vvvz+7du/n617/OokWLuPbaa7nnnnuYPXs2nTvv/Rt13rx5PPzww7z55pu4O8ceeyzDhw+nY8eOLF++nCeeeIIHH3yQ888/n9/97ndp5xe4+OKLue+++xg+fDi33nort99+O5MnT+aOO+7ggw8+oEWLFmXVUXfffTdTp07l+OOP54svvqBly5Y1ONupxVYicPdSYDzwErCU0DtosZlNNLNR0W7/AVxpZguBJ4BLvaGNeSEisUuuHkquFnJ3brrpJgYMGMCpp57Khx9+yMcff1zpcV5//fWyC/KAAQMYMGBA2baZM2cyePBgBg0axOLFi6scUO6NN97gnHPOoXXr1rRp04Zzzz2Xv/zlLwD07NmTgQMHAumHuoYwP8KWLVsYPnw4AJdccgmvv/56WYxjx47lscceK7uD+fjjj+eGG25gypQpbNmyJSN3Nsd6Z7G7P09oBE5ed2vS6yVA5keGEpFYpPvlHqezzz6bG264gbfffpvt27eX/ZKfPn06GzduZN68eRQUFFBUVJRy6OlkqUoLH3zwAXfffTdz5syhY8eOXHrppVUeJ91v1sQQ1hCGsa6qaqgyf/jDH3j99deZNWsWP/3pT1m8eDETJkzgjDPO4Pnnn2fo0KG88sorHH744bU6foLuLBaRnNemTRtOOukkvvOd7+zVSPzZZ59xwAEHUFBQwOzZs1mdaoLyJCeeeGLZBPXvvvsuixYtAsIQ1q1bt6Z9+/Z8/PHHvPDCC2Xvadu2bcp6+BNPPJHf//73bNu2jS+//JKnn36aE044ocbfrX379nTs2LGsNPHoo48yfPhw9uzZw9q1azn55JO588472bJlC1988QXvv/8+Rx55JDfeeCPFxcW89957Nf7MijTWkIg0CGPGjOHcc8/dqwfR2LFjOfPMMykuLmbgwIFV/jK+5ppruOyyyxgwYAADBw5kyJAhQJhtbNCgQfTr12+fIazHjRvHyJEj6dq1K7Nnzy5bP3jwYC699NKyY1xxxRUMGjQobTVQZX7zm99w9dVXs23bNnr16sXDDz/M7t27ueiii/jss89wd77//e/ToUMHfvzjHzN79myaNm1K3759y2ZbqwsNQy0iaWkY6oanpsNQq2pIRCTPKRGIiOQ5JQIRqVJDq0LOZ7X5WykRiEhaLVu2ZPPmzUoGDYC7s3nz5hrfZKZeQyKSVmFhISUlJWjAx4ahZcuWFBYW1ug9SgQiklZBQQE9e/bMdhgSI1UNiYjkOSUCEZE8p0QgIpLnlAhERPKcEoGISJ5TIhARyXNKBCIieS7WRGBmI8xsmZmtMLMJKbbfa2YLosc/zWxLnPGIiMi+YksEZtYUmAqMBPoCY8ysb/I+7v59dx/o7gOB+4D/F1c8ixbBT38a19FFRBquOEsEQ4AV7r7S3XcCM4Cz0uw/hjBvcSxmz4Zbb4U5c+L6BBGRhinORNANWJu0XBKt24eZ9QB6Aq9Wsn2cmc01s7m1He/kssugbVv4xS9q9XYRkUYrzkSw7wzRUNnwhaOBp9x9d6qN7j7N3YvdvbhLly61CqZdO/jOd2DmTFi3rlaHEBFplOJMBCXAIUnLhUBll+DRxFgtlPC970FpKTzwQNyfJCLScMSZCOYAfcysp5k1J1zsZ1Xcycy+BnQE/h5jLAD07g1nngm//CV89VXcnyYi0jDElgjcvRQYD7wELAVmuvtiM5toZqOSdh0DzPB6mvXi+uth0yZ4/PH6+DQRkdxnDW3WoeLiYp87d26t3+8OAweG54ULwVK1ZIiINDJmNs/di1Nty7s7i83guuvgnXfgtdeyHY2ISPblXSIAuPBC6NwZJk8Oy9OnQ1ERNGkSnqdPz2Z0IiL1Ky8TQcuWcPXV8OyzcM89MG4crF4dqotWrw7LSgYiki/yro0gYd268Ou/ZUvYunXf7T16wKpVdf4YEZGcoDaCFA4+GC64IHUSAFizpn7jERHJlrxNBBAajSvTvXv9xSEikk15nQiKi+Gww/btQtqqFUyalJ2YRETqW14nAggXfHfo0iUkhB49YNo0GDs225GJiNSPZtkOINvOPjtUA/XuDa+mHPtURKRxy/sSQbNmMH58mK9g4cJsRyMiUv/yPhEAXHFFaBdI3GAmIpJPlAiAjh3hyivht7/VDGYikn+UCCK33w4HHRRKB7t2ZTsaEZH6o0QQad8e/ud/wiT3d96Z7WhEROqPEkGSs86Cb38bJk6E997LdjQiIvVDiaCC++6D1q1Dm8GePdmORkQkfrEmAjMbYWbLzGyFmU2oZJ/zzWyJmS02s6zPG3bggWFE0jfeCFNaiog0drElAjNrCkwFRgJ9gTFm1rfCPn2AHwHHu3s/4Pq44qmJSy6Bb3wDbrwR1q7VfAUi0rjFeWfxEGCFu68EMLMZwFnAkqR9rgSmuvunAO6+IcZ4qs0MfvUr6N8fRo2CZctg+/awLTFfAWgYChFpHOKsGuoGrE1aLonWJTsMOMzM/mpm/zCzEakOZGbjzGyumc3duHFjTOHurWdP+NnPYMGC8iSQsG0b3HxzvYQhIhK7OBNBqmnhK86C0wzoA5wEjAF+bWYd9nmT+zR3L3b34i5dumQ80Mpce23l2zRfgYg0FnEmghLgkKTlQmBdin2ecfdd7v4BsIyQGHJC06bQtWvqbZqvQEQaizgTwRygj5n1NLPmwGhgVoV9fg+cDGBmnQlVRStjjKnG7rorDEyXTPMViEhjElsicPdSYDzwErAUmOnui81sopmNinZ7CdhsZkuA2cAP3X1zXDHVxtix8OCDUFAQlg8+WPMViEjjkreT19fUypUwdCi0aQN//3u430BEpKHQ5PUZ0KsXPPssfPRR6FK6bVu2IxIRyQwlgho49lh44okwVPWFF8Lu3dmOSESk7pQIauiss2DKFHjmGbj++jDfsYhIQ6ZEUAvjx8MNN8D998O992oIChFp2PJ+8vrauuuuMNzED34QehTt3BnWawgKEWloVCKopSZN4NFHoXnz8iSQoCEoRKQhUSKog/32gx07Um/TEBQi0lAoEdRRjx6p12sIChFpKJQI6mjSpDDkRLIWLTQEhYg0HEoEdTR2bBhyIlEySIxLVNlgdSIiuUaJIAPGjoVVq8I9BWvWQJ8+cPrp4U5kEZFcp0SQYV27wmuvwYABcM454U5kEZFcpkQQg06d4JVXYNiw8qojEZFcpUQQk3bt4IUXYORIuOoq6NhRdx6LSG5SIojRfvvB+eeHmc62bAltCIk7j5UMRCRXxJoIzGyEmS0zsxVmNiHF9kvNbKOZLYgeV8QZTzb85Cf7jlKqO49FJJfENtaQmTUFpgLfIMxNPMfMZrn7kgq7Punu4+OKI9squ8N49er6jUNEpDJxlgiGACvcfaW77wRmAGfF+Hk5qbI7jJs0gdmz6zcWEZFU4kwE3YC1Scsl0bqKvmVmi8zsKTM7JMZ4siLVncctW8JBB8Fpp6lHkYhkX5yJwFKsqziNy7NAkbsPAF4BfpPyQGbjzGyumc3duHFjhsOMV/Kdx2bh+de/hiVL4NRTQ4+i666D0tJsRyoi+apak9ebWW+gxN13mNlJwADgt+6+Jc17jgNuc/d/jZZ/BODu/1XJ/k2BT9y9fbpYsjV5fRxKS+GHP4TJk0Pp4PHHwz0IIiKZlonJ638H7DazQ4H/BXoCj1fxnjlAHzPraWbNgdHArAqBJY/IMwpYWs14GoVmzaC4GPbfH/74RzjwQLjttmxHJSL5prqJYI+7lwLnAJPd/ftA2mHVov3HAy8RLvAz3X2xmU00s1HRbtea2WIzWwhcC1xamy/RUE2fHu4p+OSTsLx7N9x+e6hO0lzIIlJfqls19CYwGbgZONPdPzCzd929f9wBVtSYqoaKiirvRnrmmfDII6G0ICJSV5moGroMOA6YFCWBnsBjmQowX6WbxezFF2HwYHjzzfqLR0TyU7USgbsvcfdr3f0JM+sItHX3O2KOrdGr7B6DHj3gjTfC6xNOCI3JqioSkbhUKxGY2Wtm1s7M9gcWAg+b2T3xhtb4pbrHoFWrsH7IEJg/P8xr8P3vw9lnQwPrOSsiDUR1q4bau/vnwLnAw+5+NHBqfGHlh1T3GEybFtZDGLH06afhnntCVVH//prsRkQyr7qJoFnU1fN84LkY48k7idnN9uwJz4kkkPD44/CLX8DOnfDppzBqFFxxBWzdmo1oRaQxqm4imEjoBvq+u88xs17A8vjCEijvXproWbRrV7j34KGH4KijytsRRETqorqNxf/n7gPc/ZpoeaW7fyve0OTmm8OQ1clKS+GAA0JV0oknwo03wo4d2YlPRBqH6jYWF5rZ02a2wcw+NrPfmVlh3MHlu8q6l27YAAsWhCqiO+8MDcsLF9ZvbCLSeFS3auhhwvAQBxNGEH02Wicxqqx7affu0LZtaFh+9ln4+OMwVMWECfuWIEREqlLdRNDF3R9299Lo8QjQJca4hPTdSxO++U1YvBguvhh+/nM48sgwbpGISHVVNxFsMrOLzKxp9LgI2BxnYFJ199KETp3gf/83THTTrBn867/CRReFKiQRkapUNxF8h9B19CNgPXAeYdgJiVlV3UunTw9jFjVpApdeGqqHbr0VZs6EI44IPYx0V7KIpFPdXkNr3H2Uu3dx9wPc/WzCzWWSRcndS93D8/jxcNhhoTG5b1+4/HI4+WRYmlcDfItITdRlhrIbMhaF1Eqq7qXbtoX1ffvCn/8cqpIWLgxtB9dfD1sqnUpIRPJVXRJBqqkopR5V1r00sb5JE7jySvjnP0NX0ylToE+fkBx2766/OEUkt9UlEajmOcvSdS9N1qUL/PKX8PbboaRw1VWhu+nrr8cfo4jkvrSJwMy2mtnnKR5bCfcUpGVmI8xsmZmtMLMJafY7z8zczFJOmiCpVad7abKBA+G11+DJJ2HzZhg+HC64oPLJcUQkP6RNBO7e1t3bpXi0dfdm6d4bTUY/FRgJ9AXGmFnfFPu1JUxTqSlYaqi63UuTmcH558N774X5kZ99Fg4/HP7zP2HTpnoLXURySF2qhqoyBFgRjUu0E5gBnJViv58CdwJfxRhLo1WT7qVFRWEZQsnhJz8JCeHb34a774aePUPXUzUoi+SXOBNBN2Bt0nJJtK6MmQ0CDnF3DW0dg1TdS8eNK08GENoTfvtbePddGDkSfvrTkBAmTdJQ1yL5Is5EkKpXUVkDs5k1Ae4F/qPKA5mNM7O5ZjZ3o6bpqrZ03Usr6ts33IQ2f36YHvOWW6BXL/jv/4bt2+snXhHJjjgTQQlwSNJyIbAuabkt0B94zcxWAUOBWakajN19mrsXu3txly4a4qi6qupemsrAgTBrFvzjHzBoEPzgByEh3HFHmBhHRBqfOBPBHKCPmfU0s+bAaMIIpgC4+2fu3tndi9y9CPgHMMrd58YYU16pbvfSVI49Ngxe9+c/hykyf/QjOOSQMH+yehmJNC6xJQJ3LwXGE2Y2WwrMdPfFZjbRzEbF9blSrqbdS1M58UR4+eVQZXTOOXD//dC7N4wZE+5LEJGGL84SAe7+vLsf5u693X1StO5Wd5+VYt+TVBrIrOp0L62sV1FFAwfCo4/CypWhVPCHP8DRR8Mpp8Bzz+lOZZGGzLyBDU1ZXFzsc+cqX2RColdRcoNyq1ZV34sA8Nln8OCDMHkyfPghFBbCd74THj16xBu3iNScmc1z95Q37SoR5LGiotT1/T16hHsSqmPXrtC4/OCD5RPinHZaGONo1CgoKMhUtCJSF+kSQaxVQ5LbatOrqKKCAvjWt+DFF0O10Y9/HGZMO++8UEq48cZwj0ID+70hkjM2bQpdu8eNg7/9LZ7PUCLIY3XpVZRKURHcfnsoTTz3HBx3XLgP4cgjQxfUa68NpYYdO2obsUjjt3176KBx442hHe6AA8KYYE8+CcuXx/OZqhrKY3VpI6iu9evDeEbPPguvvAJffQVt2oTqozPPhNNPD//QRfLNjh2wdm2onl29OvyA+tvf4I03wraCAviXf4FTTw2P4uIwFW1tqY1AKjV9erjTeM2aUBKYNKk8CaTbVhvbtsGrr4bSwnPPhUZmMxg6NLQnnHlmuMPZNNOFNCLbt8Nbb4Vh3999tzvZDHMAABD/SURBVPzC/9FHe+9nFu7Z+cY3woX/hBPCj6ZMUSKQGou7tOAeptNMlBYSf9JevUJCGDUq/EdQY7M0NFu3hl/2r78eHm+9BTt3hgt9796hCrV799Apo0eP8teFhdC8eXxxKRFIjWWiR1FNrFsXSgmzZsGf/hSqkNq3DwPhffObMGIEdOqU+c8VqY3t20NJOfFYvTo8L14cbrTcsweaNg3VOSeeGB7HHw8dO2YvZiUCqbEmTVL39DEL/8jj9OWXoT1h1qyQHDZsCPEMHRqSwhlnhAZoVSFJnPbsgQ8+CBf3JUvC83vvhYt+xbEvmzSBgw+GQw+FYcPCpE9Dh2a2aqeulAikxuq7RFCZPXtCtdEf/hCSQmJYi0MOCQlh5MjwH04NzlIbX34JJSWhverDD0Pj7dKl5Rf95JF3CwvhiCPCMO2JKp1Etc7BB+d+NaYSgdRYffQoqo116+CFF0JSePnl8B8ZQmI45phQFD/mmNDtLpvFcMkNW7fC++/DihXlz2vWhIt+SUm4Q76iwkLo1y90XOjXr/x1u3b1H38mKRFIrVTVayjTvYpqascOePPNUGKYMyc8r1hRvv3QQ2Hw4PL/zP36hXV16YInuWHXrnCj1caNoepw48byx9q14d/BihXw8cd7v++AA8Iv+G7dwgU/+TnxaN06O98pbkoEknG5WmL49FOYN688MSxYEOp5E//MCwrga18rTwxHHBGSQ+/e0LZt9uKWvX3+efi7rVoVnpNfl5RUPjdGkybQtWv4m1Z85PvfWIlAMi5X2hCq48svQ33v4sV7PyrGeeCB+144uncP9b9du0LLllkJv0FyDyPS7twZfr3v2lX+eseO8Ct+/fpQ1bd+/d6Pdevgk0/2Pl7r1qFuvmfPUA144IHQpUv544ADwnPHjqG3juxLiUAyLpu9ijLliy/KqxASj0Q9cknJvvvvv39IColH166hS+v++4dHx457P++3X/1/p0wpLQ0Npdu3h1Jf4nViecuWUO2SeGzYsPfrRNtNdRQUwEEHhfPZtWs4t0VF4ZG4+HfqpF5idZUuEai2VGqle/fUJYLajlOUDW3ahHkWBg7cd9v27WEQvZKS8Au14mPJkvDrNd08DC1aQIcO4X6IDh32ft2+faimaNEi3ESU6nn37nBB3bYtPCc/tm0LF8bmzfd+X+J1QUH45Z3qIp5uObGutLR657BJE+jcOfxCP/DAcEPgAQeE71ZQEOKp+Ny8efj1nrjw779/OI5kT6yJwMxGAL8AmgK/dvc7Kmy/GvgusBv4Ahjn7kvijEkyY9Kk1G0EybOfZbsxuS7226+8HaEye/aEXimffBLqrFM9f/ZZ+PWceF6zJjxv2RJumqupFi1CNUli5rmdO8MFf+fO8KiYmMzCd0l+tGpV/vqgg/ZdV3G/5G2J1+3bhwt/p06qimkMYksEZtYUmAp8gzCR/Rwzm1XhQv+4u/8y2n8UcA8wIq6YJHMSF/R04xQlJ4rVq8Ny8nsbuiZNwgWxfftQfVFTpaXlF/DExTz5uWnTcOFt3br84l9Vj6fkevlE6UBVKlKV2NoIzOw44DZ3/9do+UcA7v5flew/BrjY3UemO67aCBqGhtSYLJIPstVG0A1Ym7RcAhxbcScz+y5wA9AcOCXVgcxsHDAOoHtDqoTOY5mY9EZE6kecTTSpCqT7FD/cfaq79wZuBG5JdSB3n+buxe5e3KVLlwyHKXGozqQ306eHkkOTJuF5+vT6iExEKoozEZQAhyQtFwLr0uw/Azg7xnikHk2aVN6gmZDcmJxoQ1i9OnRDTbQhKBmI1L84E8EcoI+Z9TSz5sBoYFbyDmbWJ2nxDCCmidikvo0dG+4y7tEjNFb26LH3Xcc337x3jyMIyzffXP+xiuS72NoI3L3UzMYDLxG6jz7k7ovNbCIw191nAePN7FRgF/ApcElc8Uj9Gzu28h5CakMQyR2x3sbh7s+7+2Hu3tvdJ0Xrbo2SAO5+nbv3c/eB7n6yuy+OMx7JHWpDEMkdup9PskJtCCK5Q4lAskJtCCK5Q4POSU5qDIPaieSSdDeUqUQgOamqNgS1H4hkjhKB5KR0bQhqPxDJLCUCyUnp2hDUfiCSWWojkAZH7QciNac2AmlUdA+CSGYpEUiDo3sQRDJLiUAaHN2DIJJZaiOQRkdtCCL7UhuB5BW1IYjUjBKBNDpqQxCpGSUCaXTUhiBSM0oE0iiNHQurVoU2gVWr9p4XoTpzIajqSPKJEoHkneqMY6SqI8knsSYCMxthZsvMbIWZTUix/QYzW2Jmi8zsT2bWI854RKDqNoTqVB2pxCCNSWyJwMyaAlOBkUBfYIyZ9a2w23yg2N0HAE8Bd8YVj0hCVW0IVVUdqcQgjU2cJYIhwAp3X+nuO4EZwFnJO7j7bHdP/Pb6B1AYYzwiZdK1IVRVdaTGZmls4kwE3YC1Scsl0brKXA68kGqDmY0zs7lmNnfjxo0ZDFFkX1VVHVWnxKBqI2lI4kwElmJdytuYzewioBi4K9V2d5/m7sXuXtylS5cMhiiyr6qqjtKVGFRtJA1RnImgBDgkabkQWFdxJzM7FbgZGOXuO2KMR6Ta0lUdpSsxqNpIGqI4E8EcoI+Z9TSz5sBoYFbyDmY2CPgVIQlsiDEWkYxJV2LQPQrSEDWL68DuXmpm44GXgKbAQ+6+2MwmAnPdfRahKqgN8H9mBrDG3UfFFZNIpowdu3cpIaF791AdlGo9lFcdJUoNiaqjxDFFsiHW+wjc/Xl3P8zde7v7pGjdrVESwN1PdfcD3X1g9FASkAZN9yhIQ6Q7i0UySPcoSEOkRCCSYXHfo6ASg2SaEoFIPcrEPQoqMUimKRGI1KO63KMAKjFIPJQIROpZbe9RAJUYJB5KBCI5JO4Sg0oLkooSgUiOiavEoNKCVEaJQKQBqUuJQe0LUhklApEGprYlBrUvSGWUCEQakXQlBvVIksooEYg0MpWVGNQjSSqjRCCSJ3QPg1RGiUAkj+geBklFiUBEAJUY8pkSgYiUyWaJQUkie2KbmEZEGpdEUrj55nBx7949JIHkEkO6SXmqKjFowp7sibVEYGYjzGyZma0wswkptp9oZm+bWamZnRdnLCJSd3GVGFStlF2xJQIzawpMBUYCfYExZta3wm5rgEuBx+OKQ0TqR13aGDLREK1EUXtxlgiGACvcfaW77wRmAGcl7+Duq9x9EbAnxjhEpJ7UtsSQicH0lChqL85E0A1Ym7RcEq2rMTMbZ2ZzzWzuxo0bMxKciNSvdCWGujZEZyJR5LM4E4GlWOe1OZC7T3P3Yncv7tKlSx3DEpFsqazEUNeuq3VNFJDfJYY4E0EJcEjSciGwLsbPE5EGrC4N0XVNFPnetTXORDAH6GNmPc2sOTAamBXj54lII1VViaGuiSJdiSEv2h/cPbYHcDrwT+B94OZo3URgVPT6GELJ4UtgM7C4qmMeffTRLiJS0WOPuffo4W4Wnh97bO9trVq5h0t5eLRqVb6P2d7bEo/EsVJt69GjeseuKrb6Asz1yq7VlW3I1YcSgYjURrqLcbqLfbokUdV7E5+bC4lCiUBEJI10F+uqLvQNJVGkSwQaa0hE8l5durbG3aOpPrq+KhGIiFD7rq3ZThSZoEQgIlKFdF1bs50oMkGJQESkjrKZKDJBiUBEJGZxJopM0HwEIiJZNnZs5fMuVDUPRCYoEYiI5Lh0iSITVDUkIpLnlAhERPKcEoGISJ5TIhARyXNKBCIiec7CWEQNh5ltBFZXsrkzsKkew6mpXI5PsdWOYqsdxVY7dYmth7unnOKxwSWCdMxsrrsXZzuOyuRyfIqtdhRb7Si22okrNlUNiYjkOSUCEZE819gSwbRsB1CFXI5PsdWOYqsdxVY7scTWqNoIRESk5hpbiUBERGpIiUBEJM81mkRgZiPMbJmZrTCzCdmOJ5mZrTKzd8xsgZnNzXIsD5nZBjN7N2nd/mb2spktj5475lBst5nZh9G5W2Bmp2cptkPMbLaZLTWzxWZ2XbQ+6+cuTWxZP3dm1tLM3jKzhVFst0fre5rZm9F5e9LMmudQbI+Y2QdJ521gfceWFGNTM5tvZs9Fy/Gct8pmtW9ID6Ap8D7QC2gOLAT6ZjuupPhWAZ2zHUcUy4nAYODdpHV3AhOi1xOAn+dQbLcBP8iB89YVGBy9bgv8E+ibC+cuTWxZP3eAAW2i1wXAm8BQYCYwOlr/S+CaHIrtEeC8bP+bi+K6AXgceC5ajuW8NZYSwRBghbuvdPedwAzgrCzHlJPc/XXgkwqrzwJ+E73+DXB2vQYVqSS2nODu69397ej1VmAp0I0cOHdpYss6D76IFguihwOnAE9F67N13iqLLSeYWSFwBvDraNmI6bw1lkTQDVibtFxCjvxHiDjwRzObZ2bjsh1MCge6+3oIFxXggCzHU9F4M1sUVR1lpdoqmZkVAYMIvyBz6txViA1y4NxF1RsLgA3Ay4TS+xZ3L412ydr/14qxuXvivE2Kztu9ZtYiG7EBk4H/BPZEy52I6bw1lkRgKdblTGYHjnf3wcBI4LtmdmK2A2pAHgB6AwOB9cB/ZzMYM2sD/A643t0/z2YsFaWILSfOnbvvdveBQCGh9H5Eqt3qN6roQyvEZmb9gR8BhwPHAPsDN9Z3XGb2TWCDu89LXp1i14yct8aSCEqAQ5KWC4F1WYplH+6+LnreADxN+M+QSz42s64A0fOGLMdTxt0/jv6z7gEeJIvnzswKCBfa6e7+/6LVOXHuUsWWS+cuimcL8BqhHr6DmSWmys36/9ek2EZEVW3u7juAh8nOeTseGGVmqwhV3acQSgixnLfGkgjmAH2iFvXmwGhgVpZjAsDMWptZ28Rr4DTg3fTvqnezgEui15cAz2Qxlr0kLrKRc8jSuYvqZ/8XWOru9yRtyvq5qyy2XDh3ZtbFzDpEr/cDTiW0YcwGzot2y9Z5SxXbe0mJ3Qh18PV+3tz9R+5e6O5FhOvZq+4+lrjOW7ZbxTPYun46obfE+8DN2Y4nKa5ehF5MC4HF2Y4NeIJQTbCLUJK6nFD3+CdgefS8fw7F9ijwDrCIcNHtmqXYhhGK4YuABdHj9Fw4d2liy/q5AwYA86MY3gVujdb3At4CVgD/B7TIodhejc7bu8BjRD2LsvUATqK811As501DTIiI5LnGUjUkIiK1pEQgIpLnlAhERPKcEoGISJ5TIhARyXNKBCIRM9udNOLkAsvgKLZmVpQ8qqpILmlW9S4ieWO7h+EGRPKKSgQiVbAwn8TPo7Hr3zKzQ6P1PczsT9HgZH8ys+7R+gPN7OlonPuFZvYv0aGamtmD0dj3f4zuZsXMrjWzJdFxZmTpa0oeUyIQKbdfhaqhC5K2fe7uQ4D7CWO+EL3+rbsPAKYDU6L1U4A/u/tRhPkVFkfr+wBT3b0fsAX4VrR+AjAoOs7VcX05kcrozmKRiJl94e5tUqxfBZzi7iujwd0+cvdOZraJMGzDrmj9enfvbGYbgUIPg5YljlFEGOa4T7R8I1Dg7j8zsxeBL4DfA7/38jHyReqFSgQi1eOVvK5sn1R2JL3eTXkb3RnAVOBoYF7S6JIi9UKJQKR6Lkh6/nv0+m+EkSEBxgJvRK//BFwDZROftKvsoGbWBDjE3WcTJiHpAOxTKhGJk355iJTbL5qtKuFFd090IW1hZm8SfjyNidZdCzxkZj8ENgKXReuvA6aZ2eWEX/7XEEZVTaUp8JiZtSdMPHKvh7HxReqN2ghEqhC1ERS7+6ZsxyISB1UNiYjkOZUIRETynEoEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikuf+P67M0nHIRvuJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history_dict['accuracy']\n",
    "val_acc = history_dict['val_accuracy']\n",
    "loss = history_dict['loss']\n",
    "val_loss = history_dict['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "# \"bo\" is for \"blue dot\"\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "# b is for \"solid blue line\"\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZwU9Z3/8dcHBLkPAY0yOoOGjQoMMI4YV7wVkXhE1AjB3XgSNRqjm3VVTDRu1E3UaDSuWYxJVPBAExX9GcX7SDwABRSJioA4goAccsxwf35/fKtneobunp6jp3u638/Hox5dXVVd/ekaqE/V9ypzd0REpHC1yXYAIiKSXUoEIiIFTolARKTAKRGIiBQ4JQIRkQKnRCAiUuCUCGQHZtbWzNab2V7NuW02mdk3zazZ20qb2TFmtiju/Udmdmg62zbiu/5gZlc39vMiyeyU7QCk6cxsfdzbTsAmYFv0/ofuPrkh+3P3bUCX5t62ELj7t5pjP2Z2HnCmux8Rt+/zmmPfInUpEeQBd68+EUdXnOe5+wvJtjezndx9a0vEJlIf/XvMPhUNFQAz+6WZPWJmD5nZOuBMMzvYzN4yszVmttTM7jCzdtH2O5mZm1lJ9H5StP5vZrbOzN40s34N3TZaf7yZfWxmX5vZnWb2dzM7K0nc6cT4QzObb2arzeyOuM+2NbPbzGylmX0KjExxfK4xs4frLLvLzH4TzZ9nZvOi3/NpdLWebF8VZnZENN/JzB6IYpsLHJDgexdE+51rZidFywcBvwMOjYrdvoo7ttfFff6C6LevNLMnzGz3dI5NQ45zLB4ze8HMVpnZl2Z2Rdz3/Cw6JmvNbIaZ7ZGoGM7M3oj9naPj+Vr0PauAa8ysv5m9HP2Wr6Lj1j3u88XRb1wRrf+tmXWIYt4vbrvdzazSzHol+72SgLtryqMJWAQcU2fZL4HNwImE5N8ROBA4iHBXuDfwMXBxtP1OgAMl0ftJwFdAOdAOeASY1IhtdwXWASdH6y4HtgBnJfkt6cT4JNAdKAFWxX47cDEwFygCegGvhX/uCb9nb2A90Dlu38uB8uj9idE2BhwFVAGl0bpjgEVx+6oAjojmbwFeAXoCxcCHdbb9HrB79Df5fhTDbtG684BX6sQ5Cbgumh8RxTgE6AD8L/BSOsemgce5O7AMuBTYGegGDIvWXQXMBvpHv2EIsAvwzbrHGngj9neOfttW4EKgLeHf478ARwPto38nfwduifs9H0THs3O0/SHRuonADXHf8x/A49n+f9japqwHoKmZ/6DJE8FL9Xzup8Cj0Xyik/vv47Y9CfigEdueA7wet86ApSRJBGnG+O249X8FfhrNv0YoIoutG1X35FRn328B34/mjwc+TrHt08CPovlUiWBx/N8CuCh+2wT7/QD4TjRfXyK4D7gxbl03Qr1QUX3HpoHH+d+AGUm2+zQWb53l6SSCBfXEcBowPZo/FPgSaJtgu0OAhYBF72cBo5v7/1W+TyoaKhyfx78xs33N7P9Ft/prgeuB3ik+/2XcfCWpK4iTbbtHfBwe/udWJNtJmjGm9V3AZyniBXgQGBvNfx+ormA3sxPM7O2oaGQN4Wo81bGK2T1VDGZ2lpnNjoo31gD7prlfCL+ven/uvhZYDfSN2yatv1k9x3lPYH6SGPYkJIPGqPvv8RtmNsXMvohi+HOdGBZ5aJhQi7v/nXB3MdzMBgJ7Af+vkTEVLCWCwlG36eT/Ea5Av+nu3YCfE67QM2kp4YoVADMzap+46mpKjEsJJ5CY+pq3PgIcY2ZFhKKrB6MYOwKPATcRim16ANPSjOPLZDGY2d7A3YTikV7Rfv8Zt9/6mrouIRQ3xfbXlVAE9UUacdWV6jh/DuyT5HPJ1m2IYuoUt+wbdbap+/t+RWjtNiiK4aw6MRSbWdskcdwPnEm4e5ni7puSbCdJKBEUrq7A18CGqLLthy3wnU8DZWZ2opntRCh37pOhGKcAPzGzvlHF4X+l2tjdlxGKL/4EfOTun0SrdiaUW68AtpnZCYSy7HRjuNrMeljoZ3Fx3LouhJPhCkJOPI9wRxCzDCiKr7St4yHgXDMrNbOdCYnqdXdPeoeVQqrjPBXYy8wuNrP2ZtbNzIZF6/4A/NLM9rFgiJntQkiAXxIaJbQ1s/HEJa0UMWwAvjazPQnFUzFvAiuBGy1UwHc0s0Pi1j9AKEr6PiEpSAMpERSu/wB+QKi8/T/CFXFGRSfbM4DfEP5j7wO8R7gSbO4Y7wZeBN4HphOu6uvzIKHM/8G4mNcAlwGPEypcTyMktHRcS7gzWQT8jbiTlLvPAe4A3om22Rd4O+6zzwOfAMvMLL6IJ/b5ZwlFOI9Hn98LGJdmXHUlPc7u/jVwLHAqoXL6Y+DwaPXNwBOE47yWUHHbISryOx+4mtBw4Jt1flsi1wLDCAlpKvCXuBi2AicA+xHuDhYT/g6x9YsIf+fN7v6PBv52oaaCRaTFRbf6S4DT3P31bMcjrZeZ3U+ogL4u27G0RupQJi3KzEYSbvU3EpofbiVcFYs0SlTfcjIwKNuxtFYqGpKWNhxYQCgyGAl8V5V70lhmdhOhL8ON7r442/G0VioaEhEpcLojEBEpcK2ujqB3795eUlKS7TBERFqVmTNnfuXuCZtrZywRmNkfCU2+lrv7wATrDfgtoet/JaH7+bv17bekpIQZM2Y0d7giInnNzJL2rs9k0dCfSTHiI2E8l/7RNJ7Q7ltERFpYxhKBu79G6ICTzMnA/R68BfSIDaMrIiItJ5uVxX2pPfBUBUnGnTGz8dFY5zNWrFjRIsGJiBSKbFYWJxq0K2FbVnefSOi+Tnl5+Q7bbNmyhYqKCjZu3Ni8EUqz6tChA0VFRbRrl2z4HBHJhmwmggpqj8xYRBhuoOE7qqiga9eulJSUEOqgJde4OytXrqSiooJ+/frV/wERaTHZLBqaCvx7NGrht4Gv3X1pY3a0ceNGevXqpSSQw8yMXr166a5NpBEmT4aSEmjTJrxOnlzfJxomY4nAzB4ijCnzLQvPcT3XwjNWL4g2eYYw1MB84B7C05ua8n1NilcyT38jKVT1nchTrZ88GcaPh88+A/fwOn58MyeDbD8iraHTAQcc4HV9+OGHOyyT3KS/leSjSZPci4vdzcLrpEm113Xq5B5O42Hq1Klmm/rWFxfXXhebiosbFiNJHjnqelRl81i5ciVDhgxhyJAhfOMb36Bv377V7zdv3pzWPs4++2w++uijlNvcddddTG7ue0IRqVdTrtgnTIDKytr7q6wMy9NZvzjJUHrJljdKsgyRq1Nz3BGkyt5Nde211/rNN9+8w/Lt27f7tm3bmu+LWindEUiuSnZeaOoVu1ni9WbprdcdQQa0SHlbZP78+QwcOJALLriAsrIyli5dyvjx4ykvL2fAgAFcf/311dsOHz6cWbNmsXXrVnr06MGVV17J4MGDOfjgg1m+fDkA11xzDbfffnv19ldeeSXDhg3jW9/6Fv/4R3gw04YNGzj11FMZPHgwY8eOpby8nFmzZu0Q27XXXsuBBx5YHV/4dwIff/wxRx11FIMHD6asrIxFixYBcOONNzJo0CAGDx7MhNilikgOyVQ5fFOv2PdK8rTs2PL61t9wA3TqVHtdp05hebNJliFydWrqHUFzZddk4u8IPvnkEzczf+edd6rXr1y50t3dt2zZ4sOHD/e5c+e6u/shhxzi7733nm/ZssUBf+aZZ9zd/bLLLvObbrrJ3d0nTJjgt912W/X2V1xxhbu7P/nkk37ccce5u/tNN93kF110kbu7z5o1y9u0aePvvffeDnHG4ti+fbuPGTOm+vvKysp86tSp7u5eVVXlGzZs8KlTp/rw4cO9srKy1mcbQ3cEkgmZLIdv6hV7U2OLbdPUUgx0R1CjRcrb4uyzzz4ceOCB1e8feughysrKKCsrY968eXz44Yc7fKZjx44cf/zxABxwwAHVV+V1jR49eodt3njjDcaMGQPA4MGDGTBgQMLPvvjiiwwbNozBgwfz6quvMnfuXFavXs1XX33FiSeeCIQOYJ06deKFF17gnHPOoWPHjgDssssuDT8QIk2U6oo+k+XwTb1iHzcOJk6E4mIwC68TJ4bl6ayPbbNoEWzfHl7HNfbp1EkUXCKo74/a3Dp37lw9/8knn/Db3/6Wl156iTlz5jBy5MiE7erbt29fPd+2bVu2bt2acN8777zzDtu41/+gocrKSi6++GIef/xx5syZwznnnFMdR6Imnu6upp+ScU2pkK3vAq8pxTdNPdHHtkl1Is/0ib4+BZcIWqS8LYm1a9fStWtXunXrxtKlS3nuueea/TuGDx/OlClTAHj//fcT3nFUVVXRpk0bevfuzbp16/jLX/4CQM+ePenduzdPPfUUEDrqVVZWMmLECO69916qqqoAWLUq1ViCIsklO9k3teVNJsvhc+GKPdMKLhGk80fNlLKyMvbff38GDhzI+eefzyGHHNLs33HJJZfwxRdfUFpayq233srAgQPp3r17rW169erFD37wAwYOHMgpp5zCQQcdVL1u8uTJ3HrrrZSWljJ8+HBWrFjBCSecwMiRIykvL2fIkCHcdtttzR635IdsVcjWd4HXHMU3rflEX69klQe5OqlDWWpbtmzxqqoqd3f/+OOPvaSkxLds2ZLlqGrob9W6NaXjVCYrZOuLLZ31+Y4UlcVZP7E3dFIiSG316tVeVlbmpaWlPmjQIH/uueeyHVIt+lu1XplsT9/UljdSv1SJoOCKhvJdjx49mDlzJrNnz2bOnDmMGDEi2yFJK9KUljnZrpCVxlMiECkgmWyZowrZVizZrUKuTioaat30t8qephbtNEfxTaGX02cTKhoSKRzJrvoz3TJHV/WtVzafUCYizSxWvBM74ceKdyC9op3PPttxfaxoJ3bSnjChpsdtrFgnZtw4ndxbI90RNIMjjjhih85ht99+OxddlPpZO126dAFgyZIlnHbaaUn3PWPGjJT7uf3226mMu9QbNWoUa9asSSd0aYUaW6HbHIOb6Yo+PykRNIOxY8fy8MMP11r28MMPM3bs2LQ+v8cee/DYY481+vvrJoJnnnmGHj16NHp/kruaUqGrljmSjBJBMzjttNN4+umn2bRpEwCLFi1iyZIlDB8+nPXr13P00UdTVlbGoEGDePLJJ3f4/KJFixg4cCAQhn8YM2YMpaWlnHHGGdXDOgBceOGF1UNYX3vttQDccccdLFmyhCOPPJIjjzwSgJKSEr766isAfvOb3zBw4EAGDhxYPYT1okWL2G+//Tj//PMZMGAAI0aMqPU9MU899RQHHXQQQ4cO5ZhjjmHZsmUArF+/nrPPPptBgwZRWlpaPUTFs88+S1lZGYMHD+boo49ulmNbiJrShDPVVb/K8CWpZLXIuTrV12ro0kvdDz+8eadLL62/Rn7UqFH+xBNPuHsYCvqnP/2pu4eevl9//bW7u69YscL32Wcf3759u7u7d+7c2d3dFy5c6AMGDHB391tvvdXPPvtsd3efPXu2t23b1qdPn+7uNcM/b9261Q8//HCfPXu2u7sXFxf7ihUrqmOJvZ8xY4YPHDjQ169f7+vWrfP999/f3333XV+4cKG3bdu2enjq008/3R944IEdftOqVauqY73nnnv88ssvd3f3K664wi+NOyirVq3y5cuXe1FRkS9YsKBWrHWp1VDTeufW1wNXHa8kGdRqKPPii4fii4XcnauvvprS0lKOOeYYvvjii+or60Ree+01zjzzTABKS0spLS2tXjdlyhTKysoYOnQoc+fOTTigXLw33niDU045hc6dO9OlSxdGjx7N66+/DkC/fv0YMmQIkHyo64qKCo477jgGDRrEzTffzNy5cwF44YUX+NGPflS9Xc+ePXnrrbc47LDD6NevH6ChqpPJ9OBqKt6Rxsi7VkNR6UeL++53v8vll1/Ou+++S1VVFWVlZUAYxG3FihXMnDmTdu3aUVJSknDo6XiJhnxeuHAht9xyC9OnT6dnz56cddZZ9e4nXAQkFhvCGsIw1omKhi655BIuv/xyTjrpJF555RWuu+666v3WjTHRskIVG0QtUcuaVCf6cePSa8IZ3yoIElfo6sQvDaE7gmbSpUsXjjjiCM4555xalcRff/01u+66K+3atePll1/ms0Tt8+Icdthh1Q+o/+CDD5gzZw4QhrDu3Lkz3bt3Z9myZfztb3+r/kzXrl1Zt25dwn098cQTVFZWsmHDBh5//HEOPfTQtH/T119/Td++fQG47777qpePGDGC3/3ud9XvV69ezcEHH8yrr77KwoULgcIdqjrTvXN1xS+ZoETQjMaOHcvs2bOrnxAGMG7cOGbMmEF5eTmTJ09m3333TbmPCy+8kPXr11NaWsqvf/1rhg0bBoSnjQ0dOpQBAwZwzjnn1BrCevz48Rx//PHVlcUxZWVlnHXWWQwbNoyDDjqI8847j6FDh6b9e6677jpOP/10Dj30UHr37l29/JprrmH16tUMHDiQwYMH8/LLL9OnTx8mTpzI6NGjGTx4MGeccUba35NPmlq0oyackhXJKg9yddIQE61bvvytklX4NkdlroZhkEwgRWVx3tURiGRaqt676p0rrZGKhkQSaGxbfhXtSGuUN4nAU7SQkdzQWv5GTanwVWWutEZ5kQg6dOjAypUrW82JphC5OytXrqRDhw7ZDgXIXO9d0BW/tD55UUdQVFRERUUFK1asyHYokkKHDh0oKirKdhgpy/ibqy2/SGtire0qury83OsbjVMklZKSxBW6xcXhCr6+9ZC605hILjKzme5enmhdXhQNidSVquinqQ9gARX/SH5RIpC8U19lr3rvitSmoiHJO/UV7dStI4Bwxa+TveQzFQ1J3mlK0Y+u+EVqy4tWQ1JY6mv1U1/v3th2OvGLBLojkFanvnb+6VT2ikiNjCYCMxtpZh+Z2XwzuzLB+mIze9HM5pjZK2aW/UbmkvNU9CPSvDKWCMysLXAXcDywPzDWzPavs9ktwP3uXgpcD9yUqXik9UlWD1Bfqx9Q806RhsjkHcEwYL67L3D3zcDDwMl1ttkfeDGafznBeilQqZqAquhHpHllMhH0BT6Pe18RLYs3Gzg1mj8F6GpmveruyMzGm9kMM5uhYSQKQ32PdFTRj0jzyWQiSPQA27qdFn4KHG5m7wGHA18AW3f4kPtEdy939/I+ffo0f6SSFU1tAqqiH5HmkcnmoxXAnnHvi4Al8Ru4+xJgNICZdQFOdfevMxiT5IjmaAIqIs0jk3cE04H+ZtbPzNoDY4Cp8RuYWW8zi8VwFfDHDMYjOURNQEVyR8YSgbtvBS4GngPmAVPcfa6ZXW9mJ0WbHQF8ZGYfA7sB+m9eINQEVCR3aKwhyZhUQzWnM9SziDQfjTUkLa6+EUBV9COSO5QIJCPqqwNQ0Y9I7lDRkGREmzbhTqAus9DkU0RaloqGpMWlMwyEiOQGDUMtjZaqMlgPeG+d1q4Nf89Vq2CnncLUrt2Or126QM+e4c4vXevXw9KlsGJFzT5iU+fO0L595n6XpKZEII1SX4ewWELQA95bTlUVrFwZptWrYfNm2LZtx2n79rBtRQV8/nmYFi8Or2vXpv99bdpAr17Qpw/07l3z2rt3+HexdCksWRJely4NiSCVWHLo2hV22SXsO9HUrVsoYkxm06bwO9atqz3FllVWht9fVQUbN9bMV1WFz3bsGGKITd261cx36ZI6+e20U4i9d+/EsXfsmP7xbUmqI5BGUfPPxtu6NZysV6yAr75K/rpxYzjptG2beFq3rubEv3LljpXz6ejTB/bcM0x77VUz37t3SBpbtoR4t26tmd+yJXx33Zhj8ytXhhPe7rsnnnbdNexn/XrYsCG8xqYNG8IJe9WqsJ/Y/lavbnzdUocOtU/snTuH+Dp0CK/x0847h+Men0hi82vXhvhSnTI3bw7bJNO2bc2dVfxdVmx+111h773D1K9fzXzfvuGzTZGqjkB3BNIo9XUIK1TuMH8+TJ8OH3xQc4KMP2GuXp388z161Fxdd+xYczLeuHHHK/suXaCoCAYPrrnijF2J9uwZTmqxpFE3oey8M+yxR2auULdvD1fsqa7aG7PPNWtCUli3LvW27dvXPvG3a9d8caRj06baCTp+Wr++dlKNn9+8GZYtgzffhEceCX/jmHbtwsXX9dfDmDHNH7MSgTSKxgIKJ/0vvggn/dg0Y0Y4YUG4yosvMhkyZMdilD59aheptPRJKxMaUm/QkH3uskuYcl0sye6xR+P3sWVLKKpbuBAWLKiZevduvjjjKRFIUqoMDiorw1X+J5+EKTb/0UfhCg7CSX/QIPje96C8HA48EAYMyI8Tu7S8du1qioWOPjrz36dEIAkVamXwxo3wzjvw6qvw+uswd26o8Iy3227Qvz+MGgVDh4aT/uDBuVsRKFIfVRZLQoVSGVxVFcpkX301TG+9Fcp4zaC0NBTn9O8P3/xmzWu3btmOWqThVFksDZZPlcEbNoTktWhRKHONvS5cGK74N28OZdBDhsBFF8ERR8Dw4a2jPFqkOSgRSEKtrTLYPbRVnzsXPvyw5vXjj0NLnXgdOoQ7npISOOYYOPzwcOLv3j0bkYtknxKBJJTrlcHu8OKL8OijoZnmhx/WtNaBcDU/YAB897uhPXa/fjUn/912a96mjSKtnRJBgUvWMihXK4M3bIAHHoA77wwn/+7dQ0XtmDHhxD9gAOy/f+iYo5O9SHqUCApYOi2Dsn3ij1m0CO66C/7wh3DlX1YG990Xmmt26JDt6ERaN7UaKmC53jJo61Z47bVw9T91arjCP/VU+PGP4V//VVf8Ig2hVkOSUC62DFq8GJ57Lkwvvhiu/nv1giuvhAsvDEMqiEjzUiIoYNloGbR9e+3xcjZtCm33Yyf/jz4K2xUVwejRcNxxcOKJ6qwlkklKBAUsEy2Dqqrg3XfDyf2tt+Dtt+HLL2sSQDIdO4ZmnBdcACNGwH77qehHpKUoEeS5VOMFNUfLoC+/hBdeCCf8t96CWbNC2T6EJpvDh4f9JhtKeaedQg/e4cNV6SuSLaoszmN1WwVBuOJv6kPiN2yAxx+HSZPg+efD1X6XLjBsGHz722E66KDQhFNEckOqymIlgjzWnK2Ctm0LlbeTJsFf/xqSQXExnHkmnHZaGHmzqQ/OEJHMUauhAtUcrYIqKuC22+DBB0MxUPfu8P3vw7/9GxxySGbGnheRlqVEkMea0ipo2za4+264+uowNPOoUeHk/53vqCxfJN/oei6P3XBDqBOIl06roDlzwtX+JZfAwQfDP/8JTzwROnMpCYjkHyWCPDZuXKgYLi4OTTGLi1NXFFdVwVVXwQEHhMfiTZoEzz4bnpIkIvlLRUN5Lt3xgl54IbTh//RTOPtsuPnm0KNXRPKf7ggKXEUF/Pu/w7HHhorfl16CP/5RSUCkkCgRtHKTJ4dmom3ahNfJk9P73LJlcNll4dGLDz8M11wT6gaOPDKT0YpILlLRUCtW3zDSiaxaBbfcAr/9bRjn5wc/gJ/9LCQRESlMuiNoxSZMqN1rGML7CRN23HbtWvjv/w7DPvzP/8DJJ4cHu9x7r5KASKHTHUErlk6HsW3b4I47QpPRlSvhlFPgF78IPYFFRCCNOwIzu9jMerZEMNIwyTqGxZYvXRoqgS+/HMrLYfr0MDyEkoCIxEunaOgbwHQzm2JmI800OHCuSNVh7PnnYciQMCLon/4U+gOUJxxlREQKXb2JwN2vAfoD9wJnAZ+Y2Y1mtk99n40Sx0dmNt/Mrkywfi8ze9nM3jOzOWY2qhG/oWAl6jB2990wb154oEvv3jBjBpx1VrYjFZFcllZlsYchSr+Mpq1AT+AxM/t1ss+YWVvgLuB4YH9grJntX2eza4Ap7j4UGAP8b4N/QYEbNy6MJLp9O/z976Hy94YbQqew6dNh/7pHXESkjnTqCH5sZjOBXwN/Bwa5+4XAAcCpKT46DJjv7gvcfTPwMHBynW0c6BbNdweWNDD+vJduP4HnngtFQTNnwv33h4RQt9hIRCSRdFoN9QZGu3utcSzdfbuZnZDic32Bz+PeVwAH1dnmOmCamV0CdAaOSbQjMxsPjAfYK5MP1M0x6fQT2LIFfv7z0CR04EB49FHYd9/sxCsirVM6RUPPAKtib8ysq5kdBODu81J8LlGlct2n4IwF/uzuRcAo4AEz2yEmd5/o7uXuXt6nT580Qs4P9fUTWLgQDjssJIHzzguPi1QSEJGGSicR3A2sj3u/IVpWnwpgz7j3RexY9HMuMAXA3d8EOhDuQITU/QSmTAlFQfPmwSOPwD33qChIRBonnURgHvc8S3ffTnpFStOB/mbWz8zaEyqDp9bZZjFwNICZ7UdIBCvSCbwQJCsF69wZzjgjVATPmgXf+17LxiUi+SWdRLAgqjBuF02XAgvq+5C7bwUuBp4D5hFaB801s+vN7KRos/8Azjez2cBDwFne2h6inEGJ+gmYhecFX301vPaahocQkaar9+H1ZrYrcAdwFKGM/0XgJ+6+PPPh7ajQHl4/eXI46ceKiXr0gMceg6OPzm5cItK6NOnh9dEJf0yzRyVpGTUqnPgXLw7zf/4zFFB9uYi0gHoTgZl1IFTqDiCU4QPg7udkMC4B3nwTxowJYwb95jfwk5+EoiERkeaUTh3BA4Txho4DXiW0/lmXyaAKSaIOY9u3w69+BYceCjvtBP/4R3iIjJKAiGRCOnUE77n7UDOb4+6lZtYOeM7dj2qZEGvLpzqCuh3GADp2DE8Ne//90Bpo4kTo3j17MYpIfkhVR5DOHcGW6HWNmQ0kDAVR0kyxFbREHcaqqkIS+P3vwyMklQREJNPS6Q8wMXoewTWEfgBdgJ9lNKoCkazDGMAPf9hycYhIYUuZCKLhHta6+2rgNWDvFomqQOy1Vxg/qK7i4paPRUQKV8qioagX8cUtFEvBueEGaNeu9rLYg2VERFpKOnUEz5vZT81sTzPbJTZlPLIC0LkzbN1a03u4uDhUDsdGFhURaQnp1BHE+gv8KG6Zo2KiJnn33XDCP/BAeOWV0FpIRCQb0ulZ3K8lAikkX3wBJ54IvXrBk08qCYhIdqXTs/jfEy139/ubP5z8t2EDnHQSrF0bHuAtFuIAAA/FSURBVC35jW9kOyIRKXTpFA0dGDffgTBs9LuAEkGaJk8OfQY++yxc/W/cCE89BaWl2Y5MRCS9oqFL4t+bWXfCsBOShrq9h6uqQkuhNWuyG5eISEw6rYbqqgT6N3cg+SpR7+EtW2oeNykikm3p1BE8Rc2zhtsA+xM9XlLql+pxkyIiuSCdOoJb4ua3Ap+5e0WG4sk7RUXw+ec7Lk/2GEoRkZaWTiJYDCx1940AZtbRzErcfVFGI8sTe+65YyJQ72ERySXp1BE8CmyPe78tWib1mDQpPEvg1FNDr2Ez9R4WkdyTzh3BTu6+OfbG3TebWfsMxpQXPv0ULrooPFzmkUegbdtsRyQiklg6dwQrzOyk2BszOxn4KnMhtX5btoQr/jZtwl2BkoCI5LJ07gguACab2e+i9xVAwt7GEvziF/D22zBliiqFRST3pdOh7FPg22bWhfBoSz2vOIVXXoEbb4Rzz4XTT892NCIi9au3aMjMbjSzHu6+3t3XmVlPM/tlSwTX2qxaBWeeCf37w+23ZzsaEZH0pFNHcLy7Vw+IED2tbFTmQmqd3OH882H5cnjwQejSJdsRiYikJ51E0NbMdo69MbOOwM4pti9If/gD/PWvoVjogAOyHY2ISPrSqSyeBLxoZn+K3p8N3Je5kFqfO+6An/wkzN95J+y+u/oJiEjrkU5l8a/NbA5wDGDAs4Aerx6ZPBkuvzwUDUEYQ2j8+DCvZCAirUG6o49+SehdfCrheQTzMhZRK/Of/wnbttVeVlmp0UVFpPVIekdgZv8CjAHGAiuBRwjNR49sodhahaVLEy/X6KIi0lqkKhr6J/A6cKK7zwcws8taJKpW4qWXkq9TRzIRaS1SFQ2dSigSetnM7jGzowl1BAJs3w5XXBEeQF/34fMaXVREWpOkicDdH3f3M4B9gVeAy4DdzOxuMxvRQvHlrEcegZkz4bbb4J57NLqoiLRe5rHmLulsbLYLcDpwhrsflbGoUigvL/cZM2Zk46urbdoE++4L3bvDu++GweVERHKZmc109/JE69LpR1DN3VcB/xdNBet//xcWLYJp05QERKT102msgdasgV/+EkaMgGOPzXY0IiJNl9FEYGYjzewjM5tvZlcmWH+bmc2Kpo/NbE2i/eSS//kfWL0afvWrbEciItI8GlQ01BBm1ha4CziW8AyD6WY21d0/jG3j7pfFbX8JMDRT8TSHxYvDqKJnnglDhmQ7GhGR5pHJO4JhwHx3XxA96vJh4OQU248FHspgPE3285+H1//+7+zGISLSnDKZCPoCn8e9r4iW7cDMioF+QMIuWmY23sxmmNmMFStWNHug6ZgzB+6/H37849BEVEQkX2QyESTqfJasreoY4DF335ZopbtPdPdydy/v06dPswXYEFddBT16hFcRkXySyURQAewZ974IWJJk2zHkcLHQvHnwzDNhlNGePbMdjYhI88pkIpgO9DezfmbWnnCyn1p3IzP7FtATeDODsTTJpZeG15/9DEpKwtDTIiL5ImOJwN23AhcDzxGGrZ7i7nPN7HozOylu07HAw96QLs4t6J574Pnna95/9ll43oCSgYjkiwYNMZELWnqIiV12Cf0G6iouDr2LRURag1RDTKhncQrbtydOAqDnDYhI/lAiSOGFF5Kv0/MGRCRfKBGk8LvfQbduet6AiOQ3JYIkFiyAp58OHcj0vAERyWcZG2uotbv77jDE9A9/CEVFOvGLSP7SHUEClZVw770wenRIAiIi+UyJIIEHHwythS6+ONuRiIhknhJBHe5w551QWgqHHprtaEREMk91BHW88UYYaXTixFA5LCKS73RHUMedd4ZRRlU5LCKFQokgzhdfwF//CueeG/oKiIgUAiWCOL//fRhW4qKLsh2JiEjLUSKIbNoU6gW+8x3Ye+9sRyMi0nKUCCKPPgrLl8Mll2Q7EhGRlqVEEJk0CfbZB445JtuRiIi0LCUCYONGePVVOOGEMKyEiEgh0WmP0Hdg40YYMSLbkYiItDwlAmDaNGjXDg4/PNuRiIi0PCUCQiIYPhw6d852JCIiLa/gE8GXX8Ls2SoWEpHCVfCJIPY4SiUCESlUBZ8Ipk2D3r1hyJBsRyIikh0FnQjc4fnnQ98BNRsVkUJV0Ke/Dz4IdQRdu0JJSUgGJSUweXK2IxMRaTkF/TyCadPC6wMPhH4EAJ99BuPHh3kNRS0ihaCg7whi/QdiSSCmshImTMhOTCIiLa1gE0FVFbz2GmzZknj94sUtG4+ISLYUbCKIDSux666J1++1V8vGIyKSLQWbCKZNg/bt4cYbd3waWadOcMMN2YlLRKSlFXQiOOSQ8FjKiROhuDg8rL64OLxXRbGIFIqCbDW0dCnMmQM33RTejxunE7+IFK6CvCPQsBIiIjUKMhFoWAkRkRoFlwhiw0oce6yGlRARgQJMBO+/D8uWqVhIRCQmo4nAzEaa2UdmNt/MrkyyzffM7EMzm2tmD2YyHqgZVuLYYzP9TSIirUPGWg2ZWVvgLuBYoAKYbmZT3f3DuG36A1cBh7j7ajNL0r2r+UybBgMGQN++mf4mEZHWIZN3BMOA+e6+wN03Aw8DJ9fZ5nzgLndfDeDuyzMYT/WwEioWEhGpkclE0Bf4PO59RbQs3r8A/2Jmfzezt8xsZAbj4fXXYdMmFQuJiMTLZIcyS7DME3x/f+AIoAh43cwGuvuaWjsyGw+MB9irCYMAxYaVOOywRu9CRCTvZPKOoALYM+59EbAkwTZPuvsWd18IfERIDLW4+0R3L3f38j59+jQ6oGnTYPhw6Ny50bsQEck7mUwE04H+ZtbPzNoDY4CpdbZ5AjgSwMx6E4qKFmQimKVLQ9NR1Q+IiNSWsUTg7luBi4HngHnAFHefa2bXm9lJ0WbPASvN7EPgZeA/3X1lJuJ5/vnwqkQgIlJbRgedc/dngGfqLPt53LwDl0dTRnXpAiNHwuDBmf4mEZHWpWB6Fo8eDX/7m4aVEBGpS6dFEZECp0QgIlLglAhERAqcEoGISIFTIhARKXBKBCIiBU6JQESkwCkRiIgUOCUCEZECp0QgIlLglAhERAqcEoGISIFTIhARKXBKBCIiBU6JQESkwBVEIpg8GUpKwrMISkrCexERCTL6hLJcMHkyjB8PlZXh/WefhfcA48ZlLy4RkVyR93cEEybUJIGYysqwXERECiARLF7csOUiIoUm7xPBXns1bLmISKHJ+0Rwww3QqVPtZZ06heUiIlIAiWDcOJg4EYqLwSy8TpyoimIRkZi8bzUE4aSvE7+ISGJ5f0cgIiKpKRGIiBQ4JQIRkQKnRCAiUuCUCERECpy5e7ZjaBAzWwF8lmR1b+CrFgynoXI5PsXWOIqtcRRb4zQltmJ375NoRatLBKmY2Qx3L892HMnkcnyKrXEUW+MotsbJVGwqGhIRKXBKBCIiBS7fEsHEbAdQj1yOT7E1jmJrHMXWOBmJLa/qCEREpOHy7Y5AREQaSIlARKTA5U0iMLORZvaRmc03syuzHU88M1tkZu+b2Swzm5HlWP5oZsvN7IO4ZbuY2fNm9kn02jOHYrvOzL6Ijt0sMxuVpdj2NLOXzWyemc01s0uj5Vk/diliy/qxM7MOZvaOmc2OYvtFtLyfmb0dHbdHzKx9DsX2ZzNbGHfchrR0bHExtjWz98zs6eh9Zo6bu7f6CWgLfArsDbQHZgP7ZzuuuPgWAb2zHUcUy2FAGfBB3LJfA1dG81cCv8qh2K4DfpoDx213oCya7wp8DOyfC8cuRWxZP3aAAV2i+XbA28C3gSnAmGj574ELcyi2PwOnZfvfXBTX5cCDwNPR+4wct3y5IxgGzHf3Be6+GXgYODnLMeUkd38NWFVn8cnAfdH8fcB3WzSoSJLYcoK7L3X3d6P5dcA8oC85cOxSxJZ1HqyP3raLJgeOAh6LlmfruCWLLSeYWRHwHeAP0XsjQ8ctXxJBX+DzuPcV5Mh/hIgD08xsppmNz3YwCezm7kshnFSAXbMcT10Xm9mcqOgoK8VW8cysBBhKuILMqWNXJzbIgWMXFW/MApYDzxPu3te4+9Zok6z9f60bm7vHjtsN0XG7zcx2zkZswO3AFcD26H0vMnTc8iURWIJlOZPZgUPcvQw4HviRmR2W7YBakbuBfYAhwFLg1mwGY2ZdgL8AP3H3tdmMpa4EseXEsXP3be4+BCgi3L3vl2izlo0q+tI6sZnZQOAqYF/gQGAX4L9aOi4zOwFY7u4z4xcn2LRZjlu+JIIKYM+490XAkizFsgN3XxK9LgceJ/xnyCXLzGx3gOh1eZbjqebuy6L/rNuBe8jisTOzdoQT7WR3/2u0OCeOXaLYcunYRfGsAV4hlMP3MLPYo3Kz/v81LraRUVGbu/sm4E9k57gdApxkZosIRd1HEe4QMnLc8iURTAf6RzXq7YExwNQsxwSAmXU2s66xeWAE8EHqT7W4qcAPovkfAE9mMZZaYifZyClk6dhF5bP3AvPc/Tdxq7J+7JLFlgvHzsz6mFmPaL4jcAyhDuNl4LRos2wdt0Sx/TMusRuhDL7Fj5u7X+XuRe5eQjifveTu48jUcct2rXgz1q6PIrSW+BSYkO144uLam9CKaTYwN9uxAQ8Rigm2EO6kziWUPb4IfBK97pJDsT0AvA/MIZx0d89SbMMJt+FzgFnRNCoXjl2K2LJ+7IBS4L0ohg+An0fL9wbeAeYDjwI751BsL0XH7QNgElHLomxNwBHUtBrKyHHTEBMiIgUuX4qGRESkkZQIREQKnBKBiEiBUyIQESlwSgQiIgVOiUAkYmbb4kacnGXNOIqtmZXEj6oqkkt2qn8TkYJR5WG4AZGCojsCkXpYeJ7Er6Kx698xs29Gy4vN7MVocLIXzWyvaPluZvZ4NM79bDP712hXbc3snmjs+2lRb1bM7Mdm9mG0n4ez9DOlgCkRiNToWKdo6Iy4dWvdfRjwO8KYL0Tz97t7KTAZuCNafgfwqrsPJjxfYW60vD9wl7sPANYAp0bLrwSGRvu5IFM/TiQZ9SwWiZjZenfvkmD5IuAod18QDe72pbv3MrOvCMM2bImWL3X33ma2AijyMGhZbB8lhGGO+0fv/wto5+6/NLNngfXAE8ATXjNGvkiL0B2BSHo8yXyybRLZFDe/jZo6uu8AdwEHADPjRpcUaRFKBCLpOSPu9c1o/h+EkSEBxgFvRPMvAhdC9YNPuiXbqZm1AfZ095cJDyHpAexwVyKSSbryEKnRMXpaVcyz7h5rQrqzmb1NuHgaGy37MfBHM/tPYAVwdrT8UmCimZ1LuPK/kDCqaiJtgUlm1p3w4JHbPIyNL9JiVEcgUo+ojqDc3b/KdiwimaCiIRGRAqc7AhGRAqc7AhGRAqdEICJS4JQIREQKnBKBiEiBUyIQESlw/x+hEvn4tS2K1QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.clf()   # clear figure\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}