tensorflow2/001-googles-7-steps-of-machine-learning-in-practice/001-googles-7-steps-of-machine-learning-in-practice.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Google's 7 steps of Machine Learning in practice\n",
    "\n",
    "This is a notebook for the medium article [The Google's 7 step of Machine Learning in Practice: a TensorFlow example for structured data](https://towardsdatascience.com/the-googles-7-steps-of-machine-learning-in-practice-a-tensorflow-example-for-structured-data-96ccbb707d77)\n",
    "\n",
    "Please check out article for instructions\n",
    "\n",
    "**License**: [BSD 2-Clause](https://opensource.org/licenses/BSD-2-Clause)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the question\n",
    "Data: [Titanic Dataset from Kaggle](https://www.kaggle.com/c/titanic/overview). \n",
    "\n",
    "Let's pretend that we've been asked to create a system that predicts survival on the Titanic."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Gathering data\n",
    "Let's import some libraries and load data to get started!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by reading in the CSV file into a pandas DataFrame.\n",
    "\n",
    "**Note:** the test.csv from Kaggle doesn't come with source of truth (No Survived column).\n",
    "\n",
    "Both the train.csv and test.csv we are using in this notebook are random shuffled and splited from the original train.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train_raw = pd.read_csv('data/titanic/train.csv')\n",
    "df_test_raw = pd.read_csv('data/titanic/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>493</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Molson, Mr. Harry Markland</td>\n",
       "      <td>male</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113787</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>C30</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Harper, Mrs. Henry Sleeper (Myna Haxtun)</td>\n",
       "      <td>female</td>\n",
       "      <td>49.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17572</td>\n",
       "      <td>76.7292</td>\n",
       "      <td>D33</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>388</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Buss, Miss. Kate</td>\n",
       "      <td>female</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27849</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>192</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Carbines, Mr. William</td>\n",
       "      <td>male</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28424</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>687</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Panula, Mr. Jaako Arnold</td>\n",
       "      <td>male</td>\n",
       "      <td>14.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3101295</td>\n",
       "      <td>39.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass                                      Name  \\\n",
       "0          493         0       1                Molson, Mr. Harry Markland   \n",
       "1           53         1       1  Harper, Mrs. Henry Sleeper (Myna Haxtun)   \n",
       "2          388         1       2                          Buss, Miss. Kate   \n",
       "3          192         0       2                     Carbines, Mr. William   \n",
       "4          687         0       3                  Panula, Mr. Jaako Arnold   \n",
       "\n",
       "      Sex   Age  SibSp  Parch    Ticket     Fare Cabin Embarked  \n",
       "0    male  55.0      0      0    113787  30.5000   C30        S  \n",
       "1  female  49.0      1      0  PC 17572  76.7292   D33        C  \n",
       "2  female  36.0      0      0     27849  13.0000   NaN        S  \n",
       "3    male  19.0      0      0     28424  13.0000   NaN        S  \n",
       "4    male  14.0      4      1   3101295  39.6875   NaN        S  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Preparing data with Exploratory Data Analysis (EDA)\n",
    "\n",
    "Let's begin some exploratory data analysis! We'll start by checking out missing data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Missing data\n",
    "\n",
    "We can use seaborn to create a simple heatmap to see where we are missing data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a25685850>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAErCAYAAAB981BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAaTUlEQVR4nO3debxlVXXg8d+qAkSGikIQMV0gDkQcIBAR1HxsEeMsRIGgoh+jtrTdGjHGockgikP602ltjYqKIQQSooiEKCIBRFSwGyhmHDCtIKZbhiZRGaLIsPqPfR5169Wrx3tVe++69c7v+/m8D3Xvo86q9+656+6z9tr7RGYiSepj2cb+B0jSmJh0Jakjk64kdWTSlaSOTLqS1JFJV5I62my+b/72skPtJ5NU3dk/vqprvOc+Ys+u8c6979RY1/cc6UpSR/OOdCWphd4jz2li0pXU3VIvL8zH8oIkdWTSlaSOTLqS1JFJV5I6MulKUkcmXUnqyJYxSd1NUwtXbyZdVTHmvkst3pjPF5Ouqpimk1qaZiZdVTHmkYu0GCZdVWESlBbG7gVJ6siRrqqwvCAtjElXVZgEpYUx6Urqbswf0tZ0JakjR7qSuhvzHIBJV1J305QEezPpSupuzCNda7qS1JFJV5I6MulKUkfWdCV1N0011t5MuqpizBMj0mKYdFWFSVCLMeYPaZOupO6mKQn2ZtKV1J0jXUnqaJqSYG8mXVUx5pGLtBgmXVVhEtRijPlD2sURktSRSVeSOrK8oCrGfLkoLYZJV1WYBKWFMelK6m7MH9ImXUndjbkcZdKV1N00JcHe7F6QpI4c6UrqzvKCtIHG/CaSFsPygiR15EhXVTjy1GKM+Xwx6aoKywtajDGfLyZdVTFNJ7U0zazpSlJHJl1J6sjygqTuxlyOMulK6m7ME2mWFySpI0e6qmLMIxdpMUy6qsIkqMUY8/lieUGSOnKkK6m7MZejTLqqYsxvImkxTLqqwiSoxRjz+WLSldTdmK+MnEiTpI5MupLUkUlXkjoy6UpSR06kqYoxT4xo8cb8+pl0VcWY30TSYph0JXU35isjk66k7qYpCfZm0pXU3ZhHunYvSFJHjnQlddd75Nl7ZD0fk66qGPPlohZvqZ8v59637u+ZdFWFSVBaGGu6ktSRI11V0/OS0ZG1NlUmXVVjItRCjflcMemqiqU+MaK6xny+mHRVxTSd1NI0cyJNkjpypKsqxny5KC2GSVdVmAS1GGM+X0y6krob85WRNV1J6sikK0kdWV6Q1N00Xe73ZtJVFWOu0Wnxxny+WF6QpI5MupLUkeUFVTFNl2/SNHOkK0kdmXQlqSOTriR1ZE1XVYy5BUhaDJOuqjAJajHGfL6YdCV1N+YrI2u6ktSRSVeSOjLpSlJH1nQldTdNNdbeTLqSuhvzRJpJV1WM+U0kLYZJV1WYBLUYYz5fTLqSuhvzlZFJV1J305QEezPpqooxj1y0eGM+X0y6qmKaTmpNvzGfLyZdSd2NeaTrijRJ6siRrqTupmnk2ZtJV1J3Yy4vmHRVxZjfRNJimHRVhUlQWhiTrqpwpCstjElXVZgEpYWxZUySOjLpSlJHJl1J6siarqTuxjwH4EhXkjpypKsqbBnTYoz5fDHpqoppOqmlaWZ5QZI6MulKUkcmXUnqyJqupO7GPAfgSFeSOnKkK6k7W8YkqaNpSoK9mXQldTfmka41XUnqyJGupO6maeTZm0lXUndjLi+YdCV1N01JsDdrupLUkSNdSd1ZXpCkjqYpCfZm0pXUnSNdaQON+U0kLYZJV1WYBKWFMemqCke60sKYdFWFSVBaGPt0Jakjk64kdWR5QVVY05UWxqSrKkyC0sJYXpCkjky6ktSR5QVJ3Y25HGXSldTdmCdeTbqSupumJNibSVdSd2Me6TqRJkkdmXQlqSOTriR1ZE1XUnfTVGPtzaSrKsY8MaLFG/P5YtJVFdN0UkvTzKSrKsY8cpEWw6SrKkyC0sLYvSBJHZl0Jakjk64kdWTSlaSOTLqS1JFJV5I6MulKUkcmXUnqyKQrSR2ZdCWpI5cBS+puzMvGTbqSuhvzBkmWFySpI5OuJHVkeUFVjPlyUVoMk66qMAlKC2N5QZI6cqSranqWGBxZa1Nl0lU1JkLpgVlekKSOTLqS1JHlBVVhy5i0MI50JakjR7qqwpGntDCOdCWpI5OuJHVkeUFSd2MuR5l0JXU35m4XywuS1JFJV5I6MulKUkfWdFXFmGt0Wrwxv34mXVUx5jeRtBgmXUndjfnKyKSrKsb8JpIWw6SrKkyC0sKYdFWFI11pYUy6qsIkKC2MfbqS1JFJV5I6MulKUkcmXUnqyKQrSR3ZvaAqbBmTFsakqypMgtLCWF6QpI5MupLUkeUFVWFNV1oYk66qMAlqMcZ8vph0VYUjXS3GmM8Xk66qmKaTWppmJl1J3Y35Q9qkK6m7MZcXbBmTpI5MupLUkUlXkjoy6UpSR06kqZqekyPTNDGi9TPa1zAzq38BR7Q4rvGMt6nFW8o/m/HW76tVeeGIRsc1nvE2tXhL+Wcz3nqwpitJHZl0JamjVkn3uEbHNZ7xNrV4S/lnM956iKFYLEnqwPKCJHVk0pWkjky60iYsIp6+kOc0PazpSpuwiLg8M/d+oOc0PTZ4GXBEfBRYZ+bOzDdvaIxpEBGPBv5PZt4VEc8E9gBOysyfNoq3I/AB4BGZ+fyIeDzw1Mw8vkGs100eNyKWA3+Sme+pHWs4frefbSLmw4GnUM7VVZl5U8NYDwIOBh7JxHssM4+pGOOpwNOAHSLirRPfWgEsrxVnnvi/BTw2M0+IiB2AbTLz+kaxdgBez9q/z9dWjvPS+b6fmX9fI06N8sKlwGXAlsDewP8evn4D2KLC8dcQEbdHxG3r+qodb8JpwL0R8RhKG8lK4O8axvtr4GzgEcPjfwLe0ijWARHx5YjYKSKeAFwEbNsoFvT92YiI/wBcArwUOAS4KCKqvmFn+QJwEHAPcOfEV01bANtQktC2E1+3UX7GZiLiaOCdwFHDU5sDf9sw5BeAXwG+Apw58VXbi4ev1wHHA4cPX38J1DtfKq5RvgjYbOLx5sBFDddEvxf4z5QTbQXwn4BjGsa7fPjv24HfH/58RcN4q2bHAK5sGO8w4FbgBuDpreJspJ/te8D2E4+3B77XMN63Wv7+ZsXaZfjvVh1jXgnErNfv6pbxev1sQ7xzgJ0mHu8EnF3r+DUn0h46JL8Z2wzPtXJgZh6bmbdn5m2Z+QnK6KKVuyPi5cCrgS8Nz23eMN6dEbE9Q+kmIvYDftYiUEQ8FjiSMpq/AXhVRGzVItag2882+Bfg9onHtw/PtfI/I+JJDY8/6RER8R3gWoCI2DMijm0c85dZstHM67d143hfiogXNI4xaWVm3jjx+GZg51oHr7m1438FroiI8ymfgs8A3l3x+LPdGRGHA5+lvPgvp/4l3KTXAG8A3p+Z10fErsDfNIz3VuCLwKMj4pvADrS7bDwDeFNmfiUiYoi9CnhCo3g9fzaA7wMXR8QXKOfKQcDVM7XQzPxQjSARcc1w/M2A10TEdcBdlPdDZuYeNeLM8mHguZTfJ5l5VUQ8o0GcSZ+LiE8BD4mI11MuvT/dMN6RwB9FxF3A3az+fa6Y/6+tt/Mi4mzgM8PjwyiljSqqdi8MkxX7Dg8vzraTFY8EPgI8nXKifxN4S2b+sFXMidgPpXwaXt04zmbAr1NOsu9l5t2N4qzIzNtmPbdbZv5Ti3jD8bv8bEOso+f7flaaMIyIXR4gzg014syKeXFm7hsRV2TmXsNzV2Vmk81qhw/lfwc8DngO5fU7OzPPbRFvY4mIl1AGjgDfyMzTqx17Q5NuRMzbmpKZl29QgCkREV8DDqSMYi4DbgG+mZlvne/vbUC85cALWXvGtsqobFasmW6CX8vM57XuJljHLPHPgGsy85YWMSdiPxT4adYcbawdYz/g25l5+/B4BbB7Zl7cINbngQ8BH6MMeI4EnpyZL6sdayLmNZnZvHwSEY/LzGvXlWNa5pbhA/Sxw9XfVsDymddzg49dIemeP8+3MzOftUEB1h13N+ATwI6Z+cSI2INS531fo3hXZOZew0z4ysw8OiKubnTJSER8GfgFcA1w38zztUZls2KdBZwA/HFm7jmMQq9o9caKiDOBpwIz584zKR9ku1ImQ6uUbSLiXcDnhjfug4CzKF019wCvyMxql4yz4l4B7D2T2CNiGXBpNuidjYhfpVzxPZsy6jwHeHNm/mvtWBMxTwQ+lpmrWsUY4hyXmUesI8e0zC2vp+yju11mPnqY8/hkZh5QJUCl2b5lNJ7xniPm1yl9l5MzqM1mjSnJbyfKSb3P8FzLGdtmx54jVu9ugrMpH5Yzj3ccntuu5msIfJvVA4sjgK9Relh3By5p+POt9btr9XoCz5/juTc0Pl+upXxw/QC4enhvdDtfW39RujO2mPV+uKbW8atMpGXmfRHxMWCvGsdboK0y85JSYrrfPQ3jHUNJDBdm5qqIeBSlH7mVsyLiOZl5TsMYM3p3E6zMzJsnHt8yPPevEVGztjszyw5lsukzmXkv8N1hNN/KdRHxZsqVGJTWxusaxfrTiLgrM78KEBFvB54FfLJRPCi/y24iYkvK7/C3KOfoBZSR5y8ahbwrM385k1uGc6VaOarmiXdeRBwM/P3Eid7SrcMqsZlEcQhw4/x/Zf1l5qnAqROPr6OsOmrlIuD04dK09Yxt726Cr0XEl1j9+zx4eG5roOYKv7si4omUlp/9gbdNfK9lS9wbgL8A/oRyfp5Hu9vMHEhpqXo78DzKBFfL1klymBCMiIdRFkW1dhKlze+jw+NXUDqHDm0U7+sR8UfAgyPitykJ/4xaB6/WvRARtwNbA/cCP6dxW8cw0jyOshTyJ8D1wOHZYIZ4iLclZaXKE5g40bLyUsSJeNdT3jzXtPoQi4h9gH/OzJuGT/P/SEmA3wHelY3qgsMM+EspIxcor9+OmfnGynH2BU6kfIh8ODPfOzz/AuBVmfnymvGGYy+nLA8/vPax54n5MEpL02XAa1sPeiLiQOCDlBWFtwC7AN/NzCYthhHxncx8/AM9VzHeMsp7fbI7o15L3Maun2xA3WX58N+tgW07xDuVsgruB5QFEucAH2kY7xvAssY/0+WUyQIo7TE/piTd9wKfbxx7L+DPgR9SJtTetLHPqYo/24XAFo1j3E5Z8jvz318Ad8w8bhz7KsqqviuGx/sDxzeM97fAfhOP96V8sLWKd8ysx8uBk2sdv1p5YRi9HA7smpnvjYiVlKV0l9SKMcv1EfGPwCnAVxvFmPSYzDw0Ig7KzBMj4u8otaVWrqNccp9FabAHqreMLc/Vo9nDgOMy8zTgtIi4smIc4P6Ok5cPX7dSXrvIzP1rx5oVd3vgaFbXBC+kvLFarUq7DvhmRHyRiQU7NV+7zGy5N8YDuTsz/yUilkXEssw8PyI+XDvIxGKTzSmr/H40PN6FYQVeIysj4qjM/LOI2AL4HGVyrYqaNd1jKa1Nz6KMlO4APg7sUzHGpMcBLwLeCBw/1Ag/m5kXNoo3M8Hz06FOeBPwsEaxoJRLrqfMolbfOGiwPCI2y8x7gANYs+7YYqLpWsoH1Ysy8/sAEfEHDeLM9lnKlcNMDf5wSsJ/dqN4Pxi+ltF246CZJv6vZubPhscPAZ6Zmf/QMOxPI2Ibyu/05Ii4hTarQV/U4JgL8VrKz3UUZRR/Vmb+j1oHr1nTvTwz9+61MmZW7IdSehUPz8wm29oN/bmnUbZ0PIGyt8S7MrPlLHFTEfHHwAsoo86dGXpLo+ykdmJmVt0MOyJ+B3gZZRXhP1KS4V9m5q4148wR91uZ+cRZz3Vp8G8tIq7MzN+Y9dz978HKsXbOzB8NE54/p3yoHE7ZAezkhlcOM/HXmLjLzB9VPv5kH/XmwKcoK12PH+JVWYxRM+leTJnUWjUk3x2Ac1q8+BMx/z3lsvh5lC0mTxkujzd5w+/vHaw9cVe1IXxoD9uJ8lrdOTy3G2V/1CYrfoY37UGUMsOzKLPTp2ej9riI+BBla8fPDU8dAjwlM9+27r+1QfG6vHZDrLUW6LT6QImJzdEj4rTMbNm9Mxm3y8Rdr4VeNZPu4ZQEuDdlxvgQykbYp877F9c/3g+BKyhvpC/OJIwGceZd5lu5xjoZ9xzKJfDbKC1Irwb+X2a+s0W8jWW4SjkUOCxrrfhZfezbKTXAYHVnDZSJkTuyXWdNt9cuIv6K0mb38eGpN1ImR3+vQazJq9gmo+l1xL2K8uH8lSyrQvcHXpmZr2sQaxlwaGaeUvvY98eolXShrJWm1AYDOC8zv1vt4GvHWmuTlkZxumyWMkfcyzLzNydHMhGxKjNb1chVSc/Xbrhq+FNW16fPBd7XYhAya6Tb7ZZAEXFpZj55SL57ZVmM1XJTn0sz88ktjg0VJ0siYjvK0P8zE89tnpV3j4qId2TmfwPeHxFrfWJk5dsDtUqqCzDze7sxIl5IaefabiP9WzZJsfE2TOn22g3J9b+0OPYc9oxyd5agLByYGfS03mqx18TdjK9ExNsoVyuT3SdV+tZrlhd+SLmFzU8oL8JDKDP8NwOvz8zLKsV5cWaeERGvnuv7mXlijThzxD0RODKHe6INl8UfzHaLI15EmelfSVmJswJ4T2Z+sUW8pSjm3jDl/hO+RY11iNv8tYuIj2XmmyLiDOZYopqZB9aKtbEME7o7Utq1JifudgHOrJVT5og7173eMjMfVeX4FZPupykN9WcPj59DadE5gbKIYN/5/v56xNu74Uhlrnhr1bB61rW0eBHxFOBHOezrPHxQH0xZkPHuWiOXiXhbUmq4j6FsAnP80I5XXUTclpkrhsnktWTm11vE7WloAz0qM6+Z9fyTgA9k5os3zr9sw9RMumvNmM7UtOZqa6kQ73zg4cDnKV0L36p5/DniXUXpf/zJ8Hg74Ou1Z4mjbEe4LpnDUlY9sIi4HHh2lo10nkFpUft9yvaOu2dm1f0lIuIUSmnhAuD5wA2ZeWTNGBOxlvwH/nx18NYtf0Mv/uNZs/vkpBrHrtkAf2NEvJNyYkPpZLg5ylr0+9b919ZPZu4f5U4Vvwt8KspG0adko/10KS0rF0XETNvRocD7G8SZq1a1NWUt+PaUhSdamK4r7oDHzySCiDie0qbWyuxbr6+hVVdNZw+Z53sPbhV0mDx/JiXpfpnyAXohpbVxg9W8MeUrKLfx+Ifha+fhueWUxFhdZt6UmX9BuaS7EphvlLihsU4CXkKpUd8MvDQrbbY9K84HZ74oG/o8mHJ/ts8CVWpKI7I8Vm/heABrLhdvseLu/knjVmWFCcspC3S2XcfXUnBplA3F1zAsVGpSzx0cQjlfbsrM1wB7UhaAVFHtxMvMWymXbnP5fq04MyJid8ro5WDKnV1PAf6wQZzZdbpPtn5DDaWLt1ImDU6krBT7ScuYS9RnKNv03UqZiLkA7p+gabFf8MzsPqw5w99idv/GzDym4vGm0Vso25sezuok+2TKsviXNIz786Et7Z7hCvoWyqRoFTVbxnajNIM/kjXv6dVkhhj4K8ro77mZ+eNGMaAkvck63e6Uk6GJiPhzyraHxwFPysw7WsVa6jLz/RFxHqtX3M1MYCxj3QOEDYnXZAn6OsQD/y+btiwb3T9tWAwxs4z7zBw2bG/o0ih7WHyakuzvAP5XrYPXnEi7irJb/WWsXvlDi7aOoU78N5n5itrHniPWNRN1us0ot3lp1hQeEfdRdhW7hzVbgVr3QmoTEhHb1e6+0Nqi3HV8RVa883fNutY9mfmJB/7fNlxm3hsRKyNii8z8ZeNwa9TpItoOMDKzZp1dS5QJt60od6ye3Aq0WtKtOdJ9N6X2cTpr7v/a6u4DJ1Eu9ZvtWTrEuXfi+EGZ2Po3HHlKS1JEHEuZw5lZXXsY8IOsdGeTmkm36SqOOeLNuSfCRly2K2kJiIhrKX3cM/dfXAZ8OzN3r3H8mt0LTfdEnSOeyVVSC9+ntLzO3G9xJRU7sGp2L2xFaXPaeVjv/ljg1zPzS7VizIp3PnOvOW/VLSFpCZvYx2Jb4LsRccnweF8qLnSpOZF2AqVz4WnD4/9LuZljk6TLmrfT3pLSr9u6IV3S0vXfewSpWdOd2fOy++16Jv4Nl2TmU3rFk7R0DQsjJtccVGkKqDnS/WVEPJjhkj8iHs1EF0Ntw6qtGcsoK1WqLdWTNE4RcQRwDOW29vcxdCpRaRl+zaR7NOVmgysj4mTKzQd/r+LxZ7uM1TXdeyjb9VW/fYek0Xk78MRha4PqanYvnDtspbcf5ZPhyBb/6IjYB/jnmW6JWXukfqd2PEmj8wNKL34TNWu6TweuzMw7I+KVlBtUfiQzb3iAv7rYOF33SJU0LhGxF6Ux4GLWXOhV5VZgNZPu1ZQt0Pag/IOPB343M+fc2X4D4tw/ORcRH6fcZfXdw+Pqm6VLGpehVexCyq6C9+8FXutWYLX3XsiIOAj4eGYeHxEtaqzLI2KzYXvFA4AjJr7XYo9USeOyeWauc4P4DVUzSd0eEUcBrwSeMSyd27zi8Wf03iNV0ricNXQwnEGDfWRqlhceTrlTxKrMvCAidqbcU6zKLS5mxdqP1Xuk3jk8txuwTc+bVUpaejaluwFvDfxi2HZxN+BxwFmZefcD/FVJGo2ae7d+A3hQRPwacA7wKuCvKx5fkpqJiHdM/PnQWd/7QK04NZNuZOa/UW41c2xmHsrqW2xI0rR72cSfj5r1vefVClI16UbEUyk3UzyzwfElqaVYx5/nerzeaibFIymfDqdn5rcj4lHA+RWPL0kt5Tr+PNfj9VZtIk2SNmUTt+aavC0Xw+MtM7NKC2zN7oUdgHcAT6Dsbwu4qbgkTapZXjgZuBbYFXgPZQOaVRWPL0mbvJoj3csy8zcj4urM3GN4blVm7lMlgCQtATWXAc8sgrgxIl4I/BjYbp7/X5JGp2bSfV9E/Arwh8BHgRXAH1Q8viRt8ja4vBARWwJvAB5D2Qrt+GEHMEnSLDWS7imU0sIFwPOBGzLzyAr/Nklacmok3Wsy80nDnzcDLsnMvWv84yRpqanRMnb/LmKWFSRpfjVGujOrOGDNlRxB2YNyxQYFkKQlxGXAktSRu4BJUkcmXUnqyKQrSR2ZdCWpI5OuJHX0/wFyztIjzhiiqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(df_train_raw.isnull(), \n",
    "            yticklabels=False, \n",
    "            cbar=False, \n",
    "            cmap='viridis')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Age, Cabin and Embarked have missing values\n",
    "\n",
    "Let's continue on by visualizing some more of the data! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a25ce26d0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAOIklEQVR4nO3dfayedX3H8feHFmSTCUjPWG2blSjRsExRG5/4YwZcBs4JccBwuFXXpFuCBuMWx7Y/Jotb9I/JUCdZM9RCNiniJh0xcwzs3Aw+HBRRIM7KJLSiPTwqOtnKvvvj/vXHsZzSm7bXfR8471dycq7rd1333e8hTd9c97kfUlVIkgRw2LQHkCQtHkZBktQZBUlSZxQkSZ1RkCR1y6c9wMFYsWJFrV27dtpjSNJTys0333xvVc0sdOwpHYW1a9cyOzs77TEk6SklyV37OubDR5KkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpO4p/YrmQ+GTX/7WtEfQInTWS5477RGkqfBKQZLUGQVJUmcUJEmdUZAkdUZBktQZBUlSZxQkSZ1RkCR1RkGS1BkFSVJnFCRJ3eBRSLIsyVeSXNf2T0jyhSTbk2xJckRbf0bb396Orx16NknST5rElcKFwB3z9t8LXFJVzwMeADa09Q3AA239knaeJGmCBo1CktXArwJ/2/YDnApc007ZDJzVts9s+7Tjp7XzJUkTMvSVwl8B7wT+r+0fBzxYVbvb/g5gVdteBdwN0I4/1M7/CUk2JplNMjs3Nzfk7JK05AwWhSSvA3ZV1c2H8n6ralNVrauqdTMzM4fyriVpyRvyQ3ZOAV6f5LXAkcCzgEuBY5Isb1cDq4Gd7fydwBpgR5LlwNHAfQPOJ0nay2BXClX1R1W1uqrWAucBN1bV+cBngLPbaeuBa9v21rZPO35jVdVQ80mSHm8ar1P4Q+AdSbYz+p3B5W39cuC4tv4O4KIpzCZJS9pEPqO5qrYB29r2ncDLFjjnx8A5k5hHkrQwX9EsSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpG6wKCQ5MskXk3w1yW1JLm7rJyT5QpLtSbYkOaKtP6Ptb2/H1w41myRpYUNeKTwCnFpVLwJOBk5P8grgvcAlVfU84AFgQzt/A/BAW7+knSdJmqDBolAjD7fdw9tXAacC17T1zcBZbfvMtk87flqSDDWfJOnxBv2dQpJlSW4BdgHXA98CHqyq3e2UHcCqtr0KuBugHX8IOG6B+9yYZDbJ7Nzc3JDjS9KSM2gUqurRqjoZWA28DHjBIbjPTVW1rqrWzczMHPSMkqTHTOTZR1X1IPAZ4JXAMUmWt0OrgZ1teyewBqAdPxq4bxLzSZJGhnz20UySY9r2TwG/DNzBKA5nt9PWA9e27a1tn3b8xqqqoeaTJD3e8v2fcsBWApuTLGMUn6ur6roktwNXJXk38BXg8nb+5cCVSbYD9wPnDTibJGkBg0Whqm4FXrzA+p2Mfr+w9/qPgXOGmkeStH++olmS1BkFSVJnFCRJnVGQJHVDPvtI0kG491//ftojaBFa8ZrfHPT+vVKQJHVGQZLUGQVJUmcUJEmdUZAkdUZBktQZBUlSZxQkSZ1RkCR1RkGS1BkFSVJnFCRJnVGQJHVjRSHJDeOsSZKe2p7wrbOTHAn8NLAiybFA2qFnAasGnk2SNGH7+zyF3wXeDjwHuJnHovB94IMDziVJmoInjEJVXQpcmuRtVfWBCc0kSZqSsT55rao+kORVwNr5t6mqKwaaS5I0BWNFIcmVwHOBW4BH23IBRkGSnkbG/YzmdcBJVVVDDiNJmq5xX6fwdeDnhhxEkjR9414prABuT/JF4JE9i1X1+kGmkiRNxbhReNeQQ0iSFodxn330b0MPIkmavnGfffQDRs82AjgCOBz4YVU9a6jBJEmTN+6Vws/s2U4S4EzgFUMNJUmajif9Lqk18kngVwaYR5I0ReM+fPSGebuHMXrdwo8HmUiSNDXjPvvo1+Zt7wa+zeghJEnS08i4v1N4y9CDSJKmb9wP2Vmd5B+T7Gpfn0iyeujhJEmTNe4vmj8CbGX0uQrPAf6prUmSnkbGjcJMVX2kqna3r48CMwPOJUmagnGjcF+SNyVZ1r7eBNw35GCSpMkbNwq/A5wLfBe4BzgbePMT3SDJmiSfSXJ7ktuSXNjWn53k+iTfbN+PbetJ8v4k25PcmuQlB/xTSZIOyLhR+DNgfVXNVNXPMorExfu5zW7g96vqJEavfr4gyUnARcANVXUicEPbBzgDOLF9bQQue1I/iSTpoI0bhRdW1QN7dqrqfuDFT3SDqrqnqr7ctn8A3AGsYvT6hs3ttM3AWW37TOCK9orpzwPHJFk59k8iSTpo40bhsD0P88DoISDGf+EbSdYyisgXgOOr6p526LvA8W17FXD3vJvtaGt739fGJLNJZufm5sYdQZI0hnH/Yf9L4KYkH2/75wB/Ps4NkxwFfAJ4e1V9f/R+eiNVVUme1Ed8VtUmYBPAunXr/HhQSTqExn1F8xVJZoFT29Ibqur2/d0uyeGMgvB3VfUPbfl7SVZW1T3t4aFdbX0nsGbezVe3NUnShIz9EFCLwH5DsEd7i+3LgTuq6n3zDm0F1gPvad+vnbf+1iRXAS8HHpr3MJMkaQLGjsIBOAX4LeBrSW5pa3/MKAZXJ9kA3MXoqa4AnwJeC2wHfgT4fkuSNGGDRaGq/gPIPg6ftsD5BVww1DySpP170h+yI0l6+jIKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSusGikOTDSXYl+fq8tWcnuT7JN9v3Y9t6krw/yfYktyZ5yVBzSZL2bcgrhY8Cp++1dhFwQ1WdCNzQ9gHOAE5sXxuBywacS5K0D4NFoao+C9y/1/KZwOa2vRk4a976FTXyeeCYJCuHmk2StLBJ/07h+Kq6p21/Fzi+ba8C7p533o629jhJNiaZTTI7Nzc33KSStARN7RfNVVVAHcDtNlXVuqpaNzMzM8BkkrR0TToK39vzsFD7vqut7wTWzDtvdVuTJE3QpKOwFVjfttcD185b/+32LKRXAA/Ne5hJkjQhy4e64yQfA14NrEiyA/hT4D3A1Uk2AHcB57bTPwW8FtgO/Ah4y1BzSZL2bbAoVNUb93HotAXOLeCCoWaRJI3HVzRLkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpM4oSJI6oyBJ6oyCJKkzCpKkzihIkjqjIEnqjIIkqVtUUUhyepJvJNme5KJpzyNJS82iiUKSZcBfA2cAJwFvTHLSdKeSpKVl0UQBeBmwvarurKr/Aa4CzpzyTJK0pCyf9gDzrALunre/A3j53icl2QhsbLsPJ/nGBGZbKlYA9057CGkB/t3szj8Ud/Lz+zqwmKIwlqraBGya9hxPR0lmq2rdtOeQ9ubfzclZTA8f7QTWzNtf3dYkSROymKLwJeDEJCckOQI4D9g65ZkkaUlZNA8fVdXuJG8FPg0sAz5cVbdNeaylxofltFj5d3NCUlXTnkGStEgspoePJElTZhQkSZ1RkG8vokUryYeT7Ery9WnPslQYhSXOtxfRIvdR4PRpD7GUGAX59iJatKrqs8D9055jKTEKWujtRVZNaRZJU2YUJEmdUZBvLyKpMwry7UUkdUZhiauq3cCetxe5A7jatxfRYpHkY8BNwPOT7EiyYdozPd35NheSpM4rBUlSZxQkSZ1RkCR1RkGS1BkFSVJnFCQgyZ8kuS3JrUluSfLyQ3Cfrz9U7zqb5OFDcT/S/viUVC15SV4JvA94dVU9kmQFcERVfWeM2y5vr/UYesaHq+qoof8cySsFCVYC91bVIwBVdW9VfSfJt1sgSLIuyba2/a4kVyb5HHBlks8n+YU9d5ZkWzv/zUk+mOToJHclOawdf2aSu5McnuS5Sf45yc1J/j3JC9o5JyS5KcnXkrx7wv89tIQZBQn+BViT5D+TfCjJL41xm5OA11TVG4EtwLkASVYCK6tqds+JVfUQcAuw535fB3y6qv6X0QfSv62qXgr8AfChds6lwGVV9YvAPQf9E0pjMgpa8qrqYeClwEZgDtiS5M37udnWqvrvtn01cHbbPhe4ZoHztwC/0bbPa3/GUcCrgI8nuQX4G0ZXLQCnAB9r21c+qR9IOgjLpz2AtBhU1aPANmBbkq8B64HdPPY/TkfudZMfzrvtziT3JXkho3/4f2+BP2Ir8BdJns0oQDcCzwQerKqT9zXWAf440gHzSkFLXpLnJzlx3tLJwF3Atxn9Aw7w6/u5my3AO4Gjq+rWvQ+2q5EvMXpY6LqqerSqvg/8V5Jz2hxJ8qJ2k88xuqIAOP/J/1TSgTEKEhwFbE5ye5JbGf2+4F3AxcClSWaBR/dzH9cw+kf86ic4ZwvwpvZ9j/OBDUm+CtzGYx+FeiFwQbtq8ZPwNDE+JVWS1HmlIEnqjIIkqTMKkqTOKEiSOqMgSeqMgiSpMwqSpO7/Ad6gRl1CIO2KAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='Survived', data=df_train_raw, palette='RdBu_r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a25de6090>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAX20lEQVR4nO3de5BV5bnn8e8jcCAGoon0OEibgUqMIoOgtJec3FBDwGRyMBcVczmoqRCjIVqTy0miliSBVE4V0YoazWCMaErj/cJYmXOiBqMejdqtQEB0RIMBDgktCooOpoFn/ujFsgMN7Ibeezf091O1a6/9rvWu/ey26Z/rXWu/KzITSZIA9ql3AZKknsNQkCSVDAVJUslQkCSVDAVJUqlvvQvYHYMHD85hw4bVuwxJ2qO0tLS8lJkNna3bo0Nh2LBhNDc317sMSdqjRMSL21vn8JEkqWQoSJJKhoIkqbRHn1OQpC3a2tpYsWIFGzZsqHcpPcaAAQNobGykX79+FfcxFCTtFVasWMGgQYMYNmwYEVHvcuouM1mzZg0rVqxg+PDhFfdz+EjSXmHDhg0ccMABBkIhIjjggAO6fORkKEjaaxgIf29Xfh6GgiSpZChIUhfNnDmTkSNHcsQRRzBmzBgee+yxepfUbXr9iea7nny+3iX0GCcf9Z56lyD1eI8++ij33HMPTz75JP379+ell17ib3/7W73L6jYeKUhSF6xatYrBgwfTv39/AAYPHsxBBx1ES0sLH/nIRxg7diwTJkxg1apVrFu3jkMPPZRnn30WgNNPP52rr766nuXvlKEgSV3wsY99jOXLl/O+972Pc845h9///ve0tbUxbdo0brvtNlpaWjjrrLO44IIL2G+//bjiiis444wzuOmmm3jllVf48pe/XO+PsEO9fvhIkrpi4MCBtLS08NBDDzFv3jxOO+00LrzwQhYtWsT48eMB2LRpE0OGDAFg/Pjx3HrrrZx77rksWLCgnqVXxFCQpC7q06cP48aNY9y4cYwaNYqf/exnjBw5kkcffXSbbTdv3sySJUvYd999eeWVV2hsbKxDxZVz+EiSuuDZZ5/lueeeK1/Pnz+fESNG0NraWoZCW1sbixcvBuDSSy9lxIgR3HjjjZx55pm0tbXVpe5KeaQgSV2wfv16pk2bxtq1a+nbty/vfe97mT17NlOnTuXrX/8669atY+PGjZx//vn07duXX/ziFzz++OMMGjSID3/4w8yYMYPvf//79f4Y22UoSFIXjB07lkceeWSb9sGDB/Pggw9u075kyZJy+ZJLLqlqbd2hasNHETEgIh6PiAURsTgivl+0z4mIP0XE/OIxpmiPiLgsIpZGxMKIOKpatUmSOlfNI4U3gRMyc31E9AMejoj/U6z7VmbettX2JwGHFI9jgauKZ0lSjVTtSCHbrS9e9iseuYMuk4Dri35/APaPiCHVqk+StK2qXn0UEX0iYj6wGrg3M7dMEDKzGCK6NCL6F21DgeUduq8o2rbe59SIaI6I5tbW1mqWL0m9TlVDITM3ZeYYoBE4JiL+O/Bd4DDgaOBdwL90cZ+zM7MpM5saGhq6vWZJ6s1q8j2FzFwLzAMmZuaqYojoTeBa4Jhis5XAwR26NRZtkqQaqdqJ5ohoANoyc21EvA0YD/xrRAzJzFXRfveHk4FFRZe5wNci4ibaTzCvy8xV1apP0t6tu2dAruYswg888ACzZs3innvuqdp7VKqaVx8NAa6LiD60H5Hckpn3RMTvisAIYD5wdrH9b4CPA0uBN4Azq1ibJKkTVQuFzFwIHNlJ+wnb2T6Bc6tVjyRV07Jly5g4cSLHHXccjzzyCEcffTRnnnkmF198MatXr+aGG24A4LzzzmPDhg287W1v49prr+XQQw/9u/28/vrrTJs2jUWLFtHW1sb06dOZNGlSzT6Hcx9JUjdZunQp3/jGN3jmmWd45plnuPHGG3n44YeZNWsWP/rRjzjssMN46KGHeOqpp/jBD37A9773vW32MXPmTE444QQef/xx5s2bx7e+9S1ef/31mn0Gp7mQpG4yfPhwRo0aBcDIkSM58cQTiQhGjRrFsmXLWLduHVOmTOG5554jIjqdHO+3v/0tc+fOZdasWQBs2LCBP//5z4wYMaImn8FQkKRusuVubAD77LNP+XqfffZh48aNXHTRRRx//PHceeedLFu2jHHjxm2zj8zk9ttv32ZYqVYcPpKkGlm3bh1Dh7Z/J3fOnDmdbjNhwgQuv/xy2k+zwlNPPVWr8gCPFCTtpap5Cemu+va3v82UKVOYMWMGn/jEJzrd5qKLLuL888/niCOOYPPmzQwfPryml6rGljTaEzU1NWVzc/Nu7aO7r2Xek/XEf0RSpZYsWVKzcfc9SWc/l4hoycymzrZ3+EiSVDIUJEklQ0GSVDIUJEklQ0GSVDIUJEklv6cgaa/00n03duv+Bn/0czvd5rLLLuOqq67iqKOOKifA607Tp09n4MCBfPOb3+z2fW9hKEhSN7nyyiu57777aGxsrHcpu8zhI0nqBmeffTYvvPACJ510EjNnzuSss87imGOO4cgjj+Tuu+8G2qe2OPnkkxk/fjzDhg3jiiuu4JJLLuHII4/kuOOO4+WXXwbg6quv5uijj2b06NF85jOf4Y033tjm/Z5//nkmTpzI2LFj+dCHPsQzzzzTLZ/DUJCkbvDzn/+cgw46iHnz5vH6669vd/rrRYsWcccdd/DEE09wwQUXsO+++/LUU0/x/ve/n+uvvx6AT3/60zzxxBMsWLCAESNGcM0112zzflOnTuXyyy+npaWFWbNmcc4553TL53D4SJK62famvwY4/vjjGTRoEIMGDWK//fbjk5/8JACjRo1i4cKFQHtwXHjhhaxdu5b169czYcKEv9v/+vXreeSRRzjllFPKtjfffLNbajcUJKmbbW/668cee2yn02sDnHHGGdx1112MHj2aOXPm8MADD/zdfjZv3sz+++/P/Pnzu712h48kqZvt7vTXr732GkOGDKGtra3Tq5je8Y53MHz4cG699VagPYQWLFiw+4VTxSOFiBgAPAj0L97ntsy8OCKGAzcBBwAtwBcz828R0R+4HhgLrAFOy8xl1apP0t6tkktIq2V3p7/+4Q9/yLHHHktDQwPHHnssr7322jbb3HDDDXz1q19lxowZtLW1MXnyZEaPHr3btVdt6uyICODtmbk+IvoBDwPnAf8TuCMzb4qInwMLMvOqiDgHOCIzz46IycCnMvO0Hb2HU2d3L6fO1p7MqbM712Omzs5264uX/YpHAicAtxXt1wEnF8uTitcU608sgkWSVCNVPacQEX0iYj6wGrgXeB5Ym5kbi01WAEOL5aHAcoBi/Trah5i23ufUiGiOiObW1tZqli9JvU5VQyEzN2XmGKAROAY4rBv2OTszmzKzqaGhYbdrlLT32JPvJFkNu/LzqMnVR5m5FpgHvB/YPyK2nOBuBFYWyyuBgwGK9fvRfsJZknZqwIABrFmzxmAoZCZr1qxhwIABXepXzauPGoC2zFwbEW8DxgP/Sns4fJb2K5CmAHcXXeYWrx8t1v8u/a8rqUKNjY2sWLECh5XfMmDAgC7Pw1TNL68NAa6LiD60H5Hckpn3RMTTwE0RMQN4Ctjy/e1rgF9FxFLgZWByFWuTtJfp168fw4cPr3cZe7yqhUJmLgSO7KT9BdrPL2zdvgE4Zet2SVLt+I1mSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVLJUJAklQwFSVKpaqEQEQdHxLyIeDoiFkfEeUX79IhYGRHzi8fHO/T5bkQsjYhnI2JCtWqTJHWubxX3vRH4RmY+GRGDgJaIuLdYd2lmzuq4cUQcDkwGRgIHAfdFxPsyc1MVa5QkdVC1I4XMXJWZTxbLrwFLgKE76DIJuCkz38zMPwFLgWOqVZ8kaVs1OacQEcOAI4HHiqavRcTCiPhlRLyzaBsKLO/QbQWdhEhETI2I5ohobm1trWLVktT7VD0UImIgcDtwfma+ClwFvAcYA6wCftKV/WXm7MxsysymhoaGbq9XknqzqoZCRPSjPRBuyMw7ADLzr5m5KTM3A1fz1hDRSuDgDt0bizZJUo1U8+qjAK4BlmTmJR3ah3TY7FPAomJ5LjA5IvpHxHDgEODxatUnSdpWNa8++gDwReCPETG/aPsecHpEjAESWAZ8BSAzF0fELcDTtF+5dK5XHklSbVUtFDLzYSA6WfWbHfSZCcysVk2SpB3zG82SpJKhIEkqGQqSpJKhIEkqGQqSpJKhIEkqGQqSpJKhIEkqGQqSpJKhIEkqGQqSpJKhIEkqGQqSpJKhIEkqVRQKEXF/JW2SpD3bDu+nEBEDgH2BwRHxTt66P8I7gKFVrk2SVGM7u8nOV4DzgYOAFt4KhVeBK6pYlySpDnYYCpn5U+CnETEtMy+vUU2SpDqp6HacmXl5RPwjMKxjn8y8vkp1SZLqoNITzb8CZgEfBI4uHk076XNwRMyLiKcjYnFEnFe0vysi7o2I54rndxbtERGXRcTSiFgYEUft1ieTJHVZRUcKtAfA4ZmZXdj3RuAbmflkRAwCWiLiXuAM4P7M/HFEfAf4DvAvwEnAIcXjWOCq4lmSVCOVfk9hEfBfu7LjzFyVmU8Wy68BS2i/YmkScF2x2XXAycXyJOD6bPcHYP+IGNKV95Qk7Z5KjxQGA09HxOPAm1saM/OfKukcEcOAI4HHgAMzc1Wx6i/AgcXyUGB5h24rirZVHdqIiKnAVIB3v/vdFZYvSapEpaEwfVffICIGArcD52fmqxFRrsvMjIiuDEmRmbOB2QBNTU1d6itJ2rFKrz76/a7sPCL60R4IN2TmHUXzXyNiSGauKoaHVhftK4GDO3RvLNokSTVS6dVHr0XEq8VjQ0RsiohXd9IngGuAJZl5SYdVc4EpxfIU4O4O7f9cXIV0HLCuwzCTJKkGKj1SGLRlufhjPwk4bifdPgB8EfhjRMwv2r4H/Bi4JSK+BLwInFqs+w3wcWAp8AZwZoWfQdJe7qX7bqx3CT3G4I9+rqr7r/ScQqm4LPWuiLiY9stJt7fdw7w1LcbWTtzOfs/taj2SpO5TUShExKc7vNyH9u8tbKhKRZKkuqn0SOGTHZY3AstoH0KSJO1FKj2n4Pi+JPUClV591BgRd0bE6uJxe0Q0Vrs4SVJtVTrNxbW0XzJ6UPH430WbJGkvUmkoNGTmtZm5sXjMARqqWJckqQ4qDYU1EfGFiOhTPL4ArKlmYZKk2qs0FM6i/Utmf6F9grrP0j4FtiRpL1LpJak/AKZk5ivQfqMc2m+6c1a1CpMk1V6lRwpHbAkEgMx8mfapsCVJe5FKQ2GfLbfNhPJIoctTZEiSerZK/7D/BHg0Im4tXp8CzKxOSZKkeqn0G83XR0QzcELR9OnMfLp6ZUmS6qHiIaAiBAwCSdqLVXpOQZLUCxgKkqSSoSBJKhkKkqSSoSBJKlUtFCLil8W9FxZ1aJseESsjYn7x+HiHdd+NiKUR8WxETKhWXZKk7avmkcIcYGIn7Zdm5pji8RuAiDgcmAyMLPpcGRF9qlibJKkTVQuFzHwQeLnCzScBN2Xmm5n5J2ApcEy1apMkda4e5xS+FhELi+GlLfMpDQWWd9hmRdG2jYiYGhHNEdHc2tpa7VolqVepdShcBbwHGEP7fRl+0tUdZObszGzKzKaGBm/+JkndqaahkJl/zcxNmbkZuJq3hohWAgd32LSxaJMk1VBNQyEihnR4+Slgy5VJc4HJEdE/IoYDhwCP17I2SVIV74kQEb8GxgGDI2IFcDEwLiLGAAksA74CkJmLI+IW2ifc2wicm5mbqlWbJKlzVQuFzDy9k+ZrdrD9TLxHgyTVld9oliSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVqhYKEfHLiFgdEYs6tL0rIu6NiOeK53cW7RERl0XE0ohYGBFHVasuSdL2VfNIYQ4wcau27wD3Z+YhwP3Fa4CTgEOKx1TgqirWJUnajqqFQmY+CLy8VfMk4Lpi+Trg5A7t12e7PwD7R8SQatUmSepcrc8pHJiZq4rlvwAHFstDgeUdtltRtG0jIqZGRHNENLe2tlavUknqhep2ojkzE8hd6Dc7M5sys6mhoaEKlUlS71XrUPjrlmGh4nl10b4SOLjDdo1FmySphmodCnOBKcXyFODuDu3/XFyFdBywrsMwkySpRvpWa8cR8WtgHDA4IlYAFwM/Bm6JiC8BLwKnFpv/Bvg4sBR4AzizWnVJkravaqGQmadvZ9WJnWybwLnVqkWSVBm/0SxJKhkKkqRS1YaPJO2eu558vt4l9BgfrHcBvYhHCpKkkqEgSSoZCpKkkqEgSSoZCpKkkqEgSSoZCpKkkqEgSSoZCpKkkqEgSSoZCpKkkqEgSSo5IZ5KL913Y71L6DEGf/Rz9S5BqguPFCRJJUNBklQyFCRJpbqcU4iIZcBrwCZgY2Y2RcS7gJuBYcAy4NTMfKUe9UlSb1XPI4XjM3NMZjYVr78D3J+ZhwD3F68lSTXUk4aPJgHXFcvXASfXsRZJ6pXqFQoJ/DYiWiJiatF2YGauKpb/AhzYWceImBoRzRHR3NraWotaJanXqNf3FD6YmSsj4r8A90bEMx1XZmZGRHbWMTNnA7MBmpqaOt1GkrRr6nKkkJkri+fVwJ3AMcBfI2IIQPG8uh61SVJvVvNQiIi3R8SgLcvAx4BFwFxgSrHZFODuWtcmSb1dPYaPDgTujIgt739jZv5bRDwB3BIRXwJeBE6tQ22S1KvVPBQy8wVgdCfta4ATa12PJOktPemSVElSnRkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKvW4UIiIiRHxbEQsjYjv1LseSepNelQoREQf4GfAScDhwOkRcXh9q5Kk3qNHhQJwDLA0M1/IzL8BNwGT6lyTJPUafetdwFaGAss7vF4BHNtxg4iYCkwtXq6PiGdrVFtvMBh4qd5F9Ayfr3cB+nv+bpa65Xfzv21vRU8LhZ3KzNnA7HrXsTeKiObMbKp3HdLW/N2snZ42fLQSOLjD68aiTZJUAz0tFJ4ADomI4RHxD8BkYG6da5KkXqNHDR9l5saI+Brw70Af4JeZubjOZfUmDsupp/J3s0YiM+tdgySph+hpw0eSpDoyFCRJJUNBTi2iHisifhkRqyNiUb1r6S0MhV7OqUXUw80BJta7iN7EUJBTi6jHyswHgZfrXUdvYiios6lFhtapFkl1ZihIkkqGgpxaRFLJUJBTi0gqGQq9XGZuBLZMLbIEuMWpRdRTRMSvgUeBQyNiRUR8qd417e2c5kKSVPJIQZJUMhQkSSVDQZJUMhQkSSVDQZJUMhQkICIuiIjFEbEwIuZHxLHdsM9/6q5ZZyNifXfsR9oZL0lVrxcR7wcuAcZl5psRMRj4h8z8zwr69i2+61HtGtdn5sBqv4/kkYIEQ4CXMvNNgMx8KTP/MyKWFQFBRDRFxAPF8vSI+FVE/Afwq4j4Q0SM3LKziHig2P6MiLgiIvaLiBcjYp9i/dsjYnlE9IuI90TEv0VES0Q8FBGHFdsMj4hHI+KPETGjxj8P9WKGggS/BQ6OiP8bEVdGxEcq6HM48NHMPB24GTgVICKGAEMys3nLhpm5DpgPbNnv/wD+PTPbaL8h/bTMHAt8E7iy2OanwFWZOQpYtdufUKqQoaBeLzPXA2OBqUArcHNEnLGTbnMz8/8Vy7cAny2WTwVu62T7m4HTiuXJxXsMBP4RuDUi5gP/i/ajFoAPAL8uln/VpQ8k7Ya+9S5A6gkycxPwAPBARPwRmAJs5K3/cRqwVZfXO/RdGRFrIuII2v/wn93JW8wFfhQR76I9gH4HvB1Ym5ljtlfWLn4caZd5pKBeLyIOjYhDOjSNAV4EltH+BxzgMzvZzc3At4H9MnPh1iuLo5EnaB8WuiczN2Xmq8CfIuKUoo6IiNFFl/+g/YgC4PNd/1TSrjEUJBgIXBcRT0fEQtrPF0wHvg/8NCKagU072cdttP8Rv2UH29wMfKF43uLzwJciYgGwmLduhXoecG5x1OKd8FQzXpIqSSp5pCBJKhkKkqSSoSBJKhkKkqSSoSBJKhkKkqSSoSBJKv1/dFQUQ2fViwMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='Survived', \n",
    "              hue='Sex', \n",
    "              data=df_train_raw,\n",
    "              palette='RdBu_r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a25e69b90>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEICAYAAACwDehOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAVmklEQVR4nO3df7RXdZ3v8ec7JPEGZsbRiCNBLSZBheN4Eh1/VHq7oywbSpRRSaG4MbPGjJYzzmqstUbNXONKm5x+aLh0QJa/sGnK1HQcjdtkqUEQAd5GMlgchhRwSLyKAb3vH9/N9oQH+B7Od5/vOZznY63vOnt/9t6f/T4u1nn52fuz9zcyE0mSAN7U7AIkSX2HoSBJKhkKkqSSoSBJKhkKkqSSoSBJKlUWChExJCKejoifR8TKiLi6aB8TEU9FxOqIuDci3ly0H1ysry62j66qNklS16Kq5xQiIoC3ZObLETEY+BEwB7gc+HZm3hMRtwA/z8ybI+KvgAmZ+ZcRcQHw0cz8872dY/jw4Tl69OhK6pekA9WSJUs2ZWZLV9sOquqkWUubl4vVwcUngTOAi4r2+cBVwM3AlGIZ4FvA1yIici+pNXr0aBYvXtzw2iXpQBYRa/e0rdJ7ChExKCKWAS8AjwK/ArZk5o5ilw5gZLE8ElgHUGz/LfD2KuuTJP2hSkMhM3dmZhvQCpwIHN3TPiNidkQsjojFGzdu7HGNkqTX9crso8zcAvwAOBk4LCJ2XbZqBdYXy+uBowCK7W8FNnfR19zMbM/M9paWLi+JSZL2U2X3FCKiBdiemVsi4hDgQ8D11MLhPOAeYAbw3eKQ+4v1nxTbH9/b/QRJaqbt27fT0dHBtm3bml3KHg0ZMoTW1lYGDx5c9zGVhQIwApgfEYOojUgWZuYDEbEKuCcirgWWArcV+98GLIiI1cCLwAUV1iZJPdLR0cGwYcMYPXo0tcmWfUtmsnnzZjo6OhgzZkzdx1U5+2g5cHwX7c9Ru7+we/s24Pyq6pGkRtq2bVufDQSAiODtb3873b336hPNkrSf+mog7LI/9RkKkqSSoSBJDTRo0CDa2to49thjOf/883nllVf2uO9VV13FDTfc0IvV7VuVN5olADZ974pml9Btwz/8pWaXoH7qkEMOYdmyZQBMnz6dW265hcsvv7zJVdXPkYIkVeS0005j9erVANxxxx1MmDCBiRMncvHFF79h31tvvZX3ve99TJw4kalTp5YjjPvuu49jjz2WiRMncvrppwOwcuVKTjzxRNra2pgwYQLPPvtsw2p2pCBJFdixYwff//73Oeuss1i5ciXXXnstP/7xjxk+fDgvvvjiG/Y/99xz+eQnPwnA5z//eW677TYuu+wyrrnmGh555BFGjhzJli1bALjllluYM2cO06dP53e/+x07d+5sWN2OFCSpgV599VXa2tpob29n1KhRzJo1i8cff5zzzz+f4cOHA3D44Ye/4bgVK1Zw2mmncdxxx3HnnXeycuVKAE455RRmzpzJrbfeWv7xP/nkk7nuuuu4/vrrWbt2LYccckjD6nekIEkN1PmeQnfMnDmT73znO0ycOJF58+axaNEioDYqeOqpp3jwwQc54YQTWLJkCRdddBGTJk3iwQcfZPLkyXzzm9/kjDPOaEj9jhQkqWJnnHEG9913H5s3117n1tXlo61btzJixAi2b9/OnXfeWbb/6le/YtKkSVxzzTW0tLSwbt06nnvuOd797nfz6U9/milTprB8+fKG1epIQZIqdswxx/C5z32O97///QwaNIjjjz+eefPm/cE+X/jCF5g0aRItLS1MmjSJrVu3AnDFFVfw7LPPkpmceeaZTJw4keuvv54FCxYwePBg3vGOd3DllVc2rNbKvnmtN7S3t6dfstP3OSVVB6JnnnmGcePGNbuMfeqqzohYkpntXe3v5SNJUslQkCSVDAVJUslQkCSVDAVJUslQkCSVfE5BkhrgXXctbWh/ay96wxdXvsEnPvEJHnjgAY444ghWrFjRkPM6UpCkfmrmzJk8/PDDDe3TUJCkfur000/v8uV6PWEoSJJKhoIkqWQoSJJKhoIkqeSUVElqgHqmkDbahRdeyKJFi9i0aROtra1cffXVzJo1q0d9GgqS1E/dfffdDe/Ty0eSpFJloRARR0XEDyJiVUSsjIg5RftVEbE+IpYVn8mdjvm7iFgdEb+MiD+tqjZJUteqvHy0A/jrzPxZRAwDlkTEo8W2f8zMGzrvHBHjgQuAY4B3Av8eEX+UmTsrrFGS1EllI4XM3JCZPyuWtwLPACP3csgU4J7MfC0zfw2sBk6sqj5J0hv1yj2FiBgNHA88VTR9KiKWR8TtEfG2om0ksK7TYR3sPUQkSQ1WeShExFDgX4DPZOZLwM3Ae4A2YANwYzf7mx0RiyNi8caNGxteryQNZJVOSY2IwdQC4c7M/DZAZj7fafutwAPF6nrgqE6HtxZtfyAz5wJzAdrb27OayiWpe27a0ti3lc457Ky9bl+3bh2XXHIJzz//PBHB7NmzmTNnTo/PW+XsowBuA57JzC93ah/RabePArteAn4/cEFEHBwRY4CxwNNV1SdJ/dlBBx3EjTfeyKpVq3jyySf5+te/zqpVq3rebwNq25NTgIuBX0TEsqLtSuDCiGgDElgD/AVAZq6MiIXAKmozly515pEkdW3EiBGMGFH7f+xhw4Yxbtw41q9fz/jx43vUb2WhkJk/AqKLTQ/t5ZgvAl+sqiZJOhCtWbOGpUuXMmnSpB735RPNktSPvfzyy0ydOpWvfOUrHHrooT3uz1CQpH5q+/btTJ06lenTp3Puuec2pE9DQZL6ocxk1qxZjBs3jssvv7xh/fqWVElqgH1NIW20J554ggULFnDcccfR1tYGwHXXXcfkyZP3ceTeGQqS1A+deuqpZDb+US0vH0mSSoaCJKlkKEiSSoaCJKlkKEiSSoaCJKnklFRJaoBN37uiof0N//CX9rp927ZtnH766bz22mvs2LGD8847j6uvvrrH5zUUJKkfOvjgg3n88ccZOnQo27dv59RTT+Xss8/mpJNO6lG/Xj6SpH4oIhg6dChQewfS9u3bqX2NTc8YCpLUT+3cuZO2tjaOOOIIPvShD/nqbEkayAYNGsSyZcvo6Ojg6aefZsWKFfs+aB8MBUnq5w477DA++MEP8vDDPf+eaENBkvqhjRs3smXLFgBeffVVHn30UY4++uge9+vsI0lqgH1NIW20DRs2MGPGDHbu3Mnvf/97pk2bxjnnnNPjfg0FSeqHJkyYwNKlSxver5ePJEklQ0GSVDIUJGk/VfHNZ420P/UZCpK0H4YMGcLmzZv7bDBkJps3b2bIkCHdOs4bzZK0H1pbW+no6GDjxo3NLmWPhgwZQmtra7eOMRQkaT8MHjyYMWPGNLuMhvPykSSpVFkoRMRREfGDiFgVESsjYk7RfnhEPBoRzxY/31a0R0T8U0SsjojlEfHHVdUmSepalSOFHcBfZ+Z44CTg0ogYD3wWeCwzxwKPFesAZwNji89s4OYKa5MkdaGyUMjMDZn5s2J5K/AMMBKYAswvdpsPfKRYngLckTVPAodFxIiq6pMkvVGv3FOIiNHA8cBTwJGZuaHY9BvgyGJ5JLCu02EdRdvufc2OiMURsbgv3/WXpP6o8lCIiKHAvwCfycyXOm/L2gTfbk3yzcy5mdmeme0tLS0NrFSSVGkoRMRgaoFwZ2Z+u2h+ftdloeLnC0X7euCoToe3Fm2SpF5S5eyjAG4DnsnML3fadD8wo1ieAXy3U/slxSykk4DfdrrMJEnqBVU+vHYKcDHwi4hYVrRdCfwDsDAiZgFrgWnFtoeAycBq4BXg4xXWJknqQmWhkJk/AmIPm8/sYv8ELq2qHknSvvlEsySpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkp1hUJEPFZPmySpf9vrdzRHxBDgfwDDI+JtvP6dy4cCIyuuTZLUy/YaCsBfAJ8B3gks4fVQeAn4WoV1SZKaYK+hkJk3ATdFxGWZ+dVeqkmS1CT7GikAkJlfjYg/AUZ3PiYz76ioLklSE9QVChGxAHgPsAzYWTQnYChI0gGkrlAA2oHxmZlVFiNJaq56n1NYAbyjykIkSc1X70hhOLAqIp4GXtvVmJl/VklVkqSmqDcUrupuxxFxO3AO8EJmHlu0XQV8EthY7HZlZj5UbPs7YBa1exafzsxHuntOSVLP1Dv76P/sR9/zqD3LsPvN6H/MzBs6N0TEeOAC4Bhqz0T8e0T8UWbuRJLUa+p9zcXWiHip+GyLiJ0R8dLejsnMHwIv1lnHFOCezHwtM38NrAZOrPNYSVKD1BUKmTksMw/NzEOBQ4CpwDf285yfiojlEXF78eoMqL0yY12nfTrYw2s0ImJ2RCyOiMUbN27sahdJ0n7q9ltSs+Y7wJ/ux/lupva8QxuwAbhxP84/NzPbM7O9paVlP0qQJO1JvQ+vndtp9U3UnlvY1t2TZebznfq8FXigWF0PHNVp19aiTZLUi+qdffThTss7gDXU7gN0S0SMyMwNxepHqT3/AHA/cFdEfJnajeaxwNPd7V+S1DP1zj76eHc7joi7gQ9Qe+12B/D3wAcioo3aKzLWUHsLK5m5MiIWAquohc6lzjySpN5X7+WjVuCrwClF038AczKzY0/HZOaFXTTftpf9vwh8sZ56JEnVqPdG8z9Tu8TzzuLzvaJNknQAqTcUWjLznzNzR/GZBzj1R5IOMPWGwuaI+FhEDCo+HwM2V1mYJKn31RsKnwCmAb+h9nzBecDMimqSJDVJvVNSrwFmZOZ/A0TE4cAN1MJCknSAqHekMGFXIABk5ovA8dWUJElqlnpD4U2d3lO0a6RQ7yhDktRP1PuH/UbgJxFxX7F+Pj5TIEkHnHqfaL4jIhYDZxRN52bmqurKkiQ1Q92XgIoQMAgk6QDW7VdnS5IOXIaCJKlkKEiSSoaCJKlkKEiSSoaCJKlkKEiSSoaCJKlkKEiSSoaCJKlkKEiSSoaCJKlkKEiSSoaCJKnkt6dJfdS77lra7BK6be1Ffktvf+dIQZJUMhQkSaXKQiEibo+IFyJiRae2wyPi0Yh4tvj5tqI9IuKfImJ1RCyPiD+uqi5J0p5VOVKYB5y1W9tngccycyzwWLEOcDYwtvjMBm6usC5J0h5UFgqZ+UPgxd2apwDzi+X5wEc6td+RNU8Ch0XEiKpqkyR1rbdnHx2ZmRuK5d8ARxbLI4F1nfbrKNo2sJuImE1tNMGoUaOqq1TSgLDpe1c0u4RuGf7hL1Xaf9NuNGdmArkfx83NzPbMbG9paamgMkkauHo7FJ7fdVmo+PlC0b4eOKrTfq1FmySpF/V2KNwPzCiWZwDf7dR+STEL6STgt50uM0mSekll9xQi4m7gA8DwiOgA/h74B2BhRMwC1gLTit0fAiYDq4FXgI9XVZckac8qC4XMvHAPm87sYt8ELq2qFklSfXyiWZJUMhQkSSVDQZJUMhQkSSVDQZJUMhQkSSVDQZJUMhQkSSVDQZJUMhQkSSVDQZJUMhQkSSVDQZJU6u2v41QD3LTl4WaX0C3Tm12ApLo5UpAklRwpSGqY/jaKBUeyuxvwofCuu5Y2u4Ruu3xysyuQdKDy8pEkqWQoSJJKhoIkqWQoSJJKhoIkqWQoSJJKhoIkqWQoSJJKhoIkqdSUJ5ojYg2wFdgJ7MjM9og4HLgXGA2sAaZl5n83oz5JGqiaOVL4YGa2ZWZ7sf5Z4LHMHAs8VqxLknpRX7p8NAWYXyzPBz7SxFokaUBqVigk8G8RsSQiZhdtR2bmhmL5N8CRzSlNkgauZr0l9dTMXB8RRwCPRsT/7bwxMzMisqsDixCZDTBq1KjqK5WkAaQpI4XMXF/8fAH4V+BE4PmIGAFQ/HxhD8fOzcz2zGxvaWnprZIlaUDo9VCIiLdExLBdy8D/AlYA9wMzit1mAN/t7dokaaBrxuWjI4F/jYhd578rMx+OiJ8CCyNiFrAWmNaE2iRpQOv1UMjM54CJXbRvBs7s7XokSa/rS1NSJUlNZihIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSpZChIkkqGgiSp1OdCISLOiohfRsTqiPhss+uRpIGkT4VCRAwCvg6cDYwHLoyI8c2tSpIGjj4VCsCJwOrMfC4zfwfcA0xpck2SNGD0tVAYCazrtN5RtEmSekFkZrNrKEXEecBZmfm/i/WLgUmZ+alO+8wGZher7wV+2euFHriGA5uaXYTUBf9tNta7MrOlqw0H9XYl+7AeOKrTemvRVsrMucDc3ixqoIiIxZnZ3uw6pN35b7P39LXLRz8FxkbEmIh4M3ABcH+Ta5KkAaNPjRQyc0dEfAp4BBgE3J6ZK5tcliQNGH0qFAAy8yHgoWbXMUB5WU59lf82e0mfutEsSWquvnZPQZLURIaCfLWI+qyIuD0iXoiIFc2uZaAwFAY4Xy2iPm4ecFazixhIDAX5ahH1WZn5Q+DFZtcxkBgK8tUikkqGgiSpZChon68WkTRwGAry1SKSSobCAJeZO4BdrxZ5Bljoq0XUV0TE3cBPgPdGREdEzGp2TQc6n2iWJJUcKUiSSoaCJKlkKEiSSoaCJKlkKEiSSoaCBETE5yJiZUQsj4hlETGpAX3+WaPeOhsRLzeiH2lfnJKqAS8iTga+DHwgM1+LiOHAmzPzv+o49qDiWY+qa3w5M4dWfR7JkYIEI4BNmfkaQGZuysz/iog1RUAQEe0RsahYvioiFkTEE8CCiHgyIo7Z1VlELCr2nxkRX4uIt0bE2oh4U7H9LRGxLiIGR8R7IuLhiFgSEf8REUcX+4yJiJ9ExC8i4tpe/u+hAcxQkODfgKMi4j8j4hsR8f46jhkP/M/MvBC4F5gGEBEjgBGZuXjXjpn5W2AZsKvfc4BHMnM7te8eviwzTwD+BvhGsc9NwM2ZeRywoce/oVQnQ0EDXma+DJwAzAY2AvdGxMx9HHZ/Zr5aLC8EziuWpwHf6mL/e4E/L5YvKM4xFPgT4L6IWAZ8k9qoBeAU4O5ieUG3fiGpBw5qdgFSX5CZO4FFwKKI+AUwA9jB6//jNGS3Q/5fp2PXR8TmiJhA7Q//X3ZxivuB6yLicGoB9DjwFmBLZrbtqaz9/HWk/eZIQQNeRLw3IsZ2amoD1gJrqP0BB5i6j27uBf4WeGtmLt99YzEa+Sm1y0IPZObOzHwJ+HVEnF/UERExsTjkCWojCoDp3f+tpP1jKEgwFJgfEasiYjm1+wVXAVcDN0XEYmDnPvr4FrU/4gv3ss+9wMeKn7tMB2ZFxM+Blbz+VahzgEuLUYvfhKde45RUSVLJkYIkqWQoSJJKhoIkqWQoSJJKhoIkqWQoSJJKhoIkqWQoSJJK/x/zL+l10qhC/QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='Survived',\n",
    "              hue='Pclass', \n",
    "              data=df_train_raw,\n",
    "              palette='rainbow')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a25edf510>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXwV1fn48c+TnRAISxLWQAIBlLATWcQFcQNEsRUV6oIrWqW22laxv+rX2tpWq6VW0X4RFXEDi6Ui0CAoKFJEgoCAGA1hSZAlLAkQCNme3x934BtjQm7CvZnk3uf9euWVmTNnzn3mdZP73Dkzc46oKsYYY4JPiNsBGGOMcYclAGOMCVKWAIwxJkhZAjDGmCBlCcAYY4JUmNsB1EZcXJwmJSW5HYYxxjQqa9eu3a+q8ZXLG1UCSEpKIiMjw+0wjDGmURGRHVWVWxeQMcYEKUsAxhgTpCwBGGNMkLIEYIwxQcoSgDHGBClLAMYYE6QsARhjTJCyBGCMMUHKEoAxxgQpr54EFpGRwLNAKDBDVf9caXskMAsYCBwArlfV7RW2dwK+Ah5T1ae9adMEpg3Tp592e99Jk+opEmNMjWcAIhIKTANGAT2BCSLSs1K124FDqpoCTAWerLT9r8B/atmmMcYYP/KmC2gQkKWq2apaDMwGxlaqMxZ4zVmeC1wsIgIgIlcD24DNtWzTGGOMH3mTADoAORXWc52yKuuoailQALQWkRjgIeB3dWgTABGZJCIZIpKRl5fnRbjGGGO84e+LwI8BU1X1aF0bUNXpqpqmqmnx8T8YzdQYY0wdeXMReBeQWGG9o1NWVZ1cEQkDYvFcDB4MjBORp4AWQLmIFAFrvWjTGGOMH3mTANYA3UQkGc+H9HjgJ5XqzAcmAquAccBHqqrA+ScriMhjwFFVfd5JEjW1aYwxxo9qTACqWioik4HFeG7ZfEVVN4vI40CGqs4HXgZeF5Es4CCeD/Rat3mGx2KMMaYWvHoOQFUXAYsqlT1aYbkIuLaGNh6rqU1jjDH1x54ENsaYIGUJwBhjgpQlAGOMCVKWAIwxJkhZAjDGmCBlCcAYY4KUJQBjjAlSlgCMMSZIWQIwxpggZQnAGGOClCUAY4wJUpYAjDEmSFkCMMaYIGUJwBhjgpRXw0Eb0xBsmD79tNv7TppUT5EYExjsDMAYY4KUVwlAREaKSKaIZInIlCq2R4rIHGf7ahFJcsoHich652eDiPyowj7bRWSjsy3DVwdkjDHGOzV2AYlIKDANuBTIBdaIyHxV/apCtduBQ6qaIiLjgSeB64FNQJozBWQ7YIOIvK+qpc5+F6nqfl8ekDHGGO94cwYwCMhS1WxVLQZmA2Mr1RkLvOYszwUuFhFR1WMVPuyjAPVF0MYYY86cNwmgA5BTYT3XKauyjvOBXwC0BhCRwSKyGdgI3F0hISjwgYisFZFqr96JyCQRyRCRjLy8PG+OyRhjjBf8fhFYVVeraipwDvCwiEQ5m85T1QHAKOBeEbmgmv2nq2qaqqbFx8f7O1xjjAka3iSAXUBihfWOTlmVdUQkDIgFDlSsoKpbgKNAL2d9l/N7HzAPT1eTMcaYeuJNAlgDdBORZBGJAMYD8yvVmQ9MdJbHAR+pqjr7hAGISGfgLGC7iDQVkWZOeVPgMjwXjI0xxtSTGu8Ccu7gmQwsBkKBV1R1s4g8DmSo6nzgZeB1EckCDuJJEgDnAVNEpAQoB+5R1f0i0gWYJyInY3hLVdN9fXDGGGOq59WTwKq6CFhUqezRCstFwLVV7Pc68HoV5dlA39oGa4wxxnfsSWBjjAlSlgCMMSZIWQIwxpggZaOBGp+qacROY0zDYWcAxhgTpCwBGGNMkLIEYIwxQcoSgDHGBClLAMYYE6QsARhjTJCyBGAanLKSErdDMCYoWAIwrisrLiZn2TI2vvwyL3XtytSICN4+7zzWTZtG4b59bodnTMCyBGBcVXTwIGueeoqvZ8/m0DffkNCvH+f8+tcU5efz4eTJ/G+HDmRMnYqqzSZqjK/Zk8DGNfnZ2Wx48UXKiovpd++9xPfpQ99JntlBL3zqKfI2bmTlo4+y/IEHKMjOpnVqKhJS/XeWmp5CPtm2McbDzgCMKw5s2ULGM88QGhnJoIceIr5Pnx/Uie/dm7Hvvkvar37FuuefZ/0LL1B24oQL0RoTmCwBmHpXcuwYm2fOJDo+nkFTphDTvn21dSUkhOF/+QsXT5vG/k2b2DxrlnUHGeMjlgBMvcucM4fiw4dJveUWImJivNqn/z33kDJ2LHszMsj95BM/R2hMcPAqAYjISBHJFJEsEZlSxfZIEZnjbF8tIklO+SARWe/8bBCRH3nbpglM+9avZ/dnn5E8ahSxSUm12jfp8stpnZrKN++8w+GdO/0ToDFBpMYEICKhwDRgFNATmCAiPStVux04pKopwFTgSad8E5Cmqv2AkcD/ikiYl22aAFN85Ahb3niDZomJJI8eXev9JSSEXrfeSnhMDF9On07p8eN+iNKY4OHNGcAgIEtVs1W1GJgNjK1UZyzwmrM8F7hYRERVj6lqqVMeBZzsvPWmTRNgshcsoOTYMVJvvZWQsLrdgBbRrBm977yTogMH+OZf//JxhMYEF28SQAcgp8J6rlNWZR3nA78AaA0gIoNFZDOwEbjb2e5Nmzj7TxKRDBHJyMvL8yJc0xAVHz7MrpUraTdkCM06VPlWe61lSgqJw4eza8UKjuTk1LyDMaZKfr8IrKqrVTUVOAd4WESiarn/dFVNU9W0+Ph4/wRp/G7nsmWUl5aSdNllPmmvy5gxhEdH8/WcOXZXkDF15E0C2AUkVljv6JRVWUdEwoBY4EDFCqq6BTgK9PKyTRMgSouKyFm+nIS+fWnatq1P2gxv2pSUq68m/9tv2ffFFz5p05hg400CWAN0E5FkEYkAxgPzK9WZD0x0lscBH6mqOvuEAYhIZ+AsYLuXbZoAkbtiBaXHjpE0cqRP2+1w3nnEdOzIN3PnUlZc7NO2jQkGNSYAp89+MrAY2AK8o6qbReRxEbnKqfYy0FpEsoAHgJO3dZ4HbBCR9cA84B5V3V9dm748MNMwlJeUsHPpUlr26EFscrJP25aQEHpcfz1FBw+yY+lSn7ZtTDDw6lYMVV0ELKpU9miF5SLg2ir2ex143ds2TeDZ/fnnnMjPJ3XixJor10Gr7t2J69OHHUuW0Omiiwhr0sQvr2NMILIngY1ffbdyJU3btaPV2Wf77TW6XHEFpceOkfPxx357DWMCkSUA4zfH9+8nf+tW2g0ahIj47XVik5Jo3asXO5YsscHijKkFSwDGb/Z8/jkAbQcN8vtrdRk9mpKjR8mxcYKM8ZolAOMXqsruzz+nRUoKTeLi/P56Lbp2pdXZZ7Pjgw/sjiBjvGQJwPjF0dxcCnfvrpdv/yd1GT3a88Txp5/W22sa05hZAjB+sXv1aiQkhDYDB9bba7bs3p0WKSnsWLqU8rKyentdYxorSwDG57S8nD1r1hDXu7fX4/37SudLLqHowAHy1q+v19c1pjGyBGB87tA333AiP79eu39Oiu/blyZxcez88MN6f21jGhtLAMbn9qxZQ2hkZJXz/PqbhITQacQI8rdupWD79np/fWMaE0sAxqdUlf2bNtE6NZXQiAhXYmg/bBhhUVHstOEhjDktSwDGp47m5nIiP5/43r1diyEsKor2w4axd+1aig4dci0OYxo6SwDGp/I2bgSgdWqqq3F0GjECVSVn2TJX4zCmIbMEYHxq/6ZNNO/cmcjYWFfjaBIXR0K/fuz69FPKSkpcjcWYhsoSgPGZ4wcOUJCdTVyvXm6HAkDi8OGUFBayNyPD7VCMaZAsARif2b54MagS52L/f0Ute/Sgadu25Cxf7nYoxjRIlgCMz2QvXEh4s2Y079zZ7VAAEBE6Xnghh7dvt1tCjamCVwlAREaKSKaIZInIlCq2R4rIHGf7ahFJcsovFZG1IrLR+T2iwj7LnTbXOz8JvjooU//Ky8rYlp5OXGoqEtJwvle0GzqU0MhIcu0swJgfqPE/VURCgWnAKKAnMEFEelaqdjtwSFVTgKnAk075fuBKVe2NZ87gyrOD3aCq/ZyffWdwHMZlu1evpujgwQbT/39SeJMmtBs8mD0ZGRw/cMDtcIxpULz5qjYIyFLVbFUtBmYDYyvVGQu85izPBS4WEVHVdar6nVO+GWgiIpG+CNw0LNkLFyKhobTuWfm7gfsShw+nvKSEja+84nYoxjQo3iSADkBOhfVcp6zKOs6E7wVA60p1rgG+UNWKUza96nT/PCLVTBklIpNEJENEMvLy8rwI17hhx5IltB8yhPCmTd0O5QdiOnSgRbdubHjxRbS83O1wjGkw6qWzVkRS8XQL3VWh+Aana+h85+emqvZV1emqmqaqafHx8f4P1tTaicOH2bt2LYkXXeR2KNVKvPBCCrZtY/sHH7gdijENhjcJYBeQWGG9o1NWZR0RCQNigQPOekdgHnCzqm49uYOq7nJ+HwHewtPVZBqh3BUr0PJyOjXgBJDQvz/RCQmsf+EFt0MxpsHwJgGsAbqJSLKIRADjgfmV6szHc5EXYBzwkaqqiLQAFgJTVHXlycoiEiYicc5yODAG2HRmh2LckrN8OaEREbQbOtTtUKoVEhZG7zvuIHvhQg7v3Ol2OMY0CDUmAKdPfzKwGNgCvKOqm0XkcRG5yqn2MtBaRLKAB4CTt4pOBlKARyvd7hkJLBaRL4H1eM4gXvLlgZn6k7NsGe2GDCG8SRO3QzmtPpMmoap8OX2626EY0yCEeVNJVRcBiyqVPVphuQi4tor9/gD8oZpm62+uQOM3Rfn57Fu3jiG//a3bodQotnNnulxxBV++9BJDH33UteGqjWkoGs4TO6ZR2uX0/zfkC8AV9fvpTzm2bx/fzpvndijGuM4SgDkjO5cvJzQykvZDhrgdileSLr+c5klJdjHYGCwBmDOUs2wZ7YcOJSwqyu1QvBISGkrfu+8m95NPyNtk9x2Y4ObVNQBjqlJ06BD71q/n3Mce81mbG+rhAm3v22/nv//zP6x/4QUutTMBE8TsDMDUWe6KFaBK4vDhbodSK9FxcZw1fjxfvf46Jw4fdjscY1xjCcDUWc6yZYRFRdFu8GC3Q6m1fvfeS8nRo2yeNcvtUIxxjSUAU2c5H39Mu6FDCYtsfOP7tTvnHNqecw7rp01DVd0OxxhXWAIwdVJ85Ah5GzbQ4bzz3A6lzvpPnszBr79m50cfuR2KMa6wBGDqZPfq1Wh5OR2GDXM7lDrrcd11NImLY/20aW6HYowrLAGYOtm1ciWINJr7/6sSFhVF7zvuIOu99yjYscPtcIypd5YATJ3sWrmS+N69iYyNdTuUM9LvnntAhHXPPed2KMbUO0sAptbKy8rY/dlntG/E3T8nNU9MpPu4cWycMYPiI0fcDseYemUJwNTa/o0bKT5ypFH3/1eUdv/9nCgoYNPMmW6HYky9sgRgam3XSs/UDoGSANoNHky7IUP44tlnbcpIE1QsAZha27VyJTHt29O8c2e3Q/GZgb/4Bflbt5K9cKHboRhTbywBmFrbtXIl7YcNQ0TcDsVnul9zDc0SE8mYOtXtUIypN5YATK0cyc3lyM6dAdP9c1JIWBj9J08mZ9ky9qxd63Y4xtQLr0YDFZGRwLNAKDBDVf9caXskMAvPLF8HgOtVdbuIXAr8GYgAioFfq+pHzj4DgZlAEzyzjf1c7Zn8Bi/Q+v8rComMJCwqiiV33UWfSZNqtW/fWtY3piGo8QxAREKBacAooCcwQUR6Vqp2O3BIVVOAqcCTTvl+4EpV7Y1n0vjXK+zzInAn0M35GXkGx2Hqya6VKwmLjia+b1+3Q/G58CZN6Dh8OHu/+ILCvXvdDscYv/OmC2gQkKWq2apaDMwGxlaqMxZ4zVmeC1wsIqKq61T1O6d8M9BERCJFpB3QXFU/c771zwKuPuOjMX733X//S7vBgwkND3c7FL/oNGIEIaGh7FiyxO1QjPE7b7qAOgA5FdZzgcrj/56qo6qlIlIAtMZzBnDSNcAXqnpCRDo47VRss0NVLy4ik4BJAJ06dfIiXOMvxYWF7Fu3jqTLL6+XiVvcEBkbS7tzz+W7//6Xrlde2eifdDbmdOrlIrCIpOLpFrqrtvuq6nRVTVPVtPj4eN8HZ7y2NyMDLS8ntmtXt0Pxq6RLL0XLyti5dKnboRjjV94kgF1AYoX1jk5ZlXVEJAyIxXMxGBHpCMwDblbVrRXqd6yhTdPAfLdqFQCxyckuR+Jf0QkJtBk4kJxPPqGksNDtcIzxG28SwBqgm4gki0gEMB6YX6nOfDwXeQHGAR+pqopIC2AhMEVVV56srKq7gcMiMkQ8N5PfDLx3hsdi/Oy7VauIbtOGiJgYt0Pxu+RRoygrKmKHnQWYAFZjAlDVUmAysBjYAryjqptF5HERucqp9jLQWkSygAeAKU75ZCAFeFRE1js/Cc62e4AZQBawFfiPrw7K+J6q8t2qVcR26eJ2KPWiWceOJAwYwM4PP6T46FG3wzHGL7x6DkBVF+G5V79i2aMVlouAa6vY7w/AH6ppMwPoVZtgjXsKsrM5npdH0mWXuR1Kvek6Zgz71q1jx5IldPvRj9wOxxifsyeBjVdO9f8HyRkAQEyHDrRJSyNn2TKKDx92OxxjfM4SgPHKd6tWEdGsGTHt27sdSr3qOmYMZcXFbP/gA7dDMcbnLAEYr3y3ahVtBw1CQoLrT6Zp27a0GzSInOXLOVFQ4HY4xvhUcP03mzopLiwk78svaT90qNuhuKLLmDFoWRlb51e++c2Yxs2ri8AmuFR+yvdgZiZaVkZRfj7NEhOr2StwRSckkDh8ODuXLaPTiBHEdKjyoXVjGh07AzA1KsjOBqBFEF0Ariz5iisIi4rim3ffdTsUY3zGEoCpUX52NtFt2hDetKnbobgmIiaG5NGjObB5Mwe++srtcIzxCUsA5rRUlYLs7KD+9n9Sp4suIqp1a755912bO9gEBLsGYE7r2L59lBw9SmxKituhnLEzHcE0JDycbj/6ERtnzCB3xQoSL7zQR5EZ4w47AzCnVbDVM36fnQF4tElLo2WPHmTNm8cJezjMNHKWAMxp5W/dSlh0NE3btnU7lAZBRDj7Jz+hrKSEb+fOdTscY86IJQBzWvlbt9KiS5egewDsdJq2bUvSZZexe/VqDmZmuh2OMXVm/9WmWiWFhRTu3k2LAJ8Api6SR42iSVwcW958k/KSErfDMaZOLAGYauU79/8H+gxgdREaEcFZEyZwbO9eshcudDscY+rE7gJyyenuSOk7aVI9RlK9/KwsJCQk4GcAq6u4Xr1of+65bEtP57vPPqP9kCFuh2RMrdgZgKlWQXY2zTp1IjQiwu1QGqwe111HVKtW/Ofmmyk5dsztcIypFa8SgIiMFJFMEckSkSlVbI8UkTnO9tUikuSUtxaRZSJyVESer7TPcqfNyjOFmQagvKyMgm3b7PbPGoQ1aULqxIkc+vZbPnnoIbfDMaZWakwAIhIKTANGAT2BCSLSs1K124FDqpoCTAWedMqLgEeAX1XT/A2q2s/52VeXAzD+cWTnTspLSmgRAA+A+VurHj0YeP/9rHv+ebalp7sdjjFe8+YawCAgS1WzAURkNjAWqDggyljgMWd5LvC8iIiqFgKfioh9ijQy+c4DYME0A9iZOO+JJ9ixdCkLb7iBm774gtjOnb+3vaankBvKdR8TXLzpAuoA5FRYz3XKqqzjTCJfALT2ou1Xne6fR0REqqogIpNEJENEMvLy8rxo0vhC/tatRLVuTVTLlm6H0iiEN2nC2Hffpby0lPevvZbSEyfcDsmYGrl5EfgGVe0NnO/83FRVJVWdrqppqpoWHx9frwEGK1U99QCY8V7Lbt0Y9dpr7FmzhmX33+92OMbUyJsEsAuoOAtIR6esyjoiEgbEAgdO16iq7nJ+HwHewtPVZBqA43l5FBcU0KJbN7dDaXS6XX015zz4IBtefJGNr77qdjjGnJY3CWAN0E1EkkUkAhgPVJ4bbz4w0VkeB3ykqlpdgyISJiJxznI4MAbYVNvgjX8c/OYbwHNx09Te+U88QedLLmHJpEns+PBDt8Mxplo1JgCnT38ysBjYAryjqptF5HERucqp9jLQWkSygAeAU7eKish24K/ALSKS69xBFAksFpEvgfV4ziBe8t1hmTNxKDOTiObNiW7Txu1QGqWQsDCumjuXVmedxXs//jF5m+y7jWmYvHoSWFUXAYsqlT1aYbkIuLaafZOqaXagdyGa+qSqHPrmG1p270411+WNFyJjY/nxokW8OXgw/xo9mn6TJxPVooXbYRnzPfYksPme/KwsTuTnW/ePDzRPTOTHCxdSdOgQX/ztbxQfOeJ2SMZ8jyUA8z05y5cD0LJ7d3cDCRBt+vfnR++/z/H9+1n7t79RUljodkjGnGIJwHxPzvLl1v/vY52GD6ffPfdwbM8eTxKwMYNMA2EJwJyiquxctoyWPXpY/7+Pte7Zkz53383RXbs83UFHj7odkjGWAMz/OfTttxTu3k0r6/7xi/jevenrJIG1zzzDiYICt0MyQc4SgDnlVP+/XQD2m/g+fej/s59x/MAB1jz9NMcPnPZ5SWP8yiaEMafkLF9O03btiE6wkbn9qdVZZzHg5z9n3XPPseYvf2HAffc1igmCTOCxMwADePr/c5Yvp9NFF1n/fz1o0bUrab/8JVpWRsbTT5+aftOY+mQJwACwf9MmCnfvptOIEW6HEjSaJSZyzoMPEhYdzdqpU9lvTwybemYJwACQvWABAMmjR7scSXCJjo/nnF//mqZt2rB+2jT2rFnjdkgmiFgCMABsXbCANgMHEtOunduhBJ3I2FgG/vKXxHbtysaXXz51Md4Yf7MEYDi2fz+7P/uMLmPGuB1K0Apv0oQB991HXO/efP3222QvXMhpBtQ1xicsARi2p6ej5eV0tQTgqtCICPrefTfthgxh6/z5ZM2bZ0nA+JXdBmrYumABTdu2pc2AAW6HEvRCQkNJnTiR0MhIti9eTFlxMX3vvBMJse9qxvfsryrIlZWUsD09nS5XXGEfMg2EhIRw1oQJdLr4YnKWLWPJ3Xej5eVuh2UCkJ0BBLldK1dyoqDA+v8bGBGh+7XXEhIezpcvvYSEhnLJCy/YMxrGpywBBLnsBQsIjYig8yWXuB1Ko3a6J3nrSkRIufpqEvr25fMnnyQkPJwRzz5rScD4jFfn/CIyUkQyRSRLRKZUsT1SROY421eLSJJT3lpElonIURF5vtI+A0Vko7PP38X+ql2RvWABicOHExET43Yopgoiwvl/+hMDH3iAdc89x/Jf/couDBufqTEBiEgoMA0YBfQEJjjz+lZ0O3BIVVOAqcCTTnkR8AjwqyqafhG4E+jm/IysywGYujuwZQsHMzOt+6eBExGGP/00A+67j7V//SurHn/c7ZBMgPDmDGAQkKWq2apaDMwGxlaqMxZ4zVmeC1wsIqKqhar6KZ5EcIqItAOaq+pn6vk6Mwu4+kwOxNTepldfRUJD6XHddW6HYmogIlw0dSqpt9zCfx97jC/+/ne3QzIBwJtrAB2AnArrucDg6uqoaqmIFACtgf2naTO3UpsdqqooIpOASQCdOnXyIlzjjbKSEjbPmkXXMWNoarN/NQoSEsLlL71EcUEBH/3850S2aEHqzTe7HZZpxBr8fX+qOl1V01Q1LT4+3u1wAsb29HSO7d1Lr9tuczsUUwshYWFc8dZbdLr4YtJvu42tzhhOxtSFNwlgF5BYYb2jU1ZlHREJA2KB0810sctp53RtGj/a+MorRLdpQ/KoUW6HYmopLCqKq+fNI6F/f96/7jq+W7XK7ZBMI+VNAlgDdBORZBGJAMYD8yvVmQ9MdJbHAR/paW5VUNXdwGERGeLc/XMz8F6tozd1UrhvH9kLFtDzppsIDQ93OxxTBxHNmvHjhQuJ6dCBf40Zw4EtW9wOyTRCNSYAVS0FJgOLgS3AO6q6WUQeF5GrnGovA61FJAt4ADh1q6iIbAf+CtwiIrkV7iC6B5gBZAFbgf/45pBMTba88QblpaX0uvVWt0MxZ6BpQgLjFi8mJDycuZdfzpHc3Jp3MqYCrx4EU9VFwKJKZY9WWC4Crq1m36RqyjOAXt4GanxDVdn4yiu0GzKEuJ6V7+Y1jU2LLl0Yl57O7Asu4N1Roxj/ySdEtWzpdlimkWjwF4GNb+V8/DEHNm+2b/8BJKFfP67+9785mJnJv8eOpeT4cbdDMo2EDQURRFSVTx58kGYdO9LzppvcDsf4yMlhKFInTmTjyy/z1rnn0veuu04N7meTypvq2BlAEMl85x32rFnDsD/8gfAmTdwOx/hY23POocd115G3fj1fvfGGDRlhamRnAEGi9MQJVjz8MPF9+tDzxhvdDsf4SacRIyg+epRtCxcSHh1Nt2uucTsk04BZAggSG/7xDwq2beOa9HRCQkPdDsf4Udcrr6S0sJAdS5YQHh1d4+ih1kUUvCwBBIHjBw/y2e9/T+dLLyX58svdDsf4mYjQ4/rrKT1+nKz33iPEhvs21bAEEOBKjh3j31ddxYnDh7nwqafcDsfUEwkJoefEiZSVlPDNP/+JiNDp4ovdDss0MHYROICVFRczf9w4vlu1iiveeouEfv3cDsnUo5DQUHrfcQcJ/fuT+c477PzoI7dDMg2MJYAAVV5ayn9uuYVt//kPl/7jH/QYN87tkIwLQkJD6X3nnST060fmnDlsX7zY7g4yp1gXUANTWlTEt//+N9vT09mWns6RnBxCIyMJi4oiOiGBNgMH0iYtjbZpacT37Utk8+bf27/k+HE2z5zJmqefpiA7m/P//Gf63HmnS0djGoKQ0FB6T5rEpldf5dt//YsTBQV0Hzfu1HMCJnhZAmhA9q1fz5Y336T48GEimjWj8yWXcPYNN1BWXExZURFHcnLI/eQTtrz11ql9WqSk0CIlhfKSEsqKijiYmcnx/ftpO2gQw595hm5X2zw7xkkCt91GRLNm7PzwQ4qPHCF14kRCwuwjIJjZu98AFB89Subs2exZs4aYjh0Z++67dLzgAkIjIqqsX7hnD3vWrmXfunXsW7eOwzt3EhYZSWhUFBy+cCYAABAjSURBVJ0vuYS+d99NxwsusMnDA4SvJpyXkBB6XHcdkbGxZM2bR9GhQ3YLaJCzBOCy4/v3k/HMM5woKKDLlVeSPGpUjbfsNW3blq5XXEHXK66opyhNoBARkkeOJKpVK76aNYvP/vhHEocPp21amtuhGRdYJ6CLig4dYu3UqZQWFXHOgw/SdcwYe0jL1It2gwZxzoMPIiEhvH3eeWyYPt0uDgchOwNwyYnDh1k7dSrFR48y8Be/IDYp6dS2Mz3lt9N6443mnTox+De/YXt6OkvuuovshQu5fMYMom3q1aBhZwAuKDl2jC+efZaiQ4fo/7OfEZuc7HZIJkhFxMQwLj2d4X/9K9vT05nZuzdb33/f7bBMPfEqAYjISBHJFJEsEZlSxfZIEZnjbF8tIkkVtj3slGeKyOUVyreLyEYRWS8iGb44mMZi6b33cnTXLvredRctU1LcDscEOQkJIe3++7lxzRqiExKYd9VVvD9+PIV797odmvGzGhOAiIQC04BRQE9gQoVpHU+6HTikqinAVOBJZ9+eeOYQTgVGAi847Z10kar2U9WguQK1aeZMNs+cSfLo0cT1sgnRTMMR36cPN2VkMOz3vydr3jxePftsvpwxAy0vdzs04yfeXAMYBGSpajaAiMwGxgJfVagzFnjMWZ4LPO9M9j4WmK2qJ4BtzpzBg4BVvgm/ccnbtIml99xD4kUX0XXMGL+9Tk3XEOwagalOaEQEQ3/7W7qPG8eSu+7igzvvZPUf/8hZP/kJzTt1+kF9+1tq3LzpAuoA5FRYz3XKqqzjTCJfALSuYV8FPhCRtSJS7V+RiEwSkQwRycjLy/Mi3Iap5Phx3r/uOiKaN2fMW2/ZU5imQWt91llcv3w5qbfcwvH9+1n9xz/y9dtvU3LsmNuhGR9y8y6g81R1l4gkAEtE5GtV/aRyJVWdDkwHSEtLa7T3qX36//4fB7dsYdwHH9C0bVu3wzGmRiJC+6FDie/bl63vvUfOxx+z94sv6H7NNbQdPNgeNAwA3iSAXUBihfWOTllVdXJFJAyIBQ6cbl9VPfl7n4jMw9M19IMEEAhyV6xg7d/+Rr977iHp0kvdDseYWgmPjuasCRNoP2wYW958k02vvsqulSs5+8YbrbuxkfOmH2IN0E1EkkUkAs9F3fmV6swHJjrL44CP1PNUyXxgvHOXUDLQDfhcRJqKSDMAEWkKXAZsOvPDaXiKCwtJv/VWYpOSuODJJ90Ox5g6a96pE4Meeoizb7iBI7m5fPb442QvXEh5aanboZk6qvEMQFVLRWQysBgIBV5R1c0i8jiQoarzgZeB152LvAfxJAmceu/guWBcCtyrqmUi0gaY55xChgFvqWq6H47PdSumTCF/61auX76ciJgYt8Mx5oxISAgdL7iA+L59yZwzh63z57M3I4PUW2+t8iKxadi8ugagqouARZXKHq2wXARcW82+TwBPVCrLBvrWNtjGZueyZax7/nkG3HcfiRde6HY4xlSpLk+eR8bG0mfSJPI2bOCrN9/k8z/9ieRRo0gePdpGGG1E7J3yk+KjR0m/7TZapKRw/p/+5HY4xvhFfN++nJuSQuacOWQvXEjel1/S67bbiGnf3u3QjBfsXkQ/+fjBBzm8YwejZs4kPDra7XCM8Zvwpk3pddtt9P3pTynKz2f1E0+wY+lSe4CsEbAzAD/Y8eGHbHjxRQY+8AAdhg1zOxxj6kVCv37EdunCV7Nm8c0//8n+TZvoOmYMzTp2dDs0Uw07A/CxE4cPs/j222nZvTvn/eEPbodjTL2KbN6cfvfey9k33khBdjYze/fm6zlz3A7LVMMSgA+pKkvuvpsjubmerp8mTdwOyZh6JyJ0PP98hvz2t7Q66ywWjB/Pwhtu4PjBg26HZiqxBOBDm197ja/ffptzH3uM9kOHuh2OMa6KTkhgwooVDHv8cTLfeYeZvXrZUNMNjF0D8JGDmZksvfdeEocPZ/DDD7sdjjENQkhYGEMfeYQuY8aQfsstzLvqKnrefDPDn3mG6Li40+5rTxn7n50B+EBpURELxo8nvEkTRr/xhk3raEwlbfr358Y1axj66KN8/dZbvNKjhw013QBYAjhDWl5O+q23sm/9eka++irNOlQeKNUYA56hpof97nfcvH49cb168cGdd/LWsGF8tyooR4dvECwBnKFPH3mEr2fP5vw//5muV17pdjjGNHhxqalcv3w5o2bNomDbNt4691ze+/GPOfD1126HFnTsGsAZ+HLGDFb/8Y/0ufNOBj34oNvhGNPg1NSPf0dWFmunTuXzp54i67336HH99Qx68EES+vWrpwiDm50B1FHm3Lksuftuki67jIunTbOx0Y2pg4iYGIY+8gh3Zmcz8IEH2Pr++8zq35+5l19O3saNdo3AzywB1MG6adN4/7rraDdoEFf+85+Ehoe7HZIxjVp0fDzD//IX7srJ4fw//Ym8L79k/fPP8+lvfsPW99/n+P79bocYkKwLqBZUlZWPPMJnTzxB16uuYszbbzfKcX7qMvqjMfUhqkULBk+ZQtovf8mHP/sZu1asIHvBArIXLCA2OZm2gwaR0L8/US1buh1qQLAE4KXDO3fywV13sT09nd533MGlL75ow94a4yeh4eG0GTCANgMGcHz/fvZkZLBnzRoy58whc84cmnfuzLF9++gyejQJ/frZHNt1ZJ9gNdDyctb/4x988tBDoMqIv/+d/pMnW5+/MfWkSVwcySNHkjxyJEd37yZv/XryNmxg5SOPsPKRR2gSF0eniy8mcfhwOgwbRlxqqiUEL1kCqEbx0aNsmjmTdX//O4e+/ZbOl17KZdOnE5uU5HZoxgStmHbtiGnXjuRRo0i56iq2L1nCjqVL2bFkCZnOoHORsbG0Pecc4vv1I6FvX+J69aJFSsoPZuQ7XVdosDxl7FUCEJGRwLN4poScoap/rrQ9EpgFDMQzGfz1qrrd2fYwcDtQBtynqou9adMNx/Ly2LF0Kds/+ICsefM4UVBAu8GDGTNnDj2uvda+9RvTgDRt25bUm24i9aabUFUKsrPZtXIlu1auZO/atax77jnKTpz4Xv3Y5GRiOnQgpn17Du/YQXhMDOFNmxIWHU1YVBRhTZoQGhlJUX4+ETExAd/NW+PRiUgoMA24FMgF1ojIfFX9qkK124FDqpoiIuOBJ4HrRaQnnvmBU4H2wFIR6e7sU1ObPlNcWEjJ0aOen8JCig4d4lheHsf27ePw9u0c2LKFg1u2kL91KwBRrVrRZcwY+k+eTPshQ/wRkjHGh0SEFl270qJrV1JvvhmA8tJSDnz9ted/OyuLQ1lZHN6+nf2bNrH9gw8oPny42vZWTJkCeJ5ePpkkwmNiiIiJOfU7olkzIpo186yf/F0hoYQ3bUp4dDShTmIJi4wkJCLi1O+QsDBCwsMJCQ1FQkORkJB6/5LpTXobBGQ58/giIrOBsXgmej9pLPCYszwXeF48RzIWmK2qJ4BtzqTxg5x6NbXpM28MHMjBzMwqt4VGRtKye3cSBgyg12230fnSS2kzYICN52NMIxcSFkZ8r17E9+pV5fYvnn+eksJCSo4do7SwkNITJyg9fpyyoiLaDBzo2eZ8aSx2vkCe/H14585T68VHjlB67JhvghbxJIKTyUDk1O97Dxzw+RDzoqo1xCPjgJGqeoezfhMwWFUnV6izyamT66xvBQbjSQqfqeobTvnLwH+c3U7bZoW2JwEnO+R6AFV/kjcecUCw3NQcLMcaLMcJwXOsgXacnVU1vnJhg+/gUtXpQMDcuC4iGaqa5nYc9SFYjjVYjhOC51iD5Ti9uVdqF5BYYb2jU1ZlHREJA2LxXAyubl9v2jTGGONH3iSANUA3EUkWkQg8F3XnV6ozH5joLI8DPlJP39J8YLyIRIpIMtAN+NzLNo0xxvhRjV1AqloqIpOBxXhu2XxFVTeLyONAhqrOB14GXncu8h7E84GOU+8dPBd3S4F7VbUMoKo2fX94DVLAdGd5IViONViOE4LnWIPiOGu8CGyMMSYw2fPSxhgTpCwBGGNMkLIEUI9EZKSIZIpIlohMcTseXxGRRBFZJiJfichmEfm5U95KRJaIyLfO74AZw1dEQkVknYgscNaTRWS1897OcW5uaNREpIWIzBWRr0Vki4gMDdT3VETud/52N4nI2yISFYjvaWWWAOpJhSE1RgE9gQnOUBmBoBT4par2BIYA9zrHNgX4UFW7AR8664Hi58CWCutPAlNVNQU4hGd4lMbuWSBdVc8C+uI53oB7T0WkA3AfkKaqvfDcmHJySJtAe0+/xxJA/Tk1pIaqFgMnh79o9FR1t6p+4SwfwfNB0QHP8b3mVHsNuNqdCH1LRDoCVwAznHUBRuAZBgUC4FhFJBa4AM8dfqhqsarmE6DvKZ47Ips4zzFFA7sJsPe0KpYA6k8HIKfCeq5TFlBEJAnoD6wG2qjqbmfTHqCNS2H52t+AB4GTE9a2BvJVtdRZD4T3NhnIA151urpmiEhTAvA9VdVdwNPATjwf/AXAWgLvPf0BSwDGZ0QkBngX+IWqfm+oRefBwEZ/z7GIjAH2qepat2PxszBgAPCiqvYHCqnU3RNA72lLPGc2yXhGLW4KjHQ1qHpiCaD+BPTwFyISjufD/01V/ZdTvFdE2jnb2wH73IrPh4YBV4nIdjzdeCPw9JW3cLoPIDDe21wgV1VXO+tz8SSEQHxPLwG2qWqeqpYA/8LzPgfae/oDlgDqT8AOf+H0gb8MbFHVv1bYVHGIkInAe/Udm6+p6sOq2lFVk/C8hx+p6g3AMjzDoEAAHKuq7gFyRKSHU3Qxnif6A+49xdP1M0REop2/5ZPHGlDvaVXsSeB6JCKj8fQfnxz+4gmXQ/IJETkPWAFs5P/6xX+D5zrAO0AnYAdwnaoedCVIPxCR4cCvVHWMiHTBc0bQClgH3OjMg9FoiUg/PBe6I4Bs4FY8XxoD7j0Vkd8B1+O5o20dcAeePv+Aek8rswRgjDFByrqAjDEmSFkCMMaYIGUJwBhjgpQlAGOMCVKWAIwxJkhZAjDGCyJytYioiJzldizG+IolAGO8MwH41PltTECwBGBMDZwxjs7DMxzweKcsRERecMbKXyIii0RknLNtoIh8LCJrRWTxyaETjGloLAEYU7OxeMbF/wY4ICIDgR8DSXjmdrgJGAqnxkR6DhinqgOBV4CAeOLbBJ6wmqsYE/Qm4BnwDTxDA0zA87/zT1UtB/aIyDJnew+gF7DEM6wMoXiGGDamwbEEYMxpiEgrPCN+9hYRxfOBrsC86nYBNqvq0HoK0Zg6sy4gY05vHPC6qnZW1SRVTQS2AQeBa5xrAW2A4U79TCBeRE51CYlIqhuBG1MTSwDGnN4Efvht/12gLZ4x878C3gC+AAqc6T7HAU+KyAZgPXBu/YVrjPdsNFBj6khEYlT1qIi0Bj4Hhjnj6BvTKNg1AGPqboGItMAzXv7v7cPfNDZ2BmCMMUHKrgEYY0yQsgRgjDFByhKAMcYEKUsAxhgTpCwBGGNMkPr/YNgTltHY2tQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df_train_raw['Age'].dropna(),\n",
    "             kde=True,\n",
    "             color='darkred',\n",
    "             bins=30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a260f82d0>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAASUklEQVR4nO3df4zkdX3H8ef7QPxxax3Oo5srR3o0SyCElNPbIAZjdkEtUnGJIQRjzLU9c/+oxZamhTZpY9Kmmva0/GGaXsR6aSwLxaN3kKil521Nm4ruCipwUraIClFO7W11aaIi7/4x39V1WG7mOzuz8/0cz0ey2fl+9/uded3Md1/32c/O97uRmUiSyrNp1AEkSf2xwCWpUBa4JBXKApekQlngklSo0zfywbZu3Zo7duyotc/TTz/N5s2bhxNonZqazVz1NDUXNDebuepZb66FhYXvZeZZz/lCZm7Yx65du7Kuo0eP1t5nozQ1m7nqaWquzOZmM1c9680FzOcaneoUiiQVygKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFaqnU+kjogV8FLgISOB3gEeA24EdwOPAdZl5YigpNRAHr766p+3edvfdQ04iaRB6HYHfAnw6My8ALgaOATcBRzLzPOBItSxJ2iBdCzwiXgG8HrgVIDN/nJlLwAxwoNrsAHDNsEJKkp4rssvfxIyIncB+4GHao+8F4AbgycxsVdsEcGJluWP/vcBegPHx8V2zs7O1Ai4vLzM2NlZrn43S1GzPl2tpcbGn/VsTE4OOBJT3fDVBU7OZq5715pqenl7IzMnO9b0U+CTweeCyzLwvIm4BfgC8d3VhR8SJzDzzZPc1OTmZ8/PztYLPzc0xNTVVa5+N0tRsz5dr1HPgpT1fTdDUbOaqZ725ImLNAu9lDvwJ4InMvK9avhN4NfBURGyr7nwbcLzvdJKk2roWeGZ+B/hWRJxfrbqC9nTKYWB3tW43cGgoCSVJa+r1L/K8F/hERJwBPAb8Nu3yvyMi9gDfAK4bTkRJ0lp6KvDMfAB4zvwL7dG4JGkEPBNTkgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqEscEkqlAUuSYWywCWpUBa4JBWq1+uBS8/R659og+H9mTbphcwRuCQVygKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RC9XQqfUQ8DvwQ+CnwTGZORsQW4HZgB/A4cF1mnhhOTElSpzoj8OnM3JmZk9XyTcCRzDwPOFItS5I2yHqmUGaAA9XtA8A1648jSepVZGb3jSK+DpwAEvi7zNwfEUuZ2aq+HsCJleWOffcCewHGx8d3zc7O1gq4vLzM2NhYrX02SlOyLS0u/uKKVguWlvq+v9bERH+P2+U+m/J8dWpqLmhuNnPVs95c09PTC6tmP36m1wI/OzOfjIhfBu4F3gscXl3YEXEiM8882f1MTk7m/Px8reBzc3NMTU3V2mejNCVb52VdN83M8OyhQ33fX6+Xfq17OdmmPF+dmpoLmpvNXPWsN1dErFngPU2hZOaT1efjwF3AJcBTEbGtuvNtwPG+00mSauta4BGxOSJevnIbeBPwIHAY2F1tthvof8gnSaqtl7cRjgN3tae5OR34x8z8dER8EbgjIvYA3wCuG15MSVKnrgWemY8BF6+x/vvAFcMIJUnqzjMxJalQFrgkFcoCl6RCWeCSVKieLmYlrdfBq69m08wMB/ftO+l2vZ5EJMkRuCQVywKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqEscEkqlAUuSYWywCWpUBa4JBXKApekQlngklQoC1ySCtVzgUfEaRFxf0TcUy2fGxH3RcRiRNweEWcML6YkqVOdEfgNwLFVyx8EPpyZE8AJYM8gg0mSTq6nAo+I7cBvAh+tlgO4HLiz2uQAcM0wAkqS1haZ2X2jiDuBvwReDvwB8FvA56vRNxFxDvCpzLxojX33AnsBxsfHd83OztYKuLy8zNjYWK19NkpTsi0tLv7iilYLlpb6vr/WxER/j9v1jrvn6vWxB6kpr+NamprNXPWsN9f09PRCZk52rj+9244R8RbgeGYuRMRU3QfOzP3AfoDJycmcmqp3F3Nzc9TdZ6M0JdvBfft+YXnTzAzPHjrU9/1N3X13X4/bTS+5en3sQWrK67iWpmYzVz3DytW1wIHLgLdGxFXAS4BfAm4BWhFxemY+A2wHnhx4OknS8+o6B56ZN2fm9szcAVwPfDYz3wEcBa6tNtsN9D/kkyTVtp73gf8R8PsRsQi8Erh1MJEkSb3oZQrlZzJzDpirbj8GXDL4SJKkXngmpiQVygKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVKha10LRC8PBq68edQRJPXAELkmFssAlqVAWuCQVygKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVKiuVyOMiJcAnwNeXG1/Z2b+WUScC8wCrwQWgHdm5o+HGVZaMcgrJm6amYGpqYHdn7RRehmB/wi4PDMvBnYCV0bEpcAHgQ9n5gRwAtgzvJiSpE5dCzzblqvFF1UfCVwO3FmtPwBcM5SEkqQ1RWZ23yjiNNrTJBPAR4C/Aj5fjb6JiHOAT2XmRWvsuxfYCzA+Pr5rdna2VsDl5WXGxsZq7bNRhp1taXGxvx1bLVhaGmyYQeghV2tioqe76vu5WfNBW7S2bh3c/Q1QU49/c9Wz3lzT09MLmTnZub6nv8iTmT8FdkZEC7gLuKDXB87M/cB+gMnJyZyqOdc4NzdH3X02yrCzHdy3r6/9Ns3M8OyhQwNOs3695Jq6++6e7qvf52Ytm2ZmmLr22oHd3yA19fg3Vz3DylXrXSiZuQQcBV4LtCJi5T+A7cCTA84mSTqJrgUeEWdVI28i4qXAG4FjtIt8ZdiyG2jekE+STmG9TKFsAw5U8+CbgDsy856IeBiYjYg/B+4Hbh1iTklSh64FnplfAV61xvrHgEuGEUqS1J1nYkpSoSxwSSqUBS5JhbLAJalQPZ3Io8Ea5IWYJL1wOQKXpEJZ4JJUKAtckgrlHLgaxd8PSL1zBC5JhbLAJalQFrgkFcoCl6RCWeCSVKhi3oXS67sT3tbjn+SSpNI5ApekQlngklQoC1ySClXMHLjUBHXOFPX3MRo2R+CSVCgLXJIKZYFLUqEscEkqlAUuSYWywCWpUF0LPCLOiYijEfFwRDwUETdU67dExL0R8Wj1+czhx5UkrehlBP4McGNmXghcCrw7Ii4EbgKOZOZ5wJFqWZK0QboWeGZ+OzO/VN3+IXAMOBuYAQ5Umx0ArhlWSEnSc0Vm9r5xxA7gc8BFwDczs1WtD+DEynLHPnuBvQDj4+O7ZmdnawVcXl5mbGyMpcXFnrZvTUzUuv/1WMlWV6//lr61WrC0NNzH6EeDc7W2bu1p0zqv3SCOxX6PsWEzVz3rzTU9Pb2QmZOd63su8IgYA/4N+IvMPBgRS6sLOyJOZOZJ58EnJydzfn6+VvC5uTmmpqYaeTnZlWx1DfsP926ameHZQ4eG+hj9aHKua971rp623ehT6fs9xobNXPWsN1dErFngPb0LJSJeBHwS+ERmHqxWPxUR26qvbwOO951OklRb14tZVdMjtwLHMvNDq750GNgNfKD63LyhlTRCTfypUaeWXq5GeBnwTuCrEfFAte6PaRf3HRGxB/gGcN1wIkqS1tK1wDPz34F4ni9fMdg4kqReeSamJBXKApekQlngklQoC1ySCmWBS1KhLHBJKpQFLkmF6uVEHumUN+zr00jD4AhckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqE8lV4qhH8kWZ0cgUtSoSxwSSqUUyjSKcaplhcOR+CSVCgLXJIKZYFLUqG6FnhEfCwijkfEg6vWbYmIeyPi0erzmcONKUnq1MsI/OPAlR3rbgKOZOZ5wJFqWZK0gboWeGZ+DvifjtUzwIHq9gHgmgHnkiR1EZnZfaOIHcA9mXlRtbyUma3qdgAnVpbX2HcvsBdgfHx81+zsbK2Ay8vLjI2NsbS42NP2rYmJWve/HivZ6ur139K3VguWlob7GP0wV31DzLae75V+j/1hO1VzTU9PL2TmZOf6dRd4tXwiM7vOg09OTub8/Hyd3MzNzTE1NdXI97auZKtr2H8BfdPMDM8eOjTUx+iHueobZrb1fK/0e+wP26maKyLWLPB+34XyVERsq+54G3C872SSpL70eybmYWA38IHqc2OGL4Me3Xq2mqSm6uVthLcB/wmcHxFPRMQe2sX9xoh4FHhDtSxJ2kBdR+CZ+fbn+dIVA84iSarBMzElqVBejbCLk82pb5qZ4eC+fT9bdr5cp6K1vgc6j33w+B8FR+CSVCgLXJIK5RSKpIGo8xZep1sGwxG4JBXKApekQjmFIr1ADfuaPBo+R+CSVCgLXJIKZYFLUqEscEkqlAUuSYWywCWpUL6NUFLxVt4SudZFtlY71c4AdQQuSYWywCWpUBa4JBXKOXBJG67X0/hPtTnrQXMELkmFssAlqVBOoQyQV3eTtJEcgUtSoSxwSSqUUyiS1Kdep0233HjjUB5/XSPwiLgyIh6JiMWIuGlQoSRJ3fVd4BFxGvAR4M3AhcDbI+LCQQWTJJ3cekbglwCLmflYZv4YmAVmBhNLktRNZGZ/O0ZcC1yZme+qlt8JvCYz39Ox3V5gb7V4PvBIzYfaCnyvr5DD19Rs5qqnqbmgudnMVc96c/1qZp7VuXLov8TMzP3A/n73j4j5zJwcYKSBaWo2c9XT1FzQ3GzmqmdYudYzhfIkcM6q5e3VOknSBlhPgX8ROC8izo2IM4DrgcODiSVJ6qbvKZTMfCYi3gN8BjgN+FhmPjSwZD/X9/TLBmhqNnPV09Rc0Nxs5qpnKLn6/iWmJGm0PJVekgplgUtSoRpd4E05VT8iPhYRxyPiwVXrtkTEvRHxaPX5zBHkOicijkbEwxHxUETc0KBsL4mIL0TEl6ts76/WnxsR91Wv6e3VL8A3XEScFhH3R8Q9TckVEY9HxFcj4oGImK/WNeG1bEXEnRHxtYg4FhGvbUiu86vnauXjBxHxvoZk+73quH8wIm6rvh8Gfow1tsAbdqr+x4ErO9bdBBzJzPOAI9XyRnsGuDEzLwQuBd5dPUdNyPYj4PLMvBjYCVwZEZcCHwQ+nJkTwAlgzwiyAdwAHFu13JRc05m5c9V7hpvwWt4CfDozLwAupv28jTxXZj5SPVc7gV3A/wF3jTpbRJwN/C4wmZkX0X6Tx/UM4xjLzEZ+AK8FPrNq+Wbg5hHm2QE8uGr5EWBbdXsb8EgDnrNDwBublg14GfAl4DW0z0Y7fa3XeAPzbKf9jX05cA8QDcn1OLC1Y91IX0vgFcDXqd7w0JRca+R8E/AfTcgGnA18C9hC+51+9wC/MYxjrLEjcH7+JKx4olrXFOOZ+e3q9neA8VGGiYgdwKuA+2hItmqa4gHgOHAv8N/AUmY+U20yqtf0b4A/BJ6tll/ZkFwJ/EtELFSXoIDRv5bnAt8F/r6acvpoRGxuQK5O1wO3VbdHmi0znwT+Gvgm8G3gf4EFhnCMNbnAi5Ht/1JH9n7MiBgDPgm8LzN/sPpro8yWmT/N9o+322lf/OyCUeRYLSLeAhzPzIVRZ1nD6zLz1bSnDd8dEa9f/cURvZanA68G/jYzXwU8TceURAOO/zOAtwL/1Pm1UWSr5txnaP/n9yvAZp47BTsQTS7wpp+q/1REbAOoPh8fRYiIeBHt8v5EZh5sUrYVmbkEHKX9Y2MrIlZOIBvFa3oZ8NaIeJz2FTQvpz3HO+pcKyM3MvM47bncSxj9a/kE8ERm3lct30m70Eeda7U3A1/KzKeq5VFnewPw9cz8bmb+BDhI+7gb+DHW5AJv+qn6h4Hd1e3dtOefN1REBHArcCwzP9SwbGdFRKu6/VLac/PHaBf5taPKlpk3Z+b2zNxB+5j6bGa+Y9S5ImJzRLx85TbtOd0HGfFrmZnfAb4VEedXq64AHh51rg5v5+fTJzD6bN8ELo2Il1XfoyvP2eCPsVH+4qGHXwZcBfwX7bnTPxlhjttoz2X9hPaIZA/tedMjwKPAvwJbRpDrdbR/PPwK8ED1cVVDsv06cH+V7UHgT6v1vwZ8AVik/SPvi0f4uk4B9zQhV/X4X64+Hlo53hvyWu4E5qvX8p+BM5uQq8q2Gfg+8IpV60aeDXg/8LXq2P8H4MXDOMY8lV6SCtXkKRRJ0klY4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQ/w9CC6dYIgBIdwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_train_raw['Age'].hist(bins=30,color='darkred',alpha=0.7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a261c28d0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEHCAYAAABBW1qbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAARgElEQVR4nO3de7BdZX3G8e8jkXoXJKcUk9g4lbGlF1EzGIujFWqLV5gOUtuq0dLGP9Bq61SxduqldcZOVby0dYaKNWirUJSClrEyXHTsKDZRBCS1RgtCiiYg4K3aBn/9Y795PSYnZp/A2isn+X5m9py13vXuzRMGzpN13akqJEkCuNfYASRJ+w9LQZLUWQqSpM5SkCR1loIkqbMUJEndsiE/PMkNwLeAu4AdVbUmyUOA84DVwA3AaVV1e5IAbwOeBnwXeEFVffbHff7y5ctr9erVg+WXpAPRpk2bbq2quYW2DVoKzZOr6tZ562cCl1XVG5Oc2dZfCTwVOLq9Hge8s/3co9WrV7Nx48ZhUkvSASrJjXvaNsbho5OBDW15A3DKvPFza+LTwGFJjhohnyQdtIYuhQI+lmRTkvVt7MiquqUtfw04si2vAG6a996b25gkaUaGPnz0hKramuQngUuT/Mf8jVVVSRb1nI1WLusBHvawh91zSSVJw+4pVNXW9nMbcCFwHPD1nYeF2s9tbfpWYNW8t69sY7t+5tlVtaaq1szNLXieRJK0jwYrhST3T/LAncvArwHXARcD69q0dcBFbfli4PmZWAvcOe8wkyRpBoY8fHQkcOHkSlOWAf9YVR9N8u/A+UlOB24ETmvzL2FyOeoWJpekvnDAbJKkBQxWClX1FeBRC4zfBpy4wHgBZwyVR5K0d97RLEnqLAVJUjeLO5pn6rF/fO7YERa06a+eP3YESdor9xQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkKkqTOUpAkdZaCJKmzFCRJnaUgSeosBUlSZylIkjpLQZLUWQqSpM5SkCR1loIkqbMUJEmdpSBJ6iwFSVJnKUiSOktBktRZCpKkzlKQJHWWgiSpsxQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkKkqTOUpAkdYOXQpJDknwuyUfa+sOTXJVkS5Lzkhzaxn+irW9p21cPnU2S9KNmsafwUmDzvPW/BM6qqkcAtwOnt/HTgdvb+FltniRphgYthSQrgacD72rrAU4ALmhTNgCntOWT2zpt+4ltviRpRobeU3gr8ArgB239COCOqtrR1m8GVrTlFcBNAG37nW2+JGlGBiuFJM8AtlXVpnv4c9cn2Zhk4/bt2+/Jj5akg96QewrHA89KcgPwASaHjd4GHJZkWZuzEtjalrcCqwDa9gcDt+36oVV1dlWtqao1c3NzA8aXpIPPYKVQVa+qqpVVtRp4DnB5Vf0OcAVwapu2DrioLV/c1mnbL6+qGiqfJGl3Y9yn8Ergj5JsYXLO4Jw2fg5wRBv/I+DMEbJJ0kFt2d6n3H1VdSVwZVv+CnDcAnO+Bzx7FnkkSQvzjmZJUmcpSJI6S0GS1FkKkqTOUpAkdZaCJKmzFCRJnaUgSeosBUlSZylIkjpLQZLUWQqSpM5SkCR1loIkqbMUJEmdpSBJ6iwFSVJnKUiSOktBktRZCpKkzlKQJHWWgiSpsxQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkKkqTOUpAkdZaCJKmzFCRJnaUgSeosBUlSZylIkrrBSiHJfZJ8Jsnnk3whyeva+MOTXJVkS5Lzkhzaxn+irW9p21cPlU2StLAh9xS+D5xQVY8CjgVOSrIW+EvgrKp6BHA7cHqbfzpwexs/q82TJM3QYKVQE99uq/durwJOAC5o4xuAU9ryyW2dtv3EJBkqnyRpd4OeU0hySJKrgW3ApcCXgTuqakebcjOwoi2vAG4CaNvvBI5Y4DPXJ9mYZOP27duHjC9JB51BS6Gq7qqqY4GVwHHAz94Dn3l2Va2pqjVzc3N3O6Mk6YdmcvVRVd0BXAE8HjgsybK2aSWwtS1vBVYBtO0PBm6bRT5J0sSQVx/NJTmsLd8XeAqwmUk5nNqmrQMuassXt3Xa9surqobKJ0na3bK9T9lnRwEbkhzCpHzOr6qPJLke+ECSvwA+B5zT5p8DvDfJFuAbwHMGzCZJWsBgpVBV1wCPXmD8K0zOL+w6/j3g2UPlkSTtnXc0S5I6S0GS1E1VCkkum2ZMkrS0/dhzCknuA9wPWJ7kcGDnHcYP4oc3nUmSDhB7O9H8IuBlwEOBTfywFL4J/PWAuSRJI/ixpVBVbwPeluQlVfWOGWWSJI1kqktSq+odSX4ZWD3/PVV17kC5JEkjmKoUkrwX+BngauCuNlyApSBJB5Bpb15bAxzjYyck6cA27X0K1wE/NWQQSdL4pt1TWA5cn+QzTL5RDYCqetYgqSRJo5i2FF47ZAhJ0v5h2quPPj50EEnS+Ka9+uhbTK42AjiUyfctf6eqHjRUMEnS7E27p/DAnctJApwMrB0qlCRpHIt+SmpN/DPw6wPkkSSNaNrDR78xb/VeTO5b+N4giSRJo5n26qNnzlveAdzA5BCSJOkAMu05hRcOHUSSNL5pv2RnZZILk2xrrw8mWTl0OEnSbE17ovnvgYuZfK/CQ4EPtzFJ0gFk2lKYq6q/r6od7fUeYG7AXJKkEUxbCrcleW6SQ9rrucBtQwaTJM3etKXwu8BpwNeAW4BTgRcMlEmSNJJpL0l9PbCuqm4HSPIQ4E1MykKSdICYdk/hl3YWAkBVfQN49DCRJEljmbYU7pXk8J0rbU9h2r0MSdISMe0v9jcDn0ryT2392cAbhokkSRrLtHc0n5tkI3BCG/qNqrp+uFiSpDFMfQiolYBFIEkHsEU/OluSdOCyFCRJnaUgSeosBUlSN1gpJFmV5Iok1yf5QpKXtvGHJLk0yZfaz8PbeJK8PcmWJNckecxQ2SRJCxtyT2EH8PKqOgZYC5yR5BjgTOCyqjoauKytAzwVOLq91gPvHDCbJGkBg5VCVd1SVZ9ty98CNgMrmHyN54Y2bQNwSls+GTi3Jj4NHJbkqKHySZJ2N5NzCklWM3lW0lXAkVV1S9v0NeDItrwCuGne225uY7t+1vokG5Ns3L59+2CZJelgNHgpJHkA8EHgZVX1zfnbqqqAWsznVdXZVbWmqtbMzfk9P5J0Txq0FJLcm0kh/ENVfagNf33nYaH2c1sb3wqsmvf2lW1MkjQjQ159FOAcYHNVvWXepouBdW15HXDRvPHnt6uQ1gJ3zjvMJEmagSEff3088Dzg2iRXt7E/Ad4InJ/kdOBGJt/oBnAJ8DRgC/Bd4IUDZpMkLWCwUqiqTwLZw+YTF5hfwBlD5ZEk7Z13NEuSOktBktRZCpKkzlKQJHWWgiSpsxQkSd2Q9yloH3z19b84doQFPezPrh07gqQZcE9BktRZCpKkzlKQJHWWgiSpsxQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkKkqTOUpAkdZaCJKmzFCRJnaUgSeosBUlSZylIkjpLQZLUWQqSpM5SkCR1loIkqbMUJEmdpSBJ6iwFSVJnKUiSOktBktRZCpKkbrBSSPLuJNuSXDdv7CFJLk3ypfbz8DaeJG9PsiXJNUkeM1QuSdKeDbmn8B7gpF3GzgQuq6qjgcvaOsBTgaPbaz3wzgFzSZL2YLBSqKpPAN/YZfhkYENb3gCcMm/83Jr4NHBYkqOGyiZJWtiszykcWVW3tOWvAUe25RXATfPm3dzGJEkzNNqJ5qoqoBb7viTrk2xMsnH79u0DJJOkg9esS+HrOw8LtZ/b2vhWYNW8eSvb2G6q6uyqWlNVa+bm5gYNK0kHm1mXwsXAura8Drho3vjz21VIa4E75x1mkiTNyLKhPjjJ+4FfAZYnuRl4DfBG4PwkpwM3Aqe16ZcATwO2AN8FXjhULknSng1WClX1W3vYdOICcws4Y6gskqTpeEezJKmzFCRJnaUgSeosBUlSZylIkjpLQZLUWQqSpM5SkCR1loIkqbMUJEmdpSBJ6iwFSVJnKUiSOktBktRZCpKkzlKQJHWWgiSpsxQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkKkqTOUpAkdZaCJKmzFCRJnaUgSeosBUlSZylIkrplYwfQgeX4dxw/doQF/dtL/m3sCNKS4J6CJKlzT0E6QPz1yz88doQFvfjNzxw7ghbBPQVJUueegtR8/IlPGjvCHj3pEx8fO4IOEvvVnkKSk5J8McmWJGeOnUeSDjb7TSkkOQT4G+CpwDHAbyU5ZtxUknRw2Z8OHx0HbKmqrwAk+QBwMnD9qKkkaQqb33D52BEW9HOvPmFR8/enUlgB3DRv/WbgcSNlkTRjb3juqWNHWNCr33fB2BFmKlU1dgYAkpwKnFRVv9fWnwc8rqpevMu89cD6tvpI4IsDxloO3Drg5w/N/ONZytnB/GMbOv9PV9XcQhv2pz2FrcCqeesr29iPqKqzgbNnESjJxqpaM4t/1hDMP56lnB3MP7Yx8+83J5qBfweOTvLwJIcCzwEuHjmTJB1U9ps9harakeTFwL8ChwDvrqovjBxLkg4q+00pAFTVJcAlY+eYZyaHqQZk/vEs5exg/rGNln+/OdEsSRrf/nROQZI0MkthAUv9cRtJ3p1kW5Lrxs6yWElWJbkiyfVJvpDkpWNnWowk90nymSSfb/lfN3amfZHkkCSfS/KRsbMsVpIbklyb5OokG8fOsxhJ/rD9d3Ndkvcnuc+sM1gKuzhAHrfxHuCksUPsox3Ay6vqGGAtcMYS+/f/feCEqnoUcCxwUpK1I2faFy8FNo8d4m54clUdu5QuS02yAvgDYE1V/QKTC26eM+sclsLu+uM2qup/gZ2P21gyquoTwDfGzrEvquqWqvpsW/4Wk19MK8ZNNb2a+HZbvXd7LakTd0lWAk8H3jV2loPQMuC+SZYB9wP+e9YBLIXdLfS4jSXzS+lAkmQ18GjgqnGTLE479HI1sA24tKqWVH7grcArgB+MHWQfFfCxJJvaExCWhKraCrwJ+CpwC3BnVX1s1jksBe2XkjwA+CDwsqr65th5FqOq7qqqY5nclX9ckl8YO9O0kjwD2FZVm8bOcjc8oaoew+QQ8BlJnjh2oGkkOZzJUYmHAw8F7p/kubPOYSnsbqrHbWg4Se7NpBD+oao+NHaefVVVdwBXsLTO7xwPPCvJDUwOnZ6Q5H3jRlqc9jduqmobcCGTQ8JLwa8C/1VV26vq/4APAb886xCWwu583MaIkgQ4B9hcVW8ZO89iJZlLclhbvi/wFOA/xk01vap6VVWtrKrVTP7bv7yqZv631X2V5P5JHrhzGfg1YKlchfdVYG2S+7X/D05khJP9lsIuqmoHsPNxG5uB85fa4zaSvB/4FPDIJDcnOX3sTItwPPA8Jn9Dvbq9njZ2qEU4CrgiyTVM/oJxaVUtucs6l7AjgU8m+TzwGeBfquqjI2eaSjv3dAHwWeBaJr+fZ35ns3c0S5I69xQkSZ2lIEnqLAVJUmcpSJI6S0GS1FkK0hSSvLo9vfKadpns45K8a+fD+pJ8ew/vW5vkqvaezUleO9Pg0iLtV9+8Ju2PkjweeAbwmKr6fpLlwKFV9XtTvH0DcFpVfb49gfeRQ2aV7i73FKS9Owq4taq+D1BVt1bVfye5Mkl/NHOSs9rexGVJ5trwTzJ5uNnOZyJd3+a+Nsl7k3wqyZeS/P6M/0zSgiwFae8+BqxK8p9J/jbJkxaYc39gY1X9PPBx4DVt/Czgi0kuTPKiXb405ZeAE4DHA3+W5KED/hmkqVgK0l6070d4LLAe2A6cl+QFu0z7AXBeW34f8IT23tcDa5gUy28D8x+5cFFV/U9V3crkwXlL5cFtOoB5TkGaQlXdBVwJXJnkWmDd3t4y771fBt6Z5O+A7UmO2HXOHtalmXNPQdqLJI9McvS8oWOBG3eZdi/g1Lb828An23uf3p54CXA0cBdwR1s/uX2n8xHArzB5gJ40KvcUpL17APCO9kjsHcAWJoeSLpg35ztMvlDnT5l849pvtvHnAWcl+W577+9U1V2tJ65hcthoOfDnVTXzr16UduVTUqURtPsVvl1Vbxo7izSfh48kSZ17CpKkzj0FSVJnKUiSOktBktRZCpKkzlKQJHWWgiSp+38XSo7P2OQBTgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='SibSp',data=df_train_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a262a7750>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAD4CAYAAAA5OEWQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAATUUlEQVR4nO3db4xdd33n8fenSRqqDKpJQ0fexFoH4W6VRm3KjNJU9MFMorYhrepUYtlYFbhtKvdBugIVaxu60oaoi5RqMdlF7aZyG4RpqYdsAMWKoG1qMop4EIINIeRPCYaGXVvuWGBDGKr1bsJ3H8xxuJ11Mtf33sn9+c77JV3dc37n3HO/9zu+/sw599wzqSokSVIbfmjcBUiSpB8wmCVJaojBLElSQwxmSZIaYjBLktSQC8ddAMBll11WW7duHdn2vve973HJJZeMbHsbjf0bnL0bnL0bnL0bzrj6d/jw4W9W1etXjzcRzFu3buXQoUMj297i4iJzc3Mj295GY/8GZ+8GZ+8GZ++GM67+JfnG2cY9lC1JUkMMZkmSGmIwS5LUEINZkqSGGMySJDXEYJYkqSEGsyRJDTGYJUlqyJrBnOQ1SR5L8qUkTyW5sxu/MsnnkhxJ8rEkP9yNX9zNH+mWb13flyBJ0uTo58pfp4Hrq2o5yUXAZ5N8Gvh94O6qWkjyZ8CtwD3d/amqemOSW4A/Bv7dOtV/VoePH2b+zvm+1q07ap2rkSSpf2vuMdeK5W72ou5WwPXA/d34PuDmbnp7N0+3/IYkGVnFkiRNsFStvceY5ALgMPBG4E+B/wI8WlVv7JZvAT5dVVcneRK4saqOdsu+BvxcVX1z1TZ3AbsApqenZxYWFkb2opZOLnH09NG+1p3ZPDOy550Uy8vLTE1NjbuM85K9G5y9G5y9G864+jc/P3+4qmZXj/f1Ryyq6kXgmiSbgE8CPzlsQVW1F9gLMDs7W6O8gPie/XvY/ezu/urY4aHs1bwg/uDs3eDs3eDs3XBa6985nZVdVd8GHgZ+HtiU5EywXwEc66aPAVsAuuU/CnxrJNVKkjTh+jkr+/XdnjJJfgT4ReAZVgL6rd1qO4EHuukD3Tzd8s9UP8fLJUlSX4eyNwP7us+Zfwi4r6oeTPI0sJDkPwNfBO7t1r8X+MskR4CTwC3rULckSRNpzWCuqieAnz3L+NeBa88y/r+BfzuS6iRJ2mC88pckSQ0xmCVJaojBLElSQwxmSZIaYjBLktQQg1mSpIYYzJIkNcRgliSpIQazJEkNMZglSWqIwSxJUkMMZkmSGmIwS5LUEINZkqSGGMySJDXEYJYkqSEGsyRJDTGYJUlqiMEsSVJDDGZJkhpiMEuS1BCDWZKkhhjMkiQ1xGCWJKkhBrMkSQ1ZM5iTbEnycJKnkzyV5J3d+HuTHEvyeHe7qecx70lyJMlXkvzyer4ASZImyYV9rPMC8O6q+kKS1wKHkzzULbu7qt7fu3KSq4BbgJ8C/hXw90l+oqpeHGXhkiRNojX3mKvqeFV9oZv+LvAMcPkrPGQ7sFBVp6vqH4EjwLWjKFaSpEmXqup/5WQr8AhwNfD7wG8CzwOHWNmrPpXkT4BHq+qvusfcC3y6qu5fta1dwC6A6enpmYWFhWFfy0uWTi5x9PTRvtad2TwzsuedFMvLy0xNTY27jPOSvRucvRucvRvOuPo3Pz9/uKpmV4/3cygbgCRTwMeBd1XV80nuAf4IqO5+D/Db/W6vqvYCewFmZ2drbm6u34euac/+Pex+dnd/dezo/xeTjWJxcZFR/jw2Ens3OHs3OHs3nNb619dZ2UkuYiWUP1pVnwCoqqWqerGqvg/8OT84XH0M2NLz8Cu6MUmStIZ+zsoOcC/wTFV9oGd8c89qvw482U0fAG5JcnGSK4FtwGOjK1mSpMnVz6HsNwNvB76c5PFu7A+BHUmuYeVQ9nPA7wJU1VNJ7gOeZuWM7ts8I1uSpP6sGcxV9VkgZ1n0qVd4zPuA9w1RlyRJG5JX/pIkqSEGsyRJDTGYJUlqiMEsSVJDDGZJkhpiMEuS1BCDWZKkhhjMkiQ1xGCWJKkhBrMkSQ0xmCVJaojBLElSQwxmSZIaYjBLktQQg1mSpIYYzJIkNcRgliSpIQazJEkNMZglSWqIwSxJUkMMZkmSGmIwS5LUEINZkqSGGMySJDXEYJYkqSFrBnOSLUkeTvJ0kqeSvLMbvzTJQ0m+2t2/rhtPkg8mOZLkiSRvWu8XIUnSpOhnj/kF4N1VdRVwHXBbkquA24GDVbUNONjNA7wF2NbddgH3jLxqSZIm1JrBXFXHq+oL3fR3gWeAy4HtwL5utX3Azd30duAjteJRYFOSzSOvXJKkCZSq6n/lZCvwCHA18D+ralM3HuBUVW1K8iBwV1V9tlt2EPiDqjq0alu7WNmjZnp6emZhYWH4V9NZOrnE0dNH+1p3ZvPMyJ53UiwvLzM1NTXuMs5L9m5w9m5w9m444+rf/Pz84aqaXT1+Yb8bSDIFfBx4V1U9v5LFK6qqkvSf8CuP2QvsBZidna25ublzefgr2rN/D7uf3d1fHTvOqewNYXFxkVH+PDYSezc4ezc4ezec1vrX11nZSS5iJZQ/WlWf6IaXzhyi7u5PdOPHgC09D7+iG5MkSWvo56zsAPcCz1TVB3oWHQB2dtM7gQd6xt/RnZ19HfCdqjo+wpolSZpY/RzKfjPwduDLSR7vxv4QuAu4L8mtwDeAt3XLPgXcBBwB/hn4rZFWLEnSBFszmLuTuPIyi284y/oF3DZkXZIkbUhe+UuSpIYYzJIkNcRgliSpIQazJEkNMZglSWqIwSxJUkMMZkmSGmIwS5LUEINZkqSGGMySJDXEYJYkqSEGsyRJDTGYJUlqiMEsSVJDDGZJkhpiMEuS1BCDWZKkhhjMkiQ1xGCWJKkhBrMkSQ0xmCVJaojBLElSQwxmSZIaYjBLktQQg1mSpIasGcxJPpTkRJIne8bem+RYkse72009y96T5EiSryT55fUqXJKkSdTPHvOHgRvPMn53VV3T3T4FkOQq4Bbgp7rH/PckF4yqWEmSJt2awVxVjwAn+9zedmChqk5X1T8CR4Brh6hPkqQNJVW19krJVuDBqrq6m38v8JvA88Ah4N1VdSrJnwCPVtVfdevdC3y6qu4/yzZ3AbsApqenZxYWFkbwclYsnVzi6Omjfa07s3lmZM87KZaXl5mamhp3Geclezc4ezc4ezeccfVvfn7+cFXNrh6/cMDt3QP8EVDd/R7gt89lA1W1F9gLMDs7W3NzcwOW8v/bs38Pu5/d3V8dO9b+xWSjWVxcZJQ/j43E3g3O3g3O3g2ntf4NdFZ2VS1V1YtV9X3gz/nB4epjwJaeVa/oxiRJUh8GCuYkm3tmfx04c8b2AeCWJBcnuRLYBjw2XImSJG0cax7KTrIfmAMuS3IUuAOYS3INK4eynwN+F6CqnkpyH/A08AJwW1W9uD6lS5I0edYM5qracZbhe19h/fcB7xumqFdT7kxf69UdfhYtSVp/XvlLkqSGGMySJDXEYJYkqSEGsyRJDTGYJUlqiMEsSVJDDGZJkhpiMEuS1BCDWZKkhhjMkiQ1xGCWJKkhBrMkSQ0xmCVJaojBLElSQwxmSZIaYjBLktQQg1mSpIYYzJIkNcRgliSpIQazJEkNuXDcBZwvcmf6Wq/uqHWuRJI0ydxjliSpIQazJEkNMZglSWqIwSxJUkPWDOYkH0pyIsmTPWOXJnkoyVe7+9d140nywSRHkjyR5E3rWbwkSZOmnz3mDwM3rhq7HThYVduAg908wFuAbd1tF3DPaMqUJGljWDOYq+oR4OSq4e3Avm56H3Bzz/hHasWjwKYkm0dVrCRJky5Va3/vNslW4MGqurqb/3ZVbeqmA5yqqk1JHgTuqqrPdssOAn9QVYfOss1drOxVMz09PbOwsDCaVwQsnVzi6OmjI9veuZjZPDOW5x2l5eVlpqamxl3GecneDc7eDc7eDWdc/Zufnz9cVbOrx4e+wEhVVZJzvqpGVe0F9gLMzs7W3NzcsKW8ZM/+Pex+dvfItncuasf5f4GRxcVFRvnz2Ejs3eDs3eDs3XBa69+gZ2UvnTlE3d2f6MaPAVt61ruiG5MkSX0YNJgPADu76Z3AAz3j7+jOzr4O+E5VHR+yRkmSNow1D2Un2Q/MAZclOQrcAdwF3JfkVuAbwNu61T8F3AQcAf4Z+K11qFmSpIm1ZjBX1Y6XWXTDWdYt4LZhi5IkaaPyyl+SJDXEYJYkqSEGsyRJDTGYJUlqiMEsSVJDDGZJkhpiMEuS1BCDWZKkhhjMkiQ1xGCWJKkhBrMkSQ0xmCVJaojBLElSQwxmSZIaYjBLktQQg1mSpIYYzJIkNcRgliSpIQazJEkNMZglSWqIwSxJUkMMZkmSGmIwS5LUEINZkqSGGMySJDXkwmEenOQ54LvAi8ALVTWb5FLgY8BW4DngbVV1argyJUnaGEaxxzxfVddU1Ww3fztwsKq2AQe7eUmS1If1OJS9HdjXTe8Dbl6H55AkaSING8wF/F2Sw0l2dWPTVXW8m/4nYHrI55AkacNIVQ3+4OTyqjqW5MeBh4B/Dxyoqk0965yqqted5bG7gF0A09PTMwsLCwPXsdrSySWOnj46su2di5nNM2N53lFaXl5mampq3GWcl+zd4Ozd4OzdcMbVv/n5+cM9HwO/ZKiTv6rqWHd/IskngWuBpSSbq+p4ks3AiZd57F5gL8Ds7GzNzc0NU8q/sGf/HnY/u3tk2zsXtWPwX3Rasbi4yCh/HhuJvRucvRucvRtOa/0b+FB2kkuSvPbMNPBLwJPAAWBnt9pO4IFhi5QkaaMYZo95GvhkkjPb+euq+psknwfuS3Ir8A3gbcOXKUnSxjBwMFfV14GfOcv4t4AbhilKkqSNyit/SZLUEINZkqSGGMySJDXEYJYkqSEGsyRJDRnqAiMaXO5MX+vVHef/BUskSf1zj1mSpIYYzJIkNcRgliSpIQazJEkNMZglSWqIwSxJUkP8utSI9fs1qHE+r1/BkqR2uccsSVJDDGZJkhpiMEuS1BA/Y27cuD6zliSNh3vMkiQ1xGCWJKkhBrMkSQ0xmCVJaojBLElSQwxmSZIa4telNqC1voL1/p94P/N3zve9vVFf4rPfr4h5aVFJk8g9ZkmSGuIes141XiylPf7xE6k96xbMSW4E/htwAfAXVXXXej2XxsvAlaTRWZdgTnIB8KfALwJHgc8nOVBVT6/H80mtGvUvLf3utfrLktSf3Jm+zqt5NY8Yrdce87XAkar6OkCSBWA7YDBrZDxJTNIkStXo/9NK8lbgxqr6nW7+7cDPVdXv9ayzC9jVzf4b4CsjLOEy4Jsj3N5GY/8GZ+8GZ+8GZ++GM67+/euqev3qwbGd/FVVe4G967HtJIeqanY9tr0R2L/B2bvB2bvB2bvhtNa/9fq61DFgS8/8Fd2YJEl6BesVzJ8HtiW5MskPA7cAB9bpuSRJmhjrcii7ql5I8nvA37LydakPVdVT6/FcL2NdDpFvIPZvcPZucPZucPZuOE31b11O/pIkSYPxkpySJDXEYJYkqSETF8xJbkzylSRHktw+7npak+RDSU4kebJn7NIkDyX5anf/um48ST7Y9fKJJG8aX+Xjl2RLkoeTPJ3kqSTv7Mbt3xqSvCbJY0m+1PXuzm78yiSf63r0se5kUZJc3M0f6ZZvHWf9LUhyQZIvJnmwm7d3fUryXJIvJ3k8yaFurNn37UQFc8+lQN8CXAXsSHLVeKtqzoeBG1eN3Q4crKptwMFuHlb6uK277QLueZVqbNULwLur6irgOuC27t+X/VvbaeD6qvoZ4BrgxiTXAX8M3F1VbwROAbd2698KnOrG7+7W2+jeCTzTM2/vzs18VV3T833lZt+3ExXM9FwKtKr+D3DmUqDqVNUjwMlVw9uBfd30PuDmnvGP1IpHgU1JNr86lbanqo5X1Re66e+y8p/k5di/NXU9WO5mL+puBVwP3N+Nr+7dmZ7eD9yQZMNeADzJFcCvAH/RzQd7N6xm37eTFsyXA/+rZ/5oN6ZXNl1Vx7vpfwKmu2n7+TK6w4M/C3wO+9eX7lDs48AJ4CHga8C3q+qFbpXe/rzUu275d4Afe3Urbsp/Bf4D8P1u/sewd+eigL9Lcri7HDQ0/L717zHrX6iqSuJ36F5Bking48C7qur53p0R+/fyqupF4Jokm4BPAj855pLOC0l+FThRVYeTzI27nvPUL1TVsSQ/DjyU5B96F7b2vp20PWYvBTqYpTOHarr7E924/VwlyUWshPJHq+oT3bD9OwdV9W3gYeDnWTlMeGYHobc/L/WuW/6jwLde5VJb8Wbg15I8x8rHc9ez8rfu7V2fqupYd3+ClV8Kr6Xh9+2kBbOXAh3MAWBnN70TeKBn/B3dWYrXAd/pOfSz4XSf090LPFNVH+hZZP/WkOT13Z4ySX6Elb/V/gwrAf3WbrXVvTvT07cCn6kNejWkqnpPVV1RVVtZ+T/tM1X1G9i7viS5JMlrz0wDvwQ8Scvv26qaqBtwE/AsK59f/cdx19PaDdgPHAf+LyufndzKyudPB4GvAn8PXNqtG1bOcv8a8GVgdtz1j7l3v8DKZ1VPAI93t5vsX1+9+2ngi13vngT+Uzf+BuAx4AjwP4CLu/HXdPNHuuVvGPdraOEGzAEP2rtz6tkbgC91t6fO5ELL71svySlJUkMm7VC2JEnnNYNZkqSGGMySJDXEYJYkqSEGsyRJDTGYJUlqiMEsSVJD/h++ZvjVi4VD7AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_train_raw['Fare'].hist(color='green', \n",
    "                          bins=40, \n",
    "                          figsize=(8,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Data Cleaning\n",
    "We want to fill in missing age data instead of just dropping the missing age data rows. One way to do this is by filling in the mean age of all the passengers (imputation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a26e69710>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAWeklEQVR4nO3de4zdZZ3H8fennRkt4FJoh1I6YFEGXTULrhO8xlUQrGKEVSSoccuEWIld1+KahTXZRYxbMbvRuqZoumKpBpCbUGIF2tS6eMHKFMqlFDlIKQ72coq0Urdx2vLdP86v2E6n7Znpec5vzjyfV9Kc8/zO7/KdTvKZ5/wuz6OIwMzM8jGu7ALMzKy5HPxmZplx8JuZZcbBb2aWGQe/mVlm2souoB6TJ0+O6dOnl12GmVlLWbVq1ZaI6By8vCWCf/r06fT19ZVdhplZS5G0fqjlPtVjZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpaZpMEv6TJJayQ9KulGSS+XdLKklZKelHSTpI6UNZiZ2b6SBb+kacA/AT0R8QZgPHAR8FXg6xFxCvA8cEmqGszMbH+p7+NvAyZI2gkcAWwAzgQ+Vny+CPgi8K3EdRyWefPmUalUkuy7v78fgK6urobvu7u7mzlz5jR8v2bW2pL1+CPiWeC/gGeoBf42YBWwNSJ2Fav1A9OG2l7SLEl9kvqq1WqqMku3Y8cOduzYUXYZZpaRZD1+SccA5wEnA1uBW4AZ9W4fEQuABQA9PT2lzhaTstc8e/ZsAObPn5/sGGZme0t5cfc9wLqIqEbETuCHwNuBiZL2/MHpAp5NWIOZmQ2SMvifAd4i6QhJAs4CHgNWABcU68wEFieswczMBkl5jn8lcCvwAPBIcawFwOXA5yQ9CUwCrk1Vg5mZ7S/pXT0RcSVw5aDFTwFnpDyumZkdmJ/cNTPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zy9KWLVv49Kc/zXPPPVd2KU3n4DcboZyDYyxYuHAhDz30EAsXLiy7lKZLFvySXiNp9V7//ihpjqRjJS2TVClej0lVg1lKOQdHq9uyZQtLliwhIliyZEl2f7xTTr34m4g4PSJOB94E/B9wO3AFsDwiuoHlRduspeQeHK1u4cKFRAQAL774YnZ/vJt1qucs4LcRsR44D1hULF8EnN+kGswaJvfgaHVLly5l586dAOzcuZN77rmn5Iqaq1nBfxFwY/F+SkRsKN5vBKYMtYGkWZL6JPVVq9Vm1GhWt9yDo9Wdc845tLe3A9De3s573/vekitqruTBL6kD+CBwy+DPotZliqG2i4gFEdETET2dnZ2JqzQbntyDo9X19vYiCYBx48bR29tbckXN1Ywe//uAByJiU9HeJGkqQPG6uQk1mDVU7sHR6iZPnsy5556LJM4991wmTZpUdklN1Yzg/yh/Oc0DcCcws3g/E1jchBrMGir34BgLent7Oe2007L8o92WcueSjgTOBj611+KrgZslXQKsBy5MWYNZKr29vaxbty7L4BgLJk+ezDXXXFN2GaVIGvwR8Sdg0qBlz1G7y8espeUcHNba/OSumVlmHPxmZplx8JuZZcbBb2ZZynmQPQe/mWUp50H2HPxmlp3cB9lz8JtZdnIfZM/Bb2bZyX2QPQe/mWUn90H2HPxmlp3cB9lz8JuNUM63A7a63AfZc/CbjVDOtwOOBTmPzungNxuB3G8HtNbm4DcbgdxvBxwLcv7G5uA3G4Hcbwdsdbl/Y3Pwm43AOeecQ1tbbTqLtra27G4HbHW5f2NLGvySJkq6VdLjktZKequkYyUtk1QpXo9JWYNZCr29vbz44otALThyvEDYynL/xpa6x/8N4O6IeC1wGrAWuAJYHhHdwPKibdZy9vQYrfXk/o0tWfBLOhp4J3AtQEQMRMRW4DxgUbHaIuD8VDWYpZL7qYJWl/s3tpQ9/pOBKrBQ0oOSvlNMvj4lIjYU62wEpgy1saRZkvok9VWr1YRlmg3f3XffvU/7rrvuKqkSs+FLGfxtwN8C34qINwJ/YtBpnah1mYb8vhwRCyKiJyJ6Ojs7E5ZpNnzHH3/8Qds2ui1cuJBx42rxN27cuOy+saUM/n6gPyJWFu1bqf0h2CRpKkDxujlhDWZJbNy48aBtG92WLl3Krl27ANi1a5cv7jZKRGwEfifpNcWis4DHgDuBmcWymcDiVDWYpTJjxoyXBvmSxPve976SK7Lh8OicaX0GuF7Sw8DpwFzgauBsSRXgPUXbrKXsPbqjpOwuDra63EfnbEu584hYDfQM8dFZKY9rZnYwe0bnvOOOOzw6p5nVJ/eLg2OBR+c0s2HJ/eKgtTYHv9kI5H5xcCzw6JxmNiy5XxxsdR6d08yGLfep+1pd7kNuJL2rx2w0mDdvHpVKpeH7Xb9+PW1tbTzxxBPMnj274fvv7u5mzpw5Dd+vDT065+c///mSq2oe9/jNRmhgYICXvexlL53rt9aR+zUa9/htzEvVa97Ty58/f36S/Vs6vb29LFmyBMjzGo17/GaWndyv0bjHb2ZZ6u3tZd26ddn19sHBb2aZmjx5Mtdcc03ZZZTCp3rMzDLj4Dczy4yD38wsMw5+M7PM+OKumY1qqZ687u/vB6Crq6vh+x7tT10nDX5JTwMvALuBXRHRI+lY4CZgOvA0cGFEPJ+yDjOzwXbs2FF2CaVpRo//3RGxZa/2FcDyiLha0hVF+/Im1GFmLchPXjdeGef4zwMWFe8XAeeXUIOZWbZSB38ASyWtkjSrWDYlIjYU7zcCU4baUNIsSX2S+qrVauIyzczykfpUzzsi4llJxwHLJD2+94cREZJiqA0jYgGwAKCnp2fIdQZLdREopT31phjWN5XRfuHKzA4uafBHxLPF62ZJtwNnAJskTY2IDZKmApsbdbxKpcKq1WsZGH9Co3aZXNvu2peu+x7ZVnIl9enY/fuySzCzw5Qs+CUdCYyLiBeK9+cAXwLuBGYCVxevixt53IHxJ1A9+tJG7tL20rnt22WXYGaHKWWPfwpwezEvaRtwQ0TcLel+4GZJlwDrgQsT1mBmZoMkC/6IeAo4bYjlzwFnpTqumZkdnIdsMDPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMnPI4Jc0RdK1ku4q2q8rRtY0M7MWVE+P/zrgHmDP7CZPAJ5+ycysRdUT/JMj4mbgRYCI2AXsTlqVmZklU0/w/0nSJGoTpyPpLUDd8wRKGi/pQUk/KtonS1op6UlJN0nqGFHlZmY2IvUE/+eoTZf4akm/AL4HfGYYx/gssHav9leBr0fEKcDzgK8XmJk10SGDPyIeAP4OeBvwKeD1EfFwPTuX1AWcC3ynaAs4E7i1WGURcP7wyzYzs5E65NSLkj40aNGpkrYBj0TE5kNsPg/4F+AVRXsSsLW4TgDQD0wbRr1mZnaY6plz9xLgrcCKov0uYBVwsqQvRcT3h9pI0geAzRGxStK7hluYpFnALICTTjppuJubmdkB1BP8bcBfR8QmqN3XT+08/5uBe4Ehgx94O/BBSe8HXg78FfANYKKktqLX3wU8O9TGEbEAWADQ09MTdf9EZmZ2UPVc3D1xT+gXNhfL/gDsPNBGEfGvEdEVEdOBi4CfRMTHqX1zuKBYbSaweESVm5nZiNTT4/9pcSvmLUX7w8WyI4GtIzjm5cAPJH0ZeBC4dgT7MDOzEaon+GcDHwLeUbT7gCkR8Sfg3fUcJCJ+Cvy0eP8UcMZwCzUzs8ao53bOAJ4CdgF/Ty3s1x50IzMzG7UO2OOXdCrw0eLfFuAmQBFRVy/fzMxGp4Od6nkc+BnwgYh4EkDSZU2paoT6+/vp2PUCndu+XXYpY1bHrt/T3/9C2WWY2WE42KmeDwEbgBWS/kfSWYCaU5aZmaVywB5/RNwB3FHcvXMetaGYj5P0LeD2iFjapBrr1tXVxe+e30b16EvLLmXM6tz2bbq6jk6y73nz5lGpVJLsO4U9tc6ePbvkSoanu7ubOXM8snrODnlXT3H3zg3ADZKOAT5C7ZbMURf81toqlQp9D69h4KjOskupS/tA7bnCXz51qJFLRo+O7dWyS7BRoJ7bOV8SEc9Te5p2QZpyLHcDR3Wy8fQPl13GmHX86tvKLsFGAc+5a2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmhnU7p5nZUFrt4TtozQfwGvXwnYPfzA5bpVKhb82jDHROLLuUurXHbgB+ubm/5Erq01EdyfQnQ3Pwm1lDDHROZOMFHrw3leNvXXHoleqU7By/pJdL+rWkhyStkXRVsfxkSSslPSnpJkkdqWowM7P9pby4+2fgzIg4DTgdmCHpLcBXga9HxCnA88AlCWswM7NBkgV/1Gwvmu3FvwDOBG4tli8Czk9Vg5mZ7S/p7ZySxktaDWwGlgG/BbZGxK5ilX5g2gG2nSWpT1JfteoRBc3MGiVp8EfE7og4HeiiNsH6a4ex7YKI6ImIns7O1him18ysFTTlAa6I2AqsAN4KTJS0526iLuDZZtRgZmY1Ke/q6ZQ0sXg/ATgbWEvtD8AFxWozgcWpajAzs/2lvI9/KrBI0nhqf2BujogfSXoM+IGkLwMPAtcmrMFaSH9/Px3bt3mykIQ6tlfp7x8ouwwrWbLgj4iHgTcOsfwpauf7zcysBH5y10aNrq4unhno8NSLCR2/+ja6uo4ruwwrmUfnNDPLjIPfzCwzY+5UT8fu39O57dtll1G3tt1bANg1fnLJldSnY/fvgaPLLsPMDsOYCv7u7u6ySxi2SmUzAN3drRKmR7fk/7OZ/cWYCv5GTFDQbHsmgZg/f37JlZhZLsZU8JtZOfr7++nYtrWhY8bbvjqqW2nUIxi+uGtmlhn3+M3ssHV1dfFMB56BK6Hjb11B13FdDdmXe/xmZplx8JuZZcanemxU6dhebZlB2tp3bAVg54SJJVdSv47tVcBDNuTOwW+jRqs9H1CpbAOg+1WtFKTHtdz/szWeg99GjVZ7DsPPYFir8jl+M7PMOPjNzDKTcurFEyWtkPSYpDWSPlssP1bSMkmV4vWYVDWYmdn+Uvb4dwH/HBGvA94CzJb0OuAKYHlEdAPLi7aZmTVJsuCPiA0R8UDx/gVqE61PA84DFhWrLQLOT1WDmZntrynn+CVNpzb/7kpgSkRsKD7aCEw5wDazJPVJ6qtWq80o08wsC8mDX9JRwG3AnIj4496fRUQAMdR2EbEgInoioqezszN1mWZm2Uga/JLaqYX+9RHxw2LxJklTi8+nAptT1mBmZvtKeVePgGuBtRHxtb0+uhOYWbyfCSxOVYOZme0v5ZO7bwc+ATwiaXWx7AvA1cDNki4B1gMXJqzBzJqko9paE7G0b90OwM6JR5VcSX06qluhQcMyJwv+iPg5oAN8fFaq45pZ87Xi+D+VbRUAuhsUpskd19Ww/2eP1WNmh63VxlmCvMda8pANZmaZcfCbmWXGwW9mlhkHv5lZZhz8ZmaZcfCbmWXGwW9mlhkHv5lZZhz8ZmaZcfCbmWXGwW9mlhkHv5lZZhz8ZmaZcfCbmWXGwW9mlpmUUy9+V9JmSY/utexYScskVYrXY1Id38zMhpayx38dMGPQsiuA5RHRDSwv2mZm1kTJgj8i7gX+MGjxecCi4v0i4PxUxzczs6E1+xz/lIjYULzfCEw50IqSZknqk9RXrVabU52ZWQZKu7gbEQHEQT5fEBE9EdHT2dnZxMrMzMa2Zgf/JklTAYrXzU0+vplZ9pod/HcCM4v3M4HFTT6+mVn2Ut7OeSNwH/AaSf2SLgGuBs6WVAHeU7TNzKyJ2lLtOCI+eoCPzkp1TDMzOzQ/uWtmlhkHv5lZZhz8ZmaZcfCbmWXGwW9mlhkHv5lZZhz8ZmaZcfCbmWUm2QNcZqPFvHnzqFQqDd/v448/zo4dO+jt7eWII45o+P67u7uZM2dOw/dr5h6/2QgNDAwA8PTTT5dbiNkwucdvY16KXvMTTzzBxRdfDMCf//xnLrvsMk455ZSGH8csBff4zUbgqquu2qd95ZVXllSJ2fA5+M1GYN26dQdtm41mPtVTh1QXB4GX9jt79uyG79sXB9Npb29n586d+7TNWoWDv2QTJkwouwQbgb1Df6i22Wjm4K+De8022Pjx49m9e/c+bbNWUco5fkkzJP1G0pOSriijBrPDsXfoD9U2G82a3uOXNB6YD5wN9AP3S7ozIh5rdi1mNvqlusaW8/W1Mnr8ZwBPRsRTETEA/AA4r4Q6zEbshBNO2Kc9bdq0kiqxkZowYUK219jKOMc/DfjdXu1+4M2DV5I0C5gFcNJJJzWnMrM6zZ0796UHuAC+8pWvlFfMGDeae86tatTexx8RCyKiJyJ6Ojs7yy7HbB+nnnrqS73+adOm+aldayllBP+zwIl7tbuKZWYtZe7cuRx55JHu7VvLKeNUz/1At6STqQX+RcDHSqjD7LCceuqpLFu2rOwyzIat6cEfEbsk/SNwDzAe+G5ErGl2HWZmuSrlAa6I+DHw4zKObWaWu1F7cdfMzNJw8JuZZcbBb2aWGUVE2TUckqQqsL7sOhKaDGwpuwgbEf/uWttY//29MiL2exCqJYJ/rJPUFxE9Zddhw+ffXWvL9ffnUz1mZplx8JuZZcbBPzosKLsAGzH/7lpblr8/n+M3M8uMe/xmZplx8JuZZcbBXyJJ35W0WdKjZddiwyPpREkrJD0maY2kz5Zdk9VH0ssl/VrSQ8Xv7qqya2o2n+MvkaR3AtuB70XEG8qux+onaSowNSIekPQKYBVwvueOHv0kCTgyIrZLagd+Dnw2In5VcmlN4x5/iSLiXuAPZddhwxcRGyLigeL9C8BaatOK2igXNduLZnvxL6sesIPf7DBJmg68EVhZbiVWL0njJa0GNgPLIiKr352D3+wwSDoKuA2YExF/LLseq09E7I6I06lN/XqGpKxOtTr4zUaoOD98G3B9RPyw7Hps+CJiK7ACmFF2Lc3k4DcbgeIC4bXA2oj4Wtn1WP0kdUqaWLyfAJwNPF5uVc3l4C+RpBuB+4DXSOqXdEnZNVnd3g58AjhT0uri3/vLLsrqMhVYIelh4H5q5/h/VHJNTeXbOc3MMuMev5lZZhz8ZmaZcfCbmWXGwW9mlhkHv5lZZhz8ZoCk3cUtmY9KukXSEQdZ94uSPt/M+swaycFvVrMjIk4vRkkdAC4tuyCzVBz8Zvv7GXAKgKR/kPRwMXb79wevKOmTku4vPr9tzzcFSR8pvj08JOneYtnri3HgVxf77G7qT2VW8ANcZoCk7RFxlKQ2auPv3A3cC9wOvC0itkg6NiL+IOmLwPaI+C9JkyLiuWIfXwY2RcQ3JT0CzIiIZyVNjIitkr4J/CoirpfUAYyPiB2l/MCWNff4zWomFMP09gHPUBuH50zglojYAhARQ82d8AZJPyuC/uPA64vlvwCuk/RJYHyx7D7gC5IuB17p0LeytJVdgNkosaMYpvcltXHYDuk6ajNvPSTpYuBdABFxqaQ3A+cCqyS9KSJukLSyWPZjSZ+KiJ808Gcwq4t7/GYH9hPgI5ImAUg6doh1XgFsKIZo/viehZJeHRErI+LfgSpwoqRXAU9FxH8Di4G/Sf4TmA3BPX6zA4iINZL+A/hfSbuBB4GLB632b9Rm3qoWr68olv9ncfFWwHLgIeBy4BOSdgIbgbnJfwizIfjirplZZnyqx8wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLz/764KzGh6h2NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.boxplot(x='Pclass',\n",
    "            y='Age',\n",
    "            data=df_train_raw,\n",
    "            palette='winter')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the wealthier passengers in the higher classes tend to be older, which makes sense. We'll use these average age values to impute based on Pclass for Age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def impute_age(cols):\n",
    "    Age = cols[0]\n",
    "    Pclass = cols[1]\n",
    "    \n",
    "    if pd.isnull(Age):\n",
    "        if Pclass == 1:\n",
    "            return 37\n",
    "        elif Pclass == 2:\n",
    "            return 29\n",
    "        else:\n",
    "            return 24\n",
    "    else:\n",
    "        return Age"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now apply that function and check if it works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_copy = df_train_raw.copy() # For test only\n",
    "train_copy['Age'] = train_copy[['Age','Pclass']].apply(impute_age, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check that heat map again!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a274e5e10>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAErCAYAAAB981BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAZGUlEQVR4nO3debxlVXXg8d+qAkRQotCImC5wJOIAgag45GOLGGehVYgD+jFqS9utEWMcmgwOOKQ/ndbWOGMIgcQoojEOSAAVB+xmHh0wrSCmW5Q2cQCiKLL6j30edevVq4ep2nvVrfd+38/nfuqdc+ud9e607j57r71PZCaSpBprtvYfIEmriUlXkgqZdCWpkElXkgqZdCWpkElXkgptt9ydv7XmCOvJJHV3+ncuLY33mLvsXxrvzJtPiU3dZ0tXkgot29KVpBGqW57zxKQrqdxK715Yjt0LklTIpCtJhUy6klTIpCtJhUy6klTIpCtJhSwZk1Runkq4qpl0JZWzTleSVMKkK0mFTLqSVMikK0mFTLqSVMikK0mFLBmTVG6eSriq2dKVpEK2dCWVW82TI0y6ksrNUxKsZtKVVG41t3Tt05WkQiZdSSpk0pWkQvbpSio3T32s1WzpSlIhW7qSyq3m6gWTrqRy85QEq5l0JZWzpStJheYpCVZzIE2SCtnSlVRuNXcv2NKVpEImXUkqZNKVpEImXUkq5ECapHLzNLBVzaQrqdxqrl4w6UoqN09JsJp9upJUyJaupHKruXvBlq4kFTLpSlIhuxcklZun0/1qJl1J5ezTlSSVMOlKUiGTriQVsk9XUrl56mOtZtKVVM6BNElSCZOuJBWye0FSuXk63a9mS1eSCtnSlVTOgTRJUglbupLKzVPLs5pJV1I5uxckSSVMupJUyKQrSYVMupJUyIE0SeXmaWCrmi1dSSpkS1dSudVcMmbSlVRunpJgNZOupHKruaVrn64kFbKlK6lcdcuzumW9HJOupHIrvXvhzJs3fZ/dC5JUyKQrSYVMupJUyD5dSeXmqYSrmklXUrmVPpC2HLsXJKmQSVeSCpl0JamQfbqSys1TH2s1k66kcg6kSZJKmHQlqZDdC5LKzdPpfjWTrqRy9ulKkkqYdCWpkElXkgqZdCWpkElXkgqZdCWpkElXkgpZpyup3DzVzVYz6Uoq5+QISVIJk64kFTLpSlIh+3QllZunPtZqJl1J5RxIkySVsKUrqdw8tTyrmXQllVvN3QsmXUnl5ikJVjPpSipnS1eSCs1TEqxm0pVUbjW3dC0Zk6RCtnQllZunlmc1k66kcnYvSJJKmHQlqZBJV5IKmXQlqZBJV5IKmXQlqZBJV5IKWacrqdw81c1Ws6UrSYVs6Uoq54w0SVIJk64kFTLpSlIhk64kFXIgTVK5eRrYqmZLV5IK2dKVVG41l4yZdCWVm6ckWM2kK6ncam7p2qcrSYVs6UoqN08tz2omXUnlVnP3gklXUrl5SoLV7NOVpEK2dCWVs3tBkgrNUxKsZtKVVG41t3Tt05WkQiZdSSpk0pWkQiZdSSpk0pWkQiZdSSpk0pWkQiZdSSpk0pWkQiZdSSrkNGBJ5eZpWm41k66kcqt57QWTrqRy85QEq5l0JZVbzS1dB9IkqZBJV5IKmXQlqZB9upLKzVMfazWTrqRyDqRJkkqYdCWpkElXkgqZdCWpkElXkgqZdCWpkElXkgqZdCWpkElXkgqZdCWpkElXkgqZdCWpkElXkgq5ypikcvO06lc1k66kci7tKEkqYdKVpEImXUkqZNKVpEImXUkqZNKVpEImXUkqZNKVpEImXUkqZNKVpEImXUkqZNKVpEImXUkqZNKVpEIu7Sip3DwttVjNpCupnOvpSpJKmHQlqZBJV5IK2acrqdw89bFWs6UrSYVs6UoqZ/WCJKmESVeSCpl0JamQSVeSCpl0JamQSVeSCpl0JamQSVeSCpl0JamQSVeSCpl0JamQSVeSCpl0JamQSVeSCrm0o6Ry87TUYjWTrqRyrqcrSSphS1dSuXlqeVYz6UoqZ/eCJKmESVeSCpl0JamQSVeSCjmQJmmrmKfBrVKZ2f0GHDXiuMYz3rYWbyU/NuNt3m1U98JRg45rPONta/FW8mMz3mawT1eSCpl0JanQqKR73KDjGs9421q8lfzYjLcZYuosliQVsHtBkgqZdCWpkElX2oZFxMN+mX2aH/bpStuwiLgoMw+8tX2aH1s8DTgi3g5sMnNn5ku2NMY8iIh7AP8nM2+MiEcA+wEnZeYPB8XbA3gTcJfMfFxE3Ad4SGYePyDW82ePGxFrgT/KzNf1jjUdv+yxzcS8M/Ag2nv1/Mz87sBYtwGeCtyVmc9YZh7bMcZDgIcCu0fEy2bu2gVY2yvOMvF/E7hXZp4QEbsDt8vMqwbF2h14ARs/n8/rHOcpy92fmX/bI06P7oULgAuBHYEDgf893X4d2KHD8TcQEddFxI83desdb8ZHgF9ExD1pZSTrgL8ZGO8vgdOBu0zb/wC8dFCsQyLiUxGxZ0TcFzgHuP2gWFD72IiI/wCcBzwFOBw4JyK6fmAX+RhwGHATcMPMracdgNvRktDtZ24/pj3GYSLiNcCrgGOmXdsDfz0w5MeAXwE+DZw6c+vtSdPt+cDxwJHT7c+Bfu+XjnOUzwG2m9neHjhn4Jzo1wP/mfZG2wX4T8CxA+NdNP37CuB3p58vHhjv/MUxgEsGxnsa8H3gauBho+Jspcf2dWC3me3dgK8PjPflkc/folh7T//uVBjzEiAWvX6XjYxX9dimeGcAe85s7wmc3uv4PQfS7jglvwW3m/aNcmhmviszr8vMH2fmu2mti1F+HhHPAJ4DfHLat/3AeDdExG5MXTcR8WDgRyMCRcS9gKNprfmrgWdHxE4jYk3KHtvkn4DrZravm/aN8j8j4v4Djz/rLhHxVeAKgIjYPyLeNTjmz7Jlo4XXb+fB8T4ZEY8fHGPWusy8Zmb7e8BevQ7ec2nH/wpcHBFn0b4FHw68tuPxF7shIo4EPkh78Z9B/1O4Wc8FXgi8MTOvioi7AX81MN7LgI8D94iILwG7M+608RPAizPz0xERU+zzgfsOilf52AC+AZwbER+jvVcOAy5b6AvNzLf0CBIRl0/H3w54bkRcCdxI+zxkZu7XI84ibwUeQ3s+ycxLI+LhA+LM+lBEvBe4Q0S8gHbq/b6B8Y4G/iAibgR+zvrnc5flf22zfSYiTgc+MG0/jda10UXX6oVpsOKgafPcHDtYcVfgbcDDaG/0LwEvzcxvjYo5E/uOtG/DywbH2Q74Ndqb7OuZ+fNBcXbJzB8v2rdPZv7DiHjT8Use2xTrNcvdn50GDCNi71uJc3WPOItinpuZB0XExZl5wLTv0swcsljt9KX8b4F7A4+mvX6nZ+aZI+JtLRHxZFrDEeALmfnRbsfe0qQbEcuWpmTmRVsUYE5ExOeAQ2mtmAuBa4EvZebLlvu9LYi3FngCG4/YdmmVLYq1UE3wq5n52NHVBJsYJf4RcHlmXjsi5kzsOwI/zJ6tjY1jPBj4SmZeN23vAuybmecOiPVh4C3AO2gNnqOBB2Tm03vHmol5eWYO7z6JiHtn5hWbyjEjc8v0BXqv6exvJ2Dtwuu5xcfukHTPWubuzMxHblGATcfdB3g3sEdm3i8i9qP1875hULyLM/OAaSR8XWa+JiIuG3TKSER8CvgpcDlw88L+Xq2yRbFOA04A/jAz959aoReP+mBFxKnAQ4CF984jaF9kd6MNhnbptomIVwMfmj64twFOo1XV3AQ8MzO7nTIuinsxcOBCYo+INcAFOaB2NiL+De2M71G0VucZwEsy8597x5qJeSLwjsw8f1SMKc5xmXnUJnLMyNzyAto6urtm5j2mMY/3ZOYhXQJ0Gu1bw+AR7yVifp5Wdzk7gjps1JiW/PakvakfOO0bOWI77NhLxKquJjid9mW5sL3HtG/Xnq8h8BXWNyyOAj5Hq2HdFzhv4OPb6Lkb9XoCj1ti3wsHv1+uoH1xfRO4bPpslL1fR99o1Rk7LPo8XN7r+F0G0jLz5oh4B3BAj+P9knbKzPNaF9MtbhoY71haYjg7M8+PiLvT6pFHOS0iHp2ZZwyMsaC6mmBdZn5vZvvaad8/R0TPvt2FUXZog00fyMxfAF+bWvOjXBkRL6GdiUErbbxyUKw/jogbM/OzABHxCuCRwHsGxYP2XJaJiB1pz+Fv0t6jX6S1PH86KOSNmfmzhdwyvVe6dUf1fON9JiKeCvztzBt9pO9Ps8QWEsXhwDXL/8rmy8xTgFNmtq+kzToa5Rzgo9Op6egR2+pqgs9FxCdZ/3w+ddq3M9Bzht+NEXE/WsnPwcDLZ+4bWRL3QuDPgD+ivT8/w7jLzBxKK6l6BfBY2gDXyNJJchoQjIg70SZFjXYSrczv7dP2M2mVQ0cMivf5iPgD4LYR8Vu0hP+JXgfvVr0QEdcBOwO/AH7C4LKOqaV5HG0q5A+Aq4Ajc8AI8RRvR9pMlfsy80bLzlMRZ+JdRfvwXD7qSywiHgj8Y2Z+d/o2/4+0BPhV4NU5qF9wGgF/Cq3lAu312yMzX9Q5zkHAibQvkbdm5uun/Y8Hnp2Zz+gZbzr2Wtr08CN7H3uZmHeilTRdCDxvdKMnIg4F3kybUXgtsDfwtcwcUmIYEV/NzPvc2r6O8dbQPuuz1Rn9SuK2dv/JFvS7rJ3+3Rm4fUG8U2iz4L5JmyBxBvC2gfG+AKwZ/Jguog0WQCuP+Q4t6b4e+PDg2AcAfwp8izag9uKt/Z7q+NjOBnYYHOM62pTfhX9/Cly/sD049qW0WX0XT9sHA8cPjPfXwINntg+ifbGNinfsou21wPt7Hb9b98LUejkSuFtmvj4i1tGm0p3XK8YiV0XE3wMnA58dFGPWPTPziIg4LDNPjIi/ofUtjXIl7ZT7NFqBPdC9ZGxtrm/NPg04LjM/AnwkIi7pGAe4peLkGdPt+7TXLjLz4N6xFsXdDXgN6/sEz6Z9sEbNSrsS+FJEfJyZCTs9X7vMHLk2xq35eWb+U0SsiYg1mXlWRLy1d5CZySbb02b5fXva3ptpBt4g6yLimMz8k4jYAfgQbXCti559uu+ilTY9ktZSuh54J/DAjjFm3Rt4IvAi4Pipj/CDmXn2oHgLAzw/nPoJvwvcaVAsaN0lV9FGUbsvHDRZGxHbZeZNwCFs2O84YqDpCtoX1RMz8xsAEfF7A+Is9kHamcNCH/yRtIT/qEHxvjnd1jB24aCFIv7PZuaPpu07AI/IzL8bGPaHEXE72nP6/oi4ljGzQZ844Ji/jOfRHtcxtFb8aZn5P3odvGef7kWZeWDVzJhFse9Iq1U8MjOHLGs31ed+hLak4wm0tSVenZkjR4mHiog/BB5Pa3XuxVRbGm0ltRMzs+ti2BHx74Gn02YR/j0tGf55Zt6tZ5wl4n45M++3aF9Jgf9oEXFJZv76on23fAY7x9orM789DXj+hPalciRtBbD3DzxzWIi/wcBdZn678/Fn66i3B95Lm+l6/BSvy2SMnkn3XNqg1vlT8t0dOGPEiz8T89/RTosfS1ti8uTp9HibNz1/r2TjgbuuBeFTedietNfqhmnfPrT1UYfM+Jk+tIfRuhkeSRud/mgOKo+LiLfQlnb80LTrcOBBmfnyTf/WFsUree2mWBtN0Bn1hRIzi6NHxEcyc2T1zmzckoG7qolePZPukbQEeCBtxPhw2kLYpyz7i5sf71vAxbQP0scXEsaAOMtO8+3cxzob9wzaKfDLaSVIzwH+X2a+akS8rWU6SzkCeFr2mvGz/tjX0foAg/WVNdAGRq7PcZU1Za9dRPwFrczundOuF9EGR39nQKzZs9ghrelNxL2U9uX86WyzQg8GnpWZzx8Qaw1wRGae3PvYt8TolXShzZWm9Q0G8JnM/Fq3g28ca6NFWgbFKVksZYm4F2bmb8y2ZCLi/Mwc1UeuTipfu+ms4Y9Z3z99JvCGEY2QRS3dsksCRcQFmfmAKfkekG0y1shFfS7IzAeMODZ0HCyJiF1pTf8PzOzbPjuvHhURr8zM/wa8MSI2+sbIzpcHGpVUfwkLz9s1EfEEWjnXrlvpb9kmxdZbMKXstZuS638Zcewl7B/t6ixBmziw0OgZvdRi1cDdgk9HxMtpZyuz1Sdd6tZ7di98i3YJmx/QXoQ70Eb4vwe8IDMv7BTnSZn5iYh4zlL3Z+aJPeIsEfdE4Oicrok2nRa/OcdNjngibaR/HW0mzi7A6zLz4yPirUSx9IIpt7zhR/SxTnGHv3YR8Y7MfHFEfIIlpqhm5qG9Ym0t04DuHrRyrdmBu72BU3vllCXiLnWtt8zMu3c5fsek+z5aQf3p0/ajaSU6J9AmERy03O9vRrwDB7ZUloq3UR9WZb+W/vUi4kHAt3Na13n6on4qbULGa3u1XGbi7Ujrw70nbRGY46dyvO4i4seZucs0mLyRzPz8iLiVpjLQYzLz8kX77w+8KTOftHX+si3TM+luNGK60Ke1VFlLh3hnAXcGPkyrWvhyz+MvEe9SWv3jD6btXYHP9x4ljrYc4aZkTlNZdesi4iLgUdkW0nk4rUTtd2nLO+6bmV3Xl4iIk2ldC18EHgdcnZlH94wxE2vFf+Ev1w8+uuRvqsW/DxtWn5zU49g9C+CviYhX0d7Y0CoZvhdtLvrNm/61zZOZB0e7UsVvA++NtlD0yTloPV1ayco5EbFQdnQE8MYBcZbqq9qZNhd8N9rEE/1ySmfcAfdZSAQRcTytTG2UxZde38Coqppid1jmvtuOCjoNnj+ClnQ/RfsCPZtW2rjFel6Y8pm0y3j83XTba9q3lpYYu8vM72bmn9FO6S4Blmslbmmsk4An0/qovwc8JTsttr0ozpsXbrQFfW5Luz7bB4EufUqryNpYv4TjIWw4XXzEjLtbBo1HdSvMWEuboHP7TdxWgguiLSi+gWmi0pD+3MnhtPfLdzPzucD+tAkgXXR742Xm92mnbkv5Rq84CyJiX1rr5am0K7ueDPz+gDiL++neM/oDNXVdvIw2aHAibabYD0bGXKE+QFum7/u0gZgvwi0DNCPWC14Y3YcNR/hHjO5fk5nHdjzePHopbXnTI1mfZB9Amxb/5IFxfzKVpd00nUFfSxsU7aJnydg+tGLwu7LhNb2GjBADf0Fr/T0mM78zKAa0pDfbT7cv7c0wRET8KW3Zw+OA+2fm9aNirXSZ+caI+AzrZ9wtDGCsYdMNhC2JN2QK+ibErf+XbVu2he4fOk2GWJjGfWpOC7YPdEG0NSzeR0v21wP/q9fBew6kXUpbrf5C1s/8YURZx9RP/FeZ+czex14i1uUz/XTb0S7zMqwoPCJupq0qdhMblgKNroXUNiQidu1dfaGNRbvq+C7Z8crfPfu1bsrMd9/6f9tymfmLiFgXETtk5s8Gh9ugny5ibAMjM3v2s2uFMuGOFe2K1bNLgXZLuj1buq+l9X18lA3Xfx119YGTaKf6w9YsneL8Yub4QRvY+hdseUorUkS8izaGszC79mnAN7PTlU16Jt2hsziWiLfkmghbcdqupBUgIq6g1XEvXH9xDfCVzNy3x/F7Vi8MXRN1iXgmV0kjfINW8rpwvcV1dKzA6lm9sBOtzGmvab77vYBfy8xP9oqxKN5ZLD3nfFS1hKQVbGYdi9sDX4uI86btg+g40aXnQNoJtMqFh07b/5d2McchSZcNL6e9I61ed3RBuqSV679XBOnZp7uw5mX55Xpm/obzMvNBVfEkrVzTxIjZOQddigJ6tnR/FhG3ZTrlj4h7MFPF0Ns0a2vBGtpMlW5T9SStThFxFHAs7bL2NzNVKtFpGn7PpPsa2sUG10XE+2kXH/ydjsdf7ELW9+neRFuur/vlOyStOq8A7jctbdBdz+qFM6el9B5M+2Y4esQfHREPBP5xoVpi0RqpX+0dT9Kq801aLf4QPft0HwZckpk3RMSzaBeofFtmXn0rv/qvjVO6Rqqk1SUiDqAVBpzLhhO9ulwKrGfSvYy2BNp+tD/4eOC3M3PJle23IM4tg3MR8U7aVVZfO213Xyxd0uoylYqdTVtV8Ja1wHtdCqz32gsZEYcB78zM4yNiRB/r2ojYblpe8RDgqJn7RqyRKml12T4zN7lA/JbqmaSui4hjgGcBD5+mzm3f8fgLqtdIlbS6nDZVMHyCAevI9OxeuDPtShHnZ+YXI2Iv2jXFulziYlGsB7N+jdQbpn37ALervFilpJVnW7oa8M7AT6dlF/cB7g2clpk/v5VflaRVo+farV8AbhMRvwqcATwb+MuOx5ekYSLilTM/H7Hovjf1itMz6UZm/gvtUjPvyswjWH+JDUmad0+f+fmYRfc9tleQrkk3Ih5Cu5jiqQOOL0kjxSZ+Xmp7s/VMikfTvh0+mplfiYi7A2d1PL4kjZSb+Hmp7c3WbSBNkrZlM5fmmr0sF9P2jpnZpQS2Z/XC7sArgfvS1rcFXFRckmb17F54P3AFcDfgdbQFaM7veHxJ2ub1bOlemJm/ERGXZeZ+077zM/OBXQJI0grQcxrwwiSIayLiCcB3gF2X+f+StOr0TLpviIhfAX4feDuwC/B7HY8vSdu8Le5eiIgdgRcC96QthXb8tAKYJGmRHkn3ZFrXwheBxwFXZ+bRHf42SVpxeiTdyzPz/tPP2wHnZeaBPf44SVppepSM3bKKmN0KkrS8Hi3dhVkcsOFMjqCtQbnLFgWQpBXEacCSVMhVwCSpkElXkgqZdCWpkElXkgqZdCWp0P8HqQ9OfyGOdqEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(train_copy.isnull(),yticklabels=False,cbar=False,cmap='viridis')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Let's go ahead"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Converting Categorical Features \n",
    "\n",
    "We'll need to convert categorical features to one-hot encoding! Otherwise our machine learning algorithm won't be able to directly take in those features as inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 712 entries, 0 to 711\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  712 non-null    int64  \n",
      " 1   Survived     712 non-null    int64  \n",
      " 2   Pclass       712 non-null    int64  \n",
      " 3   Name         712 non-null    object \n",
      " 4   Sex          712 non-null    object \n",
      " 5   Age          566 non-null    float64\n",
      " 6   SibSp        712 non-null    int64  \n",
      " 7   Parch        712 non-null    int64  \n",
      " 8   Ticket       712 non-null    object \n",
      " 9   Fare         712 non-null    float64\n",
      " 10  Cabin        168 non-null    object \n",
      " 11  Embarked     710 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 66.9+ KB\n"
     ]
    }
   ],
   "source": [
    "df_train_raw.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocessing(df):\n",
    "\n",
    "    dfresult = pd.DataFrame()\n",
    "    \n",
    "    # Let's keep features in the original order\n",
    "\n",
    "    # Pclass -> one-hot encoding\n",
    "    pclass = pd.get_dummies(df['Pclass'])\n",
    "    pclass.columns = ['Pclass_' + str(x) for x in pclass.columns ]\n",
    "    dfresult = pd.concat([dfresult, pclass],axis = 1)\n",
    "\n",
    "    # Sex -> one-hot encoding\n",
    "    sex = pd.get_dummies(df['Sex'])\n",
    "    dfresult = pd.concat([dfresult,sex],axis = 1)\n",
    "\n",
    "    # Age -> replace with imputation\n",
    "    dfresult['Age'] = df[['Age','Pclass']].apply(impute_age,axis=1)\n",
    "\n",
    "    # SibSp, Parch, Fare\n",
    "    dfresult['SibSp'] = df['SibSp']\n",
    "    dfresult['Parch'] = df['Parch']\n",
    "    dfresult['Fare'] = df['Fare']\n",
    "\n",
    "    # Embarked -> one-hot encoding\n",
    "    embarked = pd.get_dummies(df['Embarked'],dummy_na=True)\n",
    "    embarked.columns = ['Embarked_' + str(x) for x in embarked.columns]\n",
    "    dfresult = pd.concat([dfresult, embarked],axis = 1)\n",
    "\n",
    "    return dfresult"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train.shape = (712, 13)\n",
      "x_test.shape = (179, 13)\n"
     ]
    }
   ],
   "source": [
    "x_train = preprocessing(df_train_raw)\n",
    "y_train = df_train_raw['Survived'].values\n",
    "\n",
    "x_test = preprocessing(df_test_raw)\n",
    "y_test = df_test_raw['Survived'].values\n",
    "\n",
    "print(\"x_train.shape =\", x_train.shape )\n",
    "print(\"x_test.shape =\", x_test.shape )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass_1</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "      <th>female</th>\n",
       "      <th>male</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Embarked_nan</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>76.7292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>39.6875</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass_1  Pclass_2  Pclass_3  female  male   Age  SibSp  Parch     Fare  \\\n",
       "0         1         0         0       0     1  55.0      0      0  30.5000   \n",
       "1         1         0         0       1     0  49.0      1      0  76.7292   \n",
       "2         0         1         0       1     0  36.0      0      0  13.0000   \n",
       "3         0         1         0       0     1  19.0      0      0  13.0000   \n",
       "4         0         0         1       0     1  14.0      4      1  39.6875   \n",
       "\n",
       "   Embarked_C  Embarked_Q  Embarked_S  Embarked_nan  \n",
       "0           0           0           1             0  \n",
       "1           1           0           0             0  \n",
       "2           0           0           1             0  \n",
       "3           0           0           1             0  \n",
       "4           0           0           1             0  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Our data is ready for our model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Choose a model\n",
    "\n",
    "Usually there are three ways of modeling using APIs of Keras. \n",
    "\n",
    "Here we take the simplest way: sequential modeling using function Sequential()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 10)                140       \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 20)                220       \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 21        \n",
      "=================================================================\n",
      "Total params: 381\n",
      "Trainable params: 381\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf \n",
    "from tensorflow.keras import models, layers\n",
    "\n",
    "tf.keras.backend.clear_session()\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(10,activation = 'relu',input_shape=(13,)))\n",
    "model.add(layers.Dense(20,activation = 'relu' ))\n",
    "model.add(layers.Dense(1,activation = 'sigmoid' ))\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Training\n",
    "\n",
    "Let's configure modle using `model.compile`\n",
    "\n",
    "There are three ways for model training. Here we introduce the simplest way: using internal function fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 569 samples, validate on 143 samples\n",
      "Epoch 1/30\n",
      "569/569 [==============================] - 1s 2ms/sample - loss: 1.9260 - accuracy: 0.5571 - val_loss: 0.7518 - val_accuracy: 0.6224\n",
      "Epoch 2/30\n",
      "569/569 [==============================] - 0s 103us/sample - loss: 0.7476 - accuracy: 0.6731 - val_loss: 0.6442 - val_accuracy: 0.6643\n",
      "Epoch 3/30\n",
      "569/569 [==============================] - 0s 113us/sample - loss: 0.6676 - accuracy: 0.6749 - val_loss: 0.6472 - val_accuracy: 0.6713\n",
      "Epoch 4/30\n",
      "569/569 [==============================] - 0s 109us/sample - loss: 0.6323 - accuracy: 0.6801 - val_loss: 0.6468 - val_accuracy: 0.6503\n",
      "Epoch 5/30\n",
      "569/569 [==============================] - 0s 111us/sample - loss: 0.6534 - accuracy: 0.6872 - val_loss: 0.6870 - val_accuracy: 0.5944\n",
      "Epoch 6/30\n",
      "569/569 [==============================] - 0s 104us/sample - loss: 0.7362 - accuracy: 0.6397 - val_loss: 0.7172 - val_accuracy: 0.6154\n",
      "Epoch 7/30\n",
      "569/569 [==============================] - 0s 95us/sample - loss: 0.6426 - accuracy: 0.6907 - val_loss: 0.7026 - val_accuracy: 0.5944\n",
      "Epoch 8/30\n",
      "569/569 [==============================] - 0s 103us/sample - loss: 0.6277 - accuracy: 0.6784 - val_loss: 0.6456 - val_accuracy: 0.6713\n",
      "Epoch 9/30\n",
      "569/569 [==============================] - 0s 105us/sample - loss: 0.6177 - accuracy: 0.6819 - val_loss: 0.6631 - val_accuracy: 0.6573\n",
      "Epoch 10/30\n",
      "569/569 [==============================] - 0s 101us/sample - loss: 0.6458 - accuracy: 0.6819 - val_loss: 0.6406 - val_accuracy: 0.6643\n",
      "Epoch 11/30\n",
      "569/569 [==============================] - 0s 96us/sample - loss: 0.6747 - accuracy: 0.6538 - val_loss: 0.6353 - val_accuracy: 0.6713\n",
      "Epoch 12/30\n",
      "569/569 [==============================] - 0s 102us/sample - loss: 0.6237 - accuracy: 0.7012 - val_loss: 0.6365 - val_accuracy: 0.6783\n",
      "Epoch 13/30\n",
      "569/569 [==============================] - 0s 101us/sample - loss: 0.6085 - accuracy: 0.6889 - val_loss: 0.6621 - val_accuracy: 0.6364\n",
      "Epoch 14/30\n",
      "569/569 [==============================] - 0s 105us/sample - loss: 0.6256 - accuracy: 0.6907 - val_loss: 0.7056 - val_accuracy: 0.5874\n",
      "Epoch 15/30\n",
      "569/569 [==============================] - 0s 93us/sample - loss: 0.6421 - accuracy: 0.6573 - val_loss: 0.6619 - val_accuracy: 0.6783\n",
      "Epoch 16/30\n",
      "569/569 [==============================] - 0s 103us/sample - loss: 0.6026 - accuracy: 0.6837 - val_loss: 0.6685 - val_accuracy: 0.6643\n",
      "Epoch 17/30\n",
      "569/569 [==============================] - 0s 101us/sample - loss: 0.6072 - accuracy: 0.6889 - val_loss: 0.6333 - val_accuracy: 0.6643\n",
      "Epoch 18/30\n",
      "569/569 [==============================] - 0s 101us/sample - loss: 0.5996 - accuracy: 0.6854 - val_loss: 0.6673 - val_accuracy: 0.6294\n",
      "Epoch 19/30\n",
      "569/569 [==============================] - 0s 95us/sample - loss: 0.6229 - accuracy: 0.6889 - val_loss: 0.6369 - val_accuracy: 0.6853\n",
      "Epoch 20/30\n",
      "569/569 [==============================] - 0s 99us/sample - loss: 0.6087 - accuracy: 0.6942 - val_loss: 0.7957 - val_accuracy: 0.5944\n",
      "Epoch 21/30\n",
      "569/569 [==============================] - 0s 97us/sample - loss: 0.6072 - accuracy: 0.6819 - val_loss: 0.6359 - val_accuracy: 0.6853\n",
      "Epoch 22/30\n",
      "569/569 [==============================] - 0s 94us/sample - loss: 0.5975 - accuracy: 0.7012 - val_loss: 0.6339 - val_accuracy: 0.6573\n",
      "Epoch 23/30\n",
      "569/569 [==============================] - 0s 99us/sample - loss: 0.5927 - accuracy: 0.6854 - val_loss: 0.6372 - val_accuracy: 0.6643\n",
      "Epoch 24/30\n",
      "569/569 [==============================] - 0s 102us/sample - loss: 0.6027 - accuracy: 0.6907 - val_loss: 0.6480 - val_accuracy: 0.6364\n",
      "Epoch 25/30\n",
      "569/569 [==============================] - 0s 103us/sample - loss: 0.6115 - accuracy: 0.6661 - val_loss: 0.6415 - val_accuracy: 0.6783\n",
      "Epoch 26/30\n",
      "569/569 [==============================] - 0s 104us/sample - loss: 0.6035 - accuracy: 0.6907 - val_loss: 0.6406 - val_accuracy: 0.6713\n",
      "Epoch 27/30\n",
      "569/569 [==============================] - 0s 104us/sample - loss: 0.5984 - accuracy: 0.7012 - val_loss: 0.6334 - val_accuracy: 0.6713\n",
      "Epoch 28/30\n",
      "569/569 [==============================] - 0s 101us/sample - loss: 0.5947 - accuracy: 0.6995 - val_loss: 0.6607 - val_accuracy: 0.6434\n",
      "Epoch 29/30\n",
      "569/569 [==============================] - 0s 97us/sample - loss: 0.5957 - accuracy: 0.6889 - val_loss: 0.7162 - val_accuracy: 0.5944\n",
      "Epoch 30/30\n",
      "569/569 [==============================] - 0s 94us/sample - loss: 0.6119 - accuracy: 0.6643 - val_loss: 0.6352 - val_accuracy: 0.6713\n"
     ]
    }
   ],
   "source": [
    "# Convert dataframe data into np array\n",
    "x_train = np.asarray(x_train)\n",
    "y_train = np.asarray(y_train)\n",
    "\n",
    "# Get around with KMP duplicate issue\n",
    "import os\n",
    "os.environ['KMP_DUPLICATE_LIB_OK']='True'\n",
    "\n",
    "# Use binary cross entropy loss function for binary classification\n",
    "model.compile(optimizer='sgd',\n",
    "            loss='binary_crossentropy',\n",
    "            metrics=['accuracy'])\n",
    "\n",
    "history = model.fit(x_train,y_train,\n",
    "                    batch_size= 64,\n",
    "                    epochs= 30,\n",
    "                    validation_split=0.2\n",
    "                   )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'svg'\n",
    "\n",
    "def plot_metric(history, metric):\n",
    "    train_metrics = history.history[metric]\n",
    "    val_metrics = history.history['val_'+metric]\n",
    "    epochs = range(1, len(train_metrics) + 1)\n",
    "    plt.plot(epochs, train_metrics, 'bo--')\n",
    "    plt.plot(epochs, val_metrics, 'ro-')\n",
    "    plt.title('Training and validation '+ metric)\n",
    "    plt.xlabel(\"Epochs\")\n",
    "    plt.ylabel(metric)\n",
    "    plt.legend([\"train_\"+metric, 'val_'+metric])\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"277.314375pt\" version=\"1.1\" viewBox=\"0 0 385.78125 277.314375\" width=\"385.78125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 277.314375 \n",
       "L 385.78125 277.314375 \n",
       "L 385.78125 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 43.78125 239.758125 \n",
       "L 378.58125 239.758125 \n",
       "L 378.58125 22.318125 \n",
       "L 43.78125 22.318125 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"m4bb7b6015b\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"48.504134\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-48\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(45.322884 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"100.980623\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"DejaVuSans-53\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(97.799373 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"153.457112\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 10 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-49\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(147.094612 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"205.933601\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 15 -->\n",
       "      <g transform=\"translate(199.571101 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"258.41009\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-50\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(252.04759 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"310.886579\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 25 -->\n",
       "      <g transform=\"translate(304.524079 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"363.363068\" xlink:href=\"#m4bb7b6015b\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-51\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(357.000568 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-51\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_8\">\n",
       "     <!-- Epochs -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.8125 72.90625 \n",
       "L 55.90625 72.90625 \n",
       "L 55.90625 64.59375 \n",
       "L 19.671875 64.59375 \n",
       "L 19.671875 43.015625 \n",
       "L 54.390625 43.015625 \n",
       "L 54.390625 34.71875 \n",
       "L 19.671875 34.71875 \n",
       "L 19.671875 8.296875 \n",
       "L 56.78125 8.296875 \n",
       "L 56.78125 0 \n",
       "L 9.8125 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "z\n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-112\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-111\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-99\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-104\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "z\n",
       "\" id=\"DejaVuSans-115\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(193.265625 268.034687)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-112\"/>\n",
       "      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-104\"/>\n",
       "      <use x=\"306.201172\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m4477e5788e\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"228.786625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.6 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-46\"/>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "z\n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-54\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 232.585844)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"199.136017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.8 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "z\n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "z\n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "z\n",
       "\" id=\"DejaVuSans-56\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 202.935236)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-56\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"169.485409\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(20.878125 173.284628)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"139.834801\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 1.2 -->\n",
       "      <g transform=\"translate(20.878125 143.63402)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"110.184193\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 1.4 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-52\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 113.983411)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-52\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"80.533585\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 1.6 -->\n",
       "      <g transform=\"translate(20.878125 84.332803)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m4477e5788e\" y=\"50.882977\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 1.8 -->\n",
       "      <g transform=\"translate(20.878125 54.682195)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-56\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_16\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-108\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798438 140.695937)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-108\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"88.964844\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "      <use x=\"141.064453\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_15\">\n",
       "    <path clip-path=\"url(#pb281098264)\" d=\"M 58.999432 32.201761 \n",
       "L 69.49473 206.90921 \n",
       "L 79.990027 218.758153 \n",
       "L 90.485325 223.993576 \n",
       "L 100.980623 220.873551 \n",
       "L 111.475921 208.594387 \n",
       "L 121.971219 222.478245 \n",
       "L 132.466516 224.685535 \n",
       "L 142.961814 226.156246 \n",
       "L 153.457112 221.989914 \n",
       "L 163.95241 217.710343 \n",
       "L 174.447708 225.267815 \n",
       "L 184.943005 227.529434 \n",
       "L 195.438303 224.989859 \n",
       "L 205.933601 222.550411 \n",
       "L 216.428899 228.404426 \n",
       "L 226.924197 227.723 \n",
       "L 237.419495 228.851222 \n",
       "L 247.914792 225.393788 \n",
       "L 258.41009 227.489749 \n",
       "L 268.905388 227.726073 \n",
       "L 279.400686 229.156591 \n",
       "L 289.895984 229.874489 \n",
       "L 300.391281 228.388359 \n",
       "L 310.886579 227.087516 \n",
       "L 321.381877 228.271476 \n",
       "L 331.877175 229.02579 \n",
       "L 342.372473 229.570561 \n",
       "L 352.86777 229.418031 \n",
       "L 363.363068 227.018878 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"ma1304c2570\" style=\"stroke:#0000ff;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#pb281098264)\">\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"58.999432\" xlink:href=\"#ma1304c2570\" y=\"32.201761\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"69.49473\" xlink:href=\"#ma1304c2570\" y=\"206.90921\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"79.990027\" xlink:href=\"#ma1304c2570\" y=\"218.758153\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"90.485325\" xlink:href=\"#ma1304c2570\" y=\"223.993576\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"100.980623\" xlink:href=\"#ma1304c2570\" y=\"220.873551\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"111.475921\" xlink:href=\"#ma1304c2570\" y=\"208.594387\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"121.971219\" xlink:href=\"#ma1304c2570\" y=\"222.478245\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"132.466516\" xlink:href=\"#ma1304c2570\" y=\"224.685535\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"142.961814\" xlink:href=\"#ma1304c2570\" y=\"226.156246\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"153.457112\" xlink:href=\"#ma1304c2570\" y=\"221.989914\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"163.95241\" xlink:href=\"#ma1304c2570\" y=\"217.710343\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"174.447708\" xlink:href=\"#ma1304c2570\" y=\"225.267815\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"184.943005\" xlink:href=\"#ma1304c2570\" y=\"227.529434\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"195.438303\" xlink:href=\"#ma1304c2570\" y=\"224.989859\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"205.933601\" xlink:href=\"#ma1304c2570\" y=\"222.550411\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"216.428899\" xlink:href=\"#ma1304c2570\" y=\"228.404426\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"226.924197\" xlink:href=\"#ma1304c2570\" y=\"227.723\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"237.419495\" xlink:href=\"#ma1304c2570\" y=\"228.851222\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"247.914792\" xlink:href=\"#ma1304c2570\" y=\"225.393788\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"258.41009\" xlink:href=\"#ma1304c2570\" y=\"227.489749\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"268.905388\" xlink:href=\"#ma1304c2570\" y=\"227.726073\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"279.400686\" xlink:href=\"#ma1304c2570\" y=\"229.156591\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"289.895984\" xlink:href=\"#ma1304c2570\" y=\"229.874489\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"300.391281\" xlink:href=\"#ma1304c2570\" y=\"228.388359\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"310.886579\" xlink:href=\"#ma1304c2570\" y=\"227.087516\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"321.381877\" xlink:href=\"#ma1304c2570\" y=\"228.271476\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"331.877175\" xlink:href=\"#ma1304c2570\" y=\"229.02579\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"342.372473\" xlink:href=\"#ma1304c2570\" y=\"229.570561\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"352.86777\" xlink:href=\"#ma1304c2570\" y=\"229.418031\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"363.363068\" xlink:href=\"#ma1304c2570\" y=\"227.018878\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_16\">\n",
       "    <path clip-path=\"url(#pb281098264)\" d=\"M 58.999432 206.281623 \n",
       "L 69.49473 222.235555 \n",
       "L 79.990027 221.789601 \n",
       "L 90.485325 221.846432 \n",
       "L 100.980623 215.895412 \n",
       "L 111.475921 211.417923 \n",
       "L 121.971219 213.582468 \n",
       "L 132.466516 222.0302 \n",
       "L 142.961814 219.42843 \n",
       "L 153.457112 222.770844 \n",
       "L 163.95241 223.559242 \n",
       "L 174.447708 223.373413 \n",
       "L 184.943005 219.577982 \n",
       "L 195.438303 213.131515 \n",
       "L 205.933601 219.605541 \n",
       "L 216.428899 218.624453 \n",
       "L 226.924197 223.850914 \n",
       "L 237.419495 218.813835 \n",
       "L 247.914792 223.318989 \n",
       "L 258.41009 199.777907 \n",
       "L 268.905388 223.470502 \n",
       "L 279.400686 223.759591 \n",
       "L 289.895984 223.267362 \n",
       "L 300.391281 221.66414 \n",
       "L 310.886579 222.636984 \n",
       "L 321.381877 222.772269 \n",
       "L 331.877175 223.834269 \n",
       "L 342.372473 219.789929 \n",
       "L 352.86777 211.55604 \n",
       "L 363.363068 223.56228 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"m8de3cf7abc\" style=\"stroke:#ff0000;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#pb281098264)\">\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"58.999432\" xlink:href=\"#m8de3cf7abc\" y=\"206.281623\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"69.49473\" xlink:href=\"#m8de3cf7abc\" y=\"222.235555\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"79.990027\" xlink:href=\"#m8de3cf7abc\" y=\"221.789601\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"90.485325\" xlink:href=\"#m8de3cf7abc\" y=\"221.846432\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"100.980623\" xlink:href=\"#m8de3cf7abc\" y=\"215.895412\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"111.475921\" xlink:href=\"#m8de3cf7abc\" y=\"211.417923\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"121.971219\" xlink:href=\"#m8de3cf7abc\" y=\"213.582468\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"132.466516\" xlink:href=\"#m8de3cf7abc\" y=\"222.0302\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"142.961814\" xlink:href=\"#m8de3cf7abc\" y=\"219.42843\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"153.457112\" xlink:href=\"#m8de3cf7abc\" y=\"222.770844\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"163.95241\" xlink:href=\"#m8de3cf7abc\" y=\"223.559242\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"174.447708\" xlink:href=\"#m8de3cf7abc\" y=\"223.373413\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"184.943005\" xlink:href=\"#m8de3cf7abc\" y=\"219.577982\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"195.438303\" xlink:href=\"#m8de3cf7abc\" y=\"213.131515\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"205.933601\" xlink:href=\"#m8de3cf7abc\" y=\"219.605541\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"216.428899\" xlink:href=\"#m8de3cf7abc\" y=\"218.624453\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"226.924197\" xlink:href=\"#m8de3cf7abc\" y=\"223.850914\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"237.419495\" xlink:href=\"#m8de3cf7abc\" y=\"218.813835\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"247.914792\" xlink:href=\"#m8de3cf7abc\" y=\"223.318989\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"258.41009\" xlink:href=\"#m8de3cf7abc\" y=\"199.777907\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"268.905388\" xlink:href=\"#m8de3cf7abc\" y=\"223.470502\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"279.400686\" xlink:href=\"#m8de3cf7abc\" y=\"223.759591\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"289.895984\" xlink:href=\"#m8de3cf7abc\" y=\"223.267362\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"300.391281\" xlink:href=\"#m8de3cf7abc\" y=\"221.66414\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"310.886579\" xlink:href=\"#m8de3cf7abc\" y=\"222.636984\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"321.381877\" xlink:href=\"#m8de3cf7abc\" y=\"222.772269\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"331.877175\" xlink:href=\"#m8de3cf7abc\" y=\"223.834269\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"342.372473\" xlink:href=\"#m8de3cf7abc\" y=\"219.789929\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"352.86777\" xlink:href=\"#m8de3cf7abc\" y=\"211.55604\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"363.363068\" xlink:href=\"#m8de3cf7abc\" y=\"223.56228\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 43.78125 239.758125 \n",
       "L 43.78125 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 378.58125 239.758125 \n",
       "L 378.58125 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 43.78125 239.758125 \n",
       "L 378.58125 239.758125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 43.78125 22.318125 \n",
       "L 378.58125 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"text_17\">\n",
       "    <!-- Training and validation loss -->\n",
       "    <defs>\n",
       "     <path d=\"M -0.296875 72.90625 \n",
       "L 61.375 72.90625 \n",
       "L 61.375 64.59375 \n",
       "L 35.5 64.59375 \n",
       "L 35.5 0 \n",
       "L 25.59375 0 \n",
       "L 25.59375 64.59375 \n",
       "L -0.296875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-84\"/>\n",
       "     <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-114\"/>\n",
       "     <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-97\"/>\n",
       "     <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-105\"/>\n",
       "     <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-110\"/>\n",
       "     <path d=\"M 45.40625 27.984375 \n",
       "Q 45.40625 37.75 41.375 43.109375 \n",
       "Q 37.359375 48.484375 30.078125 48.484375 \n",
       "Q 22.859375 48.484375 18.828125 43.109375 \n",
       "Q 14.796875 37.75 14.796875 27.984375 \n",
       "Q 14.796875 18.265625 18.828125 12.890625 \n",
       "Q 22.859375 7.515625 30.078125 7.515625 \n",
       "Q 37.359375 7.515625 41.375 12.890625 \n",
       "Q 45.40625 18.265625 45.40625 27.984375 \n",
       "z\n",
       "M 54.390625 6.78125 \n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \n",
       "Q 42 -20.796875 29.203125 -20.796875 \n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \n",
       "L 12.109375 -9.1875 \n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \n",
       "Q 23.78125 -13.375 27.78125 -13.375 \n",
       "Q 36.625 -13.375 41.015625 -8.765625 \n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \n",
       "L 45.40625 9.625 \n",
       "Q 42.625 4.78125 38.28125 2.390625 \n",
       "Q 33.9375 0 27.875 0 \n",
       "Q 17.828125 0 11.671875 7.65625 \n",
       "Q 5.515625 15.328125 5.515625 27.984375 \n",
       "Q 5.515625 40.671875 11.671875 48.328125 \n",
       "Q 17.828125 56 27.875 56 \n",
       "Q 33.9375 56 38.28125 53.609375 \n",
       "Q 42.625 51.21875 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-103\"/>\n",
       "     <path id=\"DejaVuSans-32\"/>\n",
       "     <path d=\"M 45.40625 46.390625 \n",
       "L 45.40625 75.984375 \n",
       "L 54.390625 75.984375 \n",
       "L 54.390625 0 \n",
       "L 45.40625 0 \n",
       "L 45.40625 8.203125 \n",
       "Q 42.578125 3.328125 38.25 0.953125 \n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \n",
       "Q 5.515625 14.40625 5.515625 27.296875 \n",
       "Q 5.515625 40.1875 11.734375 48.09375 \n",
       "Q 17.96875 56 27.875 56 \n",
       "Q 33.9375 56 38.25 53.625 \n",
       "Q 42.578125 51.265625 45.40625 46.390625 \n",
       "z\n",
       "M 14.796875 27.296875 \n",
       "Q 14.796875 17.390625 18.875 11.75 \n",
       "Q 22.953125 6.109375 30.078125 6.109375 \n",
       "Q 37.203125 6.109375 41.296875 11.75 \n",
       "Q 45.40625 17.390625 45.40625 27.296875 \n",
       "Q 45.40625 37.203125 41.296875 42.84375 \n",
       "Q 37.203125 48.484375 30.078125 48.484375 \n",
       "Q 22.953125 48.484375 18.875 42.84375 \n",
       "Q 14.796875 37.203125 14.796875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-100\"/>\n",
       "     <path d=\"M 2.984375 54.6875 \n",
       "L 12.5 54.6875 \n",
       "L 29.59375 8.796875 \n",
       "L 46.6875 54.6875 \n",
       "L 56.203125 54.6875 \n",
       "L 35.6875 0 \n",
       "L 23.484375 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-118\"/>\n",
       "     <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-116\"/>\n",
       "    </defs>\n",
       "    <g transform=\"translate(129.37125 16.318125)scale(0.12 -0.12)\">\n",
       "     <use xlink:href=\"#DejaVuSans-84\"/>\n",
       "     <use x=\"46.333984\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "     <use x=\"87.447266\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"148.726562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"176.509766\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"239.888672\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"267.671875\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"331.050781\" xlink:href=\"#DejaVuSans-103\"/>\n",
       "     <use x=\"394.527344\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"426.314453\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"487.59375\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"550.972656\" xlink:href=\"#DejaVuSans-100\"/>\n",
       "     <use x=\"614.449219\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"646.236328\" xlink:href=\"#DejaVuSans-118\"/>\n",
       "     <use x=\"705.416016\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"766.695312\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "     <use x=\"794.478516\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"822.261719\" xlink:href=\"#DejaVuSans-100\"/>\n",
       "     <use x=\"885.738281\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"947.017578\" xlink:href=\"#DejaVuSans-116\"/>\n",
       "     <use x=\"986.226562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"1014.009766\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "     <use x=\"1075.191406\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"1138.570312\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"1170.357422\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "     <use x=\"1198.140625\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "     <use x=\"1259.322266\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     <use x=\"1311.421875\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 291.990625 60.230625 \n",
       "L 371.58125 60.230625 \n",
       "Q 373.58125 60.230625 373.58125 58.230625 \n",
       "L 373.58125 29.318125 \n",
       "Q 373.58125 27.318125 371.58125 27.318125 \n",
       "L 291.990625 27.318125 \n",
       "Q 289.990625 27.318125 289.990625 29.318125 \n",
       "L 289.990625 58.230625 \n",
       "Q 289.990625 60.230625 291.990625 60.230625 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_17\">\n",
       "     <path d=\"M 293.990625 35.416562 \n",
       "L 313.990625 35.416562 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_18\">\n",
       "     <g>\n",
       "      <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"303.990625\" xlink:href=\"#ma1304c2570\" y=\"35.416562\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_18\">\n",
       "     <!-- train_loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 50.984375 -16.609375 \n",
       "L 50.984375 -23.578125 \n",
       "L -0.984375 -23.578125 \n",
       "L -0.984375 -16.609375 \n",
       "z\n",
       "\" id=\"DejaVuSans-95\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(321.990625 38.916562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"232.763672\" xlink:href=\"#DejaVuSans-95\"/>\n",
       "      <use x=\"282.763672\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "      <use x=\"310.546875\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"371.728516\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "      <use x=\"423.828125\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_19\">\n",
       "     <path d=\"M 293.990625 50.372812 \n",
       "L 313.990625 50.372812 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_20\">\n",
       "     <g>\n",
       "      <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"303.990625\" xlink:href=\"#m8de3cf7abc\" y=\"50.372812\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_19\">\n",
       "     <!-- val_loss -->\n",
       "     <g transform=\"translate(321.990625 53.872812)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-118\"/>\n",
       "      <use x=\"59.179688\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"120.458984\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "      <use x=\"148.242188\" xlink:href=\"#DejaVuSans-95\"/>\n",
       "      <use x=\"198.242188\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "      <use x=\"226.025391\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"287.207031\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "      <use x=\"339.306641\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pb281098264\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"22.318125\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metric(history,\"loss\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"277.314375pt\" version=\"1.1\" viewBox=\"0 0 392.14375 277.314375\" width=\"392.14375pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 277.314375 \n",
       "L 392.14375 277.314375 \n",
       "L 392.14375 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 50.14375 239.758125 \n",
       "L 384.94375 239.758125 \n",
       "L 384.94375 22.318125 \n",
       "L 50.14375 22.318125 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"m0e4b7535ec\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.866634\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-48\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(51.685384 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"107.343123\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"DejaVuSans-53\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(104.161873 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"159.819612\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 10 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-49\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(153.457112 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"212.296101\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 15 -->\n",
       "      <g transform=\"translate(205.933601 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"264.77259\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-50\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(258.41009 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"317.249079\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 25 -->\n",
       "      <g transform=\"translate(310.886579 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"369.725568\" xlink:href=\"#m0e4b7535ec\" y=\"239.758125\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-51\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(363.363068 254.356562)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-51\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_8\">\n",
       "     <!-- Epochs -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.8125 72.90625 \n",
       "L 55.90625 72.90625 \n",
       "L 55.90625 64.59375 \n",
       "L 19.671875 64.59375 \n",
       "L 19.671875 43.015625 \n",
       "L 54.390625 43.015625 \n",
       "L 54.390625 34.71875 \n",
       "L 19.671875 34.71875 \n",
       "L 19.671875 8.296875 \n",
       "L 56.78125 8.296875 \n",
       "L 56.78125 0 \n",
       "L 9.8125 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "z\n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-112\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-111\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-99\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-104\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "z\n",
       "\" id=\"DejaVuSans-115\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(199.628125 268.034687)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"63.183594\" xlink:href=\"#DejaVuSans-112\"/>\n",
       "      <use x=\"126.660156\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "      <use x=\"187.841797\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"242.822266\" xlink:href=\"#DejaVuSans-104\"/>\n",
       "      <use x=\"306.201172\" xlink:href=\"#DejaVuSans-115\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m9a792087dc\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"225.921047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.56 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-46\"/>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "z\n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-54\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 229.720266)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"198.487928\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.58 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "z\n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "z\n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "z\n",
       "\" id=\"DejaVuSans-56\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 202.287146)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-56\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"171.054808\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.60 -->\n",
       "      <g transform=\"translate(20.878125 174.854027)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"143.621689\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.62 -->\n",
       "      <g transform=\"translate(20.878125 147.420908)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-50\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"116.188569\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.64 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-52\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 119.987788)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-52\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"88.75545\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 0.66 -->\n",
       "      <g transform=\"translate(20.878125 92.554669)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"61.32233\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 0.68 -->\n",
       "      <g transform=\"translate(20.878125 65.121549)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-56\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"50.14375\" xlink:href=\"#m9a792087dc\" y=\"33.889211\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_16\">\n",
       "      <!-- 0.70 -->\n",
       "      <defs>\n",
       "       <path d=\"M 8.203125 72.90625 \n",
       "L 55.078125 72.90625 \n",
       "L 55.078125 68.703125 \n",
       "L 28.609375 0 \n",
       "L 18.3125 0 \n",
       "L 43.21875 64.59375 \n",
       "L 8.203125 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-55\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 37.68843)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-55\"/>\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-48\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_17\">\n",
       "     <!-- accuracy -->\n",
       "     <defs>\n",
       "      <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-97\"/>\n",
       "      <path d=\"M 8.5 21.578125 \n",
       "L 8.5 54.6875 \n",
       "L 17.484375 54.6875 \n",
       "L 17.484375 21.921875 \n",
       "Q 17.484375 14.15625 20.5 10.265625 \n",
       "Q 23.53125 6.390625 29.59375 6.390625 \n",
       "Q 36.859375 6.390625 41.078125 11.03125 \n",
       "Q 45.3125 15.671875 45.3125 23.6875 \n",
       "L 45.3125 54.6875 \n",
       "L 54.296875 54.6875 \n",
       "L 54.296875 0 \n",
       "L 45.3125 0 \n",
       "L 45.3125 8.40625 \n",
       "Q 42.046875 3.421875 37.71875 1 \n",
       "Q 33.40625 -1.421875 27.6875 -1.421875 \n",
       "Q 18.265625 -1.421875 13.375 4.4375 \n",
       "Q 8.5 10.296875 8.5 21.578125 \n",
       "z\n",
       "M 31.109375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-117\"/>\n",
       "      <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-114\"/>\n",
       "      <path d=\"M 32.171875 -5.078125 \n",
       "Q 28.375 -14.84375 24.75 -17.8125 \n",
       "Q 21.140625 -20.796875 15.09375 -20.796875 \n",
       "L 7.90625 -20.796875 \n",
       "L 7.90625 -13.28125 \n",
       "L 13.1875 -13.28125 \n",
       "Q 16.890625 -13.28125 18.9375 -11.515625 \n",
       "Q 21 -9.765625 23.484375 -3.21875 \n",
       "L 25.09375 0.875 \n",
       "L 2.984375 54.6875 \n",
       "L 12.5 54.6875 \n",
       "L 29.59375 11.921875 \n",
       "L 46.6875 54.6875 \n",
       "L 56.203125 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-121\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798438 153.5975)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"61.279297\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"116.259766\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"171.240234\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"234.619141\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"275.732422\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"337.011719\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"391.992188\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_16\">\n",
       "    <path clip-path=\"url(#pa4df6105f0)\" d=\"M 65.361932 229.874489 \n",
       "L 75.85723 70.772048 \n",
       "L 86.352527 68.36144 \n",
       "L 96.847825 61.129456 \n",
       "L 107.343123 51.486945 \n",
       "L 117.838421 116.574237 \n",
       "L 128.333719 46.665649 \n",
       "L 138.829016 63.540144 \n",
       "L 149.324314 58.718848 \n",
       "L 159.819612 58.718848 \n",
       "L 170.31491 97.289135 \n",
       "L 180.810208 32.201761 \n",
       "L 191.305505 49.076256 \n",
       "L 201.800803 46.665649 \n",
       "L 212.296101 92.467839 \n",
       "L 222.791399 56.30816 \n",
       "L 233.286697 49.076256 \n",
       "L 243.781995 53.897552 \n",
       "L 254.277292 49.076256 \n",
       "L 264.77259 41.844353 \n",
       "L 275.267888 58.718848 \n",
       "L 285.763186 32.201761 \n",
       "L 296.258484 53.897552 \n",
       "L 306.753781 46.665649 \n",
       "L 317.249079 80.41464 \n",
       "L 327.744377 46.665649 \n",
       "L 338.239675 32.201761 \n",
       "L 348.734973 34.612368 \n",
       "L 359.23027 49.076256 \n",
       "L 369.725568 82.825247 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"m670be71acf\" style=\"stroke:#0000ff;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#pa4df6105f0)\">\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"65.361932\" xlink:href=\"#m670be71acf\" y=\"229.874489\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"75.85723\" xlink:href=\"#m670be71acf\" y=\"70.772048\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"86.352527\" xlink:href=\"#m670be71acf\" y=\"68.36144\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"96.847825\" xlink:href=\"#m670be71acf\" y=\"61.129456\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"107.343123\" xlink:href=\"#m670be71acf\" y=\"51.486945\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"117.838421\" xlink:href=\"#m670be71acf\" y=\"116.574237\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"128.333719\" xlink:href=\"#m670be71acf\" y=\"46.665649\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"138.829016\" xlink:href=\"#m670be71acf\" y=\"63.540144\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"149.324314\" xlink:href=\"#m670be71acf\" y=\"58.718848\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"159.819612\" xlink:href=\"#m670be71acf\" y=\"58.718848\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"170.31491\" xlink:href=\"#m670be71acf\" y=\"97.289135\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"180.810208\" xlink:href=\"#m670be71acf\" y=\"32.201761\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"191.305505\" xlink:href=\"#m670be71acf\" y=\"49.076256\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"201.800803\" xlink:href=\"#m670be71acf\" y=\"46.665649\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"212.296101\" xlink:href=\"#m670be71acf\" y=\"92.467839\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"222.791399\" xlink:href=\"#m670be71acf\" y=\"56.30816\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"233.286697\" xlink:href=\"#m670be71acf\" y=\"49.076256\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"243.781995\" xlink:href=\"#m670be71acf\" y=\"53.897552\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"254.277292\" xlink:href=\"#m670be71acf\" y=\"49.076256\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"264.77259\" xlink:href=\"#m670be71acf\" y=\"41.844353\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"275.267888\" xlink:href=\"#m670be71acf\" y=\"58.718848\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"285.763186\" xlink:href=\"#m670be71acf\" y=\"32.201761\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"296.258484\" xlink:href=\"#m670be71acf\" y=\"53.897552\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"306.753781\" xlink:href=\"#m670be71acf\" y=\"46.665649\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"317.249079\" xlink:href=\"#m670be71acf\" y=\"80.41464\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"327.744377\" xlink:href=\"#m670be71acf\" y=\"46.665649\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"338.239675\" xlink:href=\"#m670be71acf\" y=\"32.201761\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"348.734973\" xlink:href=\"#m670be71acf\" y=\"34.612368\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"359.23027\" xlink:href=\"#m670be71acf\" y=\"49.076256\"/>\n",
       "     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"369.725568\" xlink:href=\"#m670be71acf\" y=\"82.825247\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_17\">\n",
       "    <path clip-path=\"url(#pa4df6105f0)\" d=\"M 65.361932 140.360393 \n",
       "L 75.85723 82.808405 \n",
       "L 86.352527 73.21642 \n",
       "L 96.847825 101.992374 \n",
       "L 107.343123 178.728412 \n",
       "L 117.838421 149.952377 \n",
       "L 128.333719 178.728412 \n",
       "L 138.829016 73.21642 \n",
       "L 149.324314 92.400389 \n",
       "L 159.819612 82.808405 \n",
       "L 170.31491 73.21642 \n",
       "L 180.810208 63.624436 \n",
       "L 191.305505 121.176424 \n",
       "L 201.800803 188.320396 \n",
       "L 212.296101 63.624436 \n",
       "L 222.791399 82.808405 \n",
       "L 233.286697 82.808405 \n",
       "L 243.781995 130.768408 \n",
       "L 254.277292 54.03237 \n",
       "L 264.77259 178.728412 \n",
       "L 275.267888 54.03237 \n",
       "L 285.763186 92.400389 \n",
       "L 296.258484 82.808405 \n",
       "L 306.753781 121.176424 \n",
       "L 317.249079 63.624436 \n",
       "L 327.744377 73.21642 \n",
       "L 338.239675 73.21642 \n",
       "L 348.734973 111.58444 \n",
       "L 359.23027 178.728412 \n",
       "L 369.725568 73.21642 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"mbb40659787\" style=\"stroke:#ff0000;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#pa4df6105f0)\">\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"65.361932\" xlink:href=\"#mbb40659787\" y=\"140.360393\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"75.85723\" xlink:href=\"#mbb40659787\" y=\"82.808405\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"86.352527\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"96.847825\" xlink:href=\"#mbb40659787\" y=\"101.992374\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"107.343123\" xlink:href=\"#mbb40659787\" y=\"178.728412\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"117.838421\" xlink:href=\"#mbb40659787\" y=\"149.952377\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"128.333719\" xlink:href=\"#mbb40659787\" y=\"178.728412\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"138.829016\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"149.324314\" xlink:href=\"#mbb40659787\" y=\"92.400389\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"159.819612\" xlink:href=\"#mbb40659787\" y=\"82.808405\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"170.31491\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"180.810208\" xlink:href=\"#mbb40659787\" y=\"63.624436\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"191.305505\" xlink:href=\"#mbb40659787\" y=\"121.176424\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"201.800803\" xlink:href=\"#mbb40659787\" y=\"188.320396\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"212.296101\" xlink:href=\"#mbb40659787\" y=\"63.624436\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"222.791399\" xlink:href=\"#mbb40659787\" y=\"82.808405\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"233.286697\" xlink:href=\"#mbb40659787\" y=\"82.808405\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"243.781995\" xlink:href=\"#mbb40659787\" y=\"130.768408\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"254.277292\" xlink:href=\"#mbb40659787\" y=\"54.03237\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"264.77259\" xlink:href=\"#mbb40659787\" y=\"178.728412\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"275.267888\" xlink:href=\"#mbb40659787\" y=\"54.03237\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"285.763186\" xlink:href=\"#mbb40659787\" y=\"92.400389\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"296.258484\" xlink:href=\"#mbb40659787\" y=\"82.808405\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"306.753781\" xlink:href=\"#mbb40659787\" y=\"121.176424\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"317.249079\" xlink:href=\"#mbb40659787\" y=\"63.624436\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"327.744377\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"338.239675\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"348.734973\" xlink:href=\"#mbb40659787\" y=\"111.58444\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"359.23027\" xlink:href=\"#mbb40659787\" y=\"178.728412\"/>\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"369.725568\" xlink:href=\"#mbb40659787\" y=\"73.21642\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 50.14375 239.758125 \n",
       "L 50.14375 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 384.94375 239.758125 \n",
       "L 384.94375 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 50.14375 239.758125 \n",
       "L 384.94375 239.758125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 50.14375 22.318125 \n",
       "L 384.94375 22.318125 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"text_18\">\n",
       "    <!-- Training and validation accuracy -->\n",
       "    <defs>\n",
       "     <path d=\"M -0.296875 72.90625 \n",
       "L 61.375 72.90625 \n",
       "L 61.375 64.59375 \n",
       "L 35.5 64.59375 \n",
       "L 35.5 0 \n",
       "L 25.59375 0 \n",
       "L 25.59375 64.59375 \n",
       "L -0.296875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-84\"/>\n",
       "     <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-105\"/>\n",
       "     <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-110\"/>\n",
       "     <path d=\"M 45.40625 27.984375 \n",
       "Q 45.40625 37.75 41.375 43.109375 \n",
       "Q 37.359375 48.484375 30.078125 48.484375 \n",
       "Q 22.859375 48.484375 18.828125 43.109375 \n",
       "Q 14.796875 37.75 14.796875 27.984375 \n",
       "Q 14.796875 18.265625 18.828125 12.890625 \n",
       "Q 22.859375 7.515625 30.078125 7.515625 \n",
       "Q 37.359375 7.515625 41.375 12.890625 \n",
       "Q 45.40625 18.265625 45.40625 27.984375 \n",
       "z\n",
       "M 54.390625 6.78125 \n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \n",
       "Q 42 -20.796875 29.203125 -20.796875 \n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \n",
       "L 12.109375 -9.1875 \n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \n",
       "Q 23.78125 -13.375 27.78125 -13.375 \n",
       "Q 36.625 -13.375 41.015625 -8.765625 \n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \n",
       "L 45.40625 9.625 \n",
       "Q 42.625 4.78125 38.28125 2.390625 \n",
       "Q 33.9375 0 27.875 0 \n",
       "Q 17.828125 0 11.671875 7.65625 \n",
       "Q 5.515625 15.328125 5.515625 27.984375 \n",
       "Q 5.515625 40.671875 11.671875 48.328125 \n",
       "Q 17.828125 56 27.875 56 \n",
       "Q 33.9375 56 38.28125 53.609375 \n",
       "Q 42.625 51.21875 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "z\n",
       "\" id=\"DejaVuSans-103\"/>\n",
       "     <path id=\"DejaVuSans-32\"/>\n",
       "     <path d=\"M 45.40625 46.390625 \n",
       "L 45.40625 75.984375 \n",
       "L 54.390625 75.984375 \n",
       "L 54.390625 0 \n",
       "L 45.40625 0 \n",
       "L 45.40625 8.203125 \n",
       "Q 42.578125 3.328125 38.25 0.953125 \n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \n",
       "Q 5.515625 14.40625 5.515625 27.296875 \n",
       "Q 5.515625 40.1875 11.734375 48.09375 \n",
       "Q 17.96875 56 27.875 56 \n",
       "Q 33.9375 56 38.25 53.625 \n",
       "Q 42.578125 51.265625 45.40625 46.390625 \n",
       "z\n",
       "M 14.796875 27.296875 \n",
       "Q 14.796875 17.390625 18.875 11.75 \n",
       "Q 22.953125 6.109375 30.078125 6.109375 \n",
       "Q 37.203125 6.109375 41.296875 11.75 \n",
       "Q 45.40625 17.390625 45.40625 27.296875 \n",
       "Q 45.40625 37.203125 41.296875 42.84375 \n",
       "Q 37.203125 48.484375 30.078125 48.484375 \n",
       "Q 22.953125 48.484375 18.875 42.84375 \n",
       "Q 14.796875 37.203125 14.796875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-100\"/>\n",
       "     <path d=\"M 2.984375 54.6875 \n",
       "L 12.5 54.6875 \n",
       "L 29.59375 8.796875 \n",
       "L 46.6875 54.6875 \n",
       "L 56.203125 54.6875 \n",
       "L 35.6875 0 \n",
       "L 23.484375 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-118\"/>\n",
       "     <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-108\"/>\n",
       "     <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-116\"/>\n",
       "    </defs>\n",
       "    <g transform=\"translate(120.251875 16.318125)scale(0.12 -0.12)\">\n",
       "     <use xlink:href=\"#DejaVuSans-84\"/>\n",
       "     <use x=\"46.333984\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "     <use x=\"87.447266\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"148.726562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"176.509766\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"239.888672\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"267.671875\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"331.050781\" xlink:href=\"#DejaVuSans-103\"/>\n",
       "     <use x=\"394.527344\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"426.314453\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"487.59375\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"550.972656\" xlink:href=\"#DejaVuSans-100\"/>\n",
       "     <use x=\"614.449219\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"646.236328\" xlink:href=\"#DejaVuSans-118\"/>\n",
       "     <use x=\"705.416016\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"766.695312\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "     <use x=\"794.478516\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"822.261719\" xlink:href=\"#DejaVuSans-100\"/>\n",
       "     <use x=\"885.738281\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"947.017578\" xlink:href=\"#DejaVuSans-116\"/>\n",
       "     <use x=\"986.226562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "     <use x=\"1014.009766\" xlink:href=\"#DejaVuSans-111\"/>\n",
       "     <use x=\"1075.191406\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "     <use x=\"1138.570312\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "     <use x=\"1170.357422\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"1231.636719\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "     <use x=\"1286.617188\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "     <use x=\"1341.597656\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "     <use x=\"1404.976562\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "     <use x=\"1446.089844\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "     <use x=\"1507.369141\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "     <use x=\"1562.349609\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 272.55 234.758125 \n",
       "L 377.94375 234.758125 \n",
       "Q 379.94375 234.758125 379.94375 232.758125 \n",
       "L 379.94375 203.845625 \n",
       "Q 379.94375 201.845625 377.94375 201.845625 \n",
       "L 272.55 201.845625 \n",
       "Q 270.55 201.845625 270.55 203.845625 \n",
       "L 270.55 232.758125 \n",
       "Q 270.55 234.758125 272.55 234.758125 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_18\">\n",
       "     <path d=\"M 274.55 209.944062 \n",
       "L 294.55 209.944062 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_19\">\n",
       "     <g>\n",
       "      <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"284.55\" xlink:href=\"#m670be71acf\" y=\"209.944062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_19\">\n",
       "     <!-- train_accuracy -->\n",
       "     <defs>\n",
       "      <path d=\"M 50.984375 -16.609375 \n",
       "L 50.984375 -23.578125 \n",
       "L -0.984375 -23.578125 \n",
       "L -0.984375 -16.609375 \n",
       "z\n",
       "\" id=\"DejaVuSans-95\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(302.55 213.444062)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-105\"/>\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-110\"/>\n",
       "      <use x=\"232.763672\" xlink:href=\"#DejaVuSans-95\"/>\n",
       "      <use x=\"282.763672\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"344.042969\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"399.023438\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"454.003906\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"517.382812\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"558.496094\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"619.775391\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"674.755859\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_20\">\n",
       "     <path d=\"M 274.55 224.900312 \n",
       "L 294.55 224.900312 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_21\">\n",
       "     <g>\n",
       "      <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"284.55\" xlink:href=\"#mbb40659787\" y=\"224.900312\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_20\">\n",
       "     <!-- val_accuracy -->\n",
       "     <g transform=\"translate(302.55 228.400312)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-118\"/>\n",
       "      <use x=\"59.179688\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"120.458984\" xlink:href=\"#DejaVuSans-108\"/>\n",
       "      <use x=\"148.242188\" xlink:href=\"#DejaVuSans-95\"/>\n",
       "      <use x=\"198.242188\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"259.521484\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"314.501953\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"369.482422\" xlink:href=\"#DejaVuSans-117\"/>\n",
       "      <use x=\"432.861328\" xlink:href=\"#DejaVuSans-114\"/>\n",
       "      <use x=\"473.974609\" xlink:href=\"#DejaVuSans-97\"/>\n",
       "      <use x=\"535.253906\" xlink:href=\"#DejaVuSans-99\"/>\n",
       "      <use x=\"590.234375\" xlink:href=\"#DejaVuSans-121\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pa4df6105f0\">\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"50.14375\" y=\"22.318125\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metric(history,\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 49us/sample - loss: 0.6453 - accuracy: 0.6536\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.6185047003809966, 0.65363127]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Run against test dataset\n",
    "\n",
    "x_test = np.asarray(x_test)\n",
    "y_test = np.asarray(y_test)\n",
    "model.evaluate(x = x_test,y = y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Hyperparameter tuning\n",
    "\n",
    "Let's use tensorboard hparams for hyperparameter tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the TensorBoard notebook extension\n",
    "%load_ext tensorboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clear any logs from previous runs\n",
    "!rm -rf ./logs/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the TensorBoard HParams plugin:\n",
    "from tensorboard.plugins.hparams import api as hp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "HP_NUM_UNITS_ONE = hp.HParam('num_units_one', hp.Discrete([5, 10, 20]))\n",
    "HP_NUM_UNITS_TWO = hp.HParam('num_units_two', hp.Discrete([10, 20, 40]))\n",
    "HP_OPTIMIZER = hp.HParam('optimizer', hp.Discrete(['adam', 'sgd']))\n",
    "\n",
    "METRIC_ACCURACY = 'accuracy'\n",
    "\n",
    "with tf.summary.create_file_writer('logs/hparam_tuning').as_default():\n",
    "  hp.hparams_config(\n",
    "    hparams=[HP_NUM_UNITS_ONE, HP_NUM_UNITS_TWO, HP_OPTIMIZER],\n",
    "    metrics=[hp.Metric(METRIC_ACCURACY, display_name='Accuracy')],\n",
    "  )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_test_model(hparams):\n",
    "  model = tf.keras.models.Sequential()\n",
    "  model.add(layers.Dense(hparams[HP_NUM_UNITS_ONE], activation = 'relu',input_shape=(13,)))\n",
    "  model.add(layers.Dense(hparams[HP_NUM_UNITS_TWO],activation = 'relu' ))\n",
    "  model.add(layers.Dense(1,activation = 'sigmoid' ))\n",
    "  model.compile(\n",
    "      optimizer=hparams[HP_OPTIMIZER],\n",
    "      loss='binary_crossentropy',\n",
    "      metrics=['accuracy'],\n",
    "  )\n",
    "\n",
    "  model.fit(x_train, y_train, epochs=30) \n",
    "  _, accuracy = model.evaluate(x_test, y_test)\n",
    "  return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run(run_dir, hparams):\n",
    "  with tf.summary.create_file_writer(run_dir).as_default():\n",
    "    hp.hparams(hparams)  # record the values used in this trial\n",
    "    accuracy = train_test_model(hparams)\n",
    "    tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">> Starting trial: run-0\n",
      "{'num_units_one': 5, 'num_units_two': 10, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 973us/sample - loss: 1.1533 - accuracy: 0.3904\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.6801 - accuracy: 0.5618\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6507 - accuracy: 0.6180\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6226 - accuracy: 0.6601\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6027 - accuracy: 0.6882\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.5814 - accuracy: 0.7149\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.5674 - accuracy: 0.7247\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.5588 - accuracy: 0.7317\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.5525 - accuracy: 0.7233\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.5448 - accuracy: 0.7402\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5393 - accuracy: 0.7472\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.5379 - accuracy: 0.7317\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.5270 - accuracy: 0.7542\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5217 - accuracy: 0.7584\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5149 - accuracy: 0.7542\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 63us/sample - loss: 0.5087 - accuracy: 0.7837\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.5041 - accuracy: 0.7725\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5018 - accuracy: 0.7767\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.4946 - accuracy: 0.7823\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.4987 - accuracy: 0.7851\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4901 - accuracy: 0.7907\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.4790 - accuracy: 0.7963\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.4781 - accuracy: 0.7978\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.4819 - accuracy: 0.7921\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4706 - accuracy: 0.8020\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.4662 - accuracy: 0.7921\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.4657 - accuracy: 0.8020\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 64us/sample - loss: 0.4619 - accuracy: 0.7949\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4624 - accuracy: 0.8048\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 65us/sample - loss: 0.4578 - accuracy: 0.7978\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 831us/sample - loss: 0.4132 - accuracy: 0.8101\n",
      ">> Starting trial: run-1\n",
      "{'num_units_one': 5, 'num_units_two': 10, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 789us/sample - loss: 0.6616 - accuracy: 0.6643\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6519 - accuracy: 0.6699\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6424 - accuracy: 0.6643\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6350 - accuracy: 0.6854\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 65us/sample - loss: 0.6362 - accuracy: 0.6854\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6236 - accuracy: 0.6742\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6272 - accuracy: 0.6671\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 65us/sample - loss: 0.6205 - accuracy: 0.6826\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 63us/sample - loss: 0.6254 - accuracy: 0.6742\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6153 - accuracy: 0.6671\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6177 - accuracy: 0.6728\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6199 - accuracy: 0.6671\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6179 - accuracy: 0.6798\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6128 - accuracy: 0.6671\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6071 - accuracy: 0.6643\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6118 - accuracy: 0.6826\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6063 - accuracy: 0.6938\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6044 - accuracy: 0.6784\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5970 - accuracy: 0.6840\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6105 - accuracy: 0.6713\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6058 - accuracy: 0.6756\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6102 - accuracy: 0.6699\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6046 - accuracy: 0.6798\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 63us/sample - loss: 0.5981 - accuracy: 0.6854\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6006 - accuracy: 0.6671\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6097 - accuracy: 0.6770\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5940 - accuracy: 0.6854\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5964 - accuracy: 0.6868\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6088 - accuracy: 0.6798\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6022 - accuracy: 0.6713\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 819us/sample - loss: 0.6137 - accuracy: 0.6760\n",
      ">> Starting trial: run-2\n",
      "{'num_units_one': 5, 'num_units_two': 20, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 954us/sample - loss: 1.3237 - accuracy: 0.3624\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.7326 - accuracy: 0.5716\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6137 - accuracy: 0.6868\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6101 - accuracy: 0.6938\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6061 - accuracy: 0.6910\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6036 - accuracy: 0.6966\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6005 - accuracy: 0.6994\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5984 - accuracy: 0.7022\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5963 - accuracy: 0.6924\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5939 - accuracy: 0.6952\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5925 - accuracy: 0.6994\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5884 - accuracy: 0.6980\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5866 - accuracy: 0.6994\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5879 - accuracy: 0.7079\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5818 - accuracy: 0.7037\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5805 - accuracy: 0.7022\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5814 - accuracy: 0.7065\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5791 - accuracy: 0.7065\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.5786 - accuracy: 0.7051\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5698 - accuracy: 0.7121\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5680 - accuracy: 0.7093\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5641 - accuracy: 0.7163\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5618 - accuracy: 0.7107\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5602 - accuracy: 0.7191\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5610 - accuracy: 0.7107\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5531 - accuracy: 0.7163\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5465 - accuracy: 0.7289\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.5429 - accuracy: 0.7205\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 63us/sample - loss: 0.5387 - accuracy: 0.7303\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5334 - accuracy: 0.7247\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 841us/sample - loss: 0.5559 - accuracy: 0.6983\n",
      ">> Starting trial: run-3\n",
      "{'num_units_one': 5, 'num_units_two': 20, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 947us/sample - loss: 1.2226 - accuracy: 0.6334\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.7198 - accuracy: 0.6798\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.6630 - accuracy: 0.6896\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.6593 - accuracy: 0.6826\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 66us/sample - loss: 0.6223 - accuracy: 0.6770\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.6633 - accuracy: 0.6896\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.6197 - accuracy: 0.6896\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6381 - accuracy: 0.6728\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6285 - accuracy: 0.6742\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6097 - accuracy: 0.7051\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 66us/sample - loss: 0.6037 - accuracy: 0.7107\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.6144 - accuracy: 0.6980\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5937 - accuracy: 0.6980\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6144 - accuracy: 0.6938\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5952 - accuracy: 0.7177\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6066 - accuracy: 0.6952\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5969 - accuracy: 0.7037\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5883 - accuracy: 0.7008\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 63us/sample - loss: 0.5920 - accuracy: 0.7022\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 66us/sample - loss: 0.5967 - accuracy: 0.6952\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.5910 - accuracy: 0.7022\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5835 - accuracy: 0.7051\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5925 - accuracy: 0.7135\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5893 - accuracy: 0.7079\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6072 - accuracy: 0.6938\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5801 - accuracy: 0.7163\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5817 - accuracy: 0.7093\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5821 - accuracy: 0.7079\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5959 - accuracy: 0.6966\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5888 - accuracy: 0.6980\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 1ms/sample - loss: 0.5986 - accuracy: 0.6704\n",
      ">> Starting trial: run-4\n",
      "{'num_units_one': 5, 'num_units_two': 40, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 942us/sample - loss: 2.4439 - accuracy: 0.3104\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 1.5168 - accuracy: 0.5028\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.9719 - accuracy: 0.6110\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6447 - accuracy: 0.6713\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6080 - accuracy: 0.6854\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6050 - accuracy: 0.6938\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6052 - accuracy: 0.6952\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.6052 - accuracy: 0.6952\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6050 - accuracy: 0.6966\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6039 - accuracy: 0.6966\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6016 - accuracy: 0.6966\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6026 - accuracy: 0.6980\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6013 - accuracy: 0.6994\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6007 - accuracy: 0.6994\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6004 - accuracy: 0.6994\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6010 - accuracy: 0.6966\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6007 - accuracy: 0.6994\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5987 - accuracy: 0.6994\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5982 - accuracy: 0.7008\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5967 - accuracy: 0.7008\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5957 - accuracy: 0.7022\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5977 - accuracy: 0.7008\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5962 - accuracy: 0.7022\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5957 - accuracy: 0.7022\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5939 - accuracy: 0.7022\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5941 - accuracy: 0.7037\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5967 - accuracy: 0.7022\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.5959 - accuracy: 0.7022\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.5921 - accuracy: 0.7037\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.5970 - accuracy: 0.7037\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 875us/sample - loss: 0.6339 - accuracy: 0.6648\n",
      ">> Starting trial: run-5\n",
      "{'num_units_one': 5, 'num_units_two': 40, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 786us/sample - loss: 0.7819 - accuracy: 0.5407\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6877 - accuracy: 0.6531\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6755 - accuracy: 0.6742\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6679 - accuracy: 0.6699\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6593 - accuracy: 0.6699\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6317 - accuracy: 0.6728\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.6410 - accuracy: 0.6699\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6318 - accuracy: 0.6699\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6421 - accuracy: 0.6713\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6363 - accuracy: 0.6615\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6312 - accuracy: 0.6854\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6292 - accuracy: 0.6798\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6655 - accuracy: 0.6671\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6352 - accuracy: 0.6784\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6233 - accuracy: 0.6784\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6289 - accuracy: 0.6784\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6208 - accuracy: 0.6812\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6249 - accuracy: 0.6784\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.6201 - accuracy: 0.6728\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6173 - accuracy: 0.6812\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6222 - accuracy: 0.6798\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6287 - accuracy: 0.6770\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.6199 - accuracy: 0.6728\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6204 - accuracy: 0.6756\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6227 - accuracy: 0.6671\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6262 - accuracy: 0.6798\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6157 - accuracy: 0.6784\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6117 - accuracy: 0.6840\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6133 - accuracy: 0.6826\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6202 - accuracy: 0.6812\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 828us/sample - loss: 0.6536 - accuracy: 0.6592\n",
      ">> Starting trial: run-6\n",
      "{'num_units_one': 10, 'num_units_two': 10, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 968us/sample - loss: 1.3478 - accuracy: 0.5843\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.9577 - accuracy: 0.6447\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.7450 - accuracy: 0.6390\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6392 - accuracy: 0.6713\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6146 - accuracy: 0.6966\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5991 - accuracy: 0.6854\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5833 - accuracy: 0.7289\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5698 - accuracy: 0.7360\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5582 - accuracy: 0.7402\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5453 - accuracy: 0.7444\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5482 - accuracy: 0.7626\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5308 - accuracy: 0.7556\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5242 - accuracy: 0.7626\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5168 - accuracy: 0.7865\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5156 - accuracy: 0.7669\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5120 - accuracy: 0.7739\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4989 - accuracy: 0.7893\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4949 - accuracy: 0.7879\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4948 - accuracy: 0.7978\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4885 - accuracy: 0.7978\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4866 - accuracy: 0.8006\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.4817 - accuracy: 0.8034\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4796 - accuracy: 0.7992\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.4773 - accuracy: 0.8006\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4746 - accuracy: 0.8062\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4731 - accuracy: 0.8006\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4721 - accuracy: 0.8048\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4715 - accuracy: 0.8062\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.4697 - accuracy: 0.8006\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.4653 - accuracy: 0.8048\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 847us/sample - loss: 0.4218 - accuracy: 0.7821\n",
      ">> Starting trial: run-7\n",
      "{'num_units_one': 10, 'num_units_two': 10, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 1ms/sample - loss: 0.6911 - accuracy: 0.6039\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6124 - accuracy: 0.6517\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.6030 - accuracy: 0.6910\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6022 - accuracy: 0.6896\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6088 - accuracy: 0.6966\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5965 - accuracy: 0.6882\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6003 - accuracy: 0.6840\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5950 - accuracy: 0.6994\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5956 - accuracy: 0.6910\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5904 - accuracy: 0.7022\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5782 - accuracy: 0.7093\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5799 - accuracy: 0.7065\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5807 - accuracy: 0.7065\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5808 - accuracy: 0.7107\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5902 - accuracy: 0.7022\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6015 - accuracy: 0.6994\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5794 - accuracy: 0.7121\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5846 - accuracy: 0.7093\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5657 - accuracy: 0.7121\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5798 - accuracy: 0.7037\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5864 - accuracy: 0.7247\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5703 - accuracy: 0.7107\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5742 - accuracy: 0.7121\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5749 - accuracy: 0.7037\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5750 - accuracy: 0.7093\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5955 - accuracy: 0.7022\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5690 - accuracy: 0.7135\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5801 - accuracy: 0.7037\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5768 - accuracy: 0.7079\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5788 - accuracy: 0.7022\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 839us/sample - loss: 0.5371 - accuracy: 0.7151\n",
      ">> Starting trial: run-8\n",
      "{'num_units_one': 10, 'num_units_two': 20, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 953us/sample - loss: 3.6371 - accuracy: 0.6067\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 1.0606 - accuracy: 0.6039\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.8849 - accuracy: 0.6390\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 71us/sample - loss: 0.7767 - accuracy: 0.6475\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.7189 - accuracy: 0.6531\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.6711 - accuracy: 0.6685\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6379 - accuracy: 0.6713\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6181 - accuracy: 0.6756\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6084 - accuracy: 0.6728\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5965 - accuracy: 0.6854\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5864 - accuracy: 0.6952\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5795 - accuracy: 0.6938\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5736 - accuracy: 0.7065\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5670 - accuracy: 0.7093\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5719 - accuracy: 0.7065\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5578 - accuracy: 0.7275\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5552 - accuracy: 0.7317\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5574 - accuracy: 0.7191\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5443 - accuracy: 0.7317\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5363 - accuracy: 0.7331\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5310 - accuracy: 0.7430\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5264 - accuracy: 0.7444\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5240 - accuracy: 0.7514\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5230 - accuracy: 0.7514\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5168 - accuracy: 0.7500\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5292 - accuracy: 0.7444\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5044 - accuracy: 0.7626\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.4996 - accuracy: 0.7598\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5300 - accuracy: 0.7486\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5027 - accuracy: 0.7669\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 857us/sample - loss: 0.4732 - accuracy: 0.7598\n",
      ">> Starting trial: run-9\n",
      "{'num_units_one': 10, 'num_units_two': 20, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 786us/sample - loss: 1.1424 - accuracy: 0.6278\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6878 - accuracy: 0.6489\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6387 - accuracy: 0.6657\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6621 - accuracy: 0.6559\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6087 - accuracy: 0.6910\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6048 - accuracy: 0.6938\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6020 - accuracy: 0.6980\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6005 - accuracy: 0.7037\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 45us/sample - loss: 0.6349 - accuracy: 0.6938\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6022 - accuracy: 0.6854\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5981 - accuracy: 0.6994\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5924 - accuracy: 0.7008\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5958 - accuracy: 0.7008\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5847 - accuracy: 0.7008\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5852 - accuracy: 0.7051\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5884 - accuracy: 0.7037\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5831 - accuracy: 0.7093\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.5791 - accuracy: 0.7079\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5927 - accuracy: 0.7149\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5702 - accuracy: 0.7121\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5685 - accuracy: 0.7135\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5693 - accuracy: 0.7135\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5667 - accuracy: 0.7205\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5786 - accuracy: 0.7093\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5707 - accuracy: 0.7093\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5640 - accuracy: 0.7135\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5662 - accuracy: 0.7205\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5677 - accuracy: 0.7163\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5762 - accuracy: 0.7135\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5552 - accuracy: 0.7374\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 828us/sample - loss: 0.6055 - accuracy: 0.6927\n",
      ">> Starting trial: run-10\n",
      "{'num_units_one': 10, 'num_units_two': 40, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 949us/sample - loss: 0.6791 - accuracy: 0.6489\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.6568 - accuracy: 0.6671\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6370 - accuracy: 0.6601\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6384 - accuracy: 0.6770\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6285 - accuracy: 0.6770\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6213 - accuracy: 0.6840\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.6159 - accuracy: 0.6784\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6160 - accuracy: 0.6854\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6110 - accuracy: 0.6868\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6056 - accuracy: 0.6910\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6171 - accuracy: 0.6896\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6035 - accuracy: 0.6896\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.6004 - accuracy: 0.6910\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5971 - accuracy: 0.6952\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5909 - accuracy: 0.6952\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5904 - accuracy: 0.6868\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5925 - accuracy: 0.6924\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 64us/sample - loss: 0.5843 - accuracy: 0.6994\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5804 - accuracy: 0.6938\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5763 - accuracy: 0.6910\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5692 - accuracy: 0.6910\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5657 - accuracy: 0.7022\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5585 - accuracy: 0.6966\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5599 - accuracy: 0.7079\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5506 - accuracy: 0.7135\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5429 - accuracy: 0.7205\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 67us/sample - loss: 0.5393 - accuracy: 0.7205\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 73us/sample - loss: 0.5464 - accuracy: 0.7416\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 70us/sample - loss: 0.5288 - accuracy: 0.7514\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5193 - accuracy: 0.7626\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 845us/sample - loss: 0.4896 - accuracy: 0.7207\n",
      ">> Starting trial: run-11\n",
      "{'num_units_one': 10, 'num_units_two': 40, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 790us/sample - loss: 1.4981 - accuracy: 0.5646\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.8595 - accuracy: 0.6250\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.7443 - accuracy: 0.6517\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6931 - accuracy: 0.6699\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6226 - accuracy: 0.6713\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6411 - accuracy: 0.6643\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6014 - accuracy: 0.7008\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5918 - accuracy: 0.6924\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6047 - accuracy: 0.6924\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5951 - accuracy: 0.7121\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6088 - accuracy: 0.7051\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5846 - accuracy: 0.7107\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6094 - accuracy: 0.7037\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5718 - accuracy: 0.7163\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5782 - accuracy: 0.7051\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5692 - accuracy: 0.7275\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5912 - accuracy: 0.7107\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5774 - accuracy: 0.7107\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5654 - accuracy: 0.7163\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5634 - accuracy: 0.7233\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5828 - accuracy: 0.7135\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5607 - accuracy: 0.7177\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6050 - accuracy: 0.7051\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5747 - accuracy: 0.7205\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5669 - accuracy: 0.7233\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5643 - accuracy: 0.7303\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5545 - accuracy: 0.7275\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5576 - accuracy: 0.7233\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5810 - accuracy: 0.7065\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 64us/sample - loss: 0.5612 - accuracy: 0.7275\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 875us/sample - loss: 0.5548 - accuracy: 0.7151\n",
      ">> Starting trial: run-12\n",
      "{'num_units_one': 20, 'num_units_two': 10, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 972us/sample - loss: 1.3074 - accuracy: 0.4705\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6292 - accuracy: 0.6742\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6025 - accuracy: 0.7022\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5834 - accuracy: 0.6952\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5850 - accuracy: 0.7051\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6126 - accuracy: 0.6742\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5678 - accuracy: 0.7219\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5556 - accuracy: 0.7331\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5555 - accuracy: 0.7289\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5434 - accuracy: 0.7317\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5379 - accuracy: 0.7374\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5392 - accuracy: 0.7317\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5239 - accuracy: 0.7542\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5252 - accuracy: 0.7514\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5171 - accuracy: 0.7640\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4923 - accuracy: 0.7837\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4920 - accuracy: 0.7978\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4821 - accuracy: 0.7851\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4756 - accuracy: 0.7893\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4821 - accuracy: 0.7963\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4631 - accuracy: 0.7907\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4539 - accuracy: 0.8258\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4641 - accuracy: 0.8188\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4512 - accuracy: 0.8202\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4484 - accuracy: 0.8132\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.4621 - accuracy: 0.8104\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4588 - accuracy: 0.8146\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4442 - accuracy: 0.8188\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.4382 - accuracy: 0.8202\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.4386 - accuracy: 0.8188\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 862us/sample - loss: 0.4026 - accuracy: 0.7989\n",
      ">> Starting trial: run-13\n",
      "{'num_units_one': 20, 'num_units_two': 10, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 815us/sample - loss: 1.1456 - accuracy: 0.6362\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.7807 - accuracy: 0.6938\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6502 - accuracy: 0.6713\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6313 - accuracy: 0.6826\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6810 - accuracy: 0.6728\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6377 - accuracy: 0.6924\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5999 - accuracy: 0.7008\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6087 - accuracy: 0.6854\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6008 - accuracy: 0.7135\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5936 - accuracy: 0.7008\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6000 - accuracy: 0.6868\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5825 - accuracy: 0.6980\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5919 - accuracy: 0.7065\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5685 - accuracy: 0.7233\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6107 - accuracy: 0.6966\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5885 - accuracy: 0.6910\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5730 - accuracy: 0.6966\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5698 - accuracy: 0.7093\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5865 - accuracy: 0.6952\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5730 - accuracy: 0.7051\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5712 - accuracy: 0.7037\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5833 - accuracy: 0.6994\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5602 - accuracy: 0.7149\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5588 - accuracy: 0.7261\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5628 - accuracy: 0.7051\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5762 - accuracy: 0.7177\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5563 - accuracy: 0.7247\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5639 - accuracy: 0.7219\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5496 - accuracy: 0.7247\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5986 - accuracy: 0.6742\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 847us/sample - loss: 0.5653 - accuracy: 0.6760\n",
      ">> Starting trial: run-14\n",
      "{'num_units_one': 20, 'num_units_two': 20, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 930us/sample - loss: 1.0741 - accuracy: 0.6250\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6419 - accuracy: 0.6980\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6006 - accuracy: 0.7037\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5843 - accuracy: 0.7191\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5822 - accuracy: 0.7233\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5719 - accuracy: 0.7261\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5574 - accuracy: 0.7303\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5559 - accuracy: 0.7331\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5566 - accuracy: 0.7402\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5301 - accuracy: 0.7458\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5331 - accuracy: 0.7430\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5352 - accuracy: 0.7683\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5137 - accuracy: 0.7725\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.5160 - accuracy: 0.7795\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4989 - accuracy: 0.7851\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4928 - accuracy: 0.7879\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4877 - accuracy: 0.7963\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4833 - accuracy: 0.7992\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5221 - accuracy: 0.7935\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.4715 - accuracy: 0.7992\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.4658 - accuracy: 0.8076\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4840 - accuracy: 0.8006\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4655 - accuracy: 0.7978\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4691 - accuracy: 0.8076\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.4591 - accuracy: 0.8174\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4446 - accuracy: 0.8062\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.4484 - accuracy: 0.8076\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4417 - accuracy: 0.8160\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.4355 - accuracy: 0.8104\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4453 - accuracy: 0.8048\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 871us/sample - loss: 0.4328 - accuracy: 0.7989\n",
      ">> Starting trial: run-15\n",
      "{'num_units_one': 20, 'num_units_two': 20, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 797us/sample - loss: 1.0742 - accuracy: 0.5815\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.7213 - accuracy: 0.6545\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6705 - accuracy: 0.6840\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6299 - accuracy: 0.6798\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 46us/sample - loss: 0.6195 - accuracy: 0.6868\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6338 - accuracy: 0.6742\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6199 - accuracy: 0.6868\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6027 - accuracy: 0.6994\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6122 - accuracy: 0.6910\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6292 - accuracy: 0.6896\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6070 - accuracy: 0.6840\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6036 - accuracy: 0.6980\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6111 - accuracy: 0.6966\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5970 - accuracy: 0.6896\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6007 - accuracy: 0.7037\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6058 - accuracy: 0.6868\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5909 - accuracy: 0.7093\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5929 - accuracy: 0.7051\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 61us/sample - loss: 0.5897 - accuracy: 0.7065\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.5971 - accuracy: 0.6924\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 65us/sample - loss: 0.5918 - accuracy: 0.7022\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5939 - accuracy: 0.6910\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5813 - accuracy: 0.7093\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5796 - accuracy: 0.7093\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5907 - accuracy: 0.7037\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.5840 - accuracy: 0.7107\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5803 - accuracy: 0.7093\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5831 - accuracy: 0.7008\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5781 - accuracy: 0.7219\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.5751 - accuracy: 0.7191\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 850us/sample - loss: 0.6249 - accuracy: 0.6816\n",
      ">> Starting trial: run-16\n",
      "{'num_units_one': 20, 'num_units_two': 40, 'optimizer': 'adam'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 1ms/sample - loss: 0.9252 - accuracy: 0.5478\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6445 - accuracy: 0.6868\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.6012 - accuracy: 0.6938\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5897 - accuracy: 0.7177\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6002 - accuracy: 0.7177\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5948 - accuracy: 0.7022\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5900 - accuracy: 0.7247\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 54us/sample - loss: 0.6004 - accuracy: 0.7051\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5722 - accuracy: 0.7261\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5748 - accuracy: 0.7219\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.5505 - accuracy: 0.7416\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5590 - accuracy: 0.7289\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5652 - accuracy: 0.7374\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5281 - accuracy: 0.7458\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5268 - accuracy: 0.7444\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5254 - accuracy: 0.7556\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5251 - accuracy: 0.7753\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5217 - accuracy: 0.7795\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.5117 - accuracy: 0.7697\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.5067 - accuracy: 0.7907\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.4974 - accuracy: 0.7823\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.5086 - accuracy: 0.7879\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.4766 - accuracy: 0.8076\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 56us/sample - loss: 0.4755 - accuracy: 0.8048\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 57us/sample - loss: 0.4695 - accuracy: 0.8104\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.4717 - accuracy: 0.8020\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.4589 - accuracy: 0.8118\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 64us/sample - loss: 0.4515 - accuracy: 0.8132\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 65us/sample - loss: 0.4469 - accuracy: 0.8202\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 68us/sample - loss: 0.4646 - accuracy: 0.8048\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 950us/sample - loss: 0.4218 - accuracy: 0.7933\n",
      ">> Starting trial: run-17\n",
      "{'num_units_one': 20, 'num_units_two': 40, 'optimizer': 'sgd'}\n",
      "Train on 712 samples\n",
      "Epoch 1/30\n",
      "712/712 [==============================] - 1s 795us/sample - loss: 1.2117 - accuracy: 0.6067\n",
      "Epoch 2/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.7174 - accuracy: 0.6461\n",
      "Epoch 3/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.6249 - accuracy: 0.6826\n",
      "Epoch 4/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.6803 - accuracy: 0.6517\n",
      "Epoch 5/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6181 - accuracy: 0.6952\n",
      "Epoch 6/30\n",
      "712/712 [==============================] - 0s 47us/sample - loss: 0.6163 - accuracy: 0.6882\n",
      "Epoch 7/30\n",
      "712/712 [==============================] - 0s 48us/sample - loss: 0.6216 - accuracy: 0.6966\n",
      "Epoch 8/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6088 - accuracy: 0.6980\n",
      "Epoch 9/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.6035 - accuracy: 0.6854\n",
      "Epoch 10/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6099 - accuracy: 0.6784\n",
      "Epoch 11/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.6071 - accuracy: 0.7008\n",
      "Epoch 12/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5932 - accuracy: 0.7079\n",
      "Epoch 13/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5961 - accuracy: 0.7135\n",
      "Epoch 14/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5928 - accuracy: 0.6994\n",
      "Epoch 15/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5980 - accuracy: 0.7037\n",
      "Epoch 16/30\n",
      "712/712 [==============================] - 0s 49us/sample - loss: 0.5989 - accuracy: 0.7079\n",
      "Epoch 17/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5865 - accuracy: 0.7233\n",
      "Epoch 18/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5978 - accuracy: 0.7051\n",
      "Epoch 19/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5872 - accuracy: 0.7205\n",
      "Epoch 20/30\n",
      "712/712 [==============================] - 0s 52us/sample - loss: 0.5937 - accuracy: 0.6980\n",
      "Epoch 21/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5929 - accuracy: 0.7037\n",
      "Epoch 22/30\n",
      "712/712 [==============================] - 0s 58us/sample - loss: 0.5790 - accuracy: 0.7079\n",
      "Epoch 23/30\n",
      "712/712 [==============================] - 0s 60us/sample - loss: 0.5991 - accuracy: 0.7065\n",
      "Epoch 24/30\n",
      "712/712 [==============================] - 0s 59us/sample - loss: 0.5858 - accuracy: 0.7135\n",
      "Epoch 25/30\n",
      "712/712 [==============================] - 0s 62us/sample - loss: 0.5849 - accuracy: 0.7247\n",
      "Epoch 26/30\n",
      "712/712 [==============================] - 0s 55us/sample - loss: 0.5910 - accuracy: 0.6966\n",
      "Epoch 27/30\n",
      "712/712 [==============================] - 0s 53us/sample - loss: 0.5890 - accuracy: 0.7121\n",
      "Epoch 28/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5890 - accuracy: 0.7205\n",
      "Epoch 29/30\n",
      "712/712 [==============================] - 0s 51us/sample - loss: 0.5808 - accuracy: 0.7247\n",
      "Epoch 30/30\n",
      "712/712 [==============================] - 0s 50us/sample - loss: 0.5816 - accuracy: 0.7191\n",
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 930us/sample - loss: 0.7432 - accuracy: 0.6927\n"
     ]
    }
   ],
   "source": [
    "session_num = 0\n",
    "\n",
    "for num_units_one in HP_NUM_UNITS_ONE.domain.values:\n",
    "  for num_units_two in HP_NUM_UNITS_TWO.domain.values:\n",
    "    for optimizer in HP_OPTIMIZER.domain.values:\n",
    "      hparams = {\n",
    "          HP_NUM_UNITS_ONE: num_units_one,\n",
    "          HP_NUM_UNITS_TWO: num_units_two,\n",
    "          HP_OPTIMIZER: optimizer,\n",
    "      }\n",
    "      run_name = \"run-%d\" % session_num\n",
    "      print('>> Starting trial: %s' % run_name)\n",
    "      print({h.name: hparams[h] for h in hparams})\n",
    "      run('logs/hparam_tuning/' + run_name, hparams)\n",
    "      session_num += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the results in TensorBoard’s HParams plugin\n",
    "Open your terminal, and cd into your project, then run `tensorboard --logdir logs/hparam_tuning`\n",
    "\n",
    "Please refer to medium article for the details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.56895125],\n",
       "       [0.37735564],\n",
       "       [0.5005745 ],\n",
       "       [0.60003537],\n",
       "       [0.5371451 ],\n",
       "       [0.36402294],\n",
       "       [0.49169463],\n",
       "       [0.49049523],\n",
       "       [0.4984674 ],\n",
       "       [0.1470165 ]], dtype=float32)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(x_test[0:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [0],\n",
       "       [1],\n",
       "       [1],\n",
       "       [1],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0]], dtype=int32)"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_classes(x_test[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "179/1 [==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 907us/sample - loss: 0.5889 - accuracy: 0.6872\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.5728170005968829, 0.68715084]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.save('data/keras_model.h5')\n",
    "\n",
    "del model  # Deleting current model\n",
    "\n",
    "# Identical to the previous one\n",
    "model = models.load_model('data/keras_model.h5')\n",
    "model.evaluate(x_test,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## That's it\n",
    "\n",
    "Thanks for reading\n",
    "\n",
    "This is a notebook for the medium article [The Google's 7 step of Machine Learning in Practice: a TensorFlow example for structured data](https://towardsdatascience.com/the-googles-7-steps-of-machine-learning-in-practice-a-tensorflow-example-for-structured-data-96ccbb707d77)\n",
    "\n",
    "Please check out article for instructions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf-tutorial",
   "language": "python",
   "name": "tf-tutorial"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}