lectures/10_regression-metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](img/330-banner.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "# Lecture 10: Regression Evaluation Metrics\n",
    "\n",
    "UBC 2022-23\n",
    "\n",
    "Instructor: Varada Kolhatkar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.compose import (\n",
    "    ColumnTransformer,\n",
    "    TransformedTargetRegressor,\n",
    "    make_column_transformer,\n",
    ")\n",
    "from sklearn.dummy import DummyRegressor\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.linear_model import LinearRegression, Ridge, RidgeCV\n",
    "from sklearn.metrics import make_scorer, mean_squared_error, r2_score\n",
    "from sklearn.model_selection import cross_val_score, cross_validate, train_test_split\n",
    "from sklearn.pipeline import Pipeline, make_pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, StandardScaler\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# import warnings\n",
    "\n",
    "# warnings.simplefilter(action=\"ignore\", category=FutureWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Announcements\n",
    "\n",
    "- Updates on HW2 autograder issues  \n",
    "- Update on HW5 \n",
    "- Midterm coming up soon on Oct 27th during class time. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{note} \n",
    "When you calculate precision, recall, f1 score, by default only the positive label is evaluated, assuming by default that the positive class is labeled 1. This is configurable through the `pos_label` parameter. \n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## Learning outcomes \n",
    "\n",
    "From this lecture, students are expected to be able to:\n",
    "\n",
    "- Carry out feature transformations on somewhat complicated dataset. \n",
    "- Visualize transformed features as a dataframe. \n",
    "- Use `Ridge` and `RidgeCV`.\n",
    "- Explain how `alpha` hyperparameter of `Ridge` relates to the fundamental tradeoff. \n",
    "- Examine coefficients of transformed features.  \n",
    "- Appropriately select a scoring metric given a regression problem.\n",
    "- Interpret and communicate the meanings of different scoring metrics on regression problems.\n",
    "    - MSE, RMSE, $R^2$, MAPE\n",
    "- Apply log-transform on the target values in a regression problem with `TransformedTargetRegressor`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dataset\n",
    "\n",
    "In this lecture, we'll be using [Kaggle House Prices dataset](https://www.kaggle.com/c/home-data-for-ml-course/). As usual, to run this notebook you'll need to download the data. For this dataset, train and test have already been separated. We'll be working with the train portion in this lecture. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>303</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>118.0</td>\n",
       "      <td>13704</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>205000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767</th>\n",
       "      <td>768</td>\n",
       "      <td>50</td>\n",
       "      <td>RL</td>\n",
       "      <td>75.0</td>\n",
       "      <td>12508</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Shed</td>\n",
       "      <td>1300</td>\n",
       "      <td>7</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>160000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429</th>\n",
       "      <td>430</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>130.0</td>\n",
       "      <td>11457</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2009</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>175000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1139</th>\n",
       "      <td>1140</td>\n",
       "      <td>30</td>\n",
       "      <td>RL</td>\n",
       "      <td>98.0</td>\n",
       "      <td>8731</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>144000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>558</th>\n",
       "      <td>559</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>57.0</td>\n",
       "      <td>21872</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR2</td>\n",
       "      <td>HLS</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>175000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 81 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Id  MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "302    303          20       RL        118.0    13704   Pave   NaN      IR1   \n",
       "767    768          50       RL         75.0    12508   Pave   NaN      IR1   \n",
       "429    430          20       RL        130.0    11457   Pave   NaN      IR1   \n",
       "1139  1140          30       RL         98.0     8731   Pave   NaN      IR1   \n",
       "558    559          60       RL         57.0    21872   Pave   NaN      IR2   \n",
       "\n",
       "     LandContour Utilities  ... PoolArea PoolQC Fence MiscFeature MiscVal  \\\n",
       "302          Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "767          Lvl    AllPub  ...        0    NaN   NaN        Shed    1300   \n",
       "429          Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "1139         Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "558          HLS    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "\n",
       "     MoSold YrSold  SaleType  SaleCondition  SalePrice  \n",
       "302       1   2006        WD         Normal     205000  \n",
       "767       7   2008        WD         Normal     160000  \n",
       "429       3   2009        WD         Normal     175000  \n",
       "1139      5   2007        WD         Normal     144000  \n",
       "558       8   2008        WD         Normal     175000  \n",
       "\n",
       "[5 rows x 81 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"data/housing-kaggle/train.csv\")\n",
    "train_df, test_df = train_test_split(df, test_size=0.10, random_state=123)\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- The supervised machine learning problem is predicting housing price given features associated with properties. \n",
    "- Here, the target is `SalePrice`, which is continuous. So it's a **regression problem** (as opposed to classification)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 81)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "### Let's separate `X` and `y`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "X_train = train_df.drop(columns=[\"SalePrice\"])\n",
    "y_train = train_df[\"SalePrice\"]\n",
    "\n",
    "X_test = test_df.drop(columns=[\"SalePrice\"])\n",
    "y_test = test_df[\"SalePrice\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### EDA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>...</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1089.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1307.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>734.182648</td>\n",
       "      <td>56.472603</td>\n",
       "      <td>69.641873</td>\n",
       "      <td>10273.261035</td>\n",
       "      <td>6.076104</td>\n",
       "      <td>5.570015</td>\n",
       "      <td>1970.995434</td>\n",
       "      <td>1984.659056</td>\n",
       "      <td>102.514155</td>\n",
       "      <td>441.425419</td>\n",
       "      <td>...</td>\n",
       "      <td>94.281583</td>\n",
       "      <td>45.765601</td>\n",
       "      <td>21.726788</td>\n",
       "      <td>3.624049</td>\n",
       "      <td>13.987062</td>\n",
       "      <td>3.065449</td>\n",
       "      <td>46.951294</td>\n",
       "      <td>6.302131</td>\n",
       "      <td>2007.840183</td>\n",
       "      <td>179802.147641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>422.224662</td>\n",
       "      <td>42.036646</td>\n",
       "      <td>23.031794</td>\n",
       "      <td>8997.895541</td>\n",
       "      <td>1.392612</td>\n",
       "      <td>1.112848</td>\n",
       "      <td>30.198127</td>\n",
       "      <td>20.639754</td>\n",
       "      <td>178.301563</td>\n",
       "      <td>459.276687</td>\n",
       "      <td>...</td>\n",
       "      <td>125.436492</td>\n",
       "      <td>65.757545</td>\n",
       "      <td>60.766423</td>\n",
       "      <td>30.320430</td>\n",
       "      <td>53.854129</td>\n",
       "      <td>42.341109</td>\n",
       "      <td>522.283421</td>\n",
       "      <td>2.698206</td>\n",
       "      <td>1.332824</td>\n",
       "      <td>79041.260572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1300.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1872.000000</td>\n",
       "      <td>1950.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2006.000000</td>\n",
       "      <td>34900.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>369.250000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>7500.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1953.000000</td>\n",
       "      <td>1966.250000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>2007.000000</td>\n",
       "      <td>129600.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>735.500000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>69.000000</td>\n",
       "      <td>9391.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1972.000000</td>\n",
       "      <td>1993.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>376.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2008.000000</td>\n",
       "      <td>162000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1099.750000</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>11509.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2000.000000</td>\n",
       "      <td>2004.000000</td>\n",
       "      <td>165.500000</td>\n",
       "      <td>704.750000</td>\n",
       "      <td>...</td>\n",
       "      <td>168.000000</td>\n",
       "      <td>66.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>2009.000000</td>\n",
       "      <td>212975.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1460.000000</td>\n",
       "      <td>190.000000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>215245.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>1378.000000</td>\n",
       "      <td>5644.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>857.000000</td>\n",
       "      <td>547.000000</td>\n",
       "      <td>552.000000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>480.000000</td>\n",
       "      <td>738.000000</td>\n",
       "      <td>15500.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>755000.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                Id   MSSubClass  LotFrontage        LotArea  OverallQual  \\\n",
       "count  1314.000000  1314.000000  1089.000000    1314.000000  1314.000000   \n",
       "mean    734.182648    56.472603    69.641873   10273.261035     6.076104   \n",
       "std     422.224662    42.036646    23.031794    8997.895541     1.392612   \n",
       "min       1.000000    20.000000    21.000000    1300.000000     1.000000   \n",
       "25%     369.250000    20.000000    59.000000    7500.000000     5.000000   \n",
       "50%     735.500000    50.000000    69.000000    9391.000000     6.000000   \n",
       "75%    1099.750000    70.000000    80.000000   11509.000000     7.000000   \n",
       "max    1460.000000   190.000000   313.000000  215245.000000    10.000000   \n",
       "\n",
       "       OverallCond    YearBuilt  YearRemodAdd   MasVnrArea   BsmtFinSF1  ...  \\\n",
       "count  1314.000000  1314.000000   1314.000000  1307.000000  1314.000000  ...   \n",
       "mean      5.570015  1970.995434   1984.659056   102.514155   441.425419  ...   \n",
       "std       1.112848    30.198127     20.639754   178.301563   459.276687  ...   \n",
       "min       1.000000  1872.000000   1950.000000     0.000000     0.000000  ...   \n",
       "25%       5.000000  1953.000000   1966.250000     0.000000     0.000000  ...   \n",
       "50%       5.000000  1972.000000   1993.000000     0.000000   376.000000  ...   \n",
       "75%       6.000000  2000.000000   2004.000000   165.500000   704.750000  ...   \n",
       "max       9.000000  2010.000000   2010.000000  1378.000000  5644.000000  ...   \n",
       "\n",
       "        WoodDeckSF  OpenPorchSF  EnclosedPorch    3SsnPorch  ScreenPorch  \\\n",
       "count  1314.000000  1314.000000    1314.000000  1314.000000  1314.000000   \n",
       "mean     94.281583    45.765601      21.726788     3.624049    13.987062   \n",
       "std     125.436492    65.757545      60.766423    30.320430    53.854129   \n",
       "min       0.000000     0.000000       0.000000     0.000000     0.000000   \n",
       "25%       0.000000     0.000000       0.000000     0.000000     0.000000   \n",
       "50%       0.000000    24.000000       0.000000     0.000000     0.000000   \n",
       "75%     168.000000    66.750000       0.000000     0.000000     0.000000   \n",
       "max     857.000000   547.000000     552.000000   508.000000   480.000000   \n",
       "\n",
       "          PoolArea       MiscVal       MoSold       YrSold      SalePrice  \n",
       "count  1314.000000   1314.000000  1314.000000  1314.000000    1314.000000  \n",
       "mean      3.065449     46.951294     6.302131  2007.840183  179802.147641  \n",
       "std      42.341109    522.283421     2.698206     1.332824   79041.260572  \n",
       "min       0.000000      0.000000     1.000000  2006.000000   34900.000000  \n",
       "25%       0.000000      0.000000     5.000000  2007.000000  129600.000000  \n",
       "50%       0.000000      0.000000     6.000000  2008.000000  162000.000000  \n",
       "75%       0.000000      0.000000     8.000000  2009.000000  212975.000000  \n",
       "max     738.000000  15500.000000    12.000000  2010.000000  755000.000000  \n",
       "\n",
       "[8 rows x 38 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1314 entries, 302 to 1389\n",
      "Data columns (total 81 columns):\n",
      " #   Column         Non-Null Count  Dtype  \n",
      "---  ------         --------------  -----  \n",
      " 0   Id             1314 non-null   int64  \n",
      " 1   MSSubClass     1314 non-null   int64  \n",
      " 2   MSZoning       1314 non-null   object \n",
      " 3   LotFrontage    1089 non-null   float64\n",
      " 4   LotArea        1314 non-null   int64  \n",
      " 5   Street         1314 non-null   object \n",
      " 6   Alley          81 non-null     object \n",
      " 7   LotShape       1314 non-null   object \n",
      " 8   LandContour    1314 non-null   object \n",
      " 9   Utilities      1314 non-null   object \n",
      " 10  LotConfig      1314 non-null   object \n",
      " 11  LandSlope      1314 non-null   object \n",
      " 12  Neighborhood   1314 non-null   object \n",
      " 13  Condition1     1314 non-null   object \n",
      " 14  Condition2     1314 non-null   object \n",
      " 15  BldgType       1314 non-null   object \n",
      " 16  HouseStyle     1314 non-null   object \n",
      " 17  OverallQual    1314 non-null   int64  \n",
      " 18  OverallCond    1314 non-null   int64  \n",
      " 19  YearBuilt      1314 non-null   int64  \n",
      " 20  YearRemodAdd   1314 non-null   int64  \n",
      " 21  RoofStyle      1314 non-null   object \n",
      " 22  RoofMatl       1314 non-null   object \n",
      " 23  Exterior1st    1314 non-null   object \n",
      " 24  Exterior2nd    1314 non-null   object \n",
      " 25  MasVnrType     1307 non-null   object \n",
      " 26  MasVnrArea     1307 non-null   float64\n",
      " 27  ExterQual      1314 non-null   object \n",
      " 28  ExterCond      1314 non-null   object \n",
      " 29  Foundation     1314 non-null   object \n",
      " 30  BsmtQual       1280 non-null   object \n",
      " 31  BsmtCond       1280 non-null   object \n",
      " 32  BsmtExposure   1279 non-null   object \n",
      " 33  BsmtFinType1   1280 non-null   object \n",
      " 34  BsmtFinSF1     1314 non-null   int64  \n",
      " 35  BsmtFinType2   1280 non-null   object \n",
      " 36  BsmtFinSF2     1314 non-null   int64  \n",
      " 37  BsmtUnfSF      1314 non-null   int64  \n",
      " 38  TotalBsmtSF    1314 non-null   int64  \n",
      " 39  Heating        1314 non-null   object \n",
      " 40  HeatingQC      1314 non-null   object \n",
      " 41  CentralAir     1314 non-null   object \n",
      " 42  Electrical     1313 non-null   object \n",
      " 43  1stFlrSF       1314 non-null   int64  \n",
      " 44  2ndFlrSF       1314 non-null   int64  \n",
      " 45  LowQualFinSF   1314 non-null   int64  \n",
      " 46  GrLivArea      1314 non-null   int64  \n",
      " 47  BsmtFullBath   1314 non-null   int64  \n",
      " 48  BsmtHalfBath   1314 non-null   int64  \n",
      " 49  FullBath       1314 non-null   int64  \n",
      " 50  HalfBath       1314 non-null   int64  \n",
      " 51  BedroomAbvGr   1314 non-null   int64  \n",
      " 52  KitchenAbvGr   1314 non-null   int64  \n",
      " 53  KitchenQual    1314 non-null   object \n",
      " 54  TotRmsAbvGrd   1314 non-null   int64  \n",
      " 55  Functional     1314 non-null   object \n",
      " 56  Fireplaces     1314 non-null   int64  \n",
      " 57  FireplaceQu    687 non-null    object \n",
      " 58  GarageType     1241 non-null   object \n",
      " 59  GarageYrBlt    1241 non-null   float64\n",
      " 60  GarageFinish   1241 non-null   object \n",
      " 61  GarageCars     1314 non-null   int64  \n",
      " 62  GarageArea     1314 non-null   int64  \n",
      " 63  GarageQual     1241 non-null   object \n",
      " 64  GarageCond     1241 non-null   object \n",
      " 65  PavedDrive     1314 non-null   object \n",
      " 66  WoodDeckSF     1314 non-null   int64  \n",
      " 67  OpenPorchSF    1314 non-null   int64  \n",
      " 68  EnclosedPorch  1314 non-null   int64  \n",
      " 69  3SsnPorch      1314 non-null   int64  \n",
      " 70  ScreenPorch    1314 non-null   int64  \n",
      " 71  PoolArea       1314 non-null   int64  \n",
      " 72  PoolQC         7 non-null      object \n",
      " 73  Fence          259 non-null    object \n",
      " 74  MiscFeature    50 non-null     object \n",
      " 75  MiscVal        1314 non-null   int64  \n",
      " 76  MoSold         1314 non-null   int64  \n",
      " 77  YrSold         1314 non-null   int64  \n",
      " 78  SaleType       1314 non-null   object \n",
      " 79  SaleCondition  1314 non-null   object \n",
      " 80  SalePrice      1314 non-null   int64  \n",
      "dtypes: float64(3), int64(35), object(43)\n",
      "memory usage: 841.8+ KB\n"
     ]
    }
   ],
   "source": [
    "train_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `pandas_profiler`\n",
    "\n",
    "We do not have `pandas_profiling` in our course environment. You will have to install it in the environment on your own if you want to run the code below. \n",
    "\n",
    "```pip install -U pandas-profiling[notebook]```\n",
    "\n",
    "The above worked for me. If this doesn't work for you, check the instructions [here](https://pypi.org/project/pandas-profiling/). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pandas_profiling import ProfileReport"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# profile = ProfileReport(train_df, title=\"Pandas Profiling Report\")  # , minimal=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature types \n",
    "\n",
    "- Do not blindly trust all the info given to you by automated tools. \n",
    "- How does pandas profiling figure out the data type?\n",
    "    - You can look at the Python data type and say floats are numeric, strings are categorical.\n",
    "    - However, in doing so you would miss out on various subtleties such as some of the string features being ordinal rather than truly categorical.\n",
    "    - Also, it will think free text is categorical."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- In addition to tools such as above, it's important to go through data description to understand the data.\n",
    "- The data description for our dataset is available [here](https://www.kaggle.com/c/home-data-for-ml-course/data?select=data_description.txt).     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature types \n",
    "\n",
    "- We have mixed feature types and a bunch of missing values. \n",
    "- Now, let's identify feature types and transformations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Let's get the numeric-looking columns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Id', 'MSSubClass', 'LotFrontage', 'LotArea', 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 'KitchenAbvGr', 'TotRmsAbvGrd', 'Fireplaces', 'GarageYrBlt', 'GarageCars', 'GarageArea', 'WoodDeckSF', 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 'MiscVal', 'MoSold', 'YrSold']\n"
     ]
    }
   ],
   "source": [
    "numeric_looking_columns = X_train.select_dtypes(include=np.number).columns.tolist()\n",
    "print(numeric_looking_columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Not all numeric looking columns are necessarily numeric. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 20,  50,  30,  60, 160,  85,  90, 120, 180,  80,  70,  75, 190,\n",
       "        45,  40])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[\"MSSubClass\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MSSubClass: Identifies the type of dwelling involved in the sale.\t\n",
    "\n",
    "        20\t1-STORY 1946 & NEWER ALL STYLES\n",
    "        30\t1-STORY 1945 & OLDER\n",
    "        40\t1-STORY W/FINISHED ATTIC ALL AGES\n",
    "        45\t1-1/2 STORY - UNFINISHED ALL AGES\n",
    "        50\t1-1/2 STORY FINISHED ALL AGES\n",
    "        60\t2-STORY 1946 & NEWER\n",
    "        70\t2-STORY 1945 & OLDER\n",
    "        75\t2-1/2 STORY ALL AGES\n",
    "        80\tSPLIT OR MULTI-LEVEL\n",
    "        85\tSPLIT FOYER\n",
    "        90\tDUPLEX - ALL STYLES AND AGES\n",
    "       120\t1-STORY PUD (Planned Unit Development) - 1946 & NEWER\n",
    "       150\t1-1/2 STORY PUD - ALL AGES\n",
    "       160\t2-STORY PUD - 1946 & NEWER\n",
    "       180\tPUD - MULTILEVEL - INCL SPLIT LEV/FOYER\n",
    "       190\t2 FAMILY CONVERSION - ALL STYLES AND AGES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Also, month sold is more of a categorical feature than a numeric feature. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  7,  3,  5,  8, 10,  6,  9, 12,  2,  4, 11])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[\"MoSold\"].unique() # Month Sold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "drop_features = [\"Id\"]\n",
    "numeric_features = [\n",
    "    \"BedroomAbvGr\",\n",
    "    \"KitchenAbvGr\", \n",
    "    \"LotFrontage\",\n",
    "    \"LotArea\",\n",
    "    \"OverallQual\",\n",
    "    \"OverallCond\",\n",
    "    \"YearBuilt\",\n",
    "    \"YearRemodAdd\",\n",
    "    \"MasVnrArea\",\n",
    "    \"BsmtFinSF1\",\n",
    "    \"BsmtFinSF2\",\n",
    "    \"BsmtUnfSF\",\n",
    "    \"TotalBsmtSF\",\n",
    "    \"1stFlrSF\",\n",
    "    \"2ndFlrSF\",\n",
    "    \"LowQualFinSF\",\n",
    "    \"GrLivArea\",\n",
    "    \"BsmtFullBath\",\n",
    "    \"BsmtHalfBath\",\n",
    "    \"FullBath\",\n",
    "    \"HalfBath\",\n",
    "    \"TotRmsAbvGrd\",\n",
    "    \"Fireplaces\",\n",
    "    \"GarageYrBlt\",\n",
    "    \"GarageCars\",\n",
    "    \"GarageArea\",\n",
    "    \"WoodDeckSF\",\n",
    "    \"OpenPorchSF\",\n",
    "    \"EnclosedPorch\",\n",
    "    \"3SsnPorch\",\n",
    "    \"ScreenPorch\",\n",
    "    \"PoolArea\",\n",
    "    \"MiscVal\",\n",
    "    \"YrSold\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "```{note}\n",
    "I've not looked at all the features carefully. It might be appropriate to apply some other encoding on some of the numeric features above. \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'MSSubClass', 'MoSold'}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(numeric_looking_columns) - set(numeric_features) - set(drop_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll treat the above numeric-looking features as categorical features. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- There are a bunch of ordinal features in this dataset. \n",
    "- Ordinal features with the same scale \n",
    "    - Poor (Po), Fair (Fa), Typical (TA), Good (Gd), Excellent (Ex)\n",
    "    - These we'll be calling `ordinal_features_reg`.\n",
    "- Ordinal features with different scales\n",
    "    - These we'll be calling `ordinal_features_oth`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex']]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ordinal_features_reg = [\n",
    "    \"ExterQual\",\n",
    "    \"ExterCond\",\n",
    "    \"BsmtQual\",\n",
    "    \"BsmtCond\",\n",
    "    \"HeatingQC\",\n",
    "    \"KitchenQual\",\n",
    "    \"FireplaceQu\",\n",
    "    \"GarageQual\",\n",
    "    \"GarageCond\",\n",
    "    \"PoolQC\",\n",
    "]\n",
    "ordering = [\n",
    "    \"Po\",\n",
    "    \"Fa\",\n",
    "    \"TA\",\n",
    "    \"Gd\",\n",
    "    \"Ex\",\n",
    "]  # if N/A it will just impute something, per below\n",
    "ordering_ordinal_reg = [ordering] * len(ordinal_features_reg)\n",
    "ordering_ordinal_reg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll pass the above as categories in our `OrdinalEncoder`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- There are a bunch more ordinal features using different scales.\n",
    "  - These we'll be calling `ordinal_features_oth`. \n",
    "  - We are encoding them separately. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "ordinal_features_oth = [\n",
    "    \"BsmtExposure\",\n",
    "    \"BsmtFinType1\",\n",
    "    \"BsmtFinType2\",\n",
    "    \"Functional\",\n",
    "    \"Fence\",\n",
    "]\n",
    "ordering_ordinal_oth = [\n",
    "    ['NA', 'No', 'Mn', 'Av', 'Gd'],\n",
    "    ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ', 'ALQ', 'GLQ'],\n",
    "    ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ', 'ALQ', 'GLQ'],\n",
    "    ['Sal','Sev','Maj2','Maj1','Mod','Min2','Min1','Typ'],    \n",
    "    ['NA', 'MnWw', 'GdWo', 'MnPrv', 'GdPrv']\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The remaining features are categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Foundation',\n",
       " 'Utilities',\n",
       " 'SaleCondition',\n",
       " 'CentralAir',\n",
       " 'LandContour',\n",
       " 'Condition2',\n",
       " 'GarageType',\n",
       " 'MasVnrType',\n",
       " 'MiscFeature',\n",
       " 'BldgType',\n",
       " 'RoofStyle',\n",
       " 'HouseStyle',\n",
       " 'Heating',\n",
       " 'RoofMatl',\n",
       " 'Exterior1st',\n",
       " 'MoSold',\n",
       " 'Exterior2nd',\n",
       " 'MSSubClass',\n",
       " 'SaleType',\n",
       " 'Alley',\n",
       " 'Street',\n",
       " 'Condition1',\n",
       " 'LotShape',\n",
       " 'LotConfig',\n",
       " 'MSZoning',\n",
       " 'GarageFinish',\n",
       " 'LandSlope',\n",
       " 'Electrical',\n",
       " 'Neighborhood',\n",
       " 'PavedDrive']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_features = list(\n",
    "    set(X_train.columns)\n",
    "    - set(numeric_features)\n",
    "    - set(ordinal_features_reg)\n",
    "    - set(ordinal_features_oth)    \n",
    "    - set(drop_features)\n",
    ")\n",
    "categorical_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- We are not doing it here but we can engineer our own features too. \n",
    "- Would price per square foot be a good feature to add in here?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Applying feature transformations\n",
    "\n",
    "- Since we have mixed feature types, let's use `ColumnTransformer` to apply different transformations on different features types.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer, make_column_transformer\n",
    "\n",
    "numeric_transformer = make_pipeline(SimpleImputer(strategy=\"median\"), StandardScaler())\n",
    "ordinal_transformer_reg = make_pipeline(\n",
    "    SimpleImputer(strategy=\"most_frequent\"),\n",
    "    OrdinalEncoder(categories=ordering_ordinal_reg),\n",
    ")\n",
    "\n",
    "ordinal_transformer_oth = make_pipeline(\n",
    "    SimpleImputer(strategy=\"most_frequent\"),\n",
    "    OrdinalEncoder(categories=ordering_ordinal_oth),\n",
    ")\n",
    "\n",
    "categorical_transformer = make_pipeline(\n",
    "    SimpleImputer(strategy=\"constant\", fill_value=\"missing\"),\n",
    "    OneHotEncoder(handle_unknown=\"ignore\", sparse=False),\n",
    ")\n",
    "\n",
    "preprocessor = make_column_transformer(\n",
    "    (\"drop\", drop_features),\n",
    "    (numeric_transformer, numeric_features),\n",
    "    (ordinal_transformer_reg, ordinal_features_reg),\n",
    "    (ordinal_transformer_oth, ordinal_features_oth),    \n",
    "    (categorical_transformer, categorical_features),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Examining the preprocessed data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'drop': 'drop',\n",
       " 'pipeline-1': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='median')),\n",
       "                 ('standardscaler', StandardScaler())]),\n",
       " 'pipeline-2': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='most_frequent')),\n",
       "                 ('ordinalencoder',\n",
       "                  OrdinalEncoder(categories=[['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex']]))]),\n",
       " 'pipeline-3': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='most_frequent')),\n",
       "                 ('ordinalencoder',\n",
       "                  OrdinalEncoder(categories=[['NA', 'No', 'Mn', 'Av', 'Gd'],\n",
       "                                             ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ',\n",
       "                                              'ALQ', 'GLQ'],\n",
       "                                             ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ',\n",
       "                                              'ALQ', 'GLQ'],\n",
       "                                             ['Sal', 'Sev', 'Maj2', 'Maj1',\n",
       "                                              'Mod', 'Min2', 'Min1', 'Typ'],\n",
       "                                             ['NA', 'MnWw', 'GdWo', 'MnPrv',\n",
       "                                              'GdPrv']]))]),\n",
       " 'pipeline-4': Pipeline(steps=[('simpleimputer',\n",
       "                  SimpleImputer(fill_value='missing', strategy='constant')),\n",
       "                 ('onehotencoder',\n",
       "                  OneHotEncoder(handle_unknown='ignore', sparse=False))])}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor.fit(X_train) # Calling fit to examine all the transformers.\n",
    "preprocessor.named_transformers_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "ohe_columns = list(\n",
    "    preprocessor.named_transformers_[\"pipeline-4\"]\n",
    "    .named_steps[\"onehotencoder\"]\n",
    "    .get_feature_names_out(categorical_features)\n",
    ")\n",
    "new_columns = numeric_features + ordinal_features_reg + ordinal_features_oth + ohe_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BedroomAbvGr</th>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>...</th>\n",
       "      <th>Neighborhood_SWISU</th>\n",
       "      <th>Neighborhood_Sawyer</th>\n",
       "      <th>Neighborhood_SawyerW</th>\n",
       "      <th>Neighborhood_Somerst</th>\n",
       "      <th>Neighborhood_StoneBr</th>\n",
       "      <th>Neighborhood_Timber</th>\n",
       "      <th>Neighborhood_Veenker</th>\n",
       "      <th>PavedDrive_N</th>\n",
       "      <th>PavedDrive_P</th>\n",
       "      <th>PavedDrive_Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>2.312501</td>\n",
       "      <td>0.381428</td>\n",
       "      <td>0.663680</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.993969</td>\n",
       "      <td>0.840492</td>\n",
       "      <td>0.269972</td>\n",
       "      <td>-0.961498</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767</th>\n",
       "      <td>1.372763</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>0.260890</td>\n",
       "      <td>0.248457</td>\n",
       "      <td>-0.054669</td>\n",
       "      <td>1.285467</td>\n",
       "      <td>-1.026793</td>\n",
       "      <td>0.016525</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.476092</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>2.885044</td>\n",
       "      <td>0.131607</td>\n",
       "      <td>-0.054669</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.563314</td>\n",
       "      <td>0.161931</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>1.227559</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1139</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>1.358264</td>\n",
       "      <td>-0.171468</td>\n",
       "      <td>-0.773017</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>-1.689338</td>\n",
       "      <td>-1.679877</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.443419</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>558</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>-0.597924</td>\n",
       "      <td>1.289541</td>\n",
       "      <td>0.663680</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.828332</td>\n",
       "      <td>0.598149</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.354114</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 263 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      BedroomAbvGr  KitchenAbvGr  LotFrontage   LotArea  OverallQual  \\\n",
       "302       0.154795     -0.222647     2.312501  0.381428     0.663680   \n",
       "767       1.372763     -0.222647     0.260890  0.248457    -0.054669   \n",
       "429       0.154795     -0.222647     2.885044  0.131607    -0.054669   \n",
       "1139      0.154795     -0.222647     1.358264 -0.171468    -0.773017   \n",
       "558       0.154795     -0.222647    -0.597924  1.289541     0.663680   \n",
       "\n",
       "      OverallCond  YearBuilt  YearRemodAdd  MasVnrArea  BsmtFinSF1  ...  \\\n",
       "302     -0.512408   0.993969      0.840492    0.269972   -0.961498  ...   \n",
       "767      1.285467  -1.026793      0.016525   -0.573129    0.476092  ...   \n",
       "429     -0.512408   0.563314      0.161931   -0.573129    1.227559  ...   \n",
       "1139    -0.512408  -1.689338     -1.679877   -0.573129    0.443419  ...   \n",
       "558     -0.512408   0.828332      0.598149   -0.573129    0.354114  ...   \n",
       "\n",
       "      Neighborhood_SWISU  Neighborhood_Sawyer  Neighborhood_SawyerW  \\\n",
       "302                  0.0                  0.0                   0.0   \n",
       "767                  0.0                  0.0                   0.0   \n",
       "429                  0.0                  0.0                   0.0   \n",
       "1139                 0.0                  0.0                   0.0   \n",
       "558                  0.0                  0.0                   0.0   \n",
       "\n",
       "      Neighborhood_Somerst  Neighborhood_StoneBr  Neighborhood_Timber  \\\n",
       "302                    0.0                   0.0                  0.0   \n",
       "767                    0.0                   0.0                  0.0   \n",
       "429                    0.0                   0.0                  1.0   \n",
       "1139                   0.0                   0.0                  0.0   \n",
       "558                    0.0                   0.0                  0.0   \n",
       "\n",
       "      Neighborhood_Veenker  PavedDrive_N  PavedDrive_P  PavedDrive_Y  \n",
       "302                    0.0           0.0           0.0           1.0  \n",
       "767                    0.0           0.0           0.0           1.0  \n",
       "429                    0.0           0.0           0.0           1.0  \n",
       "1139                   0.0           0.0           0.0           1.0  \n",
       "558                    0.0           0.0           0.0           1.0  \n",
       "\n",
       "[5 rows x 263 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_enc = pd.DataFrame(\n",
    "    preprocessor.transform(X_train), index=X_train.index, columns=new_columns\n",
    ")\n",
    "X_train_enc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 80)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 263)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_enc.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We went from 80 features to 263 features!! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Other possible preprocessing?  \n",
    "\n",
    "- There is a lot of room for improvement ...\n",
    "- We're just using `SimpleImputer`.\n",
    "    - In reality we'd want to go through this more carefully.\n",
    "    - We may also want to drop some columns that are almost entirely missing.    \n",
    "- We could also check for outliers, and do other exploratory data analysis (EDA).\n",
    "- But for now this is good enough ...    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<br><br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Model building "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `DummyRegressor`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.001081</td>\n",
       "      <td>0.000150</td>\n",
       "      <td>-0.003547</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000521</td>\n",
       "      <td>0.000116</td>\n",
       "      <td>-0.001266</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000474</td>\n",
       "      <td>0.000099</td>\n",
       "      <td>-0.011767</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000453</td>\n",
       "      <td>0.000092</td>\n",
       "      <td>-0.006744</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000391</td>\n",
       "      <td>0.000087</td>\n",
       "      <td>-0.076533</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.000426</td>\n",
       "      <td>0.000085</td>\n",
       "      <td>-0.003133</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.000390</td>\n",
       "      <td>0.000087</td>\n",
       "      <td>-0.000397</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.000435</td>\n",
       "      <td>0.000088</td>\n",
       "      <td>-0.003785</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.000397</td>\n",
       "      <td>0.000086</td>\n",
       "      <td>-0.001740</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.000499</td>\n",
       "      <td>0.000108</td>\n",
       "      <td>-0.000117</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.001081    0.000150   -0.003547          0.0\n",
       "1  0.000521    0.000116   -0.001266          0.0\n",
       "2  0.000474    0.000099   -0.011767          0.0\n",
       "3  0.000453    0.000092   -0.006744          0.0\n",
       "4  0.000391    0.000087   -0.076533          0.0\n",
       "5  0.000426    0.000085   -0.003133          0.0\n",
       "6  0.000390    0.000087   -0.000397          0.0\n",
       "7  0.000435    0.000088   -0.003785          0.0\n",
       "8  0.000397    0.000086   -0.001740          0.0\n",
       "9  0.000499    0.000108   -0.000117          0.0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy = DummyRegressor()\n",
    "pd.DataFrame(cross_validate(dummy, X_train, y_train, cv=10, return_train_score=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Apply `Ridge`\n",
    "\n",
    "- Recall that we are going to use `Ridge()` instead of `LinearRegression()` in this course. \n",
    "    - It has a hyperparameter `alpha` which controls the fundamental tradeoff. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.023278</td>\n",
       "      <td>0.005241</td>\n",
       "      <td>0.861355</td>\n",
       "      <td>0.911906</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.021812</td>\n",
       "      <td>0.006070</td>\n",
       "      <td>0.812301</td>\n",
       "      <td>0.913861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.020669</td>\n",
       "      <td>0.004597</td>\n",
       "      <td>0.775283</td>\n",
       "      <td>0.915963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.021527</td>\n",
       "      <td>0.004293</td>\n",
       "      <td>0.874519</td>\n",
       "      <td>0.910849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.019674</td>\n",
       "      <td>0.005023</td>\n",
       "      <td>0.851969</td>\n",
       "      <td>0.911622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.018961</td>\n",
       "      <td>0.004333</td>\n",
       "      <td>0.826198</td>\n",
       "      <td>0.910176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.019353</td>\n",
       "      <td>0.004159</td>\n",
       "      <td>0.825533</td>\n",
       "      <td>0.913781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.020317</td>\n",
       "      <td>0.004146</td>\n",
       "      <td>0.872238</td>\n",
       "      <td>0.910071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.018580</td>\n",
       "      <td>0.004274</td>\n",
       "      <td>0.196663</td>\n",
       "      <td>0.921448</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.020550</td>\n",
       "      <td>0.004561</td>\n",
       "      <td>0.890474</td>\n",
       "      <td>0.908221</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.023278    0.005241    0.861355     0.911906\n",
       "1  0.021812    0.006070    0.812301     0.913861\n",
       "2  0.020669    0.004597    0.775283     0.915963\n",
       "3  0.021527    0.004293    0.874519     0.910849\n",
       "4  0.019674    0.005023    0.851969     0.911622\n",
       "5  0.018961    0.004333    0.826198     0.910176\n",
       "6  0.019353    0.004159    0.825533     0.913781\n",
       "7  0.020317    0.004146    0.872238     0.910071\n",
       "8  0.018580    0.004274    0.196663     0.921448\n",
       "9  0.020550    0.004561    0.890474     0.908221"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_pipe = make_pipeline(preprocessor, Ridge())\n",
    "pd.DataFrame(cross_validate(lr_pipe, X_train, y_train, cv=10, return_train_score=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Quite a bit of variation in the test scores. \n",
    "- Performing poorly in fold 8. Not sure why. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Tuning `alpha` hyperparameter of `Ridge`\n",
    "- Recall that `Ridge` has a hyperparameter `alpha` that controls the fundamental tradeoff.\n",
    "- This is like `C` in `LogisticRegression` but, annoyingly, `alpha` is the inverse of `C`.\n",
    "- That is, large `C` is like small `alpha` and vice versa.\n",
    "- Smaller `alpha`: lower training error (overfitting)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "alphas = 10.0 ** np.arange(-5, 5, 1)\n",
    "train_scores = []\n",
    "cv_scores = []\n",
    "for alpha in alphas:\n",
    "    lr = make_pipeline(preprocessor, Ridge(alpha=alpha))\n",
    "    results = cross_validate(lr, X_train, y_train, return_train_score=True)\n",
    "    train_scores.append(np.mean(results[\"train_score\"]))\n",
    "    cv_scores.append(np.mean(results[\"test_score\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAG1CAYAAAARLUsBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUVElEQVR4nO3deXxU9b3/8dfMZA9ZSchGFvYdAkHC6hpFqSi9rZciFaRVf/W61dRWKAouVW7dilValGLVVq9Ur/VasWib1gXZJEGRfc0C2clK9szM749JJgQCkvVMMu/n43EeSc6cc+YzDGHefM93MdntdjsiIiIibsRsdAEiIiIiPU0BSERERNyOApCIiIi4HQUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNtRABIRERG342F0Aa7IZrORm5tLQEAAJpPJ6HJERETkItjtdiorK4mOjsZsvnAbjwJQG3Jzc4mNjTW6DBEREemAnJwcBg4ceMFjFIDaEBAQADj+AAMDAw2uRkRERC5GRUUFsbGxzs/xC1EAakPzba/AwEAFIBERkV7mYrqvqBO0iIiIuB0FIBEREXE7ugUmIiLSg2w2G/X19UaX0St5enpisVi65FoKQCIiIj2kvr6e48ePY7PZjC6l1woODiYyMrLT09QoAImIiPQAu91OXl4eFouF2NjYb52nRlqz2+1UV1dTWFgIQFRUVKeupwAkIiLSAxobG6muriY6Oho/Pz+jy+mVfH19ASgsLGTAgAGduh2m+CkiItIDrFYrAF5eXgZX0rs1h8eGhoZOXUcBSEREpAdpiaXO6ao/PwUgERERcTsKQCIiIuJ2FIBERESkRyQkJLB69WqjywA0CkxEREQu4PLLLycxMbFLgsuXX36Jv79/54vqAgpAPehvX+fydvoJo8uQLtDeLngd6bPX3lMsZhMeZjMeFhMeZhMeFjOelpZ9nhYzFrMJz6bHPCwmPM1N+yxN+8xnHNfWuWft8zjzOZuu52FpXYc6fIr0bXa7HavViofHt0eK8PDwHqjo4igA9aCc0mo+O1RkdBkiPcoRxs4MRU3BrHlfq7BmIiLQh0lxIUyKD2ZMdBA+nl0z7b2Iq7Hb7dQ0WA15bl9Py0X95+TWW2/l008/5dNPP+X5558H4I9//CNLlizhww8/5KGHHuKbb77h448/JjY2ltTUVLZt20ZVVRWjRo1i1apVpKSkOK+XkJDAT3/6U376058CjhFd69atY+PGjXz00UfExMTw7LPPcsMNN3TL6z6TAlAPumpkBJGBPkaXIZ1kt7fz+A49R/vOsgM2m50Gm51Gqw2rzU6D1fF9875Gm51Gq51Gm835WKPN3rS/aZ/NRqPVTkPzNZrPtdppaHrMcW3HuQ3WM/bZbG3+2TQ/B1z81P9/35MPgJfFzJiYQCbFhZAUH8KkuBAig/Q7JH1DTYOV0Ss+MuS59z02Gz+vb48Azz//PIcOHWLs2LE89thjAOzduxeApUuX8swzzzB48GBCQkLIyclhzpw5PPHEE3h7e/P6668zd+5cDh48SFxc3Hmf49FHH+Wpp57i6aef5oUXXmDhwoVkZWURGhraNS/2PBSAetCIyABGRAYYXYZItzkzHJ0Zutrc1xSomkOa1WajvtHO8eIqMrJLycgq5VRVPbuyy9iVXcb6zccBiA7yYVJTGEqKD2FUVCBeHhrPIdIdgoKC8PLyws/Pj8jISAAOHDgAwGOPPcbVV1/tPDY0NJQJEyY4f3788cf561//yvvvv8/dd9993ue49dZbWbBgAQBPPvkkv/3tb9mxYwfXXnttd7wkJwUgEekyFrMJi7lrblnZ7XayS6pJzyptCkRlHMivILe8ltzdeXywOw8Abw8z4wcGOUPRpLgQwgO8u6QGke7k62lh32OzDXvuzpo8eXKrn0+fPs0jjzzCxo0bycvLo7GxkZqaGrKzsy94nfHjxzu/9/f3JzAw0LneV3dSABIRl2QymYjv7098f3/+Y9JAAE7XNbI7p4yM7FLSs0rZlVNGWXUDX2aW8mVmqfPcuFA/JsUFkxQfwsS4EEZGBuBhUSuRuBaTyXRRt6Fc1dmjuR544AH+8Y9/8MwzzzB06FB8fX35/ve/T319/QWv4+np2epnk8mEzXbxt8w7qvf+yYuI2+nn7cH0oWFMHxoGOFqJjhVXOcJQUyvRocJKskuqyS6p5r2vcgHw87IwYWAwk+KbQlFsCCH+Wo9J5GJ4eXk51zG7kC+++IJbb72V7373u4CjRSgzM7Obq+s4BSAR6bVMJhNDwvsxJLwf/zk5FoCK2ga+yi5z3jr7KruMyrpGth47xdZjp5znDg7zb7ltFh/MsAEBWMwasi9ytoSEBLZv305mZib9+vU7b+vMsGHDePfdd5k7dy4mk4mHH364R1pyOkoBSET6lEAfTy4dHs6lwx3zjdhsdg4XnnZ2rE7PLuVYURXHih3bO01zcwV4e5AYF9wUiEJIjA0myNfzQk8l4hYeeOABFi9ezOjRo6mpqeGPf/xjm8c999xz/OhHP2L69OmEhYXx4IMPUlFR0cPVXjyTvb3jbd1ARUUFQUFBlJeXExgYaHQ5ItLFSqvq+SrnjFainDKq61s38ZtMMGxAP2cgmhQXwuAwf8xqJZIOqq2t5fjx4wwaNAgfH03n0FEX+nNsz+e3WoBExO2E+HtxxcgBXDFyAACNVhsHCyrJyC4joykUZZ2q5lDBaQ4VnOatL3MACPL1ZFJTK1FSfAgTYoPx99Y/oyK9kX5zRcTteVjMjIkOYkx0ELdMjQeg+HRdUxhyjDrbfaKM8poG/n2wiH8fdMzobjbBiMhAkuKDnUPw4/v7afkPkV5AAUhEpA1h/by5Zkwk14xxTP7WYLWxP6+iqR+Ro6XoZFkN+/Mq2J9XwZ+3OeY66e/vxcS4EKYP6c93J8ZotJmIi1IfoDaoD5CIXIyCilrnLbP0rFL2nKyg3toy6sXLw8z146JYODWOSXEhahlyc+oD1DXUB0hExGARgT5cNy6K68ZFAVDXaGVvbgXpmaW899VJ9uZW8O6uk7y76yQjIwNYmBzHvIkxBPhodJmI0dQC1Aa1AIlIZ9ntdnafKOfP27L42+5cahscLUN+XhZuTIxhYXIcY2OCDK5SepJagLqGWoBERFyYyWRiQmwwE2KDeeg7o3l31wne2J7NkcLT/M+ObP5nRzYTYoNZmBzH3PHR+Hp1zRpqInJxFIBERLpZkJ8nS2YM4tbpCew4XsIb27P5+548vs4p4+ucMh7/YB/fmzSQhclxDIsIMLpcEbegACQi0kNMJhPJg/uTPLg/xadH8076Cd7cnk12STWvbsnk1S2ZTBkUysLkOK4dG4m3h1qFRLqLApCIiAHC+nnzk8uGcMeswWw+Uswb27P45/5CdhwvYcfxEkL9vbhp8kBunhJHfH//b7+giLSLApCIiIHMZpNz7bL88lre+jKbt3bkkF9Ry0ufHuOlT48xa1gYP5waz1UjB+BhMRtdskifoN8kEREXERnkw09ThrP5wSt4+ZYkLhsejskEnx8u5v/9KZ0Zv/4Xv/nHIfLKa4wuVdyMzWbjqaeeYujQoXh7exMXF8cTTzzB9OnTefDBB1sdW1RUhKenJ5999plB1V4cDYNvg4bBi4iryD5Vzf98mc1fvszhVFU94FiC46pRESxMjuPSYeFaoLWXOGf4tt0ODdXGFOPp51jx9yI9+OCDrFu3jt/85jfMnDmTvLw8Dhw4QF1dHU899RSZmZnOiT5ffPFFnn766Vb7ulJXDYNXAGqDApCIuJq6Risf7y3gje1ZbDtW4twfG+rLgilx/OfkWML6eRtYoXybcz6466vgyWhjivllLnhdXN+yyspKwsPDefHFF7nttttaPVZUVER0dDT/+te/mDVrFgDTp0/n0ksv5b//+7+7vGzougCkW2AiIr2At4eFuROieeuOafwz9VKWzEgg0MeDnJIantp0kGmr0rj7zQy2HTuF/l8rXWn//v3U1dVx1VVXnfNYeHg411xzDW+88QYAx48fZ+vWrSxcuLCny2w3wztBr1mzhqeffpr8/HwmTJjACy+8wJQpU9o8tqGhgVWrVvHaa69x8uRJRowYwa9//WuuvfbaDl9TRKS3GToggJVzx/CL2SP5YHcub2zP5qucMj7YnccHu/MYEu7PwuR4vjdpIEF+WnbDZXn6OVpijHrui+Tr63vBxxcuXMi9997LCy+8wJtvvsm4ceMYN25cZyvsdoa2AG3YsIHU1FRWrlxJRkYGEyZMYPbs2RQWFrZ5/EMPPcRLL73ECy+8wL59+/jJT37Cd7/7XXbt2tXha4qI9Fa+XhZumhzLe3fN4IN7ZnJzchx+XhaOFlXx2Af7mPLkP3ng7a/ZlV2qViFXZDI5bkMZsbWjb86wYcPw9fUlLS2tzcdvvPFGamtr2bRpE2+++WavaP0Bg/sAJScnc8kll/Diiy8Cjl7msbGx3HPPPSxduvSc46Ojo1m+fDl33XWXc9/3vvc9fH19+fOf/9yha7ZFfYBEpLeqrG3g/77K5c/bsjiQX+ncPzoqkIVT47gxMYZ+3oY3/rul3rwW2KOPPsrzzz/P6tWrmTFjBkVFRezdu5cf//jHAPzwhz9kz5497N69m8zMTOLi4rqtll6/Flh9fT3p6eksW7bMuc9sNpOSksLWrVvbPKeuru6cF+vr68vmzZs7fM3m69bV1Tl/rqio6NBrEhExWoCPJz+cGs/C5Dh25ZTxxrZsPtidy768Cpb/dQ9PbtzPvIkxLEyOZ3S0/oMnF+fhhx/Gw8ODFStWkJubS1RUFD/5yU+cjy9cuJA5c+Zw6aWXdmv46UqGBaDi4mKsVisRERGt9kdERHDgwIE2z5k9ezbPPfccl156KUOGDCEtLY13330Xq9Xa4WsCrFq1ikcffbSTr0hExHWYTCYmxYUwKS6Eh68f5Vx241hxFW9sz+aN7dlMigtmYXI83xkfhY+nlt2Q8zObzSxfvpzly5e3+fh1113X626z9qpRYM8//zzDhg1j5MiReHl5cffdd7NkyRLM5s69jGXLllFeXu7ccnJyuqhiERHjBft5cduswaT97DLevD2Z74yPwsNsIiO7jJ+9/TXJT6bx+Af7OFp02uhSRXqMYS1AYWFhWCwWCgoKWu0vKCggMjKyzXPCw8N57733qK2t5dSpU0RHR7N06VIGDx7c4WsCeHt74+2t+TNEpG8zmUxMHxLG9CFhFFbW8vZOR6vQybIa1m8+zvrNx5k2uD8Lp8ZxzehIvDx61f+RRdrFsL/dXl5eJCUltepVbrPZSEtLY9q0aRc818fHh5iYGBobG/nf//1fbrzxxk5fU0TEnQwI8OGuK4by2S+u4I9LLiFlVARmE2w9doq739zF9P9O46lNB8gpMWimYpFuZuhQgNTUVBYvXszkyZOZMmUKq1evpqqqiiVLlgCwaNEiYmJiWLVqFQDbt2/n5MmTJCYmcvLkSR555BFsNhu/+MUvLvqaIiLSwmI2ccWIAVwxYgAny2rYsCObt77MobCyjt99cpSXPzvG4ukJ3JcyjEAfzSkkfYehAWj+/PkUFRWxYsUK8vPzSUxMZNOmTc5OzNnZ2a3699TW1vLQQw9x7Ngx+vXrx5w5c/jTn/5EcHDwRV9TRETaFhPsS+o1I7jnqmGk7S/gT9uy+OLIKdZvPs7/fXWSX8weyfeTBmrtMekTtBZYGzQPkIiIw6eHinj0b3s5VlQFwISBQay8YQyT4kIMrqz3aZ6/JiEh4VtnV5bzq6mpITMzU2uBiYhI97lseDib7ruUh74zin7eHnx9opz/+N0WfvaXrymsrDW6vF7FYnFMNVBfX29wJb1bdbWjX5qnZ+duyaoFqA1qARIROVdhZS1PbzrI2+knAOjn7cG9Vw3l1umDNGLsItjtdrKzs2loaCA6OrrTU7i4G7vdTnV1NYWFhQQHBxMVFXXOMe35/FYAaoMCkIjI+e3KLuWR9/fy9YlyAAaH+7Pi+tFcPmKAwZW5vvr6eo4fP47NZjO6lF4rODiYyMhITG2sZ6YA1EkKQCIiF2az2Xkn4wRPbTpA8WnHLZ2UURE8fP0o4vv7G1yda7PZbLoN1kGenp7OW4ltUQDqJAUgEZGLU1HbwG//eZhXt2TSaLPjZTFz+6WD+K/Lh+KvRVelhykAdZICkIhI+xwprOTRv+3j88PFAEQG+rBszkhumBDd5q0Kke6gANRJCkAiIu1nt9v5x74CHt+4j5ySGgAuSQjhkRvGMCY6yODqxB0oAHWSApCISMfVNlj5w+fHWPPvo9Q0WDGbYMGUOH52zQhC/b2MLk/6MAWgTlIAEhHpvNyyGp78cD8f7M4DIMjXk59dM5ybp8ThYdEQcOl6CkCdpAAkItJ1th07xSPv7+VAfiUAIyMDWDl3DNOG9De4MulrFIA6SQFIRKRrNVpt/M+ObJ75+BDlNQ0AfGd8FL+cM4qYYC0LIV1DAaiTFIBERLpHaVU9z/7jIG9uz8ZmBx9PM/91+VDuuHQwPp7nn99F5GIoAHWSApCISPfam1vOo+/vY0dmCQCxob489J3RXDM6QsPmpcMUgDpJAUhEpPvZ7Xb+tjuPJzfuJ7/CsbDqrGFhrJw7mqEDAgyuTnojBaBOUgASEek51fWN/O7fR3n5s2PUW214mE0snp7AfSnDCPTp3Irf4l4UgDpJAUhEpOdlnariVxv38499BQCE9fPiF7NH8v2kgZjNui0m304BqJMUgEREjPPpoSIe/dtejhVVATBhYBArbxjDpLgQgysTV6cA1EkKQCIixqpvtPHalkyeTzvM6bpGAL43aSAPXjeCAQE+BlcnrkoBqJMUgEREXENhZS1PbTrIO+knAOjn7cF9Vw1j8fQEvDw0m7S0pgDUSQpAIiKuZVd2KY+8v5evT5QDMDjcnxXXj+byEQMMrkxciQJQJykAiYi4HpvNzjsZJ3hq0wGKT9cDkDIqgoevH0V8f3+DqxNXoADUSQpAIiKuq6K2gd/+8zCvbsmk0WbHy2Lm9ksH8V+XD8Xf28Po8sRACkCdpAAkIuL6jhRW8ujf9vH54WIAIgN9WDZnJDdMiNZs0m5KAaiTFIBERHoHu93Ox/sKePyDfZworQHgkoQQHrlhDGOigwyuTnqaAlAnKQCJiPQutQ1W1n12jDWfHKG2wYbZBAumxPHANSMI8fcyujzpIQpAnaQAJCLSO+WW1fDkh/v5YHceAEG+nvzsmuHcPCUOD4uGzfd1CkCdpAAkItK7bTt2ikfe38uB/EoARkYG8PItk4nr72dwZdKd2vP5rTgsIiJ9ztTB/fngnpk8fuMYgnw9OZBfyfyXt3K8uMro0sRFKACJiEif5GExc8u0BD6+/1KGhPuTV17L/Je2cqTwtNGliQtQABIRkT4tItCHt+6YxoiIAAor6/jBy1s52HRrTNyXApCIiPR54QHe/M8dUxkdFUjx6XoWrNvGvtwKo8sSAykAiYiIWwj19+LN25MZFxNESZUjBH3TtLaYuB8FIBERcRvBfl78+bZkJsYFU17TwM1/2Mau7FKjyxIDKACJiIhbCfL15PUfTeGShBAqaxu5Zf0OdmaWGF2W9DAFIBERcTsBPp68umQKUweHcrqukUWv7GD7sVNGlyU9SAFIRETckr+3B3+8dQozh4ZRXW9l8R938MWRYqPLkh5ieABas2YNCQkJ+Pj4kJyczI4dOy54/OrVqxkxYgS+vr7ExsZy//33U1tb63z8kUcewWQytdpGjhzZ3S9DRER6IV8vC39YPJnLR4RT22DjR69+yScHC40uS3qAoQFow4YNpKamsnLlSjIyMpgwYQKzZ8+msLDtv3xvvvkmS5cuZeXKlezfv5/169ezYcMGfvnLX7Y6bsyYMeTl5Tm3zZs398TLERGRXsjH08JLtySRMmoAdY027ng9nbT9BUaXJd3M0AD03HPPcfvtt7NkyRJGjx7N2rVr8fPz45VXXmnz+C1btjBjxgxuvvlmEhISuOaaa1iwYME5rUYeHh5ERkY6t7CwsJ54OSIi0kt5e1j43cIkrh0TSb3Vxk/+nM6mPflGlyXdyLAAVF9fT3p6OikpKS3FmM2kpKSwdevWNs+ZPn066enpzsBz7NgxPvzwQ+bMmdPquMOHDxMdHc3gwYNZuHAh2dnZF6ylrq6OioqKVpuIiLgXLw8zL9w8kevHR9FgtXPXmxl8sDvX6LKkm3gY9cTFxcVYrVYiIiJa7Y+IiODAgQNtnnPzzTdTXFzMzJkzsdvtNDY28pOf/KTVLbDk5GReffVVRowYQV5eHo8++iizZs1iz549BAQEtHndVatW8eijj3bdixMRkV7J02Jm9fxEPC1m/rrrJPf+zy4arXbmTYwxujTpYoZ3gm6PTz75hCeffJLf/e53ZGRk8O6777Jx40Yef/xx5zHXXXcdN910E+PHj2f27Nl8+OGHlJWV8Ze//OW81122bBnl5eXOLScnpydejoiIuCAPi5lnbprATUkDsdnh/r98xTvpJ4wuS7qYYS1AYWFhWCwWCgpadzQrKCggMjKyzXMefvhhbrnlFm677TYAxo0bR1VVFXfccQfLly/HbD43zwUHBzN8+HCOHDly3lq8vb3x9vbuxKsREZG+xGI28evvjcfTw8yb27P5+Ttf02C1sWBKnNGlSRcxrAXIy8uLpKQk0tLSnPtsNhtpaWlMmzatzXOqq6vPCTkWiwUAu93e5jmnT5/m6NGjREVFdVHlIiLiDsxmE0/MG8ut0xOw22HZu9/wp62ZRpclXcSwFiCA1NRUFi9ezOTJk5kyZQqrV6+mqqqKJUuWALBo0SJiYmJYtWoVAHPnzuW5555j4sSJJCcnc+TIER5++GHmzp3rDEIPPPAAc+fOJT4+ntzcXFauXInFYmHBggWGvU4REemdTCYTK+eOxsNs4g+bj/Pw/+2l3mrnxzMHGV2adJKhAWj+/PkUFRWxYsUK8vPzSUxMZNOmTc6O0dnZ2a1afB566CFMJhMPPfQQJ0+eJDw8nLlz5/LEE084jzlx4gQLFizg1KlThIeHM3PmTLZt20Z4eHiPvz4REen9TCYTy78zCk8PM7//5CiPf7CPBquNn1w2xOjSpBNM9vPdO3JjFRUVBAUFUV5eTmBgoNHliIiIC7Db7fzmn4f5bdphAH529XDuuWqYwVXJmdrz+d2rRoGJiIgYxWQykXr1cH529XAAnv3HIZ77+OB5+6CKa1MAEhERaYd7rhrG0usca0z+9l9HeOojhaDeSAFIRESknX5y2RAevn40AL//5ChPbNyvENTLKACJiIh0wI9nDuKxG8cA8IfNx3nk/b0KQb2IApCIiEgHLZqWwKr/GIfJBK9tzWL5e3uw2RSCegMFIBERkU5YMCWOp78/AZMJ3tyezYP/uxurQpDLUwASERHppO8nDWT1/ETMJng7/QQPvP01jVab0WXJBSgAiYiIdIEbE2N4YcEkLGYTf911kp9u+IoGhSCXpQAkIiLSRb4zPoo1N0/C02Lig9153PPmLuobFYJckQKQiIhIF7p2bCRrf5iEl8XMpr35/Ncb6dQ1Wo0uS86iACQiItLFrhoVwbrFk/H2MPPP/YXc8Xo6tQ0KQa5EAUhERKQbXDY8nFduvQQfTzOfHirittd2UlOvEOQqFIBERES6yYyhYby6ZAp+XhY2Hylmyas7qKprNLosQQFIRESkW00d3J/XfzSFft4ebDtWwuJXdlBZ22B0WW5PAUhERKSbTU4I5c+3JRPg48HOrFIWvbKD8hqFICMpAImIiPSAxNhg/uf2qQT7ebIru4wf/mE7ZdX1RpflthSAREREesjYmCDevG0qof5efHOynAXrtlNSpRBkBAUgERGRHjQ6OpC37phKWD9v9udVsODlbRRV1hldlttRABIREelhwyMCeOuOqQwI8OZgQSU/eHkrhRW1RpflVhSAREREDDB0QD82/L9pRAX5cLSoivkvbyOvvMbostyGApCIiIhBBoX585f/N42YYF+OF1cx/6VtnCitNrost6AAJCIiYqDYUD82/L+pxIX6kV1SzfyXtpF9SiGouykAiYiIGGxgiCMEDQrz52RZDfNf3srx4iqjy+rTFIBERERcQFSQLxvumMrQAf3IK69l/ktbOVJ42uiy+iwFIBERERcxINCHt+6YysjIAAor6/jBy1s5mF9pdFl9kgKQiIiICwnr582bt09ldFQgxafrWbBuG/tyK4wuq89RABIREXExof5evHl7MuMHBlFS5QhB35woN7qsPkUBSERExAUF+3nx59uSmRgXTHlNAzf/YRu7skuNLqvPUAASERFxUYE+nrz+oylckhBCZW0jt6zfoY7RXUQBSERExIUF+Hjy6pIpTIwL5nRdI+9mnDC6pD5BAUhERMTF+Xt7MH9yLAA7s3QbrCsoAImIiPQCkxNCAPg6p4wGq83gano/BSAREZFeYHBYP4L9PKlrtLFXw+I7TQFIRESkFzCbTUyKc7QC7cwsMbia3k8BSEREpJdIincEoAwNh+80wwPQmjVrSEhIwMfHh+TkZHbs2HHB41evXs2IESPw9fUlNjaW+++/n9ra2k5dU0REpDeYHN/cAlSK3W43uJrezdAAtGHDBlJTU1m5ciUZGRlMmDCB2bNnU1hY2Obxb775JkuXLmXlypXs37+f9evXs2HDBn75y192+JoiIiK9xfiBwXiYTRRW1nGitMbocno1QwPQc889x+23386SJUsYPXo0a9euxc/Pj1deeaXN47ds2cKMGTO4+eabSUhI4JprrmHBggWtWnjae00REZHewtfLwpiYIADSNRy+UwwLQPX19aSnp5OSktJSjNlMSkoKW7dubfOc6dOnk56e7gw8x44d48MPP2TOnDkdviZAXV0dFRUVrTYRERFX5LwNlqWO0J1hWAAqLi7GarUSERHRan9ERAT5+fltnnPzzTfz2GOPMXPmTDw9PRkyZAiXX3658xZYR64JsGrVKoKCgpxbbGxsJ1+diIhI90g6ox+QdJzhnaDb45NPPuHJJ5/kd7/7HRkZGbz77rts3LiRxx9/vFPXXbZsGeXl5c4tJyeniyoWERHpWs0tQAcLKqmsbTC4mt7Lw6gnDgsLw2KxUFBQ0Gp/QUEBkZGRbZ7z8MMPc8stt3DbbbcBMG7cOKqqqrjjjjtYvnx5h64J4O3tjbe3dydfkYiISPcbEOhDbKgvOSU17Mou49Lh4UaX1CsZ1gLk5eVFUlISaWlpzn02m420tDSmTZvW5jnV1dWYza1LtlgsANjt9g5dU0RE+iibDWpKoeQ4lGZCHxo2ntQ8IaI6QneYYS1AAKmpqSxevJjJkyczZcoUVq9eTVVVFUuWLAFg0aJFxMTEsGrVKgDmzp3Lc889x8SJE0lOTubIkSM8/PDDzJ071xmEvu2aIiLSi9isUFsOtWVQU9a+r7UVwBmhJ3AgDLkChl4Fgy4Dv9CefCVdKikhlPe+yiVDAajDDA1A8+fPp6ioiBUrVpCfn09iYiKbNm1ydmLOzs5u1eLz0EMPYTKZeOihhzh58iTh4eHMnTuXJ5544qKvKSIiPaw5xNSUtjPIlENdeeef39MPbI1QcQJ2/cmxmcwQPQmGXOnYBk4Gi2fnn6uHNPcD2pVdSqPVhoelV3XpdQkmu6aSPEdFRQVBQUGUl5cTGBhodDkiIsazNp4npJwZas5uqWn6ua4Lphbx9AffYPAJbufXIPDwhvpqyN4CR/4FR/8FRftbX987EAZd2hKIQgd1vuZuZLXZSXz0YyrrGvngnpmMbZobyN215/Pb0BYgERFxQbUVcGgT7Ps/yPvaEXLqT3f+ul79OhBggptCjFcnn9sPhqY4NoDyk3Ds344wdPTfUFMCBz5wbAAhgxy3yoZcCQmzwMe1/jNsMZtIjAvm88PFZGSXKgB1gAKQiIg4Wm8O/h32vgdH08Ba3/ZxXgHntrCcE1xC2m6JcaVbTEExMPGHjs1mdQS9o2mOMJSzHUqPw5d/cGxmDxg4xRGGhl4JUYlgthj9CpgcH8rnh4vZmVnKomkJRpfT6ygAiYi4q5oyR+jZ956jJeTM0NN/GIyZ52gx8Q9vaYmx9MGPDbMFYiY5tkt/DnWVcPzzptahNCg55rh9lr0F/v0rR8AbfEXL7bKgGEPKbp4QUUtidEwf/JssIiLnVVN6RkvPv8B2xkR6YSMcoWf0PBgwCkwmg4o0mHcAjJzj2MAxjP7Yv+FIGhz/zPFnuPddxwYQPrIpDF0F8dMdt9t6QGJcMGYTnCyrIa+8hqgg3x553r5CAUhEpK+rKYUDGx2h59gnrUNP+EhH4BkzzxF65Fyhgxzb5B85OoOf3NnUOvQvOJkORQcc27bfgcUL4qa19B+KGNttQbKftwejogLZm1tBelYp149XAGoPBSARkb6ousQReva91xR6GlseCx91RkvPSGPq660sHhA31bFd8UvHn/Pxzxy3yo78yzHU/vinju0fK8B/QMutsiFXQL8BXVpOUnwIe3Mr2JlZyvXjo7v02n2dApCISF9R3TSSae97jg/gM0PPgDFNoedGCB9hVIV9j1+o4891zDzHTNOnjjhulR39F2R+DlWFsPstxwYQOa7ldlncVMcQ/U5Iig/h9a1ZZGSrH1B7KQCJiPRmVafgwN+aQs9nYLe2PBYxtuX2Vtgwgwp0IyaT4885bBhM/Qk01jlGlB39lyMU5e+G/G8c2xfPOyZoTJjZ0kIUNrzdt8smJzhms96bW0F1fSN+XvpYv1j6kxIR6W2qimH/3xy3t45/3jr0RI5zhJ7R8yBsqEEFCuBo3Rl0qWNLeQROFzluRx5taiE6XQCHP3Zs0KGlOqKDfIgM9CG/opavcsqYPiSsW19SX6IAJCLSG5wuamnpydx8VugZ39Knp/8QgwqUb9UvHMbf5NjsdijY29KZOmtL66U6MDmG5Q+56oJLdZhMJpISQti4O4+MrFIFoHZQABIRcVWnC1taejI3g93W8ljUhKaWnhsVenojkwkixzq2Gfe2vVTHyXTH9tlTLUt1DLsGEm9uFYYmxzsCkFaGbx8FIBERV1JZAPvfdyxDkfVF69ATPbEl9Lj4WlXSTmcv1VGR29I6dPZSHXYbTF7iPLV5QsSMrFJsNjtms5vO39ROCkAiIkZrDj1733OEHs5Yozp6UsvorZAEY+qTnhcYfe5SHV883zRrd1qrADQqKhBfTwsVtY0cKTrN8IgA4+ruRRSARESMUJkP+953fKBlbaFV6ImZ7Ag9o26AkHiDChSX0bxUx9T/arkdarOB2QyAp8VMYmwwW4+dYmdmqQLQRVIAEhHpKRW5LaEnexutQs/AS1pubwXHGlSguLSYSeDp75jZu3CvY8Rfk6T4EEcAyirh5uQ4A4vsPRSARES6U/nJlttbOdtaPzZwSsvtraCBRlQnvYnFE+KnwZF/OqY/ODMAJWhh1PZSABIR6Up2O5RmtqyynrO99eOxU1tubxm0irj0YgmzmgLQZzDtv5y7J8WFYDJB1qlqiirrCA/o3AzT7kABSESkM+x2KD3u6JeR+YXja8WJMw4wOZY8GD0PRt/g6Nwq0lGDZjm+Zm1xdI42WwAI8vVk+IAADhZUkp5VyrVjIw0ssndQABIRaQ+7HUqOOdZ5ag48lbmtjzF7QuwUx62tUTdAYJQxtUrfEznBMSdQXbljZFjMJOdDk+JDmgJQiQLQRVAAEhG5kOYFLs8MPKfzWx9j9nR0Yk6Y4VjbaeAUx7wuIl3N4gHxM+DQ3x1/J88IQJPjQ/ifHdnqB3SRFIBERM5kt0PxoZbAk/WFY82mM1m8HCHHGXguAU9fY+oV9zNoliMAHf8cZtzn3D25qSP0npMV1DZY8fG0GFVhr6AAJCLuzW6HogNNfXg2OwJPVVHrYyzejltaCTMd//seOFmBR4yTcEY/IGuDc1mMuFA/wvp5UXy6nm9OlnNJwrcvpurOFIBExL3YbI51ljK/cLTyZG2B6uLWx3j4NAWeWY7AE5MEnj7G1Ctytoix4BvimA8od5fj7ypNC6PGh/DR3gLSs0oVgL6FApCI9G02m2PSuDMDT01J62M8fCEuuamFZ6ajX4WHhhGLizKbHcH8wAeO4fBNAQhgcnwoH+0tYGdmKVxmYI29QIcDUFlZGe+88w5Hjx7l5z//OaGhoWRkZBAREUFMjOa2EBGD2KxQsKelw3L2Fsf/lM/k6ecYmh4/w9HKEz0RPLyMqVekIwZd5ghAmZ/DpQ84d09qXhg1uxS73Y7JpIVRz6dDAWj37t2kpKQQFBREZmYmt99+O6Ghobz77rtkZ2fz+uuvd3WdIiJts1kh/5uWPjzZW6C2vPUxXv3OCjyJzn4TIr1S83xA2dugsc7ZYjk2JhAvDzMlVfUcL65icHg/A4t0bR0KQKmpqdx666089dRTBAS0LLo2Z84cbr755i4rTkTkHNZGyN/d0mE5a6tjTpQzeQU4lgxoDjxRExzDh0X6ivCR4B/u6LB/YqdjRCLg7WFhwsAgvswsZWdWqQLQBXToX4Qvv/ySl1566Zz9MTEx5Ofnt3GGiEgHWRsdE75lft4SeOorWx/jHQhx0xx9eBJmOCaLU+CRvsxkcvx93/tXx+9GUwACx22wLzNLSc8s5T8na2Hd8+nQvxDe3t5UVFScs//QoUOEh4d3uigRcWPWBsj9qiXwZG+D+tOtj/EJgrjpZwSe8c4lAUTcxqBLHQHo+Odw+VLn7snxobzEMdKzNSHihXQoAN1www089thj/OUvfwEcQ++ys7N58MEH+d73vtelBYpIH1N3GirzoTLvjC0fKnId3+fvgYaq1uf4BDfdzmoKPBFjFXhEEi51fD2xAxpqnHNTJTV1hD5SeJqy6nqC/dTBvy0dCkDPPvss3//+9xkwYAA1NTVcdtll5OfnM23aNJ544omurlFEeoPGescSEc4w00bIqcyHunNbj8/hG3JG4JkJA8Y4hv6KSIv+QyAgyvH7lbMdBl8OQKi/F4PD/DlWXEV6VilXjYowtk4X1aEAFBQUxD/+8Q+++OILvv76a06fPs2kSZNISUnp6vpExGg2m6OjpTPEnBFuKvJavj97MsEL8ern+Ic7INLxNTCq5eew4RA+SoFH5NuYTI5O/t/8xXEbrCkAgaMVSAHowtodgBoaGvD19eWrr75ixowZzJgx49tPEhHXY7dDbdlZYSbvrJabplYbu/Xirmnxagk1AZEQEN12yPEO+PZrici3G9QUgDI/b7V7ckIIb6efYKcWRj2vdgcgT09P4uLisFov8h9EEel59dXnCTNnBZ3Gmou7nskM/gMc4SUw+vwhxy/U8b9SEekZg5r6AZ1Md/Sv83YMe2/uB/R1Thn1jTa8PNSierYO3QJbvnw5v/zlL/nTn/5EaKjWGrlojfVgrTe6ii5iP+tHe9uP2c867oKPdeX12lGf3eZo4bBZm35u/v6Mr3Z7G/tsjttD5+w786u15eczvz9nn7XlWm2ee/a+5uPPuEZNSUvIOXsiwAvxDWkKM2e00JwddPwHaFi5iCsKSYCgOCjPhpxtMNTRFWVwWD+C/Twpq25gX14FibHBhpbpijr0L9qLL77IkSNHiI6OJj4+Hn9//1aPZ2RkdElxfc7WFyHtUaOrEHfh6dd06ym6JdQEnN16E6VFPkV6u0Gz4Ks3HOuCNQUgs9lEUlwIaQcK2ZlZogDUhg4FoHnz5nVpEWvWrOHpp58mPz+fCRMm8MILLzBlypQ2j7388sv59NNPz9k/Z84cNm7cCMCtt97Ka6+91urx2bNns2nTpi6tW1zNWbdeWt2KucBjJjOYLI5h1SaL47Hm751fzeceZ7Y07TO7xvG+wa1DjneAbkeJuIOE5gDUuh/QpHhHAErPKuW2WQbV5sI6FIBWrlzZZQVs2LCB1NRU1q5dS3JyMqtXr2b27NkcPHiQAQMGnHP8u+++S319y22kU6dOMWHCBG666aZWx1177bX88Y9/dP7s7e0CKztPvweSf9Izz9UjH3wdDBwXfOysx89+TB/oIiKtNa8LlveV4/a3TxAAk5v6Ae3M0sKobenUTf309HT2798PwJgxY5g4cWK7r/Hcc89x++23s2TJEgDWrl3Lxo0beeWVV1i6dOk5x5/d5+itt97Cz8/vnADk7e1NZGRku+vpVhZPLcAoIiJdK2gghA6GkmOOpWJGXAvAhNhgPMwmiirrOFFaQ2yon8GFupYOdQsvLCzkyiuv5JJLLuHee+/l3nvvJSkpiauuuoqioqKLvk59fT3p6emt5g8ym82kpKSwdevWi7rG+vXr+cEPfnBOP6RPPvmEAQMGMGLECO68805OnTp13mvU1dVRUVHRahMREek1Eppagc4YDu/jaWFMjKM1aGdWiRFVubQOBaB77rmHyspK9u7dS0lJCSUlJezZs4eKigruvffei75OcXExVquViIjWkzRFRERc1KKqO3bsYM+ePdx2222t9l977bW8/vrrpKWl8etf/5pPP/2U66677rxD91etWkVQUJBzi43V4nEiItKLNA+HP966j2zzbbB0zQd0jg7dAtu0aRP//Oc/GTVqlHPf6NGjWbNmDddcc02XFfdt1q9fz7hx487pMP2DH/zA+f24ceMYP348Q4YM4ZNPPuGqq6465zrLli0jNTXV+XNFRYVCkIiI9B4JMx1f8/dAdYljTi4cAWj95uPszFQAOluHWoBsNhuenuf2ZfH09MRms130dcLCwrBYLBQUFLTaX1BQ8K39d6qqqnjrrbf48Y9//K3PM3jwYMLCwjhy5Eibj3t7exMYGNhqExER6TUCIiFsBGCHrC+cu5snRDxYUElFbYNBxbmmDgWgK6+8kvvuu4/c3FznvpMnT3L//fe32cJyPl5eXiQlJZGWlubcZ7PZSEtLY9q0aRc89+2336auro4f/vCH3/o8J06c4NSpU0RFRV10bSIiIr1K82iwM4bDDwj0ITbUF7sdvsouM6YuF9WhAPTiiy9SUVFBQkICQ4YMYciQIQwaNIiKigpeeOGFdl0rNTWVdevW8dprr7F//37uvPNOqqqqnKPCFi1axLJly845b/369cybN4/+/fu32n/69Gl+/vOfs23bNjIzM0lLS+PGG29k6NChzJ49uyMvV0RExPU1d4Q+/lmr3ZPjHbfDtC5Yax3qAxQbG0tGRgb//Oc/OXDgAACjRo3q0Grw8+fPp6ioiBUrVpCfn09iYiKbNm1ydozOzs7GfNaq0AcPHmTz5s18/PHH51zPYrGwe/duXnvtNcrKyoiOjuaaa67h8ccfd425gERERLpDcwAq2g+ni6BfOOCYEPGvu06SrpFgrZjs9gstyOSeKioqCAoKory8XP2BRESk9/jddCjcC9//I4z9DwD251Vw3fOf4+9l4euV1+Bh6bsLo7bn87tDfwr33nsvv/3tb8/Z/+KLL/LTn/60I5cUERGRzmoeDn/GfEDDIwII8Pagqt7KgfxKgwpzPR0KQP/7v//LjBkzztk/ffp03nnnnU4XJSIiIh3QRkdoi9lEYlwwoPmAztShAHTq1CmCgoLO2R8YGEhxcXGnixIREZEOiJ8OmODUYajIc+5u7gitANSiQwFo6NChba6s/ve//53Bgwd3uigRERHpAN8QiBrv+P6M22CTEzQj9Nk6NAosNTWVu+++m6KiIq688koA0tLSeOaZZ3j++ee7tEARERFph4RZkPe1Yzj8+P8EHAujmk1wsqyGvPIaooJ8DS7SeB0KQD/60Y+oq6vjiSee4PHHHwdg0KBBrF27lkWLFnVpgSIiItIOgy6DrS+2agHq5+3BqKhA9uZWkJ5VyvXjFYA6dAuspqaGxYsXc+LECQoKCti9ezd33333OYuaioiISA+LnwYmC5RmQlm2c3fzwqhaF8yhQwHoxhtv5PXXXwcc63+lpKTw3HPPMW/ePH7/+993aYEiIiLSDt4BED3R8f0Zo8GSEtQR+kwdCkAZGRnMmuUYavfOO+8QERFBVlYWr7/+epvzA4mIiEgPah4Of8ZtsOaFUfflVVBd32hEVS6lQwGourqagIAAAD7++GP+4z/+A7PZzNSpU8nKyurSAkVERKSdmidEPP45NC34EBPsS1SQD1abna9yyoyrzUV0eBj8e++9R05ODh999BHXXHMNAIWFhVo6QkRExGixU8HsCRUnoPS4c3dzK1C6+gF1LACtWLGCBx54gISEBJKTk5k2bRrgaA2aOHFilxYoIiIi7eTlBwMnO74/Y3X45gCkleE7GIC+//3vk52dzc6dO1tNiHjVVVfxm9/8psuKExERkQ5KOHdZjOYZoTOyS7HZ3Hst9A4vCRsZGcnEiRMxm1suMWXKFEaOHNklhYmIiEgnnNkRuqkf0KioAHw9LVTWNnK48LSBxRmvwwFIREREXNjAKWDxhtMFUHwYAA+LmcTYYAB2ZpUYWJzxFIBERET6Ik8fiJ3i+D6zpR+Q1gVzUAASERHpq5zD4c/tCK0AJCIiIn1Tc0fozM1gswEwMS4EkwmyTlVTVFlnYHHGUgASERHpq2KSwNMPqk9B0X4Agnw9GT7AMZmxO7cCKQCJiIj0VR5eEDfV8X2rdcGab4O5b0doBSAREZG+LKGNdcHiNCGiApCIiEhf1twROvNzsFmBlpFge06WU9tgNaoyQykAiYiI9GVRieAVALXlkP8NAHGhfoT186bBauebk+XG1mcQBSAREZG+zOIB8dMd3zfdBjOZTCTFBwOw000XRlUAEhER6esGnX9dMHcdCaYAJCIi0tc1d4TO2gLWRqBlJFhGdil2u/stjKoAJCIi0tdFjgOfIKivhLyvABgTHYiXh5mSqnqOFVcZW58BFIBERET6OrMF4mc6vm9aFsPbw8KEgUGAe94GUwASERFxB2cOh2+S1NwPyA07QisAiYiIuIPmjtDZ26CxHmhZGHWnG84IrQAkIiLiDsJHgV9/aKiGk+lASwA6WlRFaVW9kdX1OAUgERERd2A2Q0JTP6Cm22Ch/l4MDvcHHKPB3IkCkIiIiLtoHg7f1BEa3HddMAUgERERdzHoMsfXnB3QUAu0rAvmbiPBFIBERETcRdgw6BcB1jo48SXQMhLs65wy6httRlbXoxSARERE3IXJdM5tsMFh/gT7eVLXaGNvrvssjOoSAWjNmjUkJCTg4+NDcnIyO3bsOO+xl19+OSaT6ZztO9/5jvMYu93OihUriIqKwtfXl5SUFA4fPtwTL0VERMS1NQ+Hb+oIbTabnP2A3Ok2mOEBaMOGDaSmprJy5UoyMjKYMGECs2fPprCwsM3j3333XfLy8pzbnj17sFgs3HTTTc5jnnrqKX7729+ydu1atm/fjr+/P7Nnz6a2tranXpaIiIhrap4Q8cROqK8GWtYFUwDqQc899xy33347S5YsYfTo0axduxY/Pz9eeeWVNo8PDQ0lMjLSuf3jH//Az8/PGYDsdjurV6/moYce4sYbb2T8+PG8/vrr5Obm8t577/XgKxMREXFBIYMgcCDYGiBnG9B6JJi7LIxqaACqr68nPT2dlJQU5z6z2UxKSgpbt269qGusX7+eH/zgB/j7O+YxOH78OPn5+a2uGRQURHJy8nmvWVdXR0VFRatNRESkTzKZWm6DHXfcBpsQG4ynxURRZR0nSmsMLK7nGBqAiouLsVqtREREtNofERFBfn7+t56/Y8cO9uzZw2233ebc13xee665atUqgoKCnFtsbGx7X4qIiEjvcVZHaB9PC2OiHQujusuyGIbfAuuM9evXM27cOKZMmdKp6yxbtozy8nLnlpOT00UVioiIuKDmFqDcXVBXCZyxLpibLIxqaAAKCwvDYrFQUFDQan9BQQGRkZEXPLeqqoq33nqLH//4x632N5/Xnmt6e3sTGBjYahMREemzguMgJAHsVshydA+ZHO9eHaENDUBeXl4kJSWRlpbm3Gez2UhLS2PatGkXPPftt9+mrq6OH/7wh632Dxo0iMjIyFbXrKioYPv27d96TREREbfRfBss03EbrLkF6GBBJRW1DUZV1WMMvwWWmprKunXreO2119i/fz933nknVVVVLFmyBIBFixaxbNmyc85bv3498+bNo3///q32m0wmfvrTn/KrX/2K999/n2+++YZFixYRHR3NvHnzeuIliYiIuL7m4fBN/YAGBPoQG+qL3Q67ssuMq6uHeBhdwPz58ykqKmLFihXk5+eTmJjIpk2bnJ2Ys7OzMZtb57SDBw+yefNmPv744zav+Ytf/IKqqiruuOMOysrKmDlzJps2bcLHx6fbX4+IiEiv0NwClLcbakrBN4TJ8aHklJwkPauUy4aHG1tfNzPZ3WXAfztUVFQQFBREeXm5+gOJiEjf9UISnDoCP3gTRn6HP2/L4qH39jBjaH/euG2q0dW1W3s+vw2/BSYiIiIGcd4Gc8wH1Lwy/K7sMhqtfXthVAUgERERd5XQel2wYQMCCPD2oLreyoH8SgML634KQCIiIu6qOQAV7IGqYixmExPdZDi8ApCIiIi76hcO4aMc32duBlrmA9qpACQiIiJ9VnM/oKbbYM3zAWUoAImIiEifddbCqImxwVjMJk6W1ZBX3ncXRlUAEhERcWfxMwATFB+EygL8vT0YFRUA9O11wRSARERE3JlfKESOdXzffBssru93hFYAEhERcXcJrZfFSEoIBRSAREREpC87qyN080iwfXkVVNU1GlVVt1IAEhERcXfx08BkhpJjUH6C6GBfooJ8sNrsfJ1TZnR13UIBSERExN35BEFUouP7462Hw/fV22AKQCIiItIyHP6s22B9dUJEBSARERE5oyN0cwuQoyN0RnYpNpvdqKq6jQKQiIiIQNxUMHtAeTaUZjIqKgA/LwuVtY0cLjxtdHVdTgFIREREwLsfxCQ5vj/+OR4WM4mxwQDszCoxrq5uogAkIiIiDs2rwzfPB9TcEboPzgitACQiIiIOZ3aEtttbAlC2ApCIiIj0VbHJYPGCyjw4dZSJcSGYTJB1qpqiyjqjq+tSCkAiIiLi4OkLA6c4vs/8jCBfT4YPcCyMmt7H+gEpAImIiEiL5ttgzcPhE/rmhIgKQCIiItIioXU/oL46IaICkIiIiLQYOBk8fKCqCIoOODtC7zlZTm2D1eDiuo4CkIiIiLTw8HZMighw/HPiQv0I6+dNg9XONyfLja2tCykAiYiISGvO22CfYTKZWm6D9aH5gBSAREREpLVBTeuCZW4Gm+2MleH7zkgwBSARERFpLXoiePpDTSkU7Gk1Esxu7xsLoyoAiYiISGsWT4if5vg+83PGRgfh5WGmtLqBY8VVxtbWRRSARERE5FzNt8GOf46Xh5kJA4OAvrMumAKQiIiInKu5I3TWF2BtJCk+FOg7EyIqAImIiMi5oiaAdxDUVUD+12dMiNg3OkIrAImIiMi5zBaIn+74/vjnTGoKQEeLqiitqjewsK6hACQiIiJtcw6H/5xQfy8Gh/sDkJHd+2+DKQCJiIhI25oXRs3aCtaGPrUumAKQiIiItG3AGPANhYYqyN3VMiFiHxgJZngAWrNmDQkJCfj4+JCcnMyOHTsueHxZWRl33XUXUVFReHt7M3z4cD788EPn44888ggmk6nVNnLkyO5+GSIiIn2P2QwJMxzfH//UORLs6xNl1DfaDCys8wwNQBs2bCA1NZWVK1eSkZHBhAkTmD17NoWFhW0eX19fz9VXX01mZibvvPMOBw8eZN26dcTExLQ6bsyYMeTl5Tm3zZs398TLERER6XsSWuYDGhLuT4ifJ3WNNvbm9u6FUT2MfPLnnnuO22+/nSVLlgCwdu1aNm7cyCuvvMLSpUvPOf6VV16hpKSELVu24OnpCUBCQsI5x3l4eBAZGdmttYuIiLiF5o7QOdsxWetJig/hn/sLSc8qZWJciLG1dYJhLUD19fWkp6eTkpLSUozZTEpKClu3bm3znPfff59p06Zx1113ERERwdixY3nyySexWq2tjjt8+DDR0dEMHjyYhQsXkp2dfcFa6urqqKioaLWJiIgIED4C/AdAYy2c2OkcDt/bJ0Q0LAAVFxdjtVqJiIhotT8iIoL8/Pw2zzl27BjvvPMOVquVDz/8kIcffphnn32WX/3qV85jkpOTefXVV9m0aRO///3vOX78OLNmzaKysvK8taxatYqgoCDnFhsb2zUvUkREpLczmSBhpuP7zM+Z3NQPaGcvXxjV8E7Q7WGz2RgwYAAvv/wySUlJzJ8/n+XLl7N27VrnMddddx033XQT48ePZ/bs2Xz44YeUlZXxl7/85bzXXbZsGeXl5c4tJyenJ16OiIhI79A8HP74Z4wfGISnxURRZR05JTXG1tUJhvUBCgsLw2KxUFBQ0Gp/QUHBefvvREVF4enpicVice4bNWoU+fn51NfX4+Xldc45wcHBDB8+nCNHjpy3Fm9vb7y9vTv4SkRERPq45o7QJ77Eh3rGRAfxVU4Z6dklxPX3M7a2DjKsBcjLy4ukpCTS0tKc+2w2G2lpaUybNq3Nc2bMmMGRI0ew2VqG3h06dIioqKg2ww/A6dOnOXr0KFFRUV37AkRERNxF/yEQEA3WesjZ3jIhYi+eD8jQW2CpqamsW7eO1157jf3793PnnXdSVVXlHBW2aNEili1b5jz+zjvvpKSkhPvuu49Dhw6xceNGnnzySe666y7nMQ888ACffvopmZmZbNmyhe9+97tYLBYWLFjQ469PRESkTzCZzrgN9jmTE3p/R2hDh8HPnz+foqIiVqxYQX5+PomJiWzatMnZMTo7OxuzuSWjxcbG8tFHH3H//fczfvx4YmJiuO+++3jwwQedx5w4cYIFCxZw6tQpwsPDmTlzJtu2bSM8PLzHX5+IiEifkTALdm+A458xacrPAThYUElFbQOBPp4GF9d+Jntv7sLdTSoqKggKCqK8vJzAwECjyxERETFeaSY8PwHMHvBgFpc+/yXZJdW89qMpXDbcNRoZ2vP53atGgYmIiIhBQhIgKA5sjZC9zdkPKD2zxNi6OkgBSERERC5O86zQmZ+1TIiY3Tv7ASkAiYiIyMVpoyP0ruwyGq29b2FUBSARERG5OAlNASjvK4YH2gjw9qC63sqB/POvtuCqFIBERETk4gTFQOhgsNsw52xlonM+oN7XD0gBSERERC5ecz+g45+3dITOLjOung5SABIREZGL13wbLPOzXj0STAFIRERELl5zAMrfw4T+NixmE7nlteSW9a6FURWARERE5OIFREDYCMCOf942RkUFAL1vWQwFIBEREWmf5uHwmZ8zOT4UUAASERGRvu6MjtDNEyLuzOpd/YAUgERERKR94mc6vhbtZ0pYIwD78yqpqms0sKj2UQASERGR9vHvDxFjAYgs/ZLoIB+sNjtf55QZW1c7KACJiIhI+yW0LIvRchus9/QDUgASERGR9mvVEbppPiAFIBEREenT4meAyQynjjA1vB6AjOxSbDa7wYVdHAUgERERaT/fYIgcD8Cw6l34eVmorG3kUGHvWBhVAUhEREQ6puk2mCXrcxJjg4HecxtMAUhEREQ6JqGNhVEzFYBERESkL4ufBiYLlGUxLawa6D0jwRSAREREpGO8AyBmEgCJjd9gMkF2STWFlbUGF/btFIBERESk45rmA/I9uYUREY6FUTN6QSuQApCIiIh0XPN8QMc/Y1JcMAA7e0E/IAUgERER6bjYqWD2hIqTXBbmGAKfnq0AJCIiIn2Zlx8MvASAyfa9AOw5WU5tg9XIqr6VApCIiIh0TtNtsNCi7YT186bBamf3iXKDi7owBSARERHpnKaO0KbjnzG5qR+Qq0+IqAAkIiIinTPwErB4Q1UhKQPKAEjPKjG2pm+hACQiIiKd4+kDsVMAmGLaBzhagOx2110YVQFIREREOm/QZQAMLNuJt4eZ0uoGjhVXGVzU+SkAiYiISOc1dYQ2Z20mMSYQcO11wRSAREREpPOiJ4GnH1SfYvYAR/+fnS7cD0gBSERERDrPwwvipgIww7IfcO2RYApAIiIi0jUGXer4cjoDgKNFVZRW1RtZ0XkpAImIiEjXSHAEIK8TWxgW5gO4biuQApCIiIh0jagJ4BUAteVcH3EKcN11wQwPQGvWrCEhIQEfHx+Sk5PZsWPHBY8vKyvjrrvuIioqCm9vb4YPH86HH37YqWuKiIhIF7B4QPx0AC73PAC47kgwQwPQhg0bSE1NZeXKlWRkZDBhwgRmz55NYWFhm8fX19dz9dVXk5mZyTvvvMPBgwdZt24dMTExHb6miIiIdKGm4fBDa3YB8PWJMuobbUZW1CaT3cBpGpOTk7nkkkt48cUXAbDZbMTGxnLPPfewdOnSc45fu3YtTz/9NAcOHMDT07NLrtmWiooKgoKCKC8vJzAwsIOvTkRExA3lfQ0vXYrdK4BLGtZRXGPjr/81nYlxId3+1O35/DasBai+vp709HRSUlJaijGbSUlJYevWrW2e8/777zNt2jTuuusuIiIiGDt2LE8++SRWq7XD1wSoq6ujoqKi1SYiIiIdEDEOfIIx1Vfy3cgiwDU7QhsWgIqLi7FarURERLTaHxERQX5+fpvnHDt2jHfeeQer1cqHH37Iww8/zLPPPsuvfvWrDl8TYNWqVQQFBTm32NjYTr46ERERN2U2Q8JMAK7yOQTAThfsB2R4J+j2sNlsDBgwgJdffpmkpCTmz5/P8uXLWbt2baeuu2zZMsrLy51bTk5OF1UsIiLihhIc/YBG1Tr6AaVnu97CqB5GPXFYWBgWi4WCgoJW+wsKCoiMjGzznKioKDw9PbFYLM59o0aNIj8/n/r6+g5dE8Db2xtvb+9OvBoRERFxauoIHVicgZ/FSlFlHTklNcT19zO4sBaGtQB5eXmRlJREWlqac5/NZiMtLY1p06a1ec6MGTM4cuQINltLb/JDhw4RFRWFl5dXh64pIiIiXSx8FPiFYWqoZl64owuKq60LZugtsNTUVNatW8drr73G/v37ufPOO6mqqmLJkiUALFq0iGXLljmPv/POOykpKeG+++7j0KFDbNy4kSeffJK77rrroq8pIiIi3eyMfkDX+Dn6AblaR2jDboEBzJ8/n6KiIlasWEF+fj6JiYls2rTJ2Yk5Ozsbs7klo8XGxvLRRx9x//33M378eGJiYrjvvvt48MEHL/qaIiIi0gMGzYJ97zGu4WvgCpcLQIbOA+SqNA+QiIhIJxUdgjWXYLd4M7LqJepNXny14hqCfNuex68r9Ip5gERERKQPCxsG/SIxWeu4NigHux12udC6YApAIiIi0vVMJudosO8EHAYgw4VugykAiYiISPdomg8o0foNADsVgERERKTPa2oBCi/fgy+1fJVTRqPVNRZGVQASERGR7hEyCAIHYrI1MMvnKNX1Vg7kVxpdFaAAJCIiIt3FZIJBlwJwQ+BRAHZmusaEiApAIiIi0n2aboNNtu8BXKcfkAKQiIiIdJ+mjtARp/fTj2qXGQmmACQiIiLdJzgWQhIw2a0kWw6SW15LblmN0VUpAImIiEg3S2ieD+gI4Bq3wRSAREREpHsNugyAqaZ9gGtMiKgAJCIiIt2rqSN0VM0hAjnNzizjR4IpAImIiEj3CoiE/sMwYWeqeT/78yqpqms0tCQFIBEREel+Ta1AKb6HsNrsfJ1TZmg5CkAiIiLS/ZomRJxh2Q8Y3xFaAUhERES6X9NIsJj6Y0zq30g/bw9DyzH22UVERMQ9+IfBgNFQuI9359hhzCBDy1ELkIiIiPSMplYgjn9mbB0oAImIiEhPaeoITebnxtaBApCIiIj0lPgZgAmKD0FlvqGlKACJiIhIz/ALhchxju8zNxtaigKQiIiI9Jym4fBG9wPSKDARERHpOeNugrBhMPhyQ8tQABIREZGeE53o2AymW2AiIiLidhSARERExO0oAImIiIjbUQASERERt6MAJCIiIm5HAUhERETcjgKQiIiIuB0FIBEREXE7CkAiIiLidhSARERExO0oAImIiIjbUQASERERt6MAJCIiIm5Hq8G3wW63A1BRUWFwJSIiInKxmj+3mz/HL0QBqA2VlZUAxMbGGlyJiIiItFdlZSVBQUEXPMZkv5iY5GZsNhu5ubkEBARgMpmMLsclVVRUEBsbS05ODoGBgUaX4/b0frgWvR+uRe+Ha+nO98Nut1NZWUl0dDRm84V7+agFqA1ms5mBAwcaXUavEBgYqH9QXIjeD9ei98O16P1wLd31fnxby08zdYIWERERt6MAJCIiIm5HAUg6xNvbm5UrV+Lt7W10KYLeD1ej98O16P1wLa7yfqgTtIiIiLgdtQCJiIiI21EAEhEREbejACQiIiJuRwFIRERE3I4CkIiIiLgdBSDpEdXV1cTHx/PAAw8YXYpby8nJ4fLLL2f06NGMHz+et99+2+iS3M4HH3zAiBEjGDZsGH/4wx+MLsft6XfCNfXEZ4aGwUuPWL58OUeOHCE2NpZnnnnG6HLcVl5eHgUFBSQmJpKfn09SUhKHDh3C39/f6NLcQmNjI6NHj+bf//43QUFBJCUlsWXLFvr37290aW5LvxOuqSc+M9QCJN3u8OHDHDhwgOuuu87oUtxeVFQUiYmJAERGRhIWFkZJSYmxRbmRHTt2MGbMGGJiYujXrx/XXXcdH3/8sdFluTX9TrienvrMUAByc5999hlz584lOjoak8nEe++9d84xa9asISEhAR8fH5KTk9mxY0e7nuOBBx5g1apVXVRx39YT70ez9PR0rFYrsbGxnazafXT2/cnNzSUmJsb5c0xMDCdPnuyJ0vusrvyd0e9E53XF+9FTnxkKQG6uqqqKCRMmsGbNmjYf37BhA6mpqaxcuZKMjAwmTJjA7NmzKSwsdB6TmJjI2LFjz9lyc3P5v//7P4YPH87w4cN76iX1at39fjQrKSlh0aJFvPzyy93+mvqSrnh/pGt11Xui34mu0dn3o0c/M+wiTQD7X//611b7pkyZYr/rrrucP1utVnt0dLR91apVF3XNpUuX2gcOHGiPj4+39+/f3x4YGGh/9NFHu7LsPqs73g+73W6vra21z5o1y/766693ValuqSPvzxdffGGfN2+e8/H77rvP/sYbb/RIve6go78z+p3oHh15P3ryM0MtQHJe9fX1pKenk5KS4txnNptJSUlh69atF3WNVatWkZOTQ2ZmJs888wy33347K1as6K6S+7SueD/sdju33norV155Jbfcckt3leqWLub9mTJlCnv27OHkyZOcPn2av//978yePduokvu8i3lP9DvRcy7m/ejJzwwFIDmv4uJirFYrERERrfZHRESQn59vUFXuqyvejy+++IINGzbw3nvvkZiYSGJiIt988013lOt2Lub98fDw4Nlnn+WKK64gMTGRn/3sZxoB1o0u5j3R70TPcbXPFI8ef0ZxW7feeqvRJbi9mTNnYrPZjC7Drd1www3ccMMNRpchTfQ74bq6+zNDLUByXmFhYVgsFgoKClrtLygoIDIy0qCq3JfeD9em98f16D1xLa72figAyXl5eXmRlJREWlqac5/NZiMtLY1p06YZWJl70vvh2vT+uB69J67F1d4P3QJzc6dPn+bIkSPOn48fP85XX31FaGgocXFxpKamsnjxYiZPnsyUKVNYvXo1VVVVLFmyxMCq+y69H65N74/r0XviWnrV+9EtY8uk1/j3v/9tB87ZFi9e7DzmhRdesMfFxdm9vLzsU6ZMsW/bts24gvs4vR+uTe+P69F74lp60/uhtcBERETE7agPkIiIiLgdBSARERFxOwpAIiIi4nYUgERERMTtKACJiIiI21EAEhEREbejACQiIiJuRwFIRERE3I4CkIj0GZmZmZhMJr766quLPufVV18lODi422oSEdekACQiIiJuRwFIRERE3I4CkIj0Kps2bWLmzJkEBwfTv39/rr/+eo4ePdrmsZ988gkmk4mNGzcyfvx4fHx8mDp1Knv27Dnn2I8++ohRo0bRr18/rr32WvLy8pyPffnll1x99dWEhYURFBTEZZddRkZGRre9RhHpfgpAItKrVFVVkZqays6dO0lLS8NsNvPd734Xm8123nN+/vOf8+yzz/Lll18SHh7O3LlzaWhocD5eXV3NM888w5/+9Cc+++wzsrOzeeCBB5yPV1ZWsnjxYjZv3sy2bdsYNmwYc+bMobKysltfq4h0Hw+jCxARaY/vfe97rX5+5ZVXCA8PZ9++ffTr16/Nc1auXMnVV18NwGuvvcbAgQP561//yn/+538C0NDQwNq1axkyZAgAd999N4899pjz/CuvvLLV9V5++WWCg4P59NNPuf7667vstYlIz1ELkIj0KocPH2bBggUMHjyYwMBAEhISAMjOzj7vOdOmTXN+HxoayogRI9i/f79zn5+fnzP8AERFRVFYWOj8uaCggNtvv51hw4YRFBREYGAgp0+fvuBziohrUwuQiPQqc+fOJT4+nnXr1hEdHY3NZmPs2LHU19d3+Jqenp6tfjaZTNjtdufPixcv5tSpUzz//PPEx8fj7e3NtGnTOvWcImIsBSAR6TVOnTrFwYMHWbduHbNmzQJg8+bN33retm3biIuLA6C0tJRDhw4xatSoi37eL774gt/97nfMmTMHgJycHIqLizvwCkTEVSgAiUivERISQv/+/Xn55ZeJiooiOzubpUuXfut5jz32GP379yciIoLly5cTFhbGvHnzLvp5hw0bxp/+9CcmT55MRUUFP//5z/H19e3EKxERo6kPkIj0Gmazmbfeeov09HTGjh3L/fffz9NPP/2t5/33f/839913H0lJSeTn5/O3v/0NLy+vi37e9evXU1payqRJk7jlllu49957GTBgQGdeiogYzGQ/80a3iEgf8sknn3DFFVdQWlqq5S5EpBW1AImIiIjbUQASERERt6NbYCIiIuJ21AIkIiIibkcBSERERNyOApCIiIi4HQUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNtRABIRERG3owAkIiIibuf/A05HnE0aLkiaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogx(alphas, train_scores, label=\"train\")\n",
    "plt.semilogx(alphas, cv_scores, label=\"cv\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"alpha\")\n",
    "plt.ylabel(\"score\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100.0"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_alpha = alphas[np.argmax(cv_scores)]\n",
    "best_alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- It seems alpha=100 is the best choice here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- General intuition: larger `alpha` leads to smaller coefficients.\n",
    "- Smaller coefficients mean the predictions are less sensitive to changes in the data.\n",
    "- Hence less chance of overfitting (seeing big dependencies when you shouldn't)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `RidgeCV`\n",
    "\n",
    "BTW, because it's so common to want to tune `alpha` with `Ridge`, sklearn provides a class called `RidgeCV`, which automatically tunes `alpha` based on cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "ridgecv_pipe = make_pipeline(preprocessor, RidgeCV(alphas=alphas, cv=10))\n",
    "ridgecv_pipe.fit(X_train, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100.0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_alpha = ridgecv_pipe.named_steps['ridgecv'].alpha_\n",
    "best_alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Let's examine the coefficients "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([228728.1963872 , 104718.39905565, 155778.96723311, 246316.71119031,\n",
       "       127633.10676873, 243207.19441128, 304930.24461291, 145374.59435295,\n",
       "       157059.38983893, 128487.51979632])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_tuned = make_pipeline(preprocessor, Ridge(alpha=best_alpha))\n",
    "lr_tuned.fit(X_train, y_train)\n",
    "lr_preds = lr_tuned.predict(X_test)\n",
    "lr_preds[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(390726.1064742327, 30791.09250542063)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_preds.max(), lr_preds.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's get the feature names of the transformed data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "ohe_columns = list(\n",
    "    preprocessor.named_transformers_[\"pipeline-4\"]\n",
    "    .named_steps[\"onehotencoder\"]\n",
    "    .get_feature_names_out(categorical_features)\n",
    ")\n",
    "new_columns = numeric_features + ordinal_features_reg + ordinal_features_oth + ohe_columns\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(\n",
    "    data={\n",
    "        \"features\": new_columns,\n",
    "        \"coefficients\": lr_tuned.named_steps[\"ridge\"].coef_,\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>features</th>\n",
       "      <th>coefficients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OverallQual</td>\n",
       "      <td>14484.902165</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>GrLivArea</td>\n",
       "      <td>11704.053037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>251</th>\n",
       "      <td>Neighborhood_NridgHt</td>\n",
       "      <td>9662.969631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>250</th>\n",
       "      <td>Neighborhood_NoRidge</td>\n",
       "      <td>9497.598615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>BsmtQual</td>\n",
       "      <td>8073.088562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119</th>\n",
       "      <td>RoofMatl_ClyTile</td>\n",
       "      <td>-3992.399179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>LandContour_Bnk</td>\n",
       "      <td>-5001.996997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>243</th>\n",
       "      <td>Neighborhood_Gilbert</td>\n",
       "      <td>-5197.585536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>Neighborhood_CollgCr</td>\n",
       "      <td>-5467.463086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>Neighborhood_Edwards</td>\n",
       "      <td>-5796.508529</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>263 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 features  coefficients\n",
       "4             OverallQual  14484.902165\n",
       "16              GrLivArea  11704.053037\n",
       "251  Neighborhood_NridgHt   9662.969631\n",
       "250  Neighborhood_NoRidge   9497.598615\n",
       "36               BsmtQual   8073.088562\n",
       "..                    ...           ...\n",
       "119      RoofMatl_ClyTile  -3992.399179\n",
       "65        LandContour_Bnk  -5001.996997\n",
       "243  Neighborhood_Gilbert  -5197.585536\n",
       "240  Neighborhood_CollgCr  -5467.463086\n",
       "242  Neighborhood_Edwards  -5796.508529\n",
       "\n",
       "[263 rows x 2 columns]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(\"coefficients\",ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So according to this model:\n",
    "\n",
    "- As `OverallQual` feature gets bigger the housing price will get bigger.\n",
    "- Presence of `Neighborhood_Edwards` will result in smaller median house value. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<br><br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Regression score functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "- We aren't doing classification anymore, so we can't just check for equality:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302     False\n",
       "767     False\n",
       "429     False\n",
       "1139    False\n",
       "558     False\n",
       "        ...  \n",
       "1041    False\n",
       "1122    False\n",
       "1346    False\n",
       "1406    False\n",
       "1389    False\n",
       "Name: SalePrice, Length: 1314, dtype: bool"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_tuned.predict(X_train) == y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([205000, 160000, 175000, ..., 262500, 133000, 131000])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([212894.62756285, 178502.78223444, 189937.18327372, ...,\n",
       "       245233.6751565 , 129863.13373552, 135439.89186716])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_tuned.predict(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We need a score that reflects how right/wrong each prediction is."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "A number of popular scoring functions for regression. We are going to look at some common metrics: \n",
    "\n",
    "- mean squared error (MSE)\n",
    "- $R^2$\n",
    "- root mean squared error (RMSE)\n",
    "- MAPE\n",
    "\n",
    "See [sklearn documentation](https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics) for more details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Mean squared error (MSE)\n",
    "\n",
    "- A common metric is mean squared error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "preds = lr_tuned.predict(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "873230473.3636105"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train - preds) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Perfect predictions would have MSE=0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train - y_train) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "This is also implemented in sklearn:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "873230473.3636105"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "mean_squared_error(y_train, preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- MSE looks huge and unreasonable. There is an error of ~\\$1 Billion!\n",
    "- Is this score good or bad?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Unlike classification, with regression **our target has units**. \n",
    "- The target is in dollars, the mean squared error is in $dollars^2$ \n",
    "- The score also depends on the scale of the targets. \n",
    "- If we were working in cents instead of dollars, our MSE would be $10,000 \\times (100^2$) higher!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8732304733636.104"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train * 100 - preds * 100) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Root mean squared error or RMSE\n",
    "\n",
    "- The MSE above is in $dollars^2$.\n",
    "- A more relatable metric would be the root mean squared error, or RMSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29550.473318774617"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_train, lr_tuned.predict(X_train)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Error of \\$30,000 makes more sense.\n",
    "- Can we dig deeper?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAAGwCAYAAACAZ5AeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAC28klEQVR4nOzdeZgU1bn48W9VdVdv092zz7AOCCiigAKKxCySoETJYtQbY7xqYrwaQzRK4pabqMlNQn5JbtTEuORqxMTdxCyKSxC3RHFDUVBBRGDYZp/pvbu6qs7vj2ZaRgaYwYEe4P08Tz8yXaerX1pg3jnnPe/RlFIKIYQQQgjxkeilDkAIIYQQYn8gSZUQQgghxACQpEoIIYQQYgBIUiWEEEIIMQAkqRJCCCGEGACSVAkhhBBCDABJqoQQQgghBoCn1AEcSFzXZfPmzYTDYTRNK3U4QgghhOgDpRSJRIKhQ4ei6zuej5Kkai/avHkzI0aMKHUYQgghhNgNGzZsYPjw4Tu8LknVXhQOh4HC/5RIJFLiaIQQQgjRF/F4nBEjRhS/j++IJFV7UfeSXyQSkaRKCCGE2MfsqnRHCtWFEEIIIQaAJFVCCCGEEANAkiohhBBCiAEgSZUQQgghxACQpEoIIYQQYgBIUiWEEEIIMQAkqRJCCCGEGACSVAkhhBBCDABJqoQQQgghBoAkVUIIIYQQA0CSKiGEEEKIASBJlRBCCCHEAJCkSgghhBBiAEhSJYQQQoh92qZNmzjllFNYv359SePwlPTdhRBCCCE+gr///e+ce+65dHR0kMlkeOyxx0oWi8xUCSGEEGKftWbNGjo6Opg6dSo33HBDSWORmSohhBBC7FNs28bjKaQwl1xyCWVlZXzta1/DNM2SxiUzVUIIIYTYJyiluOGGG5g6dSqpVAoAXdc5//zzS55QgSRVQgghhNgHNDc3c9JJJ3HJJZfw5ptvcscdd5Q6pO1IUiWEEEKIQe2xxx5j0qRJPP744/j9fm666Sbmzp1b6rC2IzVVQgghhBiUstksV155ZbEAfeLEidx7770cdthhJY6sdzJTJYQQQohB6Xvf+14xobrooot4+eWXB21CBZJUCSGEEGKQ+u///m8mTJjAI488wm9+8xv8fn+pQ9opSaqEEEIIMSi0tbVx6623Fr8eMmQIy5cvZ86cOSWMqu+kpkoIIYQQJbd48WLOOusstmzZQnV1NaeeeipQaJmwr9h3IhVCCCHEfseyLK688kqOP/54tmzZwiGHHMJBBx1U6rB2i8xUCSGEEKIkVq9ezVe/+lVeffVVAM4//3x+/etfEwqFShzZ7pGkSgghhBB73X333cd5551HKpWioqKC2267jVNOOaXUYX0kklQJIYQQYq8LhUKkUimOO+44/vSnPzF8+PBSh/SRSVIlhBBCiL0iHo8TiUQA+PznP8/jjz/OrFmzMAyjxJENDClUF0IIIcQeZds21157LWPHjmXjxo3F52fPnr3fJFQgSZUQQggh9qB169bxqU99ih/96Ee0trZy3333lTqkPUaSKiGEEELsEffddx+TJ0/mhRdeIBKJcPfdd/O9732v1GHtMVJTJYQQQogBlUgkuPjii1mwYAEAM2bM4O6772b06NGlDWwPk5kqIYQQQgyoX/3qVyxYsABd17n66qt57rnn9vuECkqcVI0aNQpN07Z7zJ07F4BsNsvcuXOpqqqirKyMU089lebm5h73aGxsZM6cOQSDQWpra7nsssuwbbvHmGeeeYYpU6bg8/kYO3ZsMXPe1u9+9ztGjRqF3+9n+vTpvPzyyz2u9yUWIYQQQsCVV17JiSeeyDPPPMOPfvQjPJ4DY2GspEnVK6+8wpYtW4qPRYsWAfAf//EfAFx66aU8/PDDPPjggzz77LNs3ry5R2Mwx3GYM2cOlmXxwgsvcOedd7JgwQKuvvrq4pi1a9cyZ84cZs6cybJly7jkkks477zzeOKJJ4pj7r//fubNm8c111zDa6+9xuTJk5k9ezYtLS3FMbuKRQghhDhQbdq0iSuuuALHcQAIBAI8+uijfOITnyhxZHuZGkS+853vqDFjxijXdVVXV5fyer3qwQcfLF5/5513FKCWLFmilFLq0UcfVbquq6ampuKYm2++WUUiEZXL5ZRSSl1++eXqsMMO6/E+p59+upo9e3bx66OPPlrNnTu3+LXjOGro0KFq/vz5SinVp1j6IhaLKUDFYrE+v0YIIYQYzB566CFVWVmpAPXzn/+81OHsEX39/j1oaqosy+Kuu+7i3HPPRdM0li5dSj6fZ9asWcUx48ePZ+TIkSxZsgSAJUuWMHHiROrq6opjZs+eTTwe56233iqO2fYe3WO672FZFkuXLu0xRtd1Zs2aVRzTl1h6k8vliMfjPR5CCCHE/iCdTvPNb36TU045hY6ODqZOncqXvvSlUodVUoMmqfrb3/5GV1cXX/va1wBoamrCNE3Ky8t7jKurq6Opqak4ZtuEqvt697WdjYnH42QyGdra2nAcp9cx295jV7H0Zv78+USj0eJjxIgRu/4ghBBCiEFu2bJlTJ06lVtvvRWAyy+/nBdeeIGDDz64xJGV1qBJqm6//XZOPPFEhg4dWupQBsxVV11FLBYrPjZs2FDqkIQQQoiP5K677mL69OmsXLmSIUOGsGjRIv7f//t/mKZZ6tBKblCU469fv54nn3yShx56qPhcfX09lmXR1dXVY4aoubmZ+vr64pgP79Lr3pG37ZgP79Jrbm4mEokQCAQwDAPDMHods+09dhVLb3w+Hz6fr4+fghBCCDH4TZo0CYAvfOEL3H777VRXV5c4osFjUMxU3XHHHdTW1jJnzpzic1OnTsXr9bJ48eLic6tWraKxsZEZM2YAhWZiy5cv77FLb9GiRUQiESZMmFAcs+09usd038M0TaZOndpjjOu6LF68uDimL7EIIYQQ+6t169YVfz1p0iReffVV/va3v0lC9WF7qXB+hxzHUSNHjlRXXHHFdte++c1vqpEjR6qnnnpKvfrqq2rGjBlqxowZxeu2bavDDz9cnXDCCWrZsmXq8ccfVzU1Neqqq64qjnn//fdVMBhUl112mXrnnXfU7373O2UYhnr88ceLY+677z7l8/nUggUL1Ntvv63OP/98VV5e3mNX4a5i6QvZ/SeEEGJfkslk1He+8x3l9XrVyy+/XOpwSqav379LnlQ98cQTClCrVq3a7lomk1Hf+ta3VEVFhQoGg+pLX/qS2rJlS48x69atUyeeeKIKBAKqurpaffe731X5fL7HmKefflodccQRyjRNddBBB6k77rhju/f67W9/q0aOHKlM01RHH320evHFF/sdy65IUiWEEGJf8dZbb6lJkyYpQAHFNkMHor5+/9aUUqqEE2UHlHg8TjQaJRaLEYlESh2OEEIIsR2lFLfeeiuXXnop2WyWmpoa7rjjjh4lOgeavn7/HhSF6kIIIYQovba2Ns477zz+/ve/A3DCCSdw55137nRTlvjAoChUF0IIIUTp/fnPf+bvf/87pmny61//mscee0wSqn6QmSohhBBCAHDBBRfw1ltvce6553LkkUeWOpx9jsxUCSGEEAeo1atXc/rpp5NMJgHQNI3f/va3klDtJpmpEkIIIQ4wSinuvPNOvv3tb5NKpaipqeHGG28sdVj7PEmqhBBCiANIV1cXF1xwAQ888AAAxx13HFdeeWWJo9o/yPKfEEIIcYD497//zeTJk3nggQfweDzMnz+fJ598kuHDh5c6tP2CzFQJIYQQB4B77rmHs846C9d1GTNmDPfccw9HH310qcPar0hSJYQQQhwAPv3pT1NdXc1nP/tZbrzxRsLhcKlD2u9IUiWEEELsp1566SWmT58OQH19PW+88Yb0ndqDpKZKCCGE2M8kEgm+9rWvccwxxxQL0gFJqPYwmakSQggh9iMvv/wyX/3qV1mzZg26rrN+/fpSh3TAkJkqIYQQYj/gOA7z58/n2GOPZc2aNYwcOZJnn32Wyy67rNShHTBkpkoIIYTYx23cuJGzzjqLZ555BoAvf/nL3HrrrZSXl5c0rgONzFQJIYQQ+7g333yTZ555hlAoxB133MF9990nCVUJyEyVEEIIsQ9SSqFpGgAnnXQS119/PSeddBLjxo0rcWQHLpmpEkIIIfYxy5Yt49hjj6WxsbH43He+8x1JqEpMkiohhBBiH+G6Ltdddx3Tp09nyZIlfO973yt1SGIbsvwnhBBC7AOampr42te+xhNPPAHAF7/4RW666aYSRyW2JTNVQgghxCD36KOPMnnyZJ544gn8fj8333wzf/3rX6muri51aGIbMlMlhBBCDGIPPfQQp556KgCTJk3i3nvvZcKECSWOSvRGkiohhBBiEDvppJOYNGkSM2fO5Oc//zl+v7/UIYkdkKRKCCGEGESUUjz00EOcfPLJGIaB3+9nyZIlBIPBUocmdkFqqoQQQohBoq2tjZNPPpnTTjuNn//858XnJaHaN8hMlRBCCDEILF68mLPOOostW7ZgmibhcLjUIYl+kpkqIYQQooQsy+KKK67g+OOPZ8uWLYwfP56XXnqJiy++uNShiX6SmSohhBCiRN577z2+8pWvsHTpUgAuuOACfv3rX8ty3z5KkiohhBCiRDKZDCtWrKCyspLbbruNL33pS6UOSXwEklQJIYQQe5Ft23g8hW+/EydO5L777mPatGkMHz68xJGJj0pqqoQQQoi95F//+lexZqrbySefLAnVfkKSKiGEEGIPs22ba665huOOO441a9bwwx/+sNQhiT1Alv+EEEKIPWjt2rWceeaZLFmyBIBzzjmH3/72tyWOSuwJMlMlhBBC7CH33nsvRxxxBEuWLCESiXDPPfewYMEC6UG1n5KZKiGEEGIP+Oc//8lXv/pVAD72sY9x9913M2rUqNIGJfaoks9Ubdq0if/8z/+kqqqKQCDAxIkTefXVV4vXlVJcffXVDBkyhEAgwKxZs1i9enWPe3R0dHDmmWcSiUQoLy/nG9/4BslksseYN998k0984hP4/X5GjBjBL37xi+1iefDBBxk/fjx+v5+JEyfy6KOP9rjel1iEEEIIgOOPP54vfvGLXHPNNTz77LOSUB0ASppUdXZ2cuyxx+L1ennsscd4++23+d///V8qKiqKY37xi1/wm9/8hltuuYWXXnqJUCjE7NmzyWazxTFnnnkmb731FosWLeKRRx7hueee4/zzzy9ej8fjnHDCCTQ0NLB06VJ++ctfcu211/L73/++OOaFF17gjDPO4Bvf+Aavv/46J598MieffDIrVqzoVyxCCCEOTI7j8Nvf/pZ4PA6Apmk89NBDXHvttcUWCmI/p0roiiuuUB//+Md3eN11XVVfX69++ctfFp/r6upSPp9P3XvvvUoppd5++20FqFdeeaU45rHHHlOapqlNmzYppZS66aabVEVFhcrlcj3e+5BDDil+/eUvf1nNmTOnx/tPnz5dXXDBBX2OZVdisZgCVCwW69N4IYQQ+4YNGzao4447TgHq7LPPLnU4YoD19ft3SWeq/vGPfzBt2jT+4z/+g9raWo488kj+7//+r3h97dq1NDU1MWvWrOJz0WiU6dOnF3dRLFmyhPLycqZNm1YcM2vWLHRdL/YBWbJkCZ/85CcxTbM4Zvbs2axatYrOzs7imG3fp3tM9/v0JZYPy+VyxOPxHg8hhBD7l4ceeohJkybxzDPPEAqFmDlzZqlDEiVS0qTq/fff5+abb2bcuHE88cQTXHjhhVx88cXceeedADQ1NQFQV1fX43V1dXXFa01NTdTW1va47vF4qKys7DGmt3ts+x47GrPt9V3F8mHz588nGo0WHyNGjNjVRyKEEGIfkUqluOCCCzj11FPp7Oxk2rRpvP7663zta18rdWiiREqaVLmuy5QpU/jZz37GkUceyfnnn89//dd/ccstt5QyrAFz1VVXEYvFio8NGzaUOiQhhBADYOXKlUybNo3f//73aJrGlVdeyfPPP8+4ceNKHZoooZImVUOGDGHChAk9njv00ENpbGwEoL6+HoDm5uYeY5qbm4vX6uvraWlp6XHdtm06Ojp6jOntHtu+x47GbHt9V7F8mM/nIxKJ9HgIIYTY91VWVtLZ2cnQoUNZtGgR8+fP71FiIg5MJU2qjj32WFatWtXjuXfffZeGhgYARo8eTX19PYsXLy5ej8fjvPTSS8yYMQOAGTNm0NXVxdKlS4tjnnrqKVzXZfr06cUxzz33HPl8vjhm0aJFHHLIIcWdhjNmzOjxPt1jut+nL7EIIYTYf8ViseKva2trefjhh3njjTf4zGc+U8KoxKCylwrne/Xyyy8rj8ejfvrTn6rVq1eru+++WwWDQXXXXXcVx/z85z9X5eXl6u9//7t688031Re/+EU1evRolclkimM++9nPqiOPPFK99NJL6t///rcaN26cOuOMM4rXu7q6VF1dnTrrrLPUihUr1H333aeCwaC69dZbi2Oef/555fF41K9+9Sv1zjvvqGuuuUZ5vV61fPnyfsWyM7L7Twgh9k0LFy5UNTU16p577il1KKIE+vr9u6RJlVJKPfzww+rwww9XPp9PjR8/Xv3+97/vcd11XfXDH/5Q1dXVKZ/Ppz7zmc+oVatW9RjT3t6uzjjjDFVWVqYikYj6+te/rhKJRI8xb7zxhvr4xz+ufD6fGjZsmPr5z3++XSwPPPCAOvjgg5Vpmuqwww5TCxcu7HcsOyNJlRBC7FsymYy6+OKLFaAA9clPflK5rlvqsMRe1tfv35pSSpVypuxAEo/HiUajxGIxqa8SQohB7q233uKMM85g+fLlAFxyySXMnz8fv99f4sjE3tbX79/S4lUIIYTYhlKKW265hXnz5pHNZqmtrWXBggWceOKJpQ5NDHKSVAkhhBDbePXVV/nWt74FwGc/+1kWLFiwXY9CIXojSZUQQgixjaOOOorLL7+cIUOGcPHFF6PrJd0oL/YhUlO1F0lNlRBCDD6WZfE///M/nHfeecWWPkJsS2qqhBBCiF149913+epXv8rSpUt55plnePbZZ2VmSuw2+ZMjhBDigKOU4g9/+ANHHnkkS5cupbKyknnz5klCJT4SmakSByylFF3pPDnbxefRKQ960TSt1GEJIfawzs5OvvnNb/LAAw8AMHPmTP74xz8yfPjwEkcm9nWSVIkDUks8y4pNcTZ1pbEcF9PQGVYe5PBhEWoj0oNGiP3Vu+++y/HHH09jYyMej4ef/OQnfO9738MwjFKHJvYDklSJA05LPMszq1qJZSxqw378XoNs3mFNa4K2ZI7jDqmRxEqI/VRDQwMVFRWYpsk999zDUUcdVeqQxH5EkipxQFFKsWJTnFjGYlRVqLjcF/J5GGWGWNeeYsWmODPDPlkKFGI/sWHDBoYMGYLH48Hn8/G3v/2NqqoqwuFwqUMT+xmpyBMHlK50nk1daWrD/u2SJk3TqA372dSVpiudL1GEQoiBdM8993D44Yfzs5/9rPjcqFGjJKESe4QkVeKAkrNdLMfF7+29fsLvNbAcl5zt7uXIhBADKR6Pc/bZZ3PmmWcSj8d56qmncByn1GGJ/ZwkVeKA4vPomIZONt/7P67ZvINp6Pg88ldDiH3Vyy+/zJFHHsmf/vQndF3n2muv5cknn5RidLHHyXcOcUApD3oZVh6kJZHlw4cJKKVoSWQZVh6kPOgtUYRCiN3lOA7z58/n2GOP5f3336ehoYHnnnuOa665Bo9HSojFnidJlTigaJrG4cMiRAMm69pTpHI2jqtI5WzWtaeIBk0OHxaRInUh9kFr167lxz/+MbZtc/rpp7Ns2TKOPfbYUoclDiCSuosDTm3Ez3GH1BT7VLWlcpiGzpiasPSpEmIfNnbsWH73u9+h6zrnnHOO/HAk9jo5UHkvkgOVBxfpqC7Evi2VSvHd736Xs88+m4997GOlDkfsx+RAZSF2QdM0KkJmqcMQQuyG119/nTPOOINVq1axaNEi3nnnHUxT/j6L0pKaKiGEEPsM13X59a9/zfTp01m1ahVDhw7l97//vSRUYlCQmSohhBD7hKamJs455xz++c9/AnDyySdz2223UVVVVeLIhCiQpEoIIcSgt27dOo4++mhaW1sJBAJcd911nH/++VIHKQYVSaqEEEIMeg0NDRxzzDGsX7+ee++9lwkTJpQ6JCG2I0mVEEKIQemtt95i+PDhRKNRNE3jj3/8I36/H79f2p6IwUkK1YUQQgwqSiluuukmpk2bxre//e3i8+Xl5ZJQiUFNZqqEEEIMGm1tbZx77rk8/PDDxa+z2awkU2KfIDNVQgghBoUnn3ySSZMm8fDDD2OaJtdffz0LFy6UhErsM2SmSgghRElZlsUPfvADfvnLXwJw6KGHcu+99zJ58uQSRyZE/8hMlRBCiJKKx+P86U9/AuCb3/wmr776qiRUYp8kM1VCCCH2OqVUscdUdXU1d911F4lEgpNPPrm0gQnxEchMlRBCiL2qs7OT008/nbvuuqv43Gc+8xlJqMQ+T5IqIYQQe82//vUvJk+ezIMPPsh3vvMdkslkqUMSYsBIUiWEEGKPs22bq6++muOOO44NGzYwduxYHn/8ccrKykodmhADRmqqhBBC7FFr167lzDPPZMmSJQB8/etf5ze/+Y0kVGK/I0mVEEKIPaa1tZUjjzySWCxGNBrllltu4Stf+UqpwxL7GaUUXek8OdvF59EpD3pLcth2SZf/rr32WjRN6/EYP3588Xo2m2Xu3LlUVVVRVlbGqaeeSnNzc497NDY2MmfOHILBILW1tVx22WXYtt1jzDPPPMOUKVPw+XyMHTuWBQsWbBfL7373O0aNGoXf72f69Om8/PLLPa73JRYhhBA91dTUcN5553HssceybNkySajEgGuJZ3l6ZSuPvLmZhcs388ibm3l6ZSst8exej6XkNVWHHXYYW7ZsKT7+/e9/F69deumlPPzwwzz44IM8++yzbN68mVNOOaV43XEc5syZg2VZvPDCC9x5550sWLCAq6++ujhm7dq1zJkzh5kzZ7Js2TIuueQSzjvvPJ544onimPvvv5958+ZxzTXX8NprrzF58mRmz55NS0tLn2MRBxalFJ0pi6ZYls6UhVKq1CEJMWi89NJLrF27tvj1z372M5555hlGjRpVuqDEfqklnuWZVa2saU0Q8XsZXh4k4veypjXBM6tKkFipErrmmmvU5MmTe73W1dWlvF6vevDBB4vPvfPOOwpQS5YsUUop9eijjypd11VTU1NxzM0336wikYjK5XJKKaUuv/xyddhhh/W49+mnn65mz55d/Proo49Wc+fOLX7tOI4aOnSomj9/fp9j6YtYLKYAFYvF+vwaMfg0xzJq8dvN6o8vrFW3/WuN+uMLa9Xit5tVcyxT6tCEKCnbttVPfvITZRiGmjFjhsrn86UOSezHXNdVi99uVr9/9j31zxVb1KK3moqPf67Yon7/7Htq8dvNynXdj/xeff3+XfKZqtWrVzN06FAOOuggzjzzTBobGwFYunQp+XyeWbNmFceOHz+ekSNHFosdlyxZwsSJE6mrqyuOmT17NvF4nLfeeqs4Ztt7dI/pvodlWSxdurTHGF3XmTVrVnFMX2LpTS6XIx6P93iIfdug+6lIiEFiw4YNfPrTn+YHP/gBjuMwcuRIsln5+yD2nK50nk1daWrD/u3qpzRNozbsZ1NXmq50fq/FVNKkavr06SxYsIDHH3+cm2++mbVr1/KJT3yCRCJBU1MTpmlSXl7e4zV1dXU0NTUB0NTU1COh6r7efW1nY+LxOJlMhra2NhzH6XXMtvfYVSy9mT9/PtFotPgYMWJE3z4YMSgppVixKU4sYzGqKkTI58HQNUI+D6OqQsQyFis2xWUpUBxw/vKXvzB58mSee+45QqEQCxYs4N5775XdfWKPytkuluPi9xq9Xvd7DSzHJWe7ey2mku7+O/HEE4u/njRpEtOnT6ehoYEHHniAQCBQwsgGxlVXXcW8efOKX8fjcUms9mH9+amoImSWKEoh9p50Os0ll1zC//3f/wFw1FFHcc899zB27NgSRyYOBD6PjmnoZPMOId/26Uw272AaOj7P3ps/Kvny37bKy8s5+OCDee+996ivr8eyLLq6unqMaW5upr6+HoD6+vrtduB1f72rMZFIhEAgQHV1NYZh9Dpm23vsKpbe+Hw+IpFIj4fYdw3Gn4qEKCXDMHjllVfQNI2rrrqK559/XhIqsdeUB70MKw/Skshut0KglKIlkWVYeZDyoHevxTSokqpkMsmaNWsYMmQIU6dOxev1snjx4uL1VatW0djYyIwZMwCYMWMGy5cv77FLb9GiRUQiESZMmFAcs+09usd038M0TaZOndpjjOu6LF68uDimL7GI/d+2PxX1phQ/FQmxt7mui+MU/g74fD7uvfdeFi9ezM9+9jO83r33zUsITdM4fFiEaMBkXXuKVM7GcRWpnM269hTRoMnhwyJ7tV+VpkpYAPK9732Pz3/+8zQ0NLB582auueYali1bxttvv01NTQ0XXnghjz76KAsWLCASiXDRRRcB8MILLwCFlgpHHHEEQ4cO5Re/+AVNTU2cddZZnHfeefzsZz8DCi0VDj/8cObOncu5557LU089xcUXX8zChQuZPXs2UGipcM4553Drrbdy9NFHc/311/PAAw+wcuXKYq3VrmLpi3g8TjQaJRaLyazVPkgpxdMrC0Xqo6pCPf6iKqVY155iTE2YmeNrStJ0Tog9bcuWLZxzzjl87GMf49prry11OEIAhQ1EKzbF2dSVxnJcTENnWHmQw4dFqI34B+Q9+vz9+yPvM/wITj/9dDVkyBBlmqYaNmyYOv3009V7771XvJ7JZNS3vvUtVVFRoYLBoPrSl76ktmzZ0uMe69atUyeeeKIKBAKqurpaffe7391uG+/TTz+tjjjiCGWapjrooIPUHXfcsV0sv/3tb9XIkSOVaZrq6KOPVi+++GKP632JZVekpcK+rzmWUfe/3Kh+/+x76m+vbVSPL9+i/vbaRvX7Z99T97/SKG0VxH7r4YcfVtXV1QpQZWVlqrW1tdQhCVHkuq7qSObUlq6M6kjmBqSNwrb6+v27pDNVBxqZqdo/7I2fioQYLLLZLJdddhk33ngjAJMnT+bee+/l0EMPLXFkQuw9ff3+vVu7/7q6uvjzn//MmjVruOyyy6isrOS1116jrq6OYcOG7XbQQuwLaiN+ZoZ9g+KcKSH2pBUrVnDGGWewYsUKoHCyxPz58/H5fCWOTIjBqd9J1ZtvvsmsWbOIRqOsW7eO//qv/6KyspKHHnqIxsZG/vjHP+6JOIUYVDRNk7YJYr+WTCb51Kc+RUdHB7W1tdx555189rOfLXVYQgxq/d6mNG/ePL72ta+xevVq/P4PljpOOukknnvuuQENTgghRGmUlZUxf/58TjzxRN58801JqITog37XVEWjUV577TXGjBlDOBzmjTfe4KCDDmL9+vUccsghcizBTkhNlRBiMHvyyScJBoN87GMfAyj2/pGl7T1LKSXlBIPcHqup8vl8vZ5h9+6771JTU9Pf2wkhhCgxy7L47//+b371q1/R0NDAsmXLKC8vl2/se4FsfNm/9Hv57wtf+AI//vGPyecLBxRqmkZjYyNXXHEFp5566oAHKIQQYs959913mTFjBr/61a+AwvFhpin1gnuDHNDef0opOlMWTbEsnSlr0J212u/lv1gsxmmnncarr75KIpFg6NChNDU1MWPGDB599FFCodCeinWfJ8t/QojBQinFHXfcwUUXXUQ6naayspLbb7+dk08+udShHRCUNBPut1LO6u2x5b9oNMqiRYt4/vnneeONN0gmk0yZMoVZs2Z9pICFEELsHdlslrPPPpsHH3wQgJkzZ/KnP/1JWuLsRXJAe/90z+rFMha1YT9+r0E277CmNUFbMsdxh9QMiuXS3epTBXDsscdy7LHHDmQsQggh9gKfz0c+n8fj8fCTn/yE733vexhG7weFiz2jLwe0t6VyckA7hZm7FZvixDJWj1m9kM/DKDPEuvYUKzbFmRn2lXxWr981VRdffDG/+c1vtnv+xhtv5JJLLhmImIQQQgww27ZJpVJAYSbktttu44UXXuCKK66QhKoE5ID2vuvPrF6p9fv/1l/+8pdeZ6g+9rGP8ec//3lAghJCCDFw3n//fT7xiU9w3nnnFQt7q6qqOOqoo0oc2YGrPOhlWHmQlkR2u2JrpRQtiSzDyoOUB70linDw6MusnuW4g2JWr99JVXt7O9FodLvnI5EIbW1tAxKUEEKIgXH33XdzxBFH8OKLL/LYY4/R2NhY6pAEhRmWw4dFiAZM1rWnSOVsHFeRytmsa08RDZocPixS8uWswWBfmtXrdwRjx47l8ccf3+75xx57jIMOOmhAghJCCPHRxONxzjrrLP7zP/+TRCLBscceyxtvvEFDQ0OpQxNb1Ub8HHdIDWNqwsSzeTZ2pYln84ypCXPcwYOj8How2Jdm9fpdqD5v3jy+/e1v09rayqc//WkAFi9ezP/+7/9y/fXXD3R8Qggh+unFF1/kq1/9KmvXrkXXda655hq+//3v4/Hs9t4ksYfIAe271j2r15bMsa491WP3X0siO6hm9frdpwrg5ptv5qc//SmbN28GYNSoUVx77bWcffbZAx7g/kT6VAkh9jTLshg3bhyNjY00NDRw9913y05tsV/YF/pU7VZS1a21tZVAIEBZWdnu3uKAIkmVEGJveOqpp7j99tu56aabeq2BFWJfVapzEvdKUiX6R5IqIcSe8Oc//xnbtvnKV75S6lCE2C8NaEf1KVOmsHjxYioqKjjyyCN3mhW+9tpr/Y9WCCFEv6VSKS655BJuu+02ysrKmD59OqNHjy51WHtEqWYohOiPPiVVX/ziF/H5fAByLpQQQgwCr732GmeccQbvvvsumqZx8cUXM3z48FKHtUeUspZGiP7o1/Kf4zg8//zzTJo0ifLy8j0Y1v5Jlv+EEB+V67pcd911XHXVVeTzeYYNG8Zdd93FcccdV+rQ9ogdnfnWksgSDZiD5sw3sX/r6/fvfvWpMgyDE044gc7Ozo8coBBCiP5xHIc5c+bwve99j3w+z5e+9CXeeOON/Tah+vCZbyGfB0PXCme+VYWIZSxWbIpv17tIiFLpd/PPww8/nPfff39PxCKEEGInDMNg0qRJBAIBbr31Vv7yl79QVVVV6rD2mH3pzDchYDeSqu4TzR955BG2bNlCPB7v8RBCCDFwstksW7ZsKX79P//zPyxbtozzzz9/vy/U3pfOfBMCdqOj+kknnQTAF77whR5/oZVSaJqG4/R+No8QQoj+WbFiBWeccQahUIh//etfeL1eTNPk4IMPLnVoe8W2Z76FfNt/uxpMZ77tjOxcPHD0O6l6+umn90QcQgghtlJKcdNNN/Hd736XXC5HbW0ta9asYfz48aUOba/qPvNtTWuCUWZoux/kWxJZxtSEB8WZbzsiOxcPLP1Oqj71qU/tiTiEEEJQOKni3HPP5ZFHHgHgxBNP5I477qCurg44sGY99qUz33qzo52La1oTtCVzsnNxP7Rbp2t2dnZy++2388477wAwYcIEvv71r1NZWTmgwQkhxIFk0aJFnH322TQ1NWGaJr/85S+56KKLiknDQM567CvJWW3Ez3GH1BR/322pHKahM6YmPKhnez68c7H7sw35PIwyQ6xrT7FiU5yZYd+g/NzF7un3MTXPPfccn//854lGo0ybNg2ApUuX0tXVxcMPP8wnP/nJPRLo/kD6VAkhdsR1XaZPn86rr77KhAkTuPfee5k0aVLx+kD2a9oXl6T2lSSwW2fK4pE3NxPxe3utB0vlbOLZPJ+bNJSKkFmCCEV/DOgxNduaO3cup59+OjfffDOGUdiR4TgO3/rWt5g7dy7Lly/f/aiFEOIApes6d911FzfddBPz588nGAwWrw3krMfuLEkNhoRG07R9Kvnoy87FtlROdi7uZ/qdVL333nv8+c9/LiZUUOidMm/ePP74xz8OaHBCCLG/Ukrxhz/8gebmZr7//e8DcMghh3DDDTdsN7Y//Zp2lnjsTnK2L85qDQb7y85F0T/9/r85ZcqUYi3Vtt555x0mT548IEEJIcT+rLOzky9/+cucd955/PCHP9zlQfTdsx4+r04yZ9OZtkjmbBSF6o2+9mvqbzPN7lmtNa0JIn4vw8uDRPxe1rQmeGZVKy3x7Ef4FPZv3TsXWxLZ7Tq+d+9cHFYeHNQ7F0X/9Xum6uKLL+Y73/kO7733HscccwwAL774Ir/73e/4+c9/zptvvlkcu209gBBCiEJd6n/+53+yYcMGPB4PP/vZzzjiiCN2+hqfRyeXd3ltfRcpy8Z2XTy6TlXIR0NVAI+u92nWoz9LUlJo/dHs6zsXxe7pd1J1xhlnAHD55Zf3ek3TNGkEKoQQH5LP5/nxj3/MT3/6U5RSjB07lnvvvbe44WdnLNuhNZGjsSPN6KogptdL3nbZEksTS1uUh0wmDy/f5axHf5akBmrJsRQGQw0Y7Ls7F8Xu63dStXbt2j0RhxBC7LeUUpx44oksXrwYgHPPPZcbbriBsrKyPr32rc0Jwn4PwysCxHI2UU3DNHSifi9r29PousZhQ8O7TBz600yzOZ7bJwutB1sNWG3Ez8ywb1AkeWLP63dNVUNDQ58f/fHzn/8cTdO45JJLis9ls1nmzp1LVVUVZWVlnHrqqTQ3N/d4XWNjI3PmzCEYDFJbW8tll12Gbds9xjzzzDNMmTIFn8/H2LFjWbBgwXbv/7vf/Y5Ro0bh9/uZPn06L7/8co/rfYlFCCF6o2kaZ5xxBtFolPvvv5/bb7+9TwkVfFAHNaamjInDotRH/GTyDu2pHBnbZUxNiJqwienpPfn5cByHD4sQDZisa0+Rytk4riKVs1nXnuqxJLXtrFZvdlRorZSiM2XRFMvSmbK2qyfakwZrDVj3zsX6qJ+KkCkJ1X5sUGw7eOWVV7j11lu3q8G69NJLefjhh3nwwQd59tln2bx5M6ecckrxuuM4zJkzB8uyeOGFF7jzzjtZsGABV199dXHM2rVrmTNnDjNnzmTZsmVccsklnHfeeTzxxBPFMffffz/z5s3jmmuu4bXXXmPy5MnMnj2blpaWPscihNj/9SdhiMfjrFixovj1ueeey7vvvsuXv/zlfr3ntnVQ5UGTScOiHNVQybRRlRzVUMmUkRX4vEafZ4y6l6TG1ISJZ/Ns7EoTz+YZUxPmuIM/aKewO4XWLfEsT69s5ZE3N7Nw+WYeeXMzT6/cO8nMh2vAQj4Phq4VasCqQsQyFis2xfdqkicOPP1u/jnQkskkU6ZM4aabbuInP/kJRxxxBNdffz2xWIyamhruueceTjvtNABWrlzJoYceypIlSzjmmGN47LHH+NznPsfmzZuLRzjccsstXHHFFbS2tmKaJldccQULFy7s8Y/bV77yFbq6unj88ccBmD59OkcddRQ33ngjUGjCN2LECC666CKuvPLKPsXSF9L8U4h9V3+WlV588UW++tWvYts2b7zxBhUVFbv9vnuqiWRf6o522nA0aPZIwgayOenukGabYk/q6/fvks9UzZ07lzlz5jBr1qwezy9dupR8Pt/j+fHjxzNy5EiWLFkCwJIlS5g4cWIxoQKYPXs28Xict956qzjmw/eePXt28R6WZbF06dIeY3RdZ9asWcUxfYmlN7lcjng83uMhhNj39HVZyXEcfvKTn/Dxj3+ctWvXYhgGmzZt+kjv3ZcZo6HlAZRS/Vpy68uSVF9ntQbDLNGHdzYqVI/2Ez6v3qe2E0J8FLt19t9Aue+++3jttdd45ZVXtrvWffZVeXl5j+fr6upoamoqjtk2oeq+3n1tZ2Pi8TiZTIbOzk4cx+l1zMqVK/scS2/mz5/Pj370ox1eF0IMfn1tLTC2q4WzzjqL5557DijMiN9yyy1Eo9GP9P672pqvaxqxdJ6Fy7fskcLsvhRaD4adgtvWgNmuy/r2DO2pXLH9RMj0UB70SrNNsUft1p+urq4ubrvtNq666io6OjoAeO211/r1E9mGDRv4zne+w913343fv39uK73qqquIxWLFx4YNG0odkhCin/qSMPz1oT8zafJknnvuOcrKyrjzzju55557PnJC1W1HM0bVZT6UgvZUbo8WZu9qVqsv/a/29CxR94ze+20p3twQY0ssTdBrUBXyEfDorGlN0prIYdnS6kfsOf2eqXrzzTeZNWsW0WiUdevW8V//9V9UVlby0EMP0djY2OejapYuXUpLSwtTpkwpPuc4Ds899xw33ngjTzzxBJZl0dXV1WOGqLm5mfr6egDq6+u326XXvSNv2zEf3qXX3NxMJBIhEAhgGAaGYfQ6Ztt77CqW3vh8Pnw+X58+DyHE4LSrhMHn0Vmy6GFiXV0cddRR3HPPPYwdO3bA4/jwjJFpaLze2EVbMrfdDFqDGWTllgT/Wt3GzENq9viOs8FwJIumFdpK/Gt1Kxu7MoyuCuI1dCzbJZazGV4RIOz38NbmBLWR7RNkIQZCv/+Ez5s3j6997WusXr26xwzTSSedVJz27ovPfOYzLF++nGXLlhUf06ZN48wzzyz+2uv1Fvu6AKxatYrGxkZmzJgBwIwZM1i+fHmPXXqLFi0iEokwYcKE4pht79E9pvsepmkyderUHmNc12Xx4sXFMVOnTt1lLEKI/dOOWgt01wflbJdvXDWfH1zzY55//vk9klB123bGSNM0Nscy282gFeqXEjR2ZnjynSYeXLphj+/AGyxHspgeg5qwyZiaEBnbLbSdyDvUR/xMHBZlTE1Zj2N4hBho/Z6p6m5/8GHDhg3baX3Rh4XDYQ4//PAez4VCIaqqqorPf+Mb32DevHlUVlYSiUS46KKLmDFjRnG33QknnMCECRM466yz+MUvfkFTUxM/+MEPmDt3bnGG6Jvf/CY33ngjl19+Oeeeey5PPfUUDzzwAAsXLiy+77x58zjnnHOYNm0aRx99NNdffz2pVIqvf/3rAESj0V3GIoTYP23bMLPBDJLM2vz1j7ey4b13uOr//ZaWRJZDRg7lmyf8YK/OfvQ2gxbLWCzfGCdl5Yn6vWgK/B6DNa0J2pK5PbYDb7AcyZKzXXxegykjy8jmXfKui1fXCfmMwikfrhqUDUvF/qPfSZXP5+t1F9u7775LTU3NgATV7brrrkPXdU499VRyuRyzZ8/mpptuKl43DINHHnmECy+8kBkzZhAKhTjnnHP48Y9/XBwzevRoFi5cyKWXXsoNN9zA8OHDue2225g9e3ZxzOmnn05raytXX301TU1NHHHEETz++OM9itd3FYsQYv/UnTCsbUvytxfe5tEbf8D6N18EoHbKCcw+4fg+JQwDfXTKh5fcFIr17RlSVp66sJ+s7WJ6daIBk5DP2ONn9Q2GI1m6P5Oc7VLmL80ypDiw9btP1XnnnUd7ezsPPPAAlZWVvPnmmxiGwcknn8wnP/lJrr/++j0U6r5P+lQJsW9qiWf5n5vu4vafXU4m0YnH9DPznO9xxPGncnBdhC8cMXSnSUN/elz1NflSSvH0ykKbh1FVIVKWwyvrOgh6DUyPTksyR33Ez6RhUTRN22t9mkp57t6HP5MPH8Ozrj3FmJowM8fXSE2V6Je+fv/u90zV//7v/3LaaadRW1tLJpPhU5/6FE1NTcyYMYOf/vSnHyloIYQYbNLpNBdceBF/u+cPAIw6eAIX/+S3jB5zCEFTZ31HeqczQDtqitnbklx/kq8PL7mZHh3LcfBtTahCXg8NlcFiTHvrrL7uuq9SGCzLkOLA1e+kKhqNsmjRIp5//nneeOONYkf0DzfYFEKI/cEXTv4Sixf9E4BTz76Acy/9Pqb5wa7enfVg6muPq5lhH62JXJ+Tr+J7b7PktrolQTJjgwv15QEaKoOUBz+I50BZ+hoMy5DiwNXvpOqPf/wjp59+OsceeyzHHnts8XnLsrjvvvs4++yzBzRAIYQopQsvmserry3jsp9ez/RPfHq76zubAeprU8zOlNXn5OvD9+lutXDEiCjVIR8bu9IcWh9G1z9Inrp34I2pCe/xHXiDQV8algqxJ/T7R5avf/3rxGKx7Z5PJBLF3XJCCLG39eew451pbW3lySefLH796U/P5H///CyHH/3JXsfvbAaor00xWxNWnzuS90bTNCrLfHzi4GqGRAOs70iTytk4riKVs1nXnjrglr76cgyPEAOt3zNVSqle/3Bu3LhxwLoHCyFEf/SnFmlnFi1axNlnn00ikWDZsmWMHTuW8qCX0XWVheJnc/vi55ZEloNqyopn73XPigBkLJus5dCezFHTyyxTd0IGapfJV1/qofq69FXKYnIh9md9TqqOPPJINE1D0zQ+85nP4PF88FLHcVi7di2f/exn90iQQgixI/0pBN8Ry7L47//+b371q18BMGHCBCzLKl4fXhFgTWuSt7fEaagMEjA9Oz17L2R60DRIZm02dGZ4c1OMg2vDNFR9UOe07ZJcTdg3YB3Jd7X0NVAJqBBie31Oqk4++WQAli1bxuzZsykrKyteM02TUaNGceqppw54gEIIsSP9KQTf0UzMqlWrOOOMM3j99dcB+Na3vsWvfvUrAoFAjwQkbdm0JnK0JnJUh02qQj6qy3y0Jy3aU7liQtccz/LUymYUGsccVMGUkeW83tjF21vitCVzHDmiAp9X77EbrSJkFhuM7mg2rD/1UDvagTcQCagQYsf6nFRdc801AIwaNYqvfOUrcqadEKLk+loI3tvOPIDbb7+diy++mHQ6TVVVFX/4wx/4whe+AHyQgHSlc5T5vNRF/FSXmbQlcgRNL0ePqmRjZ6bH2XtKKVqTOQKmUfh1Is/E4QGmjapgXVua1S0JXtvQwYQhke2W5HprBZCxbNZ3pAn6PAyvCHykz2ogElAhxM71u6ZqwoQJLFu2jOnTp/d4/qWXXsIwDKZNmzZgwQkhxM70pRB8Z7VIa9asIZ1O85nPfIY//vGPDB06FPggAdnYmUYpWN+RwXZcPIZOZdAkkc2zqjlBV7ow4wOFpb6ujMWWrgzlgcJyW3sqRyrnEA2YTBrhZUi5n850nk+Mq2FUdc8Zqe56qOWbYrzXkqQ1kSOWyePz6NQAL61tZ2NnZreX6T5qAiqE2LV+7/6bO3cuGzZs2O75TZs2MXfu3AEJSggh+mJHhx13660WKZ//YAfdj370I/7whz/wz3/+s5hQQSEBWdkUozWRoymRJeA1qAr5CHgNmhJZWhM53mlKEMvmyeVd3twU45X1HSxd38n7bSk2x7PkHYXtuuSdQkKnoVEV8uH36gTMws+zve5WVJDM5lnXnqQ9lSPgNRgS9RPxe1nTmuCZVbt3OHJfdyLKuXhC7L5+z1S9/fbbTJkyZbvnjzzySN5+++0BCUoIIXame/daNu8QDXhpimcZvYtapHw+z49+9COefPJJnnvuOUzTxOv19toKJpt3aOxIYzuKusgHMzt+3cDn0WmOZ2nqylAXMXm9M4utFFG/F5+h057M0Z7MkczaVJeZeA29x31NQyeeyfP25p7F4iHTQ2fawnELiU15wKQ84KUrY/H25gQTh0cYVbX7y3QfPiuwt9/zgdAcVIg9abcOVG5ubuaggw7q8fyWLVt67AgUQog94cO713J5h9aERSrncFB1qNdjSdauXctXv/pVXnrpJQD+8Y9/cNppp+3wPbJ5h2TOobKXVgOaphE0DTpSFum8S0six/i6MjRdRylFRchHVypHZ8oiGvASMgtJSneSV13m4/XGTuLZfI/aqSXvt5PMORw9qpyU5VAeNPF7jULxeyLL+vYME4d7d3uZrjzoHdBieCHE9vr9I8kJJ5zAVVdd1aMBaFdXF9///vc5/vjjBzQ4IYTYVnfx+JrWBBG/l+HlQYaVBwn7PcQzeTZ1ZdjYlSaezTOmJsxxB9fwz3/8mSOOOIKXXnqJaDTK/fffv9OESilFLu+gAx1pC6XcDw8gZdl4DZ2g16A27KMlmSOXd1BAecBD1nZRFPpUbYllSebyhQacAS9KQTybZ1RViJDPg6FrgIah63h0WN+RJb919goATaM8YBbrs3Z3ma77XLxowGRde+qAbw4qxJ7Q76mlX/3qV3zyk5+koaGBI488Eii0Wairq+NPf/rTgAcohNh/9acJ5c52r00cFmVtW5L6SJCjRlfg9xrodoa53zqPu+++G4CPf/zj3HXXXTQ0NOwwnu5ZsNUtcZI5m3gmT1c6z8jKUGEJ0XbpyliYHoPygJegz+CgmjI2dmZpT+Wwc3nytqIyVChm39iV4elVrQwrDzC1oYLDhkZ5aW37dsXiedfFcV2qy3zEMoX+WJbj4tcL9U9ej164t+OiwW4v08m5eELsWf1OqoYNG8abb77J3XffzRtvvEEgEODrX/86Z5xxBl6vTBsLIfqmv00od7V7rS4SIJa18HsNyoNeTj7lDP7xt4cwDIOrr76a73//+zstUdi2h1Nd2M+EIVHea42TyDq835qkNuIj7PdSHwmg6xpjakJ0pS18HoOJwyOkcoXO6atbkpSHvNRHfKQsh3F1YSzbLTyc3ovFvbqOx9BRgK5Dmc9LLFvY+adpGnnbxaPreHTtIy/Tybl4Quw5u1UEFQqFOP/88wc6FiHEAaLYAypjUebzEDI8uErx3jZNKGs+9I0/m3f61D5hU1eG1xu7+MRXL2Lpm29x7hU/5eOf+AQdaZvaiKfX2TFgu1mwUdVBkjmbsD9PMmtTGTIZV1tWrHeaPrqStzYnijVKIdNgTSaPoxS1ocKS4PCKEGNry0DBuvYU77cm8eradsXiIZ9BZciksSNNyDRoqAqxtjVFSzJH1OehM2NRFfLTmsxRPgDLdDtqDiqE+Gj6lFT94x//4MQTT8Tr9fKPf/xjp2O7G+cJIURvupfxNnVlcF3F+vY0tluYiakMmqRyDv9e3UY06GVzV6Y4ixUNeMnlnR3uXtvQ2MgzzyxG/8JXsF2XcWPH8X9/fQrLUcWO4YcPi9AUy203Oza8IrDdLFg06GV0TYj17SlytktzLMuQaIBxtR8slWmaRlsyx9r2FDrQ2JHC5zFoSeYI+bw0VAXQ0ECD2rCfWDpPecBHcyLTo1hc0zRGVgRZ25rEdg2iAQ8ThkR4rzXB+o40XqOQ/I2VZTohBrU+JVUnn3wyTU1N1NbWFo+r6Y2maThO7/1ihBACuntAxWmJZ7HdQusAr6dQr9QUz5C3Faua4kwYEqGh6oPdfE3xLK0Ji2zeZeKwaI+ZmmefeJhfX/1dMqkEdUNH8NnjZxWvez0wygyxfFOMd7bEGVLup+5DR7SsaU2StuxiI89YxmJ9e4b2VI684+DRNfDoTBwWYcaY6uK9ayN+Dh8W4Ym3mlixOc6GjhQRv5ch0QAThgSIBj6YDeqeSTuoNkjWdnp0Ts/mHTrTOQ6uDxP2eWmOZzF0jeEVAQ4fWs5BtYWCfFmmE2Jw61NS5bpur78WQoj+ylg2q5sT5ByXIVE/Pq8OaPi8BrWGj9c3dpK2XGYeUluckQr5PIw2Q6RyDrG0xVubY1QETQzX4s7/vZYn/34fAA3jJzFu9KheE49s3mFLLMOk4dEe9x1lhnh7S5zWRI6GqsKOuOUb46SsfCHhC3hJZvM0J3KsakoytjZcnCnqrgsLmQYzRldS5jPwGYUjata3Z4gEvMXEqrsP1LDyILVhf49i8Vze2bocaeAqBWhUBH1MHBZlXF2ZJFJC7COksZQQYq9piWd5dlUbq1uS+L06qZxN2O+lNuwjaHrIOgrHAQ2wP/QDnKZpVJd52diZJpWz+feSl3ny5h8Qa2pE0zTO+/alTD3lfEZVR1FKkco5WI5D3lZYjkt70sLv1bFdtd19GyqDtCZyhaVIR5Gy8tSF/bD1PL+M7XJwbRjLtnlxTQdHja7A59FZvjFGLGMxurpwwHwsa9MUz1JbVqip6u4thaJHgbmmacVi8U1dGV5Z24HHcHvMoLUksizb0EV50Lvdcl9/dk0KIfaePiVVv/nNb/p8w4svvni3gxFC7L+6i9M3xVKU+Qw0NLy6Rmu8cExLQ1UIrwGZvEMk4MFr9CxIj2UsVjeniGXyZN94lL//9qfYdp5odR0X/88N/OfJJ/HS2naa41lakzk2dWZoTWTJbu3nlMzZ1JSZpC2HimDP2AKmh+pwYUbp3eY40YBJPJsnk3dJWzYRv5eo30NjZ5pX1nexsjlGxO9lY2eGg+vCxYSmoTJIImPTkrIIeA1ak1laEz5SOXu7PlCaplEe9PJ6Yxe26zK6j4cc93fXpBBi7+lTUnXdddf1+Lq1tZV0Ok15eTlQaP4ZDAapra2VpEoIsZ1te0yNqQmzuatQLL4plkW5ikzeoSOVI+z3ogHlARNzmz5MisJyWixTOMA4EQlj23k+PmsOl1z7CzocHxs60wS9Bk+vagENYpnCUl7I9JDKWSTSeQIenfdbk4RMg/LgB/VO2bxDVchHZcjk1XUdNMXixLc2xyzzeSgPenlrc7ywNKcVkp6YmWdDZxp9a4f18qBZ3Jm3viNNe8KiLZWjM53n0PpIr0lPfw853rbtQ+2H6sK6d01KYiVE6fQpqVq7dm3x1/fccw833XQTt99+O4cccggAq1at4r/+67+44IIL9kyUQoh92rbJQ9A0qAh6ea/FRbkuGjqmRyeXdwn5XNA0dF0naH6QVKVyDhubW8AboqrM5LjTzmD48OEcecwnSFsuZsbiveYEQdPAVRBL57BdjcqgF8eFnKETMA0MXSdt2azvSBMNFJbMlFI0J7IEPAarmxM4rsLj0aj0mPg9BplcnrVtKRy30C29ImQS8XuIZQqHKbckMqzv8BbvVx40iQa8tCZydKb9nHR4PaOqQ70uz/XlkOO2VI6c7e60+emOZrWEEHtXv2uqfvjDH/LnP/+5mFABHHLIIVx33XWcdtppnHnmmQMaoBBi8OlvTU+P5EED0DA9GmWmien1gFIkcoWz8CzbJeDVWdeepi7sR3Msfjv/al565p9ceP1DNFRWo+s6Y46YwfLNcTpSFlbepSNtEQ14GFMTZPkmB4VDwrIxNI2qMj9Don42dxUOOm5SGUZVhfDoGu+3pYin88SyebrSueJxMAdVBwmaBl1pi4zlEPZ5yNkuroKo30PE76EjZdGVtmlPFl5T5v/gn9RkLs+IisLuxa50vtfPqD+HHPd3VksIsff1O6nasmULtm1v97zjODQ3Nw9IUEKIwWt3anq2TR4UkLUdDqkL05WxSWbzZG0XTdOoCvmoi/jI5h3KTIOXXlvGgp9cypZ1qwGIr36J8qlj6UpbLN8UoyudJ+QzMD0afo9OeyqHrmmE/R5qwiGUAkPX8Ht1XAW2C5Uhk+ZYtthTKp7J4/FoWLaDq1SxjmrF5jhB00PGsrEdl860RTToxXUVWVvh9xqMrAyxpiXBps4MsTqLgNm9HJckkbOxXWhdkd3hZ9SfQ46b47k+z2oJIUqj30nVZz7zGS644AJuu+02pkyZAsDSpUu58MILmTVr1oAHKIQYPHa3pmfb5CEa8GK7LlUhH+VBk2zeYV1HGkPTsF3FyqYkm7rStL/8D1649wacvEV5VQ3fvubXjJw8A9d1eXtLnPdakxgatCchlXeoLfNREfCSshzSlk1d2Id/m9mfvO0Q9nsZV1vGkGiATx1cw7tNSTw66JrGsg2dKBe8hkZF0KQ1maMjVTiHz6Np6B4Nr24Qz+ZJ5PLF43AqQmZh56LtsLErTS7vkMjaRAJehpUHdvoZdR9y3JbMbde3qiWR7VHc3p9ZLSFEafQ7qfrDH/7AOeecw7Rp04pn/dm2zezZs7ntttsGPEAhxEczUNvv+1PTA4U6quzWDug+j07Eb6CgUJ9ku8TSFgpoT1pkcoVjYCzHZU3jJl6686d0rnoJgBGTj+XzF/0PVSOHoaHxyvpO3mjsxFWKgGlgOYoy08D06CRyDroGKGhNWgyvMApxKkVXxqI+Eiicx1cbpjbs45V1HdSG/by1OU7GcgGXdM7ZOtujYW6dFDIMHUPXCJk6GVvRmcpTU+bDyheaHX/q4Go+Ma6GnO3yyroOPIbe5918fT3kuD+zWkKI0uh3UlVTU8Ojjz7Ku+++y8qVKwEYP348Bx988IAHJ4T4aD7K9vsPJ2NKqT7V9KxuTrKxM8PKphirm5O0pnJkcw5+0yDs84Cm2NiZJZ1zCPsNsraLroGhQ3NzjqV/uYXOVS+hebyMOvECDp55Kl2OyYpNMY4dW0Uia9GUyOHzaMSyztbCd5PqkEkbFsmcjatcQNEczxI0DVKWjekx0HWtuEPPcgr9qxwFbckcrlKkcw5ej07acjANjbxTOOA477gEvYVZsHDAS9ay6cpYbOnKMbIyyDEHVVFZ5qMzZRHLFHpc9afuqS+HHPdnVksIURq73fxz1KhRKKUYM2bMTk9+F0KUxkfZft9bMhb0Fgqzu49y+TC/12Bte5Jn322lOZ7l3eYETbEMnWmr2C087CvswPN7C4+8o0hk86RzLptjhSNqamZ+DSfRysgTziM05CByjiLvKtLJLM+/1w4ookEvUb+XoOnB0Au7A9d3pBkS9aMUBLw+6qO+whJeOk+Zz8OIiiDjt2lt0JmyMA2dRCZPaypH0Gvg0TTiWRtNA8dVaBrgggJ0XSPk8xL0GnRmLPQOndE1If5j6vDi/TZ0FhKmmq2zdb19Rjuqe+rLIcd9ndUSQpRGv7OhdDrNRRddxJ133gnAu+++y0EHHcRFF13EsGHDuPLKKwc8SCFE/+xsqa7BDLJyS4J/rW5j5iE1VITMHrMb3clYVzpHmc9LyPTguooNnSnWt6epDHkZEg1u954Zy6YtYZH2OrzfmqQ1kSWTLyyjRQMesnmXWNbG0MDQTXRdJ++4xLesp+31J6meeRY5W2H6Q4w766copUhbNpm8Q9hngIJVTXGqwyZDIwEyto3Po6NpGtGATiyTpyNtYdkuxx1Sw6fH15KzXbJ5B7/XKNZAdf9eu5fTXl7bTtZyKfN7MXQN3bC2vq+LxygkQgrFQdVllAc95GxFmeNl1vhaPj2+Fk3TeHpla2EGKmOxqjlJImtzSH1Zj7P/YGDqnvoyqyWEKI1+J1VXXXUVb7zxBs888wyf/exni8/PmjWLa6+9VpIqIQaBHW2/7z4oeHMswztNcdqSWcbVfjB7052MbexMoxSs78hgOy4eQ6ciUNgJ98raLo47xEM44EUr9EconHXXkQYKO/s60xaGrmPZhd15Hl1HU4UddOGAh86MTchrEFv2OG8/+BvcfBazcijG+E+TUwqlbEI+A69ROFYmlrHJ5V0SuTymV8cw8lh5l7ydIxow8eoaXkNnU2eG4RVBJg0vp7Ks99mibt3Lae+1JMg7hR1/VSGz0HU9lydoK+ojPmxX4TE0Zh9ei8/j4f22JONqInxucj1tSavHbGBNmY9ExmZNa5Jc3mHSiGgxsRrIuqe+zGoJIfa+fidVf/vb37j//vs55phjevxjfdhhh7FmzZoBDU4I0Xfb1kDF0nlytoPf+8FyUCxjFQ8Kjvq9aAr8HqPHcqDX0FnZFKM1kcNWiqjfi+n3EstaLNsYI56xydopOjI5GiqCjKkNUxkqNLoM+jyUWYVDi3VNw3JsDF3D0AqzMpoGmq7hOIpcKs7ahb+hdflzAIQPOoKKg6cR3xqro1xy+UKBuKYVdu6lLBsU1Id92KrQ5yqbdzH0PIZeSO90TWPaqErG1ZX16TMrLKfVsrIpwfqOFFviGYKmhxGVQVI5m7RV2GlXHw7guhptyRwjKkMcM6YSTdN6nQ08uC5MznbZ2JnB5zWY0lC+tUmo1D0Jsb/rd1LV2tpKbW3tds+nUin5h0KIEvlwDZTtKDZ0pDENg6HlgeIxL90HBWdtF9OrEw2YhHxGcVfa+PoyGjsKhwrXRQqzXGnLpqkrh+u66LrCdSGbtXl1fSfLGjsZXhni0CFhJg8vJ2vZ5B0Xj1FInjy6hqsUhqah0DA06FzzOuse+iX5eBua4aFu5jnUfuxUbDTIFXbTKcByFF4Ks2Q5R+E1dPBAZchH2nLJ2ja65lLm81JdZtKZtqgqM/n42Ope/y3a0S7IcXVlzJk0hJfebyfvuCSyNrquEQ14sWxFMpfHZ2rYrtujdqkzZfU6G1geNJk4LIrPo7M5liHcXDjmRuqehNj/9TupmjZtGgsXLuSiiy4CKP5jcttttzFjxoyBjU4IsUu9FaRn8jZrWlO8vLadTx1ci3drY8zygIkCYtk89RE/IZ/RY1daZchLMudQWazRKeygS+QsvLpOMuuQsRyOHBEll3dZvjnGGxs6eb8tyWuNnXh0jVjaxm/quEqh6xqWo/BRmH1qffGvbHr894DCWzmUEadcQcXIQ9E1yNkOug6uW3h4PBqOAl0pAoaGaXjweQ1cNEZUBmhLWXQmCx3QI36DSMBkxkG9z1LtahfkxGFR2pMWXekc42q96LqG6xa6vPs8BtNGVTKsPNCjdmlnR8yUB02mjKwg3JrgUwfXMKIiKHVPQhwA+l0t+bOf/Yzvf//7XHjhhdi2zQ033MAJJ5zAHXfcwU9/+tN+3evmm29m0qRJRCIRIpEIM2bM4LHHHitez2azzJ07l6qqKsrKyjj11FO369re2NjInDlzigc6X3bZZdt1fH/mmWeYMmUKPp+PsWPHsmDBgu1i+d3vfseoUaPw+/1Mnz6dl19+ucf1vsQixN724YL0kM+DoWuU+bwcPaochcar6zvoSuewHAdHKVqSOUJeDw2VweLZd7araEvmSOZsQltbEKAUHSmL9e1p4pnCeXntyRwKRVM8y5ub4ySyNo5SJLN53mtJ8taWBO+3JVnXnsJ2IJsv3Ce59XBiX90Y0DSiR5zAQef9lqqR4zF0SFt2oXZJ27pMCOjKxaNrBDyFhGlIeYAxtWWE/R5SeYf6sI9x9WFqynxUhnxMayjn2F5mqbqTzjWtCSJ+L8PLg0T8Xta0JnhmVSst8WxxV93Y2giOUqSswu9rXG2EkyYO4fBh0e0K+rdtxtmbnO1SHjAZURHc7rVCiP1Tv5Oqj3/847zxxhvYts3EiRP55z//SW1tLUuWLGHq1Kn9utfw4cP5+c9/ztKlS3n11Vf59Kc/zRe/+EXeeustAC699FIefvhhHnzwQZ599lk2b97MKaecUny94zjMmTMHy7J44YUXuPPOO1mwYAFXX311cczatWuZM2cOM2fOZNmyZVxyySWcd955PPHEE8Ux999/P/PmzeOaa67htddeY/LkycyePZuWlpbimF3FIkQp7Ow8uOjWZai8o9gSy5HI2CQyhRmqw4dFKA+adKUt3twU48U1bbzblOCVdZ3YrsJxYF1HijUtSZK5PACZrclDKpvn9cZOmrrS5B0Hx1U4LoUlRZ3iLI/jOliOItGyEV0rJFjRMZOZdsnvGf8fl1FZHiHvKjJ5h7yjcBy3sESog9+r4zO9+L0GFSGThqoQ1WE/42ojTG2ooD7iJ7O1dszr0Zk4PMpxh9QWi+07UxZNsSwdyRzLN8W2SzpDPg+jqkLEMhYrNsVRShV21Y2v4XOThjJn4lA+N2koM8fvuO1EedDL0GiAde0pOlKFMwWVUsAHRenDyoPSjFOIA4imuv8V6IN8Ps8FF1zAD3/4Q0aPHr1HAqqsrOSXv/wlp512GjU1Ndxzzz2cdtppAKxcuZJDDz2UJUuWcMwxx/DYY4/xuc99js2bN1NXVwfALbfcwhVXXEFrayumaXLFFVewcOFCVqxYUXyPr3zlK3R1dfH4448DMH36dI466ihuvPFGAFzXZcSIEVx00UVceeWVxGKxXcbSF/F4nGg0SiwWIxKJDNhnJg5cTbEsC5dvZnh5EEPffpdfazJLWyLHIXVlZPIuuq5xVEMFuq7TlS4kFCkrj2W7jKgIMaY2yIvvd9CWyKFr0JQoJAuuWziwGAqzSCnLRdMKP5U5LhgGGJqO6dFxXAeFxhC/w/I/X0/biuc5/vu3YVSMoD7qpy4cIJHLowGvNXbSmc6jofAYOgHTIGc5KBQ1YR9hv5eI34OmaYyoDDFpeGE3ndp6APM7mxOMqAwyZ2I9lWU+WhO5D9WWuWzoyHBIfVmvbSBSOZt4Ns/nJg3t9266lniW599r47l3W0nkbCoCXuoiAWojPnK2QzRoctzBO07KhBD7jr5+/+7XTJXX6+Uvf/nLRw6uN47jcN9995FKpZgxYwZLly4ln8/3OE9w/PjxjBw5kiVLlgCwZMkSJk6cWEyoAGbPnk08Hi/Odi1ZsmS7Mwlnz55dvIdlWSxdurTHGF3XmTVrVnFMX2LpTS6XIx6P93gIMZB6W4Lq3uW3JZbGq2vUlPkZWh6kzO+lK2WxYnOcZDbPurY0nRkLpRQVIR+ja4KE/SbTGioxdI32dJ7x9WGqQyZdmTyZvFs4bDhf6EJuu4WEygVcBaDIOy62A8nGd1jy6/NpeX0xysnj63ifY8ZUcdLEoRw1uoKI30vOdqkq8xH2eQj6PPi9BpVBk9qIH003cBXUhH0YhkF9JEh50MSj6ziuoime4cX3O2lL5uhI53h0RRN/fW0TD7+xuccyn99j0BTPsro5RSxjbff5+b0GluP2+xDi7iXFtmSOKQ3ljK+LoDSNlc1xXm/spCrkk4RKiANQvwvVTz75ZP72t79x6aWXDkgAy5cvZ8aMGWSzWcrKyvjrX//KhAkTWLZsGaZpUl5e3mN8XV0dTU1NADQ1NfVIqLqvd1/b2Zh4PE4mk6GzsxPHcXod030MT1NT0y5j6c38+fP50Y9+1LcPQojd8OHz4NAo7vKrLfPRkrKoj/ipCfu26fJdSErWtCUI+QpLWA1VgWI/pYqQyaTh5Ty1qoW2pMWmeIZc3qV7Hmzbue3uVMR1wdUUBi7tz99P23P3gHIJVw/l85f8lAlHTCdj2Rh6oQFp0GfQ2pQj6veQtjzkbUU2X2j0aduKYVEfwysCHDWqmpRlc9whNbQmCrvt1rWnWNeWwvToHDmygrqIn4xls+T9dpI5h5mHVBcPHI4GTGrD/uLM3cThH/TWgt1rxtlbY9X6aICxuTIs26EpliUa9O6wq7oQYv/V76Rq3Lhx/PjHP+b5559n6tSphEKhHtcvvvjift3vkEMOYdmyZcRiMf785z9zzjnn8Oyzz/Y3rEHpqquuYt68ecWv4/E4I0aMKGFEYn/z4fPgQj4PrcksAa9BS8rqUZAOcFB1iHgmzyH1YTJ5xcjKIJGAp0eiAVAf9RPxGXSkchi6TnnAQ8JyyFou287pbFs7kO1qoe2R/yXdWJglbjj6eI766mUMGVJDQ6WfNzbGeWtTnAlDI9RH/Bi6RtZWaICtHNA0Ujkbv8cg4DVI5hzS+TzlQZPhFUEOHxalM2Xx9MpWAA6tD6Pr3cmQhqHreHSHxo5CPygNjZDPoKrMJGXZtCWzpHIhyrYmXLvbjLO3OjYNrXBfnwefx2BzV2a78/2EEPu/fidVt99+O+Xl5SxdupSlS5f2uKZpWr+TKtM0GTt2LABTp07llVde4YYbbuD000/Hsiy6urp6zBA1NzdTX18PQH19/Xa79Lp35G075sO79Jqbm4lEIgQCAQzDwDCMXsdse49dxdIbn8+Hzyc/rYo9a9vz4Apd0nPUhPzUR/w0VBaWzbp1nz0XDZhUl5l4ttZhJXOF/lJeo3B0zLLGDta2Z2hL5vAakMsXdgh6PIXxlq2KCVWh8QLE31xMuvEtdDNAwxcu4pMnfonOtI1h6GyJ5WhLZOlM59jQmaY27KMq5CWWsWlLWiSyeTy6jsco3L81mcPQNZ5f3caJE4cQDRTqqjRNI523GVUV2iahgrzr4rgu1WU+2lM5UjmHMl/hNQ2VQWLpPJtjGWIZi8BHPIR4Z60Utv2M+7ukKITY9/U7qVq7du2eiKPIdV1yuRxTp07F6/WyePFiTj31VABWrVpFY2NjsR/WjBkz+OlPf0pLS0uxIemiRYuIRCJMmDChOObRRx/t8R6LFi0q3sM0TaZOncrixYs5+eSTizEsXryYb3/72wB9ikWIgbSjRpU70n0e3OjqIChFRdCkJuzb7jUZy8Z2XJRyifpN3mtNoBS0p3Kk8zZZy6E1YdGezOHRNbxb+0Y5rovtghfwenSUXjhw2KGQUGlA9Jj/gHQnIz91OjXDRhLPOhi6Ri5vs7HTprrMx4ShUbbEMqzvSNMcyxHPWOga+LwG+tbsLJa2aEsUit87U3k0rYkyn4ePj6vBVfSa0Hh1HY+howDbdck7HyQ05UGTsbVluEqRzbts7Ep/pEOIt61j615m3NZAnO8nhNg39Tup2lb3xsHd7b9y1VVXceKJJzJy5EgSiQT33HMPzzzzDE888QTRaJRvfOMbzJs3j8rKSiKRCBdddBEzZswo7rY74YQTmDBhAmeddRa/+MUvaGpq4gc/+AFz584tzhB985vf5MYbb+Tyyy/n3HPP5amnnuKBBx5g4cKFxTjmzZvHOeecw7Rp0zj66KO5/vrrSaVSfP3rXwfoUyxi8OlvYjJY7KpR5Y5omsao6hCHDomypjVBDT1nSTuSOV54v61whIxdOOvuzY0x8o6izGeQzDps6EzTkSokOrVhP45LMUFRgOWC7ioCpkGscSVtS/5CzefmEQz4cFwP5V/4NkGvF0MHr64RCXjxaBqaUag7Gl5ReIwfEubR5U10pnLomkbQq5OzXXKOS97tTtQ0PAZ0pC0eX9FEZzrPJ8bV9JrQhHwGlSGTxo40IbNwZmA3pRQ522HmIbUcObK80Iz0I/x5+HAd27b3GMjz/YQQ+57dSqpuv/12rrvuOlavXg0U6qy6+z/1R0tLC2effTZbtmwhGo0yadIknnjiCY4//ngArrvuOnRd59RTTyWXyzF79mxuuumm4usNw+CRRx7hwgsvZMaMGYRCIc455xx+/OMfF8eMHj2ahQsXcumll3LDDTcwfPhwbrvtNmbPnl0cc/rpp9Pa2srVV19NU1MTRxxxBI8//niP4vVdxSIGl91NTEqtt+7o2bzT43y+XSVW29ZYdd/jvZYEz6xqIW25jKwMsLkrS2fKIpVzyOUd4pk8iWyeeDaPpoFCozmexVEKpSBk6mh2IeHJOw6dL/6Flqf/CK5NsK6BmuPPIuj14biFZcLuZKwrbWHoUBcOFHcRenUdDY2AaeD3GoUYHAfbKbzW0ME0CjNPrlvoqu71aKxqTtJQGSDi8/Jea4KDqsuKS3yapjGyIsja1iS2awAKx1U9lvkmDo/u8pDlvtjRZ/xRlhSFEPuHfvWpArj66qv59a9/XZypgULbghtvvJFLL720R0IjepI+VXvHjhKTlkSWaMDcZWJSKkopnl5Z6Py97QG93dfWtacYUxNm5vgagJ3OwrXEsyzfFOO9liQt8SxvbOwi77gcNjRCecBHIpdnxaYYubxD3ins7IvnHHJ5G6Ug76pCTZCicFSMBqahkY210fzIr8msewOAigkfZ9jnv4M3GAYg6PNQHTKJBkw0DbbEslSXmYzcWiyftmw0TSPgNVjfnmRNa5p42kLTNBxX4aIwDR20QgNRv9cgGvBw1KhK0pZDwDSojwRY254i77g0VAYZWxPG59VpSWTRNI2qkEkyZ+/xZHpfTdyFEP3X1+/f/Z6puvnmm/m///s/zjjjjOJzX/jCF5g0aRIXXXSRJFWipHrb7g6FbfyjzFDx4OCZvdQbldrOuqNvez7f6uYkGzszu/5mrkApl01daeKZPA2VIfxeD7qu4dF1QNGeyoOmSOXsQjJD4Qy+vFPoPWVohYcLxFa9SPPCG3AzcXSvj0NPuYihR8+hzOfBZ+gYHp2IzyDsN5kwNIxSGi+v62BzZ5o1rSnKg16y+UJPqHgmT0s8Tc5W6IaGsfVsGtfZ2kxUB4XCdlx0rdCkc2NXlrDfw+FDo4yoDPBeS4rGjhSbY1kmDAkzvj7K4cMi1IR9e2XZt7uObV9cYhZC7Bn9Tqry+TzTpk3b7vmpU6dud+aeEHtbXxOTwbjdvS+7yta1p3j23VZA7XB5ECjO1EX8Jj6vQcTvJWnZrG9L01AdJGe7xDIOedfFs3VWyGtqJHMuua0JlUcDTdcwgLYX/0b7U/8HQHDIGA4+/fuUDWmgYmsReCbvUhky8Xt0mhNZ2lM2hw0tQ9c10rZLynLQNI1owEvIZ5DN2+TyLrYCv0cv1HkphaaBi8J1Cq0WHE2Ry7usak6SthzKzMJnUxnycdRok0OHhFnblmZ4RZDjDqku7gjcW/9vNU0bdH+OhBCl0+/tKWeddRY333zzds///ve/58wzzxyQoITYXX1JTHang/besLMDehWKtmSW9W2FQ40bKoO9nmO3fFOM5RsLZ901VAaxbJes5eI1NMrMQjKzoSNDczyDRiFxcpXauvymYeha8R8FW23d4acUgbFHo/uC1H3sVA49/waMyhFUB32MqStjdHUIr6Hh1TUytotX19nclaY1YeHVwKNrW2fCNDy6RibvkMg6VIV8eAyt2JrBNLaeGegU3lPTQdc0Qn4vtusSNA3Q4f3WFF1pCw2NsN/LmJoQsUyeWEZ+qBNClNZuF6r/85//LO58e+mll2hsbOTss8/u0ezy17/+9cBEKUQf7cvb3Xe0qyyWsVjXluatLXFiaQtdUyzfHC/2oFJKkco5eA2dl9d24NE1wgEPyzfH2dKVoTWZI20V+kGZHp0t8Syuq/DoOq4qHJRs6Bo528Zr6OgaKFeR27SS4PBDUYC/agijL/w9NdW1eL0GVSEvk4dHSeVd3mtNEc/adKatwuzT1kOSdR1i2TyGphHye7Adl3guj+0o/F6dynI/blvh9247ioxd+H9jK4VSCq9eKGbXtEJyNyzqY1xNmFTeYX1HmmigsNQmfaGEEINFv5OqFStWMGXKFADWrFkDQHV1NdXV1T0OLZa6AlEK+/J29952leVsh9cbu2hJ5Aj7DYKmn0jQpCmeJZGxGVkZoDOTZ1NHik1dOVoSGXRNo8zvIez3MrIyQH3ET2NHmrZ4BlspAl4dr2Gg6woXMPXCDFUiV6ilyiW6aH30ejLvL2XI6f9D2egjMD0aZkU1mqZhGjpDogG8hkGtz8v6jhSbOjMoFPURP15VqMta05KiKZ4jl7epi/gJB7xUBE28hk5jRxpdU0QCXtBgTHWIroxNV9qiM1Xok6VreuE8wIBJRdBkVHWIMr8Xj6HTsXXnYpnfM6gTZSHEgaXfSdXTTz+9J+IQYkDs69vdt+2OvqmzMDvVmcpxUHWISMDD+vY0OlBb5mN9R5oNnSl0NDbFsqSy1tYZH5d41mZIVNEcM6gOewutETSwbZcOy0ajkEgFTINQ0MTKuyjlEFv9Cq2PXoeb6kIzvNiJtsIyoKWwHYeA6aGmzMfQaIDmRJaKgJdUzsZVLuUBk7Rlk845OC74TA2NQkuGSKDQOsF2cjRUBYn4vTR2pKku85HNu2iazsF1ZThKsaalcNzO2NoyDqkPM7wiwPutaZriGVCF3YGJbJ6865Y8Ud5Xe6EJIfaMj9T8U4jBqEdi0pWmLZX7SB2097buXWXr2lJsjqUxDY2UZRPLWMVu5GNqgli2Q3M8h+sWCsF9XoPyoIe2ZJZkziaWzmN6dGzXJZu3MfRC/ZKuFRIqZ2tTzJRlo2ybrmfvpHXJQwB4q0dS/YXLidSPwufRSFmFdgd52ybvOLQkc2zqzPBec4KM5eD1GJT5PSSyedKWg+HRKPeajKkpY2NnhnjGYUjUS9py2NyVwWvolJk6jlKMqwsRDZi0xLN0ZvKU+TzUDIsyeXg5Q8sDKKWoCZs0xbI0dqaI+r0Ymk7edlnXnipZoiwtFYQQHyZJldgv7evb3TVNI2U5bOjM4vfo+Lw6SavQqHNzLMuGjjR+j7612aaL3+spJCo+g3TegwbEcjZmWqMzmcdyXDQUxtaz/jy6hu4q0nmXXOsGWv7xS6yW9wEIT5lD+XHnYnh9uICLjtdQeA3I2orVLUlSVmHpzWsYxBybnJOnNZHjkLow5UEf0YCHoOnB9Gj4vAbpnE1HOo+uQXvKYsKQKPXRAI5bSJhMj04k4KE65GfisAgbOjO835akM2XRuLXLe85xiKVtNnZmGVERIO841IUDHFQbxGvoqK0F93vDR23SKoTYP0lSJfZb+/J2d6UU77cmsR0X02/Q1JUjnrVwXI0yn4fWhEVn2sJ11dZaokKvp5ReWHrTdA3XhdZEnrxbaE+gobBdcJTC5zFwVKGDeWbLu1gt76MHIlSd+B2C46YXYgAsB8p8GiHTIJHNF3cKVpd5CZgmXekcaDCiIkhd2I/PNAgpRXnARNc1cnmHmrCPg8dUsbErQ0siRyxtURf2cejQKIcNDWN6jO0S3/KQydq2FM++24Jh6FSFTPxlPjTA59WpDvvQdZ1YxuKltVlMo2uvzRLty73QhBB7liRVQgxCXek8sXSeEZVB3t4cJ5nNY9mFvk5B06AmbLK+3cZVhTYSnq2d0bsyFrbroGs6IVNHoeFaNpqCdL6wO8706CQzFnm1NRk47NM4qS5CE47DE67qJRabgFcnt7V3lEfXaE1YVJRpOE6hxsl1oabMJJbJA1sPPdZ0ujIWQ6JBhlYEGFoRoDWRozOd56TD6xlVHdph0lET9lFVZhL0efHokLJsPLpOQ1UZ5QGDpY0xklmbGQdVETA9e3WWaF/uhSaE2LMkqRJikFFK0RzLsKErQyxlsaE9TSpvowEBr4Ht6CS3zhqhIK8gnrELrRAApSBgFpKnbN5G1zS8HogVJpVIvbeM9mfupPY/rsEIFGqRotNP3WE8LpDKu/gMtp6zp2hNWnSk80QDXsIBD/FsnsbODCGfQTRg0prM4dWhzG/SUBVAQ9va+sHm0PrIThMqKCQuyZzNx8ZUAhp5x8VrFBLF5ZsSeHQw9K0zdN29uvbSLFFfeqFJiwchDkySVAnRR7uz06u/r2mJZ/n36jaeWtnMa42dhTYHeQdNKyx7WY5G1nZJ5mxcF7rbhLoUkilFIXHKWC6oPK5beCJtKTTHpuP5e4gveRBQxP59D5XHf7PPv3+voZN3Cst/IVPHY3iI+L2UB7xsiWeJZyzApC5ioGfy6LrBmJogZb7CDsH+7L7sTlwCXk+xDgwgmbNpT+WoLvORzNnk3Q8Sl701S7Qv90ITQuxZklQJ0Qe7s9Orv69piWf5x7LNLHm/jbZErtBVXCl0TeEoDduBgEfD9ECrvf056BpbZ6rorodShXP7FGTbt9D8j19ibXkXgLJJJ1D+qXP69Rlk8i6GDq7S0DWdsN8gm3dx/FBdZqIUjK4Jcc6MBpI5hw2daTZ3ZdjYle737ssdJS55x8V2XRQ6HkPHq/dMXPbGLNG+3AtNCLFnSVIlxC7szk6vvr6meyYrm3d4+f0OVjZ10ZGyiGUtHKfQW8pFw3YU4NCVc9jZEZvdiRUUkinXVaTefpqWJ25GWRl0X4jKz15EaPzHd+uz8Hs9KKB9a7sGDWhL5vCbOiPLA0T9HgKmh6EVQcbVle327ssdJS5eQ8fQNNpTFiMrg4R8PZfg9sYs0b7eC00IsedIUiXETuzOTi+lFMs3xWiKZaiP+gu9ofTtX3OYUry1OcGmrjSdKYsX329nQ0earq3F3raj0LVCYuQAjsMH6329+PDcjKsg+fpjtP3zJgD8ww+j6vPfxROp7ffnoAOmAY7rEvF7SFkOHSkLVylCppeg6UehsaEzw6auDBUh8yPtvtxR4kL3DkbHZWRFsGSzRPt6LzQhxJ4hSZUQO7E7O71WNyd5emULWdtlUyyDR9epDJqFJTLAdl2Wru9gXXsSxy20N9jUleG91gSdqTyOy9ZltkIys7Usiu0X/HZOAcEJn8Lzyl8JH/4ZKj/2H9j0Xly9I92/Y59HozrsI561yeZdDF3DclyqQj6mjKigLuIjadm0JHK8sraD2rDvIycWO0pcZhxUSXvSoitTOMuwVLNE+3ovNCHEwJOkSoid6OtOr2zeoTNlsakrw+K3m2mKZ2moDOLzGnSlcjz/XhtdmTw+r47Po5HIOIyuCfGxgypZ05rk3eY4WcstFps7Luga2Fu/9sDWRpw7p1yH9Mp/Ezz0k4Wder4QQ8/9HZrH3Nkk1055dAj4PIT9HnSgM23h93oImDCtoYK6aABU4RDlcbVh8q47YDvwdpS4tCZyg2KWaF/uhSaEGHiSVAmxE33Z6ZXLO7yyroOutMXbW+K0xHM4riJrOyRzNss3xWiJZ8k7irKtyUnSstnSleapVQ6GBm0JC0cpXLVNsbn6YHbK5oNZox2x4y20Pfy/5Da+RaWVJnzEieiA8pjFe+6MRmFmbNvkS9cKy5amoRHL2LiuQtd1DF2jLuQjHPCSyzt0ZSxCPi+jqoN4dH1Ad+D1lrjILJEQYjCSpEqIndjVTq81rUkSWRuPoVNmetDRGVbu5/32FG9uiGE7Lu3pPF6j0Ak97yri2TyactEUrG1Lbp39yeO6PdOeDydBO0uKUu/8i/YnbkTlUmhmAN0MFl/Tl2VDjcKSY8hXOPhY2/qigE/DqxcK5R3XpcxvMKLcTzLvEA56SebyeA2DIdEgDVUBogETx1V7pU+TzBIJIQYbSaqE2IltC6bXtiUp83nRdQ3XVSSyFomcTSTgZXRViK5MHke5lPt9jKvWeHZ1G50ZC5+u4fF4cCm0BAh6DZK2S1M8SzzrYLuqWDfVX66VoePJ35NavggAc8ghVH/+e3grhgB9S6j0reNct7DsqKnCsmNl0ENNxFeo8dI0bMclnXfIKxhXG+bQIVGqysxCU06fgbb1dyB9moQQBypJqoTYhdqIn8OHRXjirSbe3BQnZzv4PAbDogH8Hp26sI/OtEUsnSeXd0jk8piGTsDUaUsqknkHj9N9/h5kKPSQSudstm031d9C9FzTe7T94xfYnZsBjciML1N+7BloRv/+Wm87n+RsnS3zGRr1UT/VYT/JnE0qZ2Mrha5p+D060xoqyNou5QGv9GkSQoitJKkSgt47n0P37r8Mr6ztIGQaHDE8WtiVp8HmrgzLN8RZ25oimbfJWg5py0HXNMoDHhIZG0MDr9fAY+hYeZeM7WA5LmFTJ5H9iDHbOeyuJoxwNdWf+y7+kRM/0v00wPRo+AyDg2rKGFIeIJmzCXgNfIZOcGs9WNZyGFMbZmNnRvo0CSHENiSpEvuV3TlKprfO52U+D0pBMpfn7S1xmmI5ynwGpkfH69HJ24rWZJbmWIa2RJYyn4eAaRAwNDqzNs3xLKlcHl2DfB7yzgcdO3Vdkc+7/Z6ZAlCOXZyJ8g8/jOovXI6/YTJGILwbd/tAyAtB00Nt1I/tKA6uD1Md8pG1XRxXYeiFGap03mFLPkNVyMfY2rJBsQNPCCEGC0mqxH5jd4+S+XDn85ZEhiffaUFDcfiwKNm8QyqXpzmRpTzgZVxtGV2pLFu6MnSmLHIORPw2uq7huC7ZvEvOctlR43N3N+u306tfouPJW6j78o/xVo0A6Fdn9O5eV9v2vNIoLPXVhP00VIWIbm1XkLFcVIgerSSUUrSnLKpCPmrCJpVlPtmBJ4QQ25CkSuwXducome5u6V3pHDVhP5bj4ipFS9wi4NXQNJ117SlWNSXI5l38Xo2mWJZk1sayHdqTFlnbRdMg70Au65Jz+l8btStuPkfn038g+fpCAGJLHqD6c9/t1z10wKNr2K4q9L3SQNMo1H75DA6ui/DxcVWE/R7+tbqdRDZfWMoLmJiGjuW4xDIWjuMyZWRFcded7MATQogPSFIl9nm7c5QMFOqlVjbF6ErnWd+RwXYKS11tyRzDygOgwYpNMboyNhVBDx5dJ23lWduWIG2pYrsCjcKuvoFOpgCslrW0/eOX5NsbAYgc9SXKP3l2v++j64WkStMKRfIKMLZWzlcFfXxsbBVDy4M4rqKhqvDf5niWWNqie25L1zWOGFnBx8dVy2yUEEL0QpIqsc/bnaNkADZ1ZXh7SwK/R6c8aGL6vXRlCsefoIGORsZy8BkamZyLo2wyeRfLVj12zO2JZEopReK1R+h8+g/g5NFD5VTPmUdg9JQ+vV4DTGPrzJQqHHlju24xCTQNCJgefB6dqQ0VDKsIAIV2CJUhk+mjq9jQkea91gRpyyFoGoytCTNxeFTqpYQQYgckqRL7vL4eJbNtM0qlFO+3JrEdl/Kwr/jaoOkh4vfSnrToTFnoGjgKUrk8lu3gOAprz/a0BCD11tN0PnkrAIExR1F14ncwQuV9fn2hI7siZOqgabiOwnILx+B4dA2fRydkemioDjGloRwNrUc7hHF1ZYyrK2NKQ4XUSwkhRB9JUiX2ebs6SiaTt7EdRSydLyYHXek8sXSekZUhOtO5QlKlafi9Oj6PQWcqRTyTZ2RlgKzt0hq390oy1S004VOkVjxJYNwMwlM+t3vJjK4R9nsxdB3To5O1HHK2QzToxaPrlPk9fGxMJWG/SSpn99oOQeqlhBCi7ySpEvu8nR0l05XO8fK6LkxD41/vtRSadpYHqY2YWK7L2NoQb292t+7sM/F6NPKui+O6hd18CtqSFvk9nFApO0986cNEpn4ezeNF0w1qT/9pv5Mpj15Y3nNdcBxFImcT9BqAQtM16iJ+poyqZFxNGUGfQdpy2NiVlnYIQggxACSpEvu8bY+S2bYZZUsiw4vvdxZaI4yuoi7iL+4IXN+uk8u7VIcMJg6PsK49zZauDImcTSxtM6q6jI5UjtZEllTO3iN1U93y7Rto/ccvybe8j5vqpOLT3yj+vvpDp9CU1Kvr2HqhkMpxoTbiY0xNGWU+L6bHYHhFgBMOq6M24pd2CEIIMYAkqRL7hdqIn+MOqfmgGWUyx9q2FGU+g2kNlcVlrKDPoFr5eL81hWU7eAyNyqC3WG3uugrbcdC9OsPLg2zuyrCnzgVWSpF84wk6F/8fys6hByL4Rh7+EW9a+I9H1wj4PPi8BsccVM2EoVFCPgMUrGtP8dbmBLURvyzvCSHEAJKkSuw3aiP+YjPKlngW23WpC/sp8xeOnIllLNa3Z2hP5UhbNomMzYaONPGcTZnPw9ByPz6vTnM8S1cmTy6RJbebnc93xckk6Hj8t6TffQEAf8MRVM25FE+4qt/30oCQqReKzVF4jEKNWW3Ej6bB0PIAZd21Zho73A0phBDio5GkSuxXuptR5mwXj6ETMAt/xGMZi+Ub46SsPOUBE0ODLbEsnUmLVN6mKugjadnUhExGVoVobE/SmsjtkVqq3OZVtP71ZzjJdtA9VHzqbMJHnYym6X3/ffJBKwfTgOEVAXRNI5bNUxXyYegaHkMjZHqIBnoebNzbbkghhBAfXd//Fd8D5s+fz1FHHUU4HKa2tpaTTz6ZVatW9RiTzWaZO3cuVVVVlJWVceqpp9Lc3NxjTGNjI3PmzCEYDFJbW8tll12Gbfc8JOSZZ55hypQp+Hw+xo4dy4IFC7aL53e/+x2jRo3C7/czffp0Xn755X7HIvYepRSdKYumWJbOlIVSH8wpbbsjUKFY354hZeWpC/txlGJdW5p0ziYc8DAk4qci5CViGuScwvLfxo4MGcvZI7NUejCKa2XwVA6j/qxfETn6lJ0mVIVu6FtrprZ5aBQ6ozsKOlJ58q6iPGDSUBUENPK2YkxNmDJ/z5+dsnkH09DxeUr6118IIfY7Jf1X9dlnn2Xu3Lm8+OKLLFq0iHw+zwknnEAqlSqOufTSS3n44Yd58MEHefbZZ9m8eTOnnHJK8brjOMyZMwfLsnjhhRe48847WbBgAVdffXVxzNq1a5kzZw4zZ85k2bJlXHLJJZx33nk88cQTxTH3338/8+bN45prruG1115j8uTJzJ49m5aWlj7HIvaelniWp1e28sibm1m4fDOPvLmZp1e20hLPAh/sCGxJFI6VaU/lKA+YoLF1ec+ipsyHz6tTHvDSmcqzviPNq+s6+NfqNroy9oDWUjnZZPHX3vJ66r78I4accz2++rE7fV3AgKoyL0Gvjt+rY2zNpgod0iHkMzA0bWvbCJeKoLn1QGiDqjIfhw4pQ+OD4vPuXlTDyoOUB707fmMhhBD9pqltf7wvsdbWVmpra3n22Wf55Cc/SSwWo6amhnvuuYfTTjsNgJUrV3LooYeyZMkSjjnmGB577DE+97nPsXnzZurq6gC45ZZbuOKKK2htbcU0Ta644goWLlzIihUriu/1la98ha6uLh5//HEApk+fzlFHHcWNN94IgOu6jBgxgosuuogrr7yyT7HsSjweJxqNEovFiEQiA/rZHUh2dM5f91l13ef8dY/b2JlhQ2eKurCflGWzYlOcsM+gOuxnXXuKWNqiKZYBDVxHkXEGLlalFKm3nqZj0S3UnHxlnzqi64DHKPy3ImTiuIq05WJ6IJtXOFubeHoNnYjfA5pGKmsTCXr5xLhqRlWHqA756MrkcZXa/jMKmhx38PZnIQohhOhdX79/D6r5/1gsBkBlZSUAS5cuJZ/PM2vWrOKY8ePHM3LkSJYsWQLAkiVLmDhxYjGhApg9ezbxeJy33nqrOGbbe3SP6b6HZVksXbq0xxhd15k1a1ZxTF9i+bBcLkc8Hu/xEB/NtocgV5f5sByXjOUQ8OpUl/nY0JnixTUduK5b3BE4tjZE3imcZZfM2ng9Gh5DZ3NXhi2daTZ2ZbAchesqcgOYULm5FG0P/4r2hb9GWWmSby7a6Xid7uNjNMoDXkyvQTJnk7MdbNcluTU4j6FjGDq6UeiWHvZ7qC/3Ux/1c+iQKP8xdQSnTB3O5ycPZUxNmHg2z8auNPFsnjE1YUmohBBiDxk0hequ63LJJZdw7LHHcvjhhW3lTU1NmKZJeXl5j7F1dXU0NTUVx2ybUHVf7762szHxeJxMJkNnZyeO4/Q6ZuXKlX2O5cPmz5/Pj370oz5+AqIvejsEOe8oLMfF1DUUsLYtDZrimIOqqI34+dykIaA0VrfG8Xt01rUlac7a5BwXRynsrWf5OS4M1IpfduM7tD3yK5xYM2g65R8/k8gxp+30NWU+g4qgF10vLNiFfBppyyaTd1Cq0HNKKRfTYxDxG1SHfVSETDyajqtcyvxeEjkbTdMKZx5usxtSelEJIcSeN2iSqrlz57JixQr+/e9/lzqUAXPVVVcxb9684tfxeJwRI0aUMKJ934cPQc5rGptjKWIZi3K/yajqEIlcnvdaUmTzbnEp8JgxlTTHMyx6p5nGzgxZyyHvKpT64HDkgUiolOsQW/IAsefvBeXiidZR/fnL8A0bv9PXFdIchaFBNu/iKsXEYRGiQZP2hMX77Sla4llcBRVBLwfVlBH2e1AKYpk8ZT6DiN9TmG3bphisezekEEKIPW9QJFXf/va3eeSRR3juuecYPnx48fn6+nosy6Krq6vHDFFzczP19fXFMR/epde9I2/bMR/epdfc3EwkEiEQCGAYBoZh9Dpm23vsKpYP8/l8+Hy+fnwSYmeUUqxpSZC2bMKmD6UUbcksubxDxOchns3T2JFiaNTP6OogbckcKzbFmRku/D9oS+XY1JnBdt3CTkGl0LVCb6eBkl37GrF/3w1A6LCZVB5/IbovuMvXGTrYLiQst3jY8bCKEGV+L/XRAPUVfl5d18mWWBbHVXgNjbyjSOVs/B4dr0cn4jcpD3hlV58QQpRISf/1VUrx7W9/m7/+9a889dRTjB49usf1qVOn4vV6Wbx4cfG5VatW0djYyIwZMwCYMWMGy5cv77FLb9GiRUQiESZMmFAcs+09usd038M0TaZOndpjjOu6LF68uDimL7EcSHbWzmBPWd2c5JV1neTyDm83JVi2oYtlG2Ns6EixuiXFpq4MKzbF2dxVSDy6m1x2piyWb4qxqTODrkE04CUa9OI19AFNqAACY46i7MiTqPrcd6n+3Hf7lFABBDyFpcvKkEmZ30N1mQ+P0f3XU6Mq5OfIEeVUBL3Es4XDj7N5hzKfQcBnUBXyETANhlXIrj4hhCiVks5UzZ07l3vuuYe///3vhMPhYm1SNBolEAgQjUb5xje+wbx586isrCQSiXDRRRcxY8aM4m67E044gQkTJnDWWWfxi1/8gqamJn7wgx8wd+7c4izRN7/5TW688UYuv/xyzj33XJ566ikeeOABFi5cWIxl3rx5nHPOOUybNo2jjz6a66+/nlQqxde//vViTLuK5UDREs8Wj4OxHBfT0BlWHtyjh/G2xLM8+24rTfEsI6uCrG9Ls649TWfawjQ0yvxeXEeRcxXNiRyvrOtgakMFluPSmsjxemMX69rTdKQt8o5bqJ8agHzKtTJ0/esuojO+jBGMAlB1wrf6dQ8NyDvg9WgcXBvknaYUXn1rwqcUbK2Bqo0EmDKigqWNnfg8BmV+DxG/h4jf3JpQBTh8WERqpoQQokRKmlTdfPPNABx33HE9nr/jjjv42te+BsB1112Hruuceuqp5HI5Zs+ezU033VQcaxgGjzzyCBdeeCEzZswgFApxzjnn8OMf/7g4ZvTo0SxcuJBLL72UG264geHDh3Pbbbcxe/bs4pjTTz+d1tZWrr76apqamjjiiCN4/PHHexSv7yqWA8GO2hmsaU3QlswVa5gGUveOv7RlUxv2E/TqBE0DAI+m4SrI2S4Rn4eIR6fMb7CpK0skkGRkRZD2VI43NnbSHEtj2e6AdUnPbVlN28O/xO7cjB1rpvaUH/TpdRrg9xaW7wq/QTAMCPsMNm4tvm9JZEFTRAImNWU+Qj5PYTZQ1zh6VCXTRlXQmbFxXUV5wMuwij2b1AohhNi1QdWnan+3r/epUkrx9MpW1rQmGFUV6jEjopRiXXuKMTVhZo6vGdDZks6UxSNv/v/27jxMrrJM+P/37LV1VfXe6SSdhWyEBIgJxADKljEs4uD4U0R0ABkcFN4XBBVQAR2XMDCjgC+CjqOMr4wo/gSVTSNLkEwIEhOSsISErCTpfal9Oec87x+VLtJJk3QnnTTpvj/X1ZepOk/VuR+qru7bZ7mfnVQ4Jm+1p9nWkaE1kaUzXSBb9Mm7PoamCActYgGbyqBJV7aIY+qcP3sMAVPnJy9sZktnmkzh0L/uSvkkXvot3c//X/A9jIoaaj58A4Gm2QN6vQFEAgamXqp6nvU8DE0j4hgELJMxsQB51ydoGaBByDIZEw9Q3F0+4qwZ9Vw4p5GerCu7+oQQ4ggY6N/v98RCdXF06M4U2dGdoa4isM8fcE3TDttBvXnXp+D5BG2TCVUhmnuy9ORclNKIBk26M0UyBY+gBxqKXYk8nek8AcvkxU0d7Erk6EjlhyShcpPtdDz+fXJbXwEgNO0Uqs75XxjBigO+1tj9v2HnnaWMHhC2DMKOSXXEIWDpRIMWPekiPmBpGh3pPEXPozEe5MSmSk6bWoOu67KrTwgh3mMkqRID1pvcBCyj3+tDfVCvUoruTJGeTAHX88kWXGJBi8k1ETa3pWnL59E0HVPX0Hcf1bK5vbRmSqHQFKzbmaA7UyRbKB5yPPmd62n9zTfxswk0y6Hy7M8ROf5DBxwh0qF8vAwKqsM2mq7jej5hR6MxZtORdqkImJiGzoTqMC1WHpQiYBqkCi6+glOPqWXBlGqZ4hNCiPcoSarEgO15SHHY2ferM5QH9fZZDO/6bO/M8uqOBDURh5zr4ZgaKI180SPv+lQ4BqZpUHQL5AseRR8KxTxOwUD5Pjn3wPc8EKt6HLodxIzWUnPBl7CqD1xzTAfCto5p6ORcH9fzSRd8grZG0DaojwbIFDxMQ8dXioqARdA0GBMtHalzXGMUQ9dpS+Y5eXKVJFRCCPEeJkmVGLDeQ4rfaksy0d53TVVrMscxtRWHvKW/v8Xweddlzds9bGhNclxjnMm1FSTzLm+15nB9RSRgkE3kybqlIgmWDqChlCLnHXzhhGLXLsx4A5qmoTth6i76NmZFDZp54D72jlAFHZOQZaDnXVxfpzpi4ysImjpFV2EZOobuY+kGtREHNA3L1PHyCts0sI3Smql3GyEUQgjx3iBVAsWAaZrGrLFRYkGbLR1p0nkXzy8VoNzSkSYWsg95S3/vTr+ebIGJ1WHCjomuQbaomFAVpCrs0J7K0Z0tABqOpWNoinTOLSdUAEUfcp4ikfdxD+I8P6UUiZd/x87//Dyp1U+Wn7cqx7xrQmXs0W0NCJjsrnRuYWil6dG6iE1VyKYyZJUquqNoiAaYWBMmFrTKuxqLro+pl6Y2W5M5xsal/pQQQrzXyUiVGJTeQ4p7p+ba03lsQ+eY2ooh2dK/52J4NEjlXbozBXZ2ZYiFLCIBk2SuSMSxaIgGyBRckrpGquChvP7Xcg12hZeX7qb9ie+T27QSgNzWNUROPPeAyaKpg/JA1ygf9pwvenSm8mg6GLpOPBpg/uQqzphex6s7kuzoKe2YdD3FqzsTtKbyxByTrmyB6nCAtlSe+BAkq0IIIQ4/SarEoB3Og3p7F8PnXY+32tJ0pPO0JnNsak0T2F2fKu/6hJ3SeXfdGReFojBEi+Ozm1bS/sT38dPdYFhUnXUFkTnnD6hvrg8BW6MuEiAaMNjZk0PXNTyl0P3SmjTd0PGVVjp6JhbkufVttKfy1FUEmDkmysa2JFs7M1i7p/ymDFGyKoQQ4vCTpEoclMN1UK9j6uSLPhtbunH9UrX2bMHD831yxdIByJah05HKs63DI1ssFcA81IKeyi3StfQBki//DgCrZgI1H/kydu3EAb9H0NY5pjpMPGyxrTOLrmnUR20qAqXpvsqwjYZiQ2uKZRvbuXDO2D6jfgXPZ1xlkFmNcSbXhcpTfjJCJYQQRwdJqsSw6y2dkHd9LANyRZfWZJ7pdWG2duVQvqI64pDJu6TyLiHbJOd6JHNFdCDnHnr9qULbFpIr/wBAxfs+TPyMy9GtgR+GrQFRx8RxTMK2hablaYgFmdNUiWXoBKzdNRWUYntXhpVbuzh9Wu1hHfUTQghxZElSJYbV3ucIup5iR3eWCsfk7USOrnSesG1iGD4d6Ty5okcq7+L7UPB8iq7iINah78MZM5XKs67AjI8hNOXkQb1WA6pCBvMmVuErDdfzcSydsGPuTqj22LWnadREHHZ2Z2lL5qmKOIdt1E8IIcSRJbv/xLBp6cny+JpdrN7ejaFrjI0HCVg6PVkXy9AIWgY51ydTKJLOFfF8cD3IFlxc36PgHXxC5WUTtP3+TgptW8vPRef9/aATKgDLgFgoQFXEYUJ1qHQWYdHH8328fk5tVlDKxJDRKCGEGElkpEoMi5aeLP/90jbWNyeJBk060nmqww7VEZu6Cod0waMuZNNQEaAjk2dnT45ErojnK0pr0hX95CsDktu6hvbH/h0v1YHbtZOGf/zeQU+36UDINomHLExdLx3XE7XZ2KpIFTz0vd5WKUVHukB12KG2QkanhBBiJJGkShxxrYkcT6xt5o3mJPUVDpGARdH12dWToSdbIGgZpPMu7akcHekC2zrTpPIemlJ4nsJTcDCb/ZTn0v3CgyRe/A2gMKvGUrXo6kNavxS0SucP1kQcArsryVc4JkHbwNB0ErkimqZhGzoFz6cnW8DzfN7XVClTfkIIMcJIUiWOKKUUa3f0sLM7i2WUjm/RNXAsg3ozQEsyRzRoUeGY/G1bF/miIu8qsgUPXw2+5lSvYtdO2v9wJ4VdGwCIHP8hKs/+HLo9uFIFu4/vAyBoQkXAwjENaiI2PlAserQmc4yvCjGhOlQ+u7D3lbqulQ9FlsXoQggxskhSJY6oDS0pnn2jlZ5ckZ3dGVp7skRDDnUVpUrj8aBNRzqP6yk6MwXSOY+i7x9SQlVo3UTzgzeiCll0J0zVOf+L8IzTDqkfOhAL2hxTH6GpsnRkT0c6j6FpOJbJGdOrOOWYatbtSLCxLUmm4BGyDabUVjB7XEzqTgkhxAgkSZU4YloTOZa+2UZzIkd1yMT3YUcyy9vdWTaZJuOrgtRVOLzVlqbo+fg+aJpC1w4+oYJSzSm7bhJoOjUfvgEzWjuo1+uUqqX7qjTeZBowtb6C4xrj1FU4TKoN4/mQyhfpyRRpiAU4dUoNddEAddEA75tQKeUShBBiFJCkShwRvWf6ZQoulq6zbmeS7kyRdMHD9RUJinRlcihKO/wClk46V8QH3uX0mf3K73oTu3Yimmmj6Qa1//B1dCeMpg/8UGJHo5RFaWBbBpauEQtYnDm9lk/On0gsaPLqzmS5HIRt6MwaG+9TAV3KJQghxOghSZU4InrP9IsFTHb2ZGhL5rHNUg0n11dkCy6JrIdHaWTI9TwK3uBHqJTv0bP81/Qs+yUVcy+g6uwrATCC0UG9j2NA2DEJ7z7g2DINYkGTGfUVfHL+RKY1VAClI3ukcKcQQgiQpEocIXm3dJ7fru48rqtKx9F4ipBl4Pk+qZxfrjml4KASKrenlfbH/o38268B4GcTKOWjaQdXjs3UdSrDDvWxIDVhi7yrGFMZIhq0ym1kJEoIIUQvSarEYdV7BE1PpkAy59Lck8WxDQK+j1b0SRdckrkiOXeP1/DODruBSr/+Fzr++H9Q+TSaHaTqQ18gctyZBx23Bug6xIM24+JBgpbBrp5saSG6KTVzhRBC7EuSKjFk9jzDzzF1Cq73zpoj12d7Z4atHWkUkMi6FDyXTKF01Ay8U198MAmVX8jS+ecfkV77ZwDsxunUXPBlrHjDQffD0ABNI2SZTK4JEXZMcgWXTNFjfGWYeMg64HsIIYQYfSSpEkNi7zP88kWftmSeioDJMbURApZB0fN4ZXs3LYksBQ90pfqMSh1MgXQ/myTz5nLQdGILPkHslE+iGQf3tdaACken4ClCtkllyEI3dHIFl80dGZqqQ8yfXClrpoQQQvRLkipxSJRSbGhJsfTNNjIFlwlVIQK2wd+2drOtM8O4yiBFz8f1fVoT+dKIT0EdUokEpVQ5sTFjddScfz16IExg/KxD6kssaKChEdAU4yuDeEqxoyuLpkFTVYiPzx1HfSx4SPcQQggxcklSJQ5aayLH2h09PPtGK82JHHUVAVxfURNxSBdcJlWH6M4VWbWti55skS3taXzANumzhmow3GQ7HY/fRfTkjxKcPBeA0NT5h9yXoKlR4ZgoNAKWDhooNBpjQeZMiDN/UpUkVEIIIfZLkipxUFoTOZ5b30ZzT5ac6zOhKoSmaTQncjT3ZMm7PmbIJpEpsLE1Td7zQYHn+/jegd+/P5kNL9Lx5D342QRuTzONV/5oUHWn+tM75Rd2TEIBi/qIw8TqEK2pArZu0FQdlIRKCCHEgEhSJQatt5BnT7ZAQyzAjp4sjmWg794Zt70rw67uLNs6M/RkXLoyeXRNwzJ10nmfwiAXT/nFHF3P/Cep1U8CYNcfQ80FXz7ohEqjVCE9YJZGpGzTwDENJlWHqA47ZF2fhliQWY0VdGWKvLozSV00IGuphBBC7JckVWLQegt51lUEUJTqORVcDwV0Jou0JbJs7kiTL/qYxjsV0m1Dxx1kefRC6ybaf38nxY7tAERP/gfiH/wMmnFwO/AsHUxdY1p9hKkNUbZ3ZsgWPOpjgVL9KU1jTCzEhOogsaCNZRjs6M7QnSlKPSohhBD7JUmVGLS861PwfAKWga6XEqY1byfozuTpTBfIux65oo+pa2iajucr0vki2QIUB5FTFbt2suvn14PnYoQrqT7/eoKT5hx03BoQtHQ0XSMeckjmXGY1xok4Bk3VYXylsAydsFNasA4QsAza03ny7qEsrRdCCDEaSFIlBs0xS6NOuaJX3tXX0pOhJ+tS9BRF16eowPUVRc/DNDRcX7F7WdWAWZWNhI89HT+boPq86zBCsUOK29ShNmJjmAZj40GmNlRwyuRqXtrSialrhJ19R79yRQ/b0KXgpxBCiAOSpEoMWjxkMTYeYmNbkp50gdZkDt0oTQEWfYWmaxheqWyCp8D3FP4As6nsppXY9cdghOMAVC+6BgzzkNczmRrYpo6uG8SCNidPrmLmmBgVAZNY0KI5kWOSHe5zH6UUrckcx9RWSMFPIYQQByRJ1Si1d/XzWNCkJ+sO6GBgTdOYNTbKuh3d/OWtdrozRRKZIgVPYRug6ZRGpVTvvQYQj1uka+kDJF/+HcHJ86j9/25D0zQ089CTGZ3SKFVFwKIu6nDqMTVEHJMVmzt2Fyr1aEsWSOc9JteECVgGuaJHazJHLGQza2xUFqkLIYQ4IEmqRqH+qp/nXQ/H1HEsA9vQGRsPMWtslLpo4F3fx+ud4iv6ZIsuSkHeA1zFYKomFNu30/aHOyi2bgbAjDeA78JBLkbvT8g2qIk4HDc2hmMZbG5PU1cRKCdQuaJPIltkR3cWZ/ei+mNqKw7430AIIYToJUnVKNNbX6onW6CuIkDe9djY0k1rMk9dhcOc8ZU4ls5bbUnaU3nOmF67T1KhlGLtjh7yRZewpbO9UCwlUzCoRVNKKVKvPEXX0z9BuXn0YJTq864jNOXkoeswUBsxaaqp4P2TqplUE6IjXWBi9TtTfWHHZPbYGJvbUzREQ5w0qZKAZex3tE4IIYTYmyRVo8ie9aUmVodBg7fa0ri+z4z6CK3pAtu7Mxw/NsbE6jBbOtKs25HgzAqnT3KxoTnJI6ve5o3mJNva0xQOYmOcl0vR8eTdZN9cDkBg4hyqz/8iZqRqqLqLCYyrCnLKlBqm1VewYPei9LqKfWtOaZpGfTRIT65AwDKkfIIQQohBG9YtTc8//zwXXHABjY2NaJrGo48+2ue6Uopbb72VMWPGEAwGWbhwIRs2bOjTprOzk0suuYRoNEo8HueKK64glUr1abNmzRo+8IEPEAgEGD9+PHfcccc+sTz88MPMmDGDQCDA7NmzeeKJJwYdy3vdnvWlNE0jnffoSOeJB200XScWsOhMl9YWaZpGXUWgXKMJSv8Nlm9s4ztPvMYLG9rZ2pEeVImEPWmaXpru000qz/wsdZ/45iEnVBqlBelBC8bFHBZMqWLGmChzJ1Rx3uwxxEJ2uRREfwKWUZoOlfIJQgghDsKwJlXpdJoTTjiBe++9t9/rd9xxB/fccw/3338/K1asIBwOs2jRInK5XLnNJZdcwquvvsqSJUt47LHHeP755/nc5z5Xvp5IJPjQhz7EhAkTWLlyJXfeeSff+MY3+PGPf1xu8z//8z9cfPHFXHHFFaxatYoLL7yQCy+8kHXr1g0qlve6PetLAeWDjq3d5QJ6i3MW/VJSsWeS0ZrI8cCyzdz2+1dZubWLZM6l6JUSmYFOkCnPRe1eta47IWr+/iYaPvNvRE/+BzTt0L6Klg6Ta8KcPbOOa86YwvUfms7CYxuY0xTnzN1TmHuWguiPlE8QQghxKDSlBrI36/DTNI1HHnmECy+8ECiNijQ2NnLDDTfwpS99CYCenh7q6+t54IEH+OQnP8nrr7/OzJkz+etf/8q8efMAeOqppzjvvPN4++23aWxs5L777uNrX/sazc3N2HZpSuemm27i0Ucf5Y033gDgoosuIp1O89hjj5Xjef/738+JJ57I/fffP6BYBiKRSBCLxejp6SEajQ7Jf7fB6EoXeGzNTqIBi7Bjksq7/HVLJyHLwNm9YDtb9DhpQhWRgEk675LIFZk/qZrn32zl8bXNbOtKUyz6JPPeoGpOFbt20v6HOwkfdxbRuRcMab8cA8ZXhpg5JkokaFMdsWmqDNGdLXBMbQVnzqhF0zSUUjz7RhtvtSX7rKmC0vdtS0e6T3shhBACBv73+z37f8k3b95Mc3MzCxcuLD8Xi8WYP38+y5eX1uEsX76ceDxeTqgAFi5ciK7rrFixotzmgx/8YDmhAli0aBHr16+nq6ur3GbP+/S26b3PQGLpTz6fJ5FI9PkZTr31pVqTOZRShB2DqrBNazJHMlugLZWnKmwTdoxyjabGWJDtnWnW7UyQzhfRFHhKDfiLo5QitfZpdj1wLYVdG0i8+DB+MT9kfQqYGrGgzbiqEDVRB01TrG9O8KfXmgH6lEPoLQURC9ps6UiTzru7q727bOlIS/kEIYQQh+Q9u1C9ubn0R7G+vr7P8/X19eVrzc3N1NXV9blumiZVVVV92kyaNGmf9+i9VllZSXNz8wHvc6BY+rN48WK++c1vHrizR0hvUtGeyrOlI03A0knliuzoyvLargTRoEVV0GZXT4aujEvIKRXGfHlrJ0XXx1egaaXpvoGMUvm5FB1/+iGZ158HwBk/i5oP34BuOYfUDx3QdVA+xIIWH50zlqKneKstRSJXmmLU0GhN5Nl7ILYuGuCM6bXlkhLt6byUTxBCCDEk3rNJ1Uhw8803c/3115cfJxIJxo8fP4wRvZNULNvYzvNvtpHMuzTEgtRHg2iaYmNbktebexhbGaKpKsSKzR2sb0mSLhRxPR/PU+QKPgdayp17+3Xa/3AnXqIVNJ34Bz5NdP7H0PT+F4kPhKmVkrqIo4OmoXw4a0YdU+oirN2RIBa0aIwHCVgGRc9jW1eGJ9Y2c/7xY/okS3XRAGdWOH2Kn0r5BCGEEIfqPZtUNTQ0ANDS0sKYMWPKz7e0tHDiiSeW27S2tvZ5neu6dHZ2ll/f0NBAS0tLnza9jw/UZs/rB4qlP47j4DiHNiozWHtXSu8vWaitcIgGLCbWhBkTC2AbBiFbZ2d3jpVbO9nRk0NDMTYeIF/0SedcWhJ5EjmXbNE/4FCVl+qi5aGvglfEjDdQc8GXcRqnD0HfwDE1fB8CtkkkbDI2HuS1XUm6MwXGVQbLC9593yBfVHRm8v2WhdA0TcomCCGEGFLv2TVVkyZNoqGhgaeffrr8XCKRYMWKFSxYsACABQsW0N3dzcqVK8ttnnnmGXzfZ/78+eU2zz//PMVisdxmyZIlTJ8+ncrKynKbPe/T26b3PgOJ5b2gNZHj2TfaeGzNTh5fu5PH1uzk2TfaaE303aHYnSmysyfLxOowVWGHSMCkJ1vkhbfaeXVnks5UnhWbu3hk1Q7eak/RVBUkkSuSzhVxfR/vAHN/RqSS+KkXEz7uTMZcds+QJFQAaGAYOiHHoqkqSF3U4c2WFKu3d9OWKrC1M0um4AJQ8Eq7GsdEg33KQgghhBCHy7COVKVSKTZu3Fh+vHnzZlavXk1VVRVNTU1cd911fPvb32bq1KlMmjSJW265hcbGxvIOwWOPPZZzzjmHK6+8kvvvv59iscg111zDJz/5SRobGwH41Kc+xTe/+U2uuOIKbrzxRtatW8fdd9/N97///fJ9r732Wk4//XT+/d//nfPPP5+HHnqIl19+uVx2QdO0A8Yy3PaulN57/Ep/ldH3Lq3QnSnwwoYO3mxJYuo60aBFruCRKXiseTuBZWik8y75/cz5pV9/HqumCbt2IgDR9398SKfTKmyY1RgvJVW2ScTRcZVGrugRMHTCtkFXOk8279FUHSRV8GiIBqgK2+zoyUrtKSGEEIfdsCZVL7/8MmeeeWb5ce/6o0svvZQHHniAr3zlK6TTaT73uc/R3d3NaaedxlNPPUUg8M76mAcffJBrrrmGs88+G13X+djHPsY999xTvh6LxfjTn/7E1Vdfzdy5c6mpqeHWW2/tU8vqlFNO4b//+7/5+te/zle/+lWmTp3Ko48+yqxZs8ptBhLLcNm7Uvqex69MtPetjL5nvaaQbbClI83ORBYDjVjQRCkNy1TURBxaEjm2tGcpeqUdf3unJn4+Q+eff0x63Z+xappo+Mfvo1vOkCRUOmDooGsaE6vChAIWFQGTmohD2LEYFw+yclsXbckCtqeIBiw603k2tHocUxthQlWIvOtL7SkhhBBHxHumTtVocLjqVO1df2pvvfWmPnx8I5Vhu0+9ppqIw7KN7WztzJDMFglYBnnXIxKwqA1bvNGcpiOdw/N9UoW+X5X8rjdp/8OduF27QNOJLfgEsVMvPqTF6L10QNcg7BhYhs5ZM+qZP7maCVVBXt+VJBos9bUrXWDphjZ2dGcJW6VSEJapc+aMehpjAak9JYQQ4pAN9O/3e3ahuhi4vafz9hawDNrT+fIU2J6lFTa1p0jlXExdI2DpdGcKRIMWsaBFuqDIFos4pk4i904VcqV8Eit+S/df/i/4HkZFLTUX3EBg/Kx+7z9YpgaGrhG0dKbVV+DYBp+a38QJ4+O0JPK8sqOn3NfKsM3pU2tYubWbVMElGjApeB6gpPaUEEKII0qSqhFgz+m8/kaq+jt+pbe0wotvGaxvTpEpuNiGTjRoE7B0NDRcz6XgKZQCQy9N/nm5FO2Pfpfc1jUAhKafRtU512AEIkPSF1uHSNBERyMetAnYJnOa4hw/LrbP1GVvXyvDDvMmVrK1M0Nzd5Z03iNX9JlaJ7WnhBBCHDmSVB3llFIopQhZJls60hzbUIGu632utyZzHFNbQTxk9XltXTTAh09oQOHz59db0TSNqpBFe7pAMlck63q4nkLToZRmgW4HUZ6HZgWoWvjPhGcvHNQokEap1hSqtD6rN1K1+/mAbRC0DAKGgYciGjBZdFxDuU+9VeHfaksy0X5n/Vg8ZBMNmJi6xrh4iDNn1FIZtmWESgghxBEjSdVRrDWRK1cG70wX2NKeZmd3luMaY9RHA+SKHq3J3H6nwHRdZ8ExNbQlC6ze3kVXtkhdxCYWMHm7S2EZGolUCs/X0A0bpRvUXPAllFvAqho7qHg1Sgcfe72V2RVURSxcz0fTNHRNwzJ0bNMgaJvUVNh8+v0TmNkYe+c99qoKv+dOx9ZkjjHxIB+YVkNV5MjWBxNCCCEkqTpK7V1Coa4iQFXYYu2OJKu2dTGhJkR12On3+JXeAqG5olcqSWAZfHBaLZVhi1XbutnZkyWV9+hKF3HbtrD1l98mMOFEKhf+MwBmtHZQsfamcjoQsHQKnqLoKWxLQykYEw8yZ3wlk2pC+GigoCtTYN6ESuZPrt7n/eSoGSGEEO9FklQdhforoaBQVARs5jbF2dKRZlw8xFkz6vpMgSml2NCSYu2OHja3p2lL5kgXPCKOQVNViOn1Uc6bPYYNrUmeX9/Gm8/8irce/zHKK+Ll0sRPuwRtAGuntN0/hlaqgu7zzpSf64GhaVRWWMxoiGIaOpOqw8wcGyVomeURp0m1EWaPi7/r9J0cNSOEEOK9RpKqo1B3psiO7gx1FQE0TaMnW2BrR5aOdL5U8dyHZK6b48fFy9NgrYkcL2xo5y8b2uhI58nkPRxLZ0wshOsptndm6UgVSGSLmIUkT/z7V2h+9UUAYtPfz/iPXEvRqiB3oHLq7J7mM0pTi5Uhi3S+lPj4fukg5IZYgFOn1HD6tHoaYg7NPXl2dGfoSBcGNeIkR80IIYR4L5Gk6ii0ZwmFnmyBtW8nSBeKxIM2lmmRL3ps7cyw9M228uL0Z99o5W/buvCBiG2RL5bKK3SlC4yvCpIuuLQkXF59aSkvPvBtcolOdNOm4UNXUj//AjRNp5B796Ne9jwOsCFqAxoFz6c6bDG5Okym6DGuKsTZM2qZN6GKkGOVR5aOHXPg8wqFEEKI9zpJqo5CvWUFskWXrR1Z0oUi9RWB3XNspRGcuooAmYLL2rd7QIPmRA5D14kGdLZlMsSCNqah0ZMr0p4uEA+ZvLa5mWU/uY1iJkmofhJzLr2FTGQsrudj6D7+7jqxvelObxJlaHusm9LBMY3ybryZjTEMXVFdEeS8WQ3Ux4L79EdGnIQQQowEklQdhXrLCqzb0U17Kkc8aJcTKqUUPbkiDdEAE6pCbGxLAhrxoMXO7iyWYeAphWloaJpGyDJJ5oqEbIOM7jDnk1+mc9MaJpxzJb5hooqKPJAteij1TvJkaKDrpUe2oeH5pXubhk7ANkgWXJQG27oyTKmLMH9SVb8JlRBCCDFSSFJ1FOotK/BWW4rWZJ6mKgPfVxQ8n55ckbBlMqEqRNA2yRQ8QKMm4mAaOr5SGJqG6ylMA9587rfo0VoqTvognq+YePJZTD/l73A9xY7uLL7yMXQd21Tkiz4apYXnpga2qZdHq2wT8kVFJGDSEHWoigSYXl9BJGCSyrus25GgJuLIzjwhhBAjliRVR6m6aIDTp9WytSNNIluk0y+gATURh2n1FcRDNum8S8g2AA1Dg6qwTXMiS8QxaWlt428P/iubVz6HU1HJ+Km/IuKUdtNVBMA0dHRNo+iD67r4CmxTwzF00kUPHw0FBE0dV0He9TB0jfqow6yxlUyoDhEPlab0aiNqn0OdhRBCiJFGkqqj2NT6CHPGx/nLxjagVFYhXXDZ1pUBoDtbYEptBWiwqS1FU2WIZNZl46oX+eMPbyXb3YZmmMxc9Bmqq6pId+foSHsYBY+6CosJ1SHa0wVae3K4vk/AMqmtCBANmLQk8mSKLigIGzpVIYv6WIDzZo8p70rs1bvGa0d3hu5MUdZPCSGEGJEkqTqKtSXzdGeLFD0wdUVNxEEB2zozbG5LcWJTJbPHlaqRd6QKtCdSvPyb/8MTD/4IpRSRuibmXnorx8+ZwzF1YTxNwzA0asM2qbyHpxRVIZvGCofOTJHJNWEm14bJe4pxMYe2dJHOVIGC59MQc8i7ippIoN+RqL0PdRZCCCFGGkmqjlK9BUB9pThzeg3bOnPlOlVh28D1DarDNrW7p9tOGhfi3EUfYf3a1QC8/9xP8JnrbuHY8TVMb4iSd330dbsI2yZFz6Mq7KDpGspXZIsu1dEA4ytDnHVsPW93ZdnRncEydcZXhxgbDzGuMsiKzR2DOtRZCCGEGEkkqTpK7VkANOyYxEI26bxH0fOxDB1QJHNuebpt0phqFsw9kebtW7jy5n9lyvyzMHSNt7tzKHTqojaOZTCnKc7bXbsTNNfD1HUa42HGVQZI5l2iQYsz6yP71JUCeLsru89Bx7D/Q52FEEKIkUKSqqPUngVAATQ0InuMEHm+YntzOzuaDSqPGQ/Ard+5k5M+dhVWtKbPQcRvtSXZ2qGTL/rUhA1mj4v2SdDCjkEm75WTqHerK7W/g473d6izEEIIMRLIXMxRqrcAaK7o9Xt91V9f5F8++2Gu+/w/4fs+Sik29/hY0RomVocJOyaGrhF2TCZWhyl6HnnXoyWZAwURx6QyZJcSNQWtyRxj46H9jjT1HnR8TG0FiVyRt7szJHJFjqmt4IxptVJOQQghxIgmI1VHqd4CoHtPt3muy4M/uotf3P89lO+zOeTQ3NxMMFbT57zAPWmaRn00WFonpeuDGmlSqu8RM7UVDmfOqJVjZ4QQQow6klQdpXoLgO453dbdupN/velqXl/9VwA+cfEl/Mf9PyQajdLck+szXbi3gGXgWAYnTaqiNVE64Lg9nd/vAcetiRzrdiTY0Z2h4PnYhs7YeGhAhyELIYQQI40kVUex3um2dTsS/P8P/4qf3fFVsukU4UgFd951D5+/4rJy2z2nC/fcnaeUIp336MkWcD2fxliA4xqjBxxpak3keG59Gz3Zwj7rs9pTec6YLtN9QgghRhdJqo5yddEAp9hw5c/vJptOcdL89/PQfz/I5MmT+7Trb7qwO1Nga2eGjlSB1mSOhmiAVdu7mT02tt+EqLecQ0+2wMTqd6Yew47JRDss1dOFEEKMSpJUjQCBQICHfvlLHnvsMW655RZMc9+Pde/pQsc02NiaoidXBKVojAeZWhdmU1uKjlRhvyNNe5Zz6G99llRPF0IIMRpJUjVCnHTSSZx00kn7bdM7Xbj27R6eXd9KcyJHXYVDddihpsIhYBlEHI22VH6/I017l3PYm1RPF0IIMRpJUjXK1EUDzGnS2NiWYmp9BNBoT+V5syWJ6/uYuk7YNnG9BHOa4v2ONL3b+qxeUj1dCCHEaCR/9UahgqcwDY2gZbCpLcXWjjTKV4Rtk6Cl05HO8dqu0q6+/vSuz2pN5lBK9bnWWz39QDWthBBCiJFGRqpGIcfUsXWdNW93s6Ujg65pdKYLGLpGRcAi6pik8gU2tWY4rjHW77opqZ4uhBBC9CUjVaNQ7wjS6u09FFwfxzSIOha2YdCZzvNma4qasEN3Nk93ptjve0j1dCGEEKIvGakapVxf4SmFoWtoGigNNA00BWgK29ApeP5+F5vXRQOcWeFI9XQhhBACSapGlL2PjHm3BKc7U6Tg+UyujeC6HrmiV0qwNI142CEWNEnli3i+c8DF5u92uLIQQggx2khSNUIM5siYvOtj6hqTqsI0JzKMq7TxFBi6RsDUUb7izbYUNeGALDYXQgghBkiSqhFgsEfGOKaOberUx2xSeZdErkg8aGOZOgXXpzWZo8IxmT1u30XqQgghhOifLFQ/yu19ZEzYMTF0rXRkTHWYnmyBdTsSfUof9JZEyBV9Zo2tYEwsRKbo0ZHOky64OJbJB6fV7q5jJYQQQoiBkJGqo9zBHBmzZ0mErkyBybUhJlSHSOWL9GSKNMQCnDqlRkaphBBCiEGQkapBuvfee5k4cSKBQID58+fz0ksvDWs8Azkypr9dfHuWREjmXLqzBXRNY9bYOGdMr5OSCEIIIcQgyUjVIPzqV7/i+uuv5/7772f+/PncddddLFq0iPXr11NXVzcsMR3KkTFSEkEIIYQYOjJSNQjf+973uPLKK7n88suZOXMm999/P6FQiJ/+9KfDFtOhHhnTWxKhIRagMmxLQiWEEEIcJEmqBqhQKLBy5UoWLlxYfk7XdRYuXMjy5cv7fU0+nyeRSPT5GWq966NiQZstHWnSeRfPV6TzLls60nJkjBBCCHGESFI1QO3t7XieR319fZ/n6+vraW5u7vc1ixcvJhaLlX/Gjx9/WGKTI2OEEEKI4Sdrqg6jm2++meuvv778OJFIHNbEStZHCSGEEMNHkqoBqqmpwTAMWlpa+jzf0tJCQ0NDv69xHAfHcY5EeIAcGSOEEEIMJ5n+GyDbtpk7dy5PP/10+Tnf93n66adZsGDBMEYmhBBCiPcCGakahOuvv55LL72UefPmcfLJJ3PXXXeRTqe5/PLLhzs0IYQQQgwzSaoG4aKLLqKtrY1bb72V5uZmTjzxRJ566ql9Fq8LIYQQYvTR1N7FjcRhk0gkiMVi9PT0EI1GhzscIYQQQgzAQP9+y5oqIYQQQoghIEmVEEIIIcQQkKRKCCGEEGIISFIlhBBCCDEEJKkSQgghhBgCUlLhCOrdaHk4DlYWQgghxOHR+3f7QAUTJKk6gpLJJMBhO/9PCCGEEIdPMpkkFou963WpU3UE+b7P+vXrmTlzJtu3bx9Vtap6D5Mebf2G0dv30dpvkL6Pxr6P1n7D6Oi7UopkMkljYyO6/u4rp2Sk6gjSdZ2xY8cCEI1GR+yXb39Ga79h9PZ9tPYbpO+jse+jtd8w8vu+vxGqXrJQXQghhBBiCEhSJYQQQggxBCSpOsIcx+G2227DcZzhDuWIGq39htHb99Hab5C+j8a+j9Z+w+ju+95koboQQgghxBCQkSohhBBCiCEgSZUQQgghxBCQpEoIIYQQYghIUiWEEEIIMQQkqTqC7r33XiZOnEggEGD+/Pm89NJLwx1S2fPPP88FF1xAY2Mjmqbx6KOP9rmulOLWW29lzJgxBINBFi5cyIYNG/q06ezs5JJLLiEajRKPx7niiitIpVJ92qxZs4YPfOADBAIBxo8fzx133LFPLA8//DAzZswgEAgwe/ZsnnjiiUHHMhiLFy/mpJNOoqKigrq6Oi688ELWr1/fp00ul+Pqq6+murqaSCTCxz72MVpaWvq02bZtG+effz6hUIi6ujq+/OUv47punzbPPfcc73vf+3AchylTpvDAAw/sE8+BvicDiWUg7rvvPo4//vhywb4FCxbw5JNPjug+v5vbb78dTdO47rrrRnT/v/GNb6BpWp+fGTNmjOg+72nHjh18+tOfprq6mmAwyOzZs3n55ZfL10fq77mJEyfu87lrmsbVV18NjPzP/YhS4oh46KGHlG3b6qc//al69dVX1ZVXXqni8bhqaWkZ7tCUUko98cQT6mtf+5r67W9/qwD1yCOP9Ll+++23q1gsph599FH1yiuvqI985CNq0qRJKpvNltucc8456oQTTlAvvvii+stf/qKmTJmiLr744vL1np4eVV9fry655BK1bt069ctf/lIFg0H1ox/9qNxm2bJlyjAMdccdd6jXXntNff3rX1eWZam1a9cOKpbBWLRokfrZz36m1q1bp1avXq3OO+881dTUpFKpVLnNVVddpcaPH6+efvpp9fLLL6v3v//96pRTTilfd11XzZo1Sy1cuFCtWrVKPfHEE6qmpkbdfPPN5TabNm1SoVBIXX/99eq1115TP/jBD5RhGOqpp54qtxnI9+RAsQzU73//e/X444+rN998U61fv1599atfVZZlqXXr1o3YPvfnpZdeUhMnTlTHH3+8uvbaawd8z6Ox/7fddps67rjj1K5du8o/bW1tI7rPvTo7O9WECRPUZZddplasWKE2bdqk/vjHP6qNGzeW24zU33Otra19PvMlS5YoQD377LNKqZH9uR9pklQdISeffLK6+uqry489z1ONjY1q8eLFwxhV//ZOqnzfVw0NDerOO+8sP9fd3a0cx1G//OUvlVJKvfbaawpQf/3rX8ttnnzySaVpmtqxY4dSSqkf/vCHqrKyUuXz+XKbG2+8UU2fPr38+BOf+IQ6//zz+8Qzf/589c///M8DjuVQtba2KkAtXbq0/P6WZamHH3643Ob1119XgFq+fLlSqpSU6rqumpuby23uu+8+FY1Gy/39yle+oo477rg+97rooovUokWLyo8P9D0ZSCyHorKyUv3kJz8ZNX1OJpNq6tSpasmSJer0008vJ1Ujtf+33XabOuGEE/q9NlL73OvGG29Up5122rteH02/56699lp1zDHHKN/3R/znfqTJ9N8RUCgUWLlyJQsXLiw/p+s6CxcuZPny5cMY2cBs3ryZ5ubmPvHHYjHmz59fjn/58uXE43HmzZtXbrNw4UJ0XWfFihXlNh/84AexbbvcZtGiRaxfv56urq5ymz3v09um9z4DieVQ9fT0AFBVVQXAypUrKRaLfe45Y8YMmpqa+vR/9uzZ1NfX94k7kUjw6quvDqhvA/meDCSWg+F5Hg899BDpdJoFCxaMij4DXH311Zx//vn7xDiS+79hwwYaGxuZPHkyl1xyCdu2bRvxfQb4/e9/z7x58/j4xz9OXV0dc+bM4T/+4z/K10fL77lCocAvfvELPvvZz6Jp2oj/3I80SaqOgPb2djzP6/OFBKivr6e5uXmYohq43hj3F39zczN1dXV9rpumSVVVVZ82/b3Hnvd4tzZ7Xj9QLIfC932uu+46Tj31VGbNmlW+p23bxOPx/cZ1sH1LJBJks9kBfU8GEstgrF27lkgkguM4XHXVVTzyyCPMnDlzRPe510MPPcTf/vY3Fi9evM+1kdr/+fPn88ADD/DUU09x3333sXnzZj7wgQ+QTCZHbJ97bdq0ifvuu4+pU6fyxz/+kc9//vP87//9v/mv//qvPvGP9N9zjz76KN3d3Vx22WXle43kz/1IM4c7ACHeS66++mrWrVvHCy+8MNyhHBHTp09n9erV9PT08Jvf/IZLL72UpUuXDndYh9327du59tprWbJkCYFAYLjDOWLOPffc8r+PP/545s+fz4QJE/j1r39NMBgcxsgOP9/3mTdvHt/97ncBmDNnDuvWreP+++/n0ksvHebojpz//M//5Nxzz6WxsXG4QxmRZKTqCKipqcEwjH12MLS0tNDQ0DBMUQ1cb4z7i7+hoYHW1tY+113XpbOzs0+b/t5jz3u8W5s9rx8oloN1zTXX8Nhjj/Hss88ybty48vMNDQ0UCgW6u7v3G9fB9i0ajRIMBgf0PRlILINh2zZTpkxh7ty5LF68mBNOOIG77757RPcZStMMra2tvO9978M0TUzTZOnSpdxzzz2Ypkl9ff2I7n+veDzOtGnT2Lhx44j/zMeMGcPMmTP7PHfssceWpz9Hw++5rVu38uc//5l/+qd/Kj830j/3I02SqiPAtm3mzp3L008/XX7O932efvppFixYMIyRDcykSZNoaGjoE38ikWDFihXl+BcsWEB3dzcrV64st3nmmWfwfZ/58+eX2zz//PMUi8VymyVLljB9+nQqKyvLbfa8T2+b3vsMJJbBUkpxzTXX8Mgjj/DMM88wadKkPtfnzp2LZVl97rl+/Xq2bdvWp/9r167t8wt3yZIlRKPR8i/yA/VtIN+TgcRyKHzfJ5/Pj/g+n3322axdu5bVq1eXf+bNm8cll1xS/vdI7n+vVCrFW2+9xZgxY0b8Z37qqafuUyrlzTffZMKECcDI/z0H8LOf/Yy6ujrOP//88nMj/XM/4oZ7pfxo8dBDDynHcdQDDzygXnvtNfW5z31OxePxPrsphlMymVSrVq1Sq1atUoD63ve+p1atWqW2bt2qlCpt743H4+p3v/udWrNmjfr7v//7frcaz5kzR61YsUK98MILaurUqX22Gnd3d6v6+nr1mc98Rq1bt0499NBDKhQK7bPV2DRN9W//9m/q9ddfV7fddlu/W40PFMtgfP7zn1exWEw999xzfbYdZzKZcpurrrpKNTU1qWeeeUa9/PLLasGCBWrBggXl671bjj/0oQ+p1atXq6eeekrV1tb2u+X4y1/+snr99dfVvffe2++W4wN9Tw4Uy0DddNNNaunSpWrz5s1qzZo16qabblKapqk//elPI7bP+7Pn7r+R2v8bbrhBPffcc2rz5s1q2bJlauHChaqmpka1traO2D73eumll5Rpmuo73/mO2rBhg3rwwQdVKBRSv/jFL8ptRvLvOc/zVFNTk7rxxhv3uTaSP/cjTZKqI+gHP/iBampqUrZtq5NPPlm9+OKLwx1S2bPPPquAfX4uvfRSpVRpi+8tt9yi6uvrleM46uyzz1br16/v8x4dHR3q4osvVpFIREWjUXX55ZerZDLZp80rr7yiTjvtNOU4jho7dqy6/fbb94nl17/+tZo2bZqybVsdd9xx6vHHH+9zfSCxDEZ//QbUz372s3KbbDarvvCFL6jKykoVCoXURz/6UbVr164+77NlyxZ17rnnqmAwqGpqatQNN9ygisVinzbPPvusOvHEE5Vt22ry5Ml97tHrQN+TgcQyEJ/97GfVhAkTlG3bqra2Vp199tnlhGqk9nl/9k6qRmL/L7roIjVmzBhl27YaO3asuuiii/rUaRqJfd7TH/7wBzVr1izlOI6aMWOG+vGPf9zn+kj+PffHP/5RAf2+x0j/3I8kTSmlhmWITAghhBBiBJE1VUIIIYQQQ0CSKiGEEEKIISBJlRBCCCHEEJCkSgghhBBiCEhSJYQQQggxBCSpEkIIIYQYApJUCSGEEEIMAUmqhBBCCCGGgCRVQggxzJ577jk0TdvnIFkhxNFFKqoLIUaEM844gxNPPJG77rpruEMZtEKhQGdnJ/X19WiaNtzhCCEOkoxUCSFGDaUUrusOdxh9FItFbNumoaFBEiohjnKSVAkhjnqXXXYZS5cu5e6770bTNDRNY8uWLeVptSeffJK5c+fiOA4vvPACl112GRdeeGGf97juuus444wzyo9932fx4sVMmjSJYDDICSecwG9+85v9xjFx4kS+9a1vcfHFFxMOhxk7diz33ntvnzaapnHffffxkY98hHA4zHe+851+p/+WLVvGGWecQSgUorKykkWLFtHV1XXQsQkhDj9JqoQQR727776bBQsWcOWVV7Jr1y527drF+PHjy9dvuukmbr/9dl5//XWOP/74Ab3n4sWL+fnPf87999/Pq6++yhe/+EU+/elPs3Tp0v2+7s477+SEE05g1apV3HTTTVx77bUsWbKkT5tvfOMbfPSjH2Xt2rV89rOf3ec9Vq9ezdlnn83MmTNZvnw5L7zwAhdccAGe5x1SbEKIw8sc7gCEEOJQxWIxbNsmFArR0NCwz/V/+Zd/4e/+7u8G/H75fJ7vfve7/PnPf2bBggUATJ48mRdeeIEf/ehHnH766e/62lNPPZWbbroJgGnTprFs2TK+//3v97n/pz71KS6//PLy402bNvV5jzvuuIN58+bxwx/+sPzccccdd8ixCSEOL0mqhBAj3rx58wbVfuPGjWQymX0SsUKhwJw5c/b72t5EZ8/Hey+eP1A8q1ev5uMf//iQxyaEOLwkqRJCjHjhcLjPY13X2Xvjc7FYLP87lUoB8PjjjzN27Ng+7RzHGfJ49hYMBt/12uGOTQhx8CSpEkKMCLZtl9ccHUhtbS3r1q3r89zq1auxLAuAmTNn4jgO27ZtG/R02osvvrjP42OPPXZQ73H88cfz9NNP881vfnOfa4cSmxDi8JKkSggxIkycOJEVK1awZcsWIpEIVVVV79r2rLPO4s477+TnP/85CxYs4Be/+AXr1q0rT59VVFTwpS99iS9+8Yv4vs9pp51GT08Py5YtIxqNcumll77rey9btow77riDCy+8kCVLlvDwww/z+OOPD6ovN998M7Nnz+YLX/gCV111FbZt8+yzz/Lxj3+cmpqag45NCHF4ye4/IcSI8KUvfQnDMJg5cya1tbVs27btXdsuWrSIW265ha985SucdNJJJJNJ/vEf/7FPm29961vccsstLF68mGOPPZZzzjmHxx9/nEmTJu03jhtuuIGXX36ZOXPm8O1vf5vvfe97LFq0aFB9mTZtGn/605945ZVXOPnkk1mwYAG/+93vME3zkGITQhxeUlFdCCGGyMSJE7nuuuu47rrrhjsUIcQwkJEqIYQQQoghIEmVEEIIIcQQkOk/IYQQQoghICNVQgghhBBDQJIqIYQQQoghIEmVEEIIIcQQkKRKCCGEEGIISFIlhBBCCDEEJKkSQgghhBgCklQJIYQQQgwBSaqEEEIIIYbA/wMZFUqEEheHaAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_train, lr_tuned.predict(X_train), alpha=0.3)\n",
    "grid = np.linspace(y_train.min(), y_train.max(), 1000)\n",
    "plt.plot(grid, grid, \"--k\")\n",
    "plt.xlabel(\"true price\")\n",
    "plt.ylabel(\"predicte price\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- Here we can see a few cases where our prediction is way off.\n",
    "- Is there something weird about those houses, perhaps? Outliers? \n",
    "- Under the line means we're under-prediction, over the line means we're over-predicting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### $R^2$ (not in detail)\n",
    "\n",
    "A common score is the $R^2$\n",
    "\n",
    "- This is the score that `sklearn` uses by default when you call score():\n",
    "- You can [read about it](https://en.wikipedia.org/wiki/Coefficient_of_determination) if interested.\n",
    "- Intuition: similar to mean squared error, but flipped (higher is better), and normalized so the max is 1.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$R^2(y, \\hat{y}) = 1 - \\frac{\\sum_{i=1}^n (y_i - \\hat{y_i})^2}{\\sum_{i=1}^n (y_i - \\bar{y})^2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Key points:\n",
    "- The maximum is 1 for perfect predictions\n",
    "- Negative values are very bad: \"worse than DummyRegressor\" (very bad)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "(optional) Warning: MSE is \"reversible\" but $R^2$ is not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "873230473.3636105"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_squared_error(y_train, preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "873230473.3636105"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_squared_error(preds, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8601212294857902"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(y_train, preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8279622258827067"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(preds, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- When you call `fit` it minimizes MSE / maximizes $R^2$ (or something like that) by default.\n",
    "- Just like in classification, this isn't always what you want!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### MAPE\n",
    "\n",
    "- We got an RMSE of ~$30,000 before. \n",
    "\n",
    "Question: Is an error of \\$30,000 acceptable?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29550.473318774617"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_train, lr_tuned.predict(X_train)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- For a house worth \\$600k, it seems reasonable! That's 5% error.\n",
    "- For a house worth \\$60k, that is terrible. It's 50% error.\n",
    "\n",
    "We have both of these cases in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAn0ElEQVR4nO3df3SU1Z3H8U9CkkkQZsLPCSkJREXCbzFoGMG2i9lms6kLS45FD91FYaXaSIW4KukKFFZNxK5QWAjVpUGPIit7CpWisBg1rjUJEEFBPBErkKwwoVubGaCSILn7h+usA1EySbiTTN6vc+45zPPceZ7vZYbhc+7c55koY4wRAACAJdHhLgAAAHQvhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEu4ALNTc36/jx4+rdu7eioqLCXQ4AAGgFY4xOnTql5ORkRUd/89xGpwsfx48fV0pKSrjLAAAAbVBXV6fBgwd/Y59OFz569+4t6YvinU5nmKsBAACt4ff7lZKSEvh//Jt0uvDx5VctTqeT8AEAQBfTmiUTLDgFAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVMeEuAJfP0IXbgx4fLc4NUyUAAPw/Zj4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVoUUPs6fP69FixYpLS1NCQkJuuqqq/TP//zPMsYE+hhjtHjxYg0aNEgJCQnKysrS4cOHO7xwdIyhC7cHNQAALreQwsfjjz+ukpIS/eu//qs++OADPf7441q+fLlWr14d6LN8+XKtWrVK69atU1VVla644gplZ2fr7NmzHV48AADoekK6vfrbb7+tqVOnKjf3i9t0Dx06VC+88IJ2794t6YtZj5UrV+rhhx/W1KlTJUnPPvus3G63tm7dqttuu62DywcAAF1NSDMfN954o8rKyvThhx9Kkt5991299dZbysnJkSQdOXJEXq9XWVlZgee4XC5lZmaqoqKixWM2NjbK7/cHNQAAELlCmvlYuHCh/H6/0tPT1aNHD50/f16PPvqoZs6cKUnyer2SJLfbHfQ8t9sd2HehoqIiLV26tC21AwCALiikmY8XX3xRzz//vDZu3Kh33nlHzzzzjH7+85/rmWeeaXMBhYWF8vl8gVZXV9fmYwEAgM4vpJmPBx54QAsXLgys3RgzZoyOHTumoqIizZo1S0lJSZKk+vp6DRo0KPC8+vp6XXvttS0e0+FwyOFwtLF8AADQ1YQ08/HnP/9Z0dHBT+nRo4eam5slSWlpaUpKSlJZWVlgv9/vV1VVlTweTweUCwAAurqQZj5uueUWPfroo0pNTdWoUaO0b98+Pfnkk5o9e7YkKSoqSvPnz9cjjzyiYcOGKS0tTYsWLVJycrKmTZt2OepHCLiPBwCgMwgpfKxevVqLFi3Sj3/8Y508eVLJycn60Y9+pMWLFwf6PPjggzpz5ozmzp2rhoYGTZ48WTt27FB8fHyHFw8AALqeKPPV25N2An6/Xy6XSz6fT06nM9zldGltmek4Wpx7GSoBAES6UP7/5rddAACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFgVUvgYOnSooqKiLmr5+fmSpLNnzyo/P1/9+vVTr169lJeXp/r6+stSOAAA6JpCCh979uzRiRMnAm3Xrl2SpFtvvVWStGDBAm3btk2bN29WeXm5jh8/runTp3d81QAAoMuKCaXzgAEDgh4XFxfrqquu0ne+8x35fD6tX79eGzdu1JQpUyRJpaWlGjFihCorKzVx4sSOqxoAAHRZbV7z0dTUpOeee06zZ89WVFSUqqurde7cOWVlZQX6pKenKzU1VRUVFV97nMbGRvn9/qAGAAAiV5vDx9atW9XQ0KA77rhDkuT1ehUXF6fExMSgfm63W16v92uPU1RUJJfLFWgpKSltLQkAAHQBbQ4f69evV05OjpKTk9tVQGFhoXw+X6DV1dW163gAAKBzC2nNx5eOHTumV199Vb/+9a8D25KSktTU1KSGhoag2Y/6+nolJSV97bEcDoccDkdbygAAAF1Qm2Y+SktLNXDgQOXm5ga2ZWRkKDY2VmVlZYFtNTU1qq2tlcfjaX+lAAAgIoQ889Hc3KzS0lLNmjVLMTH//3SXy6U5c+aooKBAffv2ldPp1Lx58+TxeLjSBQAABIQcPl599VXV1tZq9uzZF+1bsWKFoqOjlZeXp8bGRmVnZ2vt2rUdUigAAIgMUcYYE+4ivsrv98vlcsnn88npdIa7nC5t6MLtIT/naHHupTsBAHCBUP7/5rddAACAVYQPAABgVZsutUXn05avWAAACAdmPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEuwBEhqELtwc9PlqcG6ZKAACdHTMfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAq0IOH5988ol++MMfql+/fkpISNCYMWO0d+/ewH5jjBYvXqxBgwYpISFBWVlZOnz4cIcWjc5v6MLtFzUAAKQQw8ef/vQnTZo0SbGxsXrllVd06NAh/cu//Iv69OkT6LN8+XKtWrVK69atU1VVla644gplZ2fr7NmzHV48AADoemJC6fz4448rJSVFpaWlgW1paWmBPxtjtHLlSj388MOaOnWqJOnZZ5+V2+3W1q1bddttt3VQ2QAAoKsKaebjpZde0oQJE3Trrbdq4MCBGj9+vJ5++unA/iNHjsjr9SorKyuwzeVyKTMzUxUVFS0es7GxUX6/P6gBAIDIFdLMx8cff6ySkhIVFBTopz/9qfbs2aOf/OQniouL06xZs+T1eiVJbrc76Hlutzuw70JFRUVaunRpG8vvnlg/AQDoykKa+WhubtZ1112nxx57TOPHj9fcuXN11113ad26dW0uoLCwUD6fL9Dq6urafCwAAND5hRQ+Bg0apJEjRwZtGzFihGprayVJSUlJkqT6+vqgPvX19YF9F3I4HHI6nUENAABErpDCx6RJk1RTUxO07cMPP9SQIUMkfbH4NCkpSWVlZYH9fr9fVVVV8ng8HVAuAADo6kJa87FgwQLdeOONeuyxx/SDH/xAu3fv1lNPPaWnnnpKkhQVFaX58+frkUce0bBhw5SWlqZFixYpOTlZ06ZNuxz1AwCALiak8HH99ddry5YtKiws1LJly5SWlqaVK1dq5syZgT4PPvigzpw5o7lz56qhoUGTJ0/Wjh07FB8f3+HFAwCAriek8CFJ3//+9/X973//a/dHRUVp2bJlWrZsWbsKAwAAkYnfdgEAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVSFfaovu58IfsjtanBumSgAAkYCZDwAAYBXhAwAAWEX4AAAAVrHmA0EuXN8BAEBHY+YDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGBVTLgLwKUNXbg93CUAANBhmPkAAABWET4AAIBVhA8AAGBVSOHjZz/7maKiooJaenp6YP/Zs2eVn5+vfv36qVevXsrLy1N9fX2HF42uaejC7UENANA9hTzzMWrUKJ04cSLQ3nrrrcC+BQsWaNu2bdq8ebPKy8t1/PhxTZ8+vUMLBgAAXVvIV7vExMQoKSnpou0+n0/r16/Xxo0bNWXKFElSaWmpRowYocrKSk2cOLH91QIAgC4v5JmPw4cPKzk5WVdeeaVmzpyp2tpaSVJ1dbXOnTunrKysQN/09HSlpqaqoqLia4/X2Ngov98f1AAAQOQKaeYjMzNTGzZs0PDhw3XixAktXbpUN910kw4ePCiv16u4uDglJiYGPcftdsvr9X7tMYuKirR06dI2FY/wYL0GAKA9QgofOTk5gT+PHTtWmZmZGjJkiF588UUlJCS0qYDCwkIVFBQEHvv9fqWkpLTpWAAAoPNr16W2iYmJuuaaa/TRRx8pKSlJTU1NamhoCOpTX1/f4hqRLzkcDjmdzqAGAAAiV7vCx+nTp/X73/9egwYNUkZGhmJjY1VWVhbYX1NTo9raWnk8nnYXCgAAIkNIX7v84z/+o2655RYNGTJEx48f15IlS9SjRw/dfvvtcrlcmjNnjgoKCtS3b185nU7NmzdPHo+HK10AAEBASOHjv//7v3X77bfrj3/8owYMGKDJkyersrJSAwYMkCStWLFC0dHRysvLU2Njo7Kzs7V27drLUjgAAOiaoowxJtxFfJXf75fL5ZLP52P9x/+J1KtLjhbnhrsEAEAHCeX/b37bBQAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBV7QofxcXFioqK0vz58wPbzp49q/z8fPXr10+9evVSXl6e6uvr21snItDQhdsvagCAyNfm8LFnzx798pe/1NixY4O2L1iwQNu2bdPmzZtVXl6u48ePa/r06e0uFAAARIY2hY/Tp09r5syZevrpp9WnT5/Adp/Pp/Xr1+vJJ5/UlClTlJGRodLSUr399tuqrKzssKIBAEDX1abwkZ+fr9zcXGVlZQVtr66u1rlz54K2p6enKzU1VRUVFe2rFAAARISYUJ+wadMmvfPOO9qzZ89F+7xer+Li4pSYmBi03e12y+v1tni8xsZGNTY2Bh77/f5QSwIAAF1ISDMfdXV1uu+++/T8888rPj6+QwooKiqSy+UKtJSUlA45LgAA6JxCCh/V1dU6efKkrrvuOsXExCgmJkbl5eVatWqVYmJi5Ha71dTUpIaGhqDn1dfXKykpqcVjFhYWyufzBVpdXV2bBwMAADq/kL52ufnmm3XgwIGgbXfeeafS09P10EMPKSUlRbGxsSorK1NeXp4kqaamRrW1tfJ4PC0e0+FwyOFwtLF8AADQ1YQUPnr37q3Ro0cHbbviiivUr1+/wPY5c+aooKBAffv2ldPp1Lx58+TxeDRx4sSOqxoAAHRZIS84vZQVK1YoOjpaeXl5amxsVHZ2ttauXdvRpwEAAF1UlDHGhLuIr/L7/XK5XPL5fHI6neEup1PoTnf+PFqcG+4SAABtEMr/3/y2CwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMCqmFA6l5SUqKSkREePHpUkjRo1SosXL1ZOTo4k6ezZs7r//vu1adMmNTY2Kjs7W2vXrpXb7e7wwruCoQu3Bz0+Wpwb8nO6m7b8nV3O4wAAOl5IMx+DBw9WcXGxqqurtXfvXk2ZMkVTp07V+++/L0lasGCBtm3bps2bN6u8vFzHjx/X9OnTL0vhAACgawpp5uOWW24Jevzoo4+qpKRElZWVGjx4sNavX6+NGzdqypQpkqTS0lKNGDFClZWVmjhxYsdVDQAAuqw2r/k4f/68Nm3apDNnzsjj8ai6ulrnzp1TVlZWoE96erpSU1NVUVHxtcdpbGyU3+8PagAAIHKFHD4OHDigXr16yeFw6O6779aWLVs0cuRIeb1excXFKTExMai/2+2W1+v92uMVFRXJ5XIFWkpKSsiDAAAAXUfI4WP48OHav3+/qqqqdM8992jWrFk6dOhQmwsoLCyUz+cLtLq6ujYfCwAAdH4hrfmQpLi4OF199dWSpIyMDO3Zs0e/+MUvNGPGDDU1NamhoSFo9qO+vl5JSUlfezyHwyGHwxF65QAAoEtq930+mpub1djYqIyMDMXGxqqsrCywr6amRrW1tfJ4PO09DQAAiBAhzXwUFhYqJydHqampOnXqlDZu3Kg33nhDO3fulMvl0pw5c1RQUKC+ffvK6XRq3rx58ng8XOkCAAACQgofJ0+e1N///d/rxIkTcrlcGjt2rHbu3Km//Mu/lCStWLFC0dHRysvLC7rJGAAAwJdCCh/r16//xv3x8fFas2aN1qxZ066iAABA5OK3XQAAgFWEDwAAYFXIl9oCnU13/zE+AOhqmPkAAABWET4AAIBVhA8AAGAVaz7QqbGeAwAiDzMfAADAKsIHAACwivABAACsYs2HRS2tXzhanBuGSgAACB9mPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEu4DubujC7eEuoVto6e/5aHFuGCoBADDzAQAArCJ8AAAAqwgfAADAqpDCR1FRka6//nr17t1bAwcO1LRp01RTUxPU5+zZs8rPz1e/fv3Uq1cv5eXlqb6+vkOLBgAAXVdI4aO8vFz5+fmqrKzUrl27dO7cOX3ve9/TmTNnAn0WLFigbdu2afPmzSovL9fx48c1ffr0Di8cAAB0TSFd7bJjx46gxxs2bNDAgQNVXV2tb3/72/L5fFq/fr02btyoKVOmSJJKS0s1YsQIVVZWauLEiR1XOQAA6JLatebD5/NJkvr27StJqq6u1rlz55SVlRXok56ertTUVFVUVLR4jMbGRvn9/qAGAAAiV5vDR3Nzs+bPn69JkyZp9OjRkiSv16u4uDglJiYG9XW73fJ6vS0ep6ioSC6XK9BSUlLaWhIAAOgC2hw+8vPzdfDgQW3atKldBRQWFsrn8wVaXV1du44HAAA6tzbd4fTee+/Vb3/7W7355psaPHhwYHtSUpKamprU0NAQNPtRX1+vpKSkFo/lcDjkcDjaUgYAAOiCQpr5MMbo3nvv1ZYtW/Taa68pLS0taH9GRoZiY2NVVlYW2FZTU6Pa2lp5PJ6OqRgAAHRpIc185Ofna+PGjfrNb36j3r17B9ZxuFwuJSQkyOVyac6cOSooKFDfvn3ldDo1b948eTwernQBAACSQgwfJSUlkqTvfve7QdtLS0t1xx13SJJWrFih6Oho5eXlqbGxUdnZ2Vq7dm2HFAsAALq+kMKHMeaSfeLj47VmzRqtWbOmzUUBAIDIxW+7AAAAqwgfAADAqjZdagtEgqELtwc9PlqcG/JzWvs8AMD/Y+YDAABYRfgAAABWET4AAIBVrPkAvkFLazwAAO3DzAcAALCK8AEAAKwifAAAAKtY8wH8H9Z3AIAdzHwAAACrCB8AAMAqwgcAALCK8AEAAKxiwSnQTm35gToA6M6Y+QAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWBUT6hPefPNNPfHEE6qurtaJEye0ZcsWTZs2LbDfGKMlS5bo6aefVkNDgyZNmqSSkhINGzasI+sGOq2hC7dfss/R4lwLlQBA5xTyzMeZM2c0btw4rVmzpsX9y5cv16pVq7Ru3TpVVVXpiiuuUHZ2ts6ePdvuYgEAQNcX8sxHTk6OcnJyWtxnjNHKlSv18MMPa+rUqZKkZ599Vm63W1u3btVtt93WvmoBAECX16FrPo4cOSKv16usrKzANpfLpczMTFVUVLT4nMbGRvn9/qAGAAAiV8gzH9/E6/VKktxud9B2t9sd2HehoqIiLV26tCPLACLWhetJWDsCoCsK+9UuhYWF8vl8gVZXVxfukgAAwGXUoeEjKSlJklRfXx+0vb6+PrDvQg6HQ06nM6gBAIDI1aHhIy0tTUlJSSorKwts8/v9qqqqksfj6chTAQCALirkNR+nT5/WRx99FHh85MgR7d+/X3379lVqaqrmz5+vRx55RMOGDVNaWpoWLVqk5OTkoHuBALi01twvpKU+rAMB0NmFHD727t2rv/iLvwg8LigokCTNmjVLGzZs0IMPPqgzZ85o7ty5amho0OTJk7Vjxw7Fx8d3XNUAAKDLCjl8fPe735Ux5mv3R0VFadmyZVq2bFm7CgMAAJEp7Fe7AACA7oXwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACs6tAfluvOWnNDKOCb8B4C0F0w8wEAAKwifAAAAKsIHwAAwCrWfLQC38UDLbvw3wY/agegNZj5AAAAVhE+AACAVYQPAABgVbdb88F31OgMbK4jaulcl+t9b/NcALouZj4AAIBVhA8AAGAV4QMAAFjV7dZ8tAb39UBX1pr3r821T6yzAnAhZj4AAIBVhA8AAGAV4QMAAFhF+AAAAFax4BRAq3TUQuyOuhEZNzQDui5mPgAAgFWEDwAAYBXhAwAAWMWaDwAtCveP312oo9aFtOa43enGaN1prOg8mPkAAABWET4AAIBVhA8AAGDVZVvzsWbNGj3xxBPyer0aN26cVq9erRtuuOFynQ5AO3SFH1Nsyw/m2dSa+45cznuTRMp9Ty7X+p/upCv8HV6WmY9///d/V0FBgZYsWaJ33nlH48aNU3Z2tk6ePHk5TgcAALqQyxI+nnzySd1111268847NXLkSK1bt049e/bUr371q8txOgAA0IV0+NcuTU1Nqq6uVmFhYWBbdHS0srKyVFFRcVH/xsZGNTY2Bh77fD5Jkt/v7+jSJEnNjX8OetzSeS7sAyByteYzoDWfRy19blz4vNb0aau2HrstY72cWvP5G+4aO7tw/R1+eUxjzKU7mw72ySefGEnm7bffDtr+wAMPmBtuuOGi/kuWLDGSaDQajUajRUCrq6u7ZFYI+03GCgsLVVBQEHjc3NysY8eO6dprr1VdXZ2cTmcYq7PP7/crJSWl2429u45bYuzdcezdddxS9x17dxi3MUanTp1ScnLyJft2ePjo37+/evToofr6+qDt9fX1SkpKuqi/w+GQw+EI2hYd/cVSFKfTGbEv0qV017F313FLjL07jr27jlvqvmOP9HG7XK5W9evwBadxcXHKyMhQWVlZYFtzc7PKysrk8Xg6+nQAAKCLuSxfuxQUFGjWrFmaMGGCbrjhBq1cuVJnzpzRnXfeeTlOBwAAupDLEj5mzJihP/zhD1q8eLG8Xq+uvfZa7dixQ263u1XPdzgcWrJkyUVfx3QH3XXs3XXcEmPvjmPvruOWuu/Yu+u4v06UMa25JgYAAKBj8NsuAADAKsIHAACwivABAACsInwAAACrOmX4WLNmjYYOHar4+HhlZmZq9+7d4S4p4M0339Qtt9yi5ORkRUVFaevWrUH7jTFavHixBg0apISEBGVlZenw4cNBfT799FPNnDlTTqdTiYmJmjNnjk6fPh3U57333tNNN92k+Ph4paSkaPny5RfVsnnzZqWnpys+Pl5jxozRyy+/HHItrVVUVKTrr79evXv31sCBAzVt2jTV1NQE9Tl79qzy8/PVr18/9erVS3l5eRfdbK62tla5ubnq2bOnBg4cqAceeECff/55UJ833nhD1113nRwOh66++mpt2LDhonou9R5pTS2tVVJSorFjxwZuDuTxePTKK69E/LgvVFxcrKioKM2fPz/ix/6zn/1MUVFRQS09PT3ixy1Jn3zyiX74wx+qX79+SkhI0JgxY7R3797A/kj9jBs6dOhFr3lUVJTy8/MlRfZrHhbt/zWXjrVp0yYTFxdnfvWrX5n333/f3HXXXSYxMdHU19eHuzRjjDEvv/yy+ad/+ifz61//2kgyW7ZsCdpfXFxsXC6X2bp1q3n33XfN3/zN35i0tDTz2WefBfr81V/9lRk3bpyprKw0//Vf/2Wuvvpqc/vttwf2+3w+43a7zcyZM83BgwfNCy+8YBISEswvf/nLQJ/f/e53pkePHmb58uXm0KFD5uGHHzaxsbHmwIEDIdXSWtnZ2aa0tNQcPHjQ7N+/3/z1X/+1SU1NNadPnw70ufvuu01KSoopKysze/fuNRMnTjQ33nhjYP/nn39uRo8ebbKyssy+ffvMyy+/bPr3728KCwsDfT7++GPTs2dPU1BQYA4dOmRWr15tevToYXbs2BHo05r3yKVqCcVLL71ktm/fbj788ENTU1NjfvrTn5rY2Fhz8ODBiB73V+3evdsMHTrUjB071tx3332tPl9XHfuSJUvMqFGjzIkTJwLtD3/4Q8SP+9NPPzVDhgwxd9xxh6mqqjIff/yx2blzp/noo48CfSL1M+7kyZNBr/euXbuMJPP6668bYyL3NQ+XThc+brjhBpOfnx94fP78eZOcnGyKiorCWFXLLgwfzc3NJikpyTzxxBOBbQ0NDcbhcJgXXnjBGGPMoUOHjCSzZ8+eQJ9XXnnFREVFmU8++cQYY8zatWtNnz59TGNjY6DPQw89ZIYPHx54/IMf/MDk5uYG1ZOZmWl+9KMftbqW9jh58qSRZMrLywPHjo2NNZs3bw70+eCDD4wkU1FRYYz5IrhFR0cbr9cb6FNSUmKcTmdgrA8++KAZNWpU0LlmzJhhsrOzA48v9R5pTS3t1adPH/Nv//Zv3WLcp06dMsOGDTO7du0y3/nOdwLhI5LHvmTJEjNu3LgW90XyuB966CEzefLkr93fnT7j7rvvPnPVVVeZ5ubmiH7Nw6VTfe3S1NSk6upqZWVlBbZFR0crKytLFRUVYaysdY4cOSKv1xtUv8vlUmZmZqD+iooKJSYmasKECYE+WVlZio6OVlVVVaDPt7/9bcXFxQX6ZGdnq6amRn/6058Cfb56ni/7fHme1tTSHj6fT5LUt29fSVJ1dbXOnTsXdL709HSlpqYGjX3MmDFBN5vLzs6W3+/X+++/36pxteY90ppa2ur8+fPatGmTzpw5I4/H0y3GnZ+fr9zc3Ivqi/SxHz58WMnJybryyis1c+ZM1dbWRvy4X3rpJU2YMEG33nqrBg4cqPHjx+vpp58O7O8un3FNTU167rnnNHv2bEVFRUX0ax4unSp8/M///I/Onz9/0Z1Q3W63vF5vmKpqvS9r/Kb6vV6vBg4cGLQ/JiZGffv2DerT0jG+eo6v6/PV/Zeqpa2am5s1f/58TZo0SaNHjw6cLy4uTomJid9YU1vH5ff79dlnn7XqPdKaWkJ14MAB9erVSw6HQ3fffbe2bNmikSNHRvy4N23apHfeeUdFRUUX7YvksWdmZmrDhg3asWOHSkpKdOTIEd100006depURI/7448/VklJiYYNG6adO3fqnnvu0U9+8hM988wzQbVH+mfc1q1b1dDQoDvuuCNwrkh9zcPlstxeHZEtPz9fBw8e1FtvvRXuUqwZPny49u/fL5/Pp//4j//QrFmzVF5eHu6yLqu6ujrdd9992rVrl+Lj48NdjlU5OTmBP48dO1aZmZkaMmSIXnzxRSUkJISxssurublZEyZM0GOPPSZJGj9+vA4ePKh169Zp1qxZYa7OnvXr1ysnJ6dVPw2PtulUMx/9+/dXjx49Llq1W19fr6SkpDBV1Xpf1vhN9SclJenkyZNB+z///HN9+umnQX1aOsZXz/F1fb66/1K1tMW9996r3/72t3r99dc1ePDgwPakpCQ1NTWpoaHhG2tq67icTqcSEhJa9R5pTS2hiouL09VXX62MjAwVFRVp3Lhx+sUvfhHR466urtbJkyd13XXXKSYmRjExMSovL9eqVasUExMjt9sdsWO/UGJioq655hp99NFHEf2aDxo0SCNHjgzaNmLEiMBXTt3hM+7YsWN69dVX9Q//8A+BbZH8modLpwofcXFxysjIUFlZWWBbc3OzysrK5PF4wlhZ66SlpSkpKSmofr/fr6qqqkD9Ho9HDQ0Nqq6uDvR57bXX1NzcrMzMzECfN998U+fOnQv02bVrl4YPH64+ffoE+nz1PF/2+fI8raklFMYY3XvvvdqyZYtee+01paWlBe3PyMhQbGxs0PlqampUW1sbNPYDBw4EfTDt2rVLTqcz8IF3qXG15j3Smlraq7m5WY2NjRE97ptvvlkHDhzQ/v37A23ChAmaOXNm4M+ROvYLnT59Wr///e81aNCgiH7NJ02adNEl9B9++KGGDBkiKbI/475UWlqqgQMHKjc3N7Atkl/zsAn3itcLbdq0yTgcDrNhwwZz6NAhM3fuXJOYmBi0gjicTp06Zfbt22f27dtnJJknn3zS7Nu3zxw7dswY88WlX4mJieY3v/mNee+998zUqVNbvAxt/Pjxpqqqyrz11ltm2LBhQZehNTQ0GLfbbf7u7/7OHDx40GzatMn07NnzosvQYmJizM9//nPzwQcfmCVLlrR4Gdqlammte+65x7hcLvPGG28EXY725z//OdDn7rvvNqmpqea1114ze/fuNR6Px3g8nsD+Ly9F+973vmf2799vduzYYQYMGNDipWgPPPCA+eCDD8yaNWtavBTtUu+RS9USioULF5ry8nJz5MgR895775mFCxeaqKgo85//+Z8RPe6WfPVql0ge+/3332/eeOMNc+TIEfO73/3OZGVlmf79+5uTJ09G9Lh3795tYmJizKOPPmoOHz5snn/+edOzZ0/z3HPPBfpE6mecMV9cWZKammoeeuihi/ZF6mseLp0ufBhjzOrVq01qaqqJi4szN9xwg6msrAx3SQGvv/66kXRRmzVrljHmi8u/Fi1aZNxut3E4HObmm282NTU1Qcf44x//aG6//XbTq1cv43Q6zZ133mlOnToV1Ofdd981kydPNg6Hw3zrW98yxcXFF9Xy4osvmmuuucbExcWZUaNGme3btwftb00trdXSmCWZ0tLSQJ/PPvvM/PjHPzZ9+vQxPXv2NH/7t39rTpw4EXSco0ePmpycHJOQkGD69+9v7r//fnPu3LmgPq+//rq59tprTVxcnLnyyiuDzvGlS71HWlNLa82ePdsMGTLExMXFmQEDBpibb745EDwiedwtuTB8ROrYZ8yYYQYNGmTi4uLMt771LTNjxoyge11E6riNMWbbtm1m9OjRxuFwmPT0dPPUU08F7Y/UzzhjjNm5c6eR1OIxIvk1D4coY4wJy5QLAADoljrVmg8AABD5CB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACs+l+sEMsHeu6JzgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(y_train, bins=100);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "How about looking at percent error? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302      3.851038\n",
       "767     11.564239\n",
       "429      8.535533\n",
       "1139   -16.371069\n",
       "558     17.177968\n",
       "          ...    \n",
       "1041    -0.496571\n",
       "1122   -28.696351\n",
       "1346    -6.577648\n",
       "1406    -2.358546\n",
       "1389     3.389230\n",
       "Name: SalePrice, Length: 1314, dtype: float64"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred_train = lr_tuned.predict(X_train)\n",
    "percent_errors = (pred_train - y_train) / y_train * 100.0\n",
    "percent_errors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are both positive (predict too high) and negative (predict too low)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can look at the absolute percent error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302      3.851038\n",
       "767     11.564239\n",
       "429      8.535533\n",
       "1139    16.371069\n",
       "558     17.177968\n",
       "          ...    \n",
       "1041     0.496571\n",
       "1122    28.696351\n",
       "1346     6.577648\n",
       "1406     2.358546\n",
       "1389     3.389230\n",
       "Name: SalePrice, Length: 1314, dtype: float64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.abs(percent_errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "And, like MSE, we can take the average over examples. This is called mean absolute percent error (MAPE)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_mape(true, pred):\n",
    "    return np.mean(np.abs((pred - true) / true))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10093121294225238"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_mape(y_train, pred_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use `sklearn` to calculate MAPE. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_absolute_percentage_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10093121294225238"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_percentage_error(y_train, pred_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Ok, this is quite interpretable.\n",
    "- On average, we have around 10% error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Transforming the targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- Does `.fit()` know we care about MAPE?\n",
    "- No, it doesn't. Why are we minimizing MSE (or something similar) if we care about MAPE??\n",
    "- When minimizing MSE, the expensive houses will dominate because they have the biggest error.\n",
    "- Which is better for RMSE?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Model A\n",
    "\n",
    "- Example 1: Truth: \\$50k, Prediction: \\\\$100k\n",
    "- Example 2: Truth: \\$500k, Prediction: \\\\$550k\n",
    "- RMSE: $50k\n",
    "- MAPE: 45%\n",
    "\n",
    "\n",
    "Model B\n",
    "\n",
    "- Example 1: Truth: \\$50k, Prediction: \\\\$60k\n",
    "- Example 2: Truth: \\$500k, Prediction: \\\\$600k\n",
    "- RMSE: $71k\n",
    "- MAPE: 20%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- How can we get `.fit()` to think about MAPE?\n",
    "- A common practice which tends to work is log transforming the targets.\n",
    "- That is, transform $y\\rightarrow \\log(y)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAn0ElEQVR4nO3df3SU1Z3H8U9CkkkQZsLPCSkJREXCbzFoGMG2i9lms6kLS45FD91FYaXaSIW4KukKFFZNxK5QWAjVpUGPIit7CpWisBg1rjUJEEFBPBErkKwwoVubGaCSILn7h+usA1EySbiTTN6vc+45zPPceZ7vZYbhc+7c55koY4wRAACAJdHhLgAAAHQvhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEu4ALNTc36/jx4+rdu7eioqLCXQ4AAGgFY4xOnTql5ORkRUd/89xGpwsfx48fV0pKSrjLAAAAbVBXV6fBgwd/Y59OFz569+4t6YvinU5nmKsBAACt4ff7lZKSEvh//Jt0uvDx5VctTqeT8AEAQBfTmiUTLDgFAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVMeEuAJfP0IXbgx4fLc4NUyUAAPw/Zj4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVoUUPs6fP69FixYpLS1NCQkJuuqqq/TP//zPMsYE+hhjtHjxYg0aNEgJCQnKysrS4cOHO7xwdIyhC7cHNQAALreQwsfjjz+ukpIS/eu//qs++OADPf7441q+fLlWr14d6LN8+XKtWrVK69atU1VVla644gplZ2fr7NmzHV48AADoekK6vfrbb7+tqVOnKjf3i9t0Dx06VC+88IJ2794t6YtZj5UrV+rhhx/W1KlTJUnPPvus3G63tm7dqttuu62DywcAAF1NSDMfN954o8rKyvThhx9Kkt5991299dZbysnJkSQdOXJEXq9XWVlZgee4XC5lZmaqoqKixWM2NjbK7/cHNQAAELlCmvlYuHCh/H6/0tPT1aNHD50/f16PPvqoZs6cKUnyer2SJLfbHfQ8t9sd2HehoqIiLV26tC21AwCALiikmY8XX3xRzz//vDZu3Kh33nlHzzzzjH7+85/rmWeeaXMBhYWF8vl8gVZXV9fmYwEAgM4vpJmPBx54QAsXLgys3RgzZoyOHTumoqIizZo1S0lJSZKk+vp6DRo0KPC8+vp6XXvttS0e0+FwyOFwtLF8AADQ1YQ08/HnP/9Z0dHBT+nRo4eam5slSWlpaUpKSlJZWVlgv9/vV1VVlTweTweUCwAAurqQZj5uueUWPfroo0pNTdWoUaO0b98+Pfnkk5o9e7YkKSoqSvPnz9cjjzyiYcOGKS0tTYsWLVJycrKmTZt2OepHCLiPBwCgMwgpfKxevVqLFi3Sj3/8Y508eVLJycn60Y9+pMWLFwf6PPjggzpz5ozmzp2rhoYGTZ48WTt27FB8fHyHFw8AALqeKPPV25N2An6/Xy6XSz6fT06nM9zldGltmek4Wpx7GSoBAES6UP7/5rddAACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFgVUvgYOnSooqKiLmr5+fmSpLNnzyo/P1/9+vVTr169lJeXp/r6+stSOAAA6JpCCh979uzRiRMnAm3Xrl2SpFtvvVWStGDBAm3btk2bN29WeXm5jh8/runTp3d81QAAoMuKCaXzgAEDgh4XFxfrqquu0ne+8x35fD6tX79eGzdu1JQpUyRJpaWlGjFihCorKzVx4sSOqxoAAHRZbV7z0dTUpOeee06zZ89WVFSUqqurde7cOWVlZQX6pKenKzU1VRUVFV97nMbGRvn9/qAGAAAiV5vDx9atW9XQ0KA77rhDkuT1ehUXF6fExMSgfm63W16v92uPU1RUJJfLFWgpKSltLQkAAHQBbQ4f69evV05OjpKTk9tVQGFhoXw+X6DV1dW163gAAKBzC2nNx5eOHTumV199Vb/+9a8D25KSktTU1KSGhoag2Y/6+nolJSV97bEcDoccDkdbygAAAF1Qm2Y+SktLNXDgQOXm5ga2ZWRkKDY2VmVlZYFtNTU1qq2tlcfjaX+lAAAgIoQ889Hc3KzS0lLNmjVLMTH//3SXy6U5c+aooKBAffv2ldPp1Lx58+TxeLjSBQAABIQcPl599VXV1tZq9uzZF+1bsWKFoqOjlZeXp8bGRmVnZ2vt2rUdUigAAIgMUcYYE+4ivsrv98vlcsnn88npdIa7nC5t6MLtIT/naHHupTsBAHCBUP7/5rddAACAVYQPAABgVZsutUXn05avWAAACAdmPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEuwBEhqELtwc9PlqcG6ZKAACdHTMfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAq0IOH5988ol++MMfql+/fkpISNCYMWO0d+/ewH5jjBYvXqxBgwYpISFBWVlZOnz4cIcWjc5v6MLtFzUAAKQQw8ef/vQnTZo0SbGxsXrllVd06NAh/cu//Iv69OkT6LN8+XKtWrVK69atU1VVla644gplZ2fr7NmzHV48AADoemJC6fz4448rJSVFpaWlgW1paWmBPxtjtHLlSj388MOaOnWqJOnZZ5+V2+3W1q1bddttt3VQ2QAAoKsKaebjpZde0oQJE3Trrbdq4MCBGj9+vJ5++unA/iNHjsjr9SorKyuwzeVyKTMzUxUVFS0es7GxUX6/P6gBAIDIFdLMx8cff6ySkhIVFBTopz/9qfbs2aOf/OQniouL06xZs+T1eiVJbrc76Hlutzuw70JFRUVaunRpG8vvnlg/AQDoykKa+WhubtZ1112nxx57TOPHj9fcuXN11113ad26dW0uoLCwUD6fL9Dq6urafCwAAND5hRQ+Bg0apJEjRwZtGzFihGprayVJSUlJkqT6+vqgPvX19YF9F3I4HHI6nUENAABErpDCx6RJk1RTUxO07cMPP9SQIUMkfbH4NCkpSWVlZYH9fr9fVVVV8ng8HVAuAADo6kJa87FgwQLdeOONeuyxx/SDH/xAu3fv1lNPPaWnnnpKkhQVFaX58+frkUce0bBhw5SWlqZFixYpOTlZ06ZNuxz1AwCALiak8HH99ddry5YtKiws1LJly5SWlqaVK1dq5syZgT4PPvigzpw5o7lz56qhoUGTJ0/Wjh07FB8f3+HFAwCAriek8CFJ3//+9/X973//a/dHRUVp2bJlWrZsWbsKAwAAkYnfdgEAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVSFfaovu58IfsjtanBumSgAAkYCZDwAAYBXhAwAAWEX4AAAAVrHmA0EuXN8BAEBHY+YDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGBVTLgLwKUNXbg93CUAANBhmPkAAABWET4AAIBVhA8AAGBVSOHjZz/7maKiooJaenp6YP/Zs2eVn5+vfv36qVevXsrLy1N9fX2HF42uaejC7UENANA9hTzzMWrUKJ04cSLQ3nrrrcC+BQsWaNu2bdq8ebPKy8t1/PhxTZ8+vUMLBgAAXVvIV7vExMQoKSnpou0+n0/r16/Xxo0bNWXKFElSaWmpRowYocrKSk2cOLH91QIAgC4v5JmPw4cPKzk5WVdeeaVmzpyp2tpaSVJ1dbXOnTunrKysQN/09HSlpqaqoqLia4/X2Ngov98f1AAAQOQKaeYjMzNTGzZs0PDhw3XixAktXbpUN910kw4ePCiv16u4uDglJiYGPcftdsvr9X7tMYuKirR06dI2FY/wYL0GAKA9QgofOTk5gT+PHTtWmZmZGjJkiF588UUlJCS0qYDCwkIVFBQEHvv9fqWkpLTpWAAAoPNr16W2iYmJuuaaa/TRRx8pKSlJTU1NamhoCOpTX1/f4hqRLzkcDjmdzqAGAAAiV7vCx+nTp/X73/9egwYNUkZGhmJjY1VWVhbYX1NTo9raWnk8nnYXCgAAIkNIX7v84z/+o2655RYNGTJEx48f15IlS9SjRw/dfvvtcrlcmjNnjgoKCtS3b185nU7NmzdPHo+HK10AAEBASOHjv//7v3X77bfrj3/8owYMGKDJkyersrJSAwYMkCStWLFC0dHRysvLU2Njo7Kzs7V27drLUjgAAOiaoowxJtxFfJXf75fL5ZLP52P9x/+J1KtLjhbnhrsEAEAHCeX/b37bBQAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBV7QofxcXFioqK0vz58wPbzp49q/z8fPXr10+9evVSXl6e6uvr21snItDQhdsvagCAyNfm8LFnzx798pe/1NixY4O2L1iwQNu2bdPmzZtVXl6u48ePa/r06e0uFAAARIY2hY/Tp09r5syZevrpp9WnT5/Adp/Pp/Xr1+vJJ5/UlClTlJGRodLSUr399tuqrKzssKIBAEDX1abwkZ+fr9zcXGVlZQVtr66u1rlz54K2p6enKzU1VRUVFe2rFAAARISYUJ+wadMmvfPOO9qzZ89F+7xer+Li4pSYmBi03e12y+v1tni8xsZGNTY2Bh77/f5QSwIAAF1ISDMfdXV1uu+++/T8888rPj6+QwooKiqSy+UKtJSUlA45LgAA6JxCCh/V1dU6efKkrrvuOsXExCgmJkbl5eVatWqVYmJi5Ha71dTUpIaGhqDn1dfXKykpqcVjFhYWyufzBVpdXV2bBwMAADq/kL52ufnmm3XgwIGgbXfeeafS09P10EMPKSUlRbGxsSorK1NeXp4kqaamRrW1tfJ4PC0e0+FwyOFwtLF8AADQ1YQUPnr37q3Ro0cHbbviiivUr1+/wPY5c+aooKBAffv2ldPp1Lx58+TxeDRx4sSOqxoAAHRZIS84vZQVK1YoOjpaeXl5amxsVHZ2ttauXdvRpwEAAF1UlDHGhLuIr/L7/XK5XPL5fHI6neEup1PoTnf+PFqcG+4SAABtEMr/3/y2CwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMCqmFA6l5SUqKSkREePHpUkjRo1SosXL1ZOTo4k6ezZs7r//vu1adMmNTY2Kjs7W2vXrpXb7e7wwruCoQu3Bz0+Wpwb8nO6m7b8nV3O4wAAOl5IMx+DBw9WcXGxqqurtXfvXk2ZMkVTp07V+++/L0lasGCBtm3bps2bN6u8vFzHjx/X9OnTL0vhAACgawpp5uOWW24Jevzoo4+qpKRElZWVGjx4sNavX6+NGzdqypQpkqTS0lKNGDFClZWVmjhxYsdVDQAAuqw2r/k4f/68Nm3apDNnzsjj8ai6ulrnzp1TVlZWoE96erpSU1NVUVHxtcdpbGyU3+8PagAAIHKFHD4OHDigXr16yeFw6O6779aWLVs0cuRIeb1excXFKTExMai/2+2W1+v92uMVFRXJ5XIFWkpKSsiDAAAAXUfI4WP48OHav3+/qqqqdM8992jWrFk6dOhQmwsoLCyUz+cLtLq6ujYfCwAAdH4hrfmQpLi4OF199dWSpIyMDO3Zs0e/+MUvNGPGDDU1NamhoSFo9qO+vl5JSUlfezyHwyGHwxF65QAAoEtq930+mpub1djYqIyMDMXGxqqsrCywr6amRrW1tfJ4PO09DQAAiBAhzXwUFhYqJydHqampOnXqlDZu3Kg33nhDO3fulMvl0pw5c1RQUKC+ffvK6XRq3rx58ng8XOkCAAACQgofJ0+e1N///d/rxIkTcrlcGjt2rHbu3Km//Mu/lCStWLFC0dHRysvLC7rJGAAAwJdCCh/r16//xv3x8fFas2aN1qxZ066iAABA5OK3XQAAgFWEDwAAYFXIl9oCnU13/zE+AOhqmPkAAABWET4AAIBVhA8AAGAVaz7QqbGeAwAiDzMfAADAKsIHAACwivABAACsYs2HRS2tXzhanBuGSgAACB9mPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVsWEu4DubujC7eEuoVto6e/5aHFuGCoBADDzAQAArCJ8AAAAqwgfAADAqpDCR1FRka6//nr17t1bAwcO1LRp01RTUxPU5+zZs8rPz1e/fv3Uq1cv5eXlqb6+vkOLBgAAXVdI4aO8vFz5+fmqrKzUrl27dO7cOX3ve9/TmTNnAn0WLFigbdu2afPmzSovL9fx48c1ffr0Di8cAAB0TSFd7bJjx46gxxs2bNDAgQNVXV2tb3/72/L5fFq/fr02btyoKVOmSJJKS0s1YsQIVVZWauLEiR1XOQAA6JLatebD5/NJkvr27StJqq6u1rlz55SVlRXok56ertTUVFVUVLR4jMbGRvn9/qAGAAAiV5vDR3Nzs+bPn69JkyZp9OjRkiSv16u4uDglJiYG9XW73fJ6vS0ep6ioSC6XK9BSUlLaWhIAAOgC2hw+8vPzdfDgQW3atKldBRQWFsrn8wVaXV1du44HAAA6tzbd4fTee+/Vb3/7W7355psaPHhwYHtSUpKamprU0NAQNPtRX1+vpKSkFo/lcDjkcDjaUgYAAOiCQpr5MMbo3nvv1ZYtW/Taa68pLS0taH9GRoZiY2NVVlYW2FZTU6Pa2lp5PJ6OqRgAAHRpIc185Ofna+PGjfrNb36j3r17B9ZxuFwuJSQkyOVyac6cOSooKFDfvn3ldDo1b948eTwernQBAACSQgwfJSUlkqTvfve7QdtLS0t1xx13SJJWrFih6Oho5eXlqbGxUdnZ2Vq7dm2HFAsAALq+kMKHMeaSfeLj47VmzRqtWbOmzUUBAIDIxW+7AAAAqwgfAADAqjZdagtEgqELtwc9PlqcG/JzWvs8AMD/Y+YDAABYRfgAAABWET4AAIBVrPkAvkFLazwAAO3DzAcAALCK8AEAAKwifAAAAKtY8wH8H9Z3AIAdzHwAAACrCB8AAMAqwgcAALCK8AEAAKxiwSnQTm35gToA6M6Y+QAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWBUT6hPefPNNPfHEE6qurtaJEye0ZcsWTZs2LbDfGKMlS5bo6aefVkNDgyZNmqSSkhINGzasI+sGOq2hC7dfss/R4lwLlQBA5xTyzMeZM2c0btw4rVmzpsX9y5cv16pVq7Ru3TpVVVXpiiuuUHZ2ts6ePdvuYgEAQNcX8sxHTk6OcnJyWtxnjNHKlSv18MMPa+rUqZKkZ599Vm63W1u3btVtt93WvmoBAECX16FrPo4cOSKv16usrKzANpfLpczMTFVUVLT4nMbGRvn9/qAGAAAiV8gzH9/E6/VKktxud9B2t9sd2HehoqIiLV26tCPLACLWhetJWDsCoCsK+9UuhYWF8vl8gVZXVxfukgAAwGXUoeEjKSlJklRfXx+0vb6+PrDvQg6HQ06nM6gBAIDI1aHhIy0tTUlJSSorKwts8/v9qqqqksfj6chTAQCALirkNR+nT5/WRx99FHh85MgR7d+/X3379lVqaqrmz5+vRx55RMOGDVNaWpoWLVqk5OTkoHuBALi01twvpKU+rAMB0NmFHD727t2rv/iLvwg8LigokCTNmjVLGzZs0IMPPqgzZ85o7ty5amho0OTJk7Vjxw7Fx8d3XNUAAKDLCjl8fPe735Ux5mv3R0VFadmyZVq2bFm7CgMAAJEp7Fe7AACA7oXwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACs6tAfluvOWnNDKOCb8B4C0F0w8wEAAKwifAAAAKsIHwAAwCrWfLQC38UDLbvw3wY/agegNZj5AAAAVhE+AACAVYQPAABgVbdb88F31OgMbK4jaulcl+t9b/NcALouZj4AAIBVhA8AAGAV4QMAAFjV7dZ8tAb39UBX1pr3r821T6yzAnAhZj4AAIBVhA8AAGAV4QMAAFhF+AAAAFax4BRAq3TUQuyOuhEZNzQDui5mPgAAgFWEDwAAYBXhAwAAWMWaDwAtCveP312oo9aFtOa43enGaN1prOg8mPkAAABWET4AAIBVhA8AAGDVZVvzsWbNGj3xxBPyer0aN26cVq9erRtuuOFynQ5AO3SFH1Nsyw/m2dSa+45cznuTRMp9Ty7X+p/upCv8HV6WmY9///d/V0FBgZYsWaJ33nlH48aNU3Z2tk6ePHk5TgcAALqQyxI+nnzySd1111268847NXLkSK1bt049e/bUr371q8txOgAA0IV0+NcuTU1Nqq6uVmFhYWBbdHS0srKyVFFRcVH/xsZGNTY2Bh77fD5Jkt/v7+jSJEnNjX8OetzSeS7sAyByteYzoDWfRy19blz4vNb0aau2HrstY72cWvP5G+4aO7tw/R1+eUxjzKU7mw72ySefGEnm7bffDtr+wAMPmBtuuOGi/kuWLDGSaDQajUajRUCrq6u7ZFYI+03GCgsLVVBQEHjc3NysY8eO6dprr1VdXZ2cTmcYq7PP7/crJSWl2429u45bYuzdcezdddxS9x17dxi3MUanTp1ScnLyJft2ePjo37+/evToofr6+qDt9fX1SkpKuqi/w+GQw+EI2hYd/cVSFKfTGbEv0qV017F313FLjL07jr27jlvqvmOP9HG7XK5W9evwBadxcXHKyMhQWVlZYFtzc7PKysrk8Xg6+nQAAKCLuSxfuxQUFGjWrFmaMGGCbrjhBq1cuVJnzpzRnXfeeTlOBwAAupDLEj5mzJihP/zhD1q8eLG8Xq+uvfZa7dixQ263u1XPdzgcWrJkyUVfx3QH3XXs3XXcEmPvjmPvruOWuu/Yu+u4v06UMa25JgYAAKBj8NsuAADAKsIHAACwivABAACsInwAAACrOmX4WLNmjYYOHar4+HhlZmZq9+7d4S4p4M0339Qtt9yi5ORkRUVFaevWrUH7jTFavHixBg0apISEBGVlZenw4cNBfT799FPNnDlTTqdTiYmJmjNnjk6fPh3U57333tNNN92k+Ph4paSkaPny5RfVsnnzZqWnpys+Pl5jxozRyy+/HHItrVVUVKTrr79evXv31sCBAzVt2jTV1NQE9Tl79qzy8/PVr18/9erVS3l5eRfdbK62tla5ubnq2bOnBg4cqAceeECff/55UJ833nhD1113nRwOh66++mpt2LDhonou9R5pTS2tVVJSorFjxwZuDuTxePTKK69E/LgvVFxcrKioKM2fPz/ix/6zn/1MUVFRQS09PT3ixy1Jn3zyiX74wx+qX79+SkhI0JgxY7R3797A/kj9jBs6dOhFr3lUVJTy8/MlRfZrHhbt/zWXjrVp0yYTFxdnfvWrX5n333/f3HXXXSYxMdHU19eHuzRjjDEvv/yy+ad/+ifz61//2kgyW7ZsCdpfXFxsXC6X2bp1q3n33XfN3/zN35i0tDTz2WefBfr81V/9lRk3bpyprKw0//Vf/2Wuvvpqc/vttwf2+3w+43a7zcyZM83BgwfNCy+8YBISEswvf/nLQJ/f/e53pkePHmb58uXm0KFD5uGHHzaxsbHmwIEDIdXSWtnZ2aa0tNQcPHjQ7N+/3/z1X/+1SU1NNadPnw70ufvuu01KSoopKysze/fuNRMnTjQ33nhjYP/nn39uRo8ebbKyssy+ffvMyy+/bPr3728KCwsDfT7++GPTs2dPU1BQYA4dOmRWr15tevToYXbs2BHo05r3yKVqCcVLL71ktm/fbj788ENTU1NjfvrTn5rY2Fhz8ODBiB73V+3evdsMHTrUjB071tx3332tPl9XHfuSJUvMqFGjzIkTJwLtD3/4Q8SP+9NPPzVDhgwxd9xxh6mqqjIff/yx2blzp/noo48CfSL1M+7kyZNBr/euXbuMJPP6668bYyL3NQ+XThc+brjhBpOfnx94fP78eZOcnGyKiorCWFXLLgwfzc3NJikpyTzxxBOBbQ0NDcbhcJgXXnjBGGPMoUOHjCSzZ8+eQJ9XXnnFREVFmU8++cQYY8zatWtNnz59TGNjY6DPQw89ZIYPHx54/IMf/MDk5uYG1ZOZmWl+9KMftbqW9jh58qSRZMrLywPHjo2NNZs3bw70+eCDD4wkU1FRYYz5IrhFR0cbr9cb6FNSUmKcTmdgrA8++KAZNWpU0LlmzJhhsrOzA48v9R5pTS3t1adPH/Nv//Zv3WLcp06dMsOGDTO7du0y3/nOdwLhI5LHvmTJEjNu3LgW90XyuB966CEzefLkr93fnT7j7rvvPnPVVVeZ5ubmiH7Nw6VTfe3S1NSk6upqZWVlBbZFR0crKytLFRUVYaysdY4cOSKv1xtUv8vlUmZmZqD+iooKJSYmasKECYE+WVlZio6OVlVVVaDPt7/9bcXFxQX6ZGdnq6amRn/6058Cfb56ni/7fHme1tTSHj6fT5LUt29fSVJ1dbXOnTsXdL709HSlpqYGjX3MmDFBN5vLzs6W3+/X+++/36pxteY90ppa2ur8+fPatGmTzpw5I4/H0y3GnZ+fr9zc3Ivqi/SxHz58WMnJybryyis1c+ZM1dbWRvy4X3rpJU2YMEG33nqrBg4cqPHjx+vpp58O7O8un3FNTU167rnnNHv2bEVFRUX0ax4unSp8/M///I/Onz9/0Z1Q3W63vF5vmKpqvS9r/Kb6vV6vBg4cGLQ/JiZGffv2DerT0jG+eo6v6/PV/Zeqpa2am5s1f/58TZo0SaNHjw6cLy4uTomJid9YU1vH5ff79dlnn7XqPdKaWkJ14MAB9erVSw6HQ3fffbe2bNmikSNHRvy4N23apHfeeUdFRUUX7YvksWdmZmrDhg3asWOHSkpKdOTIEd100006depURI/7448/VklJiYYNG6adO3fqnnvu0U9+8hM988wzQbVH+mfc1q1b1dDQoDvuuCNwrkh9zcPlstxeHZEtPz9fBw8e1FtvvRXuUqwZPny49u/fL5/Pp//4j//QrFmzVF5eHu6yLqu6ujrdd9992rVrl+Lj48NdjlU5OTmBP48dO1aZmZkaMmSIXnzxRSUkJISxssurublZEyZM0GOPPSZJGj9+vA4ePKh169Zp1qxZYa7OnvXr1ysnJ6dVPw2PtulUMx/9+/dXjx49Llq1W19fr6SkpDBV1Xpf1vhN9SclJenkyZNB+z///HN9+umnQX1aOsZXz/F1fb66/1K1tMW9996r3/72t3r99dc1ePDgwPakpCQ1NTWpoaHhG2tq67icTqcSEhJa9R5pTS2hiouL09VXX62MjAwVFRVp3Lhx+sUvfhHR466urtbJkyd13XXXKSYmRjExMSovL9eqVasUExMjt9sdsWO/UGJioq655hp99NFHEf2aDxo0SCNHjgzaNmLEiMBXTt3hM+7YsWN69dVX9Q//8A+BbZH8modLpwofcXFxysjIUFlZWWBbc3OzysrK5PF4wlhZ66SlpSkpKSmofr/fr6qqqkD9Ho9HDQ0Nqq6uDvR57bXX1NzcrMzMzECfN998U+fOnQv02bVrl4YPH64+ffoE+nz1PF/2+fI8raklFMYY3XvvvdqyZYtee+01paWlBe3PyMhQbGxs0PlqampUW1sbNPYDBw4EfTDt2rVLTqcz8IF3qXG15j3Smlraq7m5WY2NjRE97ptvvlkHDhzQ/v37A23ChAmaOXNm4M+ROvYLnT59Wr///e81aNCgiH7NJ02adNEl9B9++KGGDBkiKbI/475UWlqqgQMHKjc3N7Atkl/zsAn3itcLbdq0yTgcDrNhwwZz6NAhM3fuXJOYmBi0gjicTp06Zfbt22f27dtnJJknn3zS7Nu3zxw7dswY88WlX4mJieY3v/mNee+998zUqVNbvAxt/Pjxpqqqyrz11ltm2LBhQZehNTQ0GLfbbf7u7/7OHDx40GzatMn07NnzosvQYmJizM9//nPzwQcfmCVLlrR4Gdqlammte+65x7hcLvPGG28EXY725z//OdDn7rvvNqmpqea1114ze/fuNR6Px3g8nsD+Ly9F+973vmf2799vduzYYQYMGNDipWgPPPCA+eCDD8yaNWtavBTtUu+RS9USioULF5ry8nJz5MgR895775mFCxeaqKgo85//+Z8RPe6WfPVql0ge+/3332/eeOMNc+TIEfO73/3OZGVlmf79+5uTJ09G9Lh3795tYmJizKOPPmoOHz5snn/+edOzZ0/z3HPPBfpE6mecMV9cWZKammoeeuihi/ZF6mseLp0ufBhjzOrVq01qaqqJi4szN9xwg6msrAx3SQGvv/66kXRRmzVrljHmi8u/Fi1aZNxut3E4HObmm282NTU1Qcf44x//aG6//XbTq1cv43Q6zZ133mlOnToV1Ofdd981kydPNg6Hw3zrW98yxcXFF9Xy4osvmmuuucbExcWZUaNGme3btwftb00trdXSmCWZ0tLSQJ/PPvvM/PjHPzZ9+vQxPXv2NH/7t39rTpw4EXSco0ePmpycHJOQkGD69+9v7r//fnPu3LmgPq+//rq59tprTVxcnLnyyiuDzvGlS71HWlNLa82ePdsMGTLExMXFmQEDBpibb745EDwiedwtuTB8ROrYZ8yYYQYNGmTi4uLMt771LTNjxoyge11E6riNMWbbtm1m9OjRxuFwmPT0dPPUU08F7Y/UzzhjjNm5c6eR1OIxIvk1D4coY4wJy5QLAADoljrVmg8AABD5CB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACs+l+sEMsHeu6JzgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(y_train, bins=100);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAft0lEQVR4nO3df3BU1f3/8Vcwv6hkExLJJpEgaJGgiFVQWH58qhhNKaUwpFUxVXSojDZSSeoPYtEYRBMdFXQMoIhQZ0xpsZWKRajGglUSxDC0qJ0IAk007NKqSSBONpHc7x/9unZDgNxk92x2eT5m7ox77s3dN8fNzWvOnnNvlGVZlgAAAAzpF+oCAADA6YXwAQAAjCJ8AAAAowgfAADAKMIHAAAwivABAACMInwAAACjCB8AAMCo6FAX0FlHR4caGhqUkJCgqKioUJcDAAC6wbIsHTlyRBkZGerX7+RjG30ufDQ0NCgzMzPUZQAAgB6or6/X4MGDT3pMnwsfCQkJkv5bvMPhCHE1AACgO5qbm5WZmen7O34yfS58fPNVi8PhIHwAABBmujNlggmnAADAKMIHAAAwivABAACMInwAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrwAQAAjCJ8AAAAowgfAADAKMIHAAAwKjrUBQAIvaEL/3xc28GyaSGoBMDpgJEPAABglO3w8dlnn+lnP/uZUlJS1L9/f1100UV6//33ffsty9IDDzyg9PR09e/fX9nZ2dq7d29AiwYAAOHLVvj48ssvNXHiRMXExOj111/XRx99pCeeeEIDBw70HfPYY4/p6aef1sqVK7Vjxw6deeaZysnJUWtra8CLBwAA4cfWnI9HH31UmZmZWrNmja9t2LBhvv+2LEvLli3TokWLNGPGDEnSiy++KKfTqQ0bNuj6668PUNkAACBc2Rr5ePXVVzV27Fj99Kc/VWpqqi655BKtWrXKt//AgQNyu93Kzs72tSUmJmrcuHGqqqrq8pxer1fNzc1+GwAAiFy2wsf+/fu1YsUKDR8+XFu2bNHtt9+uX/7yl/rNb34jSXK73ZIkp9Pp93NOp9O3r7PS0lIlJib6tszMzJ78OwAAQJiwFT46Ojp06aWX6pFHHtEll1yiefPm6dZbb9XKlSt7XEBRUZGampp8W319fY/PBQAA+j5b4SM9PV0XXHCBX9vIkSNVV1cnSUpLS5MkeTwev2M8Ho9vX2dxcXFyOBx+GwAAiFy2wsfEiRNVW1vr1/bxxx/rnHPOkfTfyadpaWmqrKz07W9ubtaOHTvkcrkCUC4AAAh3tla7FBQUaMKECXrkkUd07bXX6r333tNzzz2n5557TpIUFRWlBQsWaMmSJRo+fLiGDRum+++/XxkZGZo5c2Yw6gcAAGHGVvi47LLL9Morr6ioqEiLFy/WsGHDtGzZMuXl5fmOueeee9TS0qJ58+apsbFRkyZN0ubNmxUfHx/w4gEAQPiJsizLCnUR/6u5uVmJiYlqampi/gdgCM92AdBbdv5+82wXAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYFR0qAsAgM6GLvzzcW0Hy6aFoBIAwcDIBwAAMIrwAQAAjCJ8AAAAowgfAADAKMIHAAAwitUuQIRj5QiAvoaRDwAAYBThAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABG2QofDz74oKKiovy2rKws3/7W1lbl5+crJSVFAwYMUG5urjweT8CLBgAA4cv2yMeFF16oQ4cO+bZ33nnHt6+goEAbN27U+vXrtW3bNjU0NGjWrFkBLRgAAIQ323c4jY6OVlpa2nHtTU1NWr16tSoqKjRlyhRJ0po1azRy5EhVV1dr/Pjxva8WAACEPdsjH3v37lVGRobOPfdc5eXlqa6uTpJUU1Oj9vZ2ZWdn+47NysrSkCFDVFVVdcLzeb1eNTc3+20AACBy2Rr5GDdunNauXasRI0bo0KFDKikp0eTJk/XBBx/I7XYrNjZWSUlJfj/jdDrldrtPeM7S0lKVlJT0qHgA5vCMGACBYit8TJ061fffo0eP1rhx43TOOefo97//vfr379+jAoqKilRYWOh73dzcrMzMzB6dCwAA9H29WmqblJSk888/X/v27VNaWpra2trU2Njod4zH4+lyjsg34uLi5HA4/DYAABC5ehU+jh49qk8++UTp6ekaM2aMYmJiVFlZ6dtfW1ururo6uVyuXhcKAAAig62vXe666y5Nnz5d55xzjhoaGlRcXKwzzjhDs2fPVmJioubOnavCwkIlJyfL4XBo/vz5crlcrHQBAAA+tsLHp59+qtmzZ+vzzz/XoEGDNGnSJFVXV2vQoEGSpKVLl6pfv37Kzc2V1+tVTk6Oli9fHpTCAQBAeLIVPtatW3fS/fHx8SovL1d5eXmvigIAAJGLZ7sAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrwAQAAjCJ8AAAAowgfAADAKFu3Vwdw+hi68M+hLgFAhGLkAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFE82wVAWOrq2TMHy6aFoBIAdjHyAQAAjCJ8AAAAowgfAADAKMIHAAAwivABAACMInwAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKO4vTqAoOp8G/Se3gK9q9upAwhPjHwAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKNY7QIgYFiRAqA7GPkAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFG9Ch9lZWWKiorSggULfG2tra3Kz89XSkqKBgwYoNzcXHk8nt7WCQAAIkSPw8fOnTv17LPPavTo0X7tBQUF2rhxo9avX69t27apoaFBs2bN6nWhAAAgMvQofBw9elR5eXlatWqVBg4c6GtvamrS6tWr9eSTT2rKlCkaM2aM1qxZo+3bt6u6ujpgRQMAgPDVo/CRn5+vadOmKTs726+9pqZG7e3tfu1ZWVkaMmSIqqqqujyX1+tVc3Oz3wYAACKX7durr1u3Trt27dLOnTuP2+d2uxUbG6ukpCS/dqfTKbfb3eX5SktLVVJSYrcMADhO59u7HyybFqJKAJyMrZGP+vp63XnnnXrppZcUHx8fkAKKiorU1NTk2+rr6wNyXgAA0DfZCh81NTU6fPiwLr30UkVHRys6Olrbtm3T008/rejoaDmdTrW1tamxsdHv5zwej9LS0ro8Z1xcnBwOh98GAAAil62vXa666irt2bPHr+2WW25RVlaW7r33XmVmZiomJkaVlZXKzc2VJNXW1qqurk4ulytwVQMAgLBlK3wkJCRo1KhRfm1nnnmmUlJSfO1z585VYWGhkpOT5XA4NH/+fLlcLo0fPz5wVQMAgLBle8LpqSxdulT9+vVTbm6uvF6vcnJytHz58kC/DQAACFO9Dh9bt271ex0fH6/y8nKVl5f39tQAACAC8WwXAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGBUwJfaAuj7Oj8DBQBMYuQDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABjFahcAIcfqG+D0wsgHAAAwivABAACMInwAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrwAQAAjOL26gCM4lbqABj5AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGsdoFQI+xcgVATzDyAQAAjCJ8AAAAowgfAADAKMIHAAAwivABAACMYrULAARAVyt/DpZNC0ElQN/HyAcAADCK8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrVLkAf0Xm1BCsleo8VKEDfxMgHAAAwivABAACMInwAAACjCB8AAMAoJpwCYaQ7k1K7mmQJAH0JIx8AAMAoW+FjxYoVGj16tBwOhxwOh1wul15//XXf/tbWVuXn5yslJUUDBgxQbm6uPB5PwIsGAADhy1b4GDx4sMrKylRTU6P3339fU6ZM0YwZM/Thhx9KkgoKCrRx40atX79e27ZtU0NDg2bNmhWUwgEAQHiyNedj+vTpfq8ffvhhrVixQtXV1Ro8eLBWr16tiooKTZkyRZK0Zs0ajRw5UtXV1Ro/fnzgqgYAAGGrx3M+jh07pnXr1qmlpUUul0s1NTVqb29Xdna275isrCwNGTJEVVVVJzyP1+tVc3Oz3wYAACKX7fCxZ88eDRgwQHFxcbrtttv0yiuv6IILLpDb7VZsbKySkpL8jnc6nXK73Sc8X2lpqRITE31bZmam7X8EAAAIH7bDx4gRI7R7927t2LFDt99+u+bMmaOPPvqoxwUUFRWpqanJt9XX1/f4XAAAoO+zfZ+P2NhYffe735UkjRkzRjt37tRTTz2l6667Tm1tbWpsbPQb/fB4PEpLSzvh+eLi4hQXF2e/cgAAEJZ6fZ+Pjo4Oeb1ejRkzRjExMaqsrPTtq62tVV1dnVwuV2/fBgAARAhbIx9FRUWaOnWqhgwZoiNHjqiiokJbt27Vli1blJiYqLlz56qwsFDJyclyOByaP3++XC4XK10AAICPrfBx+PBh3XTTTTp06JASExM1evRobdmyRVdffbUkaenSperXr59yc3Pl9XqVk5Oj5cuXB6VwAAAQnmyFj9WrV590f3x8vMrLy1VeXt6rogB0D89xARCOeLYLAAAwivABAACMInwAAACjCB8AAMAowgcAADCK8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrwAQAAjLJ1e3UACHedb0l/sGya7Z/p7s8B6BojHwAAwCjCBwAAMIrwAQAAjCJ8AAAAowgfAADAKFa7AEAPdLUCBkD3MPIBAACMInwAAACjCB8AAMAowgcAADCK8AEAAIxitQuA0xrPbQHMY+QDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUTzbBQA66ep5L8E6L8+RwemIkQ8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRtsJHaWmpLrvsMiUkJCg1NVUzZ85UbW2t3zGtra3Kz89XSkqKBgwYoNzcXHk8noAWDQAAwpet8LFt2zbl5+erurpab7zxhtrb23XNNdeopaXFd0xBQYE2btyo9evXa9u2bWpoaNCsWbMCXjgAAAhP0XYO3rx5s9/rtWvXKjU1VTU1Nfq///s/NTU1afXq1aqoqNCUKVMkSWvWrNHIkSNVXV2t8ePHB65yAAAQlno156OpqUmSlJycLEmqqalRe3u7srOzfcdkZWVpyJAhqqqq6vIcXq9Xzc3NfhsAAIhctkY+/ldHR4cWLFigiRMnatSoUZIkt9ut2NhYJSUl+R3rdDrldru7PE9paalKSkp6WgYQloYu/HOoS0AY6erzcrBsWggqAQKjxyMf+fn5+uCDD7Ru3bpeFVBUVKSmpibfVl9f36vzAQCAvq1HIx933HGHXnvtNb399tsaPHiwrz0tLU1tbW1qbGz0G/3weDxKS0vr8lxxcXGKi4vrSRkAACAM2Rr5sCxLd9xxh1555RW99dZbGjZsmN/+MWPGKCYmRpWVlb622tpa1dXVyeVyBaZiAAAQ1myNfOTn56uiokJ/+tOflJCQ4JvHkZiYqP79+ysxMVFz585VYWGhkpOT5XA4NH/+fLlcLla6AAAASTbDx4oVKyRJV1xxhV/7mjVrdPPNN0uSli5dqn79+ik3N1der1c5OTlavnx5QIoFAADhz1b4sCzrlMfEx8ervLxc5eXlPS4KACtiThed/z+zigWnA57tAgAAjCJ8AAAAowgfAADAKMIHAAAwqse3VwcABB4TjXE6YOQDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABgVHeoCgEgzdOGfj2s7WDYtBJUgknX+nPEZQzhh5AMAABhF+AAAAEYRPgAAgFGEDwAAYBQTTgHgNNHVZOjOmLgKExj5AAAARhE+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRhA8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBTPdsFpoatnWvAMCwAIDUY+AACAUYQPAABgFOEDAAAYRfgAAABGET4AAIBRrHYB/r+uVsR0xgoZAOg92yMfb7/9tqZPn66MjAxFRUVpw4YNfvsty9IDDzyg9PR09e/fX9nZ2dq7d2+g6gUAAGHOdvhoaWnRxRdfrPLy8i73P/bYY3r66ae1cuVK7dixQ2eeeaZycnLU2tra62IBAED4s/21y9SpUzV16tQu91mWpWXLlmnRokWaMWOGJOnFF1+U0+nUhg0bdP311/euWgAAEPYCOuH0wIEDcrvdys7O9rUlJiZq3LhxqqqqCuRbAQCAMBXQCadut1uS5HQ6/dqdTqdvX2der1der9f3urm5OZAlAQCAPibkq11KS0tVUlIS6jIQxrqzSiXU7x/qGhEa/H/317k/WD12+gro1y5paWmSJI/H49fu8Xh8+zorKipSU1OTb6uvrw9kSQAAoI8JaPgYNmyY0tLSVFlZ6Wtrbm7Wjh075HK5uvyZuLg4ORwOvw0AAEQu21+7HD16VPv27fO9PnDggHbv3q3k5GQNGTJECxYs0JIlSzR8+HANGzZM999/vzIyMjRz5sxA1g0AAMKU7fDx/vvv68orr/S9LiwslCTNmTNHa9eu1T333KOWlhbNmzdPjY2NmjRpkjZv3qz4+PjAVQ0AAMKW7fBxxRVXyLKsE+6PiorS4sWLtXjx4l4VBgAIX0y2xcnwYDkAAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYFTIb68OhAqz8RHpAvUZ57boCDRGPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUax2AWxghQwA9B4jHwAAwCjCBwAAMIrwAQAAjCJ8AAAAowgfAADAKFa7AEAEYCUWwgkjHwAAwCjCBwAAMIrwAQAAjCJ8AAAAo5hwCgDwYeIqTGDkAwAAGEX4AAAARhE+AACAUYQPAABgFOEDAAAYxWoXGNN5Fv3BsmkhqgRAuOjO6huuJeGHkQ8AAGAU4QMAABhF+AAAAEYRPgAAgFGEDwAAYBSrXXBKfW2VCs+eACJDV7/Lob6+wAxGPgAAgFGEDwAAYBThAwAAGEX4AAAARhE+AACAUafdape+tnLDpEDNLDc5Q52VLUDfE8zfy0Cdu69f60N9HQ11fzDyAQAAjCJ8AAAAowgfAADAKMIHAAAwivABAACMOu1Wu3RHT2Zbh3rmcE9nMwdrZnmozwPg9NGd60Z3junqmhmOfw/CQdBGPsrLyzV06FDFx8dr3Lhxeu+994L1VgAAIIwEJXz87ne/U2FhoYqLi7Vr1y5dfPHFysnJ0eHDh4PxdgAAIIwEJXw8+eSTuvXWW3XLLbfoggsu0MqVK/Wd73xHL7zwQjDeDgAAhJGAz/loa2tTTU2NioqKfG39+vVTdna2qqqqjjve6/XK6/X6Xjc1NUmSmpubA12aJKnD+5Xf667ep/Mx3RGserurq5o719STf1cwBarvASAQTP496M41O1BMvdc357Qs69QHWwH22WefWZKs7du3+7Xffffd1uWXX37c8cXFxZYkNjY2NjY2tgjY6uvrT5kVQr7apaioSIWFhb7XHR0d+uKLL5SSkqKoqKgQVnZyzc3NyszMVH19vRwOR6jLCTn641v0hT/641v0hT/641uR0BeWZenIkSPKyMg45bEBDx9nnXWWzjjjDHk8Hr92j8ejtLS0446Pi4tTXFycX1tSUlKgywoah8MRth+UYKA/vkVf+KM/vkVf+KM/vhXufZGYmNit4wI+4TQ2NlZjxoxRZWWlr62jo0OVlZVyuVyBfjsAABBmgvK1S2FhoebMmaOxY8fq8ssv17Jly9TS0qJbbrklGG8HAADCSFDCx3XXXad///vfeuCBB+R2u/W9731PmzdvltPpDMbbhURcXJyKi4uP+8rodEV/fIu+8Ed/fIu+8Ed/fOt064soy+rOmhgAAIDA4MFyAADAKMIHAAAwivABAACMInwAAACjCB/dUFZWpqioKC1YsOCkxzU2Nio/P1/p6emKi4vT+eefr02bNpkp0pDu9sWyZcs0YsQI9e/fX5mZmSooKFBra6uZIoPowQcfVFRUlN+WlZV10p9Zv369srKyFB8fr4suuiiiPhN2+2PVqlWaPHmyBg4cqIEDByo7O1vvvfeewYqDqyefj2+sW7dOUVFRmjlzZnCLNKQnfRHJ19Ce9EekXkelIC21jSQ7d+7Us88+q9GjR5/0uLa2Nl199dVKTU3Vyy+/rLPPPlv/+te/wupurafS3b6oqKjQwoUL9cILL2jChAn6+OOPdfPNNysqKkpPPvmkoWqD58ILL9Sbb77pex0dfeJfo+3bt2v27NkqLS3Vj370I1VUVGjmzJnatWuXRo0aZaLcoLPTH1u3btXs2bM1YcIExcfH69FHH9U111yjDz/8UGeffbaJcoPOTn984+DBg7rrrrs0efLkYJZmnJ2+OB2uoXb6I9Kvo4SPkzh69Kjy8vK0atUqLVmy5KTHvvDCC/riiy+0fft2xcTESJKGDh1qoEoz7PTF9u3bNXHiRN1www2S/tsPs2fP1o4dO0yUGnTR0dFdPiqgK0899ZR+8IMf6O6775YkPfTQQ3rjjTf0zDPPaOXKlcEs0xg7/fHSSy/5vX7++ef1hz/8QZWVlbrpppuCUZ5xdvpDko4dO6a8vDyVlJTob3/7mxobG4NXnGF2+iLSr6GSvf6I9OsoX7ucRH5+vqZNm6bs7OxTHvvqq6/K5XIpPz9fTqdTo0aN0iOPPKJjx44ZqDT47PTFhAkTVFNT4xtO379/vzZt2qQf/vCHwS7TiL179yojI0Pnnnuu8vLyVFdXd8Jjq6qqjuuznJwcVVVVBbtMY+z0R2dfffWV2tvblZycHMQKzbLbH4sXL1Zqaqrmzp1rqEJz7PRFpF9DJXv9EenXUUY+TmDdunXatWuXdu7c2a3j9+/fr7feekt5eXnatGmT9u3bp1/84hdqb29XcXFxkKsNLrt9ccMNN+g///mPJk2aJMuy9PXXX+u2227TfffdF+RKg2/cuHFau3atRowYoUOHDqmkpESTJ0/WBx98oISEhOOOd7vdx93Z1+l0yu12myo5qOz2R2f33nuvMjIyuhVqw4Hd/njnnXe0evVq7d6923yxQWa3LyL5GirZ749Ivo5Kkiwcp66uzkpNTbX+/ve/+9q+//3vW3feeecJf2b48OFWZmam9fXXX/vannjiCSstLS2YpQZdT/rir3/9q+V0Oq1Vq1ZZ//jHP6w//vGPVmZmprV48WIDFZv15ZdfWg6Hw3r++ee73B8TE2NVVFT4tZWXl1upqakmyjPuVP3xv0pLS62BAwf6fbYizcn6o7m52Ro6dKi1adMmX9ucOXOsGTNmGKzQnFN9NiL1Gnoip+qPSL+OMvLRhZqaGh0+fFiXXnqpr+3YsWN6++239cwzz8jr9eqMM87w+5n09HTFxMT4tY8cOVJut1ttbW2KjY01Vn8g9aQv7r//ft144436+c9/Lkm66KKL1NLSonnz5unXv/61+vWLnG/7kpKSdP7552vfvn1d7k9LS5PH4/Fr83g8tuYEhJNT9cc3Hn/8cZWVlenNN9885QTmcHay/vjkk0908OBBTZ8+3dfW0dEh6b9zA2pra3XeeecZqzXYTvXZiNRr6Imcqj8i/Toa3tUHyVVXXaU9e/Zo9+7dvm3s2LHKy8vT7t27j/tjK0kTJ07Uvn37fBcPSfr444+Vnp4e1r80PemLr7766rhfjG+OsyLsUUJHjx7VJ598ovT09C73u1wuVVZW+rW98cYbcrlcJsoz7lT9IUmPPfaYHnroIW3evFljx441WJ15J+uPrKys4363fvzjH+vKK6/U7t27lZmZGYKKg+dUn41IvYaeyKn6I+Kvo6EeegkXnb9quPHGG62FCxf6XtfV1VkJCQnWHXfcYdXW1lqvvfaalZqaai1ZsiQE1QbXqfqiuLjYSkhIsH77299a+/fvt/7yl79Y5513nnXttdeGoNrA+tWvfmVt3brVOnDggPXuu+9a2dnZ1llnnWUdPnzYsqzj++Ldd9+1oqOjrccff9z65z//aRUXF1sxMTHWnj17QvVPCCi7/VFWVmbFxsZaL7/8snXo0CHfduTIkVD9EwLKbn90Fklfu9jti0i/htrtj0i+jloWX7v0WF1dnV8qzczM1JYtW1RQUKDRo0fr7LPP1p133ql77703hFWa0bkvFi1apKioKC1atEifffaZBg0apOnTp+vhhx8OYZWB8emnn2r27Nn6/PPPNWjQIE2aNEnV1dUaNGiQpOP7YsKECaqoqNCiRYt03333afjw4dqwYUPE3OPDbn+sWLFCbW1t+slPfuJ3nuLiYj344IMmSw8Ku/0Ryez2RaRfQ+32RyRfRyUpyrIiYfwGAACEi9MjggMAgD6D8AEAAIwifAAAAKMIHwAAwCjCBwAAMIrwAQAAjCJ8AAAAowgfAADAKMIHAAAwivABAACMInwAAACjCB8AAMCo/wdi3Gb0mhqO2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(np.log10(y_train), bins=100);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can incorporate this in our pipeline using `sklearn`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.compose import TransformedTargetRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "ttr = TransformedTargetRegressor(\n",
    "    Ridge(alpha=best_alpha), func=np.log1p, inverse_func=np.expm1\n",
    ") # transformer for log transforming the target\n",
    "ttr_pipe = make_pipeline(preprocessor, ttr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "ttr_pipe.fit(X_train, y_train); # y_train automatically transformed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([221355.29528077, 170663.43286226, 182608.09768702, ...,\n",
       "       248575.94877669, 132148.9047652 , 133262.17638244])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ttr_pipe.predict(X_train)  # predictions automatically un-transformed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.07808600924240866"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_percentage_error(y_test, ttr_pipe.predict(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "We reduced MAPE from ~10% to ~8% with this trick! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Different scoring functions with `cross_validate`\n",
    "\n",
    "- Let's try using MSE instead of the default $R^2$ score. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_neg_mean_squared_error</th>\n",
       "      <th>train_neg_mean_squared_error</th>\n",
       "      <th>test_neg_mean_absolute_percentage_error</th>\n",
       "      <th>train_neg_mean_absolute_percentage_error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.023106</td>\n",
       "      <td>0.005589</td>\n",
       "      <td>-7.060346e+08</td>\n",
       "      <td>-9.383069e+08</td>\n",
       "      <td>-0.096993</td>\n",
       "      <td>-0.104071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.019548</td>\n",
       "      <td>0.006010</td>\n",
       "      <td>-1.239851e+09</td>\n",
       "      <td>-8.267971e+08</td>\n",
       "      <td>-0.108030</td>\n",
       "      <td>-0.099662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.021142</td>\n",
       "      <td>0.006150</td>\n",
       "      <td>-1.125125e+09</td>\n",
       "      <td>-8.763019e+08</td>\n",
       "      <td>-0.118362</td>\n",
       "      <td>-0.101807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.018061</td>\n",
       "      <td>0.005076</td>\n",
       "      <td>-9.819320e+08</td>\n",
       "      <td>-8.847908e+08</td>\n",
       "      <td>-0.107847</td>\n",
       "      <td>-0.102472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.018337</td>\n",
       "      <td>0.005378</td>\n",
       "      <td>-2.268434e+09</td>\n",
       "      <td>-7.397199e+08</td>\n",
       "      <td>-0.121967</td>\n",
       "      <td>-0.098286</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_neg_mean_squared_error  \\\n",
       "0  0.023106    0.005589                -7.060346e+08   \n",
       "1  0.019548    0.006010                -1.239851e+09   \n",
       "2  0.021142    0.006150                -1.125125e+09   \n",
       "3  0.018061    0.005076                -9.819320e+08   \n",
       "4  0.018337    0.005378                -2.268434e+09   \n",
       "\n",
       "   train_neg_mean_squared_error  test_neg_mean_absolute_percentage_error  \\\n",
       "0                 -9.383069e+08                                -0.096993   \n",
       "1                 -8.267971e+08                                -0.108030   \n",
       "2                 -8.763019e+08                                -0.118362   \n",
       "3                 -8.847908e+08                                -0.107847   \n",
       "4                 -7.397199e+08                                -0.121967   \n",
       "\n",
       "   train_neg_mean_absolute_percentage_error  \n",
       "0                                 -0.104071  \n",
       "1                                 -0.099662  \n",
       "2                                 -0.101807  \n",
       "3                                 -0.102472  \n",
       "4                                 -0.098286  "
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(\n",
    "    cross_validate(\n",
    "        lr_tuned,\n",
    "        X_train,\n",
    "        y_train,\n",
    "        return_train_score=True,\n",
    "        scoring=[\"neg_mean_squared_error\", \"neg_mean_absolute_percentage_error\"]\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.021977</td>\n",
       "      <td>0.005930</td>\n",
       "      <td>0.096993</td>\n",
       "      <td>0.104071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.018638</td>\n",
       "      <td>0.005066</td>\n",
       "      <td>0.108030</td>\n",
       "      <td>0.099662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.018160</td>\n",
       "      <td>0.005145</td>\n",
       "      <td>0.118362</td>\n",
       "      <td>0.101807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.019339</td>\n",
       "      <td>0.005935</td>\n",
       "      <td>0.107847</td>\n",
       "      <td>0.102472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.018742</td>\n",
       "      <td>0.005270</td>\n",
       "      <td>0.121967</td>\n",
       "      <td>0.098286</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.021977    0.005930    0.096993     0.104071\n",
       "1  0.018638    0.005066    0.108030     0.099662\n",
       "2  0.018160    0.005145    0.118362     0.101807\n",
       "3  0.019339    0.005935    0.107847     0.102472\n",
       "4  0.018742    0.005270    0.121967     0.098286"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def mape(true, pred):\n",
    "    return mean_absolute_percentage_error(true, pred)\n",
    "\n",
    "\n",
    "# make a scorer function that we can pass into cross-validation\n",
    "mape_scorer = make_scorer(mape, greater_is_better=True)\n",
    "\n",
    "pd.DataFrame(\n",
    "    cross_validate(\n",
    "        lr_tuned, X_train, y_train, return_train_score=True, scoring=mape_scorer\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['accuracy',\n",
       " 'adjusted_mutual_info_score',\n",
       " 'adjusted_rand_score',\n",
       " 'average_precision',\n",
       " 'balanced_accuracy',\n",
       " 'completeness_score',\n",
       " 'explained_variance',\n",
       " 'f1',\n",
       " 'f1_macro',\n",
       " 'f1_micro',\n",
       " 'f1_samples',\n",
       " 'f1_weighted',\n",
       " 'fowlkes_mallows_score',\n",
       " 'homogeneity_score',\n",
       " 'jaccard',\n",
       " 'jaccard_macro',\n",
       " 'jaccard_micro',\n",
       " 'jaccard_samples',\n",
       " 'jaccard_weighted',\n",
       " 'matthews_corrcoef',\n",
       " 'max_error',\n",
       " 'mutual_info_score',\n",
       " 'neg_brier_score',\n",
       " 'neg_log_loss',\n",
       " 'neg_mean_absolute_error',\n",
       " 'neg_mean_absolute_percentage_error',\n",
       " 'neg_mean_gamma_deviance',\n",
       " 'neg_mean_poisson_deviance',\n",
       " 'neg_mean_squared_error',\n",
       " 'neg_mean_squared_log_error',\n",
       " 'neg_median_absolute_error',\n",
       " 'neg_root_mean_squared_error',\n",
       " 'normalized_mutual_info_score',\n",
       " 'precision',\n",
       " 'precision_macro',\n",
       " 'precision_micro',\n",
       " 'precision_samples',\n",
       " 'precision_weighted',\n",
       " 'r2',\n",
       " 'rand_score',\n",
       " 'recall',\n",
       " 'recall_macro',\n",
       " 'recall_micro',\n",
       " 'recall_samples',\n",
       " 'recall_weighted',\n",
       " 'roc_auc',\n",
       " 'roc_auc_ovo',\n",
       " 'roc_auc_ovo_weighted',\n",
       " 'roc_auc_ovr',\n",
       " 'roc_auc_ovr_weighted',\n",
       " 'top_k_accuracy',\n",
       " 'v_measure_score']"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sklearn\n",
    "sklearn.metrics.get_scorer_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fit_time</th>\n",
       "      <td>2.274203e-02</td>\n",
       "      <td>1.794481e-02</td>\n",
       "      <td>1.888394e-02</td>\n",
       "      <td>1.950598e-02</td>\n",
       "      <td>1.934409e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score_time</th>\n",
       "      <td>6.248951e-03</td>\n",
       "      <td>5.296230e-03</td>\n",
       "      <td>5.223751e-03</td>\n",
       "      <td>5.053759e-03</td>\n",
       "      <td>8.992910e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_r2</th>\n",
       "      <td>8.668969e-01</td>\n",
       "      <td>8.200460e-01</td>\n",
       "      <td>8.262644e-01</td>\n",
       "      <td>8.511854e-01</td>\n",
       "      <td>6.109505e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_r2</th>\n",
       "      <td>8.551369e-01</td>\n",
       "      <td>8.636241e-01</td>\n",
       "      <td>8.579735e-01</td>\n",
       "      <td>8.561893e-01</td>\n",
       "      <td>8.834054e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_neg_mape</th>\n",
       "      <td>-9.699277e-02</td>\n",
       "      <td>-1.080304e-01</td>\n",
       "      <td>-1.183620e-01</td>\n",
       "      <td>-1.078469e-01</td>\n",
       "      <td>-1.219672e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_neg_mape</th>\n",
       "      <td>-1.040712e-01</td>\n",
       "      <td>-9.966190e-02</td>\n",
       "      <td>-1.018073e-01</td>\n",
       "      <td>-1.024720e-01</td>\n",
       "      <td>-9.828607e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_neg_rmse</th>\n",
       "      <td>-2.657131e+04</td>\n",
       "      <td>-3.521152e+04</td>\n",
       "      <td>-3.354288e+04</td>\n",
       "      <td>-3.133579e+04</td>\n",
       "      <td>-4.762808e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_neg_rmse</th>\n",
       "      <td>-3.063179e+04</td>\n",
       "      <td>-2.875408e+04</td>\n",
       "      <td>-2.960240e+04</td>\n",
       "      <td>-2.974543e+04</td>\n",
       "      <td>-2.719779e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_neg_mse</th>\n",
       "      <td>-7.060346e+08</td>\n",
       "      <td>-1.239851e+09</td>\n",
       "      <td>-1.125125e+09</td>\n",
       "      <td>-9.819320e+08</td>\n",
       "      <td>-2.268434e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_neg_mse</th>\n",
       "      <td>-9.383069e+08</td>\n",
       "      <td>-8.267971e+08</td>\n",
       "      <td>-8.763019e+08</td>\n",
       "      <td>-8.847908e+08</td>\n",
       "      <td>-7.397199e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           0             1             2             3  \\\n",
       "fit_time        2.274203e-02  1.794481e-02  1.888394e-02  1.950598e-02   \n",
       "score_time      6.248951e-03  5.296230e-03  5.223751e-03  5.053759e-03   \n",
       "test_r2         8.668969e-01  8.200460e-01  8.262644e-01  8.511854e-01   \n",
       "train_r2        8.551369e-01  8.636241e-01  8.579735e-01  8.561893e-01   \n",
       "test_neg_mape  -9.699277e-02 -1.080304e-01 -1.183620e-01 -1.078469e-01   \n",
       "train_neg_mape -1.040712e-01 -9.966190e-02 -1.018073e-01 -1.024720e-01   \n",
       "test_neg_rmse  -2.657131e+04 -3.521152e+04 -3.354288e+04 -3.133579e+04   \n",
       "train_neg_rmse -3.063179e+04 -2.875408e+04 -2.960240e+04 -2.974543e+04   \n",
       "test_neg_mse   -7.060346e+08 -1.239851e+09 -1.125125e+09 -9.819320e+08   \n",
       "train_neg_mse  -9.383069e+08 -8.267971e+08 -8.763019e+08 -8.847908e+08   \n",
       "\n",
       "                           4  \n",
       "fit_time        1.934409e-02  \n",
       "score_time      8.992910e-03  \n",
       "test_r2         6.109505e-01  \n",
       "train_r2        8.834054e-01  \n",
       "test_neg_mape  -1.219672e-01  \n",
       "train_neg_mape -9.828607e-02  \n",
       "test_neg_rmse  -4.762808e+04  \n",
       "train_neg_rmse -2.719779e+04  \n",
       "test_neg_mse   -2.268434e+09  \n",
       "train_neg_mse  -7.397199e+08  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scoring = {\n",
    "    \"r2\": \"r2\",\n",
    "    \"neg_mape\": \"neg_mean_absolute_percentage_error\", \n",
    "    \"neg_rmse\": \"neg_root_mean_squared_error\",\n",
    "    \"neg_mse\": \"neg_mean_squared_error\",\n",
    "}\n",
    "\n",
    "pd.DataFrame(\n",
    "    cross_validate(lr_tuned, X_train, y_train, return_train_score=True, scoring=scoring)\n",
    ").T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.09496387589495993"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_percentage_error(y_test, lr_tuned.predict(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Using regression metrics with `scikit-learn`\n",
    "\n",
    "- In `sklearn` you will notice that it has negative version of the metrics above (e.g., `neg_mean_squared_error`, `neg_root_mean_squared_error`). \n",
    "- The reason for this is that scores return a value to maximize, the higher the better.\n",
    "- If you define your own scorer function and if you do not want this interpretation, you can set the `greater_is_better` parameter to False "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## ❓❓ Questions for you"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### (iClicker) Exercise 10.1 \n",
    "\n",
    "**iClicker cloud join link: https://join.iclicker.com/3DP5H**\n",
    "\n",
    "**Select all of the following statements which are TRUE.**\n",
    "\n",
    "- (A) Price per square foot would be a good feature to add in our `X`. \n",
    "- (B) The `alpha` hyperparameter of `Ridge` has similar interpretation of `C` hyperparameter of `LogisticRegression`; higher `alpha` means more complex model. \n",
    "- (C) In regression, one should use MAPE instead of MSE when relative (percent) error matters more than absolute error.\n",
    "- (D) A lower RMSE value indicates a better model.\n",
    "- (E) We can use still use precision and recall for regression problems but now we have other metrics we can use as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Summary\n",
    "\n",
    "- House prices dataset target is price, which is numeric -> regression rather than classification\n",
    "- There are corresponding versions of all the tools we used:\n",
    "    - `DummyClassifier` -> `DummyRegressor`\n",
    "    - `LogisticRegression` -> `Ridge`\n",
    "- `Ridge` hyperparameter `alpha` is like `LogisticRegression` hyperparameter `C`, but opposite meaning\n",
    "- We'll avoid `LinearRegression` in this course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Scoring metrics\n",
    "- $R^2$ is the default .score(), it is unitless, 0 is bad, 1 is best\n",
    "- MSE (mean squared error) is in units of target squared, hard to interpret; 0 is best\n",
    "- RMSE (root mean squared error) is in the same units as the target; 0 is best\n",
    "- MAPE (mean absolute percent error) is unitless; 0 is best, 1 is bad"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [conda env:cpsc330]",
   "language": "python",
   "name": "conda-env-cpsc330-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}