lectures/mehrdad/10_regression-metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](../img/330-banner.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Lecture 10: Regression metrics\n",
    "\n",
    "UBC 2024 Summer\n",
    "\n",
    "Instructor: Mehrdad Oveisi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:00.500729Z",
     "iopub.status.busy": "2024-05-29T14:52:00.500224Z",
     "iopub.status.idle": "2024-05-29T14:52:04.729720Z",
     "shell.execute_reply": "2024-05-29T14:52:04.728063Z",
     "shell.execute_reply.started": "2024-05-29T14:52:00.500681Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.compose import (\n",
    "    ColumnTransformer,\n",
    "    TransformedTargetRegressor,\n",
    "    make_column_transformer,\n",
    ")\n",
    "from sklearn.dummy import DummyRegressor\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.linear_model import Ridge, RidgeCV\n",
    "from sklearn.metrics import make_scorer, mean_squared_error, r2_score\n",
    "from sklearn.model_selection import (\n",
    "    GridSearchCV,\n",
    "    cross_val_score,\n",
    "    cross_validate,\n",
    "    train_test_split,\n",
    ")\n",
    "from sklearn.pipeline import Pipeline, make_pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, StandardScaler\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## Learning outcomes \n",
    "\n",
    "From this lecture, students are expected to be able to:\n",
    "\n",
    "- Carry out feature transformations on somewhat complicated dataset. \n",
    "- Visualize transformed features as a dataframe. \n",
    "- Use `Ridge` and `RidgeCV`.\n",
    "- Explain how `alpha` hyperparameter of `Ridge` relates to the fundamental tradeoff. \n",
    "- Explain the effect of `alpha` on the magnitude of the learned coefficients. \n",
    "- Examine coefficients of transformed features.  \n",
    "- Appropriately select a scoring metric given a regression problem.\n",
    "- Interpret and communicate the meanings of different scoring metrics on regression problems.\n",
    "    - MSE, RMSE, $R^2$, MAPE\n",
    "- Apply log-transform on the target values in a regression problem with `TransformedTargetRegressor`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After carrying out preprocessing, why it's useful to get feature names for transformed features?   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More comments on tackling class imbalance\n",
    "\n",
    "- In lecture 9 we talked about a few rather simple approaches to deal with class imbalance. \n",
    "- If you have a problem such as fraud detection problem where you want to spot rare events, you can also think of this problem as anomaly detection problem and use different kinds of algorithms such as [isolation forests](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html).  \n",
    "- If you are interested in this area, it might be worth checking out this book on this topic.\n",
    "[Imbalanced Learning: Foundations, Algorithms, and Applications](https://www.amazon.com/dp/1118074629/ref=as_li_ss_tl?&linkCode=sl1&tag=inspiredalgor-20&linkId=615e87a9105582e292ad2b7e2c7ea339&language=en_US)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "When you calculate precision, recall, f1 score, by default only the positive label is evaluated, assuming by default that the positive class is labeled 1. This is configurable through the `pos_label` parameter.\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dataset [[video](https://youtu.be/lgGTKLwNgkQ)]\n",
    "\n",
    "In this lecture, we'll be using [Kaggle House Prices dataset](https://www.kaggle.com/c/home-data-for-ml-course/). As usual, to run this notebook you'll need to download the data. For this dataset, train and test have already been separated. We'll be working with the train portion in this lecture. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:04.735265Z",
     "iopub.status.busy": "2024-05-29T14:52:04.733880Z",
     "iopub.status.idle": "2024-05-29T14:52:04.853330Z",
     "shell.execute_reply": "2024-05-29T14:52:04.850738Z",
     "shell.execute_reply.started": "2024-05-29T14:52:04.735187Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>303</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>118.0</td>\n",
       "      <td>13704</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>205000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767</th>\n",
       "      <td>768</td>\n",
       "      <td>50</td>\n",
       "      <td>RL</td>\n",
       "      <td>75.0</td>\n",
       "      <td>12508</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Shed</td>\n",
       "      <td>1300</td>\n",
       "      <td>7</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>160000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429</th>\n",
       "      <td>430</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>130.0</td>\n",
       "      <td>11457</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2009</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>175000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1139</th>\n",
       "      <td>1140</td>\n",
       "      <td>30</td>\n",
       "      <td>RL</td>\n",
       "      <td>98.0</td>\n",
       "      <td>8731</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>144000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>558</th>\n",
       "      <td>559</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>57.0</td>\n",
       "      <td>21872</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR2</td>\n",
       "      <td>HLS</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>175000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 81 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Id  MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "302    303          20       RL        118.0    13704   Pave   NaN      IR1   \n",
       "767    768          50       RL         75.0    12508   Pave   NaN      IR1   \n",
       "429    430          20       RL        130.0    11457   Pave   NaN      IR1   \n",
       "1139  1140          30       RL         98.0     8731   Pave   NaN      IR1   \n",
       "558    559          60       RL         57.0    21872   Pave   NaN      IR2   \n",
       "\n",
       "     LandContour Utilities  ... PoolArea PoolQC Fence MiscFeature MiscVal  \\\n",
       "302          Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "767          Lvl    AllPub  ...        0    NaN   NaN        Shed    1300   \n",
       "429          Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "1139         Lvl    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "558          HLS    AllPub  ...        0    NaN   NaN         NaN       0   \n",
       "\n",
       "     MoSold YrSold  SaleType  SaleCondition  SalePrice  \n",
       "302       1   2006        WD         Normal     205000  \n",
       "767       7   2008        WD         Normal     160000  \n",
       "429       3   2009        WD         Normal     175000  \n",
       "1139      5   2007        WD         Normal     144000  \n",
       "558       8   2008        WD         Normal     175000  \n",
       "\n",
       "[5 rows x 81 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"../data/housing-kaggle/train.csv\")\n",
    "train_df, test_df = train_test_split(df, test_size=0.10, random_state=123)\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- The supervised machine learning problem is predicting housing price given features associated with properties. \n",
    "- Here, the target is `SalePrice`, which is continuous. So it's a **regression problem** (as opposed to classification)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:04.857526Z",
     "iopub.status.busy": "2024-05-29T14:52:04.856604Z",
     "iopub.status.idle": "2024-05-29T14:52:04.870312Z",
     "shell.execute_reply": "2024-05-29T14:52:04.868139Z",
     "shell.execute_reply.started": "2024-05-29T14:52:04.857451Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 81)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "### Let's separate `X` and `y`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:04.873787Z",
     "iopub.status.busy": "2024-05-29T14:52:04.872798Z",
     "iopub.status.idle": "2024-05-29T14:52:04.893752Z",
     "shell.execute_reply": "2024-05-29T14:52:04.891633Z",
     "shell.execute_reply.started": "2024-05-29T14:52:04.873699Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "X_train = train_df.drop(columns=[\"SalePrice\"])\n",
    "y_train = train_df[\"SalePrice\"]\n",
    "\n",
    "X_test = test_df.drop(columns=[\"SalePrice\"])\n",
    "y_test = test_df[\"SalePrice\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Exploratory Data Analysis (EDA)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:04.896861Z",
     "iopub.status.busy": "2024-05-29T14:52:04.896328Z",
     "iopub.status.idle": "2024-05-29T14:52:05.110785Z",
     "shell.execute_reply": "2024-05-29T14:52:05.108311Z",
     "shell.execute_reply.started": "2024-05-29T14:52:04.896828Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>...</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1089.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1307.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "      <td>1314.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>734.182648</td>\n",
       "      <td>56.472603</td>\n",
       "      <td>69.641873</td>\n",
       "      <td>10273.261035</td>\n",
       "      <td>6.076104</td>\n",
       "      <td>5.570015</td>\n",
       "      <td>1970.995434</td>\n",
       "      <td>1984.659056</td>\n",
       "      <td>102.514155</td>\n",
       "      <td>441.425419</td>\n",
       "      <td>...</td>\n",
       "      <td>94.281583</td>\n",
       "      <td>45.765601</td>\n",
       "      <td>21.726788</td>\n",
       "      <td>3.624049</td>\n",
       "      <td>13.987062</td>\n",
       "      <td>3.065449</td>\n",
       "      <td>46.951294</td>\n",
       "      <td>6.302131</td>\n",
       "      <td>2007.840183</td>\n",
       "      <td>179802.147641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>422.224662</td>\n",
       "      <td>42.036646</td>\n",
       "      <td>23.031794</td>\n",
       "      <td>8997.895541</td>\n",
       "      <td>1.392612</td>\n",
       "      <td>1.112848</td>\n",
       "      <td>30.198127</td>\n",
       "      <td>20.639754</td>\n",
       "      <td>178.301563</td>\n",
       "      <td>459.276687</td>\n",
       "      <td>...</td>\n",
       "      <td>125.436492</td>\n",
       "      <td>65.757545</td>\n",
       "      <td>60.766423</td>\n",
       "      <td>30.320430</td>\n",
       "      <td>53.854129</td>\n",
       "      <td>42.341109</td>\n",
       "      <td>522.283421</td>\n",
       "      <td>2.698206</td>\n",
       "      <td>1.332824</td>\n",
       "      <td>79041.260572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1300.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1872.000000</td>\n",
       "      <td>1950.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2006.000000</td>\n",
       "      <td>34900.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>369.250000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>7500.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1953.000000</td>\n",
       "      <td>1966.250000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>2007.000000</td>\n",
       "      <td>129600.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>735.500000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>69.000000</td>\n",
       "      <td>9391.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1972.000000</td>\n",
       "      <td>1993.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>376.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2008.000000</td>\n",
       "      <td>162000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1099.750000</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>11509.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2000.000000</td>\n",
       "      <td>2004.000000</td>\n",
       "      <td>165.500000</td>\n",
       "      <td>704.750000</td>\n",
       "      <td>...</td>\n",
       "      <td>168.000000</td>\n",
       "      <td>66.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>2009.000000</td>\n",
       "      <td>212975.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1460.000000</td>\n",
       "      <td>190.000000</td>\n",
       "      <td>313.000000</td>\n",
       "      <td>215245.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>1378.000000</td>\n",
       "      <td>5644.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>857.000000</td>\n",
       "      <td>547.000000</td>\n",
       "      <td>552.000000</td>\n",
       "      <td>508.000000</td>\n",
       "      <td>480.000000</td>\n",
       "      <td>738.000000</td>\n",
       "      <td>15500.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>2010.000000</td>\n",
       "      <td>755000.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                Id   MSSubClass  LotFrontage        LotArea  OverallQual  \\\n",
       "count  1314.000000  1314.000000  1089.000000    1314.000000  1314.000000   \n",
       "mean    734.182648    56.472603    69.641873   10273.261035     6.076104   \n",
       "std     422.224662    42.036646    23.031794    8997.895541     1.392612   \n",
       "min       1.000000    20.000000    21.000000    1300.000000     1.000000   \n",
       "25%     369.250000    20.000000    59.000000    7500.000000     5.000000   \n",
       "50%     735.500000    50.000000    69.000000    9391.000000     6.000000   \n",
       "75%    1099.750000    70.000000    80.000000   11509.000000     7.000000   \n",
       "max    1460.000000   190.000000   313.000000  215245.000000    10.000000   \n",
       "\n",
       "       OverallCond    YearBuilt  YearRemodAdd   MasVnrArea   BsmtFinSF1  ...  \\\n",
       "count  1314.000000  1314.000000   1314.000000  1307.000000  1314.000000  ...   \n",
       "mean      5.570015  1970.995434   1984.659056   102.514155   441.425419  ...   \n",
       "std       1.112848    30.198127     20.639754   178.301563   459.276687  ...   \n",
       "min       1.000000  1872.000000   1950.000000     0.000000     0.000000  ...   \n",
       "25%       5.000000  1953.000000   1966.250000     0.000000     0.000000  ...   \n",
       "50%       5.000000  1972.000000   1993.000000     0.000000   376.000000  ...   \n",
       "75%       6.000000  2000.000000   2004.000000   165.500000   704.750000  ...   \n",
       "max       9.000000  2010.000000   2010.000000  1378.000000  5644.000000  ...   \n",
       "\n",
       "        WoodDeckSF  OpenPorchSF  EnclosedPorch    3SsnPorch  ScreenPorch  \\\n",
       "count  1314.000000  1314.000000    1314.000000  1314.000000  1314.000000   \n",
       "mean     94.281583    45.765601      21.726788     3.624049    13.987062   \n",
       "std     125.436492    65.757545      60.766423    30.320430    53.854129   \n",
       "min       0.000000     0.000000       0.000000     0.000000     0.000000   \n",
       "25%       0.000000     0.000000       0.000000     0.000000     0.000000   \n",
       "50%       0.000000    24.000000       0.000000     0.000000     0.000000   \n",
       "75%     168.000000    66.750000       0.000000     0.000000     0.000000   \n",
       "max     857.000000   547.000000     552.000000   508.000000   480.000000   \n",
       "\n",
       "          PoolArea       MiscVal       MoSold       YrSold      SalePrice  \n",
       "count  1314.000000   1314.000000  1314.000000  1314.000000    1314.000000  \n",
       "mean      3.065449     46.951294     6.302131  2007.840183  179802.147641  \n",
       "std      42.341109    522.283421     2.698206     1.332824   79041.260572  \n",
       "min       0.000000      0.000000     1.000000  2006.000000   34900.000000  \n",
       "25%       0.000000      0.000000     5.000000  2007.000000  129600.000000  \n",
       "50%       0.000000      0.000000     6.000000  2008.000000  162000.000000  \n",
       "75%       0.000000      0.000000     8.000000  2009.000000  212975.000000  \n",
       "max     738.000000  15500.000000    12.000000  2010.000000  755000.000000  \n",
       "\n",
       "[8 rows x 38 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.115359Z",
     "iopub.status.busy": "2024-05-29T14:52:05.113086Z",
     "iopub.status.idle": "2024-05-29T14:52:05.288236Z",
     "shell.execute_reply": "2024-05-29T14:52:05.286090Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.115283Z"
    },
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1314., 1089., 1307., 1241.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we have missing data\n",
    "train_df.describe().loc[\"count\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.294775Z",
     "iopub.status.busy": "2024-05-29T14:52:05.293705Z",
     "iopub.status.idle": "2024-05-29T14:52:05.342488Z",
     "shell.execute_reply": "2024-05-29T14:52:05.340382Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.294715Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 1314 entries, 302 to 1389\n",
      "Data columns (total 81 columns):\n",
      " #   Column         Non-Null Count  Dtype  \n",
      "---  ------         --------------  -----  \n",
      " 0   Id             1314 non-null   int64  \n",
      " 1   MSSubClass     1314 non-null   int64  \n",
      " 2   MSZoning       1314 non-null   object \n",
      " 3   LotFrontage    1089 non-null   float64\n",
      " 4   LotArea        1314 non-null   int64  \n",
      " 5   Street         1314 non-null   object \n",
      " 6   Alley          81 non-null     object \n",
      " 7   LotShape       1314 non-null   object \n",
      " 8   LandContour    1314 non-null   object \n",
      " 9   Utilities      1314 non-null   object \n",
      " 10  LotConfig      1314 non-null   object \n",
      " 11  LandSlope      1314 non-null   object \n",
      " 12  Neighborhood   1314 non-null   object \n",
      " 13  Condition1     1314 non-null   object \n",
      " 14  Condition2     1314 non-null   object \n",
      " 15  BldgType       1314 non-null   object \n",
      " 16  HouseStyle     1314 non-null   object \n",
      " 17  OverallQual    1314 non-null   int64  \n",
      " 18  OverallCond    1314 non-null   int64  \n",
      " 19  YearBuilt      1314 non-null   int64  \n",
      " 20  YearRemodAdd   1314 non-null   int64  \n",
      " 21  RoofStyle      1314 non-null   object \n",
      " 22  RoofMatl       1314 non-null   object \n",
      " 23  Exterior1st    1314 non-null   object \n",
      " 24  Exterior2nd    1314 non-null   object \n",
      " 25  MasVnrType     528 non-null    object \n",
      " 26  MasVnrArea     1307 non-null   float64\n",
      " 27  ExterQual      1314 non-null   object \n",
      " 28  ExterCond      1314 non-null   object \n",
      " 29  Foundation     1314 non-null   object \n",
      " 30  BsmtQual       1280 non-null   object \n",
      " 31  BsmtCond       1280 non-null   object \n",
      " 32  BsmtExposure   1279 non-null   object \n",
      " 33  BsmtFinType1   1280 non-null   object \n",
      " 34  BsmtFinSF1     1314 non-null   int64  \n",
      " 35  BsmtFinType2   1280 non-null   object \n",
      " 36  BsmtFinSF2     1314 non-null   int64  \n",
      " 37  BsmtUnfSF      1314 non-null   int64  \n",
      " 38  TotalBsmtSF    1314 non-null   int64  \n",
      " 39  Heating        1314 non-null   object \n",
      " 40  HeatingQC      1314 non-null   object \n",
      " 41  CentralAir     1314 non-null   object \n",
      " 42  Electrical     1313 non-null   object \n",
      " 43  1stFlrSF       1314 non-null   int64  \n",
      " 44  2ndFlrSF       1314 non-null   int64  \n",
      " 45  LowQualFinSF   1314 non-null   int64  \n",
      " 46  GrLivArea      1314 non-null   int64  \n",
      " 47  BsmtFullBath   1314 non-null   int64  \n",
      " 48  BsmtHalfBath   1314 non-null   int64  \n",
      " 49  FullBath       1314 non-null   int64  \n",
      " 50  HalfBath       1314 non-null   int64  \n",
      " 51  BedroomAbvGr   1314 non-null   int64  \n",
      " 52  KitchenAbvGr   1314 non-null   int64  \n",
      " 53  KitchenQual    1314 non-null   object \n",
      " 54  TotRmsAbvGrd   1314 non-null   int64  \n",
      " 55  Functional     1314 non-null   object \n",
      " 56  Fireplaces     1314 non-null   int64  \n",
      " 57  FireplaceQu    687 non-null    object \n",
      " 58  GarageType     1241 non-null   object \n",
      " 59  GarageYrBlt    1241 non-null   float64\n",
      " 60  GarageFinish   1241 non-null   object \n",
      " 61  GarageCars     1314 non-null   int64  \n",
      " 62  GarageArea     1314 non-null   int64  \n",
      " 63  GarageQual     1241 non-null   object \n",
      " 64  GarageCond     1241 non-null   object \n",
      " 65  PavedDrive     1314 non-null   object \n",
      " 66  WoodDeckSF     1314 non-null   int64  \n",
      " 67  OpenPorchSF    1314 non-null   int64  \n",
      " 68  EnclosedPorch  1314 non-null   int64  \n",
      " 69  3SsnPorch      1314 non-null   int64  \n",
      " 70  ScreenPorch    1314 non-null   int64  \n",
      " 71  PoolArea       1314 non-null   int64  \n",
      " 72  PoolQC         7 non-null      object \n",
      " 73  Fence          259 non-null    object \n",
      " 74  MiscFeature    50 non-null     object \n",
      " 75  MiscVal        1314 non-null   int64  \n",
      " 76  MoSold         1314 non-null   int64  \n",
      " 77  YrSold         1314 non-null   int64  \n",
      " 78  SaleType       1314 non-null   object \n",
      " 79  SaleCondition  1314 non-null   object \n",
      " 80  SalePrice      1314 non-null   int64  \n",
      "dtypes: float64(3), int64(35), object(43)\n",
      "memory usage: 841.8+ KB\n"
     ]
    }
   ],
   "source": [
    "train_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `ydata-profiling`\n",
    "\n",
    "We do not have `ydata_profiling` in our course environment (formerly `pandas_profiling`). You will  have to install it in the environment on your own if you want to run the code below. \n",
    "\n",
    "`conda install -n cpsc330 -c conda-forge ydata-profiling`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.345145Z",
     "iopub.status.busy": "2024-05-29T14:52:05.344587Z",
     "iopub.status.idle": "2024-05-29T14:52:05.353931Z",
     "shell.execute_reply": "2024-05-29T14:52:05.351568Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.345093Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "## /!\\ This takes a while to run\n",
    "# from ydata_profiling import ProfileReport\n",
    "\n",
    "# profile = ProfileReport(train_df, title=\"Pandas Profiling Report\")  # , minimal=True)\n",
    "# profile.to_notebook_iframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature types \n",
    "\n",
    "- Do not blindly trust all the info given to you by automated tools. \n",
    "- How does pandas profiling figure out the data type?\n",
    "    - You can look at the Python data type and say floats are numeric, strings are categorical.\n",
    "    - However, in doing so you would miss out on various subtleties such as some of the string features being ordinal rather than truly categorical.\n",
    "    - Also, it will think free text is categorical."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- In addition to tools such as above, it's important to go through data description to understand the data.\n",
    "- The data description for our dataset is available [here](https://www.kaggle.com/c/home-data-for-ml-course/data?select=data_description.txt).     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature types \n",
    "\n",
    "- We have **mixed feature types** and a bunch of **missing values**. \n",
    "- Now, let's identify feature types and transformations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Let's get the numeric-looking columns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.358877Z",
     "iopub.status.busy": "2024-05-29T14:52:05.357726Z",
     "iopub.status.idle": "2024-05-29T14:52:05.371605Z",
     "shell.execute_reply": "2024-05-29T14:52:05.369509Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.358689Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Id', 'MSSubClass', 'LotFrontage', 'LotArea', 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 'KitchenAbvGr', 'TotRmsAbvGrd', 'Fireplaces', 'GarageYrBlt', 'GarageCars', 'GarageArea', 'WoodDeckSF', 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 'MiscVal', 'MoSold', 'YrSold']\n"
     ]
    }
   ],
   "source": [
    "numeric_looking_columns = X_train.select_dtypes(include=np.number).columns.tolist()\n",
    "print(numeric_looking_columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Not all numeric looking columns are necessarily numeric. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.376138Z",
     "iopub.status.busy": "2024-05-29T14:52:05.374471Z",
     "iopub.status.idle": "2024-05-29T14:52:05.389392Z",
     "shell.execute_reply": "2024-05-29T14:52:05.387272Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.376083Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 20,  50,  30,  60, 160,  85,  90, 120, 180,  80,  70,  75, 190,\n",
       "        45,  40])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[\"MSSubClass\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MSSubClass: Identifies the type of dwelling involved in the sale.\t\n",
    "\n",
    "        20\t1-STORY 1946 & NEWER ALL STYLES\n",
    "        30\t1-STORY 1945 & OLDER\n",
    "        40\t1-STORY W/FINISHED ATTIC ALL AGES\n",
    "        45\t1-1/2 STORY - UNFINISHED ALL AGES\n",
    "        50\t1-1/2 STORY FINISHED ALL AGES\n",
    "        60\t2-STORY 1946 & NEWER\n",
    "        70\t2-STORY 1945 & OLDER\n",
    "        75\t2-1/2 STORY ALL AGES\n",
    "        80\tSPLIT OR MULTI-LEVEL\n",
    "        85\tSPLIT FOYER\n",
    "        90\tDUPLEX - ALL STYLES AND AGES\n",
    "       120\t1-STORY PUD (Planned Unit Development) - 1946 & NEWER\n",
    "       150\t1-1/2 STORY PUD - ALL AGES\n",
    "       160\t2-STORY PUD - 1946 & NEWER\n",
    "       180\tPUD - MULTILEVEL - INCL SPLIT LEV/FOYER\n",
    "       190\t2 FAMILY CONVERSION - ALL STYLES AND AGES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Also, month sold is more of a categorical feature than a numeric feature. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.393696Z",
     "iopub.status.busy": "2024-05-29T14:52:05.392074Z",
     "iopub.status.idle": "2024-05-29T14:52:05.412554Z",
     "shell.execute_reply": "2024-05-29T14:52:05.409959Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.393633Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  7,  3,  5,  8, 10,  6,  9, 12,  2,  4, 11])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[\"MoSold\"].unique()  # Month Sold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.416535Z",
     "iopub.status.busy": "2024-05-29T14:52:05.415493Z",
     "iopub.status.idle": "2024-05-29T14:52:05.434903Z",
     "shell.execute_reply": "2024-05-29T14:52:05.431847Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.416497Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "drop_features = [\"Id\"]\n",
    "numeric_features = [\n",
    "    \"BedroomAbvGr\",\n",
    "    \"KitchenAbvGr\",\n",
    "    \"LotFrontage\",\n",
    "    \"LotArea\",\n",
    "    \"OverallQual\",\n",
    "    \"OverallCond\",\n",
    "    \"YearBuilt\",\n",
    "    \"YearRemodAdd\",\n",
    "    \"MasVnrArea\",\n",
    "    \"BsmtFinSF1\",\n",
    "    \"BsmtFinSF2\",\n",
    "    \"BsmtUnfSF\",\n",
    "    \"TotalBsmtSF\",\n",
    "    \"1stFlrSF\",\n",
    "    \"2ndFlrSF\",\n",
    "    \"LowQualFinSF\",\n",
    "    \"GrLivArea\",\n",
    "    \"BsmtFullBath\",\n",
    "    \"BsmtHalfBath\",\n",
    "    \"FullBath\",\n",
    "    \"HalfBath\",\n",
    "    \"TotRmsAbvGrd\",\n",
    "    \"Fireplaces\",\n",
    "    \"GarageYrBlt\",\n",
    "    \"GarageCars\",\n",
    "    \"GarageArea\",\n",
    "    \"WoodDeckSF\",\n",
    "    \"OpenPorchSF\",\n",
    "    \"EnclosedPorch\",\n",
    "    \"3SsnPorch\",\n",
    "    \"ScreenPorch\",\n",
    "    \"PoolArea\",\n",
    "    \"MiscVal\",\n",
    "    \"YrSold\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "___\n",
    "I've not looked at all the features carefully. It might be appropriate to apply some other encoding on some of the numeric features above. \n",
    "___"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.440771Z",
     "iopub.status.busy": "2024-05-29T14:52:05.438694Z",
     "iopub.status.idle": "2024-05-29T14:52:05.463871Z",
     "shell.execute_reply": "2024-05-29T14:52:05.461378Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.440687Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'MSSubClass', 'MoSold'}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(numeric_looking_columns) - set(numeric_features) - set(drop_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll treat the above numeric-looking features as categorical features. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- There are a bunch of ordinal features in this dataset. \n",
    "- Ordinal features with the same scale \n",
    "    - Poor (Po), Fair (Fa), Typical (TA), Good (Gd), Excellent (Ex)\n",
    "    - These we'll be calling `ordinal_features_reg`.\n",
    "- Ordinal features with different scales\n",
    "    - These we'll be calling `ordinal_features_oth`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.468161Z",
     "iopub.status.busy": "2024-05-29T14:52:05.466892Z",
     "iopub.status.idle": "2024-05-29T14:52:05.487617Z",
     "shell.execute_reply": "2024-05-29T14:52:05.484936Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.468072Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       " ['Po', 'Fa', 'TA', 'Gd', 'Ex']]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ordinal_features_reg = [\n",
    "    \"ExterQual\",\n",
    "    \"ExterCond\",\n",
    "    \"BsmtQual\",\n",
    "    \"BsmtCond\",\n",
    "    \"HeatingQC\",\n",
    "    \"KitchenQual\",\n",
    "    \"FireplaceQu\",\n",
    "    \"GarageQual\",\n",
    "    \"GarageCond\",\n",
    "    \"PoolQC\",\n",
    "]\n",
    "ordering = [\n",
    "    \"Po\",\n",
    "    \"Fa\",\n",
    "    \"TA\",\n",
    "    \"Gd\",\n",
    "    \"Ex\",\n",
    "]  # if N/A it will just impute something, per below\n",
    "ordering_ordinal_reg = [ordering] * len(ordinal_features_reg)\n",
    "ordering_ordinal_reg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll pass the above as categories in our `OrdinalEncoder`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- There are a bunch more ordinal features using different scales.\n",
    "  - These we'll be calling `ordinal_features_oth`. \n",
    "  - We are encoding them separately. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.491567Z",
     "iopub.status.busy": "2024-05-29T14:52:05.490537Z",
     "iopub.status.idle": "2024-05-29T14:52:05.504394Z",
     "shell.execute_reply": "2024-05-29T14:52:05.499909Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.491499Z"
    }
   },
   "outputs": [],
   "source": [
    "ordinal_features_oth = [\n",
    "    \"BsmtExposure\",\n",
    "    \"BsmtFinType1\",\n",
    "    \"BsmtFinType2\",\n",
    "    \"Functional\",\n",
    "    \"Fence\",\n",
    "]\n",
    "ordering_ordinal_oth = [\n",
    "    [\"NA\", \"No\", \"Mn\", \"Av\", \"Gd\"],\n",
    "    [\"NA\", \"Unf\", \"LwQ\", \"Rec\", \"BLQ\", \"ALQ\", \"GLQ\"],\n",
    "    [\"NA\", \"Unf\", \"LwQ\", \"Rec\", \"BLQ\", \"ALQ\", \"GLQ\"],\n",
    "    [\"Sal\", \"Sev\", \"Maj2\", \"Maj1\", \"Mod\", \"Min2\", \"Min1\", \"Typ\"],\n",
    "    [\"NA\", \"MnWw\", \"GdWo\", \"MnPrv\", \"GdPrv\"],\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The remaining features are categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.508387Z",
     "iopub.status.busy": "2024-05-29T14:52:05.507170Z",
     "iopub.status.idle": "2024-05-29T14:52:05.526025Z",
     "shell.execute_reply": "2024-05-29T14:52:05.522559Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.508347Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Alley',\n",
       " 'MSSubClass',\n",
       " 'Neighborhood',\n",
       " 'Utilities',\n",
       " 'Exterior1st',\n",
       " 'MasVnrType',\n",
       " 'LotShape',\n",
       " 'LandContour',\n",
       " 'MoSold',\n",
       " 'MSZoning',\n",
       " 'Street',\n",
       " 'GarageFinish',\n",
       " 'HouseStyle',\n",
       " 'LotConfig',\n",
       " 'PavedDrive',\n",
       " 'BldgType',\n",
       " 'LandSlope',\n",
       " 'Condition1',\n",
       " 'MiscFeature',\n",
       " 'Electrical',\n",
       " 'CentralAir',\n",
       " 'RoofMatl',\n",
       " 'Heating',\n",
       " 'RoofStyle',\n",
       " 'SaleType',\n",
       " 'Exterior2nd',\n",
       " 'Condition2',\n",
       " 'SaleCondition',\n",
       " 'Foundation',\n",
       " 'GarageType']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_features = list(\n",
    "    set(X_train.columns)\n",
    "    - set(numeric_features)\n",
    "    - set(ordinal_features_reg)\n",
    "    - set(ordinal_features_oth)\n",
    "    - set(drop_features)\n",
    ")\n",
    "categorical_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- We are not doing it here but we can engineer our own features too. \n",
    "- Would price per square foot be a good feature to add in here?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Applying feature transformations\n",
    "\n",
    "- Since we have mixed feature types, let's use `ColumnTransformer` to apply different transformations on different features types.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.532630Z",
     "iopub.status.busy": "2024-05-29T14:52:05.530806Z",
     "iopub.status.idle": "2024-05-29T14:52:05.551634Z",
     "shell.execute_reply": "2024-05-29T14:52:05.548752Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.532504Z"
    },
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.compose import make_column_transformer\n",
    "\n",
    "numeric_transformer = make_pipeline(SimpleImputer(strategy=\"median\"), StandardScaler())\n",
    "ordinal_transformer_reg = make_pipeline(\n",
    "    SimpleImputer(strategy=\"most_frequent\"),\n",
    "    OrdinalEncoder(categories=ordering_ordinal_reg),\n",
    ")\n",
    "\n",
    "ordinal_transformer_oth = make_pipeline(\n",
    "    SimpleImputer(strategy=\"most_frequent\"),\n",
    "    OrdinalEncoder(categories=ordering_ordinal_oth),\n",
    ")\n",
    "\n",
    "categorical_transformer = make_pipeline(\n",
    "    SimpleImputer(strategy=\"constant\", fill_value=\"missing\"),\n",
    "    OneHotEncoder(handle_unknown=\"ignore\", sparse_output=False),\n",
    ")\n",
    "\n",
    "preprocessor = make_column_transformer(\n",
    "    (\"drop\", drop_features),\n",
    "    (numeric_transformer, numeric_features),\n",
    "    (ordinal_transformer_reg, ordinal_features_reg),\n",
    "    (ordinal_transformer_oth, ordinal_features_oth),\n",
    "    (categorical_transformer, categorical_features),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Examining the preprocessed data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.558730Z",
     "iopub.status.busy": "2024-05-29T14:52:05.556790Z",
     "iopub.status.idle": "2024-05-29T14:52:05.816775Z",
     "shell.execute_reply": "2024-05-29T14:52:05.814589Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.558660Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                (&#x27;pipeline-1&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;standardscaler&#x27;,\n",
       "                                                  StandardScaler())]),\n",
       "                                 [&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;,\n",
       "                                  &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                  &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;,\n",
       "                                  &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;,\n",
       "                                  &#x27;TotalBsmtSF&#x27;, &#x27;...\n",
       "                                 [&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;,\n",
       "                                  &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;,\n",
       "                                  &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;,\n",
       "                                  &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;,\n",
       "                                  &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;,\n",
       "                                  &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                  &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;,\n",
       "                                  &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                  &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;,\n",
       "                                  &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;,\n",
       "                                  &#x27;GarageType&#x27;])])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">ColumnTransformer</label><div class=\"sk-toggleable__content\"><pre>ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                (&#x27;pipeline-1&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;standardscaler&#x27;,\n",
       "                                                  StandardScaler())]),\n",
       "                                 [&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;,\n",
       "                                  &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                  &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;,\n",
       "                                  &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;,\n",
       "                                  &#x27;TotalBsmtSF&#x27;, &#x27;...\n",
       "                                 [&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;,\n",
       "                                  &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;,\n",
       "                                  &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;,\n",
       "                                  &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;,\n",
       "                                  &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;,\n",
       "                                  &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                  &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;,\n",
       "                                  &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                  &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;,\n",
       "                                  &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;,\n",
       "                                  &#x27;GarageType&#x27;])])</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Id&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>drop</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-1</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;, &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;, &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;, &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;, &#x27;TotalBsmtSF&#x27;, &#x27;1stFlrSF&#x27;, &#x27;2ndFlrSF&#x27;, &#x27;LowQualFinSF&#x27;, &#x27;GrLivArea&#x27;, &#x27;BsmtFullBath&#x27;, &#x27;BsmtHalfBath&#x27;, &#x27;FullBath&#x27;, &#x27;HalfBath&#x27;, &#x27;TotRmsAbvGrd&#x27;, &#x27;Fireplaces&#x27;, &#x27;GarageYrBlt&#x27;, &#x27;GarageCars&#x27;, &#x27;GarageArea&#x27;, &#x27;WoodDeckSF&#x27;, &#x27;OpenPorchSF&#x27;, &#x27;EnclosedPorch&#x27;, &#x27;3SsnPorch&#x27;, &#x27;ScreenPorch&#x27;, &#x27;PoolArea&#x27;, &#x27;MiscVal&#x27;, &#x27;YrSold&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" ><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">StandardScaler</label><div class=\"sk-toggleable__content\"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-2</label><div class=\"sk-toggleable__content\"><pre>[&#x27;ExterQual&#x27;, &#x27;ExterCond&#x27;, &#x27;BsmtQual&#x27;, &#x27;BsmtCond&#x27;, &#x27;HeatingQC&#x27;, &#x27;KitchenQual&#x27;, &#x27;FireplaceQu&#x27;, &#x27;GarageQual&#x27;, &#x27;GarageCond&#x27;, &#x27;PoolQC&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" ><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-3</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BsmtExposure&#x27;, &#x27;BsmtFinType1&#x27;, &#x27;BsmtFinType2&#x27;, &#x27;Functional&#x27;, &#x27;Fence&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" ><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" ><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;NA&#x27;, &#x27;No&#x27;, &#x27;Mn&#x27;, &#x27;Av&#x27;, &#x27;Gd&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;Sal&#x27;, &#x27;Sev&#x27;, &#x27;Maj2&#x27;, &#x27;Maj1&#x27;, &#x27;Mod&#x27;, &#x27;Min2&#x27;, &#x27;Min1&#x27;,\n",
       "                            &#x27;Typ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;MnWw&#x27;, &#x27;GdWo&#x27;, &#x27;MnPrv&#x27;, &#x27;GdPrv&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" ><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-4</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;, &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;, &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;, &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;, &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;, &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;, &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;, &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;, &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;, &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;, &#x27;GarageType&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" ><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(fill_value=&#x27;missing&#x27;, strategy=&#x27;constant&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" ><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder(handle_unknown=&#x27;ignore&#x27;, sparse_output=False)</pre></div></div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "ColumnTransformer(transformers=[('drop', 'drop', ['Id']),\n",
       "                                ('pipeline-1',\n",
       "                                 Pipeline(steps=[('simpleimputer',\n",
       "                                                  SimpleImputer(strategy='median')),\n",
       "                                                 ('standardscaler',\n",
       "                                                  StandardScaler())]),\n",
       "                                 ['BedroomAbvGr', 'KitchenAbvGr', 'LotFrontage',\n",
       "                                  'LotArea', 'OverallQual', 'OverallCond',\n",
       "                                  'YearBuilt', 'YearRemodAdd', 'MasVnrArea',\n",
       "                                  'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF',\n",
       "                                  'TotalBsmtSF', '...\n",
       "                                 ['Alley', 'MSSubClass', 'Neighborhood',\n",
       "                                  'Utilities', 'Exterior1st', 'MasVnrType',\n",
       "                                  'LotShape', 'LandContour', 'MoSold',\n",
       "                                  'MSZoning', 'Street', 'GarageFinish',\n",
       "                                  'HouseStyle', 'LotConfig', 'PavedDrive',\n",
       "                                  'BldgType', 'LandSlope', 'Condition1',\n",
       "                                  'MiscFeature', 'Electrical', 'CentralAir',\n",
       "                                  'RoofMatl', 'Heating', 'RoofStyle',\n",
       "                                  'SaleType', 'Exterior2nd', 'Condition2',\n",
       "                                  'SaleCondition', 'Foundation',\n",
       "                                  'GarageType'])])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor.fit(X_train)  # Calling fit to examine all the transformers.\n",
    "preprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.820615Z",
     "iopub.status.busy": "2024-05-29T14:52:05.819551Z",
     "iopub.status.idle": "2024-05-29T14:52:05.854836Z",
     "shell.execute_reply": "2024-05-29T14:52:05.852003Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.820572Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'drop': 'drop',\n",
       " 'pipeline-1': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='median')),\n",
       "                 ('standardscaler', StandardScaler())]),\n",
       " 'pipeline-2': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='most_frequent')),\n",
       "                 ('ordinalencoder',\n",
       "                  OrdinalEncoder(categories=[['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex'],\n",
       "                                             ['Po', 'Fa', 'TA', 'Gd', 'Ex']]))]),\n",
       " 'pipeline-3': Pipeline(steps=[('simpleimputer', SimpleImputer(strategy='most_frequent')),\n",
       "                 ('ordinalencoder',\n",
       "                  OrdinalEncoder(categories=[['NA', 'No', 'Mn', 'Av', 'Gd'],\n",
       "                                             ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ',\n",
       "                                              'ALQ', 'GLQ'],\n",
       "                                             ['NA', 'Unf', 'LwQ', 'Rec', 'BLQ',\n",
       "                                              'ALQ', 'GLQ'],\n",
       "                                             ['Sal', 'Sev', 'Maj2', 'Maj1',\n",
       "                                              'Mod', 'Min2', 'Min1', 'Typ'],\n",
       "                                             ['NA', 'MnWw', 'GdWo', 'MnPrv',\n",
       "                                              'GdPrv']]))]),\n",
       " 'pipeline-4': Pipeline(steps=[('simpleimputer',\n",
       "                  SimpleImputer(fill_value='missing', strategy='constant')),\n",
       "                 ('onehotencoder',\n",
       "                  OneHotEncoder(handle_unknown='ignore', sparse_output=False))])}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor.named_transformers_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.860385Z",
     "iopub.status.busy": "2024-05-29T14:52:05.857051Z",
     "iopub.status.idle": "2024-05-29T14:52:05.881140Z",
     "shell.execute_reply": "2024-05-29T14:52:05.878551Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.860259Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['drop', 'pipeline-1', 'pipeline-2', 'pipeline-3', 'pipeline-4'])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor.named_transformers_.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.888914Z",
     "iopub.status.busy": "2024-05-29T14:52:05.885062Z",
     "iopub.status.idle": "2024-05-29T14:52:05.898765Z",
     "shell.execute_reply": "2024-05-29T14:52:05.896466Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.888827Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "ohe_columns = list(\n",
    "    preprocessor.named_transformers_[\"pipeline-4\"]\n",
    "    .named_steps[\"onehotencoder\"]\n",
    "    .get_feature_names_out(categorical_features)\n",
    ")\n",
    "new_columns = (\n",
    "    numeric_features + ordinal_features_reg + ordinal_features_oth + ohe_columns\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:05.917753Z",
     "iopub.status.busy": "2024-05-29T14:52:05.916211Z",
     "iopub.status.idle": "2024-05-29T14:52:06.061176Z",
     "shell.execute_reply": "2024-05-29T14:52:06.057518Z",
     "shell.execute_reply.started": "2024-05-29T14:52:05.917637Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BedroomAbvGr</th>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>...</th>\n",
       "      <th>Foundation_Slab</th>\n",
       "      <th>Foundation_Stone</th>\n",
       "      <th>Foundation_Wood</th>\n",
       "      <th>GarageType_2Types</th>\n",
       "      <th>GarageType_Attchd</th>\n",
       "      <th>GarageType_Basment</th>\n",
       "      <th>GarageType_BuiltIn</th>\n",
       "      <th>GarageType_CarPort</th>\n",
       "      <th>GarageType_Detchd</th>\n",
       "      <th>GarageType_missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>2.312501</td>\n",
       "      <td>0.381428</td>\n",
       "      <td>0.663680</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.993969</td>\n",
       "      <td>0.840492</td>\n",
       "      <td>0.269972</td>\n",
       "      <td>-0.961498</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767</th>\n",
       "      <td>1.372763</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>0.260890</td>\n",
       "      <td>0.248457</td>\n",
       "      <td>-0.054669</td>\n",
       "      <td>1.285467</td>\n",
       "      <td>-1.026793</td>\n",
       "      <td>0.016525</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.476092</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>2.885044</td>\n",
       "      <td>0.131607</td>\n",
       "      <td>-0.054669</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.563314</td>\n",
       "      <td>0.161931</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>1.227559</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1139</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>1.358264</td>\n",
       "      <td>-0.171468</td>\n",
       "      <td>-0.773017</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>-1.689338</td>\n",
       "      <td>-1.679877</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.443419</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>558</th>\n",
       "      <td>0.154795</td>\n",
       "      <td>-0.222647</td>\n",
       "      <td>-0.597924</td>\n",
       "      <td>1.289541</td>\n",
       "      <td>0.663680</td>\n",
       "      <td>-0.512408</td>\n",
       "      <td>0.828332</td>\n",
       "      <td>0.598149</td>\n",
       "      <td>-0.573129</td>\n",
       "      <td>0.354114</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 262 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      BedroomAbvGr  KitchenAbvGr  LotFrontage   LotArea  OverallQual  \\\n",
       "302       0.154795     -0.222647     2.312501  0.381428     0.663680   \n",
       "767       1.372763     -0.222647     0.260890  0.248457    -0.054669   \n",
       "429       0.154795     -0.222647     2.885044  0.131607    -0.054669   \n",
       "1139      0.154795     -0.222647     1.358264 -0.171468    -0.773017   \n",
       "558       0.154795     -0.222647    -0.597924  1.289541     0.663680   \n",
       "\n",
       "      OverallCond  YearBuilt  YearRemodAdd  MasVnrArea  BsmtFinSF1  ...  \\\n",
       "302     -0.512408   0.993969      0.840492    0.269972   -0.961498  ...   \n",
       "767      1.285467  -1.026793      0.016525   -0.573129    0.476092  ...   \n",
       "429     -0.512408   0.563314      0.161931   -0.573129    1.227559  ...   \n",
       "1139    -0.512408  -1.689338     -1.679877   -0.573129    0.443419  ...   \n",
       "558     -0.512408   0.828332      0.598149   -0.573129    0.354114  ...   \n",
       "\n",
       "      Foundation_Slab  Foundation_Stone  Foundation_Wood  GarageType_2Types  \\\n",
       "302               0.0               0.0              0.0                0.0   \n",
       "767               0.0               0.0              0.0                0.0   \n",
       "429               0.0               0.0              0.0                0.0   \n",
       "1139              0.0               0.0              0.0                0.0   \n",
       "558               0.0               0.0              0.0                0.0   \n",
       "\n",
       "      GarageType_Attchd  GarageType_Basment  GarageType_BuiltIn  \\\n",
       "302                 1.0                 0.0                 0.0   \n",
       "767                 1.0                 0.0                 0.0   \n",
       "429                 1.0                 0.0                 0.0   \n",
       "1139                0.0                 0.0                 0.0   \n",
       "558                 1.0                 0.0                 0.0   \n",
       "\n",
       "      GarageType_CarPort  GarageType_Detchd  GarageType_missing  \n",
       "302                  0.0                0.0                 0.0  \n",
       "767                  0.0                0.0                 0.0  \n",
       "429                  0.0                0.0                 0.0  \n",
       "1139                 0.0                1.0                 0.0  \n",
       "558                  0.0                0.0                 0.0  \n",
       "\n",
       "[5 rows x 262 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_enc = pd.DataFrame(\n",
    "    preprocessor.transform(X_train), index=X_train.index, columns=new_columns\n",
    ")\n",
    "X_train_enc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.068955Z",
     "iopub.status.busy": "2024-05-29T14:52:06.065060Z",
     "iopub.status.idle": "2024-05-29T14:52:06.083859Z",
     "shell.execute_reply": "2024-05-29T14:52:06.079756Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.068900Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 80)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.090308Z",
     "iopub.status.busy": "2024-05-29T14:52:06.088147Z",
     "iopub.status.idle": "2024-05-29T14:52:06.101517Z",
     "shell.execute_reply": "2024-05-29T14:52:06.098912Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.090260Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1314, 262)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_enc.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We went from 80 features to 262 features!! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Other possible preprocessing?  \n",
    "\n",
    "- There is a lot of room for improvement ...\n",
    "- We're just using `SimpleImputer`.\n",
    "    - In reality we'd want to go through this more carefully.\n",
    "    - We may also want to drop some columns that are almost entirely missing.    \n",
    "- We could also check for outliers, and do other exploratory data analysis (EDA).\n",
    "- But for now this is good enough ...    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<br><br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Model building "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `DummyRegressor`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.106452Z",
     "iopub.status.busy": "2024-05-29T14:52:06.104375Z",
     "iopub.status.idle": "2024-05-29T14:52:06.192111Z",
     "shell.execute_reply": "2024-05-29T14:52:06.188723Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.106373Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.002971</td>\n",
       "      <td>0.001457</td>\n",
       "      <td>-0.003547</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.002372</td>\n",
       "      <td>0.000798</td>\n",
       "      <td>-0.001266</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.001567</td>\n",
       "      <td>0.000826</td>\n",
       "      <td>-0.011767</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.001546</td>\n",
       "      <td>0.000743</td>\n",
       "      <td>-0.006744</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.002560</td>\n",
       "      <td>0.001202</td>\n",
       "      <td>-0.076533</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.001707</td>\n",
       "      <td>0.000882</td>\n",
       "      <td>-0.003133</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.002405</td>\n",
       "      <td>0.001830</td>\n",
       "      <td>-0.000397</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.003828</td>\n",
       "      <td>0.002762</td>\n",
       "      <td>-0.003785</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.002508</td>\n",
       "      <td>0.001337</td>\n",
       "      <td>-0.001740</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.002634</td>\n",
       "      <td>0.001301</td>\n",
       "      <td>-0.000117</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.002971    0.001457   -0.003547          0.0\n",
       "1  0.002372    0.000798   -0.001266          0.0\n",
       "2  0.001567    0.000826   -0.011767          0.0\n",
       "3  0.001546    0.000743   -0.006744          0.0\n",
       "4  0.002560    0.001202   -0.076533          0.0\n",
       "5  0.001707    0.000882   -0.003133          0.0\n",
       "6  0.002405    0.001830   -0.000397          0.0\n",
       "7  0.003828    0.002762   -0.003785          0.0\n",
       "8  0.002508    0.001337   -0.001740          0.0\n",
       "9  0.002634    0.001301   -0.000117          0.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy = DummyRegressor()\n",
    "pd.DataFrame(cross_validate(dummy, X_train, y_train, cv=10, return_train_score=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's try a linear model: `Ridge`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Recall that we are going to use `Ridge()` instead of `LinearRegression()` in this course. \n",
    "- Similar to linear regression, ridge regression is also a linear model for regression. \n",
    "- So the formula it uses to make predictions is the same one used for ordinary least squares. \n",
    "- But it has a hyperparameter `alpha` which controls the fundamental tradeoff.     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.198116Z",
     "iopub.status.busy": "2024-05-29T14:52:06.197174Z",
     "iopub.status.idle": "2024-05-29T14:52:06.531267Z",
     "shell.execute_reply": "2024-05-29T14:52:06.522042Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.198031Z"
    }
   },
   "outputs": [],
   "source": [
    "lr = make_pipeline(preprocessor, Ridge())\n",
    "lr.fit(X_train, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.534502Z",
     "iopub.status.busy": "2024-05-29T14:52:06.533461Z",
     "iopub.status.idle": "2024-05-29T14:52:06.677872Z",
     "shell.execute_reply": "2024-05-29T14:52:06.674664Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.534434Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([224865.34161762,  65243.29045704, 133337.46729488, 250900.65424895,\n",
       "       128984.32127053, 205597.8361313 , 334576.71199006, 158982.72345029,\n",
       "       148343.09009799, 129257.94933723])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_preds = lr.predict(X_test)\n",
    "lr_preds[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.681848Z",
     "iopub.status.busy": "2024-05-29T14:52:06.680664Z",
     "iopub.status.idle": "2024-05-29T14:52:06.752626Z",
     "shell.execute_reply": "2024-05-29T14:52:06.718551Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.681753Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(411412.6718650306, 38853.27502379057)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_preds.max(), lr_preds.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.795026Z",
     "iopub.status.busy": "2024-05-29T14:52:06.784722Z",
     "iopub.status.idle": "2024-05-29T14:52:06.815118Z",
     "shell.execute_reply": "2024-05-29T14:52:06.805362Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.794959Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Smallest coefficient:  -191169.0717446897\n",
      "Largest coefficient: 83603.01311996009\n"
     ]
    }
   ],
   "source": [
    "print(\"Smallest coefficient: \", lr.named_steps[\"ridge\"].coef_.min())\n",
    "print(\"Largest coefficient:\", lr.named_steps[\"ridge\"].coef_.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's carry out cross-validation with `Ridge`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:06.820510Z",
     "iopub.status.busy": "2024-05-29T14:52:06.819833Z",
     "iopub.status.idle": "2024-05-29T14:52:11.480708Z",
     "shell.execute_reply": "2024-05-29T14:52:11.477477Z",
     "shell.execute_reply.started": "2024-05-29T14:52:06.820432Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.131161</td>\n",
       "      <td>0.097820</td>\n",
       "      <td>0.862432</td>\n",
       "      <td>0.911846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.164345</td>\n",
       "      <td>0.070076</td>\n",
       "      <td>0.812363</td>\n",
       "      <td>0.913849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.279829</td>\n",
       "      <td>0.070386</td>\n",
       "      <td>0.775270</td>\n",
       "      <td>0.915952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.206024</td>\n",
       "      <td>0.085858</td>\n",
       "      <td>0.874532</td>\n",
       "      <td>0.910843</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.302397</td>\n",
       "      <td>0.097927</td>\n",
       "      <td>0.852104</td>\n",
       "      <td>0.911615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.184979</td>\n",
       "      <td>0.084040</td>\n",
       "      <td>0.826142</td>\n",
       "      <td>0.910174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.244142</td>\n",
       "      <td>0.101946</td>\n",
       "      <td>0.825697</td>\n",
       "      <td>0.913767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.175739</td>\n",
       "      <td>0.078021</td>\n",
       "      <td>0.873704</td>\n",
       "      <td>0.910018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.224848</td>\n",
       "      <td>0.083200</td>\n",
       "      <td>0.196599</td>\n",
       "      <td>0.921438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.274215</td>\n",
       "      <td>0.079731</td>\n",
       "      <td>0.890457</td>\n",
       "      <td>0.908215</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.131161    0.097820    0.862432     0.911846\n",
       "1  0.164345    0.070076    0.812363     0.913849\n",
       "2  0.279829    0.070386    0.775270     0.915952\n",
       "3  0.206024    0.085858    0.874532     0.910843\n",
       "4  0.302397    0.097927    0.852104     0.911615\n",
       "5  0.184979    0.084040    0.826142     0.910174\n",
       "6  0.244142    0.101946    0.825697     0.913767\n",
       "7  0.175739    0.078021    0.873704     0.910018\n",
       "8  0.224848    0.083200    0.196599     0.921438\n",
       "9  0.274215    0.079731    0.890457     0.908215"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_pipe = make_pipeline(preprocessor, Ridge())\n",
    "pd.DataFrame(cross_validate(lr_pipe, X_train, y_train, cv=10, return_train_score=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Quite a bit of variation in the test scores. \n",
    "- Performing poorly in fold 8. Not sure why. \n",
    "    - Probably it contains the outliers in the data which we kind of ignored. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Feature names of transformed data\n",
    "\n",
    "- If you want to get the column names of newly created columns, you need to fit the transformer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:11.495167Z",
     "iopub.status.busy": "2024-05-29T14:52:11.494523Z",
     "iopub.status.idle": "2024-05-29T14:52:11.745253Z",
     "shell.execute_reply": "2024-05-29T14:52:11.740586Z",
     "shell.execute_reply.started": "2024-05-29T14:52:11.495104Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                (&#x27;pipeline-1&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;standardscaler&#x27;,\n",
       "                                                  StandardScaler())]),\n",
       "                                 [&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;,\n",
       "                                  &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                  &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;,\n",
       "                                  &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;,\n",
       "                                  &#x27;TotalBsmtSF&#x27;, &#x27;...\n",
       "                                 [&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;,\n",
       "                                  &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;,\n",
       "                                  &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;,\n",
       "                                  &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;,\n",
       "                                  &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;,\n",
       "                                  &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                  &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;,\n",
       "                                  &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                  &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;,\n",
       "                                  &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;,\n",
       "                                  &#x27;GarageType&#x27;])])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-16\" type=\"checkbox\" ><label for=\"sk-estimator-id-16\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">ColumnTransformer</label><div class=\"sk-toggleable__content\"><pre>ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                (&#x27;pipeline-1&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;standardscaler&#x27;,\n",
       "                                                  StandardScaler())]),\n",
       "                                 [&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;,\n",
       "                                  &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                  &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;,\n",
       "                                  &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;,\n",
       "                                  &#x27;TotalBsmtSF&#x27;, &#x27;...\n",
       "                                 [&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;,\n",
       "                                  &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;,\n",
       "                                  &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;,\n",
       "                                  &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;,\n",
       "                                  &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;,\n",
       "                                  &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                  &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;,\n",
       "                                  &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                  &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;,\n",
       "                                  &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;,\n",
       "                                  &#x27;GarageType&#x27;])])</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-17\" type=\"checkbox\" ><label for=\"sk-estimator-id-17\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Id&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-18\" type=\"checkbox\" ><label for=\"sk-estimator-id-18\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>drop</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-19\" type=\"checkbox\" ><label for=\"sk-estimator-id-19\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-1</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;, &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;, &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;, &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;, &#x27;TotalBsmtSF&#x27;, &#x27;1stFlrSF&#x27;, &#x27;2ndFlrSF&#x27;, &#x27;LowQualFinSF&#x27;, &#x27;GrLivArea&#x27;, &#x27;BsmtFullBath&#x27;, &#x27;BsmtHalfBath&#x27;, &#x27;FullBath&#x27;, &#x27;HalfBath&#x27;, &#x27;TotRmsAbvGrd&#x27;, &#x27;Fireplaces&#x27;, &#x27;GarageYrBlt&#x27;, &#x27;GarageCars&#x27;, &#x27;GarageArea&#x27;, &#x27;WoodDeckSF&#x27;, &#x27;OpenPorchSF&#x27;, &#x27;EnclosedPorch&#x27;, &#x27;3SsnPorch&#x27;, &#x27;ScreenPorch&#x27;, &#x27;PoolArea&#x27;, &#x27;MiscVal&#x27;, &#x27;YrSold&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-20\" type=\"checkbox\" ><label for=\"sk-estimator-id-20\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-21\" type=\"checkbox\" ><label for=\"sk-estimator-id-21\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">StandardScaler</label><div class=\"sk-toggleable__content\"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-22\" type=\"checkbox\" ><label for=\"sk-estimator-id-22\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-2</label><div class=\"sk-toggleable__content\"><pre>[&#x27;ExterQual&#x27;, &#x27;ExterCond&#x27;, &#x27;BsmtQual&#x27;, &#x27;BsmtCond&#x27;, &#x27;HeatingQC&#x27;, &#x27;KitchenQual&#x27;, &#x27;FireplaceQu&#x27;, &#x27;GarageQual&#x27;, &#x27;GarageCond&#x27;, &#x27;PoolQC&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-23\" type=\"checkbox\" ><label for=\"sk-estimator-id-23\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-24\" type=\"checkbox\" ><label for=\"sk-estimator-id-24\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-25\" type=\"checkbox\" ><label for=\"sk-estimator-id-25\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-3</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BsmtExposure&#x27;, &#x27;BsmtFinType1&#x27;, &#x27;BsmtFinType2&#x27;, &#x27;Functional&#x27;, &#x27;Fence&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-26\" type=\"checkbox\" ><label for=\"sk-estimator-id-26\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-27\" type=\"checkbox\" ><label for=\"sk-estimator-id-27\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;NA&#x27;, &#x27;No&#x27;, &#x27;Mn&#x27;, &#x27;Av&#x27;, &#x27;Gd&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;Sal&#x27;, &#x27;Sev&#x27;, &#x27;Maj2&#x27;, &#x27;Maj1&#x27;, &#x27;Mod&#x27;, &#x27;Min2&#x27;, &#x27;Min1&#x27;,\n",
       "                            &#x27;Typ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;MnWw&#x27;, &#x27;GdWo&#x27;, &#x27;MnPrv&#x27;, &#x27;GdPrv&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-28\" type=\"checkbox\" ><label for=\"sk-estimator-id-28\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-4</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;, &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;, &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;, &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;, &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;, &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;, &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;, &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;, &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;, &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;, &#x27;GarageType&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-29\" type=\"checkbox\" ><label for=\"sk-estimator-id-29\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(fill_value=&#x27;missing&#x27;, strategy=&#x27;constant&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-30\" type=\"checkbox\" ><label for=\"sk-estimator-id-30\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder(handle_unknown=&#x27;ignore&#x27;, sparse_output=False)</pre></div></div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "ColumnTransformer(transformers=[('drop', 'drop', ['Id']),\n",
       "                                ('pipeline-1',\n",
       "                                 Pipeline(steps=[('simpleimputer',\n",
       "                                                  SimpleImputer(strategy='median')),\n",
       "                                                 ('standardscaler',\n",
       "                                                  StandardScaler())]),\n",
       "                                 ['BedroomAbvGr', 'KitchenAbvGr', 'LotFrontage',\n",
       "                                  'LotArea', 'OverallQual', 'OverallCond',\n",
       "                                  'YearBuilt', 'YearRemodAdd', 'MasVnrArea',\n",
       "                                  'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF',\n",
       "                                  'TotalBsmtSF', '...\n",
       "                                 ['Alley', 'MSSubClass', 'Neighborhood',\n",
       "                                  'Utilities', 'Exterior1st', 'MasVnrType',\n",
       "                                  'LotShape', 'LandContour', 'MoSold',\n",
       "                                  'MSZoning', 'Street', 'GarageFinish',\n",
       "                                  'HouseStyle', 'LotConfig', 'PavedDrive',\n",
       "                                  'BldgType', 'LandSlope', 'Condition1',\n",
       "                                  'MiscFeature', 'Electrical', 'CentralAir',\n",
       "                                  'RoofMatl', 'Heating', 'RoofStyle',\n",
       "                                  'SaleType', 'Exterior2nd', 'Condition2',\n",
       "                                  'SaleCondition', 'Foundation',\n",
       "                                  'GarageType'])])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:11.752479Z",
     "iopub.status.busy": "2024-05-29T14:52:11.751872Z",
     "iopub.status.idle": "2024-05-29T14:52:11.789369Z",
     "shell.execute_reply": "2024-05-29T14:52:11.787043Z",
     "shell.execute_reply.started": "2024-05-29T14:52:11.752433Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Alley_Grvl',\n",
       " 'Alley_Pave',\n",
       " 'Alley_missing',\n",
       " 'MSSubClass_20',\n",
       " 'MSSubClass_30',\n",
       " 'MSSubClass_40',\n",
       " 'MSSubClass_45',\n",
       " 'MSSubClass_50',\n",
       " 'MSSubClass_60',\n",
       " 'MSSubClass_70',\n",
       " 'MSSubClass_75',\n",
       " 'MSSubClass_80',\n",
       " 'MSSubClass_85',\n",
       " 'MSSubClass_90',\n",
       " 'MSSubClass_120',\n",
       " 'MSSubClass_160',\n",
       " 'MSSubClass_180',\n",
       " 'MSSubClass_190',\n",
       " 'Neighborhood_Blmngtn',\n",
       " 'Neighborhood_Blueste',\n",
       " 'Neighborhood_BrDale',\n",
       " 'Neighborhood_BrkSide',\n",
       " 'Neighborhood_ClearCr',\n",
       " 'Neighborhood_CollgCr',\n",
       " 'Neighborhood_Crawfor',\n",
       " 'Neighborhood_Edwards',\n",
       " 'Neighborhood_Gilbert',\n",
       " 'Neighborhood_IDOTRR',\n",
       " 'Neighborhood_MeadowV',\n",
       " 'Neighborhood_Mitchel',\n",
       " 'Neighborhood_NAmes',\n",
       " 'Neighborhood_NPkVill',\n",
       " 'Neighborhood_NWAmes',\n",
       " 'Neighborhood_NoRidge',\n",
       " 'Neighborhood_NridgHt',\n",
       " 'Neighborhood_OldTown',\n",
       " 'Neighborhood_SWISU',\n",
       " 'Neighborhood_Sawyer',\n",
       " 'Neighborhood_SawyerW',\n",
       " 'Neighborhood_Somerst',\n",
       " 'Neighborhood_StoneBr',\n",
       " 'Neighborhood_Timber',\n",
       " 'Neighborhood_Veenker',\n",
       " 'Utilities_AllPub',\n",
       " 'Utilities_NoSeWa',\n",
       " 'Exterior1st_AsbShng',\n",
       " 'Exterior1st_AsphShn',\n",
       " 'Exterior1st_BrkComm',\n",
       " 'Exterior1st_BrkFace',\n",
       " 'Exterior1st_CBlock',\n",
       " 'Exterior1st_CemntBd',\n",
       " 'Exterior1st_HdBoard',\n",
       " 'Exterior1st_ImStucc',\n",
       " 'Exterior1st_MetalSd',\n",
       " 'Exterior1st_Plywood',\n",
       " 'Exterior1st_Stone',\n",
       " 'Exterior1st_Stucco',\n",
       " 'Exterior1st_VinylSd',\n",
       " 'Exterior1st_Wd Sdng',\n",
       " 'Exterior1st_WdShing',\n",
       " 'MasVnrType_BrkCmn',\n",
       " 'MasVnrType_BrkFace',\n",
       " 'MasVnrType_Stone',\n",
       " 'MasVnrType_missing',\n",
       " 'LotShape_IR1',\n",
       " 'LotShape_IR2',\n",
       " 'LotShape_IR3',\n",
       " 'LotShape_Reg',\n",
       " 'LandContour_Bnk',\n",
       " 'LandContour_HLS',\n",
       " 'LandContour_Low',\n",
       " 'LandContour_Lvl',\n",
       " 'MoSold_1',\n",
       " 'MoSold_2',\n",
       " 'MoSold_3',\n",
       " 'MoSold_4',\n",
       " 'MoSold_5',\n",
       " 'MoSold_6',\n",
       " 'MoSold_7',\n",
       " 'MoSold_8',\n",
       " 'MoSold_9',\n",
       " 'MoSold_10',\n",
       " 'MoSold_11',\n",
       " 'MoSold_12',\n",
       " 'MSZoning_C (all)',\n",
       " 'MSZoning_FV',\n",
       " 'MSZoning_RH',\n",
       " 'MSZoning_RL',\n",
       " 'MSZoning_RM',\n",
       " 'Street_Grvl',\n",
       " 'Street_Pave',\n",
       " 'GarageFinish_Fin',\n",
       " 'GarageFinish_RFn',\n",
       " 'GarageFinish_Unf',\n",
       " 'GarageFinish_missing',\n",
       " 'HouseStyle_1.5Fin',\n",
       " 'HouseStyle_1.5Unf',\n",
       " 'HouseStyle_1Story',\n",
       " 'HouseStyle_2.5Fin',\n",
       " 'HouseStyle_2.5Unf',\n",
       " 'HouseStyle_2Story',\n",
       " 'HouseStyle_SFoyer',\n",
       " 'HouseStyle_SLvl',\n",
       " 'LotConfig_Corner',\n",
       " 'LotConfig_CulDSac',\n",
       " 'LotConfig_FR2',\n",
       " 'LotConfig_FR3',\n",
       " 'LotConfig_Inside',\n",
       " 'PavedDrive_N',\n",
       " 'PavedDrive_P',\n",
       " 'PavedDrive_Y',\n",
       " 'BldgType_1Fam',\n",
       " 'BldgType_2fmCon',\n",
       " 'BldgType_Duplex',\n",
       " 'BldgType_Twnhs',\n",
       " 'BldgType_TwnhsE',\n",
       " 'LandSlope_Gtl',\n",
       " 'LandSlope_Mod',\n",
       " 'LandSlope_Sev',\n",
       " 'Condition1_Artery',\n",
       " 'Condition1_Feedr',\n",
       " 'Condition1_Norm',\n",
       " 'Condition1_PosA',\n",
       " 'Condition1_PosN',\n",
       " 'Condition1_RRAe',\n",
       " 'Condition1_RRAn',\n",
       " 'Condition1_RRNe',\n",
       " 'Condition1_RRNn',\n",
       " 'MiscFeature_Gar2',\n",
       " 'MiscFeature_Othr',\n",
       " 'MiscFeature_Shed',\n",
       " 'MiscFeature_TenC',\n",
       " 'MiscFeature_missing',\n",
       " 'Electrical_FuseA',\n",
       " 'Electrical_FuseF',\n",
       " 'Electrical_FuseP',\n",
       " 'Electrical_Mix',\n",
       " 'Electrical_SBrkr',\n",
       " 'Electrical_missing',\n",
       " 'CentralAir_N',\n",
       " 'CentralAir_Y',\n",
       " 'RoofMatl_ClyTile',\n",
       " 'RoofMatl_CompShg',\n",
       " 'RoofMatl_Membran',\n",
       " 'RoofMatl_Metal',\n",
       " 'RoofMatl_Roll',\n",
       " 'RoofMatl_Tar&Grv',\n",
       " 'RoofMatl_WdShake',\n",
       " 'RoofMatl_WdShngl',\n",
       " 'Heating_Floor',\n",
       " 'Heating_GasA',\n",
       " 'Heating_GasW',\n",
       " 'Heating_Grav',\n",
       " 'Heating_OthW',\n",
       " 'Heating_Wall',\n",
       " 'RoofStyle_Flat',\n",
       " 'RoofStyle_Gable',\n",
       " 'RoofStyle_Gambrel',\n",
       " 'RoofStyle_Hip',\n",
       " 'RoofStyle_Mansard',\n",
       " 'RoofStyle_Shed',\n",
       " 'SaleType_COD',\n",
       " 'SaleType_CWD',\n",
       " 'SaleType_Con',\n",
       " 'SaleType_ConLD',\n",
       " 'SaleType_ConLI',\n",
       " 'SaleType_ConLw',\n",
       " 'SaleType_New',\n",
       " 'SaleType_Oth',\n",
       " 'SaleType_WD',\n",
       " 'Exterior2nd_AsbShng',\n",
       " 'Exterior2nd_AsphShn',\n",
       " 'Exterior2nd_Brk Cmn',\n",
       " 'Exterior2nd_BrkFace',\n",
       " 'Exterior2nd_CBlock',\n",
       " 'Exterior2nd_CmentBd',\n",
       " 'Exterior2nd_HdBoard',\n",
       " 'Exterior2nd_ImStucc',\n",
       " 'Exterior2nd_MetalSd',\n",
       " 'Exterior2nd_Other',\n",
       " 'Exterior2nd_Plywood',\n",
       " 'Exterior2nd_Stone',\n",
       " 'Exterior2nd_Stucco',\n",
       " 'Exterior2nd_VinylSd',\n",
       " 'Exterior2nd_Wd Sdng',\n",
       " 'Exterior2nd_Wd Shng',\n",
       " 'Condition2_Artery',\n",
       " 'Condition2_Feedr',\n",
       " 'Condition2_Norm',\n",
       " 'Condition2_PosA',\n",
       " 'Condition2_PosN',\n",
       " 'Condition2_RRAe',\n",
       " 'Condition2_RRAn',\n",
       " 'Condition2_RRNn',\n",
       " 'SaleCondition_Abnorml',\n",
       " 'SaleCondition_AdjLand',\n",
       " 'SaleCondition_Alloca',\n",
       " 'SaleCondition_Family',\n",
       " 'SaleCondition_Normal',\n",
       " 'SaleCondition_Partial',\n",
       " 'Foundation_BrkTil',\n",
       " 'Foundation_CBlock',\n",
       " 'Foundation_PConc',\n",
       " 'Foundation_Slab',\n",
       " 'Foundation_Stone',\n",
       " 'Foundation_Wood',\n",
       " 'GarageType_2Types',\n",
       " 'GarageType_Attchd',\n",
       " 'GarageType_Basment',\n",
       " 'GarageType_BuiltIn',\n",
       " 'GarageType_CarPort',\n",
       " 'GarageType_Detchd',\n",
       " 'GarageType_missing']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ohe_columns = list(\n",
    "    preprocessor.named_transformers_[\"pipeline-4\"]\n",
    "    .named_steps[\"onehotencoder\"]\n",
    "    .get_feature_names_out(categorical_features)\n",
    ")\n",
    "ohe_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:11.795583Z",
     "iopub.status.busy": "2024-05-29T14:52:11.792991Z",
     "iopub.status.idle": "2024-05-29T14:52:11.810180Z",
     "shell.execute_reply": "2024-05-29T14:52:11.806295Z",
     "shell.execute_reply.started": "2024-05-29T14:52:11.795480Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "new_columns = (\n",
    "    numeric_features + ordinal_features_reg + ordinal_features_oth + ohe_columns\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Tuning `alpha` hyperparameter of `Ridge`\n",
    "- Recall that `Ridge` has a hyperparameter `alpha` that controls the fundamental tradeoff.\n",
    "- This is like `C` in `LogisticRegression` but, annoyingly, `alpha` is the inverse of `C`.\n",
    "- That is, large `C` is like small `alpha` and vice versa.\n",
    "- **Smaller** `alpha`: lower training error (**overfitting**)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:11.816256Z",
     "iopub.status.busy": "2024-05-29T14:52:11.815129Z",
     "iopub.status.idle": "2024-05-29T14:52:22.483247Z",
     "shell.execute_reply": "2024-05-29T14:52:22.480436Z",
     "shell.execute_reply.started": "2024-05-29T14:52:11.816159Z"
    }
   },
   "outputs": [],
   "source": [
    "param_grid = {\"ridge__alpha\": 10.0 ** np.arange(-5, 5, 1)}\n",
    "\n",
    "pipe_ridge = make_pipeline(preprocessor, Ridge())\n",
    "\n",
    "search = GridSearchCV(pipe_ridge, param_grid, return_train_score=True, n_jobs=-1)\n",
    "search.fit(X_train, y_train)\n",
    "train_scores = search.cv_results_[\"mean_train_score\"]\n",
    "cv_scores = search.cv_results_[\"mean_test_score\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:22.497417Z",
     "iopub.status.busy": "2024-05-29T14:52:22.496322Z",
     "iopub.status.idle": "2024-05-29T14:52:23.649254Z",
     "shell.execute_reply": "2024-05-29T14:52:23.646497Z",
     "shell.execute_reply.started": "2024-05-29T14:52:22.497356Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAG1CAYAAAARLUsBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABWa0lEQVR4nO3deXxU5d3//9dM9oRkskEI2QFZAsoSIBBAKyIIdaEbaVUQi1p+31pFKq3cqC3c9qa4FW8t3FqlaEsRq1VpZRE3FlmEsMkuSxZIQiBkI4RsM78/JpkQEiD7mWTez8fjPJKcOXPmczLGeXNd17kuk81msyEiIiLiQsxGFyAiIiLS1hSARERExOUoAImIiIjLUQASERERl6MAJCIiIi5HAUhERERcjgKQiIiIuBwFIBEREXE57kYX4IysViuZmZn4+/tjMpmMLkdEREQawGazUVRURLdu3TCbr93GowBUj8zMTKKioowuQ0RERJogIyODyMjIax6jAFQPf39/wP4LDAgIMLgaERERaYjCwkKioqIcn+PXogBUj+pur4CAAAUgERGRdqYhw1c0CFpERERcjgKQiIiIuBx1gYmIiLSxyspKysvLjS6jXfL09LzuHV4NoQAkIiLSRmw2G9nZ2eTn5xtdSrtlNpuJi4vD09OzWedRABIREWkj1eGnS5cu+Pr6aq65Rqqepy8rK4vo6Ohm/f4UgERERNpAZWWlI/yEhIQYXU671blzZzIzM6moqMDDw6PJ59EgaBERkTZQPebH19fX4Erat+qur8rKymadRwFIRESkDanbq3la6venACQiIiIuRwFIREREXI4CkIiIiLSZ2NhYFi1aZHQZugtMREREru173/seAwcObJHgsmPHDvz8/JpfVDMpALWhYzlFzP/PIaPLkBbQlCF4jR2319jXMJtMuLuZcDebHV893C7bZzbh7la1z3HMNfZVfXXsc5zPfi4PNzNuZpNjX/Xxjn1Vx2vAp0jHZ7PZqKysxN39+rGic+fObVDR9SkAtaHCSxVsPHrW6DJE2pSb2WQPRVeErcvDU3WgcnczEeDtwYBIC4NjghgUFYTFt+nzfIg4M5vNRkl5827lbiofD7cG/+Nk2rRpbNiwgQ0bNvDKK68A8Ne//pUHH3yQtWvXMnfuXPbt28e6deuIjo5m1qxZbNu2jeLiYvr27cuCBQsYO3as43yxsbHMnDmTmTNnAva7uv7yl7/wySefsG7dOiIiInjppZe4++67W/y6L6cA1IZign15efIAo8uQZrLZmvCcRr9G41/EarNRXmmj0mqjvNJKhdVGRaW1Zp/VSkVl1T6rjcrKy/ZZrbWfW9++K85X/bjjfFXblar3l9l/atC1bLjsHwo9u3QiITqIwTGBJMQE0T20E2azWpWk/SspryT+2XWGvPbB+ePx9WxYBHjllVc4evQo/fv3Z/78+QAcOHAAgN/85je8+OKLdO/encDAQE6dOsXEiRN57rnn8Pb25u233+auu+7iyJEjREdHX/U15s2bx/PPP88LL7zAq6++yn333UdaWhrBwcHNv9irUABqQyGdvPjh4EijyxBpNVarzR6ULgtP1QGpVhi7RkA7U3iJXel57ErLIzX3IsdyLnAs5wIrd2YAEODtzqDoIBJighgcHcSAKAv+3molEmktFosFT09PfH196dq1KwCHDx8GYP78+dx+++2OY0NCQhgwoOYf+s899xwffvghq1at4tFHH73qa0ybNo2f/exnAPzP//wPr776Kt988w133HFHa1wSoAAkIi3IbDbhaTbh2cwbTO8fHgNA7oVSdqfnk1IViPaeyqfwUgUbjp51tBKZTNA7zJ/BVYEoISaI2BCtsSTOz8fDjYPzxxv22i1hyJAhtX4uLi5m3rx5/Oc//3EsV1FSUkJ6evo1z3PTTTc5vvfz88Pf35+cnJwWqfFqFIBExGmFdPJibHwYY+PDACivtHI4q4hd6XmkpOWxKz2PU3klHM4u4nB2Ef/Ybv+fbLCfJ4OjAxkUXdNK1NDmfpG2YjKZ2v1/l1fezTV79mzWrVvHiy++SM+ePfHx8eHHP/4xZWVl1zzPlWt6mUwmrFZri9d7ufb9mxcRl+LhZubGSAs3Rlp4ICkWgJzqLrP0fHal5bHvdAHni8v47FAOnx2y/wvSzWyib7i/o4VocHQQkUE+aiUSaSBPT88Grb21adMmpk2bxg9+8AMALly4QGpqaitX1zQKQCLSrnUJ8OaO/uHc0T8cgNKKSg5mFpKSlmfvPkvLI7vwEvtPF7L/dCHvbE0DILSTFwkxgQyODmJwTBA3RljwbqFuAZGOJjY2lu3bt5OamkqnTp2u2jrTs2dP/vWvf3HXXXdhMpl45plnWr0lp6kUgESkQ/Fyd2NQdBCDooMc+zLzS6oGVtvHEx3MLODchVLWHTjDugNnAPBwMxHfzeK442xwdBDdAn2MugwRp/Lkk0/ywAMPEB8fT0lJCX/961/rPe5Pf/oTP//5z0lKSiI0NJTf/va3FBYWtnG1DWOyNeV+2w6usLAQi8VCQUEBAQEBRpcjIi3sUnkl354uYFfVOKKUtHzOXSitc1y4xZvB0UEMirbfgt+vmwVPd60gJE1z6dIlTp48SVxcHN7e3kaX025d6/fYmM9vtQCJiMvx9nBjaGwwQ2Ptc4zYbDZO5ZU4BlbvSs/jUFYRWQWX+OTbLD75NgsAT3czN0ZYqsYR2VuJugTog0ykPVIAEhGXZzKZiAr2JSrYl0mDIgC4WFbB3owCx5xEu9LzyLtYTkqa/Q60apFBPrUGV/cJ98fDTa1EIs5OAUhEpB6+nu6M6BHCiB4hgL2VKDX3Yk0rUVoeR84UcSqvhFN5JazamwmAt4eZAZGBDI0N5oeDI+jeuZORlyEiV6EAJCLSACaTibhQP+JC/fhxgn1G96JL5ezNKKjVdVZ0qYLtJ8+z/eR5XvvyGEk9QrgvMYbb48M0fkjEiSgAiYg0kb+3B6NuCGXUDaGAfSmQ42cvsCs9j3UHzvDlkRy2HM9ly/FcQjt5kTw0kp8OjSYq2NfgykVEAUhEpIWYzSZuCPPnhjB/kodGcyrvIit3ZPDujgzOFpXy5y+Ps/ir43yvV2fuS4zh1j5dcNPCriKGUAASEWklkUG+/Hpcbx677QY+O3iG5dvT2XzsHF8eOcuXR87SzeLNT4dFkzw0ijDdTSbSphSARERamYebmQk3hjPhxnBOnitmxTfp/HNnBpkFl3h5/VFe+fw7bu8bxn3DoxnZIxSzWoVEWp0CkIhIG4oL9eO/JvZl1u29WLs/m+Xb09iRmsfaA9msPZBNTIgv9w6L5scJkYR08jK6XJEOSwFIRMQA3h5uTBoUwaRBERzJLuIf29P4167TpOVeZMGaw7z06VEm3NiV+xJjGBobpIVbRVqYlsKoh5bCEBEjXCyr4N97M1m+PZ19pwoc+2/o0on7EqP5weBILD4eBlYozaGlMFpGSy2FoUkpRESchK+nO8lDo1n16ChWPTqSnw6NwsfDje9yLvD7fx8k8X8+4zfv72VvRj76t6u0JavVysKFC+nZsydeXl5ER0fzhz/8gREjRvDUU0/VOvbs2bN4eHjw5ZdfGlRtw6gFqB5qARIRZ1F4qZyPdp9m+bZ0jpwpcuzvHxHAfYkx3D2gG35eGs3QHtRpubDZoPyiMcV4+EIjulV/+9vf8pe//IU//elPjBo1iqysLA4fPsylS5d44YUXSE1NdXTTvvbaa/zxj38kPT0ds7nl21laqgVIAageCkAi4mxsNhu70vNYvi2d/3ybRVmFFYBOXu78YFAE9yZG0zdc/79yZnU+uMuK4X+6GVPMf2WCp1+DDi0qKqJz58689tprPPTQQ7UeO3v2LN26deOLL75g9OjRACQlJTFq1Cief/75Fi8b1AUmIuJSTCYTCTHBvJw8kO1zbmPuxL7EhfpxobSCv21LY8Irm/jRki38a9cpLpVXGl2udCCHDh2itLSU2267rc5jnTt35vbbb2f58uUAnDx5kq1bt3Lfffe1dZmNZni76eLFi3nhhRfIysqiX79+LFq0yJEi6/PnP/+Z1157jdTUVKKjo5k7dy5Tp06tdcwHH3zAM888w/Hjx+nRowd/+MMf+MEPftDalyIi0iaC/Dx5+ObuTB8Vx9YTuSzfnsanB844Vqqf9++D/DghknsTo+mhxVidl4evvSXGqNduIB8fn2s+ft999/H444/z6quv8o9//IN+/foxYMCA5lbY6gxtAVq5ciUzZ85k7ty57N69m9GjRzNhwgTS09PrPX7JkiXMmTOH3//+9xw4cIB58+bxy1/+kn//+9+OY7Zu3UpycjJTpkxh7969TJkyhcmTJ7N9+/a2uiwRkTZhNpsY2TOUxfclsGXOGGaP701EoA8FJeW8tfkkt720gXv/so1P9tV0mYkTMZns3VBGbI0Y/3PDDTfg4+PD559/Xu/jkyZN4tKlS6xdu5Z//OMf3H///S31G2pVho4BSkxMZPDgwSxZssSxr2/fvkyaNIkFCxbUOT4pKYmRI0fywgsvOPbNnDmTnTt3snnzZgCSk5MpLCxkzZo1jmPuuOMOgoKCWLFiRYPq0hggEWmvKq02Nh49y/LtaXxxOAdr1f/htRir8drzbfDz5s3jlVdeYdGiRYwcOZKzZ89y4MABpk+fDthbgQ4cOMC+ffscPTStpaXGABnWBVZWVkZKSkqd2+fGjRvHli1b6n1OaWlpnYv18fHhm2++oby8HA8PD7Zu3coTTzxR65jx48ezaNGiq9ZSWlpKaWmp4+fCwsJGXo2IiHNwM5u4tU8Xbu3ThdP5Jaz8Jp13d2SQc9lirLdUL8bauzPubhoKKtf3zDPP4O7uzrPPPktmZibh4eHMmDHD8fh9993H97//fW6++eZWDT8tybAAdO7cOSorKwkLC6u1PywsjOzs7HqfM378eN58800mTZrE4MGDSUlJYenSpZSXl3Pu3DnCw8PJzs5u1DkBFixYwLx585p/USIiTiQi0IdZ43rzq9tu4PND9sVYN313jq+OnOWrI2cJt3jz06H2xVi7WtpXi4S0LbPZzNy5c5k7d269j0+cOLHdzU1lePS/cnp3m8121Snfn3nmGSZMmMDw4cPx8PDgnnvuYdq0aQC4ubk16ZwAc+bMoaCgwLFlZGQ08WpERJyPh5uZO/qH87fpiXz15Pf4xc3dCfL1IKvgEn/67CgjF37BL/62k41Hz2K1tq8PMZGmMiwAhYaG4ubmVqdlJicnp04LTjUfHx+WLl3KxYsXSU1NJT09ndjYWPz9/QkNDQWga9eujTongJeXFwEBAbU2EZGOKDbUjzkT+7J1zm288tOBDIsNptJqY92BM0xd+g3fe/Er/m/DcXIvlF7/ZCLtmGEByNPTk4SEBNavX19r//r160lKSrrmcz08PIiMjMTNzY13332XO++80zHb5IgRI+qc89NPP73uOUVEXIm3hxv3DIzgvRkj+PSJm5mWFIu/lzvp5y/yxzWHGbHgCx5bsZvtJ3LbXdeGSEMYOg/QrFmzmDJlCkOGDGHEiBG88cYbpKenOwZWzZkzh9OnT/POO+8AcPToUb755hsSExPJy8vj5ZdfZv/+/bz99tuOcz7++OPcfPPNLFy4kHvuuYePP/6Yzz77zHGXmIiI1NYrzJ/f392P39zRm//szWL59jT2nipg1d5MVu3NZPQNofzurnh6dvE3ulSRFmNoAEpOTiY3N5f58+eTlZVF//79Wb16NTExMQBkZWXVmhOosrKSl156iSNHjuDh4cGtt97Kli1biI2NdRyTlJTEu+++y9NPP80zzzxDjx49WLlyJYmJiW19eSIi7YqvpzuTh0YxeWgU354qYPn2NP616zSbvjvHHYs28UBSLI+PvYEAb61IL+2f1gKrh+YBEhGxS8st5rlPDrH+4BkAQvw8+c0dvflJQhRmc8Mn05Oa+WtiY2OvO7uyXF1JSQmpqalaC0xERFpPTIgff5k6hLd/Pozunf3ILS7jtx98y6TFX5OSlmd0ee2Kh4e95eziRYNWgO8gysrKgNp3fzeFWoDqoRYgEZG6yiqsvLM1lUWffceF0goAfjg4gqfu6EOXAM0j1BBZWVnk5+fTpUsXfH19rzlFi9RltVrJzMzEw8OD6OjoOr+/xnx+KwDVQwFIROTqzhaV8sK6w7y38xQAfp5u/Oq2G3hwZCxe7s37V3lHZ7PZyM7OJj8/3+hS2i2z2UxcXByenp51HlMAaiYFIBGR69uTkc/vVx1gT0Y+AHGhfjx7Zzy39ulibGHtQGVlJeXl5UaX0S55eno6pr65kgJQMykAiYg0jNVq41+7T/PHNYc5VzV54pg+XXjmznjiQv0Mrk5cjQJQMykAiYg0TtGlcl774hhLvz5JeaUNDzcT00d159ExPenkZeiMK+JCFICaSQFIRKRpjp+9wPx/H2TD0bMAdPH3Ys7EPkwaGKEBv9LqFICaSQFIRKTpbDYbXxzOYf5/DpKWa7/le3B0IL+/ux83RQYaW5x0aApAzaQAJCLSfKUVlby1+SSvfXGMi2WVmEyQPCSKJ8f3JrSTl9HlSQekANRMCkAiIi0nu+ASC9ce5sPdpwHw93Zn5theTB0Rg4eb5uOVlqMA1EwKQCIiLW9n6nl+/+8D7D9dCEDPLp34/V39GHVDqMGVSUehANRMCkAiIq2j0mrjvZ0ZvLDuCOeL7UsajO8XxtPfjycq2Nfg6qS9UwBqJgUgEZHWVXCxnEWfH+WdrWlUWm14upuZcXN3ZnyvB76eum1emkYBqJkUgERE2sbRM0X8ftUBthzPBSDc4s1/TezLnTeF67Z5aTQFoGZSABIRaTs2m411B7L57/8c4nR+CQDD4oL5/V39iO+m/wdLwykANZMCkIhI27tUXskbG0+w+KtjXCq3YjbBfYkxzLq9F0F+dRe+FLmSAlAzKQCJiBjndH4J/7P6EJ/sywIg0NeDX4/rzb3DonEzq1tMrk4BqJkUgEREjLf1eC7z/n2Aw9lFAPTp6s/v7+7H8O4hBlcmzkoBqJkUgEREnENFpZV/fJPOS58epaCkHIA7bwrnvyb2pVugj8HVibNRAGomBSAREeeSV1zGS+uP8I/t6Vht4O1h5v99ryeP3Nwdbw83o8sTJ6EA1EwKQCIizulAZgHzVh3km9TzAEQG+fD09+MZ3y9Mt82LAlBzKQCJiDgvm83Gv/dl8T+fHCK78BIAo3qG8ru74rkhzN/g6sRICkDNpAAkIuL8LpZVsOSr47y+8QRlFVbczCYeGBHL42NvwOLjYXR5YgAFoGZSABIRaT/Scy/y3CcH+fTgGQBC/DyZPb43PxkSpdvmXYwCUDMpAImItD8bj55l3r8PcPxsMQA3Rlj4/d3xJMQEG1yZtBUFoGZSABIRaZ/KK628szWNReuPUlRaAcAPB0Xw2wl9CAvwNrg6aW2N+fw2t1FNIiIirc7Dzcz0UXF8Oft7JA+JwmSCf+0+zZgXv+I/+zKNLk+ciAKQiIh0OKGdvFj445v46P+NZFB0IMVllTy2Yjf/2nXK6NLESSgAiYhIhzUgKpAPZiTxs2FRWG3w63/u5b0dGUaXJU5AAUhERDo0s9nEHybdyJThMdhs8JsP9rF8e5rRZYnBFIBERKTDM5tNzL+nHw+OjAVg7of7Wfb1SWOLEkMpAImIiEswmUw8e2c8v7i5OwC///dB3tx0wuCqxCgKQCIi4jJMJhNPTejDo7f2BOC5Tw6x+KtjBlclRlAAEhERl2IymXhyfG+eGNsLgOfXHuGVz74zuCppawpAIiLikh4fewOzx/cG4E+fHeWlT4+guYFdhwKQiIi4rF/e2pO5E/sC8OoXx/jj2sMKQS7C8AC0ePFi4uLi8Pb2JiEhgU2bNl3z+OXLlzNgwAB8fX0JDw/nwQcfJDc31/H4smXLMJlMdbZLly619qWIiEg79PDN3fn9XfEAvL7hBP/9n0MKQS7A0AC0cuVKZs6cydy5c9m9ezejR49mwoQJpKen13v85s2bmTp1KtOnT+fAgQP885//ZMeOHTz00EO1jgsICCArK6vW5u2tNWBERKR+00bG8dyk/gAs/fokz358AKtVIagjMzQAvfzyy0yfPp2HHnqIvn37smjRIqKioliyZEm9x2/bto3Y2Fgee+wx4uLiGDVqFL/4xS/YuXNnreNMJhNdu3attYmIiFzL/cNjeP5HN2Eywd+2pTH3o28VgjowwwJQWVkZKSkpjBs3rtb+cePGsWXLlnqfk5SUxKlTp1i9ejU2m40zZ87w/vvv8/3vf7/WcRcuXCAmJobIyEjuvPNOdu/efc1aSktLKSwsrLWJiIjrmTw0ipd+MgCzCVZ8k8FvPthHpUJQh2RYADp37hyVlZWEhYXV2h8WFkZ2dna9z0lKSmL58uUkJyfj6elJ165dCQwM5NVXX3Uc06dPH5YtW8aqVatYsWIF3t7ejBw5ku++u/otjgsWLMBisTi2qKiolrlIERFpd344OJI/JQ/EzWzi/ZRTzHpvDxWVVqPLkhZm+CBok8lU62ebzVZnX7WDBw/y2GOP8eyzz5KSksLatWs5efIkM2bMcBwzfPhw7r//fgYMGMDo0aN577336NWrV62QdKU5c+ZQUFDg2DIytFCeiIgru2dgBK/+bBDuZhMf78nk8ZV7KFcI6lDcjXrh0NBQ3Nzc6rT25OTk1GkVqrZgwQJGjhzJ7NmzAbjpppvw8/Nj9OjRPPfcc4SHh9d5jtlsZujQoddsAfLy8sLLy6sZVyMiIh3NxBvDcTeb+OU/dvHJviwqKq28+rPBeLob3nYgLcCwd9HT05OEhATWr19fa//69etJSkqq9zkXL17EbK5dspubG8BVb1m02Wzs2bOn3nAkIiJyLeP6deWNKUPwdDez7sAZ/r+/p1BaUWl0WdICDI2xs2bN4s0332Tp0qUcOnSIJ554gvT0dEeX1pw5c5g6darj+Lvuuot//etfLFmyhBMnTvD111/z2GOPMWzYMLp16wbAvHnzWLduHSdOnGDPnj1Mnz6dPXv21OomExERaahb+3ThzalD8HI38/nhHB55J4VL5QpB7Z1hXWAAycnJ5ObmMn/+fLKysujfvz+rV68mJiYGgKysrFpzAk2bNo2ioiJee+01fv3rXxMYGMiYMWNYuHCh45j8/HweeeQRsrOzsVgsDBo0iI0bNzJs2LA2vz4REekYbu7Vmb9OG8r0t3ey4ehZpr+9gzenDsXH083o0qSJTDZNd1lHYWEhFouFgoICAgICjC5HREScxDcnz/PgX7+huKySxLhglk4bip+XoW0JcpnGfH5rJJeIiEgDDYsL5p3pw/D3cmf7yfM8sPQbii6VG12WNIECkIiISCMkxATzt4cSCfB2Z2daHlPe+oaCEoWg9kYBSEREpJEGRgXyj4eHE+jrwZ6MfO5/czv5F8uMLksaQQFIRESkCfpHWFjx8HCC/Tz59nQBP/vLds4XKwS1FwpAIiIiTdQ3PIB3HxlOaCcvDmUV8rM3tnG2qNTosqQBFIBERESaoVeYP+8+Mpwu/l4cOVPET9/YSk7hJaPLkutQABIREWmmnl06sfIXIwi3eHP8bDHJb2wjq6DE6LLkGhSAREREWkBcqB/v/WIEEYE+nDxXTPLr2ziVd9HosuQqFIBERERaSFSwLyt/MZzoYF/Sz18k+fVtZJxXCHJGCkAiIiItKDLIHoLiQv04nV/C5Ne3knqu2Oiy5AoKQCIiIi0s3OLDykeG06OzH1kFl5j8+laO5Vwwuiy5jAKQiIhIK+gS4M27j4ygd5g/OUWl/PSNbRw9U2R0WVJFAUhERKSVdPb3YsUjw4kPD+DcBXsIOphZaHRZggKQiIhIqwr28+QfDydyY4SF88Vl3PvmNvafLjC6LJenACQiItLKAn09+ftDiQyMCiT/Yjn3/mUbezLyjS7LpSkAiYiItAGLjwd/mz6MITFBFF6q4P43t5OSdt7oslyWApCIiEgb8ff24O2fDyMxLpgLpRVMfesbtp/INbosl6QAJCIi0ob8vNxZ9uAwRvUMpbiskml/3cGWY+eMLsvlKACJiIi0MR9PN958YAi39OpMSXklDy7bwcajZ40uy6UoAImIiBjA28ONN6YmMLZvF0orrDz09k6+OHzG6LJchgKQiIiIQbzc3Vh8XwLj+4VRVmnlF39L4dMD2UaX5RIUgERERAzk6W7mtXsH8/2bwimvtPH/lu9i9bdZRpfV4SkAiYiIGMzDzcwryQOZNLAbFVYbv1qxm4/3nDa6rA5NAUhERMQJuLuZeWnyQH6cEEml1cYTK/fwQcopo8vqsBSAREREnISb2cTzP7qJnw2LwmqDJ9/fy8od6UaX1SEpAImIiDgRs9nEHybdyNQRMdhs8NsPvuXv29KMLqvDUQASERFxMmaziXl39+PnI+MAePqj/fz165MGV9WxKACJiIg4IZPJxDN39uUXt3QHYN6/D/KXjScMrqrjUAASERFxUiaTiafu6MOvxvQE4A+rD2lMUAtRABIREXFiJpOJX4/r7WgJ+sc3GQZX1DEoAImIiLQD9yfGAHDgdAElZZUGV9P+KQCJiIi0A5FBPnTx96LCamPfqXyjy2n3FIBERETaAZPJREJMEAA70/IMrqb9UwASERFpJ6oDUIoCULMpAImIiLQTQ2KDAdiVnofVajO4mvbN8AC0ePFi4uLi8Pb2JiEhgU2bNl3z+OXLlzNgwAB8fX0JDw/nwQcfJDc3t9YxH3zwAfHx8Xh5eREfH8+HH37YmpcgIiLSJuLDA/ByN5N/sZwT5y4YXU67ZmgAWrlyJTNnzmTu3Lns3r2b0aNHM2HCBNLT65/jYPPmzUydOpXp06dz4MAB/vnPf7Jjxw4eeughxzFbt24lOTmZKVOmsHfvXqZMmcLkyZPZvn17W12WiIhIq/B0NzMgKhCAnanqBmsOk81mM6wNLTExkcGDB7NkyRLHvr59+zJp0iQWLFhQ5/gXX3yRJUuWcPz4cce+V199leeff56MDPu8CMnJyRQWFrJmzRrHMXfccQdBQUGsWLGiQXUVFhZisVgoKCggICCgqZcnIiLS4p5fe5jFXx3nJwmRvPCTAUaX41Qa8/ltWAtQWVkZKSkpjBs3rtb+cePGsWXLlnqfk5SUxKlTp1i9ejU2m40zZ87w/vvv8/3vf99xzNatW+ucc/z48Vc9J0BpaSmFhYW1NhEREWekgdAtw7AAdO7cOSorKwkLC6u1PywsjOzs7Hqfk5SUxPLly0lOTsbT05OuXbsSGBjIq6++6jgmOzu7UecEWLBgARaLxbFFRUU148pERERaT3UAOnGumPPFZQZX034ZPgjaZDLV+tlms9XZV+3gwYM89thjPPvss6SkpLB27VpOnjzJjBkzmnxOgDlz5lBQUODYqrvTREREnE2gryc9u3QC1ArUHO5GvXBoaChubm51WmZycnLqtOBUW7BgASNHjmT27NkA3HTTTfj5+TF69Giee+45wsPD6dq1a6POCeDl5YWXl1czr0hERKRtJEQHcSznAjvTznN7/NU/3+TqDGsB8vT0JCEhgfXr19fav379epKSkup9zsWLFzGba5fs5uYG2Ft5AEaMGFHnnJ9++ulVzykiIh2MzQZlF6EwC3IOQdpWOP4FFOde/7ntREKsvRtsl1qAmsywFiCAWbNmMWXKFIYMGcKIESN44403SE9Pd3RpzZkzh9OnT/POO+8AcNddd/Hwww+zZMkSxo8fT1ZWFjNnzmTYsGF069YNgMcff5ybb76ZhQsXcs899/Dxxx/z2WefsXnzZsOuU0REGslqhbIiKMmHS/lVXwsa/n1lfWNjTNBtIPS4DXqMgahh4ObRNtfTwoZUjQPae6qA0opKvNzdDK6o/TE0ACUnJ5Obm8v8+fPJysqif//+rF69mpgY+4q3WVlZteYEmjZtGkVFRbz22mv8+te/JjAwkDFjxrBw4ULHMUlJSbz77rs8/fTTPPPMM/To0YOVK1eSmJjY5tcnIuLSKsvtgaQ6xDhCSv4V++v5vrQQbNbmvb7JDbwt4BMImOD8ccjcbd82vQie/hB3M/S4FXreBsHdm/d6bSgu1I9gP0/OF5ex/3ShY2C0NJyh8wA5K80DJCJSpbwESvIa3wJTkg/lxc1/fXdv8A6sCTLegVVfLdf/3rMTXH4DTFG2vSvs2Odw4ku4eEWXWFCcPQj1GGMPRl7+za+/FT309k4+O3SGuRP78vDN7Se8tabGfH4b2gIkIiJOKC8VDq6Cgx/D6Z3NP59XwDUCS2DtcHPl9x7ezX/9av5dYeC99s1qhey99jB0/EvI2AZ5J2HHm/bN7A5RifbWoR63QfhAMBt+43QtQ2KD+OzQGXamnedhFIAaSwFIREQg97g98Bz8GLL21H7s8q6kRrXGBNrDj5sTftSYzdBtkH27+UkoLYKTm+wtRMc/h/MnIO1r+/bFc+AbAt1vtbcO9RgDAeFGX0GtCRGvN92L1OWE/1WKiEibOHukJvSc2V+z32SG2FEQfw/0ngj+4bW7kjoiL3/oM9G+AZw/aQ9Cx7+EExvs3WX737dvAF36Qc+qMBSd1LItVQ10Y4QFTzcz5y6UkX7+IjEhfm1eQ3umACQi4ipsNsg5WBN6zh6ueczkBt1vsYeePneCX6hxdTqD4DgIfgiGPmQfzH1qR1V32Rf2QdQ5B+zbllfB3QdiR1a1Dt0GnXu3SWD09nCjf0QAu9Lz2ZmapwDUSApAIiIdmc0G2ftqQk/usZrHzB72D+34u+0tPb7BxtXpzNw8ICbJvt32jH0+oRNf2luHjn8ORVlw7DP7BhAQUTN2qPv3WvX3mhATZA9AaXn8KCGy1V6nI1IAEhHpaGw2OL0LDn4Eh1bZBzVXc/OCnmPtLT29xlfdIi6N4hcCN/7Yvtls9skWq8cOpW2BwtOw++/2zWSGboPtQbPnbRAxpEXHRCXEBPOXTSc1IWITKACJiHQEVqu9m+bgx/bQU3DZmobuPnDD7TWhx8lv725XTCYIi7dvSY/apw1I+9reOnTsczh7yH4n3emdsPF58LJA95trusuCYpr18tUDoY/mFFFQUo7Fp31O7GgEzQNUD80DJCLtgrUS0rfVhJ6irJrHPPyg9x320NNzLHhqfIghCk7bu8uq5x4quaKlJqRnTRiKHQVenRr9Ere88CVpuRf564NDubV3lxYqvH3SPEAiIh1VZQWkba4KPf+G4rM1j3kFQO8J9tDTYwx4+BhXp9hZImDQ/fbNWmmfYuBYVXdZxjf2MVm5x+CbN+xjsqKH13SXhd3YoLmHEmKCSMu9yK60PJcPQI2hACQi4uwqy+HkhqrQ8x8oOV/zmLfFftdW/D32AbfuXoaVKddhdoOIBPt2y2z7rNknN9nD0LHPIT8NUjfZt8/ngV9n+9xD1bNTd6o/3AyJCeZfu06zM1XjgBpDAUhExBlVlMKJr+yh5/An9uUlqvmGVIWeuyH2ZnD3NKpKaQ5vC/S9077ZbPbJF6uX6kjdZG/d+/Y9+wbQ9UZ7V1niLyCgm+M01eOA9mTkU15pxcPNuWasdlYKQCIizqK8xP7hd2gVHFljXxC0ml8X6HuXvaUnZqRzzq4sTWcyQUgP+zbsYagog1Pf1Mw9lLUHsr+1b3knYfI7jqfe0KUTAd7uFF6q4HBWETdGWoy7jnZEf0EiIkYqK4bv1ttbeo6uq72AqH849L3bHnqih9u7UMQ1uHvaB0XHjoKxv4Pic7DvPVg3xz4ztdXqGB9kNpsYHBPEV0fOsjPtvAJQAykAiYi0tdIie9g5+LE9/FSU1DxmibIHnr53Q+RQp1uAUwziFwrDHoEv/8feHXrmWwgf4Hg4Ibo6AOXx4Mg44+psRxSARETaQkk+HF1rDz3HPofK0prHgmLtoSf+HvukeR193S1pGjd3iBkB330KJzfWDkCx9nFAmhCx4RSARERay8XzcGS1PfQc/xKs5TWPBfeAfpPsoafrTQo90jBxN1cFoE2Q9CvH7oFRgbiZTWQVXOJ0fgkRgZoC4XoUgEREWoq10j5INe1r+7pQJzeCtaLm8c597Xduxd8DXeIVeqTxYkfbv6Ztsc8JVTUY3tfTnfjwAL49XcDO1PNEDIwwsMj2QQFIRKSpKsshc4898KR9bZ+V+fI7t8A+mV38Pfbg07m3IWVKB9L1Rvvt85cKIGsvRCY4HkqICeLb0wXsSsvjHgWg61IAEhFpqIpSOJ0CqVWBJ+Ob2ndtgX025ugREDvSPldPSA9japWOyewGMaPgyCeQurFWABoSG8SyLans1DigBlEAEhG5mrKL9gVG0762dzmc2gEVl2of4xNkn5cnJsn+teuNul1dWlfcaHsAOrkRRj3h2F09IeKhrEIulFbQyUsf8dei346ISLXSIkjfXtOldXpX7YHLYF+eIGakfX6WmCT7uB7dqi5tqXocUPo2+4SJVTOBh1t8iAj04XR+CXsz8hnZM9TAIp2fApCIuK6SPPuHSOpmewtP1l6wVdY+xr+bvTurOvSE9NTgZTFWl3j7cigXcyFzl32SzCoJMUGczi9hZ2qeAtB1KACJiOsozq1p3Un9Gs7sB2y1jwmMqQo7VaEnKFaBR5yL2WwP4wc/tt8Of0UAWrU3k51p569xAgEFIBHpyIqya8JO2tdw9nDdY0J6Vo3hqQo9lsi2r1OksWJH2wNQ6kb7yvJVHAujpudTabXhZlZ4v5omB6D8/Hzef/99jh8/zuzZswkODmbXrl2EhYUREaHb70TEAPkZVYGnqkvr/PG6x3TuW9O6E5ME/l3bvk6R5oq72f41fTuUXwIPbwD6dPXHz9ONotIKjp4pom94gIFFOrcmBaB9+/YxduxYLBYLqampPPzwwwQHB/Phhx+SlpbGO++8c/2TiIg0h80G50/Yg051K09B+hUHmex3ZVW37kQngV+IIeWKtKjQXtApDC6csd+dGGcfGO3uZmZgdCBfH8tlZ1qeAtA1NCkAzZo1i2nTpvH888/j7+/v2D9hwgTuvffeFitORMTBZoNzR2tad9K+hqKs2seY3KDbwKpb0kfZx0b4BBpRrUjrMpns3WD734fUTY4ABJAQE8zXx3LZlZbHlOExBhbp3JoUgHbs2MHrr79eZ39ERATZ2dnNLkpEBKsVcg7WDFpO2wLFZ2sfY/aAiISqLq0kiEoEL//6zyfS0cRVBaCTm+DWmt1DqsYBaSD0tTUpAHl7e1NYWFhn/5EjR+jcuXOzixIRF2Kz2W9HL8qybzmHawLPpfzax7p7Q+TQmi6tiCHg6WtI2SKGq54P6NQO+6SdVX8LA6MDMZkg43wJOYWX6BLgbWCRzqtJAeiee+5h/vz5vPfeewCYTCbS09N56qmn+NGPftSiBYpIO2Wz2dfFKsquCjfX+FpZVv85PPwgalhVC88oiBgM7l5tex0iziq4OwREQOFpyNgGPcYAEODtQe8wfw5nF5GSlseEG8MNLtQ5NSkAvfjii0ycOJEuXbpQUlLCLbfcQnZ2NiNGjOAPf/hDS9coIs6mrLhhwab8YsPP6RsC/uFgibKP3YkdBeEDwM2j9a5DpD2rHge07117N1hVAAL7umCHs4vYqQB0VU0KQAEBAWzevJkvvviCXbt2YbVaGTx4MGPHjm3p+kSkLVWUVgWYq4Waqu+vXPH8Wrwt9mDj3/XqXzuFqWVHpCnibrYHoNRNtXYnxATx923pWhj1GhodgCoqKvD29mbPnj2MGTOGMWPGXP9JImKsynK4kFM3yFwZcEoaMWjSw7cqwFSHmasEHI3REWk91Xd/nd5lX8uu6iaAITHBABw4XcCl8kq8PbRA75UaHYDc3d2JiYmhsrLy+geLSG02G1gr7etN2aw131sv/9l62b5K+91QDTm+9MLVu6KKz1JnyYercfO6dmtN9Vcvfy0RIWK0wGj78i35afZ17W64HYDIIB+6+HuRU1TK3ox8Ertr/qsrNakL7Omnn2bOnDn8/e9/Jzg4uKVr6rhyj8OGhUZX0TJsV36Y2q7y2BXHXe2xFjnftWq8xmvVGziu3FcdNK4MIVcLKNYrnlf1fUNDSGswu0OnrtdurfHvCj5BCjYi7UncaNidBic3OAKQyWQiISaINfuz2ZmWpwBUjyYFoP/93//l2LFjdOvWjZiYGPz8/Go9vmvXrhYprsO5eB72rTS6CnF2Jjcwu9X+ajLV3Wc2Vz1mtu/z8LGvXH61cOMbYn+OiHQssTfD7r/bB0JfpjoA7dI4oHo1KQBNmjSpxQpYvHgxL7zwAllZWfTr149FixYxevToeo+dNm0ab7/9dp398fHxHDhwAIBly5bx4IMP1jmmpKQEb2+D50KwRMK459roxdrgX/B1WglM13i8qY814jWb81h9wcJcFS4u/75WCDHXc/zlx115fEMeU0ARkUaqHgeUvQ9K8h2zn1cvjJqSnofVasOshVFraVIA+t3vftciL75y5UpmzpzJ4sWLGTlyJK+//joTJkzg4MGDREdH1zn+lVde4Y9//KPj54qKCgYMGMBPfvKTWscFBARw5MiRWvsMDz8AAeGQ9CujqxARkY4koBuE9ITcY/YJRPtMBKBfNwte7mbyL5Zz4twFenbRLOmXa9Y/N1NSUvj73//O8uXL2b17d6Of//LLLzN9+nQeeugh+vbty6JFi4iKimLJkiX1Hm+xWOjatatj27lzJ3l5eXVafEwmU63junbVas8iItKBVc8Kfdnt8J7uZgZEBQKQom6wOpoUgHJychgzZgxDhw7lscce49FHHyUhIYHbbruNs2fPXv8EQFlZGSkpKYwbN67W/nHjxrFly5YGneOtt95i7NixxMTUXuztwoULxMTEEBkZyZ133nndcFZaWkphYWGtTUREpN2o7gY7ubHW7upusJ2pCkBXalIA+tWvfkVhYSEHDhzg/Pnz5OXlsX//fgoLC3nssccadI5z585RWVlJWFhYrf1hYWENWlA1KyuLNWvW8NBDD9Xa36dPH5YtW8aqVatYsWIF3t7ejBw5ku++++6q51qwYAEWi8WxRUVFNegaREREnEJ1C9CZ/VCc69hdvTCqWoDqalIAWrt2LUuWLKFv376OffHx8fz5z39mzZo1jTqX6YrBqTabrc6++ixbtozAwMA6A7KHDx/O/fffz4ABAxg9ejTvvfcevXr14tVXX73quebMmUNBQYFjy8jIaNQ1iIiIGKpTF+hc9Zmcttmxe3C0PQCdOFfM+eKrrLnnopoUgKxWKx4eddfn8fDwwGq1NugcoaGhuLm51WntycnJqdMqdCWbzcbSpUuZMmUKnp6e1zzWbDYzdOjQa7YAeXl5ERAQUGsTERFpVxzdYDXjgIL8POnR2T5VjVqBamtSABozZgyPP/44mZmZjn2nT5/miSee4LbbbmvQOTw9PUlISGD9+vW19q9fv56kpKRrPnfDhg0cO3aM6dOnX/d1bDYbe/bsITxci8GJiEgHVs9AaKhZFmNnWiOWunEBTQpAr732GkVFRcTGxtKjRw969uxJXFwcRUVF1+xqutKsWbN48803Wbp0KYcOHeKJJ54gPT2dGTNmAPauqalTp9Z53ltvvUViYiL9+/ev89i8efNYt24dJ06cYM+ePUyfPp09e/Y4zikiItIhxY4CTHD2MBSdcexOiLV3g2lCxNqaNA9QVFQUu3btYv369Rw+fBibzUZ8fHyjV4NPTk4mNzeX+fPnk5WVRf/+/Vm9erXjrq6srCzS09NrPaegoIAPPviAV155pd5z5ufn88gjj5CdnY3FYmHQoEFs3LiRYcOGNeVSRURE2gffYAjrD2e+tbcC3fhjoOZOsL2nCiitqMTLXQujAphstmstpuSaCgsLsVgsFBQUaDyQiIi0H2v/C7b9GRKmwV32hgKbzUbCc59xvriMD/6/JEcg6oga8/ndpC6wxx57jP/93/+ts/+1115j5syZTTmliIiINFc9A6FNJpPjbjB1g9VoUgD64IMPGDlyZJ39SUlJvP/++80uSkRERJogJsm+ruD541Bw2rHbMSGiBkI7NCkA5ebmYrFY6uwPCAjg3LlzzS5KREREmsDbAuED7N9fdjfYkNiaCRE18sWuSQGoZ8+erF27ts7+NWvW0L1792YXJSIiIk0UW7cb7MYICx5uJs5dKCP9/EWDCnMuTboLbNasWTz66KOcPXuWMWPGAPD555/z4osvXvXuLBEREWkDcbfAlv+F1Jp1wbw93OgfYWF3ej47U/OICfEzsEDn0KQA9POf/5zS0lL+8Ic/8N///d8AxMXF8X//93/1ztsjIiIibSR6OJjdIT8d8tIgyD61zJCYIHan55OSnsePEiINLtJ4TeoCKykp4YEHHuDUqVOcOXOGffv28eijj153CQsRERFpZV6doNtg+/eXjQNKqJoROkUrwwNNDED33HMP77zzDmBf/2vs2LG8/PLLTJo0iSVLlrRogSIiItJI9dwOX30n2NGcIgpKyo2oyqk0KQDt2rWL0aPtv9z333+fsLAw0tLSeOedd+qdH0hERETakGMg9Eaouuurs78XMSG+2GywO12tQE0KQBcvXsTf3x+ATz/9lB/+8IeYzWaGDx9OWlpaixYoIiIijRSVCGYPKMqE8yccu6tbgbQyfDNug//oo4/IyMhg3bp1jBs3DoCcnBwtHSEiImI0T1+IqloD82TN3WCOCRE1DqhpAejZZ5/lySefJDY2lsTEREaMGAHYW4MGDRrUogWKiIhIE1R3g10+IWLVQOg9GflUVFqNqMppNCkA/fjHPyY9PZ2dO3fWmhDxtttu409/+lOLFSciIiJNdPlA6KpxQDd06YS/tzsl5ZUcyioysDjjNSkAAXTt2pVBgwZhNtecYtiwYfTp06dFChMREZFmiBwK7t5QnANnjwBgNtcsjOrq64I1OQCJiIiIE3P3qhkHVKsbTAOhQQFIRESk44q72f718oHQsQpAoAAkIiLSccVWBaDUzWC1D3oeGBWIm9lEVsElTueXGFicsRSAREREOqqIweDhByXnIecgAL6e7sSH26esceVWIAUgERGRjsrNw744KtQ7H1BKqusOhFYAEhER6cji6s4H5JgQUS1AIiIi0iFVD4RO/RqslQAMqRoIfSirkOLSCqMqM5QCkIiISEfWdQB4BUBpAWTvAyDc4kNEoA9Wm31WaFekACQiItKRublDTJL9+5M13WCDXXxdMAUgERGRjq56XbDLBkI7JkRMVwASERGRjqh6IHT6VqgsB2oGQu9Oy6PSajOqMsMoAImIiHR0YTeCdyCUXYDMPQD06eqPr6cbRaUVHD3jegujKgCJiIh0dGYzxI6yf59q7wZzdzMzKDoQcM0JERWAREREXIFjXbDL5wMKBhSAREREpKOqHgidvg0qSoHLJ0R0vRmhFYBERERcQZe+4BsKFSVwOgWAQdGBmEyQcb6EnMJLBhfYthSAREREXIHJVHM3WFU3WIC3B73D/AHX6wZTABIREXEVsVoXrJoCkIiIiKuoHgid8Q2UlwA164KpBUhEREQ6ppCe0KkrVJbaQxAwpOpOsAOZBVwqrzSyujalACQiIuIqLh8HVNUNFhnkQ2d/L8orbex1oYVRDQ9AixcvJi4uDm9vbxISEti0adNVj502bRomk6nO1q9fv1rHffDBB8THx+Pl5UV8fDwffvhha1+GiIhI+3DFfEAmk8kl1wUzNACtXLmSmTNnMnfuXHbv3s3o0aOZMGEC6enp9R7/yiuvkJWV5dgyMjIIDg7mJz/5ieOYrVu3kpyczJQpU9i7dy9Tpkxh8uTJbN++va0uS0RExHlVD4Q+nQJlxUDNQOgUF1oZ3mSz2QxbAS0xMZHBgwezZMkSx76+ffsyadIkFixYcN3nf/TRR/zwhz/k5MmTxMTEAJCcnExhYSFr1qxxHHfHHXcQFBTEihUrGlRXYWEhFouFgoICAgICGnlVIiIiTsxmg0U3QkEG3P8v6Hkbu9Pz+MHiLQT6erDr6dsxm01GV9kkjfn8NqwFqKysjJSUFMaNG1dr/7hx49iyZUuDzvHWW28xduxYR/gBewvQleccP358g88pIiLSoZlMNa1AJ+3rgvXrZsHL3Uz+xXJOnCs2sLi2Y1gAOnfuHJWVlYSFhdXaHxYWRnZ29nWfn5WVxZo1a3jooYdq7c/Ozm70OUtLSyksLKy1iYiIdFhXDIT2dDczICoQgBQXWRbD8EHQJlPtZjabzVZnX32WLVtGYGAgkyZNavY5FyxYgMVicWxRUVENK15ERKQ9qm4BytwDl+z/6HdMiOgi44AMC0ChoaG4ubnVaZnJycmp04JzJZvNxtKlS5kyZQqenp61HuvatWujzzlnzhwKCgocW0ZGRiOvRkREpB0JjIKgOLBVQvpWAJe7E8ywAOTp6UlCQgLr16+vtX/9+vUkJSVd87kbNmzg2LFjTJ8+vc5jI0aMqHPOTz/99Jrn9PLyIiAgoNYmIiLSocXVHgc0ONoegE6cLeZ8cZlRVbUZQ7vAZs2axZtvvsnSpUs5dOgQTzzxBOnp6cyYMQOwt8xMnTq1zvPeeustEhMT6d+/f53HHn/8cT799FMWLlzI4cOHWbhwIZ999hkzZ85s7csRERFpP2Kr5gOqGgcU5OdJj85+gGssi+Fu5IsnJyeTm5vL/PnzycrKon///qxevdpxV1dWVladOYEKCgr44IMPeOWVV+o9Z1JSEu+++y5PP/00zzzzDD169GDlypUkJia2+vWIiIi0G9UtQFn74OJ58A1mSEwwx88Wk5KWx+3x1x6O0t4ZOg+Qs9I8QCIi4hJeHQK530Hycuh7J+/tzOA37+9jaGwQ/5xx7eEozqhdzAMkIiIiBour3Q1WfSfY3lMFlFZ07IVRFYBERERclWMgtD0AdQ/1I9jPk7IKKwcyO/aceApAIiIirqp6PqCcA1B8DpPJ5LgbrKOvC6YAJCIi4qr8QqFLvP37K7rBdnbwGaEVgERERFxZbO1usCGxVS1Aafl05PukFIBERERc2RUDoW+MsODhZuLchVLSz180sLDWpQAkIiLiymJHAiY4dxSKsvH2cKN/hAXo2OuCKQCJiIi4Mp8g6Hqj/fvUzYBrrAumACQiIuLqqrvBTm4AICEmGOjYd4IpAImIiLi6KwZCV98JdjSniIKScqOqalUKQCIiIq4uJglMbpB3EgpO0dnfi5gQX2w22N1Bu8EUgERERFyddwB0G2j//opWoI66MrwCkIiIiNR0g105IWIHHQekACQiIiKXrQu2EWw2hlQNhN6TkU9FpdXAwlqHApCIiIhA1HAwu0NBBuSlckOXTvh7u1NSXsmhrCKjq2txCkAiIiICXp0gYoj9+9RNmM2XLYzaAdcFUwASERERu7gr1gVzLIza8cYBKQCJiIiI3eUDoW02EmI77p1gCkAiIiJiFzUM3DyhKAtyjzEwKhA3s4msgktk5pcYXV2LUgASEREROw8fiBxm//7kRnw93YkPDwA6XjeYApCIiIjUqF4X7Ir5gFJSO9ZAaAUgERERqVE9EDp1s30cUAddGV4BSERERGpEJIC7DxSfhbOHGVI1EPpQVhHFpRUGF9dyFIBERESkhrsXRCfavz+5kXCLDxGBPlRabezJyDe0tJakACQiIiK1xV62LAYwuAMujKoAJCIiIrXF3WL/mvY1WK0dckJEBSARERGprdtA8OwEJXlwZr9jIPTutDwqrTZja2shCkAiIiJSm5sHRI+wf5+6iT5d/fH1dKOotILvcjrGwqgKQCIiIlJXXM04IHc3M4OiAwHYmdoxusEUgERERKSu6oHQaVugsoKEmGCg4wyEVgASERGRusIHgJcFSgshe2/NhIgKQCIiItJhmd0gdqT9+5ObGBQdiMkE6ecvklN0ydjaWoACkIiIiNSvuhssdRMB3h70DvMHIKUDjANSABIREZH6VQ+ETtsKleUdqhtMAUhERETq16Uf+ARDeTGc3uVYF6wjTIioACQiIiL1M5trxgGlbmRI1Z1gBzILuFReaWBhzWd4AFq8eDFxcXF4e3uTkJDApk2brnl8aWkpc+fOJSYmBi8vL3r06MHSpUsdjy9btgyTyVRnu3Sp/Q/YEhERaXPVy2Kc3ERkkA+d/b0or7Sx71SBsXU1k7uRL75y5UpmzpzJ4sWLGTlyJK+//joTJkzg4MGDREdH1/ucyZMnc+bMGd566y169uxJTk4OFRUVtY4JCAjgyJEjtfZ5e3u32nWIiIh0WNUDoTO2Y6osY0hMEGv2Z7Mz7TzD4oKNra0ZDA1AL7/8MtOnT+ehhx4CYNGiRaxbt44lS5awYMGCOsevXbuWDRs2cOLECYKD7b/02NjYOseZTCa6du3aqrWLiIi4hM69wa8LFOfAqZ0kxISzZn92u78TzLAusLKyMlJSUhg3blyt/ePGjWPLli31PmfVqlUMGTKE559/noiICHr16sWTTz5JSUlJreMuXLhATEwMkZGR3HnnnezevfuatZSWllJYWFhrExEREcBkgthR9u9Pbqy5Eyw9D5ut/S6MalgAOnfuHJWVlYSFhdXaHxYWRnZ2dr3POXHiBJs3b2b//v18+OGHLFq0iPfff59f/vKXjmP69OnDsmXLWLVqFStWrMDb25uRI0fy3XffXbWWBQsWYLFYHFtUVFTLXKSIiEhHEFczH1C/bha83M3kXyzn+NliY+tqBsMHQZtMplo/22y2OvuqWa1WTCYTy5cvZ9iwYUycOJGXX36ZZcuWOVqBhg8fzv3338+AAQMYPXo07733Hr169eLVV1+9ag1z5syhoKDAsWVkZLTcBYqIiLR31QOhT+3A01bKgKhAAFLSzhtXUzMZFoBCQ0Nxc3Or09qTk5NTp1WoWnh4OBEREVgsFse+vn37YrPZOHXqVL3PMZvNDB069JotQF5eXgQEBNTaREREpEpwd/DvBpVlkLG9Q0yIaFgA8vT0JCEhgfXr19fav379epKSkup9zsiRI8nMzOTChQuOfUePHsVsNhMZGVnvc2w2G3v27CE8PLzlihcREXElJlNNN9jJTQyJaf8TIhraBTZr1izefPNNli5dyqFDh3jiiSdIT09nxowZgL1raurUqY7j7733XkJCQnjwwQc5ePAgGzduZPbs2fz85z/Hx8cHgHnz5rFu3TpOnDjBnj17mD59Onv27HGcU0RERJqg+nb4kxsZHG0PQCfOFnO+uMzAoprO0Nvgk5OTyc3NZf78+WRlZdG/f39Wr15NTEwMAFlZWaSnpzuO79SpE+vXr+dXv/oVQ4YMISQkhMmTJ/Pcc885jsnPz+eRRx4hOzsbi8XCoEGD2LhxI8OGDWvz6xMREekwqluAMncR5F5Gj85+HD9bzK60PMbG1z90xZmZbO35HrZWUlhYiMVioaCgQOOBREREqi26EfLT4b4P+O3eLqzcmcGMW3rw1IQ+RlcGNO7z2/C7wERERKSdiL3Z/jV1Iwmx1QOh2+edYApAIiIi0jCXDYSuvhNs76kCyiqsBhbVNApAIiIi0jDVA6Gz9tC9UwXBfp6UVVjZn9n+FkZVABIREZGGsUTY5wSyWTGlb3XcDdYe1wVTABIREZGGi6saB3RZN1h7nBBRAUhEREQarrobLHUjQ2JrJkRsbzeVKwCJiIhIw1UHoOz93BhUiYebiXMXSkk/f9HYuhpJAUhEREQazj8MQnsDNrxPb6V/hH19zvbWDaYAJCIiIo1TfTt8avtdF0wBSERERBqn1kDoYKD93QmmACQiIiKNEzPK/vXsIYZ0rgDgaE4RBSXlBhbVOApAIiIi0jh+IRDWH4DQs98QE+KLzQa709tPK5ACkIiIiDRe9d1gJzeSEN3+5gNSABIREZHGu2wgdM3CqApAIiIi0pHFjASTGXKPkRhaBsCejHwqKtvHwqgKQCIiItJ4PoHQ9SYAul/Yhb+3OxfLKjmUVWRsXQ2kACQiIiJNU9UNZk7dVLMwatp5IytqMAUgERERaZrY6vmANra7CREVgERERKRpYkaAyQ3y00gKsa8F1l4GQisAiYiISNN4+UPEYAD6V+zFzWwiq+ASmfklBhd2fQpAIiIi0nRV8wF5ZWwhPjwAaB/dYApAIiIi0nTV8wGd3ERCdCAAKanOPxBaAUhERESaLmo4mD2g8BSjQ+23wKe0gyUxFIBERESk6Tx9IXIIAAm2/QAcyiqiuLTCyKquSwFIREREmqdqHFBg9ja6WbyptNrYk5FvbE3XoQAkIiIizRNXNR9Q6iYSYtrHumAKQCIiItI8kUPBzQsunGFM5wLA+e8EUwASERGR5vHwhqhhACRyAIDdaXlYrTYjq7omBSARERFpvqpusK7nv8HX042i0gqO5jjvwqgKQCIiItJ8VQOhzWmbGRxVNSFiqvN2gykAiYiISPNFJICHL1zMZVyoPfjscuJxQApAIiIi0nzunhA9HIAk90OAcw+EVgASERGRllHVDRZTmILJBOnnL5JTdMngouqnACQiIiIto2ogtEfG1/Tt4gs4bzeYApCIiIi0jPCB4OkPlwr4fpdcwHkHQisAiYiISMtwc4eYJABu9jwMOO84IMMD0OLFi4mLi8Pb25uEhAQ2bdp0zeNLS0uZO3cuMTExeHl50aNHD5YuXVrrmA8++ID4+Hi8vLyIj4/nww8/bM1LEBERkWpx9nFAPYt3A3Ags4BL5ZVGVlQvQwPQypUrmTlzJnPnzmX37t2MHj2aCRMmkJ6eftXnTJ48mc8//5y33nqLI0eOsGLFCvr06eN4fOvWrSQnJzNlyhT27t3LlClTmDx5Mtu3b2+LSxIREXFtVQOhvTO307WTO+WVNvadKjC4qLpMNpvNsHmqExMTGTx4MEuWLHHs69u3L5MmTWLBggV1jl+7di0//elPOXHiBMHBwfWeMzk5mcLCQtasWePYd8cddxAUFMSKFSsaVFdhYSEWi4WCggICAgIaeVUiIiIuzFoJz8fBpQIWRv6ZJceC+M0dvfl/3+vZ6i/dmM9vw1qAysrKSElJYdy4cbX2jxs3ji1bttT7nFWrVjFkyBCef/55IiIi6NWrF08++SQlJSWOY7Zu3VrnnOPHj7/qOcHerVZYWFhrExERkSYwu0HMKABu9bKPA3LGO8EMC0Dnzp2jsrKSsLCwWvvDwsLIzs6u9zknTpxg8+bN7N+/nw8//JBFixbx/vvv88tf/tJxTHZ2dqPOCbBgwQIsFotji4qKasaViYiIuLiq2+H7XNoDQEpaHgZ2ONXL8EHQJpOp1s82m63OvmpWqxWTycTy5csZNmwYEydO5OWXX2bZsmW1WoEac06AOXPmUFBQ4NgyMjKacUUiIiIurmogtH9OCp3creRdLOf42WKDi6rNsAAUGhqKm5tbnZaZnJycOi041cLDw4mIiMBisTj29e3bF5vNxqlTpwDo2rVro84J4OXlRUBAQK1NREREmqhzX/ANwVR+kR90OQM4XzeYYQHI09OThIQE1q9fX2v/+vXrSUpKqvc5I0eOJDMzkwsXLjj2HT16FLPZTGRkJAAjRoyoc85PP/30qucUERGRFmY2Q6x9HNA43yMA7Ew7b2RFdRjaBTZr1izefPNNli5dyqFDh3jiiSdIT09nxowZgL1raurUqY7j7733XkJCQnjwwQc5ePAgGzduZPbs2fz85z/Hx8cHgMcff5xPP/2UhQsXcvjwYRYuXMhnn33GzJkzjbhEERER11R1O3y/sn2A802I6G7kiycnJ5Obm8v8+fPJysqif//+rF69mpiYGACysrJqzQnUqVMn1q9fz69+9SuGDBlCSEgIkydP5rnnnnMck5SUxLvvvsvTTz/NM888Q48ePVi5ciWJiYltfn0iIiIuK+4WAIJyd+NFGSfOwvniMoL9PA0uzM7QeYCcleYBEhERaSabDV7qDRfOMMvnOf6V1503pw5hbPzVx+Q2V7uYB0hEREQ6MJPJ0Q02sdMxwLm6wRSAREREpHVU3Q4/oMI+DsiZ7gRTABIREZHWUdUCFFrwLd6UsvdUPmUVVoOLslMAEhERkdYR3B0CIjFZy/mez3FKK6zsz3SOhVEVgERERKR1mEyObrC7Ao4DztMNpgAkIiIiraeqGyzBth+AnakKQCIiItLRVbUAhRUdxI8SdjrJwqgKQCIiItJ6AqMhMAaTrZIR7kc4d6GUjPMl139eK1MAEhERkdYVdzMAd1aNA3KGdcEUgERERKR1VQWgRA4AzjEhogKQiIiItK6qgdBdLx4lgAtOcSeYApCIiIi0roBwCOmJCSuJ5sMcOVNEQUm5oSUpAImIiEjrq2oFGud7FJsNdqcb2wqkACQiIiKtr+p2+JFuhwDjJ0RUABIREZHWV9UC1K30OEEUGj4QWgFIREREWl+nLtC5LwDjOx2jW6CPoeW4G/rqIiIi4jriRsPZQywYVIDp+wMMLUUtQCIiItI2qrrBTCc3GlyIApCIiIi0ldhRgAnOHYGiM4aWogAkIiIibcM3GLr2t3+fusnQUhSAREREpO3E2pfFUAASERER11E1HxAnjQ1AugtMRERE2k5MEtw+3zEg2igKQCIiItJ2vC0w8nGjq1AXmIiIiLgeBSARERFxOQpAIiIi4nIUgERERMTlKACJiIiIy1EAEhEREZejACQiIiIuRwFIREREXI4CkIiIiLgcBSARERFxOQpAIiIi4nIUgERERMTlKACJiIiIy9Fq8PWw2WwAFBYWGlyJiIiINFT153b15/i1KADVo6ioCICoqCiDKxEREZHGKioqwmKxXPMYk60hMcnFWK1WMjMz8ff3x2QyGV2OUyosLCQqKoqMjAwCAgKMLsfl6f1wLno/nI/eE+fSWu+HzWajqKiIbt26YTZfe5SPWoDqYTabiYyMNLqMdiEgIED/M3Eiej+ci94P56P3xLm0xvtxvZafahoELSIiIi5HAUhERERcjgKQNImXlxe/+93v8PLyMroUQe+Hs9H74Xz0njgXZ3g/NAhaREREXI5agERERMTlKACJiIiIy1EAEhEREZejACQiIiIuRwFIREREXI4CkLSJixcvEhMTw5NPPml0KS4vIyOD733ve8THx3PTTTfxz3/+0+iSXM5//vMfevfuzQ033MCbb75pdDkuTX8PzqktPjN0G7y0iblz5/Ldd98RHR3Niy++aHQ5Li0rK4szZ84wcOBAcnJyGDx4MEeOHMHPz8/o0lxCRUUF8fHxfPnllwQEBDB48GC2b99OcHCw0aW5JP09OKe2+MxQC5C0uu+++47Dhw8zceJEo0sRIDw8nIEDBwLQpUsXgoODOX/+vLFFuZBvvvmGfv36ERERgb+/PxMnTmTdunVGl+Wy9PfgfNrqM0MByMVt3LiRu+66i27dumEymfjoo4/qHLN48WLi4uLw9vYmISGBTZs2Neo1nnzySRYsWNBCFXd8bfGeVNu5cydWq5WoqKhmVu06mvv+ZGZmEhER4fg5MjKS06dPt0XpHVJL/r3o76H5WuL9aKvPDAUgF1dcXMyAAQN47bXX6n185cqVzJw5k7lz57J7925Gjx7NhAkTSE9PdxyTkJBA//7962yZmZl8/PHH9OrVi169erXVJbV7rf2eVMvNzWXq1Km88cYbrX5NHUlz35/6Rh2YTKZWrbkja4m/F9DfQ0tp7vvRpp8ZNpEqgO3DDz+stW/YsGG2GTNm1NrXp08f21NPPdWgcz711FO2yMhIW0xMjC0kJMQWEBBgmzdvXkuV3OG1xntis9lsly5dso0ePdr2zjvvtESZLqsp78/XX39tmzRpkuOxxx57zLZ8+fJWr9UVNPXvRX8PraMp70dbfmaoBUiuqqysjJSUFMaNG1dr/7hx49iyZUuDzrFgwQIyMjJITU3lxRdf5OGHH+bZZ59tjXJdQku8JzabjWnTpjFmzBimTJnSGmW6rIa8P8OGDWP//v2cPn2aoqIiVq9ezfjx440ot8NryPuhv4e205D3oy0/MxSA5KrOnTtHZWUlYWFhtfaHhYWRnZ1tUFWurSXek6+//pqVK1fy0UcfMXDgQAYOHMi3337bGuW6nIa8P+7u7rz00kvceuutDBo0iNmzZxMSEmJEuR1eQ94P/T20HWf7THFv81eUdufK8Qk2m61JYxamTZvWQhVJc96TUaNGYbVaW6MsqXK99+fuu+/m7rvvbuuyXNa13g/9PbS9hv7/q7U/M9QCJFcVGhqKm5tbnWSek5NTJ8FL29B74tz0/jgXvR/OxdneDwUguSpPT08SEhJYv359rf3r168nKSnJoKpcm94T56b3x7no/XAuzvZ+qAvMxV24cIFjx445fj558iR79uwhODiY6OhoZs2axZQpUxgyZAgjRozgjTfeID09nRkzZhhYdcem98S56f1xLno/nEu7ej9a5d4yaTe+/PJLG1Bne+CBBxzH/PnPf7bFxMTYPD09bYMHD7Zt2LDBuIJdgN4T56b3x7no/XAu7en90FpgIiIi4nI0BkhERERcjgKQiIiIuBwFIBEREXE5CkAiIiLichSARERExOUoAImIiIjLUQASERERl6MAJCIiIi5HAUhEOozU1FRMJhN79uxp8HOWLVtGYGBgq9UkIs5JAUhERERcjgKQiIiIuBwFIBFpV9auXcuoUaMIDAwkJCSEO++8k+PHj9d77FdffYXJZOKTTz5hwIABeHt7k5iYyLffflvn2HXr1tG3b186derEHXfcQVZWluOxHTt2cPvttxMaGorFYuGWW25h165drXaNItL6FIBEpF0pLi5m1qxZ7Nixg88//xyz2cwPfvADrFbrVZ8ze/ZsXnzxRXbs2EGXLl24++67KS8vdzx+8eJFXnzxRf72t7+xceNG0tPTefLJJx2PFxUV8cADD7Bp0ya2bdvGDTfcwMSJEykqKmrVaxWR1uNudAEiIo3xox/9qNbPb731Fl26dOHgwYN06tSp3uf87ne/4/bbbwfg7bffJjIykg8//JDJkycDUF5ezv/93//Ro0cPAB599FHmz5/veP6YMWNqne/1118nKCiIDRs2cOedd7bYtYlI21ELkIi0K8ePH+fee++le/fuBAQEEBcXB0B6evpVnzNixAjH98HBwfTu3ZtDhw459vn6+jrCD0B4eDg5OTmOn3NycpgxYwa9evXCYrFgsVi4cOHCNV9TRJybWoBEpF256667iIqK4i9/+QvdunXDarXSv39/ysrKGnUek8nk+N7Dw6POYzabzfHztGnTOHv2LIsWLSImJgYvLy9GjBjR6NcUEeehACQi7UZubi6HDh3i9ddfZ/To0QBs3rz5us/btm0b0dHRAOTl5XH06FH69OnT4NfdtGkTixcvZuLEiQBkZGRw7ty5JlyBiDgLBSARaTeCgoIICQnhjTfeIDw8nPT0dJ566qnrPm/+/PmEhIQQFhbG3LlzCQ0NZdKkSQ1+3Z49e/K3v/2NIUOGUFhYyOzZs/Hx8WnGlYiI0TQGSETaDbPZzLvvvktKSgr9+/fniSee4IUXXrju8/74xz/y+OOPk5CQQFZWFqtWrcLT07PBr7t06VLy8vIYNGgQU6ZM4bHHHqNLly7NuRQRMZjJdnlHt4hIB/LVV19x6623kpeXp+UuRKQWtQCJiIiIy1EAEhEREZejLjARERFxOWoBEhEREZejACQiIiIuRwFIREREXI4CkIiIiLgcBSARERFxOQpAIiIi4nIUgERERMTlKACJiIiIy1EAEhEREZfz/wPLHZTFduFd6AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogx(param_grid[\"ridge__alpha\"], train_scores.tolist(), label=\"train\")\n",
    "plt.semilogx(param_grid[\"ridge__alpha\"], cv_scores.tolist(), label=\"cv\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"alpha\")\n",
    "plt.ylabel(\"score\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:23.655629Z",
     "iopub.status.busy": "2024-05-29T14:52:23.654651Z",
     "iopub.status.idle": "2024-05-29T14:52:23.672475Z",
     "shell.execute_reply": "2024-05-29T14:52:23.669723Z",
     "shell.execute_reply.started": "2024-05-29T14:52:23.655564Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ridge__alpha': 100.0}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_alpha = search.best_params_\n",
    "best_alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:23.677330Z",
     "iopub.status.busy": "2024-05-29T14:52:23.675890Z",
     "iopub.status.idle": "2024-05-29T14:52:23.696648Z",
     "shell.execute_reply": "2024-05-29T14:52:23.693632Z",
     "shell.execute_reply.started": "2024-05-29T14:52:23.677261Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7951559890417762"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- It seems alpha=100 is the best choice here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- General intuition: **larger `alpha` leads to smaller coefficients**.\n",
    "- **Smaller coefficients** mean the predictions are **less sensitive to changes in the data**. Hence **less chance of overfitting**.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:23.701856Z",
     "iopub.status.busy": "2024-05-29T14:52:23.699771Z",
     "iopub.status.idle": "2024-05-29T14:52:24.054306Z",
     "shell.execute_reply": "2024-05-29T14:52:24.049067Z",
     "shell.execute_reply.started": "2024-05-29T14:52:23.701779Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>OverallQual</th>\n",
       "      <td>9691.378159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GrLivArea</th>\n",
       "      <td>7816.490033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1stFlrSF</th>\n",
       "      <td>5931.401623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TotRmsAbvGrd</th>\n",
       "      <td>5209.377032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageCars</th>\n",
       "      <td>5057.215412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SaleType_WD</th>\n",
       "      <td>-1214.532034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageFinish_Unf</th>\n",
       "      <td>-1281.596864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Foundation_CBlock</th>\n",
       "      <td>-1766.048889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofStyle_Gable</th>\n",
       "      <td>-1985.253914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <td>-2623.509001</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>262 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   Coefficients\n",
       "OverallQual         9691.378159\n",
       "GrLivArea           7816.490033\n",
       "1stFlrSF            5931.401623\n",
       "TotRmsAbvGrd        5209.377032\n",
       "GarageCars          5057.215412\n",
       "...                         ...\n",
       "SaleType_WD        -1214.532034\n",
       "GarageFinish_Unf   -1281.596864\n",
       "Foundation_CBlock  -1766.048889\n",
       "RoofStyle_Gable    -1985.253914\n",
       "KitchenAbvGr       -2623.509001\n",
       "\n",
       "[262 rows x 1 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe_bigalpha = make_pipeline(preprocessor, Ridge(alpha=1000))\n",
    "pipe_bigalpha.fit(X_train, y_train)\n",
    "bigalpha_coeffs = pipe_bigalpha.named_steps[\"ridge\"].coef_\n",
    "pd.DataFrame(\n",
    "    data=bigalpha_coeffs, index=new_columns, columns=[\"Coefficients\"]\n",
    ").sort_values(by=\"Coefficients\", ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- Smaller `alpha` leads to bigger coefficients. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:24.064153Z",
     "iopub.status.busy": "2024-05-29T14:52:24.061709Z",
     "iopub.status.idle": "2024-05-29T14:52:24.315684Z",
     "shell.execute_reply": "2024-05-29T14:52:24.312397Z",
     "shell.execute_reply.started": "2024-05-29T14:52:24.064050Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>RoofMatl_WdShngl</th>\n",
       "      <td>128384.117412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_Membran</th>\n",
       "      <td>127326.497787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_Metal</th>\n",
       "      <td>106023.481514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Condition2_PosA</th>\n",
       "      <td>83437.787574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_CompShg</th>\n",
       "      <td>70568.838951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Exterior1st_ImStucc</th>\n",
       "      <td>-34441.626206</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Heating_OthW</th>\n",
       "      <td>-34806.290424</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Condition2_RRAe</th>\n",
       "      <td>-63362.335309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Condition2_PosN</th>\n",
       "      <td>-194985.470060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_ClyTile</th>\n",
       "      <td>-582469.424221</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>262 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Coefficients\n",
       "RoofMatl_WdShngl     128384.117412\n",
       "RoofMatl_Membran     127326.497787\n",
       "RoofMatl_Metal       106023.481514\n",
       "Condition2_PosA       83437.787574\n",
       "RoofMatl_CompShg      70568.838951\n",
       "...                            ...\n",
       "Exterior1st_ImStucc  -34441.626206\n",
       "Heating_OthW         -34806.290424\n",
       "Condition2_RRAe      -63362.335309\n",
       "Condition2_PosN     -194985.470060\n",
       "RoofMatl_ClyTile    -582469.424221\n",
       "\n",
       "[262 rows x 1 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe_smallalpha = make_pipeline(preprocessor, Ridge(alpha=0.01))\n",
    "pipe_smallalpha.fit(X_train, y_train)\n",
    "smallalpha_coeffs = pipe_smallalpha.named_steps[\"ridge\"].coef_\n",
    "pd.DataFrame(\n",
    "    data=smallalpha_coeffs, index=new_columns, columns=[\"Coefficients\"]\n",
    ").sort_values(by=\"Coefficients\", ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the best alpha found by the grid search, the coefficients are somewhere in between. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:24.354061Z",
     "iopub.status.busy": "2024-05-29T14:52:24.337498Z",
     "iopub.status.idle": "2024-05-29T14:52:24.640546Z",
     "shell.execute_reply": "2024-05-29T14:52:24.629062Z",
     "shell.execute_reply.started": "2024-05-29T14:52:24.353974Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>OverallQual</th>\n",
       "      <td>14485.526602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GrLivArea</th>\n",
       "      <td>11693.792577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_NridgHt</th>\n",
       "      <td>9661.926657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_NoRidge</th>\n",
       "      <td>9492.921850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtQual</th>\n",
       "      <td>8076.988901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_ClyTile</th>\n",
       "      <td>-3991.413796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LandContour_Bnk</th>\n",
       "      <td>-5002.470666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_Gilbert</th>\n",
       "      <td>-5212.984664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_CollgCr</th>\n",
       "      <td>-5461.450739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_Edwards</th>\n",
       "      <td>-5789.889086</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>262 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Coefficients\n",
       "OverallQual           14485.526602\n",
       "GrLivArea             11693.792577\n",
       "Neighborhood_NridgHt   9661.926657\n",
       "Neighborhood_NoRidge   9492.921850\n",
       "BsmtQual               8076.988901\n",
       "...                            ...\n",
       "RoofMatl_ClyTile      -3991.413796\n",
       "LandContour_Bnk       -5002.470666\n",
       "Neighborhood_Gilbert  -5212.984664\n",
       "Neighborhood_CollgCr  -5461.450739\n",
       "Neighborhood_Edwards  -5789.889086\n",
       "\n",
       "[262 rows x 1 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe_bestalpha = make_pipeline(\n",
    "    preprocessor, Ridge(alpha=search.best_params_[\"ridge__alpha\"])\n",
    ")\n",
    "pipe_bestalpha.fit(X_train, y_train)\n",
    "bestalpha_coeffs = pipe_bestalpha.named_steps[\"ridge\"].coef_\n",
    "pd.DataFrame(\n",
    "    data=bestalpha_coeffs, index=new_columns, columns=[\"Coefficients\"]\n",
    ").sort_values(by=\"Coefficients\", ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To summarize: \n",
    "- Higher values of `alpha` means a more restricted model.  \n",
    "- The values of coefficients are likely to be smaller for higher values of `alpha` compared to lower values of alpha. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `RidgeCV`\n",
    "\n",
    "Because it's so common to want to tune `alpha` with `Ridge`, sklearn provides a class called `RidgeCV`, which automatically tunes `alpha` based on cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:24.669156Z",
     "iopub.status.busy": "2024-05-29T14:52:24.656970Z",
     "iopub.status.idle": "2024-05-29T14:52:43.464404Z",
     "shell.execute_reply": "2024-05-29T14:52:43.461532Z",
     "shell.execute_reply.started": "2024-05-29T14:52:24.669031Z"
    }
   },
   "outputs": [],
   "source": [
    "alphas = 10.0 ** np.arange(-6, 6, 1)\n",
    "ridgecv_pipe = make_pipeline(preprocessor, RidgeCV(alphas=alphas, cv=10))\n",
    "ridgecv_pipe.fit(X_train, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:43.473967Z",
     "iopub.status.busy": "2024-05-29T14:52:43.473217Z",
     "iopub.status.idle": "2024-05-29T14:52:43.494564Z",
     "shell.execute_reply": "2024-05-29T14:52:43.490912Z",
     "shell.execute_reply.started": "2024-05-29T14:52:43.473897Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100.0"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_alpha = ridgecv_pipe.named_steps[\"ridgecv\"].alpha_\n",
    "\n",
    "best_alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's examine the tuned model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:43.498074Z",
     "iopub.status.busy": "2024-05-29T14:52:43.497452Z",
     "iopub.status.idle": "2024-05-29T14:52:44.006724Z",
     "shell.execute_reply": "2024-05-29T14:52:44.005255Z",
     "shell.execute_reply.started": "2024-05-29T14:52:43.498033Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([228603.06797961, 104643.99931883, 155624.61029914, 246332.07245741,\n",
       "       127614.33726089, 243243.60429913, 304784.87844893, 145425.67512181,\n",
       "       157008.6526853 , 128528.91515735])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_tuned = make_pipeline(preprocessor, Ridge(alpha=best_alpha))\n",
    "ridge_tuned.fit(X_train, y_train)\n",
    "ridge_preds = ridge_tuned.predict(X_test)\n",
    "ridge_preds[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's get the feature names of the transformed data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.009280Z",
     "iopub.status.busy": "2024-05-29T14:52:44.008708Z",
     "iopub.status.idle": "2024-05-29T14:52:44.069624Z",
     "shell.execute_reply": "2024-05-29T14:52:44.065999Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.009207Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame(\n",
    "    data={\"coefficients\": ridge_tuned.named_steps[\"ridge\"].coef_}, index=new_columns\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.079941Z",
     "iopub.status.busy": "2024-05-29T14:52:44.079269Z",
     "iopub.status.idle": "2024-05-29T14:52:44.137798Z",
     "shell.execute_reply": "2024-05-29T14:52:44.134193Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.079875Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coefficients</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>OverallQual</th>\n",
       "      <td>14485.526602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GrLivArea</th>\n",
       "      <td>11693.792577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_NridgHt</th>\n",
       "      <td>9661.926657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_NoRidge</th>\n",
       "      <td>9492.921850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtQual</th>\n",
       "      <td>8076.988901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RoofMatl_ClyTile</th>\n",
       "      <td>-3991.413796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LandContour_Bnk</th>\n",
       "      <td>-5002.470666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_Gilbert</th>\n",
       "      <td>-5212.984664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_CollgCr</th>\n",
       "      <td>-5461.450739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Neighborhood_Edwards</th>\n",
       "      <td>-5789.889086</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>262 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      coefficients\n",
       "OverallQual           14485.526602\n",
       "GrLivArea             11693.792577\n",
       "Neighborhood_NridgHt   9661.926657\n",
       "Neighborhood_NoRidge   9492.921850\n",
       "BsmtQual               8076.988901\n",
       "...                            ...\n",
       "RoofMatl_ClyTile      -3991.413796\n",
       "LandContour_Bnk       -5002.470666\n",
       "Neighborhood_Gilbert  -5212.984664\n",
       "Neighborhood_CollgCr  -5461.450739\n",
       "Neighborhood_Edwards  -5789.889086\n",
       "\n",
       "[262 rows x 1 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(\"coefficients\", ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So according to this model:\n",
    "\n",
    "- As `OverallQual` feature gets bigger the housing price will get bigger.\n",
    "- `Neighborhood_Edwards` is associated with reducing the housing price. \n",
    "    - We'll talk more about interpretation of different kinds of features next week."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.148057Z",
     "iopub.status.busy": "2024-05-29T14:52:44.146267Z",
     "iopub.status.idle": "2024-05-29T14:52:44.171774Z",
     "shell.execute_reply": "2024-05-29T14:52:44.166923Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.147982Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(390725.7472092076, 30815.281220255354)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_preds.max(), ridge_preds.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ❓❓ Questions for you"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### iClicker Exercise 10.1\n",
    "\n",
    "**Select all of the following statements which are TRUE.**\n",
    "\n",
    "- (A) Price per square foot would be a good feature to add in our `X`. \n",
    "- (B) The `alpha` hyperparameter of `Ridge` has similar interpretation of `C` hyperparameter of `LogisticRegression`; higher `alpha` means more complex model. \n",
    "- (C) In `Ridge`, smaller alpha means bigger coefficients whereas bigger alpha means smaller coefficients.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can we use the metrics we looked at in the previous lecture for this problem? Why or why not? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br><br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Regression scoring functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "- We aren't doing classification anymore, so we can't just check for equality:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.175388Z",
     "iopub.status.busy": "2024-05-29T14:52:44.174123Z",
     "iopub.status.idle": "2024-05-29T14:52:44.312613Z",
     "shell.execute_reply": "2024-05-29T14:52:44.308199Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.175273Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302     False\n",
       "767     False\n",
       "429     False\n",
       "1139    False\n",
       "558     False\n",
       "        ...  \n",
       "1041    False\n",
       "1122    False\n",
       "1346    False\n",
       "1406    False\n",
       "1389    False\n",
       "Name: SalePrice, Length: 1314, dtype: bool"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_tuned.predict(X_train) == y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.324952Z",
     "iopub.status.busy": "2024-05-29T14:52:44.322320Z",
     "iopub.status.idle": "2024-05-29T14:52:44.405703Z",
     "shell.execute_reply": "2024-05-29T14:52:44.404183Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.324786Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([205000, 160000, 175000, ..., 262500, 133000, 131000])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.409792Z",
     "iopub.status.busy": "2024-05-29T14:52:44.408958Z",
     "iopub.status.idle": "2024-05-29T14:52:44.558765Z",
     "shell.execute_reply": "2024-05-29T14:52:44.543021Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.409726Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([212870.41150573, 178494.22221895, 189981.34426571, ...,\n",
       "       245329.56804591, 129904.71327467, 135422.67672595])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_tuned.predict(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We need a **score** that reflects **how right/wrong** each prediction is."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "There are a number of popular scoring functions for regression. We are going to look at some common metrics: \n",
    "\n",
    "- mean squared error (MSE)\n",
    "- $R^2$\n",
    "- root mean squared error (RMSE)\n",
    "- MAPE\n",
    "\n",
    "See [sklearn documentation](https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics) for more details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Mean squared error (MSE)\n",
    "\n",
    "- A common metric is mean squared error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.560516Z",
     "iopub.status.busy": "2024-05-29T14:52:44.560073Z",
     "iopub.status.idle": "2024-05-29T14:52:44.721731Z",
     "shell.execute_reply": "2024-05-29T14:52:44.720116Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.560472Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "preds = ridge_tuned.predict(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.724439Z",
     "iopub.status.busy": "2024-05-29T14:52:44.723569Z",
     "iopub.status.idle": "2024-05-29T14:52:44.760671Z",
     "shell.execute_reply": "2024-05-29T14:52:44.749967Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.724379Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "872961060.7986538"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train - preds) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Perfect predictions would have MSE=0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.765379Z",
     "iopub.status.busy": "2024-05-29T14:52:44.762567Z",
     "iopub.status.idle": "2024-05-29T14:52:44.862340Z",
     "shell.execute_reply": "2024-05-29T14:52:44.859641Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.765240Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train - y_train) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "This is also implemented in sklearn:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.867515Z",
     "iopub.status.busy": "2024-05-29T14:52:44.864425Z",
     "iopub.status.idle": "2024-05-29T14:52:44.891960Z",
     "shell.execute_reply": "2024-05-29T14:52:44.888990Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.867433Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "872961060.7986538"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "mean_squared_error(y_train, preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- MSE looks huge and unreasonable. There is an error of ~\\$1 Billion!\n",
    "- Is this score good or bad?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Unlike classification, with regression **our target has units**. \n",
    "- The target is in dollars, the mean squared error is in $dollars^2$ \n",
    "- The score also depends on the scale of the targets. \n",
    "- If we were working in cents instead of dollars, our MSE would be $10,000 \\times (100^2$) higher!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.896794Z",
     "iopub.status.busy": "2024-05-29T14:52:44.895873Z",
     "iopub.status.idle": "2024-05-29T14:52:44.916809Z",
     "shell.execute_reply": "2024-05-29T14:52:44.915031Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.896716Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8729610607986.537"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean((y_train * 100 - preds * 100) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Root mean squared error or RMSE\n",
    "\n",
    "- The MSE above is in $dollars^2$.\n",
    "- A more relatable metric would be the root mean squared error, or RMSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:44.922898Z",
     "iopub.status.busy": "2024-05-29T14:52:44.919280Z",
     "iopub.status.idle": "2024-05-29T14:52:45.045841Z",
     "shell.execute_reply": "2024-05-29T14:52:45.040292Z",
     "shell.execute_reply.started": "2024-05-29T14:52:44.922749Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29545.914451894256"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_train, ridge_tuned.predict(X_train)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Error of \\$30,000 makes more sense.\n",
    "- Let's dig deeper. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.047931Z",
     "iopub.status.busy": "2024-05-29T14:52:45.047449Z",
     "iopub.status.idle": "2024-05-29T14:52:45.797263Z",
     "shell.execute_reply": "2024-05-29T14:52:45.793716Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.047887Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAGwCAYAAABrUCsdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAC9SklEQVR4nOzdeXzU1bn48c/3+519MjPZEwLIJqIIuIAFbBVccLmgdrm2FYtiW23dKK1erd20tnUvXbR1a63Wjdt7f9rqVRG1VksFpYAIogjKmpB9lmTW73J+fwwZCZsJBpLA83695qWZ78nMmYEwT57znOdoSimFEEIIIYT41PTenoAQQgghxMFCAishhBBCiB4igZUQQgghRA+RwEoIIYQQoodIYCWEEEII0UMksBJCCCGE6CESWAkhhBBC9BBXb0/gUOM4DnV1dYRCITRN6+3pCCGEEKILlFK0tbVRU1ODru85LyWB1QFWV1fH4MGDe3saQgghhNgHW7ZsYdCgQXu8LoHVARYKhYD8H0w4HO7l2QghhBCiKxKJBIMHDy58ju+JBFYHWMfyXzgclsBKCCGE6Gc+qYxHiteFEEIIIXqIBFZCCCGEED1EAishhBBCiB4igZUQQgghRA+RwEoIIYQQoodIYCWEEEII0UMksBJCCCGE6CESWAkhhBBC9BAJrIQQQggheogEVkIIIYQQPUQCKyGEEEKIHiKBlRBCCCFED5HASgghhBCih0hgJYQQQoh+79lnn+Wyyy5DKdWr85DASgghhBD9ViaT4aqrruLcc8/lwQcf5C9/+UuvzkcCKyGEEEL0W7qus3jxYgCuueYaPv/5z/fqfFy9+uxCCCGEEN2klMJxHAzDwOPx8OSTT7JhwwbOPPPM3p6aZKyEEEII0X80NTVxzjnncNNNNxXuO+KII/pEUAUSWAkhhBCin1i4cCHjxo3jueeeY968eTQ0NPT2lHYhgZUQQggh+rRsNss111zDmWeeSX19PUcffTRLliyhqqqqt6e2C6mxEkIIIUSf9f7773PBBRfw9ttvA3DllVdy55134vf7e3dieyCBlRBCCCH6pFQqxcknn0xTUxNlZWU89NBDnHvuub09rb2SpUAhhBBC9EmBQIBbbrmF008/nXfeeafPB1UAmurtFqWHmEQiQSQSIR6PEw6He3s6QgghRJ/y2muv4Xa7OfHEE4F8awWlFLreu7mgrn5+S8ZKCCGEEL3ONE1++MMfcsopp/DVr36VaDQKgKZpvR5UdYfUWAkhhBCiV3344YfMnDmTt956C4AzzjgDt9vdy7PaN/0nBBRCCCHEQefRRx/l2GOP5a233qK4uJi//OUv/OEPf6CoqKi3p7ZPJGMlhBBCiAMum83y9a9/nSeeeAKAk046iccee4zDDjusl2f26UjGSgghhBAHnMfjIZvNYhgGP/vZz3j11Vf7fVAFsivwgJNdgUIIIQ5Vtm2TTqcLy3ytra188MEHTJo0qZdn9slkV6AQQggh+ozNmzdzyimn8I1vfIOOnE5paWm/CKq6Q2qshBBCCLFf/eUvf+Fb3/oWsViMUCjEhg0bGD58eG9Pa7+QjJUQQggh9ov29na+8Y1v8JWvfIVYLMbEiRNZsWLFQRtUgQRWQgghhNgPli1bxvHHH89DDz2Epmn88Ic/5J///CcjRozo7antV7IUKIQQQogeZVkW559/Phs2bGDQoEE89thjTJkypbendUD0asZq6NChaJq2y+3KK68E8ucD3XTTTdTU1OD3+5k6dSrvvvtup8fIZrNcffXVlJeXEwwGOffcc9m6dWunMdFolFmzZhGJRIhEIsyaNYtYLNZpzObNmznnnHMIBoOUl5czZ84ccrlcpzGrVq1iypQp+P1+Bg4cyM0334xsqhRCCCE6c7lcPPTQQ5x//vmsXLnykAmqoJcDq6VLl7Jt27bC7aWXXgLg/PPPB+COO+5g3rx53HPPPSxdupTq6mqmTZtGW1tb4THmzp3L008/zfz581m0aBHt7e3MmDED27YLY2bOnMnbb7/NggULWLBgAW+//TazZs0qXLdtm+nTp5NMJlm0aBHz58/n//2//8c111xTGJNIJJg2bRo1NTUsXbqUu+++m7vuuot58+bt77dJCCGE6POeeeYZ5s+fX/h66tSp/OUvf6G0tLQXZ9ULVB/yne98R40YMUI5jqMcx1HV1dXqtttuK1zPZDIqEomo++67TymlVCwWU263W82fP78wpra2Vum6rhYsWKCUUmrNmjUKUEuWLCmMWbx4sQLU+++/r5RS6vnnn1e6rqva2trCmCeffFJ5vV4Vj8eVUkr9/ve/V5FIRGUymcKYW2+9VdXU1CjHcbr8GuPxuAIKjyuEEEL0Z6lUSl1xxRUKUMFgUK1fv763p7RfdPXzu88Ur+dyOR577DG+/vWvo2kaGzZsoL6+njPOOKMwxuv1MmXKFN544w0gXxhnmmanMTU1NYwZM6YwZvHixUQiESZOnFgYM2nSJCKRSKcxY8aMoaampjDmzDPPJJvNsmzZssKYKVOm4PV6O42pq6tj48aNe3xd2WyWRCLR6SaEEEIcDN555x0mTJjA73//ewC+/e1vM2jQoF6eVe/qM4HVX//6V2KxGLNnzwagvr4egKqqqk7jqqqqCtfq6+vxeDyUlJTsdUxlZeUuz1dZWdlpzM7PU1JSgsfj2euYjq87xuzOrbfeWqjtikQiDB48eM9vghBCCNEPKKW4++67+cxnPsOaNWuoqqrixRdf5K677uqUgDgU9ZnA6o9//CNnn312p6wRgKZpnb5WSu1y3852HrO78T0xRm0vXN/bfG644Qbi8XjhtmXLlr3OXQghhOjLbNvmvPPOY86cOWSzWaZPn84777zTafXoUNYnAqtNmzbx8ssv881vfrNwX3V1NbBrNqixsbGQKaquriaXyxGNRvc6pqGhYZfnbGpq6jRm5+eJRqOYprnXMY2NjcCuWbUdeb1ewuFwp5sQQgjRXxmGwdFHH43X6+Xuu+/m2Wef3e3K0KGqTwRWf/rTn6isrGT69OmF+4YNG0Z1dXVhpyDk67Bee+01TjzxRADGjx+P2+3uNGbbtm2sXr26MGby5MnE43Heeuutwpg333yTeDzeaczq1avZtm1bYczChQvxer2MHz++MOb111/v1IJh4cKF1NTUMHTo0B58N4QQQoi+JZvNdvqMvPnmm1mxYgVXXXXVJ64iHXL2exn9J7BtWx122GHq+uuv3+XabbfdpiKRiHrqqafUqlWr1AUXXKAGDBigEolEYcy3v/1tNWjQIPXyyy+r5cuXq1NPPVUdc8wxyrKswpizzjpLjRs3Ti1evFgtXrxYjR07Vs2YMaNw3bIsNWbMGHXaaaep5cuXq5dfflkNGjRIXXXVVYUxsVhMVVVVqQsuuECtWrVKPfXUUyocDqu77rqrW69XdgUKIYToT9577z117LHHqhNOOEHlcrnenk6v6ernd68HVi+++KIC1Nq1a3e55jiOuvHGG1V1dbXyer3q5JNPVqtWreo0Jp1Oq6uuukqVlpYqv9+vZsyYoTZv3txpTEtLi7rwwgtVKBRSoVBIXXjhhSoajXYas2nTJjV9+nTl9/tVaWmpuuqqqzq1VlBKqXfeeUeddNJJyuv1qurqanXTTTd1q9WCUhJYCSGE6B8cx1EPPPCA8vv9ClBlZWVq9erVvT2tXtPVz29NKWkdfiAlEgkikQjxeFzqrYQQQvRJra2tXHrppTz11FMAnH766TzyyCO7bDA7lHT187tP1FgJIYQQom/4xz/+wbhx43jqqadwu93ceeedvPjii4d0UNUdcgizEEIIIYB8G6Hvf//71NbWcsQRR/Dkk09y/PHH9/a0+hXJWAkhhBACyPdlfOyxx7jiiitYtmyZBFX7QGqsDjCpsRJCCNFXKKV47LHH2LRpEz/60Y96ezp9Wlc/v2UpUAghhDgExeNxrrjiCp544gk0TeOMM87gM5/5TG9Pq9+TwEoIIYQ4xLzxxhtceOGFbNy4EcMw+OlPf1poiC0+HQmshBBCiEOEZVnccsst3Hzzzdi2zbBhw3jiiSeYNGlSb0/toCGBlRBCCHEIUEpx7rnn8sILLwBw4YUX8vvf/17qfXuY7AoUQgghDgGapnH++ecTCoV49NFHeeyxxySo2g9kV+ABJrsChRBCHCjt7e1s3LiRMWPGAPmsVUNDA9XV1b08s/5HOq8LIYQQh7COPlRnn302ra2tQD5rJUHV/iWBlRBCCHEQcRyHO++8k8mTJ7Nu3ToANm/e3MuzOnRI8boQQghxkKirq+Oiiy7ilVdeAeBLX/oSDzzwAKWlpb08s0OHZKyEEEKIg8AzzzzDuHHjeOWVVwgEAjz44IP8z//8jwRVB5hkrIQQQoiDwOOPP05LSwvHHXccTzzxBEceeWRvT+mQJIGVEEII0U8ppdA0DYD77ruPo48+muuvvx6v19vLMzt0yVKgEEII0c8opbj77ru56KKL6OiaVFJSwk9+8hMJqnqZZKyEEEKIfqSxsZFLLrmE559/HoCZM2dy9tln9/KsRAfJWAkhhBD9xIsvvsi4ceN4/vnn8Xq93HPPPZx11lm9PS2xA8lYCSGEEH1cNpvlBz/4AfPmzQPg6KOP5sknn2Ts2LG9PDOxMwmshBBCiD7uy1/+Ms888wwAV111FXfccQd+v7+XZyV2R5YChRBCiD7u2muvpaqqimeffZa7775bgqo+TDJWQgghRB/T2trK8uXLOf300wE46aST+OijjwgEAr08M/FJJGMlhBBC9CGvvvoq48aN47zzzmPt2rWF+yWo6h8ksBJCCCH6ANM0+cEPfsBpp51GbW0tgwYNIpvN9va0RDfJUqAQQgjRy9avX8+FF17IW2+9BcA3v/lNfv3rXxMMBnt5ZqK7JGMlhBBC9KJHH32U4447jrfeeovi4mL+53/+hwcffFCCqn5KMlZCCCFEL1q7di3t7e2cfPLJPPbYYwwePLi3pyQ+BQmshBBCiAPMsixcrvxH8I033sjQoUO55JJLMAyjl2cmPi1ZChRCCCEOEMuyuPnmm/nc5z5HLpcDwO12881vflOCqoOEZKyEEEKIA2DTpk187WtfY9GiRQD87//+LzNnzuzlWYmeJhkrIYQQYj/7y1/+wjHHHMOiRYsIhUI8+uijElQdpCRjJYQQQuwn7e3tzJkzhz/96U8ATJw4kSeeeILhw4f38szE/iIZKyGEEGI/ueyyy/jTn/6Epmn86Ec/4p///KcEVQe5Xg+samtr+drXvkZZWRmBQIBjjz2WZcuWFa4rpbjpppuoqanB7/czdepU3n333U6Pkc1mufrqqykvLycYDHLuueeydevWTmOi0SizZs0iEokQiUSYNWsWsVis05jNmzdzzjnnEAwGKS8vZ86cOYXiwg6rVq1iypQp+P1+Bg4cyM0334xSqmffFCGEEAeFn/3sZ4wePZpXX32Vn/3sZ7jd7t6ektjPejWwikajfPazn8XtdvPCCy+wZs0afvnLX1JcXFwYc8cddzBv3jzuueceli5dSnV1NdOmTaOtra0wZu7cuTz99NPMnz+fRYsW0d7ezowZM7BtuzBm5syZvP322yxYsIAFCxbw9ttvM2vWrMJ127aZPn06yWSSRYsWMX/+fP7f//t/XHPNNYUxiUSCadOmUVNTw9KlS7n77ru56667mDdv3v59o4QQQvQLtbW1PPTQQ4WvR4wYUfiFXBwiVC+6/vrr1ec+97k9XnccR1VXV6vbbrutcF8mk1GRSETdd999SimlYrGYcrvdav78+YUxtbW1Std1tWDBAqWUUmvWrFGAWrJkSWHM4sWLFaDef/99pZRSzz//vNJ1XdXW1hbGPPnkk8rr9ap4PK6UUur3v/+9ikQiKpPJFMbceuutqqamRjmO06XXHI/HFVB4TCGEEAeHv/71r6qsrExpmqZefvnl3p6O6GFd/fzu1YzVM888w4QJEzj//POprKzkuOOO48EHHyxc37BhA/X19ZxxxhmF+7xeL1OmTOGNN94AYNmyZZim2WlMTU0NY8aMKYxZvHgxkUiEiRMnFsZMmjSJSCTSacyYMWOoqakpjDnzzDPJZrOFpcnFixczZcoUvF5vpzF1dXVs3Lhxt68xm82SSCQ63YQQQhw8UqkUl19+OZ///OdpaWnh2GOPZdCgQb09LdFLejWw+uijj7j33nsZOXIkL774It/+9reZM2cOf/7znwGor68HoKqqqtP3VVVVFa7V19fj8XgoKSnZ65jKyspdnr+ysrLTmJ2fp6SkBI/Hs9cxHV93jNnZrbfeWqjrikQiclSBEEIcRFauXMmECRO47777ALj22mtZvHgxo0aN6uWZid7Sq4GV4zgcf/zx3HLLLRx33HF861vf4tJLL+Xee+/tNE7TtE5fK6V2uW9nO4/Z3fieGKO2F67vaT433HAD8Xi8cNuyZcte5y2EEKJ/uP/++/nMZz7De++9R3V1NQsXLuTOO+/stKohDj29GlgNGDCA0aNHd7rvqKOOYvPmzQBUV1cDu2aDGhsbC5mi6upqcrkc0Wh0r2MaGhp2ef6mpqZOY3Z+nmg0immaex3T2NgI7JpV6+D1egmHw51uQggh+j+/308ul2PGjBm88847TJs2rbenJPqAXg2sPvvZz7J27dpO933wwQcMGTIEgGHDhlFdXc1LL71UuJ7L5Xjttdc48cQTARg/fjxut7vTmG3btrF69erCmMmTJxOPx3nrrbcKY958803i8XinMatXr2bbtm2FMQsXLsTr9TJ+/PjCmNdff71TC4aFCxdSU1PD0KFDe+ItEUII0YfF4/HC/8+aNYsXXniBZ555hoqKil6clehT9n8d/Z699dZbyuVyqV/84hdq3bp16vHHH1eBQEA99thjhTG33XabikQi6qmnnlKrVq1SF1xwgRowYIBKJBKFMd/+9rfVoEGD1Msvv6yWL1+uTj31VHXMMccoy7IKY8466yw1btw4tXjxYrV48WI1duxYNWPGjMJ1y7LUmDFj1GmnnaaWL1+uXn75ZTVo0CB11VVXFcbEYjFVVVWlLrjgArVq1Sr11FNPqXA4rO66664uv2bZFSiEEP1PJpNR3/3ud9XAgQNVU1NTb09H9IKufn73amCllFLPPvusGjNmjPJ6verII49UDzzwQKfrjuOoG2+8UVVXVyuv16tOPvlktWrVqk5j0um0uuqqq1Rpaany+/1qxowZavPmzZ3GtLS0qAsvvFCFQiEVCoXUhRdeqKLRaKcxmzZtUtOnT1d+v1+Vlpaqq666qlNrBaWUeuedd9RJJ52kvF6vqq6uVjfddFOXWy0oJYGVEEL0N2vWrFHHHHOMAhSg/vjHP/b2lEQv6Ornt6aUtA0/kBKJBJFIhHg8LvVWQgjRhymlePDBB5k7dy7pdJry8nL+9Kc/MWPGjN6emugFXf38lkOYhRBCiJ20tLRw6aWX8vTTTwMwbdo0HnnkEQYMGNDLMxN9Xa+fFSiEEEL0NTfeeCNPP/00brebu+66iwULFkhQJbpEMlZCCCHETn7+85+zbt06br31Vo4//vjeno7oRyRjJYQQ4pC3fv16fvSjHxWaPhcXF/Piiy9KUCW6TTJWQgghDllKKf785z9z1VVX0d7eztChQ/nmN7/Z29MS/ZgEVkIIIQ5JsViMyy+/nPnz5wMwZcoUzjzzzF6elejvZClQHLKUUkSTOerjGaLJHNJ5RIhDxxtvvMGxxx7L/PnzMQyDX/ziF7zyyisMHjy4t6cm+jnJWIlDUmMiw+raBLWxFDnbwWPoDCwOMGZgmMqwr7enJ4TYj373u98xZ84cHMdh+PDhPPHEE0ycOLG3pyUOEpKxEoecxkSGf6xt4sOmNsI+N4OKA4R9bj5sauMfa5toTGR6e4pCiP3o2GOPBfJn/a1YsUKCKtGjJGMlDilKKVbXJoincwwtC6JpGgBBr4uhniAbW5Ksrk1wSshbuCaE6P82btzI0KFDAfjsZz/LypUrGTNmTO9OShyUJGMlDimxlEltLEVlyLdL4KRpGpUhH7WxFLGU2UszFEL0pLa2Ni655BJGjx7Ne++9V7hfgiqxv0hgJQ4pWcshZzv43MZur/vcBjnbIWs5B3hmQoietnTpUo4//ngefvhhstks//znP3t7SuIQIIGVOKR4XToeQydj2ru9njFtPIaO1yU/GkL0V47jcPvtt3PiiSeyfv16Bg8ezKuvvspll13W21MThwD59BCHlOKAm4HFARrbMru0V1BK0diWYWBxgOKAu5dmKIT4NGpra5k2bRrf//73sSyL//zP/2TlypWcfPLJvT01cYiQwEocUjRNY8zAMBG/h40tSZJZC9tRJLMWG1uSRAIexgwMS+G6EP3Uww8/zN///ncCgQB//OMf+ctf/kJJSUlvT0scQmRXoDjkVIZ9TB1VUehj1ZzM4jF0RlSEpI+VEP3c9ddfz5YtW/jud7/LqFGjens64hCkKWk3fUAlEgkikQjxeJxwONzb0zmkKaWIpUyyloPXpVMccEumSoh+ZuXKldx+++386U9/wuv19vZ0xEGsq5/fkrEShyxN0ygJenp7GkKIfaCU4re//S3XXXcduVyOww8/nJtvvrm3pyWEBFZCCCH6l8bGRmbPns0LL7wAwIwZM7j66qt7eVZC5EnxuhBCiH5jwYIFjBs3jhdeeAGfz8c999zDM888Q0VFRW9PTQhAMlZCCCH6id///vdceeWVQL5z+pNPPikd1EWfIxkrIYQQ/cLZZ59NOBzm6quv5q233pKgSvRJkrESQgjRJymlePPNN5k0aRIAw4YNY+3atVRXV/fyzITYM8lYCSGE6HNaWlr40pe+xOTJk3nppZcK90tQJfo6yVgJIYToU/7+979z0UUXUVtbi9vtZuPGjb09JSG6TDJWQggh+gTTNLnhhhs4/fTTqa2tZdSoUbz55ptceumlvT01IbpMMlZCCCF63fr165k5cyZLly4F4NJLL+VXv/oVwWCwl2cmRPdIYCWEEKLXvfXWWyxdupSSkhIefPBBvvSlL/X2lITYJxJYCSGE6BVKqcL5nDNnzmTr1q1ccMEFDB48uJdnJsS+kxorIYQQB9y//vUvTjzxRJqamgr3XXfddRJUiX5PAishhBAHjGVZ/PSnP+Xkk09myZIl/PjHP+7tKQnRo2QpUAghxAGxadMmLrzwQv71r38BcNFFF3HHHXf08qyE6FmSsRJCCLHfzZ8/n2OOOYZ//etfhMNhHn/8cR555BHC4XBvT02IHiUZKyGEEPvVH/7wh0IvqkmTJvHEE08wbNiwXp6VONgopYilTLKWg9elUxxwFzZHHEi9mrG66aab0DSt023H4wqUUtx0003U1NTg9/uZOnUq7777bqfHyGazXH311ZSXlxMMBjn33HPZunVrpzHRaJRZs2YRiUSIRCLMmjWLWCzWaczmzZs555xzCAaDlJeXM2fOHHK5XKcxq1atYsqUKfj9fgYOHMjNN9+MUqpn3xQhhDjInH/++Rx++OH8+Mc/5p///KcEVaLHNSYyvPp+E//3Th3Prarj/96p49X3m2hMZA74XHp9KfDoo49m27ZthduqVasK1+644w7mzZvHPffcw9KlS6murmbatGm0tbUVxsydO5enn36a+fPns2jRItrb25kxYwa2bRfGzJw5k7fffpsFCxawYMEC3n77bWbNmlW4bts206dPJ5lMsmjRIubPn8//+3//j2uuuaYwJpFIMG3aNGpqali6dCl33303d911F/PmzdvP75Doq5RSRJM56uMZosmcBNlCbOc4Dv/zP/9T+JmIRCK888473HzzzbhcslAielZjIsM/1jbxYVMbYZ+bQcUBwj43Hza18Y+1vRBcqV504403qmOOOWa31xzHUdXV1eq2224r3JfJZFQkElH33XefUkqpWCym3G63mj9/fmFMbW2t0nVdLViwQCml1Jo1axSglixZUhizePFiBaj3339fKaXU888/r3RdV7W1tYUxTz75pPJ6vSoejyullPr973+vIpGIymQyhTG33nqrqqmpUY7jdPk1x+NxBRQeV/RPDfG0emVNg/rzGxvUH/75ofrzGxvUK2saVEM83dtTE6JXbd26VZ166qkKUL///e97ezriIOc4jnplTYN64LX1auHqbeqld+sLt4Wrt6kHXluvXlnT0K3P6T3p6ud3r2es1q1bR01NDcOGDeOrX/0qH330EQAbNmygvr6eM844ozDW6/UyZcoU3njjDQCWLVuGaZqdxtTU1DBmzJjCmMWLFxOJRJg4cWJhzKRJk4hEIp3GjBkzhpqamsKYM888k2w2y7JlywpjpkyZgtfr7TSmrq5urweEZrNZEolEp5vo3/rcb0dC9BF//etfGTduHH//+98JBAIEAoHenpI4yMVSJrWxFJUh3y71VJqmURnyURtLEUuZB2xOvRpYTZw4kT//+c+8+OKLPPjgg9TX13PiiSfS0tJCfX09AFVVVZ2+p6qqqnCtvr4ej8dDSUnJXsdUVlbu8tyVlZWdxuz8PCUlJXg8nr2O6fi6Y8zu3HrrrYXarkgkIs3v+jmlFKtrE8TTOYaWBQl6XRi6RtDrYmhZkHg6x+rahCwLikNKKpXi8ssv5wtf+AKtra0cf/zxLF++nIsvvri3pyYOclnLIWc7+NzGbq/73AY52yFrOQdsTr0aWJ199tl86UtfYuzYsZx++uk899xzADzyyCOFMTtHoGqHIxD2ZOcxuxvfE2M6Pjz3Np8bbriBeDxeuG3ZsmWvcxd9W1/87UiI3rRq1SomTJjAfffdB8B//dd/sXjxYkaNGtXLMxOHAq9Lx2PoZEx7t9czpo3H0PG6Dly4s0/PZFkWL7/8Mvfff3+hkLyuro729vZPNZlgMMjYsWNZt25dYXfgztmgxsbGQqaourqaXC5HNBrd65iGhoZdnqupqanTmJ2fJxqNYprmXsc0NjYCu2bVduT1egmHw51uov/qi78dCdGb2tvb+eCDDxgwYAAvvfQSd9xxBx6Pp7enJQ4RxQE3A4sDNLZldlkpUErR2JZhYHGA4oD7gM2p24HVpk2bGDt2LOeddx5XXnll4ZynO+64g2uvvfZTTSabzfLee+8xYMAAhg0bRnV1NS+99FLhei6X47XXXuPEE08EYPz48bjd7k5jtm3bxurVqwtjJk+eTDwe56233iqMefPNN4nH453GrF69mm3bthXGLFy4EK/Xy/jx4wtjXn/99U4tGBYuXEhNTQ1Dhw79VK9b9B998bcjIQ40y7IK/z958mSefPJJVq5cyemnn96LsxKHIk3TGDMwTMTvYWNLkmTWwnYUyazFxpYkkYCHMQPDB7SfVbf/9f/Od77DhAkTiEaj+P3+wv1f+MIXeOWVV7r1WNdeey2vvfYaGzZs4M033+Q///M/SSQSXHzxxWiaxty5c7nlllt4+umnWb16NbNnzyYQCDBz5kwgv4X3G9/4Btdccw2vvPIKK1as4Gtf+1phaRHgqKOO4qyzzuLSSy9lyZIlLFmyhEsvvZQZM2YUUtVnnHEGo0ePZtasWaxYsYJXXnmFa6+9lksvvbSQYZo5cyZer5fZs2ezevVqnn76aW655Ra+973v9UoDMtE7+uJvR0IcSAsWLGDkyJGsWbOmcN/5559PRUVFL85KHMoqwz6mjqpgREWIRMZkayxFImMyoiLE1CMqqAz7DuyEurvdsKysrNCmoKioSH344YdKKaU2bNig/H5/tx7rK1/5ihowYIByu92qpqZGffGLX1Tvvvtu4brjOOrGG29U1dXVyuv1qpNPPlmtWrWq02Ok02l11VVXqdLSUuX3+9WMGTPU5s2bO41paWlRF154oQqFQioUCqkLL7xQRaPRTmM2bdqkpk+frvx+vyotLVVXXXVVp9YKSin1zjvvqJNOOkl5vV5VXV2tbrrppm5v4ZR2C/1fQzyt/vutzeqB19arvy7fqhas2qb+unyreuC19eq/l26WlgvioJTJZNTcuXMVoAB1wQUX9PaUhOjEcRzV2p5V22Jp1dqe7ZEWCzvq6ue3plT3ti+VlpayaNEiRo8eTSgUYuXKlQwfPpxFixbxpS99abf1TOJjiUSCSCRCPB6Xeqt+rDGRYXVtgtpYipzt4DF0BhYHGDMwfOB/OxJiP1uzZg0zZ85k5cqVAFx99dXccccd+Hzyd10cOrr6+d3tFrjTpk3j17/+NQ888ACQX99sb2/nxhtv5D/+4z/2fcZC9COVYR+nhLx94lwqIfYXpRQPPPAA3/3ud0mn05SXl/Pwww8zffr03p6aEH1WtzNWdXV1nHLKKRiGwbp165gwYQLr1q2jvLyc119/fbc9o8THJGMlhOgv5s+fzwUXXADka1EfeeSRTue5CnEo6ernd7cDK4B0Os38+fNZtmwZjuNw/PHHc+GFF3YqZhe7J4GVEKK/sCyLs846i+nTp/Od73wHXZfdruLQtV8DK7HvJLASQvRVuVyO3/3ud1x++eWF+inHcSSgOgCUUlJa0MfttxqrW2+9laqqKr7+9a93uv+hhx6iqamJ66+/vvuzFUII0avWrVvHzJkz+fe//83mzZv51a9+BSBB1QEgm2EOLt3+ibn//vs58sgjd7n/6KOPLhxpIIQQon9QSvHII49w3HHH8e9//5uSkhJOOumk3p7WIUMOde8+pRTRZI76eIZoMtfnzmbtdsaqvr6eAQMG7HJ/RUVFp87lQggh+rZYLMbll1/O/PnzAZg6dSqPPvoogwYN6uWZHRrUToe6dyz9Bb0uhnqCbGxJsro2wSkhrywLbtcfsnvdzlgNHjyYf/3rX7vc/69//YuampoemZQQQoj9a/ny5Rx77LHMnz8fwzC45ZZbePnllyWoOoDkUPfu6S/ZvW5nrL75zW8yd+5cTNPk1FNPBeCVV17huuuu45prrunxCQohhOh5paWlRKNRhg8fzhNPPMHEiRN7e0qHnK4c6t6czMqh7vSv7F63A6vrrruO1tZWrrjiisKBxD6fj+uvv54bbrihxycohBCiZ8TjcSKRCABDhw7lhRdeYMyYMbJDuZfseKh70Lvrx7Ec6v6x7mT3SoKeXpplXrf/tDRN4/bbb6epqYklS5awcuVKWltb+clPfrI/5ieEEKIHzJ8/n6FDh7JgwYLCfSeeeKIEVb1IDnXvuq5k93K20yeye/scBhcVFXHCCScwZswYvF5vT85JCCFED2lra+OSSy7hggsuIBaLFY4jE71P0zTGDAwT8XvY2JIkmbWwHUUya7GxJUkk4GHMwHCvL231BTtm93anL2X3urQU+MUvfpGHH36YcDjMF7/4xb2Ofeqpp3pkYkIIIT6dpUuXMnPmTNavX4+u6/zoRz/ixz/+cW9PS+ygMuxj6qiKwk635mQWj6EzoiLUp3a69baO7N6HTW0M9QQ7BZsd2b0RFaE+kd3rUmAViUQKL6JjfV4IIUTf5DgOd955Jz/60Y+wLIvBgwfz+OOPS3+qPkoOdf9kHdm95vYsG1uSVIZ8+NwGGdOmsS3Tp7J73TrSRinF5s2bqaioIBAI7M95HbTkSBshxP720ksvccYZZwBw/vnnc//991NSUtLLsxLi0+vNPlb75axAx3Hw+Xy8++67jBw5skcmeqiRwEoIcSBceeWVjB8/nksuuaRP/BYvRE/prXMVu/r53a0qL13XGTlyJC0tLZ96gkIIIXpGKpXiuuuuo7GxsXDf7373O77+9a9LUCUOOpqmURL0UB3xURL09Lm/490un7/jjjv4r//6L1avXr0/5iOEEKIbVq5cyYQJE7jzzjv5xje+0dvT2a/6+hlxQsA+NAj92te+RiqV4phjjsHj8eD3+ztdb21t7bHJCSGE2D3Hcfjtb3/L9ddfTy6XY8CAAXznO9/p7WntN/3hjDghYB8Cq1//+tf7YRpCCCG6qqGhgdmzZxeafZ577rn88Y9/pLy8vJdntn90nBEXT+c67Qb7sKmN5vYsU0dVSHAl+oxuB1YXX3zx/piHEEKILli+fDlnn302jY2N+Hw+5s2bx7e//e0+V2fSU/rTGXFCwD4EVgC2bfP000/z3nvvoWkaRx11FOeddx4u1z49nBBCiC4aOXIkRUVFVFVV8eSTT3L00Uf39pT2q/50RpwQsA+B1erVqznvvPOor69n1KhRAHzwwQdUVFTwzDPPMHbs2B6fpBBCHMo2bNjA0KFD0TSNUCjEiy++yKBBg/D5Dv7lr66cEdeczPaJM+KEgH3YFfjNb36To48+mq1bt7J8+XKWL1/Oli1bGDduHJdddtn+mKMQQhySlFLcd999jB49mt/97neF+w8//PBDIqiC/nVG3J7IbsZDS7czVitXruTf//53py6+JSUl/OIXv+CEE07o0ckJIcShqrm5mW9+85v87W9/A+Dll1/myiuvPOTqiPrTGXG7I7sZDz3dDvFHjRpFQ0PDLvc3NjZy+OGH98ikhBDiUPb3v/+dY445hr/97W+43W7mzZvHU089VQgqDqUMSMcZcRG/h40tSZJZC9tRJLMWG1uSfeqMuJ117Gb8sKmNsM/NoOIAYZ+bD5va+MfaJhoTmd6eotgPup2xuuWWW5gzZw433XQTkyZNAmDJkiXcfPPN3H777SQSicJYObJFCCG6LpfL8ZOf/IQ77rgDpRRHHnkkTzzxBMcdd1xhTE9lQHrrWJB9URn2MXVUReF1NyezeAydERWhPpv5kd2Mh65unRUI+WNtCt+8w29PO3+taRq2vfs18UOZnBUohNiTFStWcMIJJ2DbNpdddhnz5s0jGAwWru+pn1NjW4aI39Plfk79dXmqPwWD0WSO/3unjrDPTdC7aw4jmbVIZExmjKuR3Yz9RFc/v7udsXr11Vc/1cSEEELs3nHHHccvf/lLBg8ezBe/+MVO13oqA7KvzTb7QlDTcUZcfyC7GQ9d3Q6spkyZsj/mIYQQh5xYLMacOXO47rrrGDNmDMAej6XpiX5O+xqc9dcMV2/acTfj7jJW/WE3o9g38icqhBC9YNGiRRxzzDE8+uijXHTRRTjO3jMXHRkQr1unPWsRTeVoz1oo8qUYPrdBznb2mgHpTnDWQQqw903HbsbGtswumws6djMOLA702d2MYt9Jq3QhhDiALMvi5z//OT/72c9wHIcRI0Zw3333dapf3R2vSydrOizfFCOZs7AcB5euUxb0MqTMj0vXPzED0t3lKSnA3ncduxmb27NsbEnuWhPXh3czik9HAishhDhANm7cyIUXXsgbb7wBwEUXXcQ999xDKBT6xO/NWTZNbVk2t6YYVhbA43ZjWg7b4iniqRzFQQ/HDCreawaku8tT/fk4mb5QE9YfdzOKT08CKyGEOABWr17N5z73ucKOonvvvZeZM2d26XuVUrxb10bI52JQiZ941iKiaXgMnYjPzYaWFLqucXRNaK/BQ3ebbfbXAuy+VBNWGfZxSsjb60GeOHD6TI3VrbfeiqZpzJ07t3CfUoqbbrqJmpoa/H4/U6dO5d133+30fdlslquvvpry8nKCwSDnnnsuW7du7TQmGo0ya9YsIpEIkUiEWbNmEYvFOo3ZvHkz55xzDsFgkPLycubMmUMul+s0ZtWqVUyZMgW/38/AgQO5+eabD+rGfEKInnPUUUcxbtw4Jk+ezNtvv93loAo+zhyNqChi7MAI1WEfadOmJZklbTmMqAhSEfLgce0+AOrQ3Wab+3KcTG83L+2LNWEduxmrIz5Kgh4Jqg5yXcpYHXfccV3+i7B8+fJuT2Lp0qU88MADjBs3rtP9d9xxB/PmzePhhx/miCOO4Oc//znTpk1j7dq1hdT53LlzefbZZ5k/fz5lZWVcc801zJgxg2XLlmEY+X9kZs6cydatW1mwYAEAl112GbNmzeLZZ58FwLZtpk+fTkVFBYsWLaKlpYWLL74YpRR33303kO9fMW3aNE455RSWLl3KBx98wOzZswkGg1xzzTXdfs1CiP6pO0tMK1as4Mgjj8Tv92MYBk8//TSRSASXq3uLBTtmjoJeFxG/m2TWxnQc3LqOz61TG093KXPUneWp7ma4ejtTJDVhoi/o0k/35z//+cL/ZzIZfv/73zN69GgmT54M5Duvv/vuu1xxxRXdnkB7ezsXXnghDz74ID//+c8L9yul+PWvf80Pf/jDQj+XRx55hKqqKp544gm+9a1vEY/H+eMf/8ijjz7K6aefDsBjjz3G4MGDefnllznzzDN57733WLBgAUuWLGHixIkAPPjgg0yePJm1a9cyatQoFi5cyJo1a9iyZQs1NTUA/PKXv2T27Nn84he/IBwO8/jjj5PJZHj44Yfxer2MGTOGDz74gHnz5vG9731PfkiFOAR0NXCwbZs777yTH//4x1x++eX89re/BaCsrGyfnnfn2ihN0yjyffzPdzJrdWvrfleXp7pTgL2v/bF6Un+uCRMHjy79FN54442FW1NTE3PmzGHx4sXMmzePefPm8cYbbzB37tzdniH4Sa688kqmT59eCIw6bNiwgfr6es4444zCfV6vlylTphQKP5ctW4Zpmp3G1NTUMGbMmMKYxYsXE4lECkEVwKRJk4hEIp3GjBkzphBUAZx55plks1mWLVtWGDNlyhS8Xm+nMXV1dWzcuHGPry+bzZJIJDrdhBD9T1eXmLZu3cq0adO44YYbsCyLhoaGT30KRVe27tcU+1FKdXkJrqvLUx0ZrhEVIRIZk62xFImMyYiKEFOPyAdLO2eKgl4Xhq7lM0VlQeLpHKtrE/t9WXDnmjCF6tSawuvWP7ElhRCfVreL1//nf/6Hf//737vc/7WvfY0JEybw0EMPdfmx5s+fz/Lly1m6dOku1+rr6wGoqqrqdH9VVRWbNm0qjPF4PJSUlOwypuP76+vrqays3OXxKysrO43Z+XlKSkrweDydxgwdOnSX5+m4NmzYsN2+xltvvZWf/vSnu70mhOgfurrEFHv5eS699FJaW1sJBoPcfffdzJ49+1NntD8pc6RrGvGUyXOrtu2XJbhPynD1lUzRjpk9y3HY1JKmJZkttKYIelwUB9zSlFPsV93+2+X3+1m0aNEu9y9atAifr+s/wFu2bOE73/kOjz322F6/b+cf0o5zCPdm5zG7G98TY3Y+I3F3brjhBuLxeOG2ZcuWvc5dCNH3fFLgEHY5/PyGuXzpS1+itbWV8ePHs3z5ci655JIeKxPYU+aovMiLUtCSzO7XYu29Zbi6snvwQGSKOjJ7HzUneWdLnG3xFAG3QVnQi9+l82FTO01tWXKWnGMr9p9uZ6zmzp3L5ZdfzrJly5g0aRKQr7F66KGH+MlPftLlx1m2bBmNjY2MHz++cJ9t27z++uvcc889rF27FshngwYMGFAY09jYWMgUVVdXk8vliEajnbJWjY2NnHjiiYUxu1uibGpq6vQ4b775Zqfr0WgU0zQ7jenIXu34PLBrVm1HXq+30/KhEKL/+aTAwUwlePOV5wC47rrr+NnPfobH0/OZmZ0zRx5DY8XmGM3t2U6ZtIDXoFx52dCcYslHLcwYN+ATG5B+Gn3l+BZNy7ec+Oe6JrbG0gwrC+A2dHKWQzxrMajET8jn4t26NirDuwbJQvSEbv8t//73v8+f//xnVqxYwZw5c5gzZw4rVqzg4Ycf5vvf/36XH+e0005j1apVvP3224XbhAkTuPDCC3n77bcZPnw41dXVvPTSS4XvyeVyvPbaa4Wgafz48bjd7k5jtm3bxurVqwtjJk+eTDwe56233iqMefPNN4nH453GrF69mm3bthXGLFy4EK/XWwj8Jk+ezOuvv96pBcPChQupqanZZYlQCHFw6Qgc0qZVqNlpy5iF42RCZdV86ye/5Klnn+f222/fL0FVhx0zR5qmURdPd8qkxdM5Vm1N8O9NUbZEkyxc08D/razfr20G+tLxLR6XQUXIw4iKIGnLybekMG2qwz7GDowwoqJol2N7hOhJmupDjZimTp3Ksccey69//WsAbr/9dm699Vb+9Kc/MXLkSG655Rb+8Y9/dGq3cPnll/N///d/PPzww5SWlnLttdfS0tLSqd3C2WefTV1dHffffz+Qb7cwZMiQTu0Wjj32WKqqqrjzzjtpbW1l9uzZfP7zny+0W4jH44waNYpTTz2VH/zgB6xbt47Zs2fzk5/8pFvtFhKJBJFIpNAkUAjR9yml+OuKWhZ/1IpLh3hrEwt+fyOnfvFizjzrTKKpfDH3KUdWHNAsSH08w3Or6hhUHMDQtUJQlcyZFPs9GLpGQyLDoFI/g0uC+3Vn3p52BXbsHuwodN/fOt6TgRE/GdMptKQIeg00TcN2FFtjKaaPraE6Ip3PRdd19fN7nzqvx2Ix/vd//5ePPvqIa6+9ltLSUpYvX05VVRUDBw7c50nv7LrrriOdTnPFFVcQjUaZOHEiCxcu7HT8w69+9StcLhdf/vKXSafTnHbaaTz88MOFoArg8ccfZ86cOYXdg+eeey733HNP4bphGDz33HNcccUVfPazn8Xv9zNz5kzuuuuuwphIJMJLL73ElVdeyYQJEygpKeF73/se3/ve93rs9Qoh+qamtiwt7TlSWZMt77zBK/ffRDLeStOWDzEGj+OE4RVdOvetp49Z2XEJLuA12NSSJpkzqQr5QNO23+9ieHkRze3Z/drDqa8c39LxnmQtp1NLig4HallSHLq6nbF65513OP3004lEImzcuJG1a9cyfPhwfvzjH7Np0yb+/Oc/76+5HhQkYyVE/6KU4tX3m3h/azPPPfRLnp+f3/lccdhIzvvu7ZQOGs7EYWV84fiBew1YutM8s6sBWMfcPmxqo7zIy783RQm4DbxuI78E156lOuxj3MAIqZxNImMyY1zNft2Z19tn9O34nuxYd9ZxbWNLsleyi6L/228Zq+9973vMnj2bO+64o1Pm6Oyzz+7WEQ1CCNEfxFImb61Yyf03fYeN694DYPpXv84FV32foD8AKNqy1l5bCXSneWZ3ArAd2zBsaE6RylmEvC4ypk08YxJ0uxhSGkDTtAN2rl9HDVhv6U5TUyH2h24HVkuXLi3UKu1o4MCBu+yaE0KI/m7d+o/4ySXnYOayFJeWce0vfsPEkz9uaGw7ipZUbo8BS3eOWWlqy3a7e3nHEtySj1r4qLmdhkSGgNdFddjHkNIAxYF8kHMoLYH1lWVJcWjqdmDl8/l22z187dq1VFRU9MikhBCirxh5+HAmTTuX9tZGvn/r3ZRWdG44/EkBS1ebZ0aTuX0+564y7GPGuAGgNNY1JRheXkTR9qNvYNdz/Xp7ue5A6OqxPUL0tG4HVueddx4333wzf/nLX4D8PwybN2/m+9//Pl/60pd6fIJCCNEVPRks/P3vf+foo4+mqqqK4oCbH/z8LjbFspSUhzqN291BxDvrSvPM5mSWprbcp+perus6k0aUkrFsmtuz6NuX/3ZeAmtqy/bqQckHUm8vS4pDU7dzwnfddRdNTU1UVlaSTqeZMmUKhx9+OKFQiF/84hf7Y45CCLFXjYkMr77fxP+9U8dzq+r4v3fqePX97ncdz+VyfP/73+f0009n9uzZOI6DpmkcP7ySkqCPjS1JklkL21EksxYbW5KFgAUgmswVzupzHIdoMkc8lcOyHdI5a7fP2ZHxAvWpu5d/0rl+QJfOOxRC7LtuZ6zC4TCLFi3i73//O8uXL8dxHI4//vhdDlEWQogDoTuF4Xuzbt06Zs6cWTgL9bDDDsM0TTweD25DZ3RNiI8aU8TSWZqTqlPNDsCr7zcVskBZ096eOTPwunS2tKb5qCnJhCGlnTIoO2a8KkLeHulevqclsI457stSoxCi67odWP35z3/mK1/5Cqeeeiqnnnpq4f5cLsf8+fO56KKLenSCQgixJ90pDN9TsKCU4uGHH+bqq68mmUxSUlLCH/7wB774xS/md+h99HHA5NF1igMehlcUMbDYT3HAvUvBedZ0WN/QTmNblsqQl+MOK2ZUdRFLPory2geNfGZYGVVh3y5LdCVBDwOLA/k2AZ5d2wR80pLjjna3BBZNfrqlRiFE13R7KfCSSy4hHo/vcn9bWxuXXHJJj0xKCCG6oquF4Xs6viQej/PVr36Vr3/96ySTSaZOnco777xTCKo6ls1CXhfFfg+OUmxoTvJubRzTzi/J7RjYBTwGW2IpLKU4sqoIy3HYGs1QHfFzyqhyAl43HzS0sSWa7LRE13Fu3ZiBYSJ+T2HJ0bIdGhMZ3qmN4TJ0jq4J7XM2qa8clCzEwa7bGSul1G5/sLdu3UokEumRSQkhRFd0tTB8T8GCYRisWLECwzD42c9+xnXXXYdhGJ0yYcV+Dx82J2lN5mulDF2nLpbGbehMHVVBbSxFRZGXZNYmls6xLZam2O9G03WK/R5aklmSWZvigJcTR5TSkMhy0uGVVIa9uxTYV4Z9TDminDc3tPJefRv1sTSmrSjyufC784cHa5q2T0XmfeWgZCEOdl0OrI477jg0TUPTNE477TRcro+/1bZtNmzYwFlnnbVfJimEELuzL8GCZVnouo6u6xQVFTF//nxM02TixImFMR2ZMK/L4N26BEnTIuJz4/G5ydkOTe0W/1zXRHXER0syS0M8SzSdI5E2qY2lqY74qAr58HsMrKxZyG753S5cRo5IwE1xwL1LHVRTW5Z369rY0prkw6Y2cqbDYaVBjq4J43Mb3a4b21HHQck9sdQohNizLgdWn//85wF4++23OfPMMykqKipc83g8DB06VNotCCEOGKUUSikCHhebWpIcOSCErumdru8cLGzYsIGvfe1rfPnLX+Y73/kOAMcff/wuj521HLKWTSxpkTQtKos+rtHy6QY1ER/rGttZsTnKxqYkStOoKPLiNXRa2rO0tGfJ5hyqIj5cuo7byM+rI9BLpE3W1HVueRD0uIimctiOQzxtEfa6KS52E8+YvLetjbGDwgwt2/cic+lILsSB0eXA6sYbbwRg6NChfPWrX8Xr9e63SQkhxN7seOxLSzLLpuYUW2MZxg4MURny7zZYePLJJ/n2t79NIpFg3bp1fPOb3yQYDO728b0uHdtRNCTSRAKeXYINy1YU+z1sbM4HVW4dfC4dBZQEvcSSWdKmyeZWm+MGlxD0GoVAr7zIy4rNURIZsxDcpHMWiz9qoT1r85mhxSRzNsUBDz63gc9t0NCWYVNLmrGD3J+qyFw6kgux/3W7xmr06NG8/fbbndLmAG+++SaGYTBhwoQem5wQQuxs5/YKlSEfpQEv79bFWb4pxtByk9KgpxAs+DWTiy/+VuGA+BNPPJHHHntsj0FVRybMrRs0tGUpL/LsPIBYOr+cVxtNM3ZgmKY2k4a2DMV+D+VFHtoyJs3tGdyGjkvXaGrL0p61KPa7UQoSGXOnA4I1DF3Hpdtsas1g2g4R3/YlOU3rVKvl/5Rn/klHciH2r25XKV555ZVs2bJll/tra2u58sore2RSQohDg1KqU1NNpdQnjt9xF17Q68LQNWqK/Zx2ZCVDy4MMKg4wfewATjmygo3vv8Nxxx3Hn//8Z3Rd58Ybb+S1115j2LBhu338jkajz63aRmNbhngqx4otMVqTORylyJo2DW0Zgl43JQEPmgbVkQBjB4UZEAmQMm0yVr7eK+h2k87ZLNsc5Y0PW0hmLQaXBkjmrF12MZqOg+04lBd5SaRzOE6+WWgHt0vHchxM2+mRIvOOdgzVER8lwV0zckKIfdftjNWaNWt2W5Nw3HHHsWbNmh6ZlBDi4Lfjcl5Xj1bZW3sFXdcZWhYkkcm3Vli/qY4pU6eSSac57LDDePzxx/nc5z631/nsmAmrGOylPWuyqjbBh41tVEV8hHxuBkQCHFbqoy6WoSzoxdCgyOdh7CA3yaxNS3uWdY3tuA2NanwcN7gEn0unPWuyujZBMpdfAtyRW9dxGfmlRF2HIm++tsrr0tE0DdNycOn57JcUmQvRt3U7sPJ6vTQ0NDB8+PBO92/btq3TTkEhhNiTjiAmls5R5HURNFw4SrF+h11vFbtZrupKe4UNLe28uraJVM7iPy66iq3r1/DDX/yKI44aXBi387mCEb9rt41Gjx1cgq5rbGpOEvK5OG5wMS4jv7RXHfExqCRAU3uW4PYDj4Megw/TJrbj4DE0aoqDDC71o6FRoby8ty1Bc1uOdKlFke/jwCjoNSgNetjcmiLoMRhSFmRDU5LG9iwRr4toOkdZ0EdTe5ZiKTIXok/rdiQ0bdo0brjhBv72t78V+lbFYjF+8IMfMG3atB6foBDi4NKxnFcbS+M4ik0tKSwnn5EpDXhIZm0WrWsmEnBTF0t3ymYNKvHvtb3Cc8/8lVajBH38cQwpC3Lpld8lY9o0tOe7o08dlT8vb+dMWcTnYWssxcBif6eAJRJwM6IihGUrmttybIunKQ54Oh1l84+1TWxoSVLkdZHO2WxqSaKRL2IfUpYPqiC//HZYWYDGtiybWlOMHvBxcKRpGoeVBNjQ1I7lGET8LkYPCLO+qY1NrSncRj6wPFyKzIXo87odWP3yl7/k5JNPZsiQIRx33HFAvgVDVVUVjz76aI9PUAhxcImlTN6vT9CYyGA5DsV+D26XG9NyqE+kMS3F2voEoweEGVIW7HT2X1N7hqDHRUMiTUXIh+Uo3LqObme5/44bef5/H6Ns0HC+/PRCAtsDryJDJ+h1sbElyaJ1zZiOQyJtdmo3sK4pwYbmFOVFnkLAFk/n2NSSpiWZJWfbWLZD0ONm4rAyRlYVFYKiMQPDvPhuPe9sjRNP52hpzzGsPMiQUj8Rf+fCd7/bRUXIS8Dj2qXlQSyd45jBxfjdBrXRNA6KgcU+xtQUM7wywMDigBSZC9EPdDuwGjhwIO+88w6PP/44K1euxO/3c8kll3DBBRfgdsuavxBi79I5i3UNbWRthwERH163Dmh43QaVhpcVW6Okcg6njKosBDk7nv3nNnTqYhmWbY4RcBsktq7n2d98n6atH4GmMWnK6bhdnQMabXufqX9vilIZ8nLUDtmioNfF8PIiPmhoZ31jkhOGeUikTVZtzddDFfs9+VonpdGWNXl7S4zigJvKsK9QJxb0GJw4vIy0abNySwwN2NSSJux3dwquMqZNadDDxGFlbI2mO7U8KAt60TRoz1g4Kl8zVhL0MnFYCVUR/4H64xFCfEr7VBQVDAa57LLLenouQoiDXGMiw2trm1nX2I7PrZPMWoR8biq3Z3EytsK2QQMsp3M7AU3T8Ll1lm+KURH2MbjYz6tPPcxrj/0G2zIJllTw5Wtu4yvn/Qf69gDFdBxcmgYaRFMm2+JpDq8M7pL1KfK6GFIaYHNrkiOri9jUkiaZM6kK+VBAY3uW6mI/R1YV8X5DO//8oJmpR5azanuWalh5PoOllCKWNqmPp2nP5gq9pzS0Tg1LR1YVMbKqqFDnlUibrNgSLWTSKrcf0tyQyPDaB8277bS+c52YZLOE6Bu6FFg988wznH322bjdbp555pm9jj333HN7ZGJCiINLR8F6bTxJkddAQ8OtazQl8q0WhpQFcRuQNm3Cfhduo3OBukLREM/RlrU4xu/wu59czb8XvQrACVOmcd7VN5PWg9TH0zQnc7Qmc7SlTRIZE0U+WGtsy7KlNUOR101x4ONMkqZpHF4Roi6e4b1tbURTObwunZZklljKpMjrwmvorKpNsLk1xdtbonzQmCCZsTmiOtSpVmpIaYC2tEU0naMunmZIWaCwm2/n7uYlQQ9KKdbUJUikO/e22jFLt3On9X3ZUSmEODC6FFh9/vOfp76+nsrKysLRNrujaRq2bffU3IQQB4kd+0+NqAhRF8tSG0tRG8+gHEXatGlNZgn53GhAsd+DZ6c+TcmsTX1bhhK/m1BRCDObxeP18e3rf8qML19EMmfxzw+a+df6fOG729BJZCzSpgPKIWM6BD355pqraxVjBoY7BVdet87oASGUguWbo+Qsm6TpoGsQ8rh4r74NWymCbgPTcVjf2E5LMkfWsgl4jMJjdeza29ic4sPmNja35mu39tTdfG8tJDRN26XT+s5tIXasQdvXcwSFED2nS4GVs0NK3nH2rduvEOLQtWPwEPAYlATcrG90UI6Dho7HpZM1HYJeBzQNXdcJeDoHVu3JJM2xNsYMriAc8HD9bfeQam/jsBFHkMzapC2blGlhOQrHcWhJm+Qsm7DXRTJn4TKgJODFrUMyZ7KpNUXE7y4s4TW0ZQh53TS3ZzB0DbdLp8xj4HcZ1MUyRNM5fG4Dx1EUB9yUBT20JHOsa2on7HczaXhZITAqDng4vFIj5DeYekQllWHfHpfqutJCoqPT+s4NUruS3RJCHFjSeEoIsU+6U+PTKXjQADQ8Lo0ijweP2wVK0ZbN1xflLAe/W2djS4qq7RmZD95fw23XX0HF4cdw6o2354vRq2uIpXK8UxunNZkjmsqxpTXNUQOK8BguNkczGDrkHIfSIi8Rf74dgsvIn81XH0sztCyIS9f4qDlJImUST+doTmaJpbKkTcXh5X5sNFKmhVIKr0sja9o4yk1FyEsq5/BRc5L1je2MGRghtL03lVKKpvYsh1eEqAh5yVoOsZS52/fI69L32kJix07r3c1uCSEOvC4FVr/97W+7/IBz5szZ58kIIfqH7tb47Bg8KCBj2YyqChFLW7RnTDKWg67rDIwEKAt5iKdNijwGdbEUrzz9OM/cfxtmLksy1kJLSzMDIoOIp01W1caJpUwCHp2c5VDk1XEcjazK776rDvtwGzo+t46jwHKyjKwqItpu8mFTG5tbk3hdBom0Sc7JH1eTsx3SOUV71uTtrSZ+r4tE2sTQNVqTJqVBLy5dI2spKkJe4ukc9YkM9YlMvgB/+wHQmqaRyJg8t2rbXt+j4oCbgcUBPmxqY6inc2H9jgXvxQE3DYlsl7NbQoje0aXA6le/+lWnr5uamkilUhQXFwP5BqGBQIDKykoJrIQ4yO1Ljc+OwUPE78ZyHMqCXooDnvzut7YsYZ8Lr1tnc0uarbEkH22p46UHbmbDstcBOOFzp3Dn3ffRYPrZ0NzOppY0G5uTKOWQNB1cuobHlV9WTJoWGdPGrWuFIMS0bFx6vq1BWdBDyO9iyhEVfFDfns9gxU3aMxZ+j4Ghg8el0561sdLbi9+3xzuGBrajsB1F0OtiUEmAVM4mmbXZGksVWidEUzma27Of+B5pmsaYgWGa27O79LbaueC9O9ktIUTv6NJP34YNGwq3X/ziFxx77LG89957tLa20traynvvvcfxxx/Pz372s/09XyHEPujuYcd7e5zdHYIc9LoYWhYkns6xujaBUgqlFK3tWdZuS7C2vo2QN38W3obmJLatyJgWiYxJQyKLa/t5eA1tGdqyJmuXvcEj//UVNix7Hd1wM2nmd5ly9S9Z3+ZizMAwPpfB25tbaUpmMRUMjPgZVR3C0HXWN7YRdBloQFP79teqFLF0jrKgl4BHp6kty8jKEJUhL7F0PsMTz1hYjkNLMkcsbZI2HYJuHZcGHiMf0BR5XTgKUjkLXc+/HynT5sjqEF88robpY2uYPnYAkYAbR6lPfI86VIZ9TB1VwYiKEImMydZYikTGZERFiKlHfByEdQSojW2ZXf4MO7JbHY1EhRC9o9s1Vj/+8Y/53//9X0aNGlW4b9SoUfzqV7/iP//zP7nwwgt7dIJCiE/n02zN37mOSinVpRqfdQ3tvFsXZ9mmKBubk7SkchhoVITclAQ91MezvLfNxO9x4XNp2AosRxHyGLy/tZFlj/wUO5XAXzGYI776I4qHj6IuliaasjA02BxNkbNVfvefrqHp4HUZjKwM5lsiRFOEfC5chsbWWBqUIhLwUBFys6klVcgC5WxFPGPS1JYlnbNw0LAdG5euYTkKC7ABr65hOop81VI+aMxaNom0iW07jB9SwbCKfC+raDJHXSzd7TqoyrCPU3ZzPuKOj9Gd7JYQond0O7Datm0bpmnucr9t2zQ0NPTIpIQQPePTbM3fXUAWcLtoTeYfa3c6DkF+blUd79YmqE9kaG5Lk7EclIL6hE512IvHbeAoCHkNqsNe1mxroymZw7IcLM3DkHO/Q/uHyzns7G/j8vrIWA6mo4jHUzy1IkvY76I04KbI78HQNWIpk3TOZkhZkFHVRduXCKE85CGZybeAKQt6AK1T24NoMoemoD6RQdOgJuyjLpYBbFw623cYKmwdgp584b1CQ6ERTeb7Wx17WAmfG1kOQDSZY0s0HzRVhLx7fI/2VAeladonFp13ZLc6/mw6OrfvqZ2DEOLA6nZgddppp3HppZfyxz/+kfHjx6NpGv/+97/51re+xemnn74/5iiE2Ad725o/xBPg/W1t/HNdM6eMqqAk6OmU5WhMZHj1/UbqExmK/W6K/R4MDTa1ptjYnKQ06GZAJLDLc6ZzFs2J/M66hkSalrYsltIo3n6sS1vWYlNritKgm5GVETQNPmxOsvLvf0UPRAgfMZGs6RA+6rNUjDkZy3FI5SzSpk2RRyeZtUnmTCL+IOGAl5RpEvF5iPjdxNMmjW1ZBpf48LtdfPbwcqaNri7UG+VstUsWKN82wUcsbVIV8uLzuKgIe0maFqbpYDk2uq6haxrDK4oI+9zkbAfTcji8MsS4gRHGDsofRv/q+035TFQ6x9qGdtoyFqOqi3Y5L7An6qC6kt0SQvSObgdWDz30EBdffDGf+cxnCmcDWpbFmWeeyR/+8Icen6AQYt/saWt+x+HCdfE079UnaG7PMLIyXMh2KKVYtK6Z5ZujGLpOXSyNy9ApDXooC7ixHMWyTTH+Y4wXXf94d5pSik2tqcISWSpn4wBFXgOXng8igm4XGdOhPbs9YGqLs2L+XWz69yu4AmGOvPJ+snoI03bIuRV+t4Hb0LEcRTRl0ZaxsB0nX+zuV+RMB9PKEvF7CLgNosksjlKEfC4mDS9nQPHez9jTNI0xg0IUve2iJZnDresUeQxKA15ak/nMmNtl4PcYnDKqkppiH+/XtzOoOMApR+YD0qa2bKesYEWRl7a0xYdN7WRNm3GDI4Xgauddfp9GV7JbQogDr9uBVUVFBc8//zwffPAB77//PkopjjrqKI444oj9MT8hRDfsWBMVT5lkLRuf++OloXg6VzhcOOJzoynwuYxOS4OxlMk/1zXhABVFbjyGm3gmx4rNUUxbEfDotLTn+OvbDqNrwgyI+DF0aGrLEvC68LoN2jL5cgFD1zG0jzMzmga6Do6jWPv2W6yZfyuZaAOablB14n8SCEVIJh0U+fKCLArD0NG0/K6+rJWvf6os8mI64HFpZEwHQzfRNchYDmW6xkkjKxhZVdSl92xQSZAThpayujZOczKH163hc2sU+dxAPkirDvnxugw2t6YZUOznpCPKKS3y7jEreERViKzlsDWaxus2OH5IMVnTkTooIQ4B+9wgdOjQoSilGDFiBC6X9BkVorftXBNl2YotrSk8hkFNsR+F6nS4cMZy8Lh1In4PQa/BxpYkq7bGiaby5/GNrCzC0HVSOYv6WHb7qQsaqHzLgc2tqe3HtbgJ+zwMrwhyzKBiYu1ZLKVQCly6hqMUxvYgQqGBY1P3+hM0vj4flIO/fCCDvnAdgYGjyNigaQ4a4KCRsxVuHFyGTs5RGLqG32NQFfYRS1lkLAtdcyjyugn7XERTOY47rJjPjSzfbeCyu6amxQE3E4aU4tI10jmb5vYsCqiJ5JcP27MmXo+G5Ti71DHtKStYHPAwdmAEr0unLp4m1OCiOOCWOighDgHdjohSqRRXX301jzzyCAAffPABw4cPZ86cOdTU1PD973+/xycphNi73RWpp02LD5uSvLWhhSlHVOLefqhwsd+DAuIZk+qwj6DXKOxWW9/URjrnUBzwYNkKQ1c0JDK0ZXOEvG5Sps2m1hRBj4tjBofZ3JKiuS3H5tYM79Un8suHGmRyNjnLxu0yyNkKn6YBikwmxYd/+i/SW98DoPTYaYw87yrc/iJylk171soHKCpfNO42dGwFulIEXBpu3cDrNvKBlN9DU3uWaHuW5vYMtvIydlCE6WMH7DZw2dvuyI6ddrFUlmHlQXRdw3Hy3eC9LoMJQ0sZWOzfpY5pb8fRFAc8HH9YCaGmNqYcUcHgkoDUQQlxCOh29eQNN9zAypUr+cc//oHP9/E/Xqeffjr//d//3a3Huvfeexk3bhzhcJhwOMzkyZN54YUXCteVUtx0003U1NTg9/uZOnUq7777bqfHyGazXH311ZSXlxMMBjn33HPZunVrpzHRaJRZs2YRiUSIRCLMmjWLWCzWaczmzZs555xzCAaDlJeXM2fOHHK5XKcxq1atYsqUKfj9fgYOHMjNN9+8z/2AhOgpe+otVeR185mhxSg0/r2plVgqS862sZWisT1L0O1iSGmgcFae5SgaE1lSpk1VkZdYOkdLMsumlhRtaZu6eIba1hTRVI5IwIVlKbZGM9QlMqAU6azF+sZ21tS3E02ZxDMW8bSJbdukchbxjEXGMfBVDEH3Bhj4hesZe8H1OC4/lm1j2goNMDSFpm9fNiTf+NPvMogEvAwqCVAR8hHL5DuhH1bsZ2hFkKDPxbGDipn5mcOoiuxaV9UReH7Y1EbY52ZQcYCwz82HTW38Y20TAFNHVXB4ZRhbKZI5C1spRlaG+Y+xAxgzMLJLgT907ii/O1nLodjvYXBJYLffL4Q4+HQ7Y/XXv/6V//7v/2bSpEmd/pEYPXo0H374Ybcea9CgQdx2220cfvjhADzyyCOcd955rFixgqOPPpo77riDefPm8fDDD3PEEUfw85//nGnTprF27VpCoRAAc+fO5dlnn2X+/PmUlZVxzTXXMGPGDJYtW4Zh5H+LnDlzJlu3bmXBggUAXHbZZcyaNYtnn30WyNdyTJ8+nYqKChYtWkRLSwsXX3wxSinuvvtuABKJBNOmTeOUU05h6dKlfPDBB8yePZtgMMg111zT3bdRiB6zt/PjItuXpD5oaGNbPEtb2gIHqov9DCkNUBzwEEvl2NSaoj6WpjWZw2XoVIQ9ZEyHTfX5RpXFATe27ZA0bTRNoy1tsjSWbzaq6wqXoZGzNXKmg9+tYerg0XRypkMmmcSFie6PgAYjzrkcbdrXKKkeRMa0cRyHtKlwVD6w0shnrAIeA5eeP5C5JOjhsLIAHpdBVchHwGvQmszRZjs4jmJoaZCzxlRTFfHvstwX8bu6dnDxkRWccmRFt3baFQfc1ET8rK6LMyDiw2MYhQxgTxaqCyH6D011M+USCARYvXo1w4cPJxQKsXLlSoYPH87KlSs5+eSTicfjn2pCpaWl3HnnnXz961+npqaGuXPncv311wP57FRVVRW333473/rWt4jH41RUVPDoo4/yla98BYC6ujoGDx7M888/z5lnnsl7773H6NGjWbJkCRMnTgRgyZIlTJ48mffff59Ro0bxwgsvMGPGDLZs2UJNTQ0A8+fPZ/bs2TQ2NhIOh7n33nu54YYbaGhowOvN96e57bbbuPvuu9m6dWuXfxNNJBJEIhHi8TjhcPhTvVdCANTHMzy3qo5BxQEMfdfdf03tGZrbsoyqKiJtOui6xglDStB1nVgq3wU8mTPJWQ6DigPkHIcPG9vJ2TbJrENbJoeh6zgq35Xco+tkLJu2jI2hgdul4zbyzUOd7XVVOcsGTSOY2MCqR3+Ov2wgx3zjVspCPg4rDWI5ipKgm/e3tbGhOUUyZ6KjEQm4yVj542GKvS5KQx58boOwz4WmaQwuDTJuUISwz00ya5O1LTY2pRhRWcSpR1Zi2g7v1rV1Wu6L+DxsjaUYWOzf7TEwyWy+A/yMcTXd3mXXmMjwr/XNvP5BE21ZixK/m6qwn8qwl6xlEwl4OnVOF0L0X139/O72UuAJJ5zAc889V/i6I6B48MEHmTx58j5MNc+2bebPn08ymWTy5Mls2LCB+vp6zjjjjMIYr9fLlClTeOONNwBYtmwZpml2GlNTU8OYMWMKYxYvXkwkEikEVQCTJk0iEol0GjNmzJhCUAVw5plnks1mWbZsWWHMlClTCkFVx5i6ujo2bty4x9eVzWZJJBKdbkL0pN0tR3Xs/tsWT+HWNSqKfNQUByjyuYklc6yuS9CeMdnYnCKazh/7UhL0MrwyyKiqEBUhL01tOYr9LoaVF+EyNLKmQyaXP/olbTrYChwHcpZDIm3RnrXztVW2g6Epmhb9N/++ew6Z1m3kWrdyRMjiq58ZwklHVBDyu3EbOiG/O790qWk45Jcji7wuqkJe3G4XlpM/6NgwDKrD+QybS88fqNyWzbFsU5z6RIatsRTzl27mvtc+YuXWWKflvnVNCdZsS5C1dr9c53Mb5Gyn2wcXdywvNrdnOX5IMUdWhVGaxvsNCVZsjlIW9EpQJcQhqNtLgbfeeitnnXUWa9aswbIsfvOb3/Duu++yePFiXnvttW5PYNWqVUyePJlMJkNRURFPP/00o0ePLgQ9VVVVncZXVVWxadMmAOrr6/F4PJSUlOwypr6+vjCmsrJyl+etrKzsNGbn5ykpKcHj8XQaM3To0F2ep+PasGHDdvv6br31Vn76059+4vsgxL7a8YDjoZ4gaBR2/1UWeWlM5qgO+6gIeXfoBq6oT6T5sLmNoDe/nDWkzF/ot3REVYjNrSnashZFXhdtaYtYysRyHBwHOvLcFqA74ACayrdSMONNbPzbL0ltegeAwydOY/yF1xEsK8dl5M/MKwt62dTSns+SlfhwlINpK3wubXtLBxfVES9Br4sJQ8pI5iymjqqgqS1HbSzFxpYkG5uTeFw6xx1WQmXIy/LNUTa3phhU7Mcq9WPoLoJeF8PLi/igoZ31jUlOGObJLzXuYF8adu6uzUJ1xM/h2XwRfn08QyTg3mP3dSHEwavbGasTTzyRN954g1QqxYgRI1i4cCFVVVUsXryY8ePHd3sCo0aN4u2332bJkiVcfvnlXHzxxaxZs6ZwfeclNqXUJy677TxmT9uuP+2YjlXUvc3nhhtuIB6PF25btmzZ69yF6K6O8+Mifg8bW5I0tWVpas/gdxs0JnOditQ1TWN4eZDyoJfjDyvliKowk4eXMXZQuFOH8LKgl8NK/QTcBrajUJrCyYdPOCofSHXo+H8FJNe+wUcPXElq0zvobh+TZ/+IEy65iYrSEnQN1ta3EU3mKCvy4HHpxNP5fldKOXgNDctW6JqG32vgAG0Zi5SZr/EaVJJvyjl97AAGFQcYWh7ktCMrqSn2kzEdkjmbYWUBUqbFppY0ivzPZ5E3//o3tyZpz1id3rt9Pbh4d3VtGhpFXhelQS9DyoLUxdLEUrse/yWEOLh1K2NlmiaXXXYZP/7xjwvtFj4tj8dTKF6fMGECS5cu5Te/+U2hrqq+vp4BAwYUxjc2NhYyRdXV1eRyOaLRaKesVWNjIyeeeGJhzO7OMGxqaur0OG+++Wan69FoFNM0O43pyF7t+Dywa1ZtR16vt9PyoRD7w47nx+W7qWepCPqoDvsKReodOs6qi/g9lBd5MHRIZm1M28Ft6AS9Bq3JDOsakmyJJsnZDjkz37RT18DrAluBaUNHgaYGOLZJ8z8ewc604xswklFf+QFHHnMULe0mA70usqbNR81J1jW2Uxzw4DZ0QLGhOUksbWLaCpeuU+Q1yOYsNMchbSv+ta6Zs8cOIOJ3FYLDlGkxtCyIvr2ju+k4WLaDx+em2K/RksySzNoUefPfc3hFiLp4hg3NKUZUBD/1wcV7a7Ow43vc3eVFIUT/162Mldvt5umnn95fcwHyv0Fms1mGDRtGdXU1L730UuFaLpfjtddeKwRN48ePx+12dxqzbds2Vq9eXRgzefJk4vE4b731VmHMm2++STwe7zRm9erVbNu2rTBm4cKFeL3eQhZu8uTJvP76651aMCxcuJCamppdlgiF+LSUUkSTOerj+Z13XdljUhn2ccqRFfzHmGqOHVTMmIFhxg2MdAqqIH+en2U7KJVvxPmv9S28taGFpRtbWLSuif9bWcczb9fRmswScBvb+1nl+zrlbIVCx+sy8Lk0OsIKDdAMN+Xn/BdlJ57P2G//hrKBQ2hNWhi6RjKT32l33OB8LZKmFLXRFK1Jk5ztEPG5CXoMXHq+mPyj5hTvbmtjc0uSD5uSLFhdz19X1NKYyOw2qHHrer6JqO3gdulYjoNpfxzUeN06oweEOLwySCJjsjWW3+04oiK0T3VQn9RmoSfOAxRC9E/drrH6whe+wF//+le+973vfeon/8EPfsDZZ5/N4MGDaWtrY/78+fzjH/9gwYIFaJrG3LlzueWWWxg5ciQjR47klltuIRAIMHPmTAAikQjf+MY3uOaaaygrK6O0tJRrr72WsWPHFg6EPuqoozjrrLO49NJLuf/++4F8u4UZM2YwatQoAM444wxGjx7NrFmzuPPOO2ltbeXaa6/l0ksvLVT+z5w5k5/+9KfMnj2bH/zgB6xbt45bbrmFn/zkJ9Kbpo/aXZft/vBntbdGlp8UAGiaxtDyIEcNiPBhUxsVfJwtVUpRG02zbHMUn0snlbVZ29BGS3uWoNeF29BJZEzWN7aTytlUh7xUhr20Jk10HQxdw7QUtuMQ9LhQaCTfehpLGZRMOAeXDp6aw4kMH4XP58FR4DU0wj43Wcth9IAIIyryx8y0Z4Os3BKjPpHBpYFpO7h0HduxsRxF1lYYGgQMnaDHIJo2WbC6nmjK5KSRFYWgpmOXX9BrUBr0UJ/IEPG6cOn69ozYx8t9R1ZHmDqqnHja+tR/J3aua9vxMaTNghCHtm4HVocffjg/+9nPeOONNxg/fjzBYLDT9Tlz5nT5sRoaGpg1axbbtm0jEokwbtw4FixYwLRp0wC47rrrSKfTXHHFFUSjUSZOnMjChQsLPawAfvWrX+Fyufjyl79MOp3mtNNO4+GHHy70sAJ4/PHHmTNnTmH34Lnnnss999xTuG4YBs899xxXXHEFn/3sZ/H7/cycOZO77rqrMCYSifDSSy9x5ZVXMmHCBEpKSvje977XIwGm6HmfJjjpTbvroJ4x7U5n+XUluOroJL6xJUllyEfWdFi5NcbKLVFsYER5kPWNSZRSRPweamMplIL2rE1bxsoHB+1ZWlM5cpaNYWjbz/2zsRUk4y3U/m0e7R8uA91F6ZEnUFQxCBR43Do5K18zpeuQzOWX7Q4r9eeXHB2HnOXQljEp9ruJKpNoysTcXhhvOQqPDh6XTtDnwmXkgxa3S2NtQztDSv2EvW7WN7UxvLyosNw3pDRAIm2yoSXFiIoifG6dZNbqtNzX0RPr09rde/xplxeFEAeHbvex2tPuN8j/Y/PRRx996kkdzKSP1f63p+CksS1DxO/pUnDSG5RSvPp+vjv4jo0sO65tbEkyoiLEKUdWAHxiNq4xkWFVbZy3N8dYsy2ePyTZYzCqOoSjYHVtgiKvgVvX2BJLY9n5/lGJtIXXpZOzFaadb6ugaeB353tVtb7/Fs3P/xo7FUdzeag8/VKGfu4cshYo8nVSPrdBZdiH16VjOYoTh5dhK0V9PL+UZzuK2liKeMokljbJmHb++Bqno5Yr/1xFXjelQTe2Uhw3uJhUzsbvMagO+9nQksS0HYaUBji8IoTXrfNhUzttWYuKIi9et77fA+r+GsALIbqvq5/f3c5Ybdiw4VNNTIj9aXfb4GE3XbZD3j6XTdhbB/WOs/xqYynWNbSzNZru0oe5cvJtFdqzNroGYZ8HQ9fRFQS8Om1Zk9Z2k5xt056x87VUSpHM2Z0K01GQyWRoffVhYv9+BgBf1TCG/ecNlA8ejqYUug6VRV4Ml0ZlkY8JQ0vwugxeWlPPkg2tuAxQTn73XNK02dyaJGvmu7b73DqgFQriDU3hQKFWymVoJLMWW2MZQj4XY2oiDC71s74xyebWJHXxDKMHhDh2cAlH14TwuIwDsgRcGfZxSsjbL5echRD7R7cDqx11pd2AEAdSV4OTWMrskSWhntSVnWYbW5K89kEToPa6VNiRtdsWT4OCoeUBNjWnaMuabGpWVEV8OA60pS0SWROPDvl8k8o3/lT5gMptaPl2C8qm9tHryNavB6Dms1+kfOpsisNBRg8IkchYVBT5KAt58Rkaje1Z4mmbo2v8aMDG5nYOKwtQEvBi6JCKW+ho5Gwbl2HgdxtkrXx7E8dW5JSDW9ewbId0zsbj0lnbkK/9KvLk35/SoJcThnk4akCIDc0pBpUEmDqqvLBT8EDRNK3P/V0SQvSeffoX6I9//CNjxozB5/Ph8/kYM2YMf/jDH3p6bkJ0W1eCk33psn0gfNJOs3TOZGtrmoa2NOVFXgIeA0PPN9wcWhYkns4fT+M4TiFrVx32YjoKVL5VQtCTz1I1xNNkTJusZePSNCxHKyzBeV35gFQBtp3vBqUwCBx1EkawmJEX/ZyxX5pDIOBncImfkqCX0oCHAZF8tixpOvhcBs3tGZracmQsB0PX8k1FFZi2Im06BDwu3LqObTs4SsPQNdx6vgN7x4Y+TcvPq8jnxnIcAh4DdPioKUkslUNDI+RzM6IiSDxtEk9bu3nnhBDiwOl2xurHP/4xv/rVr7j66qsLR9gsXryY7373u2zcuJGf//znPT5JIbpqx+Bkd+fC9eVt8HvbaRZLZVm0voXaWJqBto9/56KUBj0MKQ0Q8efPzfMYOusaEpQEXLxXn8DQYV1TktpYGgOIpk02R+18RimRwVGglIYD5GynkKVC0/AaimRbnFy6DW/ZIHQNyid9gcMmno0/XEpF2IvPo3NYWZBk1mZrLEMiYwIatso3+VRKEfQatKVNyoIeirxuMlb+OJyc5VAW9GLoGm1ZC7/boD1noesafreBtb2nFhoU+fJ/jrajGBjxMrIiRNK02dSaIuLPL7tJ3yghRF/R7cDq3nvv5cEHH+SCCy4o3Hfuuecybtw4rr76agmsRK/qz9vg97TTrLEtzZKPomRMm/IiD9URP5ajqE9kaIhnKPK5yFoO8VSOLdEUiz9qJpGxMXTwGDpBj0571kY5DqmcSQYdjws0dEzbQQfcOpgWZCwblEbbR2/T8H/z0D0BBl38a9x+H26XQUlRGQpFKmsT8bkJuA08usa7GZNoKsfIiiAul4tMzqYpmWPZphgNbVlKgx6UymcSywJe3LpGwGOgNDAdxbGDI6BBa9IkkTZpakvTlrXxufJBcMTnJuRzMbQ8SJHPjcvQaU3m8k1Afa4+HTALIQ4t3Q6sbNtmwoQJu9w/fvx4LEvS8KJ39fdt8Dt2UK+NpWhuz7KhOYlLh2MGRdjcmsKyHXweF5btsHpbHK/LoLLIw0dN7cTSFqmcTSxpEvQalIc8gEEqZ5O2HIJuF9FUjrZM/ngal64R9Bj4vW6yVpZsLkfzPx4l8dZTAOhlQXKpGHiqMR2FL+tQHHDh0jUGlQRoaMuQytr4t7/H8axFRNNIZMxCk1O3oeN3G3hdBsmsjWUrAl4XqayFTj6r5aBRWeRlQNjHxpYU5UVeLMfhmEHFlAQ9RPwuPmpKUZ9Ig1J4DJ22TL5FQ28HzP21X5oQYv/odmD1ta99jXvvvZd58+Z1uv+BBx7gwgsv7LGJCbGvdglOklk8hs6IilC/2Aa/406z97YleH9bAreusTWaYls8zabWFCMqArRlbAxNx7IcPmhoJ5o2qQr7KfUbxFPW9gOUjXzDze11Wxkr3z5BkW/eaStoz1q05WxUtI66p24nsy1foF507NmUnvoNfF7f9h16kMjkqCnJtzJImTa10TQNbVkC23f1GRo0tWdpz1r4XAaHVxTRmsphO/nnCXpcJHMWHpdGxrTJWA5jBhQzoDhfcB9Nm4S8Lk4cUUJb1mRgcYCAJx+QlQTdNLVlqU+kCXhcGJqOaTlsbEn2WsAs7RaEEDvbp12Bf/zjH1m4cCGTJk0CYMmSJWzZsoWLLrqoU8PMnYMvIQ6U/r4NXtM0TNvh3boE0bRJRZGHeMYia9rUxTNsaU0RcBsMKPZSH8/RljGJBNyEfS7QNIp8BumcojVtETB04lmTIo+xfSevwq3reFw6qZxNxnRoX/USLS/fjzKz6L4QZWfPIXDE5PzuFo3tR9vkg7K6WJoWI0c0lc+K+d0GCkXatMlZisPLi0iaFuVFXkI+F1ujaZSCaMokadrYyiGWgeqwn4jfxZCyAB6XTtjvojzoY+ygCIdXBvnH2mbe3hJFKWhN5bDsfOuFnKVobk9RHfFh2jZVIT/DKwO4Db1Lh7T3lJ5o5iqEOPh0O7BavXo1xx9/PAAffvghABUVFVRUVLB69erCuP7yASYOXv15G3xHP650ziLkdVEby5DOWdiORpHXRVNbjkTaJJrKYitwGfnjZhoTGXwuA6XAURqao2jOZEmZ+YOUTUdhOwpH08iaDpajcFC0r3kNZWbxHjaO8hnfwxUqB8Ahf+BykdcgY+oksyaJjEVVyKAq5CGRdbY3D3UxpMxL0OMiFHCjZzVCXjem5RDyuTmiKkRze5ZtsTRpMx/MnX5UJSeNLN9jz6nqiJdtq9K0pkwGFvsIBTykcha16Qxhv5sTR+TnGE/leHNDBo8RO2DZov7cL00IsX91O7B69dVX98c8hBA76OjHdVipn3WN7TQmMmiApRRel0FNxMu2RJZU1sJR4HdrtCmTZDa/zJezLXTNYEDER1zXyNo5LMchZ+YbeWo4JE2F0jQ0Tads+ndJvfc6oQnnoemdW1VYDjS25XAZOpYCZSlMC2IpC5ehkcnl66aOG1yMx2UQTeUPKs9ZNvGMyYBIgJpiHzXFPkZUFBFP58iYDqceWUlpkXfXF08+cKmPZ6kpCVAdUbSmckTTOVy6zugBYdKmzYrNMQZEvFSF/Qc8W9Sf+6UJIfavT9UgVAixf6RzFttiGZoNjaZEZnswovC5NbKGg6bl66ZspbAcsHMKcNC1fOd0gCKvQ86ySeesfKNP28EGLMsm/q/5OJkEpdMuB8AVKif8mS/ucT6mk++C7tFB6Roais3RNKDy7REcxfsNbQwvD4Km8BgGG1pSDCrxM6TMj8b2jI7HoKnNYWRlaK8BR0fgMrw8SMCbr7Eyt7dgCHp0lm6MsSWeYtygyA4HMR+4bFFX+qVJ+wchDk0SWAnRDfuyA6y737OmLs7fVtTx2geNxFM5omkTRyl8Lh3TNnCUTc60yZoOaNsbeW7vQdURVGlAOufQkMhi2fmO6pYCM1ZPy7N3ka17H4Dg2Gl4qw/v8uvXDR2XrmEphd+lYxguiv2u7XNQrGtoJ+R3M6YmgNdlEPa7cek6tqO6tTNzx8BFI7/82aE9a9GWNfN1X07no04PVLaoP/dLE0LsXxJYCdFF+7IDrLvfs6Yuzp8WbaAunsZRiozloCmFUmA5Co+mAYqc7WAB7BBXGNs/wy1n+3KgA7aZ71PlAO1rXqNpwe9QuRSaJ0DZmVd2K6iCfODm1nWylkPI6yboM0ibDiUBN0dWFbG+OZ9lmv3ZIVi24t26tn3ambm3wMW0HTKmjd+d79y+swORLerP/dKEEPuXBFZCdMG+7ADr6vd0ZLTSOYu/raijpT2DrmmkTRvLcXDIn92XNRWOyrcusDsnatAApcBtgOPkAym232dlUzS/fB/tq/8OgHfgUZSfcy2uSFW33gMN8Ln0/Nk4jiKWNvG4oD1jE/YbNKVyDIz4KCvyoGs6VREPlWHfPu3M3Fvg4tI10jmbiiIvQe+uS3EHIlvU3/ulCSH2HwmshPgE+7IDTCnFqto49fE01RFf/hw+fdfvOVp9nNWpi6b5+/sNWLZNa8rCUYqcrXB2CKTsPfTgVeSDr5z1cVAF+QxT/X//iOy2D0DTKT7xK4RP/OouBepd4Tbyx8p4XBpul0bWcqhP5FAoEmkLn9uiPODrdBbjvu7M3Fvg0tSeZUDEv9v6pgOZLerv/dKEEPuHBFZCfIJ92QG2rqGdV99vJGM51MbTGJpG0OumOuzF5zZwGxrLN0fZ2NKO7eR3+m2JpaiPZ8haNraTD8Ty5/l1bZ7bzzjuTNMITzqf6CsPUnnONfgGH71LtqsrPAaEfC4cle+vZegapmUT8Xs4rCzAUQPCGJpGUzJHLJ1vBVEd+XSBxZ4Cl8MrQnzu8HJW1yZ6PVvU3/ulCSF6ngRWQnyCruwAa2rP0JjI1/Uk0ib/+KCR+kSGIaUBLEexLZZmxeYYadMm6DGIBFy0ZWyGlAU5fnCED5va2dSc3H5ESz7rpLannrpbKWQlmrHi9fgGjwEgcMRk/MPHo7k8+xRUuYGAx4XLMCj2u4inTVI5G5dLZ0h5gCOrt+/MUwq3DrpusCWaYmRV0acOMPYWuJQXeftEtqg/90sTQvQ8CayE+ASftAOsIZFhY3MK22nE0GFjc4q2rEXYa5A2bTa1pGhsy9CWMcnkHHKmi3TOJmPblAXc/Gt9K5ZjE0uZKMj3imI32acuSK19g5YFvwU0Bnz9blyh8nyjA9e+f/AbBnhcOj6XTs7OF8drWn6nXk0kgN9jkDVtYukcRT4PIyoC1MXSPbYrb0+Bi2SLhBB9kQRWQnyCvRVSR5M53trQQsDrpirsxXZgrdmGchxiGZtoc4pExiKVs1AKwgEXtgNp0yJrK5JZi7ZshoxpE03l0FD7FFA5uQzRvz9I+8oXAfBUj0TZFt0JMXTt45tlUzjOJuhzAYpkzsbnUlSFfOg6DCrx43XrtCSzuHSdAZEAQ8r8FHndbI2lDkgPJ8kWCSH6GgmshPgEOxZSb2hup8jrRtfzDTdXbo2h0PjM0GKKvG6iqRy6rjGoOMC6pja2JdLkbIXdUZfkKEzLwefSsWyHdU1JbNshncs374TtO/y6Mb9cw4c0PXMnVutWQCM86UsUf+5CNMP9iY+j7fBfjfyOQk3LLz8aOoS8LsqCHgzdIGNapEwHtwGHlQcZf1gJYb/n48ad3nzPqWTWkh5OQohDlgRWQnRBZdjHmIFhXny3nndqE2QtG5381v5xgyIoNNoyJtmcTc5yaMtZuHWNTM7BpWvYaKDlz/PLmjaOUhgatGXtfap7gvwOuLalfyX62iPgWBhFpZTNuAb/kGO69zjbb1rHzsPtzUZDXhfDKopAg2TWQgFuXcPnNphwWAkZyyHoMdC0j/8ZkR5OQohDnQRWQmy3c4f0iN9FPG0VCtJX1cYJel0cOyiCo/J9qpZuivLWhhhr69vJ2g4oRcZ0yNmKTM7CchSlQTfJnI3tOJi2jaZpuDUNB2efgyrIZ9LMaC04Fv6Rkyg7ew6GP9y917yb/9c1qA55GF5RhGHku6Z7DZ2A10XI5yKTsxlRGWJrNN3ru/KEEKKvkcBKHHT25diZnTukZ02HrGXnC9ddOhu310qVB91kbUVbxqQ2mqYtbZIxLFpTWdyGhkvTCXh0bMehLp4iZzk0tmexLIfs9rU+Q4eUZWPae53Snl+fbaEZ+R/dklO/iXfQ0QRHT/1UgYwO+D0ahqbhMgwmDC3lsNIgGcvBdhSGruFz6aRMm21mmrKgl8Mri/rErjwhhOhLJLASB5V9PXZmxw7pWctmfUOMxrYslSEvI6uKaExk2RpLs9nQOGpACMtWuIz8gcdboxkcRaELuGk7OErRls0Xb+eynYu497WmW1k5oq8+hNmylcqv3Iym6ehuH0VHn9Ltx9qxjsutgdvI9+MaWVHEtkSWhrYcg0sCnVpMKKVoSeYoC3qpCHkoLfLKrjwhhNiJBFbioLEvx850dFWPpbJUhHxkbZt1jUlM2+bIqiK2RNO8sb6Fj5rasR2HjKHz/rYEup4/ciaWNslYDroGpp0/+Di3HzbD5Zo20vzMnZjNmwDIbHoH/9Bju/UYLj2fmeo4S9DQwKNr+H0uXBocWR3m1CMrWbk1zootUbbEMlSGvHgMnZztEE/nsG2H4w8rKezEk115QgjRmQRW4qCwL8fOQL6r+vv1cWIpk02taZJZi9pYmtKAG5eRI5YyaUpmsR1Fkd+DbTtsaE6SNh0sR+U7o5Mv+E5knH1qlfBJr6t9xfNEX/0jysqhB4op/4+53QqqOnb86Rq4dB20/FE5GuBx6ziOoiTsY+LwMkqCXsYNilCfyOAoh3gqR0d+S9c1jj2shM+NLJeslBBC7IEEVuKgsC/HzgDUxtKs2daGz6VTHPDg0jXqE2naMhbb4lk0nO3dxCGTtWnP5kiZdj7rozovqfV0UGWn4rS88BvS698CwDdsPOXT52IES7r0/R3vglsHXdOwFFi2g6Pymasin4uQ14UDjD+shIElfgC8LoPjDythYMRPfVuaVM4m4DE4vCLE2EERqZ8SQoi9kMBKHBS6cuxMczLbqWmlUoqPmtqxbIfikLfwvV5XvpN4Q1sGA3AbOo5SpHJZ2jIWppMPTBx6PpjaUfMzt5PZ9A4YLkqmXkJo/DloWtd7Q2nkl/tCfhegkc7ZWI5C17TtbRN0DB2GlAY5fkgxGlqhXcKR1WGmjiov7IqU+ikhhOgaCazEQeGTjp1JmxaWrYinzEKQEEuZxFMmh5UGiaay+NwGPnf+cTY1p7BsG0fTqQi5yZg29YkM5va4bB839HVLySnfoPn531A+fS6eyuFd/j4X+fk5gN+tUxrwEPC4SJk2mZyNadsUB7xAPmt14ohSQj4PyazVqV2CrutSPyWEEN0kgZU4KOzt2JlYKstbG2N4DI1/rm/E6zIYWBygMuwh5zgcXhlkTZ1DQ1uGiN+Ng8rfHIVuKNwunYa2j4Oq/cVsrSW77YPCLj9P1QgGzP5Nt7JEhgaGroGjsLcv/bVlLDRNw7YVbpfBoNIAxw4upibiJ+A1SOVstsZS0i5BCCF6gARW4qCw47EzOzatbGxLs+SjKBqKMcPKqAr7CjsFN7XoZE2H8qDB2EFhNrak2NScpDGRpXR7vVU6Z9HSliWaNLt91ExXKaVIrnqJ1pfvR9k27rLBeKsPL7yurtIBl66h6xqaplA26JrOgIiPgSUB0PK1VsPKg0wfW8PIqiIAaZcghBA9SAIrcdCoDPuYOqri46aV7Vk2NCcp8hpMGFJaWNYKeA3KlZePmpLkLBuXoVEacIMC21HYjiLoM3AbHjJeF5ua2zHtfTsc+ZPYmXZaX/wdqff/CYD3sHEYweJ9eiwHsJXCBShNx60rIn4XJ44oZ2hFEW4j37x0U0uKrdE0I6uKpF2CEEL0MAmsxEGlMuwrNK1sTGSwHIeqkI8iX/7cung6x6aWNC3JLKmsRTSZw9uaJJOzKfZ7KA64iaTcmJaDaSuSWZOs6bA/VgEzW1bT/H+/xE40gW5QfNIswp/5Apq++wL8PdEBr1tHB9DAY+hYjqIk4KE06KEi7KMk8HHwtKcdkkIIIT49CazEQacjC5O1HFyGjt+T/2seT+dYtTVBMmficelkTIfWlEk6bpE1bQYWKxzA49LRdQjrOk3JDGo/pKrib/w3sUWPg3JwlQyg/Jz/wjvgiH16LJcOEa9BSchLOmvjcenkLIfioJtiv4eIv/NhyLvbISmEEKJndH3v9n5w6623csIJJxAKhaisrOTzn/88a9eu7TRGKcVNN91ETU0Nfr+fqVOn8u6773Yak81mufrqqykvLycYDHLuueeydevWTmOi0SizZs0iEokQiUSYNWsWsVis05jNmzdzzjnnEAwGKS8vZ86cOeRyuU5jVq1axZQpU/D7/QwcOJCbb74ZtT8+eUWXKKWIJnPUxzNEk7lOfxY77hRUKDa1pEnmTIq8LupjWVqTGXxunbKgh4oiLzYKjwGHVxQRcBlsaEmRypjk9sMfr+bygHIIjjmdARf/5hODqo4mn5D/odUAA/Aa+U7q7TmbVMbCZWhUhH0U+VygNEZUhPL/v4OMaeMxdLyuXv3xF0KIg1KvZqxee+01rrzySk444QQsy+KHP/whZ5xxBmvWrCEYDAJwxx13MG/ePB5++GGOOOIIfv7znzNt2jTWrl1LKBQCYO7cuTz77LPMnz+fsrIyrrnmGmbMmMGyZcswjPyyysyZM9m6dSsLFiwA4LLLLmPWrFk8++yzANi2zfTp06moqGDRokW0tLRw8cUXo5Ti7rvvBiCRSDBt2jROOeUUli5dygcffMDs2bMJBoNcc801B/rtO+R90rmAO+4ULFdeWpJZIn439YksadPEpeezWWnLIuT10JDIsnxLHJQiZzk0JLJYPRhUOZl2dF++YDx0wnm4K4d1qYO630Uhw6YUOIrC0qSmaei6QilFyrQZVOSjOuRBBzwug6MGFKHxcTF6R5+qERUhigPu3T6fEEKIfaepPpRuaWpqorKyktdee42TTz4ZpRQ1NTXMnTuX66+/Hshnp6qqqrj99tv51re+RTwep6KigkcffZSvfOUrANTV1TF48GCef/55zjzzTN577z1Gjx7NkiVLmDhxIgBLlixh8uTJvP/++4waNYoXXniBGTNmsGXLFmpqagCYP38+s2fPprGxkXA4zL333ssNN9xAQ0MDXm++D9Btt93G3XffzdatW7u0myqRSBCJRIjH44TD4f3xNh4S9nQuYGNbhojfUzgXsGPc1miaLdEkEb+bDxrasB0IegyKAx42tiSJp3LUx9P5B9cgl1OYPTRXJ5ui9aV7yW77gAEX/wbd88mtDDpySX63Rsjnoi1r49I1LFuRNvNH57h0KPK68Lg0TCt/KPRRA8JMHlFGRZGPWNrEUWrX9yfgYeoRu56bKIQQYs+6+vndp9YC4vE4AKWlpQBs2LCB+vp6zjjjjMIYr9fLlClTeOONNwBYtmwZpml2GlNTU8OYMWMKYxYvXkwkEikEVQCTJk0iEol0GjNmzJhCUAVw5plnks1mWbZsWWHMlClTCkFVx5j/396dx0lV3on+/5y19qreu2l2EVBE0AuJokZQE9GrGH8zN5vGkck2jluIOmb0zo2JScTExCQ3GY3JzdXrJDfOnYkxLkkEJ4o7OiCRRkVE9qbptZau7Zw65/n9UXSFotlpQLq/79erX1DnPFX1fLtb68uzfJ/29nY2bty4x5iKxSLpdLrqSxyeXQ9ObogGcDyfvOMRsnQaogG29GV5dX0vvu9Xdgqe2BTB9RTbUwUKrk/UNtA0jfZknu19ObYm8zheeeQnP4RJVbF9LdsfupHsmmcp9W2nsPnP+2xv6mAbGo1xi9qwScmHdLFc3NQteTiej66DZULAMgiaOomQTXNNiMnNMepjQS6c1sJfzRrDgpmtTGqMkS64bE3mSBdcJjXGJKkSQogj6AOzeF0pxU033cQ555zD9OnTAejo6ACgubm5qm1zczObNm2qtLFtm9ra2kFtBp7f0dFBU1PToPdsamqqarP7+9TW1mLbdlWbCRMmDHqfgXsTJ04c9B6LFy/mG9/4xv6/AeKA7X5wcskr7+BzPB9b11DAhu4caIozT6inKR7k0hmjQGm8ubUPr+TTm3XwlE+h5IMGXknhAZ7PkOwAVL5HevlvSL7wS1A+RryJhgW3EBwzba/P0TVojNqYern21LZUAU+B5/koDUpK4fnl5CsetGmOB6iN2JiajlKK0bUh+nIOrrfzfMRddkhKnSohhDg6PjCJ1fXXX8+bb77Jiy++OOje7h8ESqn9fjjs3mZP7YeizcBM6t76c9ttt3HTTTdVHqfTacaOHbvPvot92/3gZFfTaE9lSeUdaoI2ExoiZIou73VmKbh+ZVrwxOYIL73XSdv2FJlCCUPTcEo+mgZqZ/XPoUiqSukuup/8PsUtbQCETz6X+guvrayv2hsDcD0fHdiWKlDyfM48oQ7XVbzfk6UvWySPRyxo0RwLML4uDGik8i41YZugqe+sYPqX2X2pUyWEEEfXByKxuuGGG3j88cd5/vnnGTNmTOV6S0sLUB4NGjVqVOV6Z2dnZaSopaUFx3Ho6+urGrXq7OzkrLPOqrTZsWPHoPft6uqqep3ly5dX3e/r68N13ao2A6NXu74PDB5VGxAIBKqmDsXhUUqxvjNDzikRswP4vs/2VJ5MwSUeMCmUfDpSBRpiNhMbwnT3F2nbluYUpXhlfQ8d6QIaGpoCx/fxUGgKvCFcadj3p19Q3NKGZoeo+9g1RE45f7//EDC0chmErOOjbI2IodFcE2RUIkwkYNBcE2RdZ4b3dvRj6OXvQ87xcT2foGnQFLXpyTrURwM0xuT3TQghjpVjusZKKcX111/Po48+yp/+9KdBU2kTJ06kpaWFpUuXVq45jsOyZcsqSdOsWbOwLKuqzfbt22lra6u0mTNnDqlUitdee63SZvny5aRSqao2bW1tbN++vdJmyZIlBAIBZs2aVWnz/PPPV5VgWLJkCa2trYOmCEeCfZU6OFLW7ejn9Y19FF2PP29L8fy6bl7f1MuW3hzrOrNsS+Z4Z0caDY1o0CwXw+zL8er7vazdkcEydEKWQSRoErR0NMAf4nJOtR/9EqETZjNq4Y+ITr/ggKbeTA10XaGhqAvZNMRsokGrfO4fGnWRAKeMSjC6JkyxpOjJOvQXXBIhi5ZEgEzRpeTDrPG1MkIlhBDH0DHdFXjttdfyf//v/+V3v/sdU6dOrVxPJBKEQiEAvvOd77B48WIefPBBJk+ezF133cVzzz1XVW7h7//+73nyySd56KGHqKur45ZbbqGnp6eq3MLFF19Me3s7DzzwAFAutzB+/PiqcgunnXYazc3N3HPPPfT29rJw4UIuv/zySrmFVCrF1KlTOf/887n99ttZt24dCxcu5Gtf+9oBl1sYLrsC91fq4Ei95+9WtbN8Qw/RgM7qrRl6sgWKJUXQ0gnbJgXXw/cVsyfUc9H0ZqKB8i7AnqzDmm1JerJF2vsKuP5ffu0Pd7Sq2PEe+fWvU3P2Zw74ObueO6gD+s5F65ZpMKUxQr7kUx8NcmJThJBpwM7kLJ13WL6xFxRMrI8QDRmAjq5pTG2OsmBmqyxMF0KII+BAP7+P6VTg/fffD8C8efOqrj/44IMsXLgQgFtvvZV8Ps+1115LX18fZ5xxBkuWLKkkVQA/+MEPME2TT37yk+TzeS644AIeeuihSlIF8Ktf/Yobb7yxsnvwsssu4yc/+UnlvmEYPPXUU1x77bWcffbZhEIhrrjiCr73ve9V2iQSCZYuXcp1113H7Nmzqa2t5aabbqpaQzUS7K3UwfquDN39xcqapqE0sBMw55RoigbozhbRdQhaJuDh+YpiySceNPF8SBdcNvbkmNQYIeeUeKcjxfZUAc/3cTw1NAvUlU/m9cfoW/Yw+CXsxgmEp8zZ7/N0IGhpaGi4O4fLlF/OncKWRke6gG2adPUXUMonHioXMI0ETGxDZ1Q8yJTmGHURm7zrE7YNTmyKcurohCRVQghxjH2g6liNBMf7iJVSimff6WJ9V4YJ9ZGqaS6lFBt7skxqjHHeSY1DuvusL+vw5JvtxAIma9rTvLqhB8f1CNoG6bxHzilhaIpIyCJqmwQtg/qIzahEiKLrsXJzD50pl7TjUhiCqp+l/l56nvoBhY1vABCafCb1F9+IEdr/z1QDLEMjGjCwTZ2845FzPQytfE2h0RQPUBuyd46mKcKWyahEkB2ZIuPqwvzduRMJWKbs9hNCiKPkuBixEsefZM5lWzJHUyw46INc07QjdsBvseTjeD4h26QlEURHo1hSBGyIBQ1czyPneIS88rTajkyBHekivVmHvFNiR8YhWXBxh2CoKvfea/T8/of4+TSaGaD2gi8SnTn/gBIbSy+XVQhZOr5SuJ6i5EPEMogETOqjAQy9XPahv+hRFzHxlEZnf4FUwWXGmASfmD2Glprw4QcihBBiyEliJQ7KQIITtIw93h/KA36VUpUaTHmnhKVr5J0SQdOgPmqTc0u4rg+6hqlr6JpG3i3R1+3gOB6hgEHQ1OjMFEnnhyapSr7wK1Iv/xoAq2kijQtuxWrYd/mMgR0ilqGhUJQXo9ugaZQ8n4ht0pqw6cmWiAVNTEOnNRFiS18OpaA+YhMP2oQtnc98aBxTRx1/I51CCDFSSGIlDsquBxtHAoN/fYbqgN9Bi+N1nc29OXaki0QDJk7Jo1RSYELcNMjmy0fUmKZBrligpHwyeZ9c0aPkK5wh2vlnjzoRgNjsj1M79+ryYcr7oFFeOxW0dIKWgev5FFyP/qJHyDYI2QbN8SA5x8M0yqNYsaBFfcQmEjBI5lxOHZ0gGrRI5hwSYdnxJ4QQH2SSWImDsuvBxhPswWushuKA3z0tjt+RLrCxJ0dnukBLIsTYujCOp9jUm6O9r3zGX8DU6XcKOF55lCga0vFKHFYZCKUUpdQOrJpyTbXwiWcw6vP3YTeMO6Dna4Bt6NRGbGIBk56sg6Zp1EdtfAUhU8ctKSxDx9B9LN2gMRoATcM2DQyjRDhgYuoatnn4CasQQogjSxIrcVA0TWP66Djd/UU29mT3eMDv9NHxQ15IPbD7L5V3KovjlVJ09RepDVsETR1N09A0jZqwRXtSw/F8DF3D9Xxcr/w6PpDOH94wlZdL0fOHH1Hc+haj/vYnmPEGgL0mVdrOr4F3NQDTgIaoTTRgUfJ2LjQPWdSFbTylyBRKBFGMioeoj5aLlYbt8jSrW/IxdR1T14YkYRVCCHHkSWIlDtrAwcYDU3Xd2SK2oTOpMXbYdax2XxyvUHRmimztzREJGARMnWzRY0J9CNvQaU/myRZdoJxYDdUW1/yGN+h56l68bB8YJs72dyuJ1d4oyuf4mYCha0QDFkW3RKbgknU8dE0RsEzG14Y5a3IDs8bXsmZbhm2p8k7KkqdY056ms79IImDSl3eojwTp6i9Sc5gJqxBCiKNDEitxSI7UAb+7Lo5P5R029eTZ0pfl3c7Mzoqa5dKaxZJPV3+BZM5F1/XymqshKKGuPJfk8/9C+rVHAbDqx9Jw2T9gN51wQM/XNaiL2NRGbFJ5F103QUGx5OFroHuKoq+Y3BxjxthaWhIhnlvbRXd/kaZYkGmj4rzXlWFTbw7LKH9PTxyChFUIIcTRIYmVOGRH4oDfgcXxnZk86ztzZB0XnfJhyeW8SZWPeVGwta9AJu9g6BoF1+dwNyK6PVvpfuIenB3rAYiedjG1538e3TrwhCYeMKkJWYBGImRjaFBSiqBpUhu2qAlbpPIl/uPtTiY3RWlOhKpG/xzPZ0xtiOmtNZzQFGZ0TVhqVAkhxHFEEitxzO1aVsE2NEYlgix5qxPf92iOBdnYk8PQdQx2lifXynWdCk6Jku+jlI7nc9jTgJk3fo+zYz16MEb9xTceUBX1XZkaBG2LcMCkWFI0RG3iO9dTmYZO0NIBjbpwiS3JPMvf72PBacEjNvonhBDi6JPEShxTezpzEAWZvINtGaQKJTLFErVhi67+IiW3nHhs6Oon73p4qjx9OBRrq2rmXo0qOSTO/jRmbN/rqXZnG9ASCzB7Qg39js/2ZI7erMPYujDRYPWCc9s0CFsGW/qylUKqR2L0TwghxNEne7fFMdOZLvDs2k7atiXxlaImZBMLmnT1O5R8n7qQTdYpkc47oBS1IQsf6Mo49BdK+L7C8w59pKqwpY3uJ7+P8stbCXUrQP1F1x90UlUuqWAwriFCa22YsbUhfAXJvMuexpwczydkmfg714oJIYQYPmTEShwTSileeq+bFZuSmDpsS+UxdZ36SIC6iIWh65iGxuTGKB2pArmiS2fGIV1w8RV4AIpDOkxZeSVSL/2a1Cv/D1DYo6YQn7XgkGMxdUiELcbVRQCNoGVQG7Hp3nmcTjz0l5EopRSpgkttyCIRtKQulRBCDDOSWIljYt2Ofp5/twtfKRKxIJap45Z8tqdypPImDVGbdZ0ZmmIBSr5Pd9ahv1hCKUXRVYeUUAG4yY7yAvX2tQBEpn+U6PQLDiuWaMBgUmOUup1V0TUgHjSJB03a0wUSIRvbNHA8n1TBJWwZBC2DMbURqUslhBDDjCRW4qhTSvHm1iS9WYfx9WEU5TIFAcug2QyyI1MgYhuk8i69WYdiySNbKJFzymuqDlX/mmfpXXIfysmjBSLUz7+OyMnnHvTrlAs+lEUsjfpogDG1IXzAdT2S+fLRM2dNqscpKbYk84Qtg5BlUhuyyklVXVjqUgkhxDAkiZU46tbt6Gf5+z2kiiXe3ZFG13TCAYummE1d2MYyDN5uT5PMubieT7pYQik4jJNpSL7wS1IvPwJAYPQ0GhbcgploOqTXGuhG1NIZ3xDm1NE1aJpGT7aIqeu0xEPousbMMTVMGxXltQ1JtvRl8VEkghZjaiNSl0oIIYYpSazEUdWZLrDs3S56cw4xW6cjXcR1PRxf8b5lMromQN712ZEqoABLB0uDbMk75Ok/gPDUs0m//hjxM/6axJxPounGQT1fAwyt/Cc6NEUDXDazlfpoAB+IBkx0TcNXiv5iqVIpvSkeZMFpISmlIIQQI4QkVuKoGTgHMOeUsA2DDV39dPW7+CgMDdJ5h45UDk3TsAwd1/MxNHA8heMd5Hv5HsX2dwmOORkAu2kio6/5BUY4cVCvowMhS8OnXCYhYhucOjrO1WdNZM6kBroyxUq5iLznYRv6oErpUkpBCCFGDkmsxFEzcA5gImiyqbefnqxL2NLLa5M8n3xJ4fqgoTB1H88/tF1/pXQX3U9+n+K2t2n57D0ERk0BOKikytbBMnVsQyMeMMmVFM2xAFObo3xp7olMHRUHjtzRPkIIIY5PkliJo6ZY8imWPNr7CvQXPAKmgWlqmJpGv1KVxUsKKB1iJfXs2pfo/eOP8Qv9aHYIL9MNOxOrg6GAoGlQG7aIhizGhywawjbjGyMkwtWjTzIiJYQQYoAkVuKIGziyJpVzyBRKbOnNoWsaDVGbvOuRLrgkc27Vjr+DTap8p0Dfn35O/5+fBsAeNZmGBf+AVdt6yP0O2QY+GvGAxcT6COm8i6FpUntKCCHEXkliJYbUruf+BUwdp+Sxpj1TPrKm5LOlN8fm3iyu75Mp+mSdEv2F0mGVUXB2rKfr8Xso9W4FNOJn/jdqzrkSzTi0X29DK385JZ9RiSAnNEYwdY2c6zFWak8JIYTYB0msxJDZ/dy/ouvTlSkSC5pMaowStAycksfr7/fS3V8Aymcq+4d50F9hSxul3q0Y0XoaLr2Z4PgZh/Q6FmCa5bVRtqkTtg1OaIxi6hobenKMqw9zxgm1sn5KCCHEXkliJQ7LwAjVtmSO1zf04XoezfEQAUtn5aYkm3tzjKkN4Xo+vdkibe1p8qUSxYPc5ben9x1IcGKzLkO5RaKnXYQRih/S65k6hGwdzwcdjcZokICllacw8y7j6sJ8YtYYmhOhw+u4EEKIYU0SK3HIKiNUfTnWbE/Tl3OY0hSjLuKjgKxTYkJdiK6cwwvrumjvy5F1PExdx2DneX+HIPfea6Rf/TeaPnknuh1C0zQScz55yHHolBMrWzewbJ2QbRAO6GiaTmsixOnjazhjYp0kVUIIIfZLEitxSDrTBZ5b20Uq7xAJmGgaNMcCdGQKZAolxtaFyBRc+jxFX85hfVc/CmgIW+TcUvW5MAfId4skn3uQzMonAUgvf5Saj1x52LGELI2gbZIIWYyuCTGhPkxnv4OtG4yrD0lSJYQQ4oBJYiUO2kChz1TeYUJ9hGTexVeKaNAmBnT2F1nflWFrXx6n5FF0fTJ5l0jAZEfGIev4B71Y3enaSPfj9+B2bwIgNvvjhz1KFTA1DA1s2yAWMJncHKU+EiBf8mlJhJjeGqMv57KmPUNTPChrq4QQQuyXJFbioA2sqWqKBStV0k1NJ5138X3I5l3e7+knW/DIOh6+UvhKYRkanq/wDqLqp1KKzMon6Xv2f4PnoodraLjkK4ROmHVIfdcpL5ivC1ucMbGOrqxD3vFoTgSJhyzQNEYlwoyvD5EIlc8t3JbMkcy5UqtKCCHEfkliJQ5aseTjeD5Bq3zeXsnz6cwUWd/VT8n3Secd8q5P2DbQYWdiBclciYOd/0u98q+kXvglAMETZtHwXxdhRGoPue8hS0fXNOIhi7zrM721hmjAYFx9ZGfypxMJGGjlUwEJWgbd2SLF0uGcVCiEEGKkkMRKHLSAqWMbOgXXo+T7vLahl850HrfkkSmUyDkeJR+SpRKWDqaho2nlY2sOdgowNnM+/aueJv7h/4/YrAWHPR3XELOoCdqEAgYzx9Yw/5QWXtvYi6lrRAKD61MV3PL5f1IUVAghxIGQTwtx0GrCFqNrwuzIFHi/s59NvVlAQ0ej4JZQgKGXf7lKPpXRnpLa/3iVKrlk336h8tiI1DL6Sw8Qn33ZYSdV5cKfBrGQxaTGGPNPaaExFiBsm2zqyeKr6lEppRSdmQKja8JSFFQIIcQBkRGrEWr3CumJkEkqXzqgg4Q1TWP66Dht25K8sL6bVM7F8Xz68yV8H4K2hoZGf9EvH6KswDuA2gpuz1a6n7gHZ8d6ACInf6T8fubhr23SgaAB4+rCTG6JcVJzjK3JPK9t7KUnW2RTd46tyQKnjo7RFAtRcD06MwUSYZvpo+OycF0IIcQBkcRqBNpThfRiySNg6gQsA9vQGV0TZvroOE3x4F5fx1NQ8hWeryg4JVzPw1NQcBUKxYGuSlJK0f/mEvr+42cot4gejKFZgaEJdidDg5pIgBObozTHgyTzLr05h6ZYkKZYkLpwgDXtKVZuSjKhwaUuYjOpMbbf74EQQgixK0msRphd6081xYIUSx7v7UjSmSnSFAtw+thaApbO+q4M3f1F5k1tHJRYDJRb0FA0RCw6UwUyBY/Sznm+g9n15+Uz9P7xx+TefRmA4PgZ1F9yE2asYahCxgCiQZMZY2s4c2I9mWKJ7v4iE+ojlZGo1poQLfEAb3dkGFMT5ryTGqmN2DJSJYQQ4qBIYjWC7F5/Cg3Wd2Up+T4nNUfpzDpsSeaYMTrBhPoIG3uytG1Lc14sUJVg9GXLldTf3p7i7Y4M6XzpYGt9AlDYuobux+/By3SDblBz7lXEP/xXaNrQLf2zNBjfEGbe1CY+OXscDVGbp1Zvr5SK2JWu60yoj5AuuGiaJkmVEEKIg3ZMF68///zzLFiwgNbWVjRN47HHHqu6r5Ti61//Oq2trYRCIebNm8eaNWuq2hSLRW644QYaGhqIRCJcdtllbN26tapNX18fV111FYlEgkQiwVVXXUUymaxqs3nzZhYsWEAkEqGhoYEbb7wRx3Gq2qxevZq5c+cSCoUYPXo0d955J0od5gnCR9Hu9aeyRY+ebJGakI2m6ySCFr1Zh2zRQ9M0mmLBSg2nATtSeX62bD1P/rmd1VuT9BcOLakC8Is5vEw3Zu0oWj77PRJn/LfDTqo0IKBDxNYYXxvi9HE1XDN3El86dxJTWmI4nqoqFbG7oGWUp0elvIIQQohDcExHrLLZLDNnzuRv//Zv+eu//utB97/73e9y77338tBDDzFlyhS+9a1v8bGPfYy1a9cSi8UAWLRoEU888QSPPPII9fX13HzzzVx66aWsWLECwyh/eF5xxRVs3bqVP/7xjwB86Utf4qqrruKJJ54AwPM8LrnkEhobG3nxxRfp6enh6quvRinFj3/8YwDS6TQf+9jHOO+883j99dd59913WbhwIZFIhJtvvvlofLsO2+71p1zPp+T7WGZ5x5tt6GQKLq5fTip2r+H0VnuKh17cwLPvdpHMOwzkHgdzOo3yXDSj/H7hSR+iYcE/EDrxw+j24R8ZE7V0prTEmNwcZXxdmLBtUfQ8zj+pmbpoec3WrqUiIoHBv/5SXkEIIcTh0NQHZMhF0zR++9vfcvnllwPl0arW1lYWLVrEV7/6VaA8OtXc3Mx3vvMd/u7v/o5UKkVjYyP/8i//wqc+9SkA2tvbGTt2LL///e+ZP38+b7/9NtOmTePVV1/ljDPOAODVV19lzpw5vPPOO0ydOpU//OEPXHrppWzZsoXW1lYAHnnkERYuXEhnZyfxeJz777+f2267jR07dhAIlD+k7777bn784x+zdevWA542SqfTJBIJUqkU8Xh8KL+F+9WXdXjyzXbiQYtIwKS/WOL1jb2ELYOAZVBwPfKux4fG1xENmmSLJdIFl0tntOKUPO5d+i6vrO+lJ1vAcf0DKp+wq/41z5J8/mFarvwOZrxpSGOrj5hMrI8wvj5Ka02IcXVhknmHSY0xzjupsfLzUUrx7DtdrO/KVK2xGri3sSc76DlCCCHEgX5+f2D/Wb5hwwY6Ojq48MILK9cCgQBz587l5ZfLC51XrFiB67pVbVpbW5k+fXqlzSuvvEIikagkVQBnnnkmiUSiqs306dMrSRXA/PnzKRaLrFixotJm7ty5laRqoE17ezsbN27caxzFYpF0Ol31dawM1J/qzBRQShEJGNRHAiTzDjnHpSNVIGwbhAN6VQ2nRMjk1fd7eGt7Gtf3QIFhDtQm3z+/mKP7ye/T8+T38dJdpF//3ZDFpANhS6cuEuTE5jiJkMn6rgx/XLOdvOtxSmusKkEaKBWRCNls7MmSLZbwfEW2WGJjT1bKKwghhDgsH9jEqqOjA4Dm5uaq683NzZV7HR0d2LZNbW3tPts0NQ0eHWlqaqpqs/v71NbWYtv2PtsMPB5osyeLFy+urO1KJBKMHTt234EfQbsnFbmiRyJosCNd5KX3emhP5enKFPnPjUlWb0thGjpNcZtNPTne3ZEhky9hGxqGqYGv0A8g9yhue4ftD95Ads2zoOkkzrmS2vM+d9ixGICpQTRgMLkpyocn1OKUPNZ19dOZKdKTddjUk6NtW5rOdKHquU3xIPOmNjKpMUa64LI1mSNdcJnUGGPelMG7IIUQQogD9YHfFbj7yIFSar+jCbu32VP7oWgzMIu6r/7cdttt3HTTTZXH6XT6mCZXA0lF27Y073SkeWt7Gl2DifVhQpYFKN7dkcbUdaa0RFm+oZeC4/HO9gyFUol40ARfUfT2PQ2ofI/0q/9O8sVfgfIxEs00XHoLwTEnH1b/DQ2iAR2lwFcwvj7CR6Y0Mr4uzIrNSRIhmwn1FpahkcmXaGtP0ZN1BpWNaIoHOS8WqCqSuq+iqEIIIcSB+MAmVi0tLUB5NGjUqFGV652dnZWRopaWFhzHoa+vr2rUqrOzk7POOqvSZseOHYNev6urq+p1li9fXnW/r68P13Wr2uw+MtXZ2QkMHlXbVSAQqJo+PNJ2r6i+p2ShKR5kXtSmv1Ai75Y4oSFKxDbIOT7d/QXatqVoT+XpTBc5oSFCcOdi70LJx8m4lPaTVAFkVj5J8oV/ASB88rnUz78OPRA57PgMDSxdw/HBNnUmNkRoiJbrT+WdEuPrQmiaju8rsrrHqESQVN7ZY9kITdOojRx+VXchhBBiwAd2KnDixIm0tLSwdOnSyjXHcVi2bFklaZo1axaWZVW12b59O21tbZU2c+bMIZVK8dprr1XaLF++nFQqVdWmra2N7du3V9osWbKEQCDArFmzKm2ef/75qhIMS5YsobW1lQkTJgz9N+AQdKYLPPtOF0++2c5Tq9t58s12nn2na9BUGEAqXyJVcDixMUYsaKHrOq7n8eqGXt7cmqYzXWT5hl5++8Y21nf3M64+hFfySOXdAypFEDvtYgKtJ1F/yVdoWPAPQ5JUAZimhm4YhGyDsbVhYiGTtm0pVm1J0tXvsKk3T84p4Xg+pqFjG8Yey0YIIYQQR8IxHbHq7+/nvffeqzzesGEDq1atoq6ujnHjxrFo0SLuuusuJk+ezOTJk7nrrrsIh8NcccUVACQSCT7/+c9z8803U19fT11dHbfccgunnnoqH/3oRwE4+eSTueiii/jiF7/IAw88AJTLLVx66aVMnToVgAsvvJBp06Zx1VVXcc8999Db28stt9zCF7/4xcrK/yuuuIJvfOMbLFy4kNtvv51169Zx11138bWvfe0DMX20e0X14M5dfnuroL576YVkzuHFdT28uyODqevEQxYFxyPneKzc1EfJB8dT7O3IP98pkHnjKeIfuhxNN9BMm+bP3jMk3xsNsI3y4vtTWhNkHY/meICgadCRKtIUtQmZBmHboC9bJFcoEQ2ajG+IEAkY+IqqshFCCCHEkXJME6v//M//5Lzzzqs8HliLdPXVV/PQQw9x6623ks/nufbaa+nr6+OMM85gyZIllRpWAD/4wQ8wTZNPfvKT5PN5LrjgAh566KFKDSuAX/3qV9x4442V3YOXXXYZP/nJTyr3DcPgqaee4tprr+Xss88mFApxxRVX8L3vfa/SJpFIsHTpUq677jpmz55NbW0tN910U9X6qWNl94rqA8lMJGAywd5zBfVd6zmFbYONPVna03kMNBIhE6U0LFMRC1ps6smSLrh7nf4rdrxH9xP3UOrdBl6JxFnl0hdDkVQZGliGRtgyiIcDgMa0lgTjG8L0Zh1Ctkky56I0ha8gYpt0pAtousa4ujCaplFwSlKbSgghxFHxgaljNVIciTpWu9en2t2u9agG1hTtWs+pIRrgpfe62dSbI5N3CVoGxZJHNGChULQn8xRcn1TOrRqxUson/dpjJJ9/GPwSRrSehktvJjh+xpDEZWkQsHSCO+tszRhTw0WnNHNSS5zXNvYSD1q4ns+mnhxvbkuSzLkkQhYhSycSsDh3SiORnUmj1KYSQghxOA708/sDu3hdHLjdp/V2t3sFdfhL6YXu/iLvd/fTXyhh6hpBSyeZc4iHLEK2wY5MoXxuHopdJ9JK/b30PPUDChvfACA85SzqLroeIzQ0yaKhlQ9OrglZWJbO2Jownz97IrMm1LIjXazEGwmYJEIWDVGbtm1pir5Hfdim3ymRyjt0ZXypTSWEEOKokcRqGDjUY1oGSi+8ut5gbUc/uZ1TZvGQTdAqt/V8n5LnU/IUmgZKQX7Tn+n+3Xfw82k0M0DtBV8kOnP+kCUulq4RMHUU4CkYHQ9yyYxRzJpQi6Zpg+LVNI3RtWEiAZNNvTk6knn6iyUKrs/kphjTR8elNpUQQoijQhKrYSARMkkEbdZ1pTmhIUp0Z7IBVCqoT2qMURO2Bj23KR7k0pktKHyeebsTTdOoC1t0Zx16+4u4JXBLPpoB+s7Eyggn8J08VtMJNF72D1j1B16XS6dcqmHgT5/qswZNHZpiFoauUywpWmuCzJvcyDmT/zKNN1BBfn1Xhgl2ZJfrNvGgialrjKkJc95JjdRGbBmpEkIIcdRIYnWc60wXaNuWZmsyx4buHO/u6Gd8XZgTG2MELJ3OTGG/U2G6rjNnUgNdGYdVW/roy7s0RctJSqrgkswqitkUyooCYDdOoPnT3yLQMgXNHJys7Y0GGDqV4p6aBoaC+qhF1vEwdZ1wwEDXdTRNY1SNxaUzWrnwlJaqEaddpzE39mSrdkF2ZgqMqgnxkSkNlYOXhRBCiKNFEqvj2K4lFkbXhGiI2rzXmWVzb5b2VIFpo2Kc1JLY41TYQCHRgutRcD2ClsG5UxqpjVis3NTHpt4cecfD83x6Xn+c9v94iFGf+hZWa7lERXDMKQfV14GUTtc0bEuj6Pq4PtiGhmloTGuNc+7kRmIBi5xToi/nMn10nAUzW9H1wbv5dq0gvy2ZoztbxDZ0JjXK1J8QQohjRxKr49SeSiyEAwYnj9JpTQTZlswzuibEvKkNVYmJUop1O/pZvS3Fhu4sXZkCWccjGjAYVxemORZkakuM2rDNirUbef3n36D37VcByLY9Q23r1P1WXR9gUi7o6fuKkgJdB18pPF/DMjTiIZPmeJDGWIDzpjbRkghRcD1c32dyS4w5kxr2mFQNkGNphBBCfNBIYnWcSuZctiVzNMWCaJpGKu+wqSdPT7ZIyffxfHh9Yx8nj0owpaVc96szXeDFdd28sK6LnmyRXNEjYOmMSoQpeYp3d/SzclMfAdPA2/JnHrv7VgrpHjTDYuJ//Tsaz7yMZNal4O0/tQro5SNnfKAxGiDvlCh4Po7ro2vQkghx9okNzBxTi6ZBf7HE1mTuoEed5FgaIYQQHySSWB2ndi2xkMo7rN6aJuu41IRsLNOi6Hps6s2x7N2uyqL1Z9/pZOXmPnwgalsU3XIBhb6sw9jaIOu6sxTyRd564gHeWfprAIKN45nw375KfPTk8q7AneNVuy44H6Dzl8XotREbXdPwlKK1NkRj1Kar36U5HuCCkxqZPb6OcMCq9E1GnYQQQgwHklgdpwZKDuTdEpt68mQdl+ZYsLwinPJITlMsSM4psXprCjToSBcwdJ14UGdzLkciZGMaGqmCy7Z0kaJT4p2Xl1SSqpYzFzBm/pfwDJuCW0LTqNS0gr8kVxp/WZge0MrJlWHo1IUtmmJBzphYS8bxmDUhwH+d3kJzIjQoHhl1EkIIMRxIYnWcGig50LYtSXd/gZqQXUmqlFKkCi4t8SDj68K815UBNGpCFu3JPJZh4CmFaWjltVmWSTrvkiqUaDj9fCZtXkXLzHMJnnAG6YJLxDJI5x3yjo9tajieQqly+QVD07BNHQ2F65eTK01RTvocj2TeZVNvnlnj6zhncoMsKhdCCDGsSWJ1nBooObC+q5/OTJFxdQa+r3A8n1TBJWKZjK8LE7JNco4HaDREA5iGjq8UhqZR8hReMcPr//5TJlz0eQpu+fiYcz53Bz4+rqdIF1zyjott6jglhalroP5SiyoaNNA0DbfkY2jltV2JsM0po2JEghZj60L4Pri+HIAshBBi+JPE6jjWFA8yd0ojG7v76c4UUZSnCEclQoyvD1MTtskWS4RtA9AwNKiL2HSk80QDJmvfWM6rD95JtncHmf5+xl7+FQw0MsUium5g6BpBy6A36+ErH/CJBAJYhkFfzkHXyiNelq7hG1AslResT2uJMbEpxvi6ch+UUns8CFoIIYQYbiSxOs4lQib1UZv2VIGIXf5xDiwwH6i6fmJjDDR4v6ufcbVhkpk8z/3b/fzn4/8blCLaNIZJc/+KcMii4Pr09Jeoi2jEgwHG14cxdY2+vEPJ19A1nZNHRQiYGtuSRZJ5B8/zMTSdxpjFeVMbmTW+nkjAqCRQA+u9tiVzJHOurKcSQggxbElidRzrTBdY9m43mqbTEg/ieB5h26AjVaCn36ExFmBMXZhTxyQA6Ol3WLf+PX79zZtY/9YqAMadeQnT//oGJo1uxNQ0tibzKKUI2SbFkoehaZw8Kkax5OH6GomgycSGMBMbotiGxntdWbYmc4Rtg3jQ4pTWGgx98IjUng6CFkIIIYYbSayOU7sWCJ3eGiddF6rUsdJ16Ms5NMUDzN1lwbi+vY1vf+GT5LL9hKIxPvHlOzl3/mWc0BBmakuc9mSBXy7fiO8rEiEbTddQviLvlmiMh5jUGCZT8BhTEyZVcHA8n9qIzfTRNYypDbF8Q89BHwQthBBCDCeSWB2ndi8QmgjZnDrGIlv0cD0f1/PxPIVtGpXnnHvmLGLRCCefcipf+Kd70eINKBRbkwUUOk1xm4kNUdySKk/9lcrn97XWRBhfHyIasHC8HB+aWEvQMqrqTgFs7csPOhgZ9n8QtBBCCDFcSGJ1nNq1QOgADY3oztEiz1dsTeZ45911zDm9fK5fU1MTv/39UjYWo/S7ftXhxeu7Mmzq0bEMnfH1QSCK6/lYhl5eL4VGtljCNso7B/e0TmpfByPv7yBoIYQQYjiQeZnj1ECB0ILr7fF+tuDwh4f/mY98aCaPPPIIUB45yoea6Xd9JtRHiARMDF0jEjCZUB/B9TyKJY/OTJGIbVAbtokGTDS0yqjT6JrwXkedBg5GntQYI11w2ZrMkS64TGqMMW9Ko9SwEkIIMezJiNVxaqBA6J6m3na0b+Ebt1zLuj+/DsCyZcv49Kc/PWj6cFeaptEcD7EtmcPS9QMedVJKVR1H0xgLcN5JjXJEjRBCiBFJEqvj1ECB0N2n3v70h9/xkztvJdefJhKNcv999/HZz34W2PP04a6ClkHAMvjQxDo600W2JXN0Z4t7PRi5M12gbVuabckcjudjGzqja8IHfICyEEIIMdxIYnUcG5h6a9uWZn17Fw//4E5efOr/AXD6rNn8278+wqRJkyrtd50+3HXnnlKKbNEjlXcoeT6tiSCntMb3OerUmS7w3NouUnln0Fqt7v4i86bK1J8QQoiRR9ZYHeea4kHOO6mRxvwWXnzq/6FpGrfddhvLX3m5KqmCv0wfdmYKKFUuIprMOby5LcVrG3t54b1utvTmeWNLkq5MkdqITUsiSG3EHjT9N1DqYU9rtVJ5h7Zt6cp7CCGEECOFjFgNA5qmcfmlF/Htb3+bs846i3nz5u213a7ThwHT4L3OflIFF5SitSbE5KYI73f109Pv7HXUaX9rtaTKuhBCiJFKEqth5Pbbb99vm4Hpw9VbUzy7tpOOdIHGqE00YNNSEyAWtGmO62zqye31bL8DWaslVdaFEEKMRJJYjUBN8SCnj9N4r6uflkSQVN4lVyzx7g4HU89SHwnQGLP2Ouq0t7VaA6TKuhBCiJFKPvlGKMdT5TIK6SLJrIOuQcA00IHtyRzrdmTpzTp7HHXa01qtAQdS70oIIYQYrmTEaoSyDY2uTJHOTLmcQmd/Ec9XGLpGLGCRKhTwlcI2Btef2lupB6myLoQQYqSTxGoEy7slOjMFwrZBxLawLA3XV/TmiuQcj1hwz2uooLrUw/7qXQkhhBAjhSRWI1Sx5FPyFIamoSnQNFBa+U9NgaFplDy1zwXoTfEg58UCUmVdCCGE2EkSq2Fm9yNm9pboFFwPT8HEhgh516e/4OKpcqJVEwkQsnQKJX+vZxEO0DRNSioIIYQQO0liNYwczBEzQcsgGjBxPY8JdSEKXrCyxipoaHRkCkQD5l5LKgghhBBiMNkVOEwMHDGzvitDPGgxpiZMPGixvivDc2u76EwXqtoHLYNxdWFsw6Czv4gOhO3yrsDO/iK2Wb4viZUQQghx4GTEahjY/YiZgam/SMBkgh1hY092ULHPmrDFSS1xiiUf31f05hxKRRdT12mJh9B1jZNa4lIyQQghhDgIklgNA4dyxMyuJROSeYdRNUF0TcNXiv5iiRopmSCEEEIcNJkKPAT33XcfEydOJBgMMmvWLF544YVj2p8DOWLG8fxBO/wGSiac2BjD8xVZp4TnK05sjDFvyp7PCRRCCCHE3smI1UH613/9VxYtWsR9993H2WefzQMPPMDFF1/MW2+9xbhx445Jnw7niBkpmSCEEEIMHRmxOkj33nsvn//85/nCF77AySefzA9/+EPGjh3L/ffff8z6dLhHzAyUTGhJBKmN2JJUCSGEEIdIEquD4DgOK1as4MILL6y6fuGFF/Lyyy/v8TnFYpF0Ol31NdQG1kslQjYbe7Jki+UpvWyxxMaerBwxI4QQQhwlklgdhO7ubjzPo7m5uep6c3MzHR0de3zO4sWLSSQSla+xY8cekb4NrJea1BgjXXDZmsyRLrhMkvVSQgghxFEja6wOwe4jP0qpvY4G3Xbbbdx0002Vx+l0+ogmV7JeSgghhDh2JLE6CA0NDRiGMWh0qrOzc9Ao1oBAIEAgEDga3QPkiBkhhBDiWJKpwINg2zazZs1i6dKlVdeXLl3KWWeddYx6JYQQQogPChmxOkg33XQTV111FbNnz2bOnDn87Gc/Y/PmzVxzzTXHumtCCCGEOMYksTpIn/rUp+jp6eHOO+9k+/btTJ8+nd///veMHz/+WHdNCCGEEMeYpnYvfCSOqHQ6TSKRIJVKEY/Hj3V3hBBCCHEADvTzW9ZYCSGEEEIMEUmshBBCCCGGiCRWQgghhBBDRBIrIYQQQoghIomVEEIIIcQQkXILR9nAJswjcRizEEIIIY6Mgc/t/RVTkMTqKOvp6QE4YucFCiGEEOLIyWQyJBKJvd6XxOooq6urA2Dz5s37/MEMRwMHUG/ZsmVE1fAaqXGDxD4SYx+pcYPEPtxjV0qRyWRobW3dZztJrI4yXS8va0skEsP2l29/4vH4iIx9pMYNEvtIjH2kxg0S+3CO/UAGRGTxuhBCCCHEEJHESgghhBBiiEhidZQFAgHuuOMOAoHAse7KUTdSYx+pcYPEPhJjH6lxg8Q+UmPfnRzCLIQQQggxRGTESgghhBBiiEhiJYQQQggxRCSxEkIIIYQYIpJYCSGEEEIMEUmsjrL77ruPiRMnEgwGmTVrFi+88MKx7lLF888/z4IFC2htbUXTNB577LGq+0opvv71r9Pa2kooFGLevHmsWbOmqk2xWOSGG26goaGBSCTCZZddxtatW6va9PX1cdVVV5FIJEgkElx11VUkk8mqNps3b2bBggVEIhEaGhq48cYbcRynqs3q1auZO3cuoVCI0aNHc+edd+73DKc9Wbx4MR/60IeIxWI0NTVx+eWXs3bt2hER+/3338+MGTMqRf3mzJnDH/7wh2Ef9+4WL16MpmksWrRo2Mf+9a9/HU3Tqr5aWlqGfdwA27Zt47Of/Sz19fWEw2FOO+00VqxYMexjnzBhwqCfuaZpXHfddcM67mNGiaPmkUceUZZlqZ///OfqrbfeUl/+8pdVJBJRmzZtOtZdU0op9fvf/1799//+39VvfvMbBajf/va3VffvvvtuFYvF1G9+8xu1evVq9alPfUqNGjVKpdPpSptrrrlGjR49Wi1dulStXLlSnXfeeWrmzJmqVCpV2lx00UVq+vTp6uWXX1Yvv/yymj59urr00ksr90ulkpo+fbo677zz1MqVK9XSpUtVa2uruv766yttUqmUam5uVp/+9KfV6tWr1W9+8xsVi8XU9773vYOOe/78+erBBx9UbW1tatWqVeqSSy5R48aNU/39/cM+9scff1w99dRTau3atWrt2rXq9ttvV5Zlqba2tmEd965ee+01NWHCBDVjxgz15S9/uXJ9uMZ+xx13qFNOOUVt37698tXZ2Tns4+7t7VXjx49XCxcuVMuXL1cbNmxQzzzzjHrvvfeGfeydnZ1VP++lS5cqQD377LPDOu5jRRKro+jDH/6wuuaaa6qunXTSSeof//Efj1GP9m73xMr3fdXS0qLuvvvuyrVCoaASiYT66U9/qpRSKplMKsuy1COPPFJps23bNqXruvrjH/+olFLqrbfeUoB69dVXK21eeeUVBah33nlHKVVO8HRdV9u2bau0+fWvf60CgYBKpVJKKaXuu+8+lUgkVKFQqLRZvHixam1tVb7vH1bsnZ2dClDLli0bcbErpVRtba36X//rf42IuDOZjJo8ebJaunSpmjt3biWxGs6x33HHHWrmzJl7vDec4/7qV7+qzjnnnL3eH86x7+7LX/6ymjRpkvJ9f0TFfbTIVOBR4jgOK1as4MILL6y6fuGFF/Lyyy8fo14duA0bNtDR0VHV/0AgwNy5cyv9X7FiBa7rVrVpbW1l+vTplTavvPIKiUSCM844o9LmzDPPJJFIVLWZPn161UGX8+fPp1gsVobtX3nlFebOnVtVjG7+/Pm0t7ezcePGw4o1lUoBfzkwe6TE7nkejzzyCNlsljlz5oyIuK+77jouueQSPvrRj1ZdH+6xr1u3jtbWViZOnMinP/1p3n///WEf9+OPP87s2bP5xCc+QVNTE6effjo///nPK/eHc+y7chyHX/7yl3zuc59D07QRE/fRJInVUdLd3Y3neTQ3N1ddb25upqOj4xj16sAN9HFf/e/o6MC2bWpra/fZpqmpadDrNzU1VbXZ/X1qa2uxbXufbQYeH873UynFTTfdxDnnnMP06dOrXm+4xr569Wqi0SiBQIBrrrmG3/72t0ybNm3Yx/3II4+wcuVKFi9ePOjecI79jDPO4OGHH+bpp5/m5z//OR0dHZx11ln09PQM67jff/997r//fiZPnszTTz/NNddcw4033sjDDz9c9XrDMfZdPfbYYySTSRYuXFj1WsM97qPJPNYdGGk0Tat6rJQadO2D7FD6v3ubPbUfijZq5+LGw/l+Xn/99bz55pu8+OKLg+4N19inTp3KqlWrSCaT/OY3v+Hqq69m2bJl+3yv4z3uLVu28OUvf5klS5YQDAb32m44xn7xxRdX/n7qqacyZ84cJk2axP/5P/+HM888c6/vdbzH7fs+s2fP5q677gLg9NNPZ82aNdx///38zd/8zT7f73iPfVe/+MUvuPjii6tGjfb2XsMp7qNJRqyOkoaGBgzDGJRxd3Z2DsrOP4gGdg3tq/8tLS04jkNfX98+2+zYsWPQ63d1dVW12f19+vr6cF13n206OzuBwf/yOlA33HADjz/+OM8++yxjxoypXB/usdu2zYknnsjs2bNZvHgxM2fO5Ec/+tGwjnvFihV0dnYya9YsTNPENE2WLVvG//yf/xPTNPf6L+ThEPvuIpEIp556KuvWrRvWP/NRo0Yxbdq0qmsnn3wymzdvrrwXDM/YB2zatIlnnnmGL3zhC5VrIyHuo00Sq6PEtm1mzZrF0qVLq64vXbqUs8466xj16sBNnDiRlpaWqv47jsOyZcsq/Z81axaWZVW12b59O21tbZU2c+bMIZVK8dprr1XaLF++nFQqVdWmra2N7du3V9osWbKEQCDArFmzKm2ef/75qm26S5YsobW1lQkTJhxUbEoprr/+eh599FH+9Kc/MXHixBET+96+H8VicVjHfcEFF7B69WpWrVpV+Zo9ezZXXnklq1at4oQTThi2se+uWCzy9ttvM2rUqGH9Mz/77LMHlVF59913GT9+PDAy/jt/8MEHaWpq4pJLLqlcGwlxH3VHcGG82M1AuYVf/OIX6q233lKLFi1SkUhEbdy48Vh3TSlV3iH1xhtvqDfeeEMB6t5771VvvPFGpRzE3XffrRKJhHr00UfV6tWr1Wc+85k9bskdM2aMeuaZZ9TKlSvV+eefv8ctuTNmzFCvvPKKeuWVV9Spp566xy25F1xwgVq5cqV65pln1JgxY6q25CaTSdXc3Kw+85nPqNWrV6tHH31UxePxQ9qS+/d///cqkUio5557rmpLci6Xq7QZrrHfdttt6vnnn1cbNmxQb775prr99tuVrutqyZIlwzruPdl1V+Bwjv3mm29Wzz33nHr//ffVq6++qi699FIVi8Uq/x8arnG/9tpryjRN9e1vf1utW7dO/epXv1LhcFj98pe/rLQZrrErpZTneWrcuHHqq1/96qB7wznuY0ESq6Psn//5n9X48eOVbdvqv/yX/1LZ0v9B8Oyzzypg0NfVV1+tlCpvR77jjjtUS0uLCgQC6txzz1WrV6+ueo18Pq+uv/56VVdXp0KhkLr00kvV5s2bq9r09PSoK6+8UsViMRWLxdSVV16p+vr6qtps2rRJXXLJJSoUCqm6ujp1/fXXV22/VUqpN998U33kIx9RgUBAtbS0qK9//euHtB13TzED6sEHH6y0Ga6xf+5zn6v8PjY2NqoLLrigklQN57j3ZPfEarjGPlCjyLIs1draqv7qr/5KrVmzZtjHrZRSTzzxhJo+fboKBALqpJNOUj/72c+q7g/n2J9++mkFqLVr1w66N5zjPhY0pY6ncqZCCCGEEB9cssZKCCGEEGKISGIlhBBCCDFEJLESQgghhBgiklgJIYQQQgwRSayEEEIIIYaIJFZCCCGEEENEEishhBBCiCEiiZUQQgghxBCRxEoIIT4AnnvuOTRNI5lMHuuuCCEOg1ReF0IMG/PmzeO0007jhz/84bHuykFzHIfe3l6am5vRNO1Yd0cIcYhkxEoIMaIopSiVSse6G1Vc18W2bVpaWiSpEuI4J4mVEGJYWLhwIcuWLeNHP/oRmqahaRobN26sTLE9/fTTzJ49m0AgwAsvvMDChQu5/PLLq15j0aJFzJs3r/JYKcV3v/tdTjjhBEKhEDNnzuTf//3f99mPCRMm8M1vfpMrrriCaDRKa2srP/7xj6vaaJrGT3/6Uz7+8Y8TiUT41re+tcepwJdeeom5c+cSDoepra1l/vz59PX1HXLfhBBHniRWQohh4Uc/+hFz5szhi1/8Itu3b2f79u2MHTu2cv/WW29l8eLFvP3228yYMeOAXvOf/umfePDBB7n//vtZs2YNX/nKV/jsZz/LsmXL9vm8e+65hxkzZrBy5Upuu+02vvKVr7B06dKqNnfccQcf//jHWb16NZ/73OcGvcaqVau44IILOOWUU3jllVd48cUXWbBgAZ7nHVbfhBBHlnmsOyCEEEMhkUhg2zbhcJiWlpZB9++8804+9rGPHfDrZbNZ7r33Xv70pz8xZ84cAE444QRefPFFHnjgAebOnbvX55599tn84z/+IwBTpkzhpZde4gc/+EHV+19xxRVVCdWGDRuqXuO73/0us2fP5r777qtcO+WUUw67b0KII0sSKyHEiDB79uyDav/WW29RKBQGJWOO43D66afv87kDyc6uj3dfUL+//qxatYpPfOITQ943IcSRJYmVEGJEiEQiVY91XWf3TdGu61b+7vs+AE899RSjR4+uahcIBA76/XdflL57f3YXCoX2em+o+yaEGDqSWAkhhg3btitrkPansbGRtra2qmurVq3CsiwApk2bRiAQYPPmzQc9tfbqq68OenzSSScd1GvMmDGD//iP/+Ab3/jGoHuH0zchxJEliZUQYtiYMGECy5cvZ+PGjUSjUerq6vba9vzzz+eee+7h4YcfZs6cOfzyl7+kra2tMpUWi8W45ZZb+MpXvoLv+5xzzjmk02lefvllotEoV1999V5f+6WXXuK73/0ul19+OUuXLuXf/u3feOqppw4qlttuu41TTz2Va6+9lmuuuQbbtnn22Wf5xCc+QUNDwyH3TQhxZMmuQCHEsHHLLbdgGAbTpk2jsbGRzZs377Xt/Pnz+R//439w66238qEPfYhMJsPf/M3fVLX55je/yde+9jUWL17MySefzPz583niiSeYOHHiPvtx8803s2LFCk4//XS++c1v8v3vf5/58+cfVCxTpkxhyZIl/PnPf+bDH/4wc+bM4Xe/+x2maR5W34QQR5ZUXhdCiCE0YcIEFi1axKJFi451V4QQx4CMWAkhhBBCDBFJrIQQQgghhohMBQohhBBCDBEZsRJCCCGEGCKSWAkhhBBCDBFJrIQQQgghhogkVkIIIYQQQ0QSKyGEEEKIISKJlRBCCCHEEJHESgghhBBiiEhiJYQQQggxRP5/THBMicklQJIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_train, ridge_tuned.predict(X_train), alpha=0.3)\n",
    "grid = np.linspace(y_train.min(), y_train.max(), 1000)\n",
    "plt.plot(grid, grid, \"--k\")\n",
    "plt.xlabel(\"true price\")\n",
    "plt.ylabel(\"predicted price\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- Here we can see a few cases where our prediction is way off.\n",
    "- Is there something weird about those houses, perhaps? Outliers? \n",
    "- Under the line means we're under-predicting, over the line means we're over-predicting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### $R^2$ (not in detail)\n",
    "\n",
    "A common score is the $R^2$\n",
    "\n",
    "- This is the score that `sklearn` uses by default when you call score()\n",
    "- You can [read about it](https://en.wikipedia.org/wiki/Coefficient_of_determination) if interested.\n",
    "- $R^2$ measures the proportion of variability in $y$ that can be explained using $X$. \n",
    "- Independent of the scale of $y$. So the max is 1.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$R^2(y, \\hat{y}) = 1 - \\frac{\\sum_{i=1}^n (y_i - \\hat{y_i})^2}{\\sum_{i=1}^n (y_i - \\bar{y})^2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The denominator measures the total variance in $y$.  \n",
    "- The amount of variability that is left unexplained after performing regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Key points:\n",
    "- The maximum is 1 for perfect predictions\n",
    "- Negative values are very bad: \"worse than DummyRegressor\" (very bad)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "(Optional) Warning: MSE is \"reversible\" but $R^2$ is not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.801775Z",
     "iopub.status.busy": "2024-05-29T14:52:45.801040Z",
     "iopub.status.idle": "2024-05-29T14:52:45.830614Z",
     "shell.execute_reply": "2024-05-29T14:52:45.827224Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.801711Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "872961060.7986538"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_squared_error(y_train, preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.836676Z",
     "iopub.status.busy": "2024-05-29T14:52:45.835620Z",
     "iopub.status.idle": "2024-05-29T14:52:45.862582Z",
     "shell.execute_reply": "2024-05-29T14:52:45.859114Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.836612Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "872961060.7986538"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_squared_error(preds, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.866977Z",
     "iopub.status.busy": "2024-05-29T14:52:45.866248Z",
     "iopub.status.idle": "2024-05-29T14:52:45.888439Z",
     "shell.execute_reply": "2024-05-29T14:52:45.886109Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.866923Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8601643854446083"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(y_train, preds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.894253Z",
     "iopub.status.busy": "2024-05-29T14:52:45.892287Z",
     "iopub.status.idle": "2024-05-29T14:52:45.917479Z",
     "shell.execute_reply": "2024-05-29T14:52:45.914250Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.894169Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8280229354283185"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(preds, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- When you call `fit` it minimizes MSE / maximizes $R^2$ (or something like that) by default.\n",
    "- Just like in classification, this isn't always what you want!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### MAPE\n",
    "\n",
    "- We got an RMSE of ~$30,000 before. \n",
    "\n",
    "Question: Is an error of \\$30,000 **acceptable**?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:45.922968Z",
     "iopub.status.busy": "2024-05-29T14:52:45.920977Z",
     "iopub.status.idle": "2024-05-29T14:52:46.065596Z",
     "shell.execute_reply": "2024-05-29T14:52:46.054050Z",
     "shell.execute_reply.started": "2024-05-29T14:52:45.922898Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29545.914451894256"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sqrt(mean_squared_error(y_train, ridge_tuned.predict(X_train)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- For a house worth \\$600k, it seems reasonable! That's 5% error.\n",
    "- For a house worth \\$60k, that is terrible. It's 50% error.\n",
    "\n",
    "We have both of these cases in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:46.085536Z",
     "iopub.status.busy": "2024-05-29T14:52:46.078490Z",
     "iopub.status.idle": "2024-05-29T14:52:46.906687Z",
     "shell.execute_reply": "2024-05-29T14:52:46.904233Z",
     "shell.execute_reply.started": "2024-05-29T14:52:46.085378Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmxUlEQVR4nO3df3TU1Z3/8ddgYExwMisqM5kSSLSxKgGl4EaiNrGauDSw9bDrVqEWl+0eKKBk2V0gZrcET5kgdnPSNpoeWA8bD5vSPwSXXVpN2GrYbqBEfqwxuEgPIaTImK0bJlFoUsn9/sHy+TIkIBOSm8zk+Tjnc07nfu7MvN8Byat37mc+LmOMEQAAgCWjhroAAAAwshA+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFiVMNQFXKqnp0cffvihPB6PXC7XUJcDAACugjFGnZ2dCgQCGjXqymsbwy58fPjhh0pNTR3qMgAAQD+0trZqwoQJV5wz7MKHx+ORdL745OTkIa4GAABcjY6ODqWmpjq/x69k2IWPCx+1JCcnEz4AAIgxV7Nlgg2nAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwKmGoC8DgSVu9M+Lx8fUFQ1QJAAD/HysfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKuiCh+fffaZ/u7v/k7p6elKTEzUrbfequeff149PT3OHGOMSkpKFAgElJiYqNzcXDU1NQ144RgYaat3RhwAAAy2qMLHCy+8oB//+MeqqKjQ+++/rw0bNujFF1/Uj370I2fOhg0bVFZWpoqKCjU0NMjv9ysvL0+dnZ0DXjwAAIg9UYWPPXv26Otf/7oKCgqUlpamP/3TP1V+fr7eeecdSedXPcrLy1VcXKy5c+cqMzNTVVVVOnPmjKqrqwelAQAAEFuiCh8PPPCA/v3f/10ffPCBJOm//uu/9Mtf/lJf+9rXJEnNzc0KhULKz893nuN2u5WTk6P6+vo+X7Orq0sdHR0RBwAAiF9R3Vhu1apVCofDuuOOO3Tdddfp3LlzWrdunZ588klJUigUkiT5fL6I5/l8PrW0tPT5mqWlpVq7dm1/agcAADEoqpWPn/70p9qyZYuqq6t14MABVVVV6fvf/76qqqoi5rlcrojHxpheYxcUFRUpHA47R2tra5QtAACAWBLVysff/u3favXq1XriiSckSVOmTFFLS4tKS0u1YMEC+f1+SedXQFJSUpzntbW19VoNucDtdsvtdve3fgAAEGOiWvk4c+aMRo2KfMp1113nXGqbnp4uv9+v2tpa53x3d7fq6uqUnZ09AOUCAIBYF9XKx5w5c7Ru3TpNnDhRkydP1sGDB1VWVqaFCxdKOv9xS2FhoYLBoDIyMpSRkaFgMKikpCTNmzdvUBrA1eN7PAAAw0FU4eNHP/qR/v7v/15LlixRW1ubAoGAFi1apO9+97vOnJUrV+rs2bNasmSJ2tvblZWVpZqaGnk8ngEvHgAAxB6XMcYMdREX6+jokNfrVTgcVnJy8lCXE9P6s9JxfH3BIFQCAIh30fz+5t4uAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKyKKnykpaXJ5XL1OpYuXSpJMsaopKREgUBAiYmJys3NVVNT06AUDgAAYlNU4aOhoUGnTp1yjtraWknS448/LknasGGDysrKVFFRoYaGBvn9fuXl5amzs3PgKwcAADEpqvBxyy23yO/3O8e//du/6bbbblNOTo6MMSovL1dxcbHmzp2rzMxMVVVV6cyZM6qurh6s+gEAQIzp956P7u5ubdmyRQsXLpTL5VJzc7NCoZDy8/OdOW63Wzk5Oaqvr7/s63R1damjoyPiAAAA8avf4eP111/X6dOn9fTTT0uSQqGQJMnn80XM8/l8zrm+lJaWyuv1Okdqamp/SwIAADGg3+HjlVde0axZsxQIBCLGXS5XxGNjTK+xixUVFSkcDjtHa2trf0sCAAAxIKE/T2ppadGuXbu0bds2Z8zv90s6vwKSkpLijLe1tfVaDbmY2+2W2+3uTxkAACAG9WvlY/PmzRo/frwKCgqcsfT0dPn9fucKGOn8vpC6ujplZ2dfe6UAACAuRL3y0dPTo82bN2vBggVKSPj/T3e5XCosLFQwGFRGRoYyMjIUDAaVlJSkefPmDWjRAAAgdkUdPnbt2qUTJ05o4cKFvc6tXLlSZ8+e1ZIlS9Te3q6srCzV1NTI4/EMSLEAACD2uYwxZqiLuFhHR4e8Xq/C4bCSk5OHupyYlrZ6Z9TPOb6+4PMnAQBwiWh+f3NvFwAAYBXhAwAAWNWvS20x/PTnIxYAAIYCKx8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsShroAxIe01TsjHh9fXzBElQAAhjtWPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFZFHT5Onjypb37zm7rpppuUlJSke+65R/v373fOG2NUUlKiQCCgxMRE5ebmqqmpaUCLxvCXtnpnrwMAACnK8NHe3q77779fo0eP1s9//nMdPnxY//AP/6A/+IM/cOZs2LBBZWVlqqioUENDg/x+v/Ly8tTZ2TnQtQMAgBiUEM3kF154Qampqdq8ebMzlpaW5vxvY4zKy8tVXFysuXPnSpKqqqrk8/lUXV2tRYsWDUzVAAAgZkW18rFjxw7NmDFDjz/+uMaPH69p06Zp06ZNzvnm5maFQiHl5+c7Y263Wzk5Oaqvr+/zNbu6utTR0RFxAACA+BXVysexY8dUWVmpFStW6LnnntO+ffv07LPPyu1261vf+pZCoZAkyefzRTzP5/OppaWlz9csLS3V2rVr+1n+yMUeCgBArIpq5aOnp0df/vKXFQwGNW3aNC1atEh/+Zd/qcrKyoh5Lpcr4rExptfYBUVFRQqHw87R2toaZQsAACCWRBU+UlJSdNddd0WM3XnnnTpx4oQkye/3S5KzAnJBW1tbr9WQC9xut5KTkyMOAAAQv6IKH/fff7+OHDkSMfbBBx9o0qRJkqT09HT5/X7V1tY657u7u1VXV6fs7OwBKBcAAMS6qPZ8/NVf/ZWys7MVDAb1Z3/2Z9q3b582btyojRs3Sjr/cUthYaGCwaAyMjKUkZGhYDCopKQkzZs3b1AaAAAAsSWq8HHvvfdq+/btKioq0vPPP6/09HSVl5dr/vz5zpyVK1fq7NmzWrJkidrb25WVlaWamhp5PJ4BLx4AAMSeqMKHJM2ePVuzZ8++7HmXy6WSkhKVlJRcS10AACBOcW8XAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFgV9aW2GHkuvYnd8fUFQ1QJACAesPIBAACsInwAAACrCB8AAMAq9nwgwqX7OwAAGGisfAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArEoY6gLw+dJW7xzqEgAAGDCsfAAAAKsIHwAAwCrCBwAAsCqq8FFSUiKXyxVx+P1+57wxRiUlJQoEAkpMTFRubq6ampoGvGjEprTVOyMOAMDIFPXKx+TJk3Xq1CnnaGxsdM5t2LBBZWVlqqioUENDg/x+v/Ly8tTZ2TmgRQMAgNgVdfhISEiQ3+93jltuuUXS+VWP8vJyFRcXa+7cucrMzFRVVZXOnDmj6urqAS8cAADEpqjDx9GjRxUIBJSenq4nnnhCx44dkyQ1NzcrFAopPz/fmet2u5WTk6P6+vrLvl5XV5c6OjoiDgAAEL+i+p6PrKwsvfrqq7r99tv10Ucf6Xvf+56ys7PV1NSkUCgkSfL5fBHP8fl8amlpuexrlpaWau3atf0oHUOF/RoAgGsR1crHrFmz9Cd/8ieaMmWKHnnkEe3cef6XUFVVlTPH5XJFPMcY02vsYkVFRQqHw87R2toaTUkAACDGXNOltmPHjtWUKVN09OhR56qXCysgF7S1tfVaDbmY2+1WcnJyxAEAAOLXNYWPrq4uvf/++0pJSVF6err8fr9qa2ud893d3aqrq1N2dvY1FwoAAOJDVHs+/uZv/kZz5szRxIkT1dbWpu9973vq6OjQggUL5HK5VFhYqGAwqIyMDGVkZCgYDCopKUnz5s0brPoBAECMiSp8/OY3v9GTTz6p3/72t7rlllt03333ae/evZo0aZIkaeXKlTp79qyWLFmi9vZ2ZWVlqaamRh6PZ1CKBwAAscdljDFDXcTFOjo65PV6FQ6H2f/xf+L16pLj6wuGugQAwACJ5vc393YBAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYNU1hY/S0lK5XC4VFhY6Y8YYlZSUKBAIKDExUbm5uWpqarrWOhGH0lbv7HUAAOJfv8NHQ0ODNm7cqKlTp0aMb9iwQWVlZaqoqFBDQ4P8fr/y8vLU2dl5zcUCAIDY16/w8cknn2j+/PnatGmTbrzxRmfcGKPy8nIVFxdr7ty5yszMVFVVlc6cOaPq6uoBKxoAAMSufoWPpUuXqqCgQI888kjEeHNzs0KhkPLz850xt9utnJwc1dfXX1ulAAAgLiRE+4StW7fqwIEDamho6HUuFApJknw+X8S4z+dTS0tLn6/X1dWlrq4u53FHR0e0JQEAgBgS1cpHa2urli9fri1btuj666+/7DyXyxXx2BjTa+yC0tJSeb1e50hNTY2mJAAAEGOiCh/79+9XW1ubpk+froSEBCUkJKiurk4//OEPlZCQ4Kx4XFgBuaCtra3XasgFRUVFCofDztHa2trPVgAAQCyI6mOXhx9+WI2NjRFjf/7nf6477rhDq1at0q233iq/36/a2lpNmzZNktTd3a26ujq98MILfb6m2+2W2+3uZ/kAACDWRBU+PB6PMjMzI8bGjh2rm266yRkvLCxUMBhURkaGMjIyFAwGlZSUpHnz5g1c1QAAIGZFveH086xcuVJnz57VkiVL1N7erqysLNXU1Mjj8Qz0WwEAgBjkMsaYoS7iYh0dHfJ6vQqHw0pOTh7qcoaFkfTNn8fXFwx1CQCAfojm9zf3dgEAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYlRDN5MrKSlVWVur48eOSpMmTJ+u73/2uZs2aJUkyxmjt2rXauHGj2tvblZWVpZdeekmTJ08e8MJjQdrqnb3Gjq8v6NfzRopLe7+an9dgvg4AYOBFtfIxYcIErV+/Xu+8847eeecdffWrX9XXv/51NTU1SZI2bNigsrIyVVRUqKGhQX6/X3l5eers7ByU4gEAQOyJKnzMmTNHX/va13T77bfr9ttv17p163TDDTdo7969MsaovLxcxcXFmjt3rjIzM1VVVaUzZ86ourp6sOoHAAAxpt97Ps6dO6etW7fq008/1cyZM9Xc3KxQKKT8/HxnjtvtVk5Ojurr6y/7Ol1dXero6Ig4AABA/Io6fDQ2NuqGG26Q2+3W4sWLtX37dt11110KhUKSJJ/PFzHf5/M55/pSWloqr9frHKmpqdGWBAAAYkjU4eNLX/qSDh06pL179+o73/mOFixYoMOHDzvnXS5XxHxjTK+xixUVFSkcDjtHa2trtCUBAIAYEtXVLpI0ZswYffGLX5QkzZgxQw0NDfrBD36gVatWSZJCoZBSUlKc+W1tbb1WQy7mdrvldrujLQMAAMSoa/6eD2OMurq6lJ6eLr/fr9raWudcd3e36urqlJ2dfa1vAwAA4kRUKx/PPfecZs2apdTUVHV2dmrr1q16++239cYbb8jlcqmwsFDBYFAZGRnKyMhQMBhUUlKS5s2bN1j1AwCAGBNV+Pjoo4/01FNP6dSpU/J6vZo6dareeOMN5eXlSZJWrlyps2fPasmSJc6XjNXU1Mjj8QxK8QAAIPZEFT5eeeWVK553uVwqKSlRSUnJtdQEAADiGPd2AQAAVhE+AACAVVFfagsMNyP5RnwAEItY+QAAAFYRPgAAgFWEDwAAYBV7PjCssZ8DAOIPKx8AAMAqwgcAALCK8AEAAKxiz4dll+5hOL6+YIgqAQBgaLDyAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwKmGoCxjp0lbvHOoSRoS+fs7H1xcMQSUAAFY+AACAVYQPAABgFeEDAABYFVX4KC0t1b333iuPx6Px48frscce05EjRyLmGGNUUlKiQCCgxMRE5ebmqqmpaUCLBgAAsSuq8FFXV6elS5dq7969qq2t1Weffab8/Hx9+umnzpwNGzaorKxMFRUVamhokN/vV15enjo7Owe8eAAAEHuiutrljTfeiHi8efNmjR8/Xvv379dXvvIVGWNUXl6u4uJizZ07V5JUVVUln8+n6upqLVq0aOAqBwAAMema9nyEw2FJ0rhx4yRJzc3NCoVCys/Pd+a43W7l5OSovr6+z9fo6upSR0dHxAEAAOJXv8OHMUYrVqzQAw88oMzMTElSKBSSJPl8voi5Pp/POXep0tJSeb1e50hNTe1vSQAAIAb0O3wsW7ZM7777rn7yk5/0OudyuSIeG2N6jV1QVFSkcDjsHK2trf0tCQAAxIB+fcPpM888ox07dmj37t2aMGGCM+73+yWdXwFJSUlxxtva2nqthlzgdrvldrv7UwYAAIhBUa18GGO0bNkybdu2Tb/4xS+Unp4ecT49PV1+v1+1tbXOWHd3t+rq6pSdnT0wFQMAgJgW1crH0qVLVV1drX/5l3+Rx+Nx9nF4vV4lJibK5XKpsLBQwWBQGRkZysjIUDAYVFJSkubNmzcoDQAAgNgSVfiorKyUJOXm5kaMb968WU8//bQkaeXKlTp79qyWLFmi9vZ2ZWVlqaamRh6PZ0AKBgAAsS2q8GGM+dw5LpdLJSUlKikp6W9NAAAgjnFvFwAAYBXhAwAAWNWvS22BeJC2emfE4+PrC6J+ztU+DwDw/7HyAQAArCJ8AAAAqwgfAADAKvZ8AFfQ1x4PAMC1YeUDAABYRfgAAABWET4AAIBV7PkA/g/7OwDADlY+AACAVYQPAABgFeEDAABYRfgAAABWseEUuEb9uUEdAIxkrHwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKxKiPYJu3fv1osvvqj9+/fr1KlT2r59ux577DHnvDFGa9eu1caNG9Xe3q6srCy99NJLmjx58kDWDQxbaat3fu6c4+sLLFQCAMNT1Csfn376qe6++25VVFT0eX7Dhg0qKytTRUWFGhoa5Pf7lZeXp87OzmsuFgAAxL6oVz5mzZqlWbNm9XnOGKPy8nIVFxdr7ty5kqSqqir5fD5VV1dr0aJF11YtAACIeQO656O5uVmhUEj5+fnOmNvtVk5Ojurr6/t8TldXlzo6OiIOAAAQv6Je+biSUCgkSfL5fBHjPp9PLS0tfT6ntLRUa9euHcgygLh16X4S9o4AiEWDcrWLy+WKeGyM6TV2QVFRkcLhsHO0trYORkkAAGCYGNCVD7/fL+n8CkhKSooz3tbW1ms15AK32y232z2QZQAAgGFsQFc+0tPT5ff7VVtb64x1d3errq5O2dnZA/lWAAAgRkW98vHJJ5/o17/+tfO4ublZhw4d0rhx4zRx4kQVFhYqGAwqIyNDGRkZCgaDSkpK0rx58wa0cCDeXc33hfQ1h30gAIa7qMPHO++8o4ceesh5vGLFCknSggUL9E//9E9auXKlzp49qyVLljhfMlZTUyOPxzNwVQMAgJgVdfjIzc2VMeay510ul0pKSlRSUnItdQEAgDjFvV0AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYNWAfr36SHc1XwoFXA5/fwCMFKx8AAAAqwgfAADAKsIHAACwij0fV4nP44HeLv3vgpvaAbgarHwAAACrCB8AAMAqwgcAALBqxO356GvvBp9Twzabe4hs/p3nvy8AV4OVDwAAYBXhAwAAWEX4AAAAVo24PR9Xg+/0QCy7mr+/Nr+fg+8CAXApVj4AAIBVhA8AAGAV4QMAAFhF+AAAAFax4RTAVRmojdgD+UVkbGYFYhMrHwAAwCrCBwAAsIrwAQAArGLPB4A+DfXN7y41UPtCrua1R9IN8tg3g6HAygcAALCK8AEAAKwifAAAAKsGbc/Hyy+/rBdffFGnTp3S5MmTVV5ergcffHCw3g7ANYiFmyn254Z5tl3N/onB2mMRL/tUBnP/z0gRCz/DQVn5+OlPf6rCwkIVFxfr4MGDevDBBzVr1iydOHFiMN4OAADEkEEJH2VlZfqLv/gLffvb39add96p8vJypaamqrKycjDeDgAAxJAB/9ilu7tb+/fv1+rVqyPG8/PzVV9f32t+V1eXurq6nMfhcFiS1NHRMdClSZJ6us70Grv0vfqaAyB+Xc2/AVfzb9Klz+vrOVczpz8Gs2abrubf36Gucbgbqp/hhdc0xnz+ZDPATp48aSSZ//zP/4wYX7dunbn99tt7zV+zZo2RxMHBwcHBwREHR2tr6+dmhUHbcOpyuSIeG2N6jUlSUVGRVqxY4Tzu6elRS0uL7rnnHrW2tio5OXmwShyWOjo6lJqaOuJ6H6l9S/Q+EnsfqX1LI7f3kdC3MUadnZ0KBAKfO3fAw8fNN9+s6667TqFQKGK8ra1NPp+v13y32y232x0xNmrU+a0oycnJcfuH9HlGau8jtW+J3kdi7yO1b2nk9h7vfXu93quaN+AbTseMGaPp06ertrY2Yry2tlbZ2dkD/XYAACDGDMrHLitWrNBTTz2lGTNmaObMmdq4caNOnDihxYsXD8bbAQCAGDIo4eMb3/iGPv74Yz3//PM6deqUMjMz9bOf/UyTJk26que73W6tWbOm18cxI8FI7X2k9i3R+0jsfaT2LY3c3kdq35fjMuZqrokBAAAYGNzbBQAAWEX4AAAAVhE+AACAVYQPAABg1bAMHy+//LLS09N1/fXXa/r06fqP//iPoS7JsXv3bs2ZM0eBQEAul0uvv/56xHljjEpKShQIBJSYmKjc3Fw1NTVFzOnq6tIzzzyjm2++WWPHjtUf//Ef6ze/+U3EnPb2dj311FPyer3yer166qmndPr06Yg5J06c0Jw5czR27FjdfPPNevbZZ9Xd3R0xp7GxUTk5OUpMTNQXvvAFPf/881f3vfuXKC0t1b333iuPx6Px48frscce05EjR+K+98rKSk2dOtX5YqCZM2fq5z//eVz3fDmlpaVyuVwqLCyM+/5LSkrkcrkiDr/fH/d9S9LJkyf1zW9+UzfddJOSkpJ0zz33aP/+/XHfe1paWq8/c5fLpaVLl8Z130Omn7dwGTRbt241o0ePNps2bTKHDx82y5cvN2PHjjUtLS1DXZoxxpif/exnpri42Lz22mtGktm+fXvE+fXr1xuPx2Nee+0109jYaL7xjW+YlJQU09HR4cxZvHix+cIXvmBqa2vNgQMHzEMPPWTuvvtu89lnnzlz/uiP/shkZmaa+vp6U19fbzIzM83s2bOd85999pnJzMw0Dz30kDlw4ICpra01gUDALFu2zJkTDoeNz+czTzzxhGlsbDSvvfaa8Xg85vvf/37UfT/66KNm8+bN5r333jOHDh0yBQUFZuLEieaTTz6J69537Nhhdu7caY4cOWKOHDlinnvuOTN69Gjz3nvvxW3Pfdm3b59JS0szU6dONcuXL3fG47X/NWvWmMmTJ5tTp045R1tbW9z3/b//+79m0qRJ5umnnza/+tWvTHNzs9m1a5f59a9/Hfe9t7W1Rfx519bWGknmrbfeiuu+h8qwCx9/+Id/aBYvXhwxdscdd5jVq1cPUUWXd2n46OnpMX6/36xfv94Z+93vfme8Xq/58Y9/bIwx5vTp02b06NFm69atzpyTJ0+aUaNGmTfeeMMYY8zhw4eNJLN3715nzp49e4wk89///d/GmPMhaNSoUebkyZPOnJ/85CfG7XabcDhsjDHm5ZdfNl6v1/zud79z5pSWlppAIGB6enquqfe2tjYjydTV1Y243m+88Ubzj//4jyOm587OTpORkWFqa2tNTk6OEz7iuf81a9aYu+++u89z8dz3qlWrzAMPPHDZ8/Hc+6WWL19ubrvtNtPT0zOi+rZlWH3s0t3drf379ys/Pz9iPD8/X/X19UNU1dVrbm5WKBSKqN/tdisnJ8epf//+/fr9738fMScQCCgzM9OZs2fPHnm9XmVlZTlz7rvvPnm93og5mZmZETfwefTRR9XV1eUske7Zs0c5OTkRX2rz6KOP6sMPP9Tx48evqddwOCxJGjdu3Ijp/dy5c9q6das+/fRTzZw5c0T0LElLly5VQUGBHnnkkYjxeO//6NGjCgQCSk9P1xNPPKFjx47Ffd87duzQjBkz9Pjjj2v8+PGaNm2aNm3a5JyP594v1t3drS1btmjhwoVyuVwjpm+bhlX4+O1vf6tz5871ugGdz+frdaO64ehCjVeqPxQKacyYMbrxxhuvOGf8+PG9Xn/8+PERcy59nxtvvFFjxoy54pwLj6/l52mM0YoVK/TAAw8oMzMz4vXisffGxkbdcMMNcrvdWrx4sbZv36677rorrnu+YOvWrTpw4IBKS0t7nYvn/rOysvTqq6/qzTff1KZNmxQKhZSdna2PP/44rvs+duyYKisrlZGRoTfffFOLFy/Ws88+q1dffTXi9eKx94u9/vrrOn36tJ5++umI14r3vm0alK9Xv1YulyvisTGm19hw1p/6L53T1/yBmGP+b0PStfw8ly1bpnfffVe//OUve52Lx96/9KUv6dChQzp9+rRee+01LViwQHV1dVd8n1jvWZJaW1u1fPly1dTU6Prrr7/svHjsf9asWc7/njJlimbOnKnbbrtNVVVVuu+++y77XrHed09Pj2bMmKFgMChJmjZtmpqamlRZWalvfetbV3y/WO/9Yq+88opmzZrV69bw8d63TcNq5ePmm2/Wdddd1yu5tbW19Up5w9GF3fBXqt/v96u7u1vt7e1XnPPRRx/1ev3/+Z//iZhz6fu0t7fr97///RXntLW1Seqd4K/WM888ox07duitt97ShAkTnPF47n3MmDH64he/qBkzZqi0tFR33323fvCDH8R1z9L5ZeS2tjZNnz5dCQkJSkhIUF1dnX74wx8qISHhsv9PK176v9jYsWM1ZcoUHT16NK7/3FNSUnTXXXdFjN155506ceKE815SfPZ+QUtLi3bt2qVvf/vbzthI6Nu2YRU+xowZo+nTp6u2tjZivLa2VtnZ2UNU1dVLT0+X3++PqL+7u1t1dXVO/dOnT9fo0aMj5pw6dUrvvfeeM2fmzJkKh8Pat2+fM+dXv/qVwuFwxJz33ntPp06dcubU1NTI7XZr+vTpzpzdu3dHXKJVU1OjQCCgtLS0qHozxmjZsmXatm2bfvGLXyg9PX3E9N7Xz6Krqyvue3744YfV2NioQ4cOOceMGTM0f/58HTp0SLfeemtc93+xrq4uvf/++0pJSYnrP/f777+/1yX0H3zwgXNT0Hju/YLNmzdr/PjxKigocMZGQt/WDeJm1n65cKntK6+8Yg4fPmwKCwvN2LFjzfHjx4e6NGPM+Z3/Bw8eNAcPHjSSTFlZmTl48KBzKfD69euN1+s127ZtM42NjebJJ5/s83KsCRMmmF27dpkDBw6Yr371q31ejjV16lSzZ88es2fPHjNlypQ+L8d6+OGHzYEDB8yuXbvMhAkTIi7HOn36tPH5fObJJ580jY2NZtu2bSY5Oblfl2N95zvfMV6v17z99tsRl6OdOXPGmROPvRcVFZndu3eb5uZm8+6775rnnnvOjBo1ytTU1MRtz1dy8dUu8dz/X//1X5u3337bHDt2zOzdu9fMnj3beDwe59+heO173759JiEhwaxbt84cPXrU/PM//7NJSkoyW7ZscebEa+/GGHPu3DkzceJEs2rVql7n4rnvoTDswocxxrz00ktm0qRJZsyYMebLX/6ycznncPDWW28ZSb2OBQsWGGPOX4q2Zs0a4/f7jdvtNl/5yldMY2NjxGucPXvWLFu2zIwbN84kJiaa2bNnmxMnTkTM+fjjj838+fONx+MxHo/HzJ8/37S3t0fMaWlpMQUFBSYxMdGMGzfOLFu2LOLSK2OMeffdd82DDz5o3G638fv9pqSkpF+XYvXVsySzefNmZ0489r5w4ULn7+Itt9xiHn74YSd4xGvPV3Jp+IjX/i98h8Po0aNNIBAwc+fONU1NTXHftzHG/Ou//qvJzMw0brfb3HHHHWbjxo0R5+O59zfffNNIMkeOHOl1Lp77HgouY2LpK9EAAECsG1Z7PgAAQPwjfAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALDq/wH3ipdEUuUoVQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(y_train, bins=100);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "How about looking at percent error? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:46.911246Z",
     "iopub.status.busy": "2024-05-29T14:52:46.909410Z",
     "iopub.status.idle": "2024-05-29T14:52:47.053682Z",
     "shell.execute_reply": "2024-05-29T14:52:47.048505Z",
     "shell.execute_reply.started": "2024-05-29T14:52:46.911171Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302      3.839225\n",
       "767     11.558889\n",
       "429      8.560768\n",
       "1139   -16.405650\n",
       "558     17.194710\n",
       "          ...    \n",
       "1041    -0.496213\n",
       "1122   -28.624615\n",
       "1346    -6.541117\n",
       "1406    -2.327283\n",
       "1389     3.376089\n",
       "Name: SalePrice, Length: 1314, dtype: float64"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred_train = ridge_tuned.predict(X_train)\n",
    "percent_errors = (pred_train - y_train) / y_train * 100.0\n",
    "percent_errors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are both positive (predict too high) and negative (predict too low)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can look at the absolute percent error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.057476Z",
     "iopub.status.busy": "2024-05-29T14:52:47.056283Z",
     "iopub.status.idle": "2024-05-29T14:52:47.113372Z",
     "shell.execute_reply": "2024-05-29T14:52:47.105057Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.057415Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "302      3.839225\n",
       "767     11.558889\n",
       "429      8.560768\n",
       "1139    16.405650\n",
       "558     17.194710\n",
       "          ...    \n",
       "1041     0.496213\n",
       "1122    28.624615\n",
       "1346     6.541117\n",
       "1406     2.327283\n",
       "1389     3.376089\n",
       "Name: SalePrice, Length: 1314, dtype: float64"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.abs(percent_errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "And, like MSE, we can take the average over examples. This is called mean absolute percent error (MAPE)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.149809Z",
     "iopub.status.busy": "2024-05-29T14:52:47.137517Z",
     "iopub.status.idle": "2024-05-29T14:52:47.170464Z",
     "shell.execute_reply": "2024-05-29T14:52:47.167091Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.149694Z"
    }
   },
   "outputs": [],
   "source": [
    "def my_mape(true, pred):\n",
    "    return np.mean(np.abs((pred - true) / true))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.176083Z",
     "iopub.status.busy": "2024-05-29T14:52:47.174929Z",
     "iopub.status.idle": "2024-05-29T14:52:47.217932Z",
     "shell.execute_reply": "2024-05-29T14:52:47.214429Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.176021Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10092665203438769"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_mape(y_train, pred_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use `sklearn` to calculate MAPE. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.245695Z",
     "iopub.status.busy": "2024-05-29T14:52:47.244644Z",
     "iopub.status.idle": "2024-05-29T14:52:47.255342Z",
     "shell.execute_reply": "2024-05-29T14:52:47.251665Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.245656Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_absolute_percentage_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.261219Z",
     "iopub.status.busy": "2024-05-29T14:52:47.259596Z",
     "iopub.status.idle": "2024-05-29T14:52:47.286818Z",
     "shell.execute_reply": "2024-05-29T14:52:47.283410Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.261147Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10092665203438769"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_percentage_error(y_train, pred_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Ok, this is quite interpretable.\n",
    "- On average, we have around 10% error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Transforming the targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- When you have prices or count data, the target values are skewed. \n",
    "- Let's look at our target column. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:47.291856Z",
     "iopub.status.busy": "2024-05-29T14:52:47.291161Z",
     "iopub.status.idle": "2024-05-29T14:52:48.218248Z",
     "shell.execute_reply": "2024-05-29T14:52:48.216092Z",
     "shell.execute_reply.started": "2024-05-29T14:52:47.291800Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmxUlEQVR4nO3df3TU1Z3/8ddgYExwMisqM5kSSLSxKgGl4EaiNrGauDSw9bDrVqEWl+0eKKBk2V0gZrcET5kgdnPSNpoeWA8bD5vSPwSXXVpN2GrYbqBEfqwxuEgPIaTImK0bJlFoUsn9/sHy+TIkIBOSm8zk+Tjnc07nfu7MvN8Byat37mc+LmOMEQAAgCWjhroAAAAwshA+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFiVMNQFXKqnp0cffvihPB6PXC7XUJcDAACugjFGnZ2dCgQCGjXqymsbwy58fPjhh0pNTR3qMgAAQD+0trZqwoQJV5wz7MKHx+ORdL745OTkIa4GAABcjY6ODqWmpjq/x69k2IWPCx+1JCcnEz4AAIgxV7Nlgg2nAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwKmGoC8DgSVu9M+Lx8fUFQ1QJAAD/HysfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKuiCh+fffaZ/u7v/k7p6elKTEzUrbfequeff149PT3OHGOMSkpKFAgElJiYqNzcXDU1NQ144RgYaat3RhwAAAy2qMLHCy+8oB//+MeqqKjQ+++/rw0bNujFF1/Uj370I2fOhg0bVFZWpoqKCjU0NMjv9ysvL0+dnZ0DXjwAAIg9UYWPPXv26Otf/7oKCgqUlpamP/3TP1V+fr7eeecdSedXPcrLy1VcXKy5c+cqMzNTVVVVOnPmjKqrqwelAQAAEFuiCh8PPPCA/v3f/10ffPCBJOm//uu/9Mtf/lJf+9rXJEnNzc0KhULKz893nuN2u5WTk6P6+vo+X7Orq0sdHR0RBwAAiF9R3Vhu1apVCofDuuOOO3Tdddfp3LlzWrdunZ588klJUigUkiT5fL6I5/l8PrW0tPT5mqWlpVq7dm1/agcAADEoqpWPn/70p9qyZYuqq6t14MABVVVV6fvf/76qqqoi5rlcrojHxpheYxcUFRUpHA47R2tra5QtAACAWBLVysff/u3favXq1XriiSckSVOmTFFLS4tKS0u1YMEC+f1+SedXQFJSUpzntbW19VoNucDtdsvtdve3fgAAEGOiWvk4c+aMRo2KfMp1113nXGqbnp4uv9+v2tpa53x3d7fq6uqUnZ09AOUCAIBYF9XKx5w5c7Ru3TpNnDhRkydP1sGDB1VWVqaFCxdKOv9xS2FhoYLBoDIyMpSRkaFgMKikpCTNmzdvUBrA1eN7PAAAw0FU4eNHP/qR/v7v/15LlixRW1ubAoGAFi1apO9+97vOnJUrV+rs2bNasmSJ2tvblZWVpZqaGnk8ngEvHgAAxB6XMcYMdREX6+jokNfrVTgcVnJy8lCXE9P6s9JxfH3BIFQCAIh30fz+5t4uAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKyKKnykpaXJ5XL1OpYuXSpJMsaopKREgUBAiYmJys3NVVNT06AUDgAAYlNU4aOhoUGnTp1yjtraWknS448/LknasGGDysrKVFFRoYaGBvn9fuXl5amzs3PgKwcAADEpqvBxyy23yO/3O8e//du/6bbbblNOTo6MMSovL1dxcbHmzp2rzMxMVVVV6cyZM6qurh6s+gEAQIzp956P7u5ubdmyRQsXLpTL5VJzc7NCoZDy8/OdOW63Wzk5Oaqvr7/s63R1damjoyPiAAAA8avf4eP111/X6dOn9fTTT0uSQqGQJMnn80XM8/l8zrm+lJaWyuv1Okdqamp/SwIAADGg3+HjlVde0axZsxQIBCLGXS5XxGNjTK+xixUVFSkcDjtHa2trf0sCAAAxIKE/T2ppadGuXbu0bds2Z8zv90s6vwKSkpLijLe1tfVaDbmY2+2W2+3uTxkAACAG9WvlY/PmzRo/frwKCgqcsfT0dPn9fucKGOn8vpC6ujplZ2dfe6UAACAuRL3y0dPTo82bN2vBggVKSPj/T3e5XCosLFQwGFRGRoYyMjIUDAaVlJSkefPmDWjRAAAgdkUdPnbt2qUTJ05o4cKFvc6tXLlSZ8+e1ZIlS9Te3q6srCzV1NTI4/EMSLEAACD2uYwxZqiLuFhHR4e8Xq/C4bCSk5OHupyYlrZ6Z9TPOb6+4PMnAQBwiWh+f3NvFwAAYBXhAwAAWNWvS20x/PTnIxYAAIYCKx8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsShroAxIe01TsjHh9fXzBElQAAhjtWPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFZFHT5Onjypb37zm7rpppuUlJSke+65R/v373fOG2NUUlKiQCCgxMRE5ebmqqmpaUCLxvCXtnpnrwMAACnK8NHe3q77779fo0eP1s9//nMdPnxY//AP/6A/+IM/cOZs2LBBZWVlqqioUENDg/x+v/Ly8tTZ2TnQtQMAgBiUEM3kF154Qampqdq8ebMzlpaW5vxvY4zKy8tVXFysuXPnSpKqqqrk8/lUXV2tRYsWDUzVAAAgZkW18rFjxw7NmDFDjz/+uMaPH69p06Zp06ZNzvnm5maFQiHl5+c7Y263Wzk5Oaqvr+/zNbu6utTR0RFxAACA+BXVysexY8dUWVmpFStW6LnnntO+ffv07LPPyu1261vf+pZCoZAkyefzRTzP5/OppaWlz9csLS3V2rVr+1n+yMUeCgBArIpq5aOnp0df/vKXFQwGNW3aNC1atEh/+Zd/qcrKyoh5Lpcr4rExptfYBUVFRQqHw87R2toaZQsAACCWRBU+UlJSdNddd0WM3XnnnTpx4oQkye/3S5KzAnJBW1tbr9WQC9xut5KTkyMOAAAQv6IKH/fff7+OHDkSMfbBBx9o0qRJkqT09HT5/X7V1tY657u7u1VXV6fs7OwBKBcAAMS6qPZ8/NVf/ZWys7MVDAb1Z3/2Z9q3b582btyojRs3Sjr/cUthYaGCwaAyMjKUkZGhYDCopKQkzZs3b1AaAAAAsSWq8HHvvfdq+/btKioq0vPPP6/09HSVl5dr/vz5zpyVK1fq7NmzWrJkidrb25WVlaWamhp5PJ4BLx4AAMSeqMKHJM2ePVuzZ8++7HmXy6WSkhKVlJRcS10AACBOcW8XAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFgV9aW2GHkuvYnd8fUFQ1QJACAesPIBAACsInwAAACrCB8AAMAq9nwgwqX7OwAAGGisfAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArEoY6gLw+dJW7xzqEgAAGDCsfAAAAKsIHwAAwCrCBwAAsCqq8FFSUiKXyxVx+P1+57wxRiUlJQoEAkpMTFRubq6ampoGvGjEprTVOyMOAMDIFPXKx+TJk3Xq1CnnaGxsdM5t2LBBZWVlqqioUENDg/x+v/Ly8tTZ2TmgRQMAgNgVdfhISEiQ3+93jltuuUXS+VWP8vJyFRcXa+7cucrMzFRVVZXOnDmj6urqAS8cAADEpqjDx9GjRxUIBJSenq4nnnhCx44dkyQ1NzcrFAopPz/fmet2u5WTk6P6+vrLvl5XV5c6OjoiDgAAEL+i+p6PrKwsvfrqq7r99tv10Ucf6Xvf+56ys7PV1NSkUCgkSfL5fBHP8fl8amlpuexrlpaWau3atf0oHUOF/RoAgGsR1crHrFmz9Cd/8ieaMmWKHnnkEe3cef6XUFVVlTPH5XJFPMcY02vsYkVFRQqHw87R2toaTUkAACDGXNOltmPHjtWUKVN09OhR56qXCysgF7S1tfVaDbmY2+1WcnJyxAEAAOLXNYWPrq4uvf/++0pJSVF6err8fr9qa2ud893d3aqrq1N2dvY1FwoAAOJDVHs+/uZv/kZz5szRxIkT1dbWpu9973vq6OjQggUL5HK5VFhYqGAwqIyMDGVkZCgYDCopKUnz5s0brPoBAECMiSp8/OY3v9GTTz6p3/72t7rlllt03333ae/evZo0aZIkaeXKlTp79qyWLFmi9vZ2ZWVlqaamRh6PZ1CKBwAAscdljDFDXcTFOjo65PV6FQ6H2f/xf+L16pLj6wuGugQAwACJ5vc393YBAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYNU1hY/S0lK5XC4VFhY6Y8YYlZSUKBAIKDExUbm5uWpqarrWOhGH0lbv7HUAAOJfv8NHQ0ODNm7cqKlTp0aMb9iwQWVlZaqoqFBDQ4P8fr/y8vLU2dl5zcUCAIDY16/w8cknn2j+/PnatGmTbrzxRmfcGKPy8nIVFxdr7ty5yszMVFVVlc6cOaPq6uoBKxoAAMSufoWPpUuXqqCgQI888kjEeHNzs0KhkPLz850xt9utnJwc1dfXX1ulAAAgLiRE+4StW7fqwIEDamho6HUuFApJknw+X8S4z+dTS0tLn6/X1dWlrq4u53FHR0e0JQEAgBgS1cpHa2urli9fri1btuj666+/7DyXyxXx2BjTa+yC0tJSeb1e50hNTY2mJAAAEGOiCh/79+9XW1ubpk+froSEBCUkJKiurk4//OEPlZCQ4Kx4XFgBuaCtra3XasgFRUVFCofDztHa2trPVgAAQCyI6mOXhx9+WI2NjRFjf/7nf6477rhDq1at0q233iq/36/a2lpNmzZNktTd3a26ujq98MILfb6m2+2W2+3uZ/kAACDWRBU+PB6PMjMzI8bGjh2rm266yRkvLCxUMBhURkaGMjIyFAwGlZSUpHnz5g1c1QAAIGZFveH086xcuVJnz57VkiVL1N7erqysLNXU1Mjj8Qz0WwEAgBjkMsaYoS7iYh0dHfJ6vQqHw0pOTh7qcoaFkfTNn8fXFwx1CQCAfojm9zf3dgEAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYlRDN5MrKSlVWVur48eOSpMmTJ+u73/2uZs2aJUkyxmjt2rXauHGj2tvblZWVpZdeekmTJ08e8MJjQdrqnb3Gjq8v6NfzRopLe7+an9dgvg4AYOBFtfIxYcIErV+/Xu+8847eeecdffWrX9XXv/51NTU1SZI2bNigsrIyVVRUqKGhQX6/X3l5eers7ByU4gEAQOyJKnzMmTNHX/va13T77bfr9ttv17p163TDDTdo7969MsaovLxcxcXFmjt3rjIzM1VVVaUzZ86ourp6sOoHAAAxpt97Ps6dO6etW7fq008/1cyZM9Xc3KxQKKT8/HxnjtvtVk5Ojurr6y/7Ol1dXero6Ig4AABA/Io6fDQ2NuqGG26Q2+3W4sWLtX37dt11110KhUKSJJ/PFzHf5/M55/pSWloqr9frHKmpqdGWBAAAYkjU4eNLX/qSDh06pL179+o73/mOFixYoMOHDzvnXS5XxHxjTK+xixUVFSkcDjtHa2trtCUBAIAYEtXVLpI0ZswYffGLX5QkzZgxQw0NDfrBD36gVatWSZJCoZBSUlKc+W1tbb1WQy7mdrvldrujLQMAAMSoa/6eD2OMurq6lJ6eLr/fr9raWudcd3e36urqlJ2dfa1vAwAA4kRUKx/PPfecZs2apdTUVHV2dmrr1q16++239cYbb8jlcqmwsFDBYFAZGRnKyMhQMBhUUlKS5s2bN1j1AwCAGBNV+Pjoo4/01FNP6dSpU/J6vZo6dareeOMN5eXlSZJWrlyps2fPasmSJc6XjNXU1Mjj8QxK8QAAIPZEFT5eeeWVK553uVwqKSlRSUnJtdQEAADiGPd2AQAAVhE+AACAVVFfagsMNyP5RnwAEItY+QAAAFYRPgAAgFWEDwAAYBV7PjCssZ8DAOIPKx8AAMAqwgcAALCK8AEAAKxiz4dll+5hOL6+YIgqAQBgaLDyAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwKmGoCxjp0lbvHOoSRoS+fs7H1xcMQSUAAFY+AACAVYQPAABgFeEDAABYFVX4KC0t1b333iuPx6Px48frscce05EjRyLmGGNUUlKiQCCgxMRE5ebmqqmpaUCLBgAAsSuq8FFXV6elS5dq7969qq2t1Weffab8/Hx9+umnzpwNGzaorKxMFRUVamhokN/vV15enjo7Owe8eAAAEHuiutrljTfeiHi8efNmjR8/Xvv379dXvvIVGWNUXl6u4uJizZ07V5JUVVUln8+n6upqLVq0aOAqBwAAMema9nyEw2FJ0rhx4yRJzc3NCoVCys/Pd+a43W7l5OSovr6+z9fo6upSR0dHxAEAAOJXv8OHMUYrVqzQAw88oMzMTElSKBSSJPl8voi5Pp/POXep0tJSeb1e50hNTe1vSQAAIAb0O3wsW7ZM7777rn7yk5/0OudyuSIeG2N6jV1QVFSkcDjsHK2trf0tCQAAxIB+fcPpM888ox07dmj37t2aMGGCM+73+yWdXwFJSUlxxtva2nqthlzgdrvldrv7UwYAAIhBUa18GGO0bNkybdu2Tb/4xS+Unp4ecT49PV1+v1+1tbXOWHd3t+rq6pSdnT0wFQMAgJgW1crH0qVLVV1drX/5l3+Rx+Nx9nF4vV4lJibK5XKpsLBQwWBQGRkZysjIUDAYVFJSkubNmzcoDQAAgNgSVfiorKyUJOXm5kaMb968WU8//bQkaeXKlTp79qyWLFmi9vZ2ZWVlqaamRh6PZ0AKBgAAsS2q8GGM+dw5LpdLJSUlKikp6W9NAAAgjnFvFwAAYBXhAwAAWNWvS22BeJC2emfE4+PrC6J+ztU+DwDw/7HyAQAArCJ8AAAAqwgfAADAKvZ8AFfQ1x4PAMC1YeUDAABYRfgAAABWET4AAIBV7PkA/g/7OwDADlY+AACAVYQPAABgFeEDAABYRfgAAABWseEUuEb9uUEdAIxkrHwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKxKiPYJu3fv1osvvqj9+/fr1KlT2r59ux577DHnvDFGa9eu1caNG9Xe3q6srCy99NJLmjx58kDWDQxbaat3fu6c4+sLLFQCAMNT1Csfn376qe6++25VVFT0eX7Dhg0qKytTRUWFGhoa5Pf7lZeXp87OzmsuFgAAxL6oVz5mzZqlWbNm9XnOGKPy8nIVFxdr7ty5kqSqqir5fD5VV1dr0aJF11YtAACIeQO656O5uVmhUEj5+fnOmNvtVk5Ojurr6/t8TldXlzo6OiIOAAAQv6Je+biSUCgkSfL5fBHjPp9PLS0tfT6ntLRUa9euHcgygLh16X4S9o4AiEWDcrWLy+WKeGyM6TV2QVFRkcLhsHO0trYORkkAAGCYGNCVD7/fL+n8CkhKSooz3tbW1ms15AK32y232z2QZQAAgGFsQFc+0tPT5ff7VVtb64x1d3errq5O2dnZA/lWAAAgRkW98vHJJ5/o17/+tfO4ublZhw4d0rhx4zRx4kQVFhYqGAwqIyNDGRkZCgaDSkpK0rx58wa0cCDeXc33hfQ1h30gAIa7qMPHO++8o4ceesh5vGLFCknSggUL9E//9E9auXKlzp49qyVLljhfMlZTUyOPxzNwVQMAgJgVdfjIzc2VMeay510ul0pKSlRSUnItdQEAgDjFvV0AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYNWAfr36SHc1XwoFXA5/fwCMFKx8AAAAqwgfAADAKsIHAACwij0fV4nP44HeLv3vgpvaAbgarHwAAACrCB8AAMAqwgcAALBqxO356GvvBp9Twzabe4hs/p3nvy8AV4OVDwAAYBXhAwAAWEX4AAAAVo24PR9Xg+/0QCy7mr+/Nr+fg+8CAXApVj4AAIBVhA8AAGAV4QMAAFhF+AAAAFax4RTAVRmojdgD+UVkbGYFYhMrHwAAwCrCBwAAsIrwAQAArGLPB4A+DfXN7y41UPtCrua1R9IN8tg3g6HAygcAALCK8AEAAKwifAAAAKsGbc/Hyy+/rBdffFGnTp3S5MmTVV5ergcffHCw3g7ANYiFmyn254Z5tl3N/onB2mMRL/tUBnP/z0gRCz/DQVn5+OlPf6rCwkIVFxfr4MGDevDBBzVr1iydOHFiMN4OAADEkEEJH2VlZfqLv/gLffvb39add96p8vJypaamqrKycjDeDgAAxJAB/9ilu7tb+/fv1+rVqyPG8/PzVV9f32t+V1eXurq6nMfhcFiS1NHRMdClSZJ6us70Grv0vfqaAyB+Xc2/AVfzb9Klz+vrOVczpz8Gs2abrubf36Gucbgbqp/hhdc0xnz+ZDPATp48aSSZ//zP/4wYX7dunbn99tt7zV+zZo2RxMHBwcHBwREHR2tr6+dmhUHbcOpyuSIeG2N6jUlSUVGRVqxY4Tzu6elRS0uL7rnnHrW2tio5OXmwShyWOjo6lJqaOuJ6H6l9S/Q+EnsfqX1LI7f3kdC3MUadnZ0KBAKfO3fAw8fNN9+s6667TqFQKGK8ra1NPp+v13y32y232x0xNmrU+a0oycnJcfuH9HlGau8jtW+J3kdi7yO1b2nk9h7vfXu93quaN+AbTseMGaPp06ertrY2Yry2tlbZ2dkD/XYAACDGDMrHLitWrNBTTz2lGTNmaObMmdq4caNOnDihxYsXD8bbAQCAGDIo4eMb3/iGPv74Yz3//PM6deqUMjMz9bOf/UyTJk26que73W6tWbOm18cxI8FI7X2k9i3R+0jsfaT2LY3c3kdq35fjMuZqrokBAAAYGNzbBQAAWEX4AAAAVhE+AACAVYQPAABg1bAMHy+//LLS09N1/fXXa/r06fqP//iPoS7JsXv3bs2ZM0eBQEAul0uvv/56xHljjEpKShQIBJSYmKjc3Fw1NTVFzOnq6tIzzzyjm2++WWPHjtUf//Ef6ze/+U3EnPb2dj311FPyer3yer166qmndPr06Yg5J06c0Jw5czR27FjdfPPNevbZZ9Xd3R0xp7GxUTk5OUpMTNQXvvAFPf/881f3vfuXKC0t1b333iuPx6Px48frscce05EjR+K+98rKSk2dOtX5YqCZM2fq5z//eVz3fDmlpaVyuVwqLCyM+/5LSkrkcrkiDr/fH/d9S9LJkyf1zW9+UzfddJOSkpJ0zz33aP/+/XHfe1paWq8/c5fLpaVLl8Z130Omn7dwGTRbt241o0ePNps2bTKHDx82y5cvN2PHjjUtLS1DXZoxxpif/exnpri42Lz22mtGktm+fXvE+fXr1xuPx2Nee+0109jYaL7xjW+YlJQU09HR4cxZvHix+cIXvmBqa2vNgQMHzEMPPWTuvvtu89lnnzlz/uiP/shkZmaa+vp6U19fbzIzM83s2bOd85999pnJzMw0Dz30kDlw4ICpra01gUDALFu2zJkTDoeNz+czTzzxhGlsbDSvvfaa8Xg85vvf/37UfT/66KNm8+bN5r333jOHDh0yBQUFZuLEieaTTz6J69537Nhhdu7caY4cOWKOHDlinnvuOTN69Gjz3nvvxW3Pfdm3b59JS0szU6dONcuXL3fG47X/NWvWmMmTJ5tTp045R1tbW9z3/b//+79m0qRJ5umnnza/+tWvTHNzs9m1a5f59a9/Hfe9t7W1Rfx519bWGknmrbfeiuu+h8qwCx9/+Id/aBYvXhwxdscdd5jVq1cPUUWXd2n46OnpMX6/36xfv94Z+93vfme8Xq/58Y9/bIwx5vTp02b06NFm69atzpyTJ0+aUaNGmTfeeMMYY8zhw4eNJLN3715nzp49e4wk89///d/GmPMhaNSoUebkyZPOnJ/85CfG7XabcDhsjDHm5ZdfNl6v1/zud79z5pSWlppAIGB6enquqfe2tjYjydTV1Y243m+88Ubzj//4jyOm587OTpORkWFqa2tNTk6OEz7iuf81a9aYu+++u89z8dz3qlWrzAMPPHDZ8/Hc+6WWL19ubrvtNtPT0zOi+rZlWH3s0t3drf379ys/Pz9iPD8/X/X19UNU1dVrbm5WKBSKqN/tdisnJ8epf//+/fr9738fMScQCCgzM9OZs2fPHnm9XmVlZTlz7rvvPnm93og5mZmZETfwefTRR9XV1eUske7Zs0c5OTkRX2rz6KOP6sMPP9Tx48evqddwOCxJGjdu3Ijp/dy5c9q6das+/fRTzZw5c0T0LElLly5VQUGBHnnkkYjxeO//6NGjCgQCSk9P1xNPPKFjx47Ffd87duzQjBkz9Pjjj2v8+PGaNm2aNm3a5JyP594v1t3drS1btmjhwoVyuVwjpm+bhlX4+O1vf6tz5871ugGdz+frdaO64ehCjVeqPxQKacyYMbrxxhuvOGf8+PG9Xn/8+PERcy59nxtvvFFjxoy54pwLj6/l52mM0YoVK/TAAw8oMzMz4vXisffGxkbdcMMNcrvdWrx4sbZv36677rorrnu+YOvWrTpw4IBKS0t7nYvn/rOysvTqq6/qzTff1KZNmxQKhZSdna2PP/44rvs+duyYKisrlZGRoTfffFOLFy/Ws88+q1dffTXi9eKx94u9/vrrOn36tJ5++umI14r3vm0alK9Xv1YulyvisTGm19hw1p/6L53T1/yBmGP+b0PStfw8ly1bpnfffVe//OUve52Lx96/9KUv6dChQzp9+rRee+01LViwQHV1dVd8n1jvWZJaW1u1fPly1dTU6Prrr7/svHjsf9asWc7/njJlimbOnKnbbrtNVVVVuu+++y77XrHed09Pj2bMmKFgMChJmjZtmpqamlRZWalvfetbV3y/WO/9Yq+88opmzZrV69bw8d63TcNq5ePmm2/Wdddd1yu5tbW19Up5w9GF3fBXqt/v96u7u1vt7e1XnPPRRx/1ev3/+Z//iZhz6fu0t7fr97///RXntLW1Seqd4K/WM888ox07duitt97ShAkTnPF47n3MmDH64he/qBkzZqi0tFR33323fvCDH8R1z9L5ZeS2tjZNnz5dCQkJSkhIUF1dnX74wx8qISHhsv9PK176v9jYsWM1ZcoUHT16NK7/3FNSUnTXXXdFjN155506ceKE815SfPZ+QUtLi3bt2qVvf/vbzthI6Nu2YRU+xowZo+nTp6u2tjZivLa2VtnZ2UNU1dVLT0+X3++PqL+7u1t1dXVO/dOnT9fo0aMj5pw6dUrvvfeeM2fmzJkKh8Pat2+fM+dXv/qVwuFwxJz33ntPp06dcubU1NTI7XZr+vTpzpzdu3dHXKJVU1OjQCCgtLS0qHozxmjZsmXatm2bfvGLXyg9PX3E9N7Xz6Krqyvue3744YfV2NioQ4cOOceMGTM0f/58HTp0SLfeemtc93+xrq4uvf/++0pJSYnrP/f777+/1yX0H3zwgXNT0Hju/YLNmzdr/PjxKigocMZGQt/WDeJm1n65cKntK6+8Yg4fPmwKCwvN2LFjzfHjx4e6NGPM+Z3/Bw8eNAcPHjSSTFlZmTl48KBzKfD69euN1+s127ZtM42NjebJJ5/s83KsCRMmmF27dpkDBw6Yr371q31ejjV16lSzZ88es2fPHjNlypQ+L8d6+OGHzYEDB8yuXbvMhAkTIi7HOn36tPH5fObJJ580jY2NZtu2bSY5Oblfl2N95zvfMV6v17z99tsRl6OdOXPGmROPvRcVFZndu3eb5uZm8+6775rnnnvOjBo1ytTU1MRtz1dy8dUu8dz/X//1X5u3337bHDt2zOzdu9fMnj3beDwe59+heO173759JiEhwaxbt84cPXrU/PM//7NJSkoyW7ZscebEa+/GGHPu3DkzceJEs2rVql7n4rnvoTDswocxxrz00ktm0qRJZsyYMebLX/6ycznncPDWW28ZSb2OBQsWGGPOX4q2Zs0a4/f7jdvtNl/5yldMY2NjxGucPXvWLFu2zIwbN84kJiaa2bNnmxMnTkTM+fjjj838+fONx+MxHo/HzJ8/37S3t0fMaWlpMQUFBSYxMdGMGzfOLFu2LOLSK2OMeffdd82DDz5o3G638fv9pqSkpF+XYvXVsySzefNmZ0489r5w4ULn7+Itt9xiHn74YSd4xGvPV3Jp+IjX/i98h8Po0aNNIBAwc+fONU1NTXHftzHG/Ou//qvJzMw0brfb3HHHHWbjxo0R5+O59zfffNNIMkeOHOl1Lp77HgouY2LpK9EAAECsG1Z7PgAAQPwjfAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALDq/wH3ipdEUuUoVQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(y_train, bins=100);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- A common trick in such cases is applying a log transform on the target column to make it more normal and less skewed.  \n",
    "- That is, transform $y\\rightarrow \\log(y)$.\n",
    "- Linear regression will usually work better on something that looks more normal. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:48.222549Z",
     "iopub.status.busy": "2024-05-29T14:52:48.221053Z",
     "iopub.status.idle": "2024-05-29T14:52:48.993724Z",
     "shell.execute_reply": "2024-05-29T14:52:48.991358Z",
     "shell.execute_reply.started": "2024-05-29T14:52:48.222432Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfvUlEQVR4nO3df3BU1f3/8ddKdEk0gUphNykRoi6OiqglNhLrJ6mSdJDSdtCiBhXb6kCDPyI6lDR/sDi4wUxN0zaKxbEYRyOdsaK2KiZtxyCmaQM2LRMcpRIgCksGTZPwo0kj5/uHX1Y3CZAbds+yy/Mxc2e6597dfefd5PLy7D17XcYYIwAAAEvOiHUBAADg9EL4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGBVUqwLGOjIkSPas2ePUlNT5XK5Yl0OAAAYBmOMenp6lJGRoTPOOP7cxikXPvbs2aPMzMxYlwEAAEagvb1dEydOPO4xp1z4SE1NlfR58WlpaTGuBgAADEd3d7cyMzND/44fzykXPo5+1JKWlkb4AAAgzgznkgkuOAUAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFVJsS4AQOxNXvbaoLGdq2bHoBIApwNmPgAAgFWOw8fHH3+s2267TePGjVNKSoquuOIKbdmyJbTfGCO/36+MjAwlJycrPz9fra2tES0aAADEL0fho7OzU9dcc43OPPNMvfHGG9q2bZsee+wxjR07NnRMRUWFKisrVV1drebmZnm9XhUUFKinpyfStQMAgDjk6JqPRx99VJmZmVq7dm1obPLkyaH/bYxRVVWVysrKNHfuXElSTU2NPB6PamtrtXDhwshUDQAA4pajmY9XX31V2dnZ+sEPfqAJEyboyiuv1FNPPRXa39bWpmAwqMLCwtCY2+1WXl6eGhsbh3zN3t5edXd3h20AACBxOQofO3bs0OrVq+Xz+fTmm29q0aJFuu+++/Tss89KkoLBoCTJ4/GEPc/j8YT2DVReXq4xY8aEtszMzJH8HAAAIE44Ch9HjhzR17/+dQUCAV155ZVauHCh7r77bq1evTrsOJfLFfbYGDNo7KjS0lJ1dXWFtvb2doc/AgAAiCeOwkd6erouueSSsLGLL75Yu3fvliR5vV5JGjTL0dHRMWg25Ci32620tLSwDQAAJC5H4eOaa67R+++/Hzb2wQcfaNKkSZKkrKwseb1e1dfXh/b39fWpoaFBubm5ESgXAADEO0erXR544AHl5uYqEAho3rx5+vvf/641a9ZozZo1kj7/uKWkpESBQEA+n08+n0+BQEApKSkqKiqKyg8AAADii6PwcdVVV2n9+vUqLS3Vww8/rKysLFVVVWn+/PmhY5YuXarDhw+ruLhYnZ2dysnJUV1dnVJTUyNePAAAiD8uY4yJdRFf1t3drTFjxqirq4vrPwBLuLcLgJPl5N9v7u0CAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArEqKdQEAMNDkZa8NGtu5anYMKgEQDcx8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrWO0CJDhWjgA41TDzAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqR+HD7/fL5XKFbV6vN7TfGCO/36+MjAwlJycrPz9fra2tES8aAADEL8czH5deeqn27t0b2rZu3RraV1FRocrKSlVXV6u5uVler1cFBQXq6emJaNEAACB+OQ4fSUlJ8nq9oW38+PGSPp/1qKqqUllZmebOnaupU6eqpqZGhw4dUm1tbcQLBwAA8clx+Ni+fbsyMjKUlZWlW265RTt27JAktbW1KRgMqrCwMHSs2+1WXl6eGhsbj/l6vb296u7uDtsAAEDicnRvl5ycHD377LOaMmWK9u3bp5UrVyo3N1etra0KBoOSJI/HE/Ycj8ejXbt2HfM1y8vLtWLFihGUDsAm7hEDIFIczXzMmjVLN954oy677DLNnDlTr732+cmopqYmdIzL5Qp7jjFm0NiXlZaWqqurK7S1t7c7KQkAAMSZk1pqe/bZZ+uyyy7T9u3bQ6tejs6AHNXR0TFoNuTL3G630tLSwjYAAJC4Tip89Pb26r333lN6erqysrLk9XpVX18f2t/X16eGhgbl5uaedKEAACAxOLrm46GHHtKcOXN03nnnqaOjQytXrlR3d7cWLFggl8ulkpISBQIB+Xw++Xw+BQIBpaSkqKioKFr1AwCAOOMofHz00Ue69dZbtX//fo0fP15XX321mpqaNGnSJEnS0qVLdfjwYRUXF6uzs1M5OTmqq6tTampqVIoHAADxx1H4WLdu3XH3u1wu+f1++f3+k6kJAAAkMO7tAgAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKscfb06gNPH5GWvxboEAAmKmQ8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWcW8XAHFr4P1ndq6aHaNKADjBzAcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACs4uvVAURVpL4CfeDrAIhfzHwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKtY7QIgYliRAmA4mPkAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFUnFT7Ky8vlcrlUUlISGjPGyO/3KyMjQ8nJycrPz1dra+vJ1gkAABLEiMNHc3Oz1qxZo2nTpoWNV1RUqLKyUtXV1WpubpbX61VBQYF6enpOulgAABD/RhQ+Dhw4oPnz5+upp57SV77yldC4MUZVVVUqKyvT3LlzNXXqVNXU1OjQoUOqra2NWNEAACB+jSh8LF68WLNnz9bMmTPDxtva2hQMBlVYWBgac7vdysvLU2Nj45Cv1dvbq+7u7rANAAAkLsdfr75u3Tq9++67am5uHrQvGAxKkjweT9i4x+PRrl27hny98vJyrVixwmkZADDIUF/vvnPV7BhUAuB4HM18tLe36/7779dzzz2n0aNHH/M4l8sV9tgYM2jsqNLSUnV1dYW29vZ2JyUBAIA442jmY8uWLero6ND06dNDY5999pk2btyo6upqvf/++5I+nwFJT08PHdPR0TFoNuQot9stt9s9ktoBAEAccjTzcf3112vr1q1qaWkJbdnZ2Zo/f75aWlp0/vnny+v1qr6+PvScvr4+NTQ0KDc3N+LFAwCA+ONo5iM1NVVTp04NGzv77LM1bty40HhJSYkCgYB8Pp98Pp8CgYBSUlJUVFQUuaoBAEDccnzB6YksXbpUhw8fVnFxsTo7O5WTk6O6ujqlpqZG+q0AAEAcOunw8dZbb4U9drlc8vv98vv9J/vSAAAgAXFvFwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgVcSX2gI49Q11DxQAsIWZDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFatdAMQcq2+A0wszHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCKr1cHYBVfpQ6AmQ8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBWrXQCMGCtXAIwEMx8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCpWuwBAhAxc/bNz1ewYVQKc2pj5AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWsdoFOEWwUiI66Ctw6mHmAwAAWEX4AAAAVhE+AACAVYQPAABgFRecAnFkOBdPDjwGAE41zHwAAACrHIWP1atXa9q0aUpLS1NaWppmzJihN954I7TfGCO/36+MjAwlJycrPz9fra2tES8aAADEL0fhY+LEiVq1apU2b96szZs367rrrtP3vve9UMCoqKhQZWWlqqur1dzcLK/Xq4KCAvX09ESleAAAEH8chY85c+bohhtu0JQpUzRlyhQ98sgjOuecc9TU1CRjjKqqqlRWVqa5c+dq6tSpqqmp0aFDh1RbWxut+gEAQJwZ8TUfn332mdatW6eDBw9qxowZamtrUzAYVGFhYegYt9utvLw8NTY2HvN1ent71d3dHbYBAIDE5Th8bN26Veecc47cbrcWLVqk9evX65JLLlEwGJQkeTyesOM9Hk9o31DKy8s1ZsyY0JaZmem0JAAAEEcch4+LLrpILS0tampq0k9+8hMtWLBA27ZtC+13uVxhxxtjBo19WWlpqbq6ukJbe3u705IAAEAccfw9H2eddZYuvPBCSVJ2draam5v1y1/+Uj/96U8lScFgUOnp6aHjOzo6Bs2GfJnb7Zbb7XZaBgAAiFMn/T0fxhj19vYqKytLXq9X9fX1oX19fX1qaGhQbm7uyb4NAABIEI5mPn72s59p1qxZyszMVE9Pj9atW6e33npLGzZskMvlUklJiQKBgHw+n3w+nwKBgFJSUlRUVBSt+gEAQJxxFD727dun22+/XXv37tWYMWM0bdo0bdiwQQUFBZKkpUuX6vDhwyouLlZnZ6dycnJUV1en1NTUqBQPAADij6Pw8fTTTx93v8vlkt/vl9/vP5maAAwT93EBEI+4twsAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACscvT16gAQ74b6Svqdq2Y7ft5wngNgaMx8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrWO0CACMw1KoZAMPDzAcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrVLgBOe9y3BbCLmQ8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWcW8XABhg4L1eovna3EcGpyNmPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWET4AAIBVhA8AAGAV4QMAAFhF+AAAAFY5Ch/l5eW66qqrlJqaqgkTJuj73/++3n///bBjjDHy+/3KyMhQcnKy8vPz1draGtGiAQBA/HIUPhoaGrR48WI1NTWpvr5e/f39Kiws1MGDB0PHVFRUqLKyUtXV1WpubpbX61VBQYF6enoiXjwAAIg/SU4O3rBhQ9jjtWvXasKECdqyZYv+7//+T8YYVVVVqaysTHPnzpUk1dTUyOPxqLa2VgsXLoxc5QAAIC6d1DUfXV1dkqRzzz1XktTW1qZgMKjCwsLQMW63W3l5eWpsbBzyNXp7e9Xd3R22AQCAxOVo5uPLjDFasmSJvvnNb2rq1KmSpGAwKEnyeDxhx3o8Hu3atWvI1ykvL9eKFStGWgYQlyYvey3WJSCODPX7snPV7BhUAkTGiGc+7rnnHv3rX//SCy+8MGify+UKe2yMGTR2VGlpqbq6ukJbe3v7SEsCAABxYEQzH/fee69effVVbdy4URMnTgyNe71eSZ/PgKSnp4fGOzo6Bs2GHOV2u+V2u0dSBgAAiEOOZj6MMbrnnnv00ksv6S9/+YuysrLC9mdlZcnr9aq+vj401tfXp4aGBuXm5kamYgAAENcczXwsXrxYtbW1euWVV5Samhq6xmPMmDFKTk6Wy+VSSUmJAoGAfD6ffD6fAoGAUlJSVFRUFJUfAAAAxBdH4WP16tWSpPz8/LDxtWvX6s4775QkLV26VIcPH1ZxcbE6OzuVk5Ojuro6paamRqRgAAAQ3xyFD2PMCY9xuVzy+/3y+/0jrQmAWBFzumAlC05H3NsFAABYRfgAAABWET4AAIBVhA8AAGDViL9eHQAQHVxsjETHzAcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsCop1gUAiWbystcGje1cNTsGlSCRDfw943cM8YSZDwAAYBXhAwAAWEX4AAAAVhE+AACAVVxwCgCniaEuhh6IC1dhAzMfAADAKsIHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCrCBwAAsIrwAQAArOLeLjgtDHVPC+5hAQCxwcwHAACwivABAACsInwAAACrCB8AAMAqwgcAALCK1S7A/zfUipiBWCEDACfP8czHxo0bNWfOHGVkZMjlcunll18O22+Mkd/vV0ZGhpKTk5Wfn6/W1tZI1QsAAOKc4/Bx8OBBXX755aqurh5yf0VFhSorK1VdXa3m5mZ5vV4VFBSop6fnpIsFAADxz/HHLrNmzdKsWbOG3GeMUVVVlcrKyjR37lxJUk1NjTwej2pra7Vw4cKTqxYAAMS9iF5w2tbWpmAwqMLCwtCY2+1WXl6eGhsbI/lWAAAgTkX0gtNgMChJ8ng8YeMej0e7du0a8jm9vb3q7e0NPe7u7o5kSQAA4BQTldUuLpcr7LExZtDYUeXl5VqxYkU0ysBpYjirVGL9/rGuEbHB/+/hBvaD1WOnr4h+7OL1eiV9MQNyVEdHx6DZkKNKS0vV1dUV2trb2yNZEgAAOMVENHxkZWXJ6/Wqvr4+NNbX16eGhgbl5uYO+Ry32620tLSwDQAAJC7HH7scOHBA//73v0OP29ra1NLSonPPPVfnnXeeSkpKFAgE5PP55PP5FAgElJKSoqKioogWDgAA4pPj8LF582Z961vfCj1esmSJJGnBggV65plntHTpUh0+fFjFxcXq7OxUTk6O6urqlJqaGrmqAQBA3HIcPvLz82WMOeZ+l8slv98vv99/MnUBAOIYF9vieLixHAAAsIrwAQAArCJ8AAAAqwgfAADAKsIHAACwKipfrw7EA67GR6KL1O84X4uOSGPmAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYxWoXwAFWyADAyWPmAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYxWoXAEgArMRCPGHmAwAAWEX4AAAAVhE+AACAVYQPAABgFRecAgBCuHAVNjDzAQAArCJ8AAAAqwgfAADAKsIHAACwivABAACsYrULrBl4Ff3OVbNjVAmAeDGc1TecS+IPMx8AAMAqwgcAALCK8AEAAKwifAAAAKsIHwAAwCpWu+CETrVVKtx7AkgMQ/0tx/r8AjuY+QAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVp12q11O56urI/Wz2+whK1uAU080/y4j9dqn2iq9gWJ9Ho11P5j5AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABWnXarXYZjpFdbx/Lq4ZFezRytK8tj/ToATh/DOW8M55ihzpkjOSfFeiVJPIjazMcTTzyhrKwsjR49WtOnT9fbb78drbcCAABxJCrh43e/+51KSkpUVlamf/zjH7r22ms1a9Ys7d69OxpvBwAA4khUwkdlZaV+/OMf66677tLFF1+sqqoqZWZmavXq1dF4OwAAEEcifs1HX1+ftmzZomXLloWNFxYWqrGxcdDxvb296u3tDT3u6uqSJHV3d0e6NEnSkd5Dg8YGvtdQxwxHtGoejmj+XNEyVL9OtRoBnD4idU4azr8FwzlnR4qt9zr6msaYEx9sIuzjjz82ksw777wTNv7II4+YKVOmDDp++fLlRhIbGxsbGxtbAmzt7e0nzApRW+3icrnCHhtjBo1JUmlpqZYsWRJ6fOTIEX366acaN27ckMefKrq7u5WZman29nalpaXFupyYox9foBfh6Ec4+vEFehEu3vthjFFPT48yMjJOeGzEw8dXv/pVjRo1SsFgMGy8o6NDHo9n0PFut1tutztsbOzYsZEuK2rS0tLi8pckWujHF+hFOPoRjn58gV6Ei+d+jBkzZljHRfyC07POOkvTp09XfX192Hh9fb1yc3Mj/XYAACDOROVjlyVLluj2229Xdna2ZsyYoTVr1mj37t1atGhRNN4OAADEkaiEj5tvvlmffPKJHn74Ye3du1dTp07V66+/rkmTJkXj7WLC7XZr+fLlgz4yOl3Rjy/Qi3D0Ixz9+AK9CHc69cNlzHDWxAAAAEQGN5YDAABWET4AAIBVhA8AAGAV4QMAAFhF+BiG8vJyuVwulZSUHPe43t5elZWVadKkSXK73brgggv029/+1k6Rlgy3F88//7wuv/xypaSkKD09XT/84Q/1ySef2Ckyivx+v1wuV9jm9XqP+5yGhgZNnz5do0eP1vnnn68nn3zSUrXR57QfL730kgoKCjR+/HilpaVpxowZevPNNy1WHF0j+f046p133lFSUpKuuOKK6BZpyUh6kcjn0JH0I1HPo1KUltomkubmZq1Zs0bTpk074bHz5s3Tvn379PTTT+vCCy9UR0eH+vv7LVRpx3B7sWnTJt1xxx36xS9+oTlz5ujjjz/WokWLdNddd2n9+vWWqo2eSy+9VH/6059Cj0eNGnXMY9va2nTDDTfo7rvv1nPPPad33nlHxcXFGj9+vG688UYb5Uadk35s3LhRBQUFCgQCGjt2rNauXas5c+bob3/7m6688kob5Uadk34c1dXVpTvuuEPXX3+99u3bF83yrHLai0Q/hzrpR6KfRwkfx3HgwAHNnz9fTz31lFauXHncYzds2KCGhgbt2LFD5557riRp8uTJFqq0w0kvmpqaNHnyZN13332SpKysLC1cuFAVFRU2So26pKSkYf/X7JNPPqnzzjtPVVVVkqSLL75Ymzdv1s9//vOECR9O+nG0D0cFAgG98sor+sMf/pAw4cNJP45auHChioqKNGrUKL388svRKSwGnPQi0c+hkrN+JPp5lI9djmPx4sWaPXu2Zs6cecJjX331VWVnZ6uiokJf+9rXNGXKFD300EM6fPiwhUqjz0kvcnNz9dFHH+n111+XMUb79u3Tiy++qNmzZ1uoNPq2b9+ujIwMZWVl6ZZbbtGOHTuOeexf//pXFRYWho19+9vf1ubNm/W///0v2qVa4aQfAx05ckQ9PT2hf2wSgdN+rF27Vh9++KGWL19uqUJ7nPQi0c+hkrN+JPp5lJmPY1i3bp3effddNTc3D+v4HTt2aNOmTRo9erTWr1+v/fv3q7i4WJ9++mncf2bptBe5ubl6/vnndfPNN+u///2v+vv79d3vfle//vWvo1xp9OXk5OjZZ5/VlClTtG/fPq1cuVK5ublqbW3VuHHjBh0fDAYH3VDR4/Gov79f+/fvV3p6uq3So8JpPwZ67LHHdPDgQc2bN89CtdHntB/bt2/XsmXL9PbbbyspKbFOx057kcjnUMl5PxL5PCpJMhhk9+7dZsKECaalpSU0lpeXZ+6///5jPqegoMCMHj3a/Oc//wmN/f73vzcul8scOnQomuVG1Uh60draatLT001FRYX55z//aTZs2GAuu+wy86Mf/chCxXYdOHDAeDwe89hjjw253+fzmUAgEDa2adMmI8ns3bvXRolWnagfX1ZbW2tSUlJMfX29hcpi43j96O/vN9nZ2Wb16tWhseXLl5vLL7/cYoX2nOh3I1HPocdyon4k+nmU8DGE9evXG0lm1KhRoU2ScblcZtSoUaa/v3/Qc+644w5zwQUXhI1t27bNSDIffPCBrdIjbiS9uO2228xNN90UNvb2228bSWbPnj22Srdm5syZZtGiRUPuu/baa819990XNvbSSy+ZpKQk09fXZ6M8647Xj6PWrVtnkpOTzR//+EdLVcXOsfrR2dk56G/L5XKFxv785z/HoNroOt7vRqKeQ4/neP1I9PMo13wM4frrr9fWrVvV0tIS2rKzszV//ny1tLQMeYXyNddcoz179ujAgQOhsQ8++EBnnHGGJk6caLP8iBpJLw4dOqQzzgj/1Tp6nEmwWwn19vbqvffeO+bHJzNmzFB9fX3YWF1dnbKzs3XmmWfaKNGqE/VDkl544QXdeeedqq2tTZjPr4/leP1IS0sb9Le1aNEiXXTRRWppaVFOTk4MKo6eE/1uJOo59FhO1I+EP4/GOv3Ei4EfNSxbtszcfvvtocc9PT1m4sSJ5qabbjKtra2moaHB+Hw+c9ddd8Wg2ug6US/Wrl1rkpKSzBNPPGE+/PBDs2nTJpOdnW2+8Y1vxKDayHrwwQfNW2+9ZXbs2GGamprMd77zHZOammp27txpjBncix07dpiUlBTzwAMPmG3btpmnn37anHnmmebFF1+M1Y8QUU77UVtba5KSkszjjz9u9u7dG9q+PNUez5z2Y6BE+tjFaS8S/RzqtB+JfB41xpjEusLJor1792r37t2hx+ecc47q6+t17733Kjs7W+PGjdO8efNOuCw1EQzsxZ133qmenh5VV1frwQcf1NixY3Xdddfp0UcfjWGVkfHRRx/p1ltv1f79+zV+/HhdffXVampq0qRJkyQN7kVWVpZef/11PfDAA3r88ceVkZGhX/3qVwmzzNZpP37zm9+ov79fixcv1uLFi0PjCxYs0DPPPGO7/Ihz2o9E5rQXiX4OddqPRD6PSpLLmESYvwEAAPGCaz4AAIBVhA8AAGAV4QMAAFhF+AAAAFYRPgAAgFWEDwAAYBXhAwAAWEX4AAAAVhE+AACAVYQPAABgFeEDAABYRfgAAABW/T/+/UoFrBsKIAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(np.log10(y_train), bins=100);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can incorporate this in our pipeline using `sklearn`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:48.999550Z",
     "iopub.status.busy": "2024-05-29T14:52:48.997567Z",
     "iopub.status.idle": "2024-05-29T14:52:49.007335Z",
     "shell.execute_reply": "2024-05-29T14:52:49.004957Z",
     "shell.execute_reply.started": "2024-05-29T14:52:48.999470Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.compose import TransformedTargetRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:49.013686Z",
     "iopub.status.busy": "2024-05-29T14:52:49.011813Z",
     "iopub.status.idle": "2024-05-29T14:52:49.027082Z",
     "shell.execute_reply": "2024-05-29T14:52:49.023899Z",
     "shell.execute_reply.started": "2024-05-29T14:52:49.013614Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "ttr = TransformedTargetRegressor(\n",
    "    Ridge(alpha=best_alpha), func=np.log1p, inverse_func=np.expm1\n",
    ") # transformer for log transforming the target\n",
    "ttr_pipe = make_pipeline(preprocessor, ttr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:49.035364Z",
     "iopub.status.busy": "2024-05-29T14:52:49.031013Z",
     "iopub.status.idle": "2024-05-29T14:52:49.386280Z",
     "shell.execute_reply": "2024-05-29T14:52:49.384260Z",
     "shell.execute_reply.started": "2024-05-29T14:52:49.035220Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {color: black;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;columntransformer&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                                 (&#x27;pipeline-1&#x27;,\n",
       "                                                  Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                                   SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                                  (&#x27;standardscaler&#x27;,\n",
       "                                                                   StandardScaler())]),\n",
       "                                                  [&#x27;BedroomAbvGr&#x27;,\n",
       "                                                   &#x27;KitchenAbvGr&#x27;,\n",
       "                                                   &#x27;LotFrontage&#x27;, &#x27;LotArea&#x27;,\n",
       "                                                   &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                                   &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;,\n",
       "                                                   &#x27;MasVnrArea&#x27;, &#x27;BsmtFinSF1&#x27;,\n",
       "                                                   &#x27;Bs...\n",
       "                                                   &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;,\n",
       "                                                   &#x27;PavedDrive&#x27;, &#x27;BldgType&#x27;,\n",
       "                                                   &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                                   &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;,\n",
       "                                                   &#x27;CentralAir&#x27;, &#x27;RoofMatl&#x27;,\n",
       "                                                   &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                                   &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;,\n",
       "                                                   &#x27;Condition2&#x27;,\n",
       "                                                   &#x27;SaleCondition&#x27;,\n",
       "                                                   &#x27;Foundation&#x27;,\n",
       "                                                   &#x27;GarageType&#x27;])])),\n",
       "                (&#x27;transformedtargetregressor&#x27;,\n",
       "                 TransformedTargetRegressor(func=&lt;ufunc &#x27;log1p&#x27;&gt;,\n",
       "                                            inverse_func=&lt;ufunc &#x27;expm1&#x27;&gt;,\n",
       "                                            regressor=Ridge(alpha=100.0)))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-31\" type=\"checkbox\" ><label for=\"sk-estimator-id-31\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;columntransformer&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                                 (&#x27;pipeline-1&#x27;,\n",
       "                                                  Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                                   SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                                  (&#x27;standardscaler&#x27;,\n",
       "                                                                   StandardScaler())]),\n",
       "                                                  [&#x27;BedroomAbvGr&#x27;,\n",
       "                                                   &#x27;KitchenAbvGr&#x27;,\n",
       "                                                   &#x27;LotFrontage&#x27;, &#x27;LotArea&#x27;,\n",
       "                                                   &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                                   &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;,\n",
       "                                                   &#x27;MasVnrArea&#x27;, &#x27;BsmtFinSF1&#x27;,\n",
       "                                                   &#x27;Bs...\n",
       "                                                   &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;,\n",
       "                                                   &#x27;PavedDrive&#x27;, &#x27;BldgType&#x27;,\n",
       "                                                   &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                                   &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;,\n",
       "                                                   &#x27;CentralAir&#x27;, &#x27;RoofMatl&#x27;,\n",
       "                                                   &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                                   &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;,\n",
       "                                                   &#x27;Condition2&#x27;,\n",
       "                                                   &#x27;SaleCondition&#x27;,\n",
       "                                                   &#x27;Foundation&#x27;,\n",
       "                                                   &#x27;GarageType&#x27;])])),\n",
       "                (&#x27;transformedtargetregressor&#x27;,\n",
       "                 TransformedTargetRegressor(func=&lt;ufunc &#x27;log1p&#x27;&gt;,\n",
       "                                            inverse_func=&lt;ufunc &#x27;expm1&#x27;&gt;,\n",
       "                                            regressor=Ridge(alpha=100.0)))])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-32\" type=\"checkbox\" ><label for=\"sk-estimator-id-32\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">columntransformer: ColumnTransformer</label><div class=\"sk-toggleable__content\"><pre>ColumnTransformer(transformers=[(&#x27;drop&#x27;, &#x27;drop&#x27;, [&#x27;Id&#x27;]),\n",
       "                                (&#x27;pipeline-1&#x27;,\n",
       "                                 Pipeline(steps=[(&#x27;simpleimputer&#x27;,\n",
       "                                                  SimpleImputer(strategy=&#x27;median&#x27;)),\n",
       "                                                 (&#x27;standardscaler&#x27;,\n",
       "                                                  StandardScaler())]),\n",
       "                                 [&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;,\n",
       "                                  &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;,\n",
       "                                  &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;,\n",
       "                                  &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;,\n",
       "                                  &#x27;TotalBsmtSF&#x27;, &#x27;...\n",
       "                                 [&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;,\n",
       "                                  &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;,\n",
       "                                  &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;,\n",
       "                                  &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;,\n",
       "                                  &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;,\n",
       "                                  &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;,\n",
       "                                  &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;,\n",
       "                                  &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;,\n",
       "                                  &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;,\n",
       "                                  &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;,\n",
       "                                  &#x27;GarageType&#x27;])])</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-33\" type=\"checkbox\" ><label for=\"sk-estimator-id-33\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Id&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-34\" type=\"checkbox\" ><label for=\"sk-estimator-id-34\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">drop</label><div class=\"sk-toggleable__content\"><pre>drop</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-35\" type=\"checkbox\" ><label for=\"sk-estimator-id-35\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-1</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BedroomAbvGr&#x27;, &#x27;KitchenAbvGr&#x27;, &#x27;LotFrontage&#x27;, &#x27;LotArea&#x27;, &#x27;OverallQual&#x27;, &#x27;OverallCond&#x27;, &#x27;YearBuilt&#x27;, &#x27;YearRemodAdd&#x27;, &#x27;MasVnrArea&#x27;, &#x27;BsmtFinSF1&#x27;, &#x27;BsmtFinSF2&#x27;, &#x27;BsmtUnfSF&#x27;, &#x27;TotalBsmtSF&#x27;, &#x27;1stFlrSF&#x27;, &#x27;2ndFlrSF&#x27;, &#x27;LowQualFinSF&#x27;, &#x27;GrLivArea&#x27;, &#x27;BsmtFullBath&#x27;, &#x27;BsmtHalfBath&#x27;, &#x27;FullBath&#x27;, &#x27;HalfBath&#x27;, &#x27;TotRmsAbvGrd&#x27;, &#x27;Fireplaces&#x27;, &#x27;GarageYrBlt&#x27;, &#x27;GarageCars&#x27;, &#x27;GarageArea&#x27;, &#x27;WoodDeckSF&#x27;, &#x27;OpenPorchSF&#x27;, &#x27;EnclosedPorch&#x27;, &#x27;3SsnPorch&#x27;, &#x27;ScreenPorch&#x27;, &#x27;PoolArea&#x27;, &#x27;MiscVal&#x27;, &#x27;YrSold&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-36\" type=\"checkbox\" ><label for=\"sk-estimator-id-36\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-37\" type=\"checkbox\" ><label for=\"sk-estimator-id-37\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">StandardScaler</label><div class=\"sk-toggleable__content\"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-38\" type=\"checkbox\" ><label for=\"sk-estimator-id-38\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-2</label><div class=\"sk-toggleable__content\"><pre>[&#x27;ExterQual&#x27;, &#x27;ExterCond&#x27;, &#x27;BsmtQual&#x27;, &#x27;BsmtCond&#x27;, &#x27;HeatingQC&#x27;, &#x27;KitchenQual&#x27;, &#x27;FireplaceQu&#x27;, &#x27;GarageQual&#x27;, &#x27;GarageCond&#x27;, &#x27;PoolQC&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-39\" type=\"checkbox\" ><label for=\"sk-estimator-id-39\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-40\" type=\"checkbox\" ><label for=\"sk-estimator-id-40\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;],\n",
       "                           [&#x27;Po&#x27;, &#x27;Fa&#x27;, &#x27;TA&#x27;, &#x27;Gd&#x27;, &#x27;Ex&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-41\" type=\"checkbox\" ><label for=\"sk-estimator-id-41\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-3</label><div class=\"sk-toggleable__content\"><pre>[&#x27;BsmtExposure&#x27;, &#x27;BsmtFinType1&#x27;, &#x27;BsmtFinType2&#x27;, &#x27;Functional&#x27;, &#x27;Fence&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-42\" type=\"checkbox\" ><label for=\"sk-estimator-id-42\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(strategy=&#x27;most_frequent&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-43\" type=\"checkbox\" ><label for=\"sk-estimator-id-43\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OrdinalEncoder</label><div class=\"sk-toggleable__content\"><pre>OrdinalEncoder(categories=[[&#x27;NA&#x27;, &#x27;No&#x27;, &#x27;Mn&#x27;, &#x27;Av&#x27;, &#x27;Gd&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;Unf&#x27;, &#x27;LwQ&#x27;, &#x27;Rec&#x27;, &#x27;BLQ&#x27;, &#x27;ALQ&#x27;, &#x27;GLQ&#x27;],\n",
       "                           [&#x27;Sal&#x27;, &#x27;Sev&#x27;, &#x27;Maj2&#x27;, &#x27;Maj1&#x27;, &#x27;Mod&#x27;, &#x27;Min2&#x27;, &#x27;Min1&#x27;,\n",
       "                            &#x27;Typ&#x27;],\n",
       "                           [&#x27;NA&#x27;, &#x27;MnWw&#x27;, &#x27;GdWo&#x27;, &#x27;MnPrv&#x27;, &#x27;GdPrv&#x27;]])</pre></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-44\" type=\"checkbox\" ><label for=\"sk-estimator-id-44\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">pipeline-4</label><div class=\"sk-toggleable__content\"><pre>[&#x27;Alley&#x27;, &#x27;MSSubClass&#x27;, &#x27;Neighborhood&#x27;, &#x27;Utilities&#x27;, &#x27;Exterior1st&#x27;, &#x27;MasVnrType&#x27;, &#x27;LotShape&#x27;, &#x27;LandContour&#x27;, &#x27;MoSold&#x27;, &#x27;MSZoning&#x27;, &#x27;Street&#x27;, &#x27;GarageFinish&#x27;, &#x27;HouseStyle&#x27;, &#x27;LotConfig&#x27;, &#x27;PavedDrive&#x27;, &#x27;BldgType&#x27;, &#x27;LandSlope&#x27;, &#x27;Condition1&#x27;, &#x27;MiscFeature&#x27;, &#x27;Electrical&#x27;, &#x27;CentralAir&#x27;, &#x27;RoofMatl&#x27;, &#x27;Heating&#x27;, &#x27;RoofStyle&#x27;, &#x27;SaleType&#x27;, &#x27;Exterior2nd&#x27;, &#x27;Condition2&#x27;, &#x27;SaleCondition&#x27;, &#x27;Foundation&#x27;, &#x27;GarageType&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-45\" type=\"checkbox\" ><label for=\"sk-estimator-id-45\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SimpleImputer</label><div class=\"sk-toggleable__content\"><pre>SimpleImputer(fill_value=&#x27;missing&#x27;, strategy=&#x27;constant&#x27;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-46\" type=\"checkbox\" ><label for=\"sk-estimator-id-46\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder(handle_unknown=&#x27;ignore&#x27;, sparse_output=False)</pre></div></div></div></div></div></div></div></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-47\" type=\"checkbox\" ><label for=\"sk-estimator-id-47\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">transformedtargetregressor: TransformedTargetRegressor</label><div class=\"sk-toggleable__content\"><pre>TransformedTargetRegressor(func=&lt;ufunc &#x27;log1p&#x27;&gt;, inverse_func=&lt;ufunc &#x27;expm1&#x27;&gt;,\n",
       "                           regressor=Ridge(alpha=100.0))</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-48\" type=\"checkbox\" ><label for=\"sk-estimator-id-48\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">regressor: Ridge</label><div class=\"sk-toggleable__content\"><pre>Ridge(alpha=100.0)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-49\" type=\"checkbox\" ><label for=\"sk-estimator-id-49\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Ridge</label><div class=\"sk-toggleable__content\"><pre>Ridge(alpha=100.0)</pre></div></div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('columntransformer',\n",
       "                 ColumnTransformer(transformers=[('drop', 'drop', ['Id']),\n",
       "                                                 ('pipeline-1',\n",
       "                                                  Pipeline(steps=[('simpleimputer',\n",
       "                                                                   SimpleImputer(strategy='median')),\n",
       "                                                                  ('standardscaler',\n",
       "                                                                   StandardScaler())]),\n",
       "                                                  ['BedroomAbvGr',\n",
       "                                                   'KitchenAbvGr',\n",
       "                                                   'LotFrontage', 'LotArea',\n",
       "                                                   'OverallQual', 'OverallCond',\n",
       "                                                   'YearBuilt', 'YearRemodAdd',\n",
       "                                                   'MasVnrArea', 'BsmtFinSF1',\n",
       "                                                   'Bs...\n",
       "                                                   'HouseStyle', 'LotConfig',\n",
       "                                                   'PavedDrive', 'BldgType',\n",
       "                                                   'LandSlope', 'Condition1',\n",
       "                                                   'MiscFeature', 'Electrical',\n",
       "                                                   'CentralAir', 'RoofMatl',\n",
       "                                                   'Heating', 'RoofStyle',\n",
       "                                                   'SaleType', 'Exterior2nd',\n",
       "                                                   'Condition2',\n",
       "                                                   'SaleCondition',\n",
       "                                                   'Foundation',\n",
       "                                                   'GarageType'])])),\n",
       "                ('transformedtargetregressor',\n",
       "                 TransformedTargetRegressor(func=<ufunc 'log1p'>,\n",
       "                                            inverse_func=<ufunc 'expm1'>,\n",
       "                                            regressor=Ridge(alpha=100.0)))])"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ttr_pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Question for you: Why can't we incorporate preprocessing targets in our column transformer? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:49.389550Z",
     "iopub.status.busy": "2024-05-29T14:52:49.388397Z",
     "iopub.status.idle": "2024-05-29T14:52:49.653051Z",
     "shell.execute_reply": "2024-05-29T14:52:49.642316Z",
     "shell.execute_reply.started": "2024-05-29T14:52:49.389491Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "ttr_pipe.fit(X_train, y_train); # y_train automatically transformed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:49.655963Z",
     "iopub.status.busy": "2024-05-29T14:52:49.654983Z",
     "iopub.status.idle": "2024-05-29T14:52:49.854491Z",
     "shell.execute_reply": "2024-05-29T14:52:49.840604Z",
     "shell.execute_reply.started": "2024-05-29T14:52:49.655900Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([221329.43833466, 170670.12761659, 182639.31984311, ...,\n",
       "       248609.60023631, 132158.15026771, 133270.71866979])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ttr_pipe.predict(X_train)  # predictions automatically un-transformed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:49.856478Z",
     "iopub.status.busy": "2024-05-29T14:52:49.855903Z",
     "iopub.status.idle": "2024-05-29T14:52:50.007899Z",
     "shell.execute_reply": "2024-05-29T14:52:50.006182Z",
     "shell.execute_reply.started": "2024-05-29T14:52:49.856424Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.07808506982896254"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_percentage_error(y_test, ttr_pipe.predict(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "We reduced MAPE from ~10% to ~8% with this trick! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Does `.fit()` know we care about MAPE?\n",
    "- No, it doesn't. Why are we minimizing MSE (or something similar) if we care about MAPE??\n",
    "- When minimizing MSE, the expensive houses will dominate because they have the biggest error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Different scoring functions with `cross_validate`\n",
    "\n",
    "- Let's try using MSE instead of the default $R^2$ score. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:50.010245Z",
     "iopub.status.busy": "2024-05-29T14:52:50.009626Z",
     "iopub.status.idle": "2024-05-29T14:52:52.320823Z",
     "shell.execute_reply": "2024-05-29T14:52:52.318970Z",
     "shell.execute_reply.started": "2024-05-29T14:52:50.010176Z"
    },
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.196401</td>\n",
       "      <td>0.133432</td>\n",
       "      <td>-7.055915e+08</td>\n",
       "      <td>-9.379875e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.181491</td>\n",
       "      <td>0.082139</td>\n",
       "      <td>-1.239944e+09</td>\n",
       "      <td>-8.264283e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.189155</td>\n",
       "      <td>0.133311</td>\n",
       "      <td>-1.125440e+09</td>\n",
       "      <td>-8.758062e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.159395</td>\n",
       "      <td>0.107427</td>\n",
       "      <td>-9.801219e+08</td>\n",
       "      <td>-8.849438e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.169679</td>\n",
       "      <td>0.082599</td>\n",
       "      <td>-2.267612e+09</td>\n",
       "      <td>-7.395282e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time    test_score   train_score\n",
       "0  0.196401    0.133432 -7.055915e+08 -9.379875e+08\n",
       "1  0.181491    0.082139 -1.239944e+09 -8.264283e+08\n",
       "2  0.189155    0.133311 -1.125440e+09 -8.758062e+08\n",
       "3  0.159395    0.107427 -9.801219e+08 -8.849438e+08\n",
       "4  0.169679    0.082599 -2.267612e+09 -7.395282e+08"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(\n",
    "    cross_validate(\n",
    "        ridge_tuned,\n",
    "        X_train,\n",
    "        y_train,\n",
    "        return_train_score=True,\n",
    "        scoring=\"neg_mean_squared_error\",\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:52.327277Z",
     "iopub.status.busy": "2024-05-29T14:52:52.322740Z",
     "iopub.status.idle": "2024-05-29T14:52:54.770689Z",
     "shell.execute_reply": "2024-05-29T14:52:54.768470Z",
     "shell.execute_reply.started": "2024-05-29T14:52:52.327206Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fit_time</th>\n",
       "      <th>score_time</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.197293</td>\n",
       "      <td>0.107038</td>\n",
       "      <td>-0.096960</td>\n",
       "      <td>-0.104070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.201161</td>\n",
       "      <td>0.086505</td>\n",
       "      <td>-0.107985</td>\n",
       "      <td>-0.099685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.316449</td>\n",
       "      <td>0.084198</td>\n",
       "      <td>-0.118347</td>\n",
       "      <td>-0.101796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.292112</td>\n",
       "      <td>0.086380</td>\n",
       "      <td>-0.107947</td>\n",
       "      <td>-0.102435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.166118</td>\n",
       "      <td>0.087939</td>\n",
       "      <td>-0.121985</td>\n",
       "      <td>-0.098307</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fit_time  score_time  test_score  train_score\n",
       "0  0.197293    0.107038   -0.096960    -0.104070\n",
       "1  0.201161    0.086505   -0.107985    -0.099685\n",
       "2  0.316449    0.084198   -0.118347    -0.101796\n",
       "3  0.292112    0.086380   -0.107947    -0.102435\n",
       "4  0.166118    0.087939   -0.121985    -0.098307"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make a scorer function that we can pass into cross-validation\n",
    "mape_scorer = make_scorer(my_mape, greater_is_better=False)\n",
    "\n",
    "pd.DataFrame(\n",
    "    cross_validate(\n",
    "        ridge_tuned, X_train, y_train, return_train_score=True, scoring=mape_scorer\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:54.777470Z",
     "iopub.status.busy": "2024-05-29T14:52:54.774508Z",
     "iopub.status.idle": "2024-05-29T14:52:54.815687Z",
     "shell.execute_reply": "2024-05-29T14:52:54.812286Z",
     "shell.execute_reply.started": "2024-05-29T14:52:54.777324Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['accuracy',\n",
       " 'adjusted_mutual_info_score',\n",
       " 'adjusted_rand_score',\n",
       " 'average_precision',\n",
       " 'balanced_accuracy',\n",
       " 'completeness_score',\n",
       " 'explained_variance',\n",
       " 'f1',\n",
       " 'f1_macro',\n",
       " 'f1_micro',\n",
       " 'f1_samples',\n",
       " 'f1_weighted',\n",
       " 'fowlkes_mallows_score',\n",
       " 'homogeneity_score',\n",
       " 'jaccard',\n",
       " 'jaccard_macro',\n",
       " 'jaccard_micro',\n",
       " 'jaccard_samples',\n",
       " 'jaccard_weighted',\n",
       " 'matthews_corrcoef',\n",
       " 'max_error',\n",
       " 'mutual_info_score',\n",
       " 'neg_brier_score',\n",
       " 'neg_log_loss',\n",
       " 'neg_mean_absolute_error',\n",
       " 'neg_mean_absolute_percentage_error',\n",
       " 'neg_mean_gamma_deviance',\n",
       " 'neg_mean_poisson_deviance',\n",
       " 'neg_mean_squared_error',\n",
       " 'neg_mean_squared_log_error',\n",
       " 'neg_median_absolute_error',\n",
       " 'neg_negative_likelihood_ratio',\n",
       " 'neg_root_mean_squared_error',\n",
       " 'normalized_mutual_info_score',\n",
       " 'positive_likelihood_ratio',\n",
       " 'precision',\n",
       " 'precision_macro',\n",
       " 'precision_micro',\n",
       " 'precision_samples',\n",
       " 'precision_weighted',\n",
       " 'r2',\n",
       " 'rand_score',\n",
       " 'recall',\n",
       " 'recall_macro',\n",
       " 'recall_micro',\n",
       " 'recall_samples',\n",
       " 'recall_weighted',\n",
       " 'roc_auc',\n",
       " 'roc_auc_ovo',\n",
       " 'roc_auc_ovo_weighted',\n",
       " 'roc_auc_ovr',\n",
       " 'roc_auc_ovr_weighted',\n",
       " 'top_k_accuracy',\n",
       " 'v_measure_score']"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sklearn\n",
    "sklearn.metrics.get_scorer_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are finding `greater_is_better=False` argument confusing, here is the documentation: \n",
    "\n",
    "> greater_is_better(bool), default=True\n",
    "Whether score_func is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the score_func.\n",
    "\n",
    "Since our custom scorer `mape` gives an error and not a score, I'm passing `False` to it and it'll sign flip so that we can interpret bigger numbers as better performance.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:54.826258Z",
     "iopub.status.busy": "2024-05-29T14:52:54.823097Z",
     "iopub.status.idle": "2024-05-29T14:52:54.869003Z",
     "shell.execute_reply": "2024-05-29T14:52:54.864018Z",
     "shell.execute_reply.started": "2024-05-29T14:52:54.826183Z"
    }
   },
   "outputs": [],
   "source": [
    "# ?make_scorer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:54.873232Z",
     "iopub.status.busy": "2024-05-29T14:52:54.872368Z",
     "iopub.status.idle": "2024-05-29T14:52:57.114924Z",
     "shell.execute_reply": "2024-05-29T14:52:57.111100Z",
     "shell.execute_reply.started": "2024-05-29T14:52:54.873187Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fit_time</th>\n",
       "      <td>1.110597e-01</td>\n",
       "      <td>1.562581e-01</td>\n",
       "      <td>2.507291e-01</td>\n",
       "      <td>2.515426e-01</td>\n",
       "      <td>1.587946e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score_time</th>\n",
       "      <td>1.191244e-01</td>\n",
       "      <td>1.017010e-01</td>\n",
       "      <td>8.761001e-02</td>\n",
       "      <td>8.469224e-02</td>\n",
       "      <td>9.532523e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_r2</th>\n",
       "      <td>8.669805e-01</td>\n",
       "      <td>8.200326e-01</td>\n",
       "      <td>8.262156e-01</td>\n",
       "      <td>8.514598e-01</td>\n",
       "      <td>6.110915e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_r2</th>\n",
       "      <td>8.551862e-01</td>\n",
       "      <td>8.636849e-01</td>\n",
       "      <td>8.580539e-01</td>\n",
       "      <td>8.561645e-01</td>\n",
       "      <td>8.834356e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_mape_scorer</th>\n",
       "      <td>-9.696034e-02</td>\n",
       "      <td>-1.079852e-01</td>\n",
       "      <td>-1.183471e-01</td>\n",
       "      <td>-1.079471e-01</td>\n",
       "      <td>-1.219845e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_mape_scorer</th>\n",
       "      <td>-1.040698e-01</td>\n",
       "      <td>-9.968514e-02</td>\n",
       "      <td>-1.017959e-01</td>\n",
       "      <td>-1.024351e-01</td>\n",
       "      <td>-9.830712e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_sklearn MAPE</th>\n",
       "      <td>-9.696034e-02</td>\n",
       "      <td>-1.079852e-01</td>\n",
       "      <td>-1.183471e-01</td>\n",
       "      <td>-1.079471e-01</td>\n",
       "      <td>-1.219845e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_sklearn MAPE</th>\n",
       "      <td>-1.040698e-01</td>\n",
       "      <td>-9.968514e-02</td>\n",
       "      <td>-1.017959e-01</td>\n",
       "      <td>-1.024351e-01</td>\n",
       "      <td>-9.830712e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_neg_root_mean_square_error</th>\n",
       "      <td>-2.656297e+04</td>\n",
       "      <td>-3.521284e+04</td>\n",
       "      <td>-3.354759e+04</td>\n",
       "      <td>-3.130690e+04</td>\n",
       "      <td>-4.761945e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_neg_root_mean_square_error</th>\n",
       "      <td>-3.062658e+04</td>\n",
       "      <td>-2.874767e+04</td>\n",
       "      <td>-2.959402e+04</td>\n",
       "      <td>-2.974801e+04</td>\n",
       "      <td>-2.719427e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_neg_mean_squared_error</th>\n",
       "      <td>-7.055915e+08</td>\n",
       "      <td>-1.239944e+09</td>\n",
       "      <td>-1.125440e+09</td>\n",
       "      <td>-9.801219e+08</td>\n",
       "      <td>-2.267612e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_neg_mean_squared_error</th>\n",
       "      <td>-9.379875e+08</td>\n",
       "      <td>-8.264283e+08</td>\n",
       "      <td>-8.758062e+08</td>\n",
       "      <td>-8.849438e+08</td>\n",
       "      <td>-7.395282e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             0             1             2  \\\n",
       "fit_time                          1.110597e-01  1.562581e-01  2.507291e-01   \n",
       "score_time                        1.191244e-01  1.017010e-01  8.761001e-02   \n",
       "test_r2                           8.669805e-01  8.200326e-01  8.262156e-01   \n",
       "train_r2                          8.551862e-01  8.636849e-01  8.580539e-01   \n",
       "test_mape_scorer                 -9.696034e-02 -1.079852e-01 -1.183471e-01   \n",
       "train_mape_scorer                -1.040698e-01 -9.968514e-02 -1.017959e-01   \n",
       "test_sklearn MAPE                -9.696034e-02 -1.079852e-01 -1.183471e-01   \n",
       "train_sklearn MAPE               -1.040698e-01 -9.968514e-02 -1.017959e-01   \n",
       "test_neg_root_mean_square_error  -2.656297e+04 -3.521284e+04 -3.354759e+04   \n",
       "train_neg_root_mean_square_error -3.062658e+04 -2.874767e+04 -2.959402e+04   \n",
       "test_neg_mean_squared_error      -7.055915e+08 -1.239944e+09 -1.125440e+09   \n",
       "train_neg_mean_squared_error     -9.379875e+08 -8.264283e+08 -8.758062e+08   \n",
       "\n",
       "                                             3             4  \n",
       "fit_time                          2.515426e-01  1.587946e-01  \n",
       "score_time                        8.469224e-02  9.532523e-02  \n",
       "test_r2                           8.514598e-01  6.110915e-01  \n",
       "train_r2                          8.561645e-01  8.834356e-01  \n",
       "test_mape_scorer                 -1.079471e-01 -1.219845e-01  \n",
       "train_mape_scorer                -1.024351e-01 -9.830712e-02  \n",
       "test_sklearn MAPE                -1.079471e-01 -1.219845e-01  \n",
       "train_sklearn MAPE               -1.024351e-01 -9.830712e-02  \n",
       "test_neg_root_mean_square_error  -3.130690e+04 -4.761945e+04  \n",
       "train_neg_root_mean_square_error -2.974801e+04 -2.719427e+04  \n",
       "test_neg_mean_squared_error      -9.801219e+08 -2.267612e+09  \n",
       "train_neg_mean_squared_error     -8.849438e+08 -7.395282e+08  "
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scoring = {\n",
    "    \"r2\": \"r2\",\n",
    "    \"mape_scorer\": mape_scorer, # just for demonstration for a custom scorer\n",
    "    \"sklearn MAPE\": \"neg_mean_absolute_percentage_error\",\n",
    "    \"neg_root_mean_square_error\": \"neg_root_mean_squared_error\",\n",
    "    \"neg_mean_squared_error\": \"neg_mean_squared_error\",\n",
    "}\n",
    "\n",
    "pd.DataFrame(\n",
    "    cross_validate(\n",
    "        ridge_tuned, X_train, y_train, return_train_score=True, scoring=scoring\n",
    "    )\n",
    ").T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Are we getting the same `alpha` with mape? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:52:57.127166Z",
     "iopub.status.busy": "2024-05-29T14:52:57.125415Z",
     "iopub.status.idle": "2024-05-29T14:53:01.248197Z",
     "shell.execute_reply": "2024-05-29T14:53:01.246366Z",
     "shell.execute_reply.started": "2024-05-29T14:52:57.127095Z"
    }
   },
   "outputs": [],
   "source": [
    "param_grid = {\"ridge__alpha\": 10.0 ** np.arange(-6, 6, 1)}\n",
    "\n",
    "pipe_ridge = make_pipeline(preprocessor, Ridge())\n",
    "\n",
    "search = GridSearchCV(\n",
    "    pipe_ridge, param_grid, return_train_score=True, n_jobs=-1, scoring=mape_scorer\n",
    ")\n",
    "search.fit(X_train, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:53:01.253749Z",
     "iopub.status.busy": "2024-05-29T14:53:01.251571Z",
     "iopub.status.idle": "2024-05-29T14:53:01.416809Z",
     "shell.execute_reply": "2024-05-29T14:53:01.413481Z",
     "shell.execute_reply.started": "2024-05-29T14:53:01.253632Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best hyperparameter values:  {'ridge__alpha': 100.0}\n",
      "Best score: -0.111\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>rank_test_score</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mean_train_score</th>\n",
       "      <td>-0.101259</td>\n",
       "      <td>-0.111543</td>\n",
       "      <td>-0.096462</td>\n",
       "      <td>-0.090785</td>\n",
       "      <td>-0.085698</td>\n",
       "      <td>-0.085508</td>\n",
       "      <td>-0.085546</td>\n",
       "      <td>-0.08555</td>\n",
       "      <td>-0.085551</td>\n",
       "      <td>-0.085551</td>\n",
       "      <td>-0.203421</td>\n",
       "      <td>-0.33213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean_test_score</th>\n",
       "      <td>-0.110645</td>\n",
       "      <td>-0.115401</td>\n",
       "      <td>-0.116497</td>\n",
       "      <td>-0.122331</td>\n",
       "      <td>-0.125644</td>\n",
       "      <td>-0.127242</td>\n",
       "      <td>-0.127418</td>\n",
       "      <td>-0.127439</td>\n",
       "      <td>-0.127441</td>\n",
       "      <td>-0.127441</td>\n",
       "      <td>-0.204692</td>\n",
       "      <td>-0.33271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>param_ridge__alpha</th>\n",
       "      <td>100.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>100000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean_fit_time</th>\n",
       "      <td>0.219801</td>\n",
       "      <td>0.271209</td>\n",
       "      <td>0.262587</td>\n",
       "      <td>0.199674</td>\n",
       "      <td>0.301716</td>\n",
       "      <td>0.207903</td>\n",
       "      <td>0.216318</td>\n",
       "      <td>0.223802</td>\n",
       "      <td>0.288285</td>\n",
       "      <td>0.23024</td>\n",
       "      <td>0.26242</td>\n",
       "      <td>0.180399</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "rank_test_score           1         2         3         4         5   \\\n",
       "mean_train_score   -0.101259 -0.111543 -0.096462 -0.090785 -0.085698   \n",
       "mean_test_score    -0.110645 -0.115401 -0.116497 -0.122331 -0.125644   \n",
       "param_ridge__alpha     100.0    1000.0      10.0       1.0       0.1   \n",
       "mean_fit_time       0.219801  0.271209  0.262587  0.199674  0.301716   \n",
       "\n",
       "rank_test_score           6         7         8         9         10  \\\n",
       "mean_train_score   -0.085508 -0.085546  -0.08555 -0.085551 -0.085551   \n",
       "mean_test_score    -0.127242 -0.127418 -0.127439 -0.127441 -0.127441   \n",
       "param_ridge__alpha      0.01     0.001    0.0001   0.00001  0.000001   \n",
       "mean_fit_time       0.207903  0.216318  0.223802  0.288285   0.23024   \n",
       "\n",
       "rank_test_score           11        12  \n",
       "mean_train_score   -0.203421  -0.33213  \n",
       "mean_test_score    -0.204692  -0.33271  \n",
       "param_ridge__alpha   10000.0  100000.0  \n",
       "mean_fit_time        0.26242  0.180399  "
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Best hyperparameter values: \", search.best_params_)\n",
    "print(\"Best score: %0.3f\" % (search.best_score_))\n",
    "pd.DataFrame(search.cv_results_)[\n",
    "    [\n",
    "        \"mean_train_score\",\n",
    "        \"mean_test_score\",\n",
    "        \"param_ridge__alpha\",\n",
    "        \"mean_fit_time\",\n",
    "        \"rank_test_score\",\n",
    "    ]\n",
    "].set_index(\"rank_test_score\").sort_index().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using multiple metrics in `GridSearchCV` or `RandomizedSearchCV` \n",
    "\n",
    "- We could use multiple metrics with `GridSearchCV` or `RandomizedSearchCV`. \n",
    "- But if you do so, you need to set `refit` to the metric (string) for which the `best_params_` will be found and used to build the `best_estimator_` on the whole dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:53:01.420948Z",
     "iopub.status.busy": "2024-05-29T14:53:01.419661Z",
     "iopub.status.idle": "2024-05-29T14:53:08.039299Z",
     "shell.execute_reply": "2024-05-29T14:53:08.019290Z",
     "shell.execute_reply.started": "2024-05-29T14:53:01.420867Z"
    }
   },
   "outputs": [],
   "source": [
    "search_multi = GridSearchCV(\n",
    "    pipe_ridge,\n",
    "    param_grid,\n",
    "    return_train_score=True,\n",
    "    n_jobs=-1,\n",
    "    scoring=scoring,\n",
    "    refit=\"sklearn MAPE\",\n",
    ")\n",
    "search_multi.fit(X_train, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:53:08.067025Z",
     "iopub.status.busy": "2024-05-29T14:53:08.053241Z",
     "iopub.status.idle": "2024-05-29T14:53:08.291864Z",
     "shell.execute_reply": "2024-05-29T14:53:08.288641Z",
     "shell.execute_reply.started": "2024-05-29T14:53:08.066910Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best hyperparameter values:  {'ridge__alpha': 100.0}\n",
      "Best score: -0.111\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_ridge__alpha</th>\n",
       "      <th>params</th>\n",
       "      <th>split0_test_r2</th>\n",
       "      <th>split1_test_r2</th>\n",
       "      <th>split2_test_r2</th>\n",
       "      <th>split3_test_r2</th>\n",
       "      <th>...</th>\n",
       "      <th>mean_test_neg_mean_squared_error</th>\n",
       "      <th>std_test_neg_mean_squared_error</th>\n",
       "      <th>rank_test_neg_mean_squared_error</th>\n",
       "      <th>split0_train_neg_mean_squared_error</th>\n",
       "      <th>split1_train_neg_mean_squared_error</th>\n",
       "      <th>split2_train_neg_mean_squared_error</th>\n",
       "      <th>split3_train_neg_mean_squared_error</th>\n",
       "      <th>split4_train_neg_mean_squared_error</th>\n",
       "      <th>mean_train_neg_mean_squared_error</th>\n",
       "      <th>std_train_neg_mean_squared_error</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rank_test_mape_scorer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.362211</td>\n",
       "      <td>0.133726</td>\n",
       "      <td>0.144514</td>\n",
       "      <td>0.035490</td>\n",
       "      <td>100.0</td>\n",
       "      <td>{'ridge__alpha': 100.0}</td>\n",
       "      <td>0.866980</td>\n",
       "      <td>0.820033</td>\n",
       "      <td>0.826216</td>\n",
       "      <td>0.851460</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.263742e+09</td>\n",
       "      <td>5.328077e+08</td>\n",
       "      <td>1</td>\n",
       "      <td>-9.379875e+08</td>\n",
       "      <td>-8.264283e+08</td>\n",
       "      <td>-8.758062e+08</td>\n",
       "      <td>-8.849438e+08</td>\n",
       "      <td>-7.395282e+08</td>\n",
       "      <td>-8.529388e+08</td>\n",
       "      <td>6.685103e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.379797</td>\n",
       "      <td>0.089024</td>\n",
       "      <td>0.155092</td>\n",
       "      <td>0.078415</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>{'ridge__alpha': 1000.0}</td>\n",
       "      <td>0.827261</td>\n",
       "      <td>0.786555</td>\n",
       "      <td>0.802902</td>\n",
       "      <td>0.812272</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.362185e+09</td>\n",
       "      <td>3.262269e+08</td>\n",
       "      <td>3</td>\n",
       "      <td>-1.282021e+09</td>\n",
       "      <td>-1.156311e+09</td>\n",
       "      <td>-1.207814e+09</td>\n",
       "      <td>-1.225301e+09</td>\n",
       "      <td>-1.029950e+09</td>\n",
       "      <td>-1.180279e+09</td>\n",
       "      <td>8.521743e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.408151</td>\n",
       "      <td>0.142118</td>\n",
       "      <td>0.145437</td>\n",
       "      <td>0.060656</td>\n",
       "      <td>10.0</td>\n",
       "      <td>{'ridge__alpha': 10.0}</td>\n",
       "      <td>0.859318</td>\n",
       "      <td>0.820564</td>\n",
       "      <td>0.832866</td>\n",
       "      <td>0.846650</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.283119e+09</td>\n",
       "      <td>5.511620e+08</td>\n",
       "      <td>2</td>\n",
       "      <td>-7.034979e+08</td>\n",
       "      <td>-6.276943e+08</td>\n",
       "      <td>-6.910456e+08</td>\n",
       "      <td>-6.701186e+08</td>\n",
       "      <td>-5.915714e+08</td>\n",
       "      <td>-6.567856e+08</td>\n",
       "      <td>4.155178e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.269104</td>\n",
       "      <td>0.025912</td>\n",
       "      <td>0.081145</td>\n",
       "      <td>0.015786</td>\n",
       "      <td>1.0</td>\n",
       "      <td>{'ridge__alpha': 1.0}</td>\n",
       "      <td>0.835749</td>\n",
       "      <td>0.810073</td>\n",
       "      <td>0.831611</td>\n",
       "      <td>0.843992</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.386071e+09</td>\n",
       "      <td>6.378971e+08</td>\n",
       "      <td>4</td>\n",
       "      <td>-5.394113e+08</td>\n",
       "      <td>-4.898703e+08</td>\n",
       "      <td>-5.405227e+08</td>\n",
       "      <td>-5.290961e+08</td>\n",
       "      <td>-5.046193e+08</td>\n",
       "      <td>-5.207039e+08</td>\n",
       "      <td>2.011269e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.379829</td>\n",
       "      <td>0.115560</td>\n",
       "      <td>0.134746</td>\n",
       "      <td>0.037908</td>\n",
       "      <td>0.1</td>\n",
       "      <td>{'ridge__alpha': 0.1}</td>\n",
       "      <td>0.821807</td>\n",
       "      <td>0.792828</td>\n",
       "      <td>0.805198</td>\n",
       "      <td>0.863941</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.465279e+09</td>\n",
       "      <td>6.931439e+08</td>\n",
       "      <td>5</td>\n",
       "      <td>-4.579559e+08</td>\n",
       "      <td>-4.009514e+08</td>\n",
       "      <td>-4.345386e+08</td>\n",
       "      <td>-4.515351e+08</td>\n",
       "      <td>-4.846260e+08</td>\n",
       "      <td>-4.459214e+08</td>\n",
       "      <td>2.766317e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.354040</td>\n",
       "      <td>0.079624</td>\n",
       "      <td>0.113333</td>\n",
       "      <td>0.040005</td>\n",
       "      <td>0.01</td>\n",
       "      <td>{'ridge__alpha': 0.01}</td>\n",
       "      <td>0.824884</td>\n",
       "      <td>0.782183</td>\n",
       "      <td>0.787933</td>\n",
       "      <td>0.868003</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.501304e+09</td>\n",
       "      <td>7.090376e+08</td>\n",
       "      <td>6</td>\n",
       "      <td>-4.522563e+08</td>\n",
       "      <td>-3.926733e+08</td>\n",
       "      <td>-4.251158e+08</td>\n",
       "      <td>-4.446284e+08</td>\n",
       "      <td>-4.837022e+08</td>\n",
       "      <td>-4.396752e+08</td>\n",
       "      <td>3.014081e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.327985</td>\n",
       "      <td>0.113027</td>\n",
       "      <td>0.116361</td>\n",
       "      <td>0.047817</td>\n",
       "      <td>0.001</td>\n",
       "      <td>{'ridge__alpha': 0.001}</td>\n",
       "      <td>0.826553</td>\n",
       "      <td>0.780497</td>\n",
       "      <td>0.785399</td>\n",
       "      <td>0.868281</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.505592e+09</td>\n",
       "      <td>7.117899e+08</td>\n",
       "      <td>7</td>\n",
       "      <td>-4.521578e+08</td>\n",
       "      <td>-3.925288e+08</td>\n",
       "      <td>-4.249618e+08</td>\n",
       "      <td>-4.445124e+08</td>\n",
       "      <td>-4.836880e+08</td>\n",
       "      <td>-4.395697e+08</td>\n",
       "      <td>3.018459e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.318615</td>\n",
       "      <td>0.128902</td>\n",
       "      <td>0.110158</td>\n",
       "      <td>0.047220</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>{'ridge__alpha': 0.0001}</td>\n",
       "      <td>0.826739</td>\n",
       "      <td>0.780316</td>\n",
       "      <td>0.785134</td>\n",
       "      <td>0.868305</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.506035e+09</td>\n",
       "      <td>7.120769e+08</td>\n",
       "      <td>8</td>\n",
       "      <td>-4.521567e+08</td>\n",
       "      <td>-3.925272e+08</td>\n",
       "      <td>-4.249601e+08</td>\n",
       "      <td>-4.445111e+08</td>\n",
       "      <td>-4.836878e+08</td>\n",
       "      <td>-4.395686e+08</td>\n",
       "      <td>3.018507e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.335069</td>\n",
       "      <td>0.137801</td>\n",
       "      <td>0.106853</td>\n",
       "      <td>0.044712</td>\n",
       "      <td>0.00001</td>\n",
       "      <td>{'ridge__alpha': 1e-05}</td>\n",
       "      <td>0.826758</td>\n",
       "      <td>0.780298</td>\n",
       "      <td>0.785108</td>\n",
       "      <td>0.868308</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.506079e+09</td>\n",
       "      <td>7.121056e+08</td>\n",
       "      <td>9</td>\n",
       "      <td>-4.521567e+08</td>\n",
       "      <td>-3.925272e+08</td>\n",
       "      <td>-4.249601e+08</td>\n",
       "      <td>-4.445111e+08</td>\n",
       "      <td>-4.836878e+08</td>\n",
       "      <td>-4.395686e+08</td>\n",
       "      <td>3.018507e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.341662</td>\n",
       "      <td>0.077384</td>\n",
       "      <td>0.120612</td>\n",
       "      <td>0.053645</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>{'ridge__alpha': 1e-06}</td>\n",
       "      <td>0.826760</td>\n",
       "      <td>0.780296</td>\n",
       "      <td>0.785105</td>\n",
       "      <td>0.868308</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.506084e+09</td>\n",
       "      <td>7.121084e+08</td>\n",
       "      <td>10</td>\n",
       "      <td>-4.521567e+08</td>\n",
       "      <td>-3.925272e+08</td>\n",
       "      <td>-4.249601e+08</td>\n",
       "      <td>-4.445111e+08</td>\n",
       "      <td>-4.836878e+08</td>\n",
       "      <td>-4.395686e+08</td>\n",
       "      <td>3.018507e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.356584</td>\n",
       "      <td>0.094924</td>\n",
       "      <td>0.163695</td>\n",
       "      <td>0.047718</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>{'ridge__alpha': 10000.0}</td>\n",
       "      <td>0.593370</td>\n",
       "      <td>0.551975</td>\n",
       "      <td>0.568663</td>\n",
       "      <td>0.557739</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.686348e+09</td>\n",
       "      <td>3.316395e+08</td>\n",
       "      <td>11</td>\n",
       "      <td>-2.762093e+09</td>\n",
       "      <td>-2.571901e+09</td>\n",
       "      <td>-2.662541e+09</td>\n",
       "      <td>-2.624450e+09</td>\n",
       "      <td>-2.595569e+09</td>\n",
       "      <td>-2.643311e+09</td>\n",
       "      <td>6.665351e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.357845</td>\n",
       "      <td>0.116593</td>\n",
       "      <td>0.144385</td>\n",
       "      <td>0.039686</td>\n",
       "      <td>100000.0</td>\n",
       "      <td>{'ridge__alpha': 100000.0}</td>\n",
       "      <td>0.135384</td>\n",
       "      <td>0.113837</td>\n",
       "      <td>0.118874</td>\n",
       "      <td>0.123367</td>\n",
       "      <td>...</td>\n",
       "      <td>-5.442377e+09</td>\n",
       "      <td>5.531116e+08</td>\n",
       "      <td>12</td>\n",
       "      <td>-5.629792e+09</td>\n",
       "      <td>-5.271738e+09</td>\n",
       "      <td>-5.379416e+09</td>\n",
       "      <td>-5.342605e+09</td>\n",
       "      <td>-5.498133e+09</td>\n",
       "      <td>-5.424337e+09</td>\n",
       "      <td>1.262079e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>12 rows × 80 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                       mean_fit_time  std_fit_time  mean_score_time  \\\n",
       "rank_test_mape_scorer                                                 \n",
       "1                           0.362211      0.133726         0.144514   \n",
       "2                           0.379797      0.089024         0.155092   \n",
       "3                           0.408151      0.142118         0.145437   \n",
       "4                           0.269104      0.025912         0.081145   \n",
       "5                           0.379829      0.115560         0.134746   \n",
       "6                           0.354040      0.079624         0.113333   \n",
       "7                           0.327985      0.113027         0.116361   \n",
       "8                           0.318615      0.128902         0.110158   \n",
       "9                           0.335069      0.137801         0.106853   \n",
       "10                          0.341662      0.077384         0.120612   \n",
       "11                          0.356584      0.094924         0.163695   \n",
       "12                          0.357845      0.116593         0.144385   \n",
       "\n",
       "                       std_score_time param_ridge__alpha  \\\n",
       "rank_test_mape_scorer                                      \n",
       "1                            0.035490              100.0   \n",
       "2                            0.078415             1000.0   \n",
       "3                            0.060656               10.0   \n",
       "4                            0.015786                1.0   \n",
       "5                            0.037908                0.1   \n",
       "6                            0.040005               0.01   \n",
       "7                            0.047817              0.001   \n",
       "8                            0.047220             0.0001   \n",
       "9                            0.044712            0.00001   \n",
       "10                           0.053645           0.000001   \n",
       "11                           0.047718            10000.0   \n",
       "12                           0.039686           100000.0   \n",
       "\n",
       "                                           params  split0_test_r2  \\\n",
       "rank_test_mape_scorer                                               \n",
       "1                         {'ridge__alpha': 100.0}        0.866980   \n",
       "2                        {'ridge__alpha': 1000.0}        0.827261   \n",
       "3                          {'ridge__alpha': 10.0}        0.859318   \n",
       "4                           {'ridge__alpha': 1.0}        0.835749   \n",
       "5                           {'ridge__alpha': 0.1}        0.821807   \n",
       "6                          {'ridge__alpha': 0.01}        0.824884   \n",
       "7                         {'ridge__alpha': 0.001}        0.826553   \n",
       "8                        {'ridge__alpha': 0.0001}        0.826739   \n",
       "9                         {'ridge__alpha': 1e-05}        0.826758   \n",
       "10                        {'ridge__alpha': 1e-06}        0.826760   \n",
       "11                      {'ridge__alpha': 10000.0}        0.593370   \n",
       "12                     {'ridge__alpha': 100000.0}        0.135384   \n",
       "\n",
       "                       split1_test_r2  split2_test_r2  split3_test_r2  ...  \\\n",
       "rank_test_mape_scorer                                                  ...   \n",
       "1                            0.820033        0.826216        0.851460  ...   \n",
       "2                            0.786555        0.802902        0.812272  ...   \n",
       "3                            0.820564        0.832866        0.846650  ...   \n",
       "4                            0.810073        0.831611        0.843992  ...   \n",
       "5                            0.792828        0.805198        0.863941  ...   \n",
       "6                            0.782183        0.787933        0.868003  ...   \n",
       "7                            0.780497        0.785399        0.868281  ...   \n",
       "8                            0.780316        0.785134        0.868305  ...   \n",
       "9                            0.780298        0.785108        0.868308  ...   \n",
       "10                           0.780296        0.785105        0.868308  ...   \n",
       "11                           0.551975        0.568663        0.557739  ...   \n",
       "12                           0.113837        0.118874        0.123367  ...   \n",
       "\n",
       "                       mean_test_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                     \n",
       "1                                         -1.263742e+09   \n",
       "2                                         -1.362185e+09   \n",
       "3                                         -1.283119e+09   \n",
       "4                                         -1.386071e+09   \n",
       "5                                         -1.465279e+09   \n",
       "6                                         -1.501304e+09   \n",
       "7                                         -1.505592e+09   \n",
       "8                                         -1.506035e+09   \n",
       "9                                         -1.506079e+09   \n",
       "10                                        -1.506084e+09   \n",
       "11                                        -2.686348e+09   \n",
       "12                                        -5.442377e+09   \n",
       "\n",
       "                       std_test_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                    \n",
       "1                                         5.328077e+08   \n",
       "2                                         3.262269e+08   \n",
       "3                                         5.511620e+08   \n",
       "4                                         6.378971e+08   \n",
       "5                                         6.931439e+08   \n",
       "6                                         7.090376e+08   \n",
       "7                                         7.117899e+08   \n",
       "8                                         7.120769e+08   \n",
       "9                                         7.121056e+08   \n",
       "10                                        7.121084e+08   \n",
       "11                                        3.316395e+08   \n",
       "12                                        5.531116e+08   \n",
       "\n",
       "                       rank_test_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                     \n",
       "1                                                     1   \n",
       "2                                                     3   \n",
       "3                                                     2   \n",
       "4                                                     4   \n",
       "5                                                     5   \n",
       "6                                                     6   \n",
       "7                                                     7   \n",
       "8                                                     8   \n",
       "9                                                     9   \n",
       "10                                                   10   \n",
       "11                                                   11   \n",
       "12                                                   12   \n",
       "\n",
       "                       split0_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                        \n",
       "1                                            -9.379875e+08   \n",
       "2                                            -1.282021e+09   \n",
       "3                                            -7.034979e+08   \n",
       "4                                            -5.394113e+08   \n",
       "5                                            -4.579559e+08   \n",
       "6                                            -4.522563e+08   \n",
       "7                                            -4.521578e+08   \n",
       "8                                            -4.521567e+08   \n",
       "9                                            -4.521567e+08   \n",
       "10                                           -4.521567e+08   \n",
       "11                                           -2.762093e+09   \n",
       "12                                           -5.629792e+09   \n",
       "\n",
       "                       split1_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                        \n",
       "1                                            -8.264283e+08   \n",
       "2                                            -1.156311e+09   \n",
       "3                                            -6.276943e+08   \n",
       "4                                            -4.898703e+08   \n",
       "5                                            -4.009514e+08   \n",
       "6                                            -3.926733e+08   \n",
       "7                                            -3.925288e+08   \n",
       "8                                            -3.925272e+08   \n",
       "9                                            -3.925272e+08   \n",
       "10                                           -3.925272e+08   \n",
       "11                                           -2.571901e+09   \n",
       "12                                           -5.271738e+09   \n",
       "\n",
       "                       split2_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                        \n",
       "1                                            -8.758062e+08   \n",
       "2                                            -1.207814e+09   \n",
       "3                                            -6.910456e+08   \n",
       "4                                            -5.405227e+08   \n",
       "5                                            -4.345386e+08   \n",
       "6                                            -4.251158e+08   \n",
       "7                                            -4.249618e+08   \n",
       "8                                            -4.249601e+08   \n",
       "9                                            -4.249601e+08   \n",
       "10                                           -4.249601e+08   \n",
       "11                                           -2.662541e+09   \n",
       "12                                           -5.379416e+09   \n",
       "\n",
       "                       split3_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                        \n",
       "1                                            -8.849438e+08   \n",
       "2                                            -1.225301e+09   \n",
       "3                                            -6.701186e+08   \n",
       "4                                            -5.290961e+08   \n",
       "5                                            -4.515351e+08   \n",
       "6                                            -4.446284e+08   \n",
       "7                                            -4.445124e+08   \n",
       "8                                            -4.445111e+08   \n",
       "9                                            -4.445111e+08   \n",
       "10                                           -4.445111e+08   \n",
       "11                                           -2.624450e+09   \n",
       "12                                           -5.342605e+09   \n",
       "\n",
       "                       split4_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                        \n",
       "1                                            -7.395282e+08   \n",
       "2                                            -1.029950e+09   \n",
       "3                                            -5.915714e+08   \n",
       "4                                            -5.046193e+08   \n",
       "5                                            -4.846260e+08   \n",
       "6                                            -4.837022e+08   \n",
       "7                                            -4.836880e+08   \n",
       "8                                            -4.836878e+08   \n",
       "9                                            -4.836878e+08   \n",
       "10                                           -4.836878e+08   \n",
       "11                                           -2.595569e+09   \n",
       "12                                           -5.498133e+09   \n",
       "\n",
       "                       mean_train_neg_mean_squared_error  \\\n",
       "rank_test_mape_scorer                                      \n",
       "1                                          -8.529388e+08   \n",
       "2                                          -1.180279e+09   \n",
       "3                                          -6.567856e+08   \n",
       "4                                          -5.207039e+08   \n",
       "5                                          -4.459214e+08   \n",
       "6                                          -4.396752e+08   \n",
       "7                                          -4.395697e+08   \n",
       "8                                          -4.395686e+08   \n",
       "9                                          -4.395686e+08   \n",
       "10                                         -4.395686e+08   \n",
       "11                                         -2.643311e+09   \n",
       "12                                         -5.424337e+09   \n",
       "\n",
       "                       std_train_neg_mean_squared_error  \n",
       "rank_test_mape_scorer                                    \n",
       "1                                          6.685103e+07  \n",
       "2                                          8.521743e+07  \n",
       "3                                          4.155178e+07  \n",
       "4                                          2.011269e+07  \n",
       "5                                          2.766317e+07  \n",
       "6                                          3.014081e+07  \n",
       "7                                          3.018459e+07  \n",
       "8                                          3.018507e+07  \n",
       "9                                          3.018507e+07  \n",
       "10                                         3.018507e+07  \n",
       "11                                         6.665351e+07  \n",
       "12                                         1.262079e+08  \n",
       "\n",
       "[12 rows x 80 columns]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Best hyperparameter values: \", search_multi.best_params_)\n",
    "print(\"Best score: %0.3f\" % (search_multi.best_score_))\n",
    "pd.DataFrame(search_multi.cv_results_).set_index(\"rank_test_mape_scorer\").sort_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's the test score? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:53:08.297791Z",
     "iopub.status.busy": "2024-05-29T14:53:08.296206Z",
     "iopub.status.idle": "2024-05-29T14:53:08.415792Z",
     "shell.execute_reply": "2024-05-29T14:53:08.410739Z",
     "shell.execute_reply.started": "2024-05-29T14:53:08.297736Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.09503409246217502"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_multi.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-05-29T14:53:08.427947Z",
     "iopub.status.busy": "2024-05-29T14:53:08.420733Z",
     "iopub.status.idle": "2024-05-29T14:53:08.632655Z",
     "shell.execute_reply": "2024-05-29T14:53:08.608682Z",
     "shell.execute_reply.started": "2024-05-29T14:53:08.427884Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.09503409246217502"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_mape(y_test, ridge_tuned.predict(X_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Using regression metrics with `scikit-learn`\n",
    "\n",
    "- In `sklearn`, you will notice that it has negative version of the metrics above (e.g., `neg_mean_squared_error`, `neg_root_mean_squared_error`). \n",
    "- The reason for this is that scores return a value to maximize, **the higher the better**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## ❓❓ Questions for you"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### iClicker Exercise 10.2\n",
    "\n",
    "**Select all of the following statements which are TRUE.**\n",
    "\n",
    "- (A) We can use still use precision and recall for regression problems but now we have other metrics we can use as well.\n",
    "- (B) In `sklearn` for regression problems, using `r2_score()` and `.score()` (with default values) will produce the same results.\n",
    "- (C) RMSE is always going to be non-negative.\n",
    "- (D) MSE does not directly provide the information about whether the model is underpredicting or overpredicting.\n",
    "- (E) We can pass multiple scoring metrics to `GridSearchCV` or `RandomizedSearchCV` for regression as well as classification problems. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br><br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What did we learn today? \n",
    "\n",
    "- House prices dataset target is price, which is numeric -> regression rather than classification\n",
    "- There are corresponding versions of all the tools we used:\n",
    "    - `DummyClassifier` -> `DummyRegressor`\n",
    "    - `LogisticRegression` -> `Ridge`\n",
    "- `Ridge` hyperparameter `alpha` is like `LogisticRegression` hyperparameter `C`, but opposite meaning\n",
    "- We'll avoid `LinearRegression` in this course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Scoring metrics\n",
    "- $R^2$ is the default .score(), it is unitless, 0 is bad, 1 is best\n",
    "- MSE (mean squared error) is in units of target squared, hard to interpret; 0 is best\n",
    "- RMSE (root mean squared error) is in the same units as the target; 0 is best\n",
    "- MAPE (mean absolute percent error) is unitless; 0 is best, 1 is bad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](../img/eva-seeyou.png)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [conda env:cpsc330]",
   "language": "python",
   "name": "conda-env-cpsc330-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}