Loan_Defaulter_Prediction_Modelling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Loan Defaulter Prediction-Modelling.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KEbcxNE0vkHS"
      },
      "source": [
        "# ***Bank Loan Defaulter Prediction using Boosting Algorithms - HackerEarth***\n",
        "\n",
        "# *Part 2 - Prediction Modelling*\n",
        "\n",
        "By Nakshatra Singh"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZVvX91Ivwth"
      },
      "source": [
        "This notebook is an illustration on how to use boosting algorithms (Please see Part 1, if you haven't already) to predict if a bank customer is a defaulter or not. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "41FDenmcvri_"
      },
      "source": [
        "###**1. Retrieve and Inspect Dataset** \n",
        "\n",
        "Let's download the dataset provided by HackerEarth which is uploaded on my google drive. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jECSp4oEvTX9",
        "outputId": "a47cfadd-4397-4fea-ab7c-12d54689b6ce",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "!gdown --id 1OZGX6mpxC7ZmTZgXzq9PIvxtjuIk0-vN "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1OZGX6mpxC7ZmTZgXzq9PIvxtjuIk0-vN\n",
            "To: /content/train_indessa.csv\n",
            "144MB [00:01, 88.6MB/s]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KFLEXA2OAG20"
      },
      "source": [
        "We'll use `pandas` to parse the csv files.   "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qHyII8cQv72u"
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('/content/train_indessa.csv') "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2KRJ1WTja6BJ"
      },
      "source": [
        "We'll also define the target column right away."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rUyxKo5Sa2d1"
      },
      "source": [
        "# Class Label : Binary Classification\n",
        "train_target = pd.DataFrame(df['loan_status']) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "El_N1iKZALMp"
      },
      "source": [
        "Let's take a look at the first few rows of the table just to see what's in there.   "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JFwjdCbkv-fA",
        "outputId": "768eb17e-8e08-4ef1-b054-4a5a5fa59ca0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 465
        }
      },
      "source": [
        "df.head() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>member_id</th>\n",
              "      <th>loan_amnt</th>\n",
              "      <th>funded_amnt</th>\n",
              "      <th>funded_amnt_inv</th>\n",
              "      <th>term</th>\n",
              "      <th>batch_enrolled</th>\n",
              "      <th>int_rate</th>\n",
              "      <th>grade</th>\n",
              "      <th>sub_grade</th>\n",
              "      <th>emp_title</th>\n",
              "      <th>emp_length</th>\n",
              "      <th>home_ownership</th>\n",
              "      <th>annual_inc</th>\n",
              "      <th>verification_status</th>\n",
              "      <th>pymnt_plan</th>\n",
              "      <th>desc</th>\n",
              "      <th>purpose</th>\n",
              "      <th>title</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>addr_state</th>\n",
              "      <th>dti</th>\n",
              "      <th>delinq_2yrs</th>\n",
              "      <th>inq_last_6mths</th>\n",
              "      <th>mths_since_last_delinq</th>\n",
              "      <th>mths_since_last_record</th>\n",
              "      <th>open_acc</th>\n",
              "      <th>pub_rec</th>\n",
              "      <th>revol_bal</th>\n",
              "      <th>revol_util</th>\n",
              "      <th>total_acc</th>\n",
              "      <th>initial_list_status</th>\n",
              "      <th>total_rec_int</th>\n",
              "      <th>total_rec_late_fee</th>\n",
              "      <th>recoveries</th>\n",
              "      <th>collection_recovery_fee</th>\n",
              "      <th>collections_12_mths_ex_med</th>\n",
              "      <th>mths_since_last_major_derog</th>\n",
              "      <th>application_type</th>\n",
              "      <th>verification_status_joint</th>\n",
              "      <th>last_week_pay</th>\n",
              "      <th>acc_now_delinq</th>\n",
              "      <th>tot_coll_amt</th>\n",
              "      <th>tot_cur_bal</th>\n",
              "      <th>total_rev_hi_lim</th>\n",
              "      <th>loan_status</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>58189336</td>\n",
              "      <td>14350</td>\n",
              "      <td>14350</td>\n",
              "      <td>14350.0</td>\n",
              "      <td>36 months</td>\n",
              "      <td></td>\n",
              "      <td>19.19</td>\n",
              "      <td>E</td>\n",
              "      <td>E3</td>\n",
              "      <td>clerk</td>\n",
              "      <td>9 years</td>\n",
              "      <td>OWN</td>\n",
              "      <td>28700.0</td>\n",
              "      <td>Source Verified</td>\n",
              "      <td>n</td>\n",
              "      <td>NaN</td>\n",
              "      <td>debt_consolidation</td>\n",
              "      <td>Debt consolidation</td>\n",
              "      <td>349xx</td>\n",
              "      <td>FL</td>\n",
              "      <td>33.88</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>50.0</td>\n",
              "      <td>75.0</td>\n",
              "      <td>14.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>22515.0</td>\n",
              "      <td>73.1</td>\n",
              "      <td>28.0</td>\n",
              "      <td>f</td>\n",
              "      <td>1173.84</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>74.0</td>\n",
              "      <td>INDIVIDUAL</td>\n",
              "      <td>NaN</td>\n",
              "      <td>26th week</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>28699.0</td>\n",
              "      <td>30800.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>70011223</td>\n",
              "      <td>4800</td>\n",
              "      <td>4800</td>\n",
              "      <td>4800.0</td>\n",
              "      <td>36 months</td>\n",
              "      <td>BAT1586599</td>\n",
              "      <td>10.99</td>\n",
              "      <td>B</td>\n",
              "      <td>B4</td>\n",
              "      <td>Human Resources Specialist</td>\n",
              "      <td>&lt; 1 year</td>\n",
              "      <td>MORTGAGE</td>\n",
              "      <td>65000.0</td>\n",
              "      <td>Source Verified</td>\n",
              "      <td>n</td>\n",
              "      <td>NaN</td>\n",
              "      <td>home_improvement</td>\n",
              "      <td>Home improvement</td>\n",
              "      <td>209xx</td>\n",
              "      <td>MD</td>\n",
              "      <td>3.64</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>6.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7624.0</td>\n",
              "      <td>23.2</td>\n",
              "      <td>13.0</td>\n",
              "      <td>w</td>\n",
              "      <td>83.95</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>INDIVIDUAL</td>\n",
              "      <td>NaN</td>\n",
              "      <td>9th week</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>9974.0</td>\n",
              "      <td>32900.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>70255675</td>\n",
              "      <td>10000</td>\n",
              "      <td>10000</td>\n",
              "      <td>10000.0</td>\n",
              "      <td>36 months</td>\n",
              "      <td>BAT1586599</td>\n",
              "      <td>7.26</td>\n",
              "      <td>A</td>\n",
              "      <td>A4</td>\n",
              "      <td>Driver</td>\n",
              "      <td>2 years</td>\n",
              "      <td>OWN</td>\n",
              "      <td>45000.0</td>\n",
              "      <td>Not Verified</td>\n",
              "      <td>n</td>\n",
              "      <td>NaN</td>\n",
              "      <td>debt_consolidation</td>\n",
              "      <td>Debt consolidation</td>\n",
              "      <td>447xx</td>\n",
              "      <td>OH</td>\n",
              "      <td>18.42</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>10877.0</td>\n",
              "      <td>31.2</td>\n",
              "      <td>19.0</td>\n",
              "      <td>w</td>\n",
              "      <td>56.47</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>INDIVIDUAL</td>\n",
              "      <td>NaN</td>\n",
              "      <td>9th week</td>\n",
              "      <td>0.0</td>\n",
              "      <td>65.0</td>\n",
              "      <td>38295.0</td>\n",
              "      <td>34900.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1893936</td>\n",
              "      <td>15000</td>\n",
              "      <td>15000</td>\n",
              "      <td>15000.0</td>\n",
              "      <td>36 months</td>\n",
              "      <td>BAT4808022</td>\n",
              "      <td>19.72</td>\n",
              "      <td>D</td>\n",
              "      <td>D5</td>\n",
              "      <td>Us office of Personnel Management</td>\n",
              "      <td>10+ years</td>\n",
              "      <td>RENT</td>\n",
              "      <td>105000.0</td>\n",
              "      <td>Not Verified</td>\n",
              "      <td>n</td>\n",
              "      <td>&gt; My goal is to obtain a loan to pay off my hi...</td>\n",
              "      <td>debt_consolidation</td>\n",
              "      <td>Debt consolidation</td>\n",
              "      <td>221xx</td>\n",
              "      <td>VA</td>\n",
              "      <td>14.97</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>10.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>13712.0</td>\n",
              "      <td>55.5</td>\n",
              "      <td>21.0</td>\n",
              "      <td>f</td>\n",
              "      <td>4858.62</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>INDIVIDUAL</td>\n",
              "      <td>NaN</td>\n",
              "      <td>135th week</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>55564.0</td>\n",
              "      <td>24700.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7652106</td>\n",
              "      <td>16000</td>\n",
              "      <td>16000</td>\n",
              "      <td>16000.0</td>\n",
              "      <td>36 months</td>\n",
              "      <td>BAT2833642</td>\n",
              "      <td>10.64</td>\n",
              "      <td>B</td>\n",
              "      <td>B2</td>\n",
              "      <td>LAUSD-HOLLYWOOD HIGH SCHOOL</td>\n",
              "      <td>10+ years</td>\n",
              "      <td>RENT</td>\n",
              "      <td>52000.0</td>\n",
              "      <td>Verified</td>\n",
              "      <td>n</td>\n",
              "      <td>NaN</td>\n",
              "      <td>credit_card</td>\n",
              "      <td>refi</td>\n",
              "      <td>900xx</td>\n",
              "      <td>CA</td>\n",
              "      <td>20.16</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>11.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>35835.0</td>\n",
              "      <td>76.2</td>\n",
              "      <td>27.0</td>\n",
              "      <td>w</td>\n",
              "      <td>2296.41</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>INDIVIDUAL</td>\n",
              "      <td>NaN</td>\n",
              "      <td>96th week</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>47159.0</td>\n",
              "      <td>47033.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   member_id  loan_amnt  ...  total_rev_hi_lim  loan_status\n",
              "0   58189336      14350  ...           30800.0            0\n",
              "1   70011223       4800  ...           32900.0            0\n",
              "2   70255675      10000  ...           34900.0            0\n",
              "3    1893936      15000  ...           24700.0            0\n",
              "4    7652106      16000  ...           47033.0            0\n",
              "\n",
              "[5 rows x 45 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wwmjScOpAPNV"
      },
      "source": [
        "What's the distribution of our target variable?\n",
        " "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ij2F4yoCvuxf",
        "outputId": "54e25cb0-4649-4d7f-ef06-c37b36bff632",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df['loan_status'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    406601\n",
              "1    125827\n",
              "Name: loan_status, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lccHlDIOASap"
      },
      "source": [
        "What all columns are present in the dataframe? "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b5FnQ9pTv_u-",
        "outputId": "1c1cdd27-30a6-497a-fc20-30480efaee15",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df.columns"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['member_id', 'loan_amnt', 'funded_amnt', 'funded_amnt_inv', 'term',\n",
              "       'batch_enrolled', 'int_rate', 'grade', 'sub_grade', 'emp_title',\n",
              "       'emp_length', 'home_ownership', 'annual_inc', 'verification_status',\n",
              "       'pymnt_plan', 'desc', 'purpose', 'title', 'zip_code', 'addr_state',\n",
              "       'dti', 'delinq_2yrs', 'inq_last_6mths', 'mths_since_last_delinq',\n",
              "       'mths_since_last_record', 'open_acc', 'pub_rec', 'revol_bal',\n",
              "       'revol_util', 'total_acc', 'initial_list_status', 'total_rec_int',\n",
              "       'total_rec_late_fee', 'recoveries', 'collection_recovery_fee',\n",
              "       'collections_12_mths_ex_med', 'mths_since_last_major_derog',\n",
              "       'application_type', 'verification_status_joint', 'last_week_pay',\n",
              "       'acc_now_delinq', 'tot_coll_amt', 'tot_cur_bal', 'total_rev_hi_lim',\n",
              "       'loan_status'],\n",
              "      dtype='object')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iS-5vo5nAZ_b"
      },
      "source": [
        "###**2. Data Preprocessing**\n",
        "\n",
        "This is a very crucial step as it will sort of clean our features and help us in modelling the algorithm."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5eAXGqjGArer"
      },
      "source": [
        "###2.1 Term Feature\n",
        "\n",
        "We'll do some preprocessing for the `term` attribute."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2G18BepnwDWm"
      },
      "source": [
        "# Replaces months string with '' \n",
        "df['term'].replace(to_replace=' months', value='', regex=True, inplace=True) \n",
        "\n",
        "# Convert it to numeric\n",
        "df['term'] = pd.to_numeric(df['term'], errors='coerce')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ZbIQcWvBiJm"
      },
      "source": [
        "Let's have a look at the unique value distribution of the `term` column."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iiq1EWlAwPl4",
        "outputId": "0203695e-ef7a-489d-de9c-8343817580ba",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df['term'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "36    372793\n",
              "60    159635\n",
              "Name: term, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rt5rrzp1Bs96"
      },
      "source": [
        "###2.2 Employment Length Feature\n",
        "\n",
        "We'll do some preprocessing for the `emp_length` attribute."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xxI934YZwccO"
      },
      "source": [
        "df['emp_length'].replace('n/a', '0', inplace=True)\n",
        "df['emp_length'].replace(to_replace='\\+ years', value='', regex=True, inplace=True)\n",
        "df['emp_length'].replace(to_replace=' years', value='', regex=True, inplace=True)\n",
        "df['emp_length'].replace(to_replace='< 1 year', value='0', regex=True, inplace=True)\n",
        "df['emp_length'].replace(to_replace=' year', value='', regex=True, inplace=True)\n",
        "\n",
        "# Convert it to numeric\n",
        "df['emp_length'] = pd.to_numeric(df['emp_length'], errors='coerce') "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QXW_JGVrB0BV"
      },
      "source": [
        "Let's have a look at the unique value distribution of the `emp_length` column."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k8OdrLB3wpwu",
        "outputId": "0ae260ed-d89d-40df-bb19-039be1de3eb2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df['emp_length'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10.0    175105\n",
              "2.0      47276\n",
              "0.0      42253\n",
              "3.0      42175\n",
              "1.0      34202\n",
              "5.0      33393\n",
              "4.0      31581\n",
              "7.0      26680\n",
              "8.0      26443\n",
              "6.0      25741\n",
              "9.0      20688\n",
              "Name: emp_length, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "th8Cmx3uB3O1"
      },
      "source": [
        "###2.3 Sub Grade Feature\n",
        "\n",
        "We'll do some preprocessing for the `sub_grade` attribute."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nLWnNRBbwtr1"
      },
      "source": [
        "df['sub_grade'].replace(to_replace='A', value='0', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='B', value='1', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='C', value='2', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='D', value='3', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='E', value='4', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='F', value='5', regex=True, inplace=True)\n",
        "df['sub_grade'].replace(to_replace='G', value='6', regex=True, inplace=True) \n",
        "\n",
        "# Convert it to numeric\n",
        "df['sub_grade'] = pd.to_numeric(df['sub_grade'], errors='coerce') "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xs_n-TuhB_9k"
      },
      "source": [
        "Let's have a look at the unique value distribution of the `sub_grade` column."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WqHkd6rkw9AT",
        "outputId": "6e7d5ba0-e971-4656-fc45-2ff661a59081",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df['sub_grade'].value_counts() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "13    33844\n",
              "14    33198\n",
              "21    31975\n",
              "22    31356\n",
              "23    30080\n",
              "12    29390\n",
              "15    29313\n",
              "24    29103\n",
              "5     27016\n",
              "11    26968\n",
              "25    24985\n",
              "31    21712\n",
              "4     20823\n",
              "32    17991\n",
              "33    15771\n",
              "34    15226\n",
              "3     14082\n",
              "1     13653\n",
              "2     13533\n",
              "35    12867\n",
              "41    10928\n",
              "42    10255\n",
              "43     8488\n",
              "44     7051\n",
              "45     5773\n",
              "51     4350\n",
              "52     3196\n",
              "53     2708\n",
              "54     2056\n",
              "55     1516\n",
              "61     1112\n",
              "62      824\n",
              "63      559\n",
              "64      391\n",
              "65      335\n",
              "Name: sub_grade, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jzMGbSGdCD6d"
      },
      "source": [
        "###2.4 Last Week Pay Feature\n",
        "\n",
        "We'll do some preprocessing for the `last_week_pay` attribute."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kdndsmyhw_e8"
      },
      "source": [
        "df['last_week_pay'].replace(to_replace='th week', value='', regex=True, inplace=True)\n",
        "\n",
        "df['last_week_pay'].replace(to_replace='NA', value='', regex=True, inplace=True)\n",
        "\n",
        "# Convert it to numeric\n",
        "df['last_week_pay'] = pd.to_numeric(df['last_week_pay'], errors='coerce')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JZ2tdDhmxXIS"
      },
      "source": [
        "###**3. Handling Missing Values**\n",
        "\n",
        "Now, we'll see how many null values does each attribute have."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZxOAGsiqxSdN",
        "outputId": "68dd792e-8e3b-46d7-c428-3fc0c857689c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df.isnull().sum() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "member_id                           0\n",
              "loan_amnt                           0\n",
              "funded_amnt                         0\n",
              "funded_amnt_inv                     0\n",
              "term                                0\n",
              "batch_enrolled                  85149\n",
              "int_rate                            0\n",
              "grade                               0\n",
              "sub_grade                           0\n",
              "emp_title                       30833\n",
              "emp_length                      26891\n",
              "home_ownership                      0\n",
              "annual_inc                          3\n",
              "verification_status                 0\n",
              "pymnt_plan                          0\n",
              "desc                           456829\n",
              "purpose                             0\n",
              "title                              90\n",
              "zip_code                            0\n",
              "addr_state                          0\n",
              "dti                                 0\n",
              "delinq_2yrs                        16\n",
              "inq_last_6mths                     16\n",
              "mths_since_last_delinq         272554\n",
              "mths_since_last_record         450305\n",
              "open_acc                           16\n",
              "pub_rec                            16\n",
              "revol_bal                           0\n",
              "revol_util                        287\n",
              "total_acc                          16\n",
              "initial_list_status                 0\n",
              "total_rec_int                       0\n",
              "total_rec_late_fee                  0\n",
              "recoveries                          0\n",
              "collection_recovery_fee             0\n",
              "collections_12_mths_ex_med         95\n",
              "mths_since_last_major_derog    399448\n",
              "application_type                    0\n",
              "verification_status_joint      532123\n",
              "last_week_pay                   10614\n",
              "acc_now_delinq                     16\n",
              "tot_coll_amt                    42004\n",
              "tot_cur_bal                     42004\n",
              "total_rev_hi_lim                42004\n",
              "loan_status                         0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tRQhv50rxZch"
      },
      "source": [
        "columns = ['term', 'loan_amnt', 'funded_amnt', 'last_week_pay', 'int_rate', 'sub_grade',\n",
        "           'annual_inc', 'dti', 'mths_since_last_delinq', 'mths_since_last_record', 'open_acc',\n",
        "           'revol_bal', 'revol_util', 'total_acc', 'total_rec_int', 'mths_since_last_major_derog', \n",
        "           'tot_coll_amt', 'tot_cur_bal', 'total_rev_hi_lim', 'emp_length']\n",
        "for col in columns:\n",
        "    df[col].fillna(df[col].median(), inplace=True)  # Filling NaN values with median of each column present in columns.\n",
        "\n",
        "  \n",
        "num_cols = ['acc_now_delinq', 'total_rec_late_fee', 'recoveries', 'collection_recovery_fee', 'collections_12_mths_ex_med']\n",
        "for col in num_cols:\n",
        "    df[col].fillna(0, inplace=True)        ## Filling NaN values with 0 for each column present in columns."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2jM4QmuaxZaB"
      },
      "source": [
        "# Converts categorical variable into dummy/indicator variables.\n",
        "cate_attr = ['home_ownership', 'purpose']\n",
        "for cat in cate_attr:\n",
        "   df_col = [cat]\n",
        "   df[cat] = df[cat].astype(\"category\")\n",
        "   df[cat] = pd.get_dummies(df, columns=df_col) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jY1dX1OEx4UT"
      },
      "source": [
        "###**4. Feature Engineering**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rIKKIAQxD-7K"
      },
      "source": [
        "For convenience, I'll copy the original Dataframe to another variable and drop the target feature. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MD0TaneBx3b7"
      },
      "source": [
        "final_df = df.copy()\n",
        "final_df = final_df.drop(['loan_status'], axis=1)  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "okcEEKhLEPvd"
      },
      "source": [
        "Now, we'll create some new features which can help us in predicting the target (Defaulter or Non-Defaulter)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-1PeSYToyFF_"
      },
      "source": [
        "import math\n",
        "import numpy as np\n",
        "\n",
        "# How big the loan a person has taken with respect to his earnings, annual income to fund by bank ratio\n",
        "final_df['income_to_loan'] = np.round_(final_df['annual_inc']/final_df['funded_amnt'], decimals = 2)\n",
        "\n",
        "# How big the loan a person has taken with respect to his earnings, annual income to fund by investors ratio\n",
        "final_df['income_to_loan_inv'] = np.round_(final_df['annual_inc']/final_df['funded_amnt_inv'], decimals = 2)\n",
        "\n",
        "\n",
        "# Interest paid so far = interest received till date + Late fee received till date\n",
        "final_df['total_int_paid'] = final_df['total_rec_int'] + final_df['total_rec_late_fee']\n",
        "\n",
        "# Calculating EMIs (monthly)\n",
        "#EMI = [P x R x (1+R)^N] / [(1+R)^N-1]\n",
        "principal = final_df['loan_amnt']\n",
        "rate = final_df['int_rate'] /(100*12)\n",
        "time = final_df['term']\n",
        "\n",
        "final_df['emi_per_month'] = np.round_((principal * rate * (1+rate)**time)/((1+rate)**(time)-1), decimals = 2)\n",
        "\n",
        "# Calculating EMIs paid (in terms of percent) np.round_(in_array, decimals = 2)\n",
        "final_df['emi_paid_perc'] = np.round_(((final_df['last_week_pay'])/(final_df['term']/12*52+1))*100, decimals = 2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "defqO7jNEeuz"
      },
      "source": [
        "We engineered 5 new columns which can provide us with accurate information to predict our labels."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VCFNGwjdy7MN",
        "outputId": "33b70272-ed00-4f51-d39f-e0163ea950e9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "final_df.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(532428, 49)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KCVdJ078Euey"
      },
      "source": [
        "###**5. Attribute Vectorization**\n",
        "\n",
        "We'll use TDIDF and NLTK to vectorize some of our categorical columns. TFIDF will create feature matrices for each attribute we pass, hence increasing the dimensions. For this tutorial we wont vectorize 'emp_title', 'desc' and 'title' as it increase the data complexity hugely. Also, colab provides us with only 12gb RAM, I tried vectorizing 'desc' and it would crash the notebook. \n",
        "\n",
        "We'll download NLTK stopwords for English Language."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9m22z8qzzB-Q",
        "outputId": "f7d8bc37-8b74-4e5e-9511-73edc7d281c1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "\n",
        "from nltk.corpus import stopwords\n",
        "stop_words = stopwords.words('english') "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BJ_YfATbFziu"
      },
      "source": [
        "We'll fill remaining columns with a string called 'missing'."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b4I1avAxz_A3"
      },
      "source": [
        "cols = ['grade', 'emp_title', 'verification_status_joint', 'verification_status',\n",
        "        'pymnt_plan', 'desc', 'purpose', 'title', 'zip_code', 'addr_state', 'initial_list_status', 'application_type']\n",
        "\n",
        "for col in cols:\n",
        "    final_df[col].fillna('missing', inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FCxhomGNF-W0"
      },
      "source": [
        "Finally, if we have any left out columns, we'll fill them with 0."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rjQ4ibjHz--W"
      },
      "source": [
        "final_df.fillna(0)  \n",
        "final_df.reset_index(inplace=True)  # Resets the indices."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kFUD1FV3zfmw"
      },
      "source": [
        "###5.1 Vectorising Grade Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cXnsRfzw0ye8",
        "outputId": "bdac241a-86ec-479a-ae1f-76ef67555101",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "x_tfidf = v.fit_transform(final_df['grade'])   # Create a feature matrix using TFIDF.\n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names())\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([final_df, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'grade' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('grade',axis=1,inplace=True)\n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 6)\n",
            "(532428, 55)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b6sJLlAP31cS"
      },
      "source": [
        "###5.2 Vectorising Verification Status Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "15Z5IPiF_Nvw",
        "outputId": "a5cb3f90-a538-4803-9af4-bd0758708eba",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['verification_status'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'verification_status' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('verification_status',axis=1,inplace=True)\n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 3)\n",
            "(532428, 57)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GbDIfVV-K8TD"
      },
      "source": [
        "###5.3 Vectorising Verification Status Joint Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yzy26WNKA4Qi",
        "outputId": "ab8a65d0-fa1e-4829-996f-4d720f8f7ffe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['verification_status_joint'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'verification_status_joint' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('verification_status_joint',axis=1,inplace=True)\n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 4)\n",
            "(532428, 60)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ogRfAl-kReel"
      },
      "source": [
        "###5.4 Vectorising Payment Plan Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Af-uecdvLQf1",
        "outputId": "fdbdcc64-c3f2-426d-8ce2-03ab0dab1176",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['pymnt_plan'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'pymnt_plan' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('pymnt_plan',axis=1,inplace=True)\n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 2)\n",
            "(532428, 61)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZxT1zk8HRq45"
      },
      "source": [
        "###5.5 Vectorising Address State Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t9QsCKqlYowF",
        "outputId": "ef7397ae-560f-4b8a-9303-6b5620bc6026",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['addr_state'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'addr_state' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('addr_state',axis=1,inplace=True)\n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 46)\n",
            "(532428, 106)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zzwv804MRuPp"
      },
      "source": [
        "###5.6 Vectorising Initial List Status Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Yd8nzhFZ5zZ",
        "outputId": "3322bcb0-1fe9-42d1-ecb9-52d4de433e59",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['initial_list_status'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'initial_list_status' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('initial_list_status',axis=1,inplace=True) \n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 2)\n",
            "(532428, 107)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O08qlZomRyNz"
      },
      "source": [
        "###5.7 Vectorising Application Type Feature"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rxOAyQhkatez",
        "outputId": "b94e33be-51fd-4a51-b9bf-991fe015ca56",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "v = TfidfVectorizer(min_df=3,  max_features=None, \n",
        "                    strip_accents='unicode', analyzer='word',token_pattern=r'\\w{1,}', \n",
        "                    ngram_range=(1, 3), use_idf=1,smooth_idf=1,sublinear_tf=1,\n",
        "                    stop_words = 'english')\n",
        "\n",
        "# Create a feature matrix using TFIDF\n",
        "x_tfidf = v.fit_transform(res_train['application_type'])  # We'll use the previous concatenated dataframe here  \n",
        "\n",
        "print(x_tfidf.shape)    # No.of columns represents the number of features made by TFIDF.\n",
        "\n",
        "# Convert the feature matrix to an array and store it a pandas dataframe.\n",
        "df_tfidf = pd.DataFrame(x_tfidf.toarray(), columns=v.get_feature_names() )\n",
        "\n",
        "# Concatenate the array dataframe and our final dataframe.\n",
        "res_train = pd.concat([res_train, df_tfidf], axis=1)\n",
        "\n",
        "# Finally, we drop the 'application_type' feature as we have a more dimensional feature matrix.\n",
        "res_train.drop('application_type',axis=1,inplace=True) \n",
        "\n",
        "print(res_train.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(532428, 2)\n",
            "(532428, 108)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RYY2qesATC_4"
      },
      "source": [
        "Finally, we'll drop all the unwanted columns which provide us with no information about the target and they also increase data complexity with size."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RdBARAIZay8i"
      },
      "source": [
        "res_train.drop(['desc', 'zip_code', 'emp_title', 'title', 'batch_enrolled'], axis=1, inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iU_rJ_-iWfxs"
      },
      "source": [
        "What's the dimensions of the dataframe now?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Utzhz6PjWi8P",
        "outputId": "747f32e6-ec8d-483d-8040-b21e8ea1c7e5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "res_train.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(532428, 103)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wXyelzkEWkmV"
      },
      "source": [
        "As you see we increased to dimensions to 103 using TFIDF, you can try using Bag of Words also, but I prefer TFIDF.\n",
        "\n",
        "Finally, we'll scale our dataframe values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OdroUdp_d9hY"
      },
      "source": [
        "def scaler():\n",
        "    result = res_train.copy()\n",
        "    for feature_name in res_train.columns:\n",
        "        max_value = res_train[feature_name].max()\n",
        "        min_value = res_train[feature_name].min()\n",
        "        result[feature_name] = (res_train[feature_name] - min_value) / (max_value - min_value)\n",
        "    return result "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O9N21gVxW5Zs"
      },
      "source": [
        "We'll store the scaled dataframe in a new variable for better understanding."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UWQrMRrkftqH"
      },
      "source": [
        "res_train_scaled = scaler() "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "50B4gOM9XCOE"
      },
      "source": [
        "###**6. Train-Test Split**\n",
        "\n",
        "We'll spit our data in training and cross-validation sets. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4TC0ft9UW_mT"
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(np.array(res_train_scaled), np.array(train_target), test_size=0.30) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bGB11HXFW_h0",
        "outputId": "b5f1c46e-322f-422d-ea09-5b49f441ca99",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "X_train.shape, y_train.shape, X_test.shape, y_test.shape "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((372699, 103), (372699, 1), (159729, 103), (159729, 1))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lrExR4tVgSub"
      },
      "source": [
        "We'll import a few necessary model evaluation metrics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SKBsHmFpYJsA"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import sklearn.metrics as metrics\n",
        "from sklearn.metrics import confusion_matrix, classification_report, accuracy_score\n",
        "from sklearn.metrics import roc_curve, auc \n",
        "from sklearn.metrics import roc_auc_score"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MFhOfFwNgcKw"
      },
      "source": [
        "###**7. Cat Boost Classifier**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4c1w3sPjYRFx",
        "outputId": "a867cc75-f1f6-4a28-f60d-ef0ca02829b0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "#CATBoost\n",
        "from catboost import CatBoostClassifier\n",
        "cat_clf = CatBoostClassifier(learning_rate=0.1, n_estimators=1400, subsample=0.70, max_depth=6, scale_pos_weight=2.5, silent=True)\n",
        "cat_clf.fit(X_train, y_train)\n",
        "\n",
        "# evaluate predictions\n",
        "y_train_predict_cat = cat_clf.predict(X_train)\n",
        "print(\"==========================\")\n",
        "print('Train Accuracy: %.3f' % metrics.accuracy_score(y_train, y_train_predict_cat))\n",
        "\n",
        "# make predictions for test data\n",
        "y_pred_cat = cat_clf.predict(X_test)\n",
        "predictions = [round(value) for value in y_pred_cat]\n",
        "\n",
        "\n",
        "accuracy_per_roc_auc = roc_auc_score(y_test, predictions)\n",
        "print(\"ROC-AUC: %.10f%%\" % (accuracy_per_roc_auc * 100))\n",
        "\n",
        "print('Test Accuracy: %.3f' % metrics.accuracy_score(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.confusion_matrix(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.classification_report(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print('Precision Score: %.3f' % metrics.precision_score(y_test, predictions))\n",
        "print('Recall Score: %.3f' % metrics.recall_score(y_test, predictions))\n",
        "print('F1 Score: %.3f' % metrics.f1_score(y_test, predictions)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==========================\n",
            "Train Accuracy: 0.944\n",
            "ROC-AUC: 93.3794433030%\n",
            "Test Accuracy: 0.939\n",
            "==========================\n",
            "[[114852   6974]\n",
            " [  2849  35054]]\n",
            "==========================\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.98      0.94      0.96    121826\n",
            "           1       0.83      0.92      0.88     37903\n",
            "\n",
            "    accuracy                           0.94    159729\n",
            "   macro avg       0.90      0.93      0.92    159729\n",
            "weighted avg       0.94      0.94      0.94    159729\n",
            "\n",
            "==========================\n",
            "Precision Score: 0.834\n",
            "Recall Score: 0.925\n",
            "F1 Score: 0.877\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VtMs2U5jYXBt",
        "outputId": "8b665a59-20e1-445e-f01c-09f31d2b5e81",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "y_train_pred = cat_clf.predict(X_train)    \n",
        "y_test_pred = cat_clf.predict(X_test)\n",
        "\n",
        "\n",
        "train_fpr, train_tpr, train_thresholds = roc_curve(y_train, y_train_pred)\n",
        "test_fpr, test_tpr, test_thresholds = roc_curve(y_test, y_test_pred)\n",
        "\n",
        "plt.plot(train_fpr, train_tpr, label=\"Train AUC =\"+str(auc(train_fpr, train_tpr)))\n",
        "plt.plot(test_fpr, test_tpr, label=\"Test AUC =\"+str(auc(test_fpr, test_tpr)))\n",
        "plt.legend()\n",
        "plt.xlabel(\"True Positive Rate(TPR)\", fontweight='bold')\n",
        "plt.ylabel(\"False Positive Rate(FPR)\", fontweight='bold')\n",
        "plt.title(\"ROC-AUC\", fontweight='bold')\n",
        "plt.grid()\n",
        "plt.show() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5fX48c9JJvvGJojEGgq4ALKJoHUDLYpaEQoiVisu/WptrXXX2rr/qEWqtihKrVrXSnGFInWpEJe6AYKoqKDsqED2TJaZTHJ+f9w7w2SfhExCMuf9es1r5t65y3kmcJ97n+fe84iqYowxJnbFdXQAxhhjOpZVBMYYE+OsIjDGmBhnFYExxsQ4qwiMMSbGWUVgjDExzioCY4yJcVYRmC5JRDaLiLqvahH5TkSeEZE+YcsMFZHnRGS3iPjddf4qIt3rbCtVRG4XkXUiUiki+SKyXER+HEEcF4XFsbDOd7e5818OmzciuHydZYeKyAIR+V5EfCKyVUT+ISI9W/8rGeOwisB0dUuAv7mffwb8BUBEjgA+BKYB24CnAQ9wBfCeiGS6y6UCucAtwEHAYmApkAP8JIL9nxf2+YzgdltCRI4HVgBnA6VurGvd6QNbuj1j6vJ0dADGRNmjqvqyiHwIPA4Md+ffA6QCbwE/VtWAe7XwFXAocBVwO07FcCTgA45S1U8BRCQeGNDUjkUkGzgBCABfAkNxKp7HWliGh4FkYDlwqqr63O3v78ZlzF6xKwLT5YlIInCEO7nWPcs/zp3+h6oGAFR1JxBspjnZfT/DfX8pWAm4y1ar6vpmdv0znP9jbwFPuPN+3sLYBwGHuJOzg5WAG8P3qlrYku0Z0xC7IjBd3Uthn98GLge6s+ck6Ps6y3/nvu/nvvd237c0tgMRuRwY6E4WqOod7udgs9DLOM1Jc4ATRORAVd0WYfy9wz43GoMxe8OuCExXt4Q9Z/ljcM6uC4Ead16fOsvv777nue+73PeDmtjHNOC37usiABEZBhzufv+yqm7EadcX4Fx3foX7nhi2raQ63+0K+66pGIxpNasITFf3qKpOAf6B084+V1XLgXfd7y9w2/sRkd7AZHf+a+77Evd9iogED+yISJyIDABQ1XGqKu4rx10kvJN4m3sX0LA6333jvo8QkRT384/c96/dbW/A6bcAuEFEghUFIrKfiHSL9IcwpjFiaahNVyQim3HOoKe4ncUH4RxcPcBEoACn7T4F+Bj4BJgAZAPrgTGqWiwiaThNSqOAcuAV9/1o4D+qemUD+47DacbJBj5iT3NTHHv6HEa5+/nSXW6b+xoLxAOXq+o8d3vjgFdxrhY2uPH0cOM9TlXX7NWPZWKeXRGYmKCqW4Cn3MkbVXUFcBTwIk6F8XOc5qIHgB+parG7XhlwPPD/cA7UZwKTcA7uS2jYOJyDeyVwmqpOdl+TcG5FBTjP3faPcfoxPMAI4AvgN8BDYbHn4jRrPQdkAefj3Mn0ghuTMXvFrgiMMSbG2RWBMcbEOKsIjDEmxllFYIwxMc4qAmOMiXGd7sniXr16aU5OTqvWLSsrIy0trW0D2sdZmWODlTk27E2ZV61alaeq+zX0XaerCHJycli5cmWr1s3NzWXcuHFtG9A+zsocG6zMsWFvyiwijaYosaYhY4yJcVYRGGNMjLOKwBhjYpxVBMYYE+OsIjDGmBgXtYpARB4TkV0i8lkj34uIzBWRr0VkrYiMilYsxhhjGhfNK4LHcdL9NuZUYJD7uoSwbIvGGGPaT9SeI1DVt0Ukp4lFzgSeVCf96Qci0k1E+qrqd02sY4wxXYKqUllVQ7mviorKMvzeEnzlxfjLS6iuKCFQUUJ1ZSnqKwWfF/F7KUkZiJPlvG115ANl/aidS327O69eRSAil+BcNdCnTx9yc3NbtUOv19vqdTsrK3NssDJHR3WN4qsGX/Wed381+ALV1FT5oKqcuEAFEqhAAuXEByqID1TgqakgobqCxBrnlVRTQZJWkBJ8UUkalaRTThaV9JSa5oMBXux2YVTK3CmeLFbVh4GHAUaPHq2tfbLOnkSMDVbm2JCbm8sJJ5zgnFX7A5T7q6msqqbc77wqqgJ7Pvurqaj0UV1ZQk1lCTWVXvCVIn4vUuUlzu8lPlBGYqCMhEAZiTXlJNeUk6oVdKOCNKkknQrSpcI5gEtlRDFWE0eFpFIZl4IvPo0qTypV8d0JeNIoTEgnPzENTUxHEjMgOYP44Cslk8TULBJTM0lKyyI5PZPk1Cx6vPNOVP7OHVkR7AAODJvOducZY7qQQHUN5VXOwdg5MAfCPjsH7Ap/zZ75wWV9Vfh9leArQX1e4qq8xPnLiA+U4qkqI87v5d3/LiQd5+CcRgXp7gE7TSrpQ/CgXUE6FSRLVUTxVkkifk8qVfGpVHnSqfakUZ3YA01Ix5+UQVViOsXJGcQlZ+BJySQhJZME94DtScmExHRIyoDEdOITUkgXIT3Kv/He6siKYDFwuYgswBmntdj6B4xpf6G2avesuiLsoF33rNp5r7+cc/AOUOGrQv1liL+MuCov8QEvyTUVoYN18KAcPMMOzuvpzkujkgzZc+btobrxwMOOXlXxKQQ8aVQnpFOTkI4mZkFSNpLkHLBrkjPwp2TiSckkLikDktIhMcM5YCel1zp4J3gSSYj+z75PiVpFICLP4vRq9BKR7cCt4Py+qjofWAqchjOgeDlwYbRiMaazq6quqX3grXNW/fG3AXZ8uMVpAgk/qw4etMMO2JX+asqr9qzvr/KTqhWkU9lAE0jtg3galfSRCjLjKskUp4kkQypIpZIUrSBZK4gjbPjbRo4wKnFUe5xmEU10DsaS1IP45AzikjPDDs7pkJQZ9jnDPYCn8/6qTzl63MmQmEZCXHzMHbzbUjTvGjqnme8V+HW09m9Me2rorDrUNt3EWXXowBx2Vl3rwO0PUFFVTVV1+NjiShJVTnOIOAfwdCpY/9kHoYN5VlwFveN9ZMX5yIzzkRkXPNBXkBrqsCwnyVNOQpwvsjLGJ4bOnMU9eyZp/9CZdK330Bl3eoPfS0IKHpG9+s19yd9BcuZebcM4OkVnsTFtoaq6pk6HYv226tDBu5mz6gr3AL2nCaUa1ab2rqTiq9WO3TPBR3ePn27xPnLiK8mM95EpPjKkkvS4ctKSK0lNriC5poLkmjKSaspJrC4nIVBGnAYiK3R8ap2DcxYk9Qs7KNc54050pxs4IxdPYlv8Gcw+yCoCs89o7Ky63B+ofTdIA2fVFaGz5xq+313BnLXvNHNW3bykeKVXgp+eCVX08vjoluAjJ95HVryPzMQKMpN9YW3dwVsDy51bBaudu0881eVOx2aVF6GB/de4r1A/prgH7bADdWJ3SPpBnYP3niYSEtP55MuNDB9zbJ3v0yEufi//KiYWWEVgWiR4Vl0ROlA3flZdu0Ox4bPq8DP05s+qaxOBlIR4UhPjSUmMJyUhnowEpTsl5KRU0S2jMtQ0khFX+4CdSgXJ1eXuWXYZCdXleAJe4qvKiKsqc28rLHcL7b4qGgkkLqH2wTklHZL6hB3IG+iUrHWgD/s+IdUpWAsV7sqF7NEtXs8YsIqgy6l7Vr2jtIY124pCB+zwtuvGzqprHZz38qw6IV7cg7UndMBOTYwnKyWBvpnJpCbEkeGpolu8j6y4CjLinKaRNKkkXctJoYKUmgqSasqds2y3acQTcM+yfV7wO09eUloKNe6pdXM3IntS6h+cM7IjaiKp970nqXV/LGP2EVYRdIDws+ryOm3NDZ1V7zkwN35WHTzIN3hW/b//NRpL3bPq1ARP6Ox6/8yE0IE7NdHjfh9PagJkxvnJiCsnDR/pOGfYKVoeOmgn1pSTUOWcYeN3Ht7BV+p+9kK5d8+0RvJUpdQ5i3YP3mn71WsO2bD1ewYNGdl4p2ViOsTbP31jgux/Qwt8vauULfnl9e+1buSsuqLOcq09q06MjwsdkFMSGjirDjvTTkn01Dqwb97wFaNHHk5qXA3pUkka5aG27OSaChKrnWYQfF73IF0S9tk9WJeW7vns90KwyaQ5cZ56bdkkd4Os7AbuKGmmCSUhFeIiy5G4IzeXQSPHteg3NiaWWUUQocqqak6b+y7+QP2z18bOqlMT4+mWmuAenONqnVWnJNZvLkkJn58QR0qcn9SacjyB8rCzafeg7K9zcPaVOmfZhaVhB/JSqrwFJHztg2p/ZAX1JNe/oyR9f+jZSPt23QN9+PeepFa1dxtj2pdVBBHKL/PjD9RwxYkDOWP4AbUO5EmeOEQEaqr3nEmHzqhL6sxzz7hLG/s+7EAfUZMJdQ7E7sE49SBISmdXXgn9+h/S7H3doe+tycSYmGP/6yOUV+oDlCkFj9L/za0Nt3tXlUW2MYmvc2ad4TwYk3lAA3eUNNNpmZDWZJPJhtxc+sVYMjJjTMtYRRChPK+P7pTS/8u/QWY2dPsBpPeGHj9s5I6SjIbPuJPSneYXazIxxuwjrCKIUL7XT28pciZOvhOG/rRjAzLGmDZig9dHaLfXx35S7Eyk9+nYYIwxpg1ZRRChPK+PbE+JM5Gxf8cGY4wxbcgqggjle/3kJJU6E+m9OzYYY4xpQ9ZHEKE8r49+CaVAmtP5a4wxXYRdEUQoz+ujjxTZ1YAxpsuxiiBC+V4/vSiyjmJjTJdjFUEEAtU1FJT7yaophAyrCIwxXYtVBBEoKPejChlV+XZFYIzpcqwiiEBeqZ8k/CQGSq0iMMZ0OVYRRCC/zB4mM8Z0XVYRRCDP66M3hc6EVQTGmC7GKoII5JX691wRWGexMaaLsYogAnllPvrGW9OQMaZrsoogAnmlfn6QWAoS54yRa4wxXYilmIhAntfHAZ4S8PSCuPiODscYY9qUXRFEIL/Mx/5xxdYsZIzpkpq9IhCRvsAxQI47awvwrqp+F8W49il5pX56xhVBRnZHh2KMMW2u0YpARCYBVwPHUv/KoUZE3gHuVdV/RzG+Dqeq5Jf5yErNh/QjOjocY4xpc01dEbwM/A+4CVgBfAsIcABwJHCGu0yXbjQvqQgQqK4mPVBoTUPGmC6pqYpgiKp+0cD8L4FlwGwROSw6Ye07dnt9dMNLnAasIjDGdEmNdhY3UgkgIoki8uumlglbdqKIfCUiX4vIjQ18/wMRWS4iq0VkrYic1tICRFue17dn0Hobi8AY0wU1WhGISG8RuV9EXhGRW0TEIyKXApuBuc1tWETigXnAqcBg4BwRGVxnsT8AC1V1JDADeLCV5YiavPBB622sYmNMF9RU09CjwOnu54nuayxQCdwXwbbHAF+r6kYAEVkAnAmsC1tGgUz3cxZOP8Q+Jd/rtzxDxpguramK4FjgcZyz9lOBvwNvAOer6s4Itt0P2BY2vR2nIgl3G/C6iPwGSAN+3NCGROQS4BKAPn36kJubG8Hu6/N6vS1ed+UGP/3cK4J3Vq+n2rOtmTX2La0pc2dnZY4NVua201RFkAX8R1W/FZGXcSqCuRFWApE6B3hcVe8RkaOBp0RkqKrWhC+kqg8DDwOMHj1ax40b16qd5ebm0tJ1XytYy4HfloInjeN+fGqr9tuRWlPmzs7KHBuszG2nuSeLnxCREmATTjPOQhEpEQk2mjdpB3Bg2HS2Oy/cxcBCAFV9H0gGekUSeHvJ8/rp5ymxjmJjTJfV1BXBVpyDf1B+C7e9AhgkIv1xKoAZwM8a2MdJwOPurajJwO4W7ieq8rw+escVW0exMabLarQiUNWcvdmwqgZE5HLgNZyHzh5T1c9F5A5gpaouBq4B/i4iV+FUOheoqja+1faX5/XRUwsgPaejQzHGmKhoKsVEd+BunA7eLcBNqvppSzauqkuBpXXm3RL2eR1OHqN9Vr7XT1ZCIaTbFYExpmtqqmnoQeBs9/NQYJiIDFTVquiHtW8o9weo9leQHOe1PgJjTJfVVGfxycALOA+D/R6ns7fuA2FdmjNEZfCpYnuGwBjTNTVVEXQHnlfVL3FuHRV3XszIK/PRG7cisM5iY0wX1dx4BNeJyLlAAk5n7h9FJA9QVT0z6tF1sLxSX9gVgTUNGWO6puYqglHuK+go932furMnWvK8/j15hqyz2BjTRTVVEfRvtyj2Ufle54pAJQ5J26eeczPGmDbTXIqJzapa0l7B7GvyvD6GeYqRVBu03hjTdTXVWbwaOEVEeopItYic2F5B7SvyvH4OiC+FDLtjyBjTdTVVEYj7Iuw9puwODkpjt44aY7qw5pLOaSOfY0K+10cP7KliY0zX1txdQ08AAZxKYImIVLvzVVWzohrZPiC/tJJMCu3WUWNMlxZp9tGWZh7t9PyBGuIqC4lPrramIWNMlxa17KOdXUFZWHoJ6yw2xnRhTQ1ef0pzK0eyTGeVF+woBrsiMMZ0aU01DS0VkW3AEpxBZr7FuXvoAGA08BOcRHTN9TN0Sru9PvbDKgJjTNfX1EF8FHAtMBP4FXv6CwQox8lMek9Uo+tA+bXSS1hFYIzpuprqI/gE+LmIJAEjgR+4X20FVquqrx3i6zDBpiFNSEOS0js6HGOMiZpmm3VU1SciHwOVwCZVjWTg+k4vr9THyLhi6yg2xnR5zT1QhoiMBDYCK4EjReQLEfl71CPrYPllfvp6ShBrFjLGdHHNVgTAA4AXp2+gBnga+HE0g9oX5Fl6CWNMjIikIhgOPB42/S3Q5R+13V3qo4cWWkVgjOnyIrn1cztwgvt5GHAOsDlaAe0rvN5SUmvKLL2EMabLi6QiuBt4xP18D04T0QXRCmhfUFOjxJfvhkRsrGJjTJcXyV1Dj4nIN8Dp7qxXVPWt6IbVsQrL/fTUQmfCmoaMMV1cJHcNLQNEVa9X1euBMhF5OPqhdRxnrGJ7qtgYExsi6SweB+wXNn0kcHFUotlHOGMV21PFxpjY0FTSuVvd8QcUWOAOV1mNczvprvYKsCM4I5MVohIHNmi9MaaLa6qPYBfwBTAY586hYpxKoRD4a/RD6zh5Xj/7UYym9kJs0HpjTBfXVK6hh4CHROQfwEOq+lH7hdWx8r0++scVI5ZewhgTAyK5a+hCETlaRC4BksPmz41qZB0oz+ujb3wxkv7Djg7FGGOirtmKQERuBm5r4KtmKwIRmYjTjBQPPKKqf2pgmenu9hX4RFV/1tx2o81pGrL0EsaY2BDJA2WXAK8CE4G7cAakea25lUQkHpgHTMDpY1ghIotVdV3YMoOA3wHHqGqhiOwTj/EWlFbQTYvsqWJjTEyI5PbR3jijlAF8CDwITItgvTHA16q6UVX9wALgzDrL/B8wT9V5ektV94m7kaq8+XiotqeKjTExIZIrgjycrKNFwP1AkvtqTj9gW9j0dmBsnWUOBhCR/+E0H92mqq/W3ZDbP3EJQJ8+fcjNzY1g9/V5vd5m11VV1Ps9JMDnW/LYXdG6fe0rIilzV2Nljg1W5rYTSUXwF5xbR28H7sOpFK5vw/0PwnloLRt4W0QOV9Wi8IVU9WHgYYDRo0fruHHjWrWz3Nxcmlu3tLKKnm98AMCQsSfCQUe3al/7ikjK3NVYmWODlbntRHLX0JzgZxF5BCfpXGoE294BHBg2ne3OC7cd+FBVq4BNIrIep2JYEcH2oyLUUQzWR2CMiQlN9hGIyBEicpaIDHBnZQD/D2fEsuasAAaJSH8RSQRmAIvrLPMyztUAItILp6kokm1HTWhAGrC7howxMaHRKwIRuQm4052sFpFrcJqHulH/zL4eVQ2IyOU4dxjFA4+p6ucicgewUlUXu9+dLCLrgGrgOlXN36sS7aVgnqHqhDTibdB6Y0wMaKpp6DJgPfAUcCxOX0ER8Fvc9vrmqOpSYGmdebeEfVbgave1T9jtZh7VNGsWMsbEhqYqgr7Ahar6lIj0xbkKuEpVn2yf0DpGXqmPQVJEfKbdOmqMiQ1NVQRxwM9E5Eic1BIKTBaR0Tgn879tjwDbW57XR5+4YiT9kI4OxRhj2kVzdw2d4r6CJrvvitNE1OXke/30tvQSxpgY0lRFML7dotiHlJQWk0Y5WOZRY0yMaCoNdZcel7gxNaVulgu7IjDGxIhIcg3FFE+5VQTGmNhiFUGYyqpq0qrcxxisIjDGxAirCMLYU8XGmFjUbEUgIikiMkdE1ojIMSIy1x1MpsvJDz5MZoPWG2NiSCRXBH8BrgIOx0k/HQ9cF82gOkqe18d+FBNI7gk2aL0xJkZEUhH8FJgTNr0K6JJPWwWbhtSahYwxMSSSiqAGJ/V00HDAG51wOlae2zRk6SWMMbEkkoFpXmFPUringP2BR6IWUQfK8/roI0XE2xCVxpgYEklFcCXOFcHpQALwBHBtNIPqKPmllfSSYnuq2BgTUyKpCE4ELnFHEevSfCW7iafGbh01xsSUSPoIXgS+F5GHROSYaAfUobzfO+82RKUxJoZEUhFcDqwFfoEzuPxGEbmzmXU6pfjy3c6HdOsjMMbEjmYrAlV9UFXHAwcADwEHATdFO7D2FqiuIcWX50zYFYExJoY020cgIiOAacBUnMHlAd6NZlAdoaDcz35YegljTOyJpLP4Y/f9c5wrgX+q6rbohdQx8kr97CfFBDypeGzQemNMDImkIpgDPKOqa6MdTEdyniouJJDSO6IfxRhjuopGj3ki8lPgA+BDYKCIDAz/XlVfjHJs7Sq/zEdfKbb0EsaYmNPUye9zwDnAApwxioPEne5SWdnySv0MowhP1sHNL2yMMV1IUxXBHTj9Are3UywdKphwzmN5howxMaapMYtvBxCRI4C3VHWLO92TLph9tLikhAypsPQSxpiYE8kDZf8AxoRNTwDeiU44HSdQEnyq2CoCY0xsaaqzeBIwGadP4Fcicqr71Uigsh1ia1+lO513e6rYGBNjmuojGAlcgNMxfIL7CnomijF1CE9FML2EPVVsjIktTVUED+OMRfAR8HvgdZxKoVBVN7VDbO1GVUn27Xbug7KxCIwxMaapisAHrAH2o/bto4hID1UtiGZg7am4oooeWkgNccSl9uzocIwxpl01VRHsxnmO4NkGvtNm1u1U8rw+elOEP6kHyTZovTEmxjR1MH8bpzJ4h4YfKGuWiEwE/orT6PKIqv6pkeWmAs8DR6rqyki23ZacsYqLCaTaHUPGmNjT1HME492Py1uzYRGJB+bh3G66HVghIotVdV2d5TKA3+KksugQeV4fP5AiSM/pqBCMMabDNPscgYhcJSK/EZFuIvK2iGwQkZ9EsO0xwNequlFV/TipKs5sYLk7gdl04C2peaXuU8VZfTsqBGOM6TCRDl4/DzgP55bSYuBuYEkz6/UDwtNVbwfGhi8gIqOAA1X1FRG5rrENicglwCUAffr0ITc3N4Kw6/N6vQ2u+/H6Ss6jmO0lfja3ctv7qsbK3JVZmWODlbntRFIR9AG+xRnE/h844xM8tLc7FpE44F6cZxWapKoP49zOyujRo3XcuHGt2mdubi4NrfvurrfwfFtDzpAx5Ixt3bb3VY2VuSuzMscGK3PbiSTFxC7gWmASsApIBUojWG8HcGDYdLY7LygDGArkishm4ChgsYiMjmDbbSpQbOkljDGxK5KKYBbQH9gIvAgcQWS5hlYAg0Skv4gkAjOAxcEvVbVYVXupao6q5uCMfTCpI+4awrvLebeKwBgTg5ptGlLVvwF/E5FUd/riSDasqgERuRx4Def20cdU9XMRuQNYqaqLm95C+0mocPMMWeZRY0wMimTw+n44uYWOdSblLeDnqrqj6TVBVZcCS+vMu6WRZcdFEnA0JFfmO9dGaZZnyBgTeyJpGpqHUwl8ALwHHA/cH82g2lO5P0D3mgL88algg9YbY2JQJBXBCcBNqnqsqh6Hk4BuXFSjakd5pX72kyJ8yft1dCjGGNMhIrl9tBw4REQ8OOklDgYqohpVO9rt9bGfFFOdahWBMSY2RXJFsBC4EOfgX45z3/+/ohhTu8r3+tiPIhuQxhgTsyK5IrgB57mB4AhlrwB/jFpE7SzP6+coKSI+yyoCY0xsarIiEJEcYBSwuLG7fTq7ouJiMqWCQHfLM2SMiU2NNg2JyFnAl8BzwIciMrfdompHvqJvAfBkWkVgjIlNTfUR3I6TEfQ5YAvOAPYHtEtU7ai6JDhovT1MZoyJTU1VBAOBG1V1BjDRXXZAu0TVjsRrTxUbY2JbU30EHiDLTRWd5c4bLCJlAKr6cbSDaw+eCsszZIyJbc3dNfRHat8h9GDY5y4xuG+KL88GrTfGxLSmKoIn2i2KDuIP1JAZKKA8tQfpNmi9MSZGNTVm8YXtGUhHyC9znir2W3oJY0wMi+TJ4i4r3+untxRSnWpZR40xsSumK4JgniGxO4aMMTEspiuCvJJyelFMgqWXMMbEsEhyDSEiw3AGr18M9AM2q+q2aAbWHsqKduORGpK7d7nn5IwxJmKRjFA2A3gaJwX1WuB3gBeYEt3Qos9f+B0ASd0svYQxJnZF0jR0O/Bm2PQS4EfRCad91ZQ4FQEZ1jRkjIldkVQEBwDLwqYDQEp0wmlfUh58qtjuGjLGxK5I+gg+Bc53P/8cJ+/QJ1GLqB15ync7Hyy9hDEmhkVyRXANsD9OH8FMIAG4NppBtZcUXx6VcamQmNbRoRhjTIdp9opAVd8XkYHA0e6s91S1KLphRV91jZIRyKc8rSfJHR2MMcZ0oGavCNy7hk4FlgMzgOdE5KhoBxZtReV+eksR/mTrHzDGxLZImobuwBmb4GfAOcBRwAPRDKo95Hn99KKYmjTLM2SMiW2RVAQHAhuBMcBjwNXA4GgG1R7yvD72kyLEbh01xsS4SCqCYuAC4DTgY5xO44ooxtQuCouKyJQKErpZRWCMiW2RVATzcdJLxAEv4jxMtjaaQbWH8gLnYbJUSy9hjIlxkdw1dJuI/AXwqmpARC7HeaisU/MXfwtASo9+HRyJMcZ0rEYrAhGZ28C84EcFfhulmNpFTcn3AJaC2hgT85q6Iri8ie8iqghEZCLwV5zxjR9R1T/V+f5q4Bc4Vxi7gYtUdUtz220LcWX2VLExxkDTFcH4vdmwiMQD84AJwHZghYgsVtV1YYutBkararmIXAbcDZy9N/uNVELFLqqJI94GrTfGxLimxix+K3xaRJKApBZsewzwtapudNdfAJwJhCoCVQNiKJkAACAASURBVF0etvwHwHkt2P5eSfXn4/V0J8sGrTfGxLhIxiM4FngU56GyII1g3X5A+OA124GxTSx/MfCfRmK4BLgEoE+fPuTm5jaz64Z5vV5yc3NRVdKr8ihOzGR1K7fVWQTLHEuszLHBytx2Isk++hcgDef5gVXA4cBbTa7RQiJyHjAaOKGh71X1YeBhgNGjR+u4ceNatZ/c3FzGjRtHSWUVW5YXQVY2rd1WZxEscyyxMscGK3PbieQ5gsNwOnwVZ3Sy3wOVEay3A+ep5KBsd14tIvJjd5uTVNUXwXb3Wr7Xz35STE2a5RkyxphIKoIAkA/U4KShPh74cQTrrQAGiUh/EUnESVi3OHwBERkJ/A2nEtjVksD3RnDQ+jhLL2GMMRE1Da0DcoDXgHPdeW80t1LYw2ev4dw++piqfi4idwArVXUxMAdIx8loCrBVVSe1uBQtVJK/E4/UkJBlYxUbY0wkTxYfDSAiaey5q+eZSDauqkuBpXXm3RL2OZIrizZXUeC0UKX2tPQSxhjTaNOQiBSIyJkikikiy4CDVfVv7svbjjG2OX+x81RxmlUExhjTZB9BN5znBhKAcUD39gioPdSU7gTAk2l9BMYY01xnsTbyuVOLK3MqAksvYYwxzfcR3ABchFMJzBKRPHe+quqZUY0sihIrd1MuqaTaoPXGGNNsRTAq7HP4OMWd+uog1ZdHqacnqR0diDHG7AOaqgj6t1sU7SwjUEBFhiWbM8YYaDrpXLukg25vlVXV9NAiqlKGdnQophlVVVVs376dyspIHmTfIysriy+++CJKUe2brMyxIZIyJycnk52dTUJCQsTbjeSBsi4lz+ujtxTxXZp1FO/rtm/fTkZGBjk5OeGDIjWrtLSUjIyMKEa277Eyx4bmyqyq5Ofns337dvr3j7xRJ5IUE11KQWERGVJBXKZVBPu6yspKevbs2aJKwJhYJiL07NmzxVfRMVcRePOdp4oTu9nDZJ2BVQLGtExr/s/EXEVQURActN7yDBljDMRgRRBML5HZK7uDIzH7uvz8fEaMGMGIESPYf//96devX2ja7/c3ue7KlSu54oorWrzPNWvWICK8+uqroXmbN29m6NDaNzfcdttt/PnPfw5Nz507l0MPPZQRI0Zw5JFH8uSTT7Z43wB33XUXAwcO5JBDDuG1115rcJlly5YxatQohg4dysyZMwkEArW+X7FiBR6Ph+effz5UpqOPPpohQ4YwbNgw/vWvf4WW3bRpE2PHjmXgwIGcffbZod/13nvvZfDgwQwbNoyTTjqJLVv23LsSHx/PiBEjOOaYY5g0aU+OyjfffJNRo0YxYsQIjj32WL7++msArrrqqtDf7eCDD6Zbt24ALF++PDR/xIgRJCcn8/LLLwPwwAMPMHDgQESEvLy80D5UlSuuuIKBAwcybNgwPv7442bL2Ni2cnNzycrKCu3/jjvuCH3317/+laFDhzJkyBD+8pe/hOb/4Q9/4NBDD2XYsGFMmTKFoqKixv+YLaGqnep1xBFHaGstX75cX3nkNtVbM1VLd7V6O53J8uXLOzqEVlu3bl2r1ispKWnjSFRvvfVWnTNnTq15VVVVbb6f66+/Xo899lg9//zzQ/M2bdqkQ4YMaTSehx56SE888UQtLi5WVdXi4mJ9/PHHW7zvzz//XIcNG6aVlZW6ceNG/eEPf6iBQKDWMtXV1Zqdna1fffWVqqrefPPN+sgjj4S+DwQCOn78eD311FP1ueeeU1XVr776StevX6+qqjt27ND9999fCwsLVVX1rLPO0meffVZVVS+99FJ98MEHVVV12bJlWlZWpqqqDz74oE6fPj20j7S0NFWt/3ceNGhQ6N/MvHnzdObMmfXKOHfuXL3wwgvrzc/Pz9fu3buH9vnxxx/rpk2b9KCDDtLdu3eHlnvllVd04sSJWlNTo++//76OGTOm2TI2tq3ly5fr6aefXi+WTz/9VIcMGaJlZWVaVVWlJ510km7YsEFVVV966aXQv7vrr79er7/++nrrqzb8fwcn63ODx9WYu2sorswGre+Mbv/356z7tiSiZaurq4mPb34s6sEHZHLrGUNaFMcFF1xAcnIyq1ev5phjjmHGjBn89re/pbKykpSUFP7xj39wyCGHkJuby5///GeWLFnCbbfdxtatW9m4cSNbt27lyiuvbPBqQVV57rnneOONNzjuuOOorKwkOTm52Zj++Mc/smTJEjIzMwHIzMxk5syZLSoXwKJFi5gxYwZJSUn079+fgQMH8tFHH3H00UeHlsnPzycxMZGDDz4YgAkTJnDXXXdx8cUXA3D//fczdepUVqxYEVonuCzAAQccQO/evdm9ezdZWVksW7aMf/7znwDMnDmT2267jcsuu4zx48eH1jnqqKN4+umnm41fRCgpcf6NFBcXc8AB9fsBn332WW6//fZ6859//nlOPfVUUlOdx0xHjhzZ6G90/vnnIyIcddRRFBUV8d133zVaxm7dujW6rcZ88cUXjB07NhTLCSecwIsvvsj111/PSSedhMfjHLaPOuqo0FXX3oq5pqGkyt0UxXWHuJgrumkj27dv57333uPee+/l0EMP5Z133mH16tXccccd3HTTTQ2u8+WXX/Laa6/x0Ucfcfvtt1NVVVVvmffee4/+/fszYMAAxo0bxyuvvNJsLCUlJZSWlkZ0q+CcOXNqNYUEX8FKaceOHRx44J5BBbOzs9mxo/aggr169SIQCLBy5UrAOYBu27YttP5LL73EZZdd1mgMH330EX6/nwEDBpCfn0+3bt1CB7aG9gfw6KOPcuqpp4amKysrGT16NCeeeGKoKQfgkUce4bTTTiM7O5unnnqKG2+8sdZ2tmzZwqZNmzjxxBPr7WPBggWcc845jcYdFMlvFF7G5rz//vsMHz6cU089lc8//xyAoUOH8s4775Cfn095eTlLly4N/cbhHnvssVq/y96IuSuCVH8+Xk8P7Hqgc2nJmXu07y8/66yzQlccxcXFzJw5kw0bNiAiDR7gAU4//XSSkpJISkqid+/e7Ny5k+zs2v1Uzz77LDNmzABgxowZPPnkk0ydOrXRu0BaenfIddddx3XXXdeidRra54IFC7jqqqvw+XycfPLJod/iyiuvZPbs2cQ1cpL13Xff8fOf/5wnnnii0WXqevrpp1m5ciVvvbVnmPQtW7bQr18/1q5dy6RJkzj88MMZMGAA9913H0uXLmXs2LHMmTOHq6++mkceeSS03oIFC5g2bVq9q8XvvvuOTz/9lFNOOaWlP8delXHUqFFs2bKF9PR0li5dyuTJk9mwYQOHHXYYN9xwAyeffDJpaWmMGDGiXsyzZs3C4/Fw7rnnNrL1lom5iiAzUEBlhj1DYFovLW1PssKbb76Z8ePH89JLL7F58+ZGBxZPSkoKfY6Pj6/XwVpdXc0LL7zAokWLmDVrVujBoNLSUnr27ElhYWGt5QsKCujfvz+ZmZmkp6ezadMmhg0b1mTcc+bM4Zln6o8pdfzxxzN37lz69etX68xz+/bt9OvXr97yRx99NO+88w4Ar7/+OuvXrwecDvJgRZaXl8fSpUvxeDxMnjyZkpISTj/9dGbNmsVRRzlpy3r27ElRURGBQACPx1Nvf//973+ZNWsWb731Vq3fL7hM//79GTduHKtXryYzM5NPPvmEsWPHAnD22WczceLEWnEvWLCAefPm1SvPwoULmTJlSkRP4jb1GzVUxqYEm/IATjvtNH71q1+Rl5dHr169uPjii0PNbTfddFOtk4bHH3+cJUuW8Oabb7bZ7dUx1T5SXaP00AL8KTZovWkbxcXFoQPB448/3urtvPnmmwwbNoxt27axefNmtmzZwtSpU3nppZdIT0+nb9++LFu2DHAqgVdffZVjjz0WgN/97ndcc801ofZxr9fb4F1D1113HWvWrKn3mjt3LgCTJk1iwYIF+Hw+Nm3axIYNGxgzZky97eza5Qwv7vP5mD17Nr/85S8B5w6gzZs3s3nzZqZNm8aDDz7I5MmT8fv9TJkyhfPPP59p06aFtiMijB8/PtTO/cQTT3DmmU5S49WrV3PppZeyePFievfe8/+1sLAQn88HOP0V//vf/xg8eDDdu3enuLg4VCm98cYbHHbYYaH1vvzySwoLC2v1dwQ9++yzETULBX+jJ598ElXlgw8+ICsri759+zZaxqZ8//33OH24TnNSTU0NPXs6bRXB33jr1q28+OKL/OxnPwuV6+6772bx4sWhPoS2EFMVQakvQE9KIN0qAtM2rr/+en73u98xcuTIemf5LfHss88yZcqUWvOmTp3Ks88+C8CTTz7JnXfeyYgRIzjxxBO59dZbQ23Ql112GccffzxHHnkkQ4cO5bjjjou46SXckCFDmD59OoMHD2bixInMmzcv1CRx2mmn8e23zjM4c+bM4bDDDmPYsGGcccYZDba5h1u4cCFvv/02jz/+eKhfYs2aNQDMnj2be++9l4EDB5Kfnx86C77uuuvwer2cddZZjBgxInSb6BdffMHo0aMZPnw4p59+OjfeeCODBw/G4/Hw97//nalTpzJ8+HCeeuop5syZE4phwYIFzJgxo94Z9ObNm9m2bRsnnHBCrflz584lOzub7du3M2zYMH7xi1+Efocf/vCHDBw4kP/7v//jwQcfbLaMjW3r+eefZ+jQoQwfPpwrrriCBQsWhOKbOnUqgwcP5owzzmDevHmhW16vvfZaSktLmTBhAiNGjAhVwntLgjVSZzF69GgNdlS11MIXXmD6pxexbsTNDJ58bRtHtm/Kzc1ttLliX/fFF1/UOquLlOWgiQ1W5sY19H9HRFap6uiGlo+pK4LqcqedNbGbPVVsjDFBMVURUOFUBKk2aL0xxoTEVEUQ73Mqgsz9LL2EMcYExVRFkOB3KoK0HnZFYIwxQTFVEST7i/CSitig9cYYExJTFUF6dSHF8d07OgxjjNmnxFRFkFFdhNdjySVMZPYmDTU4t+6+9957TS4zefLkek+hXnDBBfWSiaWnp4c+r1+/ntNOO41BgwYxatQopk+fHnoAqSUaSwEdzu/3c+GFF3L44YczfPhwcnNzQ99NnDiR4cOHM2TIEH75y19SXV0NOE9bDxs2jBEjRnDyySfXev4g+PsNHTqU+Ph4CgoKgMbTLgfdc8899dI4Q/2U11u2bAmloh4yZAjz589vNt6CggImTJjAoEGDmDBhQugp7kWLFoXKMXr0aN59993Qtp544gkGDRrEoEGDeOKJJ+rFO2nSpHqpw++//34OPfRQhgwZwvXXXw84zzGkpKSEfpfgcwGlpaW18kH16tWLK6+8st5+2kxjaUn31dfepKHefMvBuvreya1evzOyNNRto6E01Hu7TmFhoWZnZ+uhhx6q33zzTWj+zJkzQymcg4KplysqKnTgwIG6ePHi0HfLly/XDz74oEWxqTaeAjrcAw88oBdccIGqqu7cuVNHjRql1dXVqqqhtNc1NTX605/+NLSt4HxV1b/+9a966aWX1tvu4sWLdfz48aradNplVdWtW7fqySefrD/4wQ9qpXEuLCysl/La5/NpZWWlqqqWlpbqQQcdpDt27Ggy3uuuu07vuusuVVW96667QqmdS0tLtaamRlVVP/nkEz3kkENU1UlZ3b9/f83Pz9eCggLt37+/FhQUhOJ64YUX9JxzzqmVOnzZsmV60kknhWLbuXOnqjacYrwho0aN0rfeeivif9uWhroRqkoPitmdsl9Hh2Ja4z83wvefRrRoSnUA4iP4p73/4XDqn1oUxqpVq7j66qvxer306tWLxx9/nL59+zJ37lzmz5+Px+Nh8ODB/OlPf2L+/PnEx8fz9NNPc//993PcccfV2taLL77IGWecQZ8+fViwYEGjmUvD/fOf/+Too4/mjDPOCM0bN24cpaWlLSqHqjaaAjrcunXrQk8O9+7dm27durFy5UrGjBkTypUTCATw+/2hp2LDc+iUlZU1mA8nPK1DU2mXwRlY5u677w6lnwiaP39+vZTXiYmJoc8+n4+amprQdGPxLlq0KHSlM3PmTMaNG8fs2bNrXYWFl+O1115jwoQJ9OjRA3BScb/66qucc845eL1e7r33Xh5++GGmT58eWv+hhx7ixhtvDOVMCk+b0Zz169eza9cujjvuOLxeb8TrtUTMNA0VFzuD1mv6/h0diumkVJXf/OY3PP/886xatYqLLrqI3//+9wD86U9/YvXq1axdu5b58+eTk5PDL3/5S6666irWrFlTrxKAPQfDc845J5RKojmfffYZRxxxRLPL1W1aCH+tW7cu4hTQw4cPZ/HixQQCATZt2sSqVatqJV075ZRT6N27NxkZGbVy7Pz+97/nwAMP5Jlnnqk18hZAeXk5r776KlOnTgWaTru8aNEi+vXrx/Dhw2ttY8eOHSxZsqTBlNfbtm1j2LBhHHjggdxwww21xiVoKN6dO3fSt6/zkOn+++/Pzp07Q8u/9NJLHHrooZx++uk89thjoX03lor65ptv5pprrqmXB2j9+vW88847jB07lhNOOKFW5bVp0yZGjhzJCSecEErmF27BggWcffbZUR2/O2auCIp2b6cbEJ9pmUc7pRacuVdEKfWAz+fjs88+Y8KECYCTMTR4ABk2bBjnnnsukydPZvLkyc1ua+fOnWzYsIFjjz0WESEhIYHPPvuMoUOHNvgfvqUHgYyMjFCum4bUbWtvzEUXXRTK73PQQQfxox/9qFZK5Ndee43KykrOPfdcli1bFvptZs2axaxZs7jrrrt44IEHag0G8+9//5tjjjkmdEbdWNrl8vJy/vjHP/L666/Xi+vKK6/k9ttvbzCn0oEHHsjatWv59ttvmTx5MtOmTaNPnz5NxhskIrV+6ylTpjBlyhTefvttbr75Zv773/82+lutWbOGb775hvvuu4/NmzfX+i4QCFBQUMAHH3zAihUrmD59Ohs3bqRv375s3bqVnj17smrVKiZPnsznn39e66pqwYIFPPXUU43uty1E9YpARCaKyFci8rWI3NjA90ki8i/3+w9FJCdasXjznA6rZEsvYVpJVRkyZEgoa+enn34aOki98sor/PrXv+bjjz/myCOPbDYB3cKFCyksLKR///7k5OSwefPm0FVB3bTTBQUF9OrVC3ASw61atarZWJu7IghPAQ2Np5z2eDzcd999rFmzhkWLFlFUVFRrNC6A5ORkzjzzTBYtWlRv/XPPPZcXXnih1ryGBoG5+OKLWbVqFW+//Tbdu3fn4IMP5ptvvmHTpk0MHz6cnJwctm/fzqhRo/j+++9ZuXIlF110ETk5OTz//PP86le/qjVIDTgjhQWvNpqKt0+fPnz33XeAM55AQ802xx9/PBs3biQvL6/RVNTvv/8+K1euJCcnh2OPPZb169eH8nxlZ2fz05/+FBFhzJgxxMXFkZeXR1JSUijj6BFHHMGAAQNCGVQBPvnkEwKBQERXgXulsc6DvX0B8cA3wA+BROATYHCdZX4FzHc/zwD+1dx2W9tZvHLpY6q3Zuqmz1veqdaZWWdx27j11lt19uzZOmDAAH3vvfdUVdXv9+tnn32m1dXVumnTptC8vn37amFhof75z3/WW265pcHtHX300aHtqGpojGBV1X//+9960kknqc/nU1XVe+65JzTObnl5uQ4YMECXLFkSWvett95qVWfxtGnTanUWz5s3r94yZWVl6vV6VVX19ddf1+OOO05VnY7Ub7/9VlWdsZunT5+u999/v6pqaOxeVWeM4KlTp4ami4qKtHv37qFtBgU7T7ds2aKHHHJIaLzfcHXH/A3+ncM717dt26bl5eWqqlpQUKCDBg3StWvXNhnvtddeW6uz+LrrrlNV1Q0bNoQ6i1etWqUHHHCA1tTUaH5+vubk5GhBQYEWFBRoTk6O5ufn14q1bifwQw89pDfffLOqOuMbZ2dna01Nje7atSs0LvQ333yjBxxwQK1t3XDDDbX+DXXGzuIxwNequhFARBYAZwLrwpY5E7jN/fw88ICIiBt0m6oq+h6AzJ71z3qMiURcXBzPP/88V1xxBcXFxQQCAa688koOPvhgzjvvPIqLi1FVrrjiCrp168YZZ5zBtGnTWLRoUa3O4uB4A+G3jfbv35+srCw+/PBDfvKTn7Bq1SqOOOII4uPjGTBgQOg2yJSUFJYsWcKVV17JlVdeSUJCAsOGDWPWrFktLs/s2bOZMWMGf/jDHxg5cmQoBfTixYtZuXIld9xxB7t27eKUU04hLi6Ofv36hZooysrKmDRpUqhDdvz48aFbH2+88Ua++uor4uLiOOigg2rdwvnSSy+FmoDCTZ06lfz8fBISEmqlXW6pL774gmuuuQYRQVW59tprOfzww9m5c2eT8U6fPp1HH32Ugw46iIULFwLwwgsv8OSTT5KQkEBKSgr/+te/EBF69OjBzTffzJFHHgnALbfcEmrmasxFF13ERRddxNChQ0lMTOSJJ55ARHj77be55ZZbSEhIIC4ujvnz59fa1sKFC1m6dGmrfouWiFoaahGZBkxU1V+40z8Hxqrq5WHLfOYus92d/sZdJq/Oti4BLgHo06fPEQsWLGhxPMVfv0fv796k4pibiItrfmDzrsLr9da6+6EzycrKYuDAgS1eL9LB67sSK3NsiLTMX3/9NcXFxbXmjR8/vtE01J2is1hVHwYeBmc8glbl1x83jtzcH3FiJ83N31qdfTyC1nT6Wp762GBlblxycjIjR46MeLvR7CzeARwYNp3tzmtwGRHxAFlAfhRjMsYYU0c0K4IVwCAR6S8iiTidwYvrLLMYmOl+ngYsi0b/gOm87J+DMS3Tmv8zUasIVDUAXA68BnwBLFTVz0XkDhGZ5C72KNBTRL4Grgbq3WJqYldycjL5+flWGRgTIVUlPz+f5OTkFq0X1T4CVV0KLK0z75awz5XAWdGMwXRewQG/d+/e3aL1KisrW/wfobOzMseGSMqcnJxMdnbLBt/qFJ3FJjYlJCTQv3//Fq+Xm5vboo6yrsDKHBuiVeaYyTVkjDGmYVYRGGNMjLOKwBhjYlzUniyOFhHZDWxp5eq9gMjSLnYdVubYYGWODXtT5oNUtcEBWTpdRbA3RGRlY49Yd1VW5thgZY4N0SqzNQ0ZY0yMs4rAGGNiXKxVBA93dAAdwMocG6zMsSEqZY6pPgJjjDH1xdoVgTHGmDqsIjDGmBjXJSsCEZkoIl+JyNciUi+jqYgkici/3O8/FJGc9o+ybUVQ5qtFZJ2IrBWRN0XkoI6Isy01V+aw5aaKiIpIp7/VMJIyi8h092/9uYj8s71jbGsR/Nv+gYgsF5HV7r/v0zoizrYiIo+JyC53BMeGvhcRmev+HmtFZNRe77SxwYw76wuIB74BfggkAp8Ag+ss8ytgvvt5BvCvjo67Hco8Hkh1P18WC2V2l8sA3gY+AEZ3dNzt8HceBKwGurvTvTs67nYo88PAZe7nwcDmjo57L8t8PDAK+KyR708D/gMIcBTw4d7usyteEYwBvlbVjarqBxYAZ9ZZ5kzgCffz88BJIiLtGGNba7bMqrpcVcvdyQ9wRozrzCL5OwPcCcwGKtszuCiJpMz/B8xT1UIAVd3VzjG2tUjKrECm+zkL+LYd42tzqvo2UNDEImcCT6rjA6CbiPTdm312xYqgH7AtbHq7O6/BZdQZQKcY6Nku0UVHJGUOdzHOGUVn1myZ3UvmA1X1lfYMLIoi+TsfDBwsIv8TkQ9EZGK7RRcdkZT5NuA8EdmOM/7Jb9ontA7T0v/vzbLxCGKMiJwHjAZO6OhYoklE4oB7gQs6OJT25sFpHhqHc9X3togcrqpFHRpVdJ0DPK6q94jI0cBTIjJUVWs6OrDOoiteEewADgybznbnNbiMiHhwLifz2yW66IikzIjIj4HfA5NU1ddOsUVLc2XOAIYCuSKyGactdXEn7zCO5O+8HVisqlWquglYj1MxdFaRlPliYCGAqr4PJOMkZ+uqIvr/3hJdsSJYAQwSkf4ikojTGby4zjKLgZnu52nAMnV7YTqpZsssIiOBv+FUAp293RiaKbOqFqtqL1XNUdUcnH6RSaq6smPCbROR/Nt+GedqABHphdNUtLE9g2xjkZR5K3ASgIgchlMRtGx8085lMXC+e/fQUUCxqn63Nxvsck1DqhoQkcuB13DuOHhMVT8XkTuAlaq6GHgU5/Lxa5xOmRkdF/Hei7DMc4B04Dm3X3yrqk7qsKD3UoRl7lIiLPNrwMkisg6oBq5T1U57tRthma8B/i4iV+F0HF/QmU/sRORZnMq8l9vvcSuQAKCq83H6QU4DvgbKgQv3ep+d+PcyxhjTBrpi05AxxpgWsIrAGGNinFUExhgT46wiMMaYGGcVgTHGxDirCMxeE5HNbnbPuq/N7bTPMhF5R0RG7MX2NJjtUUR+JCK3hW9PRHLdZdrkQSV3+8H4fW6m0DMiWC/VXfeCFuzrChEpF5EeTf2tRCSnzrwCEVkgIj3d7YSvWyQiS0Rkf/e7we78ya3+UUyH6XLPEZgO8RsgDfgJcC4wH3gLKAtfSEQ8bm6ntuLDSSFxOHAT8AIwoJXbOgcIpmH4Ec6925uBNe68O4DeQEkrt9+Yv+Bk15wNPCMi3VW1uonlU93Y3gIeb27jbjLFa4AXVbVARCL5W63Gee5kGnC2+93F7nfB33wizkOZNwO/VtV1IvI/4Gqch9pMZ9LRKVft1XVewLXseaAHnIdiFOcBmI+AlTgHEQWudZdZ4k7nuNMXAV/hHHzeA0Y1sq/NgDdseqe7nV44j9+/DBTiZKL8C5CEcwU8H+chwgpgHXCiu74Cn4XFHP7KAXLDtv8RTkoSj7vuQqAKp6I4DHgDp8LYAlzVSPy3udub5k6/4U73caefc+OvdOOcElbu8Nhuw0nP/GecNANF7rr7ucsf5y43vam/lTsvx523xJ0+LPi71P3NcZKcKfCfsPVvAGqAfh39b9FevZbaPgAAA79JREFULXtZ05BpDz8GXgLua2ohERmH89T3ZuD/4WSE/beIJDexTi8ROcld1otzkH8GOAO4G+eJ1N/i5FgaDlwK/Bf4JbCI+lfF69z1wak0zqF+uoJngB7AiSKSgvOU5xvuvhfh5MS/G/gQuLeZJp8METkE56C7OWxfK4Drgd+500+6v8NN7vQXbmzPu8tcA/wbp9I71Y0d4Niw7UUqQUT2A4LNPFvDv3Sbx4JZTT8K+2oFTo78Y1qwL7MPsKYh0x6WqOpdAM20bZ/uvp/svoIGAx83sHwaew6c5cCvcZpOjgPeU9W7RCQJOB/n4DjPXW4EsAvnimN5+AZVdZeIrMFpNvlQVRe4cYcvtgC4B6fpJMON42ngEPYkeLszbPkJOAfphjzmvlcC41S1RkTigSE4qU8Sw5bNAV53P+8Ki+0f7rxL6+wT4Afue0ty9J+M8/uAc5VxU9h34b/5h8CssO+C+8hpwb7MPsAqAtMewg9Cwfbv4L+9bg0sfw2w1v0cB2xqZLuVOGf+ZcAXqlokIunud/Vyp6jqThEZgnOmOxbnzH4w8Ie6izZelNB2/utupxvOlcjLQH93kddwmmqCvm9ic3fiNCvdAfzVTaM8Aafy+q+7nd/gVJLJTcQWwGn3D/6+da/2WzLw0oc4V1AFwDqtnam2EpgOXIFzpXcx8FAr9mH2IdY0ZNrbZvf9dBH5P5z00EHBAWTOwTmTHQvMVXe0rQZUq+p/VfV9dfPtq6oXZ2jKo8QZ3/ZBnH/nS0XkYJzmFi/OwQ7ggAa2G9zfqSIyvZF9PwPsh3NV8JI6o799BWzAaY4ZiXOF8GucYQcbs1ZV78RpOhvLnuYYcM6+B1G7qaUEpx1+oIicK87Y00twKtaZOL/bRPZcHQSbdRoqZ2PyVPVNVV2t9dOVV6vqv4Gf4/Sz3BrWdBfcx5YW7MvsA6wiMO3tXeBZnOaZn+J0IAOgqrk4mRTTcZpxLsFpvmmp83AOjjfitN/PBf6IczY7Cqev4o/utmc3sP5iYBUwFWhs8PeXcJqZBKdZCHXuiDoT+B/OVcadOE1Hn0YQ8204Z/u/x+lvWIDzG83AucLA3UcVzh093dz9Hgfc5c47DngApxnsLXeVd933Nh2HQVW/x7kS6IMzPGZwHxq2T9NJWPZRY7ow9/bRTcC7qnpelPf1Ls4VQ5ce/a4rsisCY7owdc707gV+KiI9orUfERmM04TV5J1hZt9kVwTGGBPj7IrAGGNinFUExhgT46wiMMaYGGcVgTHGxDirCIwxJsb9fxiCsNy3PDhxAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DJ9RALCbrwqM"
      },
      "source": [
        "###**8. XGBoost Classifier**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gHIZMR42dNrO",
        "outputId": "e0fc12fb-6c68-4bc3-952f-aaa23d887a4b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "#XGBOOST\n",
        "from xgboost import XGBClassifier\n",
        "\n",
        "xg = XGBClassifier(scale_pos_weight=2.5 ,silent=True)\n",
        "    \n",
        "xg.fit(X_train, y_train)\n",
        "\n",
        "\n",
        "# evaluate predictions\n",
        "y_train_predict_xg = xg.predict(X_train)\n",
        "print(\"==========================\")\n",
        "print('Train Accuracy %.3f' % metrics.accuracy_score(y_train, y_train_predict_xg))\n",
        "\n",
        "# make predictions for test data\n",
        "y_pred_xg = xg.predict(X_test)\n",
        "predictions = [round(value) for value in y_pred_xg]\n",
        "\n",
        "accuracy_per_roc_auc = roc_auc_score(y_test, predictions)\n",
        "print(\"ROC-AUC: %.10f%%\" % (accuracy_per_roc_auc * 100))\n",
        "\n",
        "\n",
        "print('Test Accuracy %.3f' % metrics.accuracy_score(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.confusion_matrix(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.classification_report(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print('Precision Score %.3f' % metrics.precision_score(y_test, predictions))\n",
        "print('Recall Score %.3f' % metrics.recall_score(y_test, predictions))\n",
        "print('F1 Score %.3f' % metrics.f1_score(y_test, predictions)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_label.py:235: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_label.py:268: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "==========================\n",
            "Train Accuracy 0.928\n",
            "ROC-AUC: 89.9874984442%\n",
            "Test Accuracy 0.927\n",
            "==========================\n",
            "[[115886   5940]\n",
            " [  5742  32161]]\n",
            "==========================\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.95      0.95      0.95    121826\n",
            "           1       0.84      0.85      0.85     37903\n",
            "\n",
            "    accuracy                           0.93    159729\n",
            "   macro avg       0.90      0.90      0.90    159729\n",
            "weighted avg       0.93      0.93      0.93    159729\n",
            "\n",
            "==========================\n",
            "Precision Score 0.844\n",
            "Recall Score 0.849\n",
            "F1 Score 0.846\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "INXh7p0VdtNm",
        "outputId": "f7754adb-a303-4d5e-b158-e0f8c72a388f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "y_train_pred = xg.predict(X_train)    \n",
        "y_test_pred = xg.predict(X_test)\n",
        "\n",
        "\n",
        "train_fpr, train_tpr, tr_thresholds = roc_curve(y_train, y_train_pred)\n",
        "test_fpr, test_tpr, te_thresholds = roc_curve(y_test, y_test_pred)\n",
        "\n",
        "plt.plot(train_fpr, train_tpr, label=\"Train AUC =\"+str(auc(train_fpr, train_tpr)))\n",
        "plt.plot(test_fpr, test_tpr, label=\"Test AUC =\"+str(auc(test_fpr, test_tpr)))\n",
        "plt.legend()\n",
        "plt.xlabel(\"True Positive Rate(TPR)\", fontweight='bold')\n",
        "plt.ylabel(\"False Positive Rate(FPR)\", fontweight='bold')\n",
        "plt.title(\"ROC-AUC\", fontweight='bold')\n",
        "plt.grid()\n",
        "plt.show() "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hc1bXw4d9S792WZckN2+COMTa92FRDYgMxBBM65JLkC4FAKAkJ4EAIcCGBQCiX0BOwLx1TQgkg4GLAhY4JJbjJRWU06l1a3x/nSIzKSCPZMyNp1vs882jmzClrj+xZOnufs7aoKsYYYyJXVLgDMMYYE16WCIwxJsJZIjDGmAhnicAYYyKcJQJjjIlwlgiMMSbCWSIwxpgIZ4nADEsislFE1H20ish2EXlERHJ91pkhIo+LSKmINLnb/EVEMrvsK0lEfi8i60WkQUQ8IvKGiBwRQBzn+MTxWJf3lrnLn/FZNrt9/S7rzhCRFSKyQ0QaRWSziDwgItkD/5SMcVgiMMPd88D/uM9/BNwKICJ7A+8DJwJbgH8AMcAFwCoRSXPXSwIKgauAccBK4EVgPPD9AI5/ms/zRe377Q8ROQRYA5wMVLuxfuK+HtPf/RnTVUy4AzAmyO5T1WdE5H3gQWBPd/mfgCTgTeAIVW1xzxa+BKYAFwG/x0kM84BGYD9V/RRARKKBib0dWEQKgEOBFuDfwAycxHN/P9twD5AAvAEco6qN7v5HuXEZs1PsjMAMeyISB+ztvvzE/Sv/YPf1A6raAqCqxUB7N81R7s9F7s+n25OAu26rqn7Vx6F/hPN/7E3gIXfZ6f2MfTKwh/vyxvYk4MawQ1W9/dmfMT2xMwIz3D3t8/wt4Hwgk+/+CNrRZf3t7s8R7s+R7s9N/g4gIucDk9yX5ap6jfu8vVvoGZzupJuAQ0VkjKpuCTD+kT7P/cZgzM6wMwIz3D3Pd3/l74Pz17UXaHOX5XZZf5T7s8z9WeL+HNfLMU4ELnQf5wCIyCxgpvv+M6r6LU6/vgCnusvr3Z9xPvuK7/Jeic97vcVgzIBZIjDD3X2qegLwAE4/+22qWgf8n/v+WW5/PyIyEjjeXf6y+/N59+cJItL+xY6IRInIRABVna+q4j7Gu6v4DhJvca8CmtXlvf+4P2eLSKL7/AD35zfuvr/GGbcAuFxE2hMFIjJCRDIC/SCM8UesDLUZjkRkI85f0Ce4g8XjcL5cY4CFQDlO330i8AHwMXAkUAB8BeyjqpUikozTpTQHqANecH/uD/xTVX/Zw7GjcLpxCoDVfNfdFMV3Yw5z3OP8211vi/vYF4gGzlfVO9z9zQdewjlb+NqNJ8uN92BV/WinPiwT8eyMwEQEVd0E/N19+WtVXQPsBzyFkzBOx+ku+itwgKpWutvVAocAf8D5oj4OWIzz5f48PZuP8+XeAByrqse7j8U4l6ICnObu+wiccYwYYDbwBfAL4C6f2AtxurUeB9KBM3CuZHrSjcmYnWJnBMYYE+HsjMAYYyKcJQJjjIlwlgiMMSbCWSIwxpgIN+TuLM7JydHx48cPaNva2lqSk5N3bUCDnLU5MlibI8POtHndunVlqjqip/eGXCIYP348a9euHdC2hYWFzJ8/f9cGNMhZmyODtTky7EybRcRviRLrGjLGmAhnicAYYyKcJQJjjIlwlgiMMSbCWSIwxpgIF7REICL3i0iJiHzm530RkdtE5BsR+URE5gQrFmOMMf4F84zgQZxyv/4cA0x2H+fhU23RGGNM6ATtPgJVfUtExveyynHAw+qUP31PRDJEJE9Vt/eyjTHGDHttra14PTvwFm+mtnQLDd6ttFVuoypxEk6V810rnDeU5dO5lnqRu6xbIhCR83DOGsjNzaWwsHBAB6ypqRnwtkOVtTkyWJuHBlWlqbGephoPrbUepM5DdGM5CY0ekpvLSWstJ1O95KiXbGklu8v2T2WcHZQ2D4k7i1X1HuAegLlz5+pA76yzOxEjg7U5Mgy2Njc0NODZsZnKki3UebbQ5N0K1duJqS0msaGEtOYysts8pEh9t21rSKQ8KouquBFsSxjP5uQ8JHUUcZn5JOUUkD5yHFm5Y8h6972gtDmciWArMMbndYG7zBhjBo2WlhY8pTuoKN5EbVkRjd4i2iq3E127g4T6EpKbSslq85BNJfk43RrtmjQaT1QWVTE5eJInsiPpAEgdRUzGaBKzx5A2YgxZo8aSkppJSrgaSHgTwUrgfBFZgTNPa6WNDxhjQkVVqajwUr5jE9WlW2jwFNFSuY2omu3E1ZWQ3FRCeouHHC0nV1rJ7bJ9OelURGdTkzCCbxNn8E3KKKLTRhOfVUDqiDFkjhpLWlYueVHR5IWlhYELWiIQkeU4oxo5IlIEXA3EAqjq3cCLwLE4E4rXAWcHKxZjTGSpraujbMcWKks20eDZSlPFVqjaTlxdMYmNJaQ2l5HTVk6m1JPZZVunmyab6tgctqfMoSg5F0kbTXxmPkk5Y8gYOYbM3DFkxcaTFZbW7XrBvGrolD7eV+DnwTq+MWb4aWxuxlOynYrizZR98z7vFK+lrWo7MbU7iK8vIbW5lKzWcrKlkq7Fmps1Gk9UNlUx2VSkTKI0KRdS84jNyCcxp+C7bprkjLB204TDkBgsNsYMb61tSrnXQ/n2zdSUbaGhvIjWym1E1ewgvr6Y5MZSMlqdbprR0spoYJrP9uWkUxGTQ21CLhuTZvFtyiiiM0aTkJVP6oixZI4aR0r6SEZFRTEqXI0cxCwRGGOCRlWpqnG6aapKNlFfvpXWiq1o9Q7i6naQ2FhKenMZ2VrOCKmn66wpNSTijc6mJnYEO9LmsC15FFFpecRl5bOltJb95x9N+oh8smKGTzdNOFgiMMYMSH1jM6Ul292rabbQ7N2GVm8npnY7CQ2lpDaVktVWTo5Ukt5l22ai8Ug2VbE5VKZNxpM0CtJGEZuRT1J2Aekjx5KZO4aUpHS/3TRlhYVk5O0W7GZGBEsExphOmlvbKPN4KC/eRG3pFhrLtzrdNLU7SKgvJrmpjIxWDyO0nLHSytgu25dLOpXROdQm5bI5aRabUvKIyRhNQlYBaSPGkjlqLAlpI6ybZhCxRGBMhGhrU8qraygv3kJVyWbqPVtprdwK1TuIqysmqbGEjJYystVLntR3u+SxhiQqorOpjhtBacJ4ilNHEZU2mvjMAlJG5JORO46U7HyyYuKsm2aIsURgzBCnqtQ1tbJh4wYqirdQ5ymiqaIIqncQU7ODxEbnrtasNg85UkVOl+2biXHuao0dQXXy7pQnOf3wsZluN03uODJGjiElITXirqaJFJYIjBnEGppbKS3zUFHi3PTUVL6VtqptRNUUk9BQTEpTKVmtHo7AS9yq1k7btiFUSDpVMTnUJo+iKGlPtqTmEdveTTNyLBkjxxKXmkNuVFS3G6ZM5LBEYEwYtLS2UVbpdtOUbqbRs5WWym1IjXPTk3O5ZBk56mWM1HeqxQLfddPUJIygLHECXzYnkj1uqs9dreNIzBxNVnSsddOYPlkiMGYXUlW8tY2UlWyjqmQzdWVFNFdsg+ptxNQVk9RQQlqzh2z1MEqqug2WNhGD172rtTZlDypTRhGVmtdRfCwzdxwpIwpIie/cTVNYWMisQVSAzQwtlgiMCVBNYwslZWVUFG+mtmwzTd5taNU2omuLSagvJqW5jKxWDyPwkiV+umlic6hLzWNb0my2ut00idljSB85hvTcccQlZ5MrYt00JqQsEZiI19jSSom3moqSzVSXFlHv2Upb1VaiaoqdbpqmUjJayhiBl916LCGcREVMDnUJIyhP2o2ylDxi0vNIyCogJWcMGaPGEp9h3TRm8LJEYIat1jbFU11PWcl293JJp7okVduJdS+XTGspI0fLGSNV3frh27tpauJyqE+bwobkXKLSRhOXmU/yiDFkjBxLcnYBKfEpdjWNGdIsEZghR1WprG+mpMxDZclmasvcSUCqnBrxMTVb+eTNy8huc7ppRvbQTVPpdtPUp41me9JebE/LIy4jv6ObJm3kWOKSrJvGRAZLBGZQqWtqodhbjbd4CzVlW2gsL6K1cjvRNduJry8hpckpPjYCL7v30E1TSxIeyaQxaRQVSRPxpowiOiOfRLf4WPrIMcSk55EZHdut/LAxkcoSgQmJ5tY2Sqrq8ZRso7p0C/VlW2ip3I5UbyfWrS6Z1lLGCC1nglQxocv2TcQ4l0vGjaAxaSqb3G6a+Kx8UnLGkpE7hvjMfJLjU1gzyKYwNGaws0Rgdkpbm1Je10RJaRlVpZuoK9tKY6e5WoudSUC0nBF4ye+pmyYqg6qYHBrS8ilO3pvitFHEZRaQnJ1PWu44krMLiEvKZqQII8PUTmOGM0sEpkeqSnVjC8XlVVQUFzndND5ztcbXF5PaVEpGWzm5lDNNGrrto1aSqIzJoS5lBJVJk6l0a8QnZjt3tabljCEqbZR10xgTZpYIIlBDcysllfWUud00DeXuXK3V24n1mat1JOVMlqpu23d00ySMpClxKptT8ohKzyMhK5+UnDGk544lNn00yfEp3WaJMsYMPpYIhpGW1jbKapooKStzJgHxFLHtm094Yf2LxNbtILGhlLSWUrLVyyi8jO2hm6YqKoOq2BwaUvIpTd6b0jTnrtaUnDGk5Y4lMbOAuKQs66YxZhixRDAEqCreumaKvVV4S7ZQV+YUH2t152pNqC8hpdkpPpYrXkb10E1TI8lUxWRTlzKSmuTdqU5xJwHJKSBtxDiSc/KJSh1FRnQsGWFoozEmfCwRhFlNYwvFlXWUd3TTOMXHomu2E1df0jFX60jKmdpDN00zMVRE51CTOIKmxGlsSc0jJm20U3xsZAH/3rCD/Y84zm56Msb4ZYkgSBpbWimpaqTUU0ZVyRbnrtYK52qa2LpiEhtLSG8uIwcvY/AysceraTKpicuhISGfsuS5lKe3d9MUkDZyHHEZ+cQmZTFCpNtcrx1xlBRCvKUAY4x/lgj6qbVN8dQ0UuKtxluyhdqyIpq9RbRVbyemZgcJDSWkNpeS3VZOrngZ0+PVNMlO8bG0kdQm78EGt/hYUs4Y0kaMISl7DFEpI+1qGmNMSFgiCMAjTz3DiC8fdYqPuf3wM/x108TkUJs0guak6WxLzSPaLT6WNmIsKSPGIGl5JMcl29U0xphBwxJBANI/vocF8j7FCbvRkDiW8uR5VKSPdu5kHTGWtBEFxGQUEJuY2Ws3jTHGDEaWCPpQ19RCZlsFnszpFFz8VrjDMcaYXS4q3AEMdp6aJnKkkpakrlN+G2PM8NDnGYGI5AEHAuPdRZuA/1PV7UGMa9Aoq2lkjFTRmGK3Txljhie/iUBEFgMXAwfR/cyhTUTeBv6sqs8FMb6wK6+qY0+qKU21RGCMGZ56OyN4BngHuAJYA2wDBBgNzAMWuetEBznGsKrxFhMlSnxG12nGjTFmeOgtEUxX1S96WP5v4HXgRhGZGpywBo/6imIAkrLywhyJMcYEh9/BYj9JABGJE5Gf97aOz7oLReRLEflGRH7dw/tjReQNEflQRD4RkWP724Bga67aAUBcmk1YaIwZnvwmAhEZKSK3i8gLInKViMSIyE+AjcBtfe1YRKKBO4BjgGnAKSIyrctqvwMeU9W9gKXAnQNsR9BodYnzJNnGCIwxw1NvXUP3Ad9zny90H/sCDcAtAex7H+AbVf0WQERWAMcB633WUSDNfZ6OMw4xqETVlzlPUuw2MWPM8NRbIjgIeBDnr/ZjgL8BrwJnqGpxAPvOB7b4vC7CSSS+lgGviMgvgGTgiJ52JCLnAecB5ObmUlhYGMDhu6upqen3tlK1nWZieOfdD0BkQMcNp4G0eaizNkcGa/MupKo9PoA24CT3ebb7+nv+1u9h+xOBe31enw78tcs6FwO/cp/vj3O2ENXbfvfee28dqDfeeKPf2zy77Dj1/mHSgI8ZbgNp81BnbY4M1ub+Adaqn+/Vvu4sfkhEqoANON04j4lIlYhUBpBjtgJjfF4XuMt8nQs85iakd4EEYNDcwtvapqS1lNMQlx3uUIwxJmh6SwSbgWLA4z42AyXu8/IA9r0GmCwiE0QkDmcweGUPxzgcwL0UNQEo7U8DgqmirolsqaQlcdDkJmOM2eX8jhGo6vid2bGqtojI+cDLODed3a+qn4vINTinKCuBXwF/E5GLcM44znJPYQYFT20T2VJFS7INFBtjhq/eSkxkAv+NM8C7CbhCVT/tz85V9UXgxS7LrvJ5vh6njtGgVFbdwDiqKLM6Q8aYYay3q4buBE52n88AZonIJFVtDn5Yg0OVt4x4aSEu3W4mM8YMX72NERwFPIlzM9hvcQZ7u94QNqzVep27ihMzrbyEMWb46i0RZAJPqOq/ce4hEHdZxGiudBJBUqYVnDPGDF99zUdwqYicCsTiDOb+UUTKAFXV44IeXZi1uuUloqwEtTFmGOsrEcxxH+32c38Omit7gqrWvZLV6gwZY4ax3hLBhJBFMUjF1pfRRhRRSVnhDsUYY4Kmt0SQDmxU1apQBTPYJDR5qI1OJzVqWM+9Y4yJcL0NFn8IHC0i2SLSKiKHhSqowSKp2UtdnJ0NGGOGt94SgbgPfH5GjPqmVjK1gqZ4qzNkjBne+io6p36eD3ue2kZyqKQtycpLGGOGt76uGnoIaMFJAs+LSKu7XFU1PaiRhZmnpomJUkW5TUhjjBnmeksEm/nuLMATglgGFW9lBSnSQJXNVWyMGeaCVn10qKv1bAcgIcPuKjbGDG+9TV5/dF8bB7LOUFXvlpdIzhod5kiMMSa4eusaelFEtgDP40wysw3n6qHRwFzg+ziF6PoaZxiSWquc8hLx6XZXsTFmeOvtS3wOcAlwJvD/+G68QIA6nMqkfwpqdGGkNU4iwOYiMMYMc72NEXwMnC4i8cBewFj3rc3Ah6raGIL4wia6vsx5YrOTGWOGuT67dVS1UUQ+ABqADaoayMT1Q15cg4e6qGSSYuLDHYoxxgRVXzeUISJ7Ad8Ca4F5IvKFiPwt6JGFWWJTObUxVl7CGDP89ZkIgL8CNThjA23AP4AjghlUuLW1KWmtXhqsvIQxJgIEkgj2BB70eb0NGNYjqJX1zWRTSWtiTrhDMcaYoAvk0s8i4FD3+SzgFGBjsAIaDDy1jWRLJZU2UGyMiQCBJIL/Bu51n/8Jp4vorGAFNBh4qmqZJDXU2BSVxpgIEMhVQ/eLyH+A77mLXlDVN4MbVnhVeZy7iuPSrbyEMWb4C+SqodcBUdXLVPUyoFZE7gl+aOFT720vL2GJwBgz/AUyWDwf8O0snwecG5RoBolmt85QUmZemCMxxpjg663o3NXu/AMKrHCnq2zFuZy0JFQBhkObW14i2sYIjDERoLcxghLgC2AazpVDlThJwQv8JfihhY/UueUlrM6QMSYC9FZr6C7gLhF5ALhLVVeHLqzwiq0vo4k44uJSwh2KMcYEXSBXDZ0tIvuLyHlAgs/y24IaWRglNHmojskkWyTcoRhjTND1mQhE5EpgWQ9v9ZkIRGQhTjdSNHCvqt7Qwzo/dPevwMeq+qO+9htsyc1e6pOtvIQxJjIEckPZecBLwELgepwJaV7uayMRiQbuAI7EGWNYIyIrVXW9zzqTgd8AB6qqV0TC3inf0NxKplbQnDC275WNMWYYCOTy0ZE4s5QBvA/cCZwYwHb7AN+o6req2gSsAI7rss5/AXeoqhdAVcN+NVJ5bRM5UklbkpWXMMZEhkDOCMpwqo5WALcD8e6jL/nAFp/XRcC+XdbZHUBE3sHpPlqmqi913ZE7PnEeQG5uLoWFhQEcvruampo+t91Y0cypVLOxAbYM8DiDSSBtHm6szZHB2rzrBJIIbsW5dPT3wC04SeGyXXj8yTg3rRUAb4nITFWt8F1JVe8B7gGYO3euzp8/f0AHKywspK9t3/74S2I/amX0xBmMGeBxBpNA2jzcWJsjg7V51wnkqqGb2p+LyL04ReeSAtj3VmCMz+sCd5mvIuB9VW0GNojIVziJYU0A+w+K2vLtACTaXcXGmAjR6xiBiOwtIieJyER3USrwB5wZy/qyBpgsIhNEJA5YCqzsss4zOGcDiEgOTldRIPsOmqbKYgBSsiwRGGMig98zAhG5ArjWfdkqIr/C6R7KoPtf9t2oaouInI9zhVE0cL+qfi4i1wBrVXWl+95RIrIeaAUuVVXPTrVoJ7VUOYkgIcMKzhljIkNvXUM/A74C/g4chDNWUAFciNtf3xdVfRF4scuyq3yeK3Cx+xgcakudn8lhv5LVGGNCordEkAecrap/F5E8nLOAi1T14dCEFh7R9aW0EkV0Yma4QzHGmJDoLRFEAT8SkXk4pSUUOF5E5uL8MX9hKAIMtfgGD9XRGWREBXKLhTHGDH19XTV0tPtod7z7U3G6iIadpGYvdfFZZIQ7EGOMCZHeEsGCkEUxSKgqaa1eGuOzwh2KMcaETG9lqIf1vMQ9qapvIZtKGhP3CHcoxhgTMtYR7qOstpFsqYIUqzNkjIkclgh8lJeXkySNRNvMZMaYCGKJwEd7eYn4TLuZzBgTOfpMBCKSKCI3ichHInKgiNzmTiYz7NRX7AAgKXN0mCMxxpjQCeSM4FbgImAmTvnpaODSYAYVLs0ddYbsjMAYEzkCSQQ/AG7yeb0OGJaX1WiNkwhi0nLDHIkxxoROIImgDaf0dLs9gZrghBNeUuvWu0vKCW8gxhgTQoFMTPMC3xWF+zswCrg3aBGFUVxjGTWSQkpMXLhDMcaYkAkkEfwS54zge0As8BBwSTCDCpfERg81sVmkhDsQY4wJoUASwWHAee4sYsNacquXhsTscIdhjDEhFcgYwVPADhG5S0QODHZA4dLU0kZmWwXNCZYIjDGRJZBEcD7wCfBjnMnlvxWRa/vYZsgpr20iRyppS7byEsaYyNJnIlDVO1V1ATAauAsYB1wR7MBCzVNZTbrUEWXlJYwxEabPMQIRmQ2cCCzBmVwe4P+CGVQ4VHm2ARCXbvcQGGMiSyCDxR+4Pz/HORN4VFW3BC+k8Kgrd24mS7Q6Q8aYCBNIIrgJeERVPwl2MOHUVOXUGUrJtjpDxpjI4jcRiMgPgPeA94FJIjLJ931VfSrIsYVUa5V7RpBhZwTGmMjS2xnB48ApwAqcOYrbifs6OohxhZzWlgIgNlhsjIkwvSWCa3DGBX4foljCKqa+jAbiSYhLDncoxhgTUr3NWfx7ABHZG3hTVTe5r7MZhtVHExo9VMdkkhDuQIwxJsQCuaHsAWAfn9dHAm8HJ5zwSWr2UhdrdxUbYyJPb4PFi4HjccYE/p+IHOO+tRfQEILYQkZVSW/z0hQ/LtyhGGNMyPU2RrAXcBbOwPCh7qPdI0GMKeSqGlrIphJPkpWXMMZEnt4SwT04cxGsBn4LvIKTFLyquiEEsYWMp7qecVThSbFEYIyJPL0lgkbgI2AEnS8fRUSyVLU8mIGFUmV5MdGixKRaeQljTOTpLRGU4txHsLyH97SPbYeUWrfOUILdTGaMiUC9fZm/hZMM3qbnG8r6JCILgb/g3Hx2r6re4Ge9JcATwDxVXRvIvnel+grnruLkrLxQH9oYY8Kut/sIFrhP3xjIjkUkGrgD53LTImCNiKxU1fVd1ksFLsQpZREWLW55iZRsSwTGmMjT530EInKRiPxCRDJE5C0R+VpEvh/AvvcBvlHVb1W1CadUxXE9rHctcCNhvCRVq0sAiE2zMQJjTOQJdPL6O4DTcC4prQT+G3i+j+3yAd9y1UXAvr4riMgcYIyqviAil/rbkYicB5wHkJubS2FhYQBhd1dTU9Pjtg2ezbQQzf+9/zGIDGjfg5W/Ng9n1ubIYG3edQJJBLnANpxJ7B/AmZ/grp09sIhEAX/GuVehV6p6D87lrMydO1fnz58/oGMWFhbS07avv38HVVEZzF+woPtGQ5y/Ng9n1ubIYG3edQIpMVECXAIsBtYBSUB1ANttBcb4vC5wl7VLBWYAhSKyEdgPWCkicwPY9y6V1FROTWxWqA9rjDGDQiCJ4DpgAvAt8BSwN4HVGloDTBaRCSISBywFVra/qaqVqpqjquNVdTzO3AeLw3HVUEqrl8Y4qzNkjIlMgUxe/z+qmg7MB1pV9VxVXRLAdi3A+cDLwBfAY6r6uYhc49YxGhSaW9vI0EqaEy0RGGMiUyCT1+fj1BY6yHkpbwKnq+rW3rcEVX0ReLHLsqv8rDs/kIB3tfKaRnKopDLZyksYYyJTIF1Dd+AkgfeAVcAhwO3BDCqUyr0eEqSZ6FSbmcwYE5kCSQSHAleo6kGqejBOAbr5QY0qhKrLtgMQl273EBhjIlMgl4/WAXuISAxOeYndgfqgRhVC9RVOIkjKtLuKjTGRKZAzgseAs3G+/Otwrvv/3yDGFFLNlVZewhgT2QI5I7gc576B9hnKXgD+GLSIQqyl2i04Z2cExpgI1WsiEJHxwBxgpb+rfYY6qS1zftpVQ8aYCOW3a0hETgL+DTwOvC8it4UsqhCKrS+jUtIgethMr2CMMf3S2xjB73Eqgj4ObMKZwH50SKIKoYQmDzUxmeEOwxhjwqa3RDAJ+LWqLgUWuutODElUIZTc7KXO6gwZYyJYb/0hMUC6Wyo63V02TURqAVT1g2AHF2yqSlqbl/qEGeEOxRhjwqavjvE/0vkKoTt9nkfv+nBCq6axhWwq2ZRkA8XGmMjVWyJ4KGRRhEl5RRXjpB5JsfISxpjI1ducxWeHMpBwqPRsAyA2zRKBMSZyBXJn8bBVW+6Ul0jIGBXmSIwxJnwiOhE0VLSXlxh2V8UaY0zAIjoRtFY5iSDVEoExJoIFdDutiMzCmbx+JZAPbFTVLcEMLCRqSgArQW2MiWyBzFC2FPgHTgnqT4DfADXACcENLfii6j3UkkhybGK4QzHGmLAJpGvo98BrPq+fBw4ITjihFd9YRnV0RrjDMMaYsAokEYwGXvd53QIMiz+hE5u81Fp5CWNMhAtkjOBT4Az3+ek4dYc+DlpEIZTaWk5DyoRwh2GMMWEVyBnBr4BROGMEZwKxwCXBDCoUWlrbyNRKWhNywh2KMcaEVZ9nBKr6rohMAvZ3F61S1YrghhV85TX15FDN1hSrM2SMiWx9nhG4Vy6MWI4AACAASURBVA0dA7wBLAUeF5H9gh1YsFWU7SBKlJhUKy9hjIlsgXQNXYMzN8GPgFOA/YC/BjOoUKjxOOUl4tKtvIQxJrIFkgjGAN8C+wD3AxcD04IZVCjUe51EkJRlk9YbYyJbIImgEjgLOBb4AGfQuD6IMYVEk1teIs0SgTEmwgWSCO7GKS8RBTyFczPZJ8EMKhTULS+RkmOJwBgT2fpMBKq6DMgCxqlqKXA+zuDxkCa1pTQRgyTYncXGmMjm9/JREbmth2XtTxW4MEgxhURsg4fKqAxGfNcmY4yJSL3dR3B+L+8FlAhEZCHwF5z5je9V1Ru6vH8x8GOcshWlwDmquqmv/e4KCU0eamIysbsIjDGRrrdEsGBndiwi0cAdwJFAEbBGRFaq6nqf1T4E5qpqnYj8DPhv4OSdOW6gUpq91Nuk9cYY0+ucxW/6vhaReCC+H/veB/hGVb91t18BHAd0JAJVfcNn/feA0/qx/wFTVdLbvJQkTgnF4YwxZlALZD6Cg4D7cG4qa6cBbJsP+E5eUwTs28v65wL/9BPDecB5ALm5uRQWFvZx6J7V1NRQWFhIQ3MbC6ji300xA97XUNHe5khibY4M1uZdJ5Dqo7cCyTj3D6wDZgJv9rpFP4nIacBc4NCe3lfVe4B7AObOnavz588f0HEKCwuZP38+W7ZtJ/6dFjIL9mD2APc1VLS3OZJYmyODtXnXCeQ+gqk4A76KMzvZb4GGALbbinNXcrsCd1knInKEu8/FqtoYwH53WpXHCSMu3eoMGWNMIGcELYAHaMMpQ50GHBHAdmuAySIyAScBLMWpV9RBRPYC/gdYqKol/Yh7p9S5dYYSMuxmMmOMCSQRrAfGAy8Dp7rLXu1rI1VtEZHz3e2igftV9XMRuQZYq6orgZuAFJyKpgCbVXVxv1vRT42VTnmJlOzRwT6UMcYMeoHMR7A/gIgk891VPY8EsnNVfRF4scuyq3yeB3Jmscu1Vrt1hrLtjMAYY/yOEYhIuYgcJyJpIvI6sLuq/o/7qAlhjLuc1pTRqkKCjREYY0yvg8UZOPcNxALzgcxQBBQKMfWlVEalQVR0uEMxxpiw6+uqIfXzfEiLb/RQHT1s8poxxuyUvsYILgfOwUkC14lImbtcVfW4oEYWREnN5dTFZYU7DGOMGRT6SgRzfJ77zlM8pM8O0lorKI8vCHcYxhgzKPSWCCaELIoQam1TMrWCksSccIdijDGDQm9F50JSDjrUvJUV5EgDpNgVQ0NRc3MzRUVFNDT4v7k9PT2dL774IoRRhZ+1OTIE0uaEhAQKCgqIjY0NeL+B3FA2rFSWbicHiEnNDXcoZgCKiopITU1l/PjxvhMldVJdXU1qamqIIwsva3Nk6KvNqorH46GoqIgJEwLv1Amk1tCwUlO+DYD4DEsEQ1FDQwPZ2dl+k4AxkUxEyM7O7vWMuScRlwjqvTsASM6yu4qHKksCxvg3kP8fEZcIWqqc8hKpVmfIGGOACEwEWuMUObU6Q2YgPB4Ps2fPZvbs2YwaNYr8/PyO101NTb1uu3btWi644IJ+H/Ojjz5CRHjppZc6lm3cuJEZM2Z0Wm/ZsmXcfPPNHa9vvvlmpkyZwuzZs5k3bx4PP/xwv48NcP311zNp0iT22GMPXn755R7Xef3115kzZw4zZszgzDPPpKWlBXD6rC+44AImTZrErFmz+OCDDzq2eeihh5g8eTKTJ0/moYceApw+8PbPc/bs2eTk5PDLX/6y07GefPJJRKRjX01NTZx99tnMnDmTPffcs9PELevWrWPmzJlMmjSJCy64AFXnyvePPvqI/fbbj9mzZzN37lxWr14NgNfr5YQTTmDWrFnss88+fPbZZwB8+eWXneJKS0vj1ltvBeDjjz9m//33Z+bMmSxatIiqqqo+P7tbbrmF6dOnM2PGDE455ZSOrpwNGzaw7777MmnSJE4++eSOf1NvvfUWc+bMITMzkyeeeKI/v77AqOqQeuy99946UG+88Yb+323naNXVowa8j6HmjTfeCHcIu9T69ev7XKeqqioEkaheffXVetNNN3Va1tzcvMuPc9lll+lBBx2kZ5xxRseyDRs26PTp0zteV1VVdYrnrrvu0qOOOkorKytVVbWyslIffPDBfh/7888/11mzZmlDQ4N+++23uttuu2lLS0undVpbW7WgoEC//PJLVVW98sor9d5771VV1RdeeEEXLlyobW1t+u677+o+++yjqqoej0cnTJigHo9Hy8vLdcKECVpeXt7t+HPmzNE333yzUzsPPvhg3XfffbWwsFBVVf/617/qWWedpaqqxcXFOmfOHG1tbVVV1Xnz5um7776rbW1tunDhQn3xxRdVVfXII4/seP7CCy/ooYceqqqql1xyiS5btkxVVb/44gs97LDDusXU0tKiubm5unHjRlVVnTt3bkcs9913n/7ud7/r9bMrKirS8ePHa11dnaqqnnTSSfrAAw90PF++fLmqqv7kJz/RO++8U1Wd3/fHH3+sS5cu1ccff7zX35lqz/9PcKo+9/i9GnFXDcU2lFEZlUFkXWswPP3+uc9Zv62q2/LW1laiowdWR2ra6DSuXjS9X9ucddZZJCQk8OGHH3LggQeydOlSLrzwQhoaGkhMTOSBBx5gjz32oLCwkJtvvpnnn3+eZcuWsXnzZr799ls2b97ML3/5yx7PFlSVxx9/nFdffZWDDz6YhoYGEhIS+ozpj3/8I4WFhaSlpQGQlpbGmWee2a92ATz77LMsXbqU+Ph4JkyYwKRJk1i9ejX7779/xzoej4e4uDh23313AI488kiuv/56zj33XJ599lnOOOMMRIT99tuPiooKtm/fTmFhIUceeSRZWVkd27z00kuccsopHfv96quvKCkp4eCDD+5YduWVV3L55Zdz0003dSxbv349hx12GAAjR44kIyODtWvXMmbMGKqqqthvP+de2DPOOINnnnmGY445BhHp+Mu9srKS0aNHd+zr17/+NQBTpkxh48aNFBcXk5v73cUlr732GhMnTmTcuHEdcR5yyCEd7Tj66KO59tpr/X52Y8eOpaWlhfr6emJjY6mrq2P06NGoKq+//jqPPvooAGeeeSbLli3jZz/7GePHjwcgKio4nTgR1zWU2FRObYyVlzC7VlFREatWreLPf/4zU6ZM4e233+bDDz/kmmuu4Yorruhxm3//+9+8/PLLrF69mt///vc0Nzd3W2fVqlVMmDCBiRMnMn/+fF544YU+Y6mqqqK6uprddtutz3VvuummTl0e7Y/2pLR161bGjPluosGCggK2bu080WBOTg4tLS2sXbsWgCeeeIItW7b0un0g+12xYgUnn3xyx+DnBx98wJYtW/je977Xab0999yTlStX0tLSwoYNG1i3bh1btmxh69atFBQU9HiMW2+9lUsvvZQxY8ZwySWXcP3113fs66mnngJg9erVbNq0iaKiom5x+Sas6dOn8+yzzwLw+OOP99n2/Px8LrnkEsaOHUteXh7p6ekcddRReDweMjIyiImJ8fuZBEvEnRGktHipTBmWN01HHH9/uYfj+vKTTjqp4yyksrKSM888k6+//hoR6fELHuB73/se8fHxxMfHM3LkSIqLizt9cQEsX76cpUuXArB06VIefvhhlixZ4vfKkP5eMXLppZdy6aWX9mubno65YsUKLrroIhobGznqqKMGfEbma8WKFfz9738HoK2tjYsvvpgHH3yw23rnnHMOX3zxBXPnzmXcuHEccMABfR7/rrvu4pZbbmHJkiU89thjnHvuufzrX//i17/+NRdeeCGzZ89m5syZ7LXXXp321dTUxMqVKzsSB8D999/PBRdcwLXXXsvixYuJi4vr9dher5dnn32WDRs2kJGRwUknncQ//vEPFi5c2I9PZ9eKuESQ0VaBJ8HKS5hdKzk5ueP5lVdeyYIFC3j66afZuHGj38nG4+PjO55HR0d3DLC2a21t5cknn+TZZ5/luuuu67hZqLq6muzsbLxeb6f1y8vLmTBhAmlpaaSkpPDtt9/2eVZw00038cgj3eeZOuSQQ7jtttvIz8/v+AsXnDOf/Pz8buvvv//+vP322wC88sorfPXVVwB+t8/Pz+80qFtUVNTpc/r4449paWlh7733Bpzk/tlnn3Wss2PHDpYuXcpzzz3H3LlzueWWWzq2PeCAA9h9993JzMzs9Ne8b+wPPfQQf/nLXwAnif/4xz8GnC60Bx54AHC65SZMmNDpM/znP//JnDlzOnUVTZkyhVdeeQVwuonaz9r8tf1f//oXEyZMYMSIEQD84Ac/YNWqVZx66qlUVFTQ0tJCTEyM3886GCKqa6ipqZlMqaYteUS4QzHDWGVlZcd/4J7+gg3Ua6+9xqxZs9iyZQsbN25k06ZNLFmyhKeffpqUlBTy8vJ4/fXXAScJvPTSSxx00EEA/OY3v+HnP/95Rz94TU1Nj1cNXXrppXz00UfdHrfddhsAixcvZsWKFTQ2NrJhwwa+/vpr9tlnn277KSlxrsZrbGzkxhtv5Kc//WnH9g8//DCqynvvvUd6ejp5eXkcffTRvPLKK3i9XrxeL6+88gpHH310x/6WL1/eqfslPT2dsrIyNm7cyMaNG9lvv/1YsWIFc+fOpa6ujtraWgBeffVVYmJimDZtGnl5eaSlpfHee++hqjz88MMcd5xTNHn06NG8+eabgHPF0+TJkwGoqKjouFLn3nvv5ZBDDukYZ+kpLt+2t7W18Yc//KFT23v67MaOHct7771HXV0dqsprr73G1KlTEREWLFjQcVXQQw891BFvsEVUImisqwQgOtXqDJngueyyy/jNb37DXnvt1e2v/P5Yvnw5J5xwQqdlS5YsYfny5QA8/PDDXHvttcyePZtFixZx9dVXM3HiRAB+9rOfsWDBAubNm8eMGTM4+OCDBzTQOH36dH74wx8ybdo0Fi5cyB133NHRVXLssceybZtzp/5NN93E1KlTmTVrFosWLeoYvD322GPZbbfdmDRpEv/1X//FnXfeCUBWVhZXXnkl8+bNY968eVx11VUdA8cAjz32WLcvXH9KSkqYM2cOU6dO5cYbb+zoTgK48847+fGPf8ykSZOYOHEixxxzDAB/+9vf+NWvfsWee+7JFVdcwT333APAF198wYwZM9hjjz345z//2XHWAFBbW8urr77KD37wg07HX758ObvvvjtTpkxh9OjRnH322b1+dvvuuy8nnngic+bMYebMmbS1tXHeeecBcOONN/LnP/+ZSZMm4fF4OPfccwFYs2YNBQUFPPPMM/zkJz9h+vT+XdDQF1EdWhWl586dq+2DUv319KP3csJXv+LTA29n5pFn7OLIBqfCwkK/XRND0RdffMHUqVN7Xcdq0EQGa7N/Pf0/EZF1qjq3p/Uj6oxAG5w+1YRMu5nMGGPaRVQikAanayjFyksYY0yHiEoEMY0VAGTkWCIwxph2EZUI4poradBYElMywh2KMcYMGhGVCBJaKqiIygArY2yMMR0iKhEkt1ZSHZ0Z7jCMMWZQiahEkNpaSV2c1RkyA7czZajBuZx31apVva5z/PHHdxRKa3fWWWd1Kz+ckpLS8fzrr7/m2GOPZfLkycyZM4cf/vCHFBcX96NlDn9lkH01Nzdz5plnMnPmTKZOndqp3MJf/vIXZsyYwfTp0zvKNIP/Us3+9tVb2ed2t99+OyJCWVkZ4L+ENPgv+9zuggsu6PR5tmsved31kvXNmzeTkpLSqez3Oeecw8iRI7uVB+9v27ds2cKCBQuYNm0a06dP73Qvw+9+9zumTJnCrFmzOOGEE6ioqOgW84D4K0s6WB87U4Z6x1XjdPWtpwx4+6HIylAHT09lqHd2G6/XqwUFBTplyhT9z3/+07H8zDPP7FZ+ODk5WVVV6+vrdbfddtOVK1d2vPfGG2/op59+2q/YVP2XQfb1yCOP6Mknn6yqqrW1tTpu3DjdsGGDfvrppzp9+nStra3V5uZmPfzww/Xrr79WVf+lmv3ty1fXss+qqps3b9bDDjtMx44dq6Wlparqv4R0b2WfVVXXrFmjp512Wsfn2c635PWaNWs6vbdkyRI98cQTO/0u33zzTV23bl2n8uADafu2bdt03bp1HTFMnjxZP//8c1VVffrppztKnV922WV62WWXdfv9qFoZar/aWlvJoooNSVZeYtj4569hx6fdFie2tkD0AP9pj5oJx9zQr03WrVvHxRdfTE1NDTk5OTz44IPk5eVx2223cffdd3eUPLjhhhu4++67iY6O5h//+Ae33357pxLLAE899RSLFi0iNzeXFStW+K1c6uvRRx9ln332YdGiRR3LBnITofZSBtmXiFBbW9tRSjkuLo60tDTWrFnDvvvuS1JSEgCHHnooTz31FJdddpnfUs3+9uWra9lngIsuuohrr72WH/3oRx3L/JWQBnos+wxOPadLL72URx99lKeffrrTcXsqeQ3wzDPPMGHChE71pcCpz7Rx48Zun2t/256VlUVennOvU2pqKlOnTmXr1q1MmzaNww8/vKM66X777bfLJqmJmK6hivJSYqUVSbFEYHYdVeUXv/gFTzzxBOvWreOcc87ht7/9LQA33HADH374IZ988gl3330348eP56c//SkXXXQRH330UbckAN/VsjnllFM6Skn05bPPPmP27Nl9rtd19i/fx/r16wMug3ziiSeSnJxMXl4eY8eO5ZJLLiErK4sZM2bw9ttv4/F4qKur48UXX+wouuavVLO/ffnqWvb52WefJT8/n5kzZ3Zaz18JaX9lnwH++te/snjx4o4v3nb+Sl7X1NRw4403cvXVV/f5ebfbmbZv3LiRDz/8kH333bfbfu+///6Okhk7K2LOCKo828gCYlJz+1zXDBF+/nKvD2HpgcbGRj777DOOPPJIwPkLs/1LZdasWZx66qkcf/zxHH/88X3uq7i4mK+//pqDDjoIESE2NpbPPvuMGTNm9Fheur8lp1NTU/noo4/8vt/e196X1atXEx0dzbZt2/B6vRx88MEcccQRTJ06lcsvv5yjjjqK5ORkZs+e3VGXyF+pZn/7aq/42bXsc11dHX/84x87qn368ldC2l/Z58MOO4zHH3+8UxVU6L3k9bJly7jooot6HE/wZ6Btr6mpYcmSJdx6663dzpKuu+46YmJiOPXUUwOOozdBTQQishD4CxAN3KuqN3R5Px54GNgb8AAnq+rGYMRS43GKY8VnWCIwu46qMn36dN59991u773wwgu89dZbPPfcc1x33XV8+mn3bixfjz32GF6vlwkTnPkyqqqqWL58Odddd123stPl5eXk5Djl1KdPn86//vWvPmOtrq7u8SwEnO6lqVOnBlQG+dFHH2XhwoXExsYycuRIDjzwQNauXctuu+3Gueee21Eo7YorruiYX8Ffqebe9gXdyz7/5z//YcOGDey5556oKlu3bmXOnDmsXr2aUaNG9VhC+uWXX+6x7HNmZibffPMNkyZNApwkM2nSJNatW9et5PXixYtZuXIl77//Pk888QSXXXYZFRUVREVFkZCQwPnnn+/3cx9I25ubm1myZAmnnnpqtyJ3Dz74IM8//zyvvfZav/8Y8Mvf4MHOPnC+/P8D7AbEAR8D07qs8/+Au93nS4H/7Wu/Ax0sXvP8vapXp+mG9Wv6XnkYscHi4Ln66qv1xhtv1IkTJ+qqVatUVbWpqUk/++wzbW1t7Rj0bGpq0ry8PPV6vXrzzTfrVVdd1eP+9t9//479qGrHPLeqqs8995wefvjh2tjYqKqqf/rTn/Tss89WVdW6ujqdMGGCPv/88x3bvvnmmwMaLD7xxBM7DRbfcccd3da54YYbOuYIrqmp0alTp+rHH3+sqs6cwaqqmzZt0j322EO9Xm+n5a2trXr66afrfffd1+e+VFVPPvlkvf/++3uMtaqqSseNG9cxWOz1ejs+n3vuuUdPP/10VVV97733dNq0aVpbW6ttbW16xhln6G233dZtf10Hi9sdeuih3QaLVXse+O86l/RA2t7W1qann366Xnjhhd2O+eSTT+rUqVO1pKSkx1jb9XewOJhjBPsA36jqt6raBKwAuhbXPg54yH3+BHC47LIU11lLlTNolJZtBefMrhMVFcUTTzzB5Zdfzp577sns2bNZtWoVra2tnHbaaR1dFBdccAEZGRksWrSIp59+mtmzZ3dM5AJ0zDfge9nohAkTSE9P5/333+f73/8+Bx98MHvvvTezZ8/mnXfe4cYbbwQgMTGRxx9/nNtvv53Jkyczbdo07rzzzo6/gPvDXxnklStXctVVVwHw85//nJqaGqZPn868efM4++yzmTVrFuCUyZ42bRqLFi3ijjvuICPDuYvfX6nm3vblr+yzP/5KSPdW9nlXOuWUU9h///358ssvKSgo4L777htQ29955x3+/ve/8/rrr3eM4bz44osAXHLJJVRXV3PkkUcye/bsjrkPdlbQylCLyInAQlX9sfv6dGBfVT3fZ53P3HWK3Nf/cdcp67Kv84DzAHJzc/desWJFv+Op/GYVI7e/Rv2BVxAVtfPT6A0VNTU1/erPHOzS09M7TuX92ZnJ64cqa3NkCLTN33zzDZWVlZ2WLViwwG8Z6iExWKyq9wD3gDMfwYDq68+fT2HhARw2jGrzB2I4zkfQ10Cw1amPDNZm/xISEthrr70C3m8wu4a2AmN8Xhe4y3pcR0RigHScQWNjjDEhEsxEsAaYLCITRCQOZzB4ZZd1VgJnus9PBF7XYPVVmWHD/okY499A/n8ELRGoagtwPvAy8AXwmKp+LiLXiMhid7X7gGwR+Qa4GPh1sOIxw0NCQgIej8eSgTE9UFU8Hg8JCQn92i6oYwSq+iLwYpdlV/k8bwBOCmYMZngpKCigqKiI0tJSv+s0NDT0+z/CUGdtjgyBtDkhIaHj/o1ADYnBYmPaxcbGdtxw5U9hYWG/BsqGA2tzZAhWmyOm1pAxxpieWSIwxpgIZ4nAGGMiXNDuLA4WESkFNg1w8xwgsBKLw4e1OTJYmyPDzrR5nKr2WHdkyCWCnSEia/3dYj1cWZsjg7U5MgSrzdY1ZIwxEc4SgTHGRLhISwT3hDuAMLA2RwZrc2QISpsjaozAGGNMd5F2RmCMMaYLSwTGGBPhhmUiEJGFIvKliHwjIt0qmopIvIj8r/v++yIyPvRR7loBtPliEVkvIp+IyGsiMi4cce5KfbXZZ70lIqIiMuQvNQykzSLyQ/d3/bmIPBrqGHe1AP5tjxWRN0TkQ/ff97HhiHNXEZH7RaTEncGxp/dFRG5zP49PRGTOTh/U32TGQ/UBRAP/AXYD4oCPgWld1vl/wN3u86XA/4Y77hC0eQGQ5D7/WSS02V0vFXgLeA+YG+64Q/B7ngx8CGS6r0eGO+4QtPke4Gfu82nAxnDHvZNtPgSYA3zm5/1jgX8CAuwHvL+zxxyOZwT7AN+o6req2gSsAI7rss5xwEPu8yeAw0VEQhjjrtZnm1X1DVWtc1++hzNj3FAWyO8Z4FrgRqAhlMEFSSBt/i/gDlX1AqhqSYhj3NUCabMCae7zdGBbCOPb5VT1LaC8l1WOAx5Wx3tAhojk7cwxh2MiyAe2+Lwucpf1uI46E+hUAtkhiS44Ammzr3Nx/qIYyvpss3vKPEZVXwhlYEEUyO95d2B3EXlHRN4TkYUhiy44AmnzMuA0ESnCmf/kF6EJLWz6+/+9TzYfQYQRkdOAucCh4Y4lmEQkCvgzcFaYQwm1GJzuofk4Z31vichMVa0Ia1TBdQrwoKr+SUT2B/4uIjNUtS3cgQ0Vw/GMYCswxud1gbusx3VEJAbndNITkuiCI5A2IyJHAL8FFqtqY4hiC5a+2pwKzAAKRWQjTl/qyiE+YBzI77kIWKmqzaq6AfgKJzEMVYG0+VzgMQBVfRdIwCnONlwF9P+9P4ZjIlgDTBaRCSIShzMYvLLLOiuBM93nJwKvqzsKM0T12WYR2Qv4H5wkMNT7jaGPNqtqparmqOp4VR2PMy6yWFXXhifcXSKQf9vP4JwNICI5OF1F34YyyF0skDZvBg4HEJGpOInA/1ymQ99K4Az36qH9gEpV3b4zOxx2XUOq2iIi5wMv41xxcL+qfi4i1wBrVXUlcB/O6eM3OIMyS8MX8c4LsM03ASnA4+64+GZVXRy2oHdSgG0eVgJs88vAUSKyHmgFLlXVIXu2G2CbfwX8TUQuwhk4Pmso/2EnIstxknmOO+5xNRALoKp344yDHAt8A9QBZ+/0MYfw52WMMWYXGI5dQ8YYY/rBEoExxkQ4SwTGGBPhLBEYY0yEs0RgjDERzhKB2WkistGt7tn1sTFEx6wVkbdFZPZO7E/bqz2KyAEissx3fyJS6K6zS25UcvffHn+jWyl0UQDbJbnbntWPY10gInUiktXb70pExndZVi4iK0Qk292P77YVIvK8iIxy35vmLj9+wB+KCZthdx+BCYtfAMnA94FTgbuBN4Fa35VEJMat7bSrNOKUkJgJXAE8CUwc4L5OAdrLMByAc+32RuAjd9k1wEigaoD79+dWnOqaNwKPiEimqrb2sn6SG9ubwIN97dwtpvgr4ClVLReRQH5XH+Lcd3IicLL73rnue+2f+UKcmzKvBH6uqutF5B3gYpyb2sxQEu6Sq/YYPg/gEr67oQecm2IU5waY1cBanC8RBS5x13nefT3efX0O8CXOl88qYI6fY20EanxeF7v7ycG5/f4ZwItTifJWIB7nDPhunJsI64H1wGHu9gp85hOz72M8UOiz/9U4JUli3G0fA5pxEsVU4FWchLEJuMhP/Mvc/Z3ovn7VfZ3rvn7cjb/BjfMEn3b7xrYMpzzzzThlBircbUe46x/srvfD3n5X7rLx7rLn3ddT2z+Xrp85TpEzBf7ps/3lQBuQH+5/i/bo38O6hkwoHAE8DdzS20oiMh/nru+NwB9wKsI+JyIJvWyTIyKHu+vWvxfwhwAAA8FJREFU4HzJPwIsAv4b547UC3FqLO0J/AT4F/BT4Fm6nxWvd7cHJ2mcQvdyBY8AWcBhIpKIc5fnq+6xn8Wpif/fwPvAn/vo8kkVkT1wvnQ3+hxrDXAZ8Bv39cPu53CF+/oLN7Yn3HV+BTyHk/SOcWMHOMhnf4GKFZERQHs3z2bfN93usfaqpqt93lqDUyP/wH4cywwC1jVkQuF5Vb0eoI++7e+5P49yH+2mAR/0sH4y331x1gE/x+k6ORhYparXi0g8cAbOl+Md7nqzgRKcM443fHeoqiUi8hFOt8n7qrrCjdt3tRXAn3C6TlLdOP4B7MF3Bd6u9Vn/SJwv6Z7c7/5sAOarapuIRAPTcUqfxPmsOx54xX1e4hPbA+6yn3Q5JsBY92d/avQfhfP5gHOWcYXPe76f+fvAdT7vtR9jfD+OZQYBSwQmFHy/hNr7v9v/7WX0sP6vgE/c51HABj/7bcD5y78W+EJVK0QkxX2vW+0UVS0Wkek4f+nui/OX/TT4/+2dz4tNYRjHP8/OwmJSUnYKCzs2YzNLRRZq1ERGFmIj1ooit0Y2yiT+ArEbIaWumqkrWUhRShbYzUIWmoVieiy+7zE/3HNzZabrnu9nc+p2zvucc7qd5/1+37fn4eLqU+sf5dc47TLOCFIi94Ft5ZQnyKqpmO8xXAvZSleAG6WM8j6UvNplnLMoSW7ocW8/kO9fvd/Var+fxksvkIL6ArzNlZVqvwETwDmk9E4Ct/8ihhkgbA2Z9eZjOR6MiFOoPHRF1UDmKJrJjgLTWbptdWExM9uZ+TxLvf3MXECtKfeG+tveQv/zxxGxE9ktC+hjB7C1y7hVvAMRMVET+w6wGamCmVT3t3fAe2TH7EYK4QxqO1jH68xsIetslCU7BjT73sFKq+Ur8uG3R8SxUO/pRyixnkDvbT9L6qCydbo9Zx2fM/NpZr7K38uVL2bmQ+A4Wme5tMy6q2J86iOWGQCcCMx60wHuIntmHC0gA5CZs6iS4kZk45xG9k2/TKKP43nk308DU2g2uwetVUyVsa91uf4B8BI4DNQ1f59BNlMgW4jUjqhDwDOkMlrIOnrzB/d8Gc32L6D1hnvoHR1BCoMS4zva0TNS4o4BV8tvY8BNZIPNlUs65fhP+zBk5jxSAltQe8wqRi6Laf4TXH3UmCGmbB/9AHQyc3KNY3WQYhjq7nfDiBWBMUNMaqZ3HRiPiE1rFScidiELq+fOMDOYWBEYY0zDsSIwxpiG40RgjDENx4nAGGMajhOBMcY0HCcCY4xpOD8Blw2CRHLCoJsAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B80Bkd4Er1eo"
      },
      "source": [
        "###**9. Light Gradient Boosting Machine**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tuM9BazodyuE",
        "outputId": "07f06b03-c28e-4041-d6d2-98d7846004cb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "#LGBM\n",
        "from lightgbm import LGBMClassifier\n",
        "\n",
        "lgbm = LGBMClassifier(objective =\"binary\",verbosity = -1, learning_rate=0.1, n_estimators=1000, scale_pos_weight=2.5)\n",
        "    \n",
        "lgbm.fit(X_train, y_train)\n",
        "\n",
        "\n",
        "# evaluate predictions\n",
        "y_train_predict_lgbm = lgbm.predict(X_train)\n",
        "\n",
        "print(\"==========================\")\n",
        "print('Train Accuracy %.3f' % metrics.accuracy_score(y_train, y_train_predict_lgbm))\n",
        "\n",
        "# make predictions for test data\n",
        "y_pred_lgbm = lgbm.predict(X_test)\n",
        "predictions = [round(value) for value in y_pred_lgbm]\n",
        "\n",
        "\n",
        "accuracy_per_roc_auc = roc_auc_score(y_test, predictions)\n",
        "print(\"ROC-AUC: %.10f%%\" % (accuracy_per_roc_auc * 100))\n",
        "\n",
        "\n",
        "print('Test Accuracy %.3f' % metrics.accuracy_score(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.confusion_matrix(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print(metrics.classification_report(y_test, predictions))\n",
        "print(\"==========================\")\n",
        "print('Precision Score %.3f' % metrics.precision_score(y_test, predictions))\n",
        "print('Recall Score %.3f' % metrics.recall_score(y_test, predictions))\n",
        "print('F1 Score %.3f' % metrics.f1_score(y_test, predictions)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_label.py:235: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_label.py:268: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "==========================\n",
            "Train Accuracy 0.951\n",
            "ROC-AUC: 93.3265914451%\n",
            "Test Accuracy 0.939\n",
            "==========================\n",
            "[[114990   6836]\n",
            " [  2932  34971]]\n",
            "==========================\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.98      0.94      0.96    121826\n",
            "           1       0.84      0.92      0.88     37903\n",
            "\n",
            "    accuracy                           0.94    159729\n",
            "   macro avg       0.91      0.93      0.92    159729\n",
            "weighted avg       0.94      0.94      0.94    159729\n",
            "\n",
            "==========================\n",
            "Precision Score 0.836\n",
            "Recall Score 0.923\n",
            "F1 Score 0.877\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uo9OoYldd1wt",
        "outputId": "7e4bce61-0d30-4b7d-adea-a1e75cb3af3b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        }
      },
      "source": [
        "y_train_pred = lgbm.predict(X_train)    \n",
        "y_test_pred = lgbm.predict(X_test)\n",
        "\n",
        "\n",
        "train_fpr, train_tpr, tr_thresholds = roc_curve(y_train, y_train_pred)\n",
        "test_fpr, test_tpr, te_thresholds = roc_curve(y_test, y_test_pred)\n",
        "\n",
        "plt.plot(train_fpr, train_tpr, label=\"Train AUC =\"+str(auc(train_fpr, train_tpr)))\n",
        "plt.plot(test_fpr, test_tpr, label=\"Test AUC =\"+str(auc(test_fpr, test_tpr)))\n",
        "plt.legend()\n",
        "plt.xlabel(\"True Positive Rate(TPR)\", fontweight='bold')\n",
        "plt.ylabel(\"False Positive Rate(FPR)\", fontweight='bold')\n",
        "plt.title(\"ROC-AUC\")\n",
        "plt.grid()\n",
        "plt.show()  "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5fXA8e/JThL2TRY1KCr7ruAetCq4AEWqqK0oWq2ttS4/rdWqqLVKta11r7UWbS3UFVFRtMW4K6C4AS4ICHFhyUYyWSaTOb8/7p1hkkxmJiGTIZnzeZ55MvfOXd53Bt5z7/vee66oKsYYY5JXSqILYIwxJrEsEBhjTJKzQGCMMUnOAoExxiQ5CwTGGJPkLBAYY0ySs0BgjDFJzgKB6fBEZJOIVIlIhYh8LyILRCQ35PPDRGS5iJSLSJmIPCciwxpso4uI3Ckim93tfOVO94qy7wUi4hORfmHm/67BvDwRURFJC5l3poiscvf5nYi8KCJH7N43Ykx9FghMsjhFVXOBMcBY4DcAInIo8DLwLNAfGAR8BLwlIvu5y2QA/wOGA1OALsChQBFwSFM7FJEc4FSgDPhxcwssIpcDdwK/B/oC+wD3AdObuy1jIkmLvogxHYeqfi8iy3ACAsAfgEdV9S8hi/1WRMYD84Cz3dc+wGRVrXCX2QbcHGV3pwKlwB3AT4HbYy2niHQFbgLOVdWnQz56zn0Z02rsjMAkFREZCEwF1otINnAY8ESYRR8HjnPf/wB4KSQIxGoOsBBYBAxxg0usDgWygGeauU9jms0CgUkWi0WkHNiCczR/A9AD5//Ad2GW/w4I9P/3bGKZJonIPsBk4N+quhWna+nsZmyiJ7BDVX3N2a8xLWGBwCSLGaraGcgHhuA08iWAH+gXZvl+wA73fVETywAgIme5g7kVIvKiO/snwDpV/dCdfgw4U0TS3WkfkN5gU+luefzuPnuFDhwbEy8WCExSUdXXgAXAHarqAd4BfhRm0dNwjuIB/guc4A7+htvmY6qa676murPPBvZzr1L6HvgTTvA50f18M5DXYFODgC2q6nfLVQPMaH4tjWkeCwQmGd0JHCcio4GrgTkicomIdBaR7u5lnYcCN7rL/xOnS+kpERkiIiki0lNErhGRExtu3L0SaX+cK4rGuK8RwL/Z1T30FHCSiBwvIqki0h/4Lc54AqpaBlwP3CsiM0QkW0TSRWSqiPwhLt+KSVoWCEzSUdXtwKPA9ar6JnACMBNnHOBrnMtLj1DVL93la3AGjD8DXgF2AitwjvDfC7OLOcCzqvqJqn4feAF/AU4WkR6qugY4A7gVKMY5A3iPXcEHVf0jcDlOgNiOE4wuBha34tdhDGIPpjHGmORmZwTGGJPkLBAYY0ySs0BgjDFJzgKBMcYkuXZ3s0qvXr00Ly+vRet6PB5ycsJeCt5hWZ2Tg9U5OexOnd9///0dqto73GftLhDk5eWxatWqFq1bUFBAfn5+6xZoD2d1Tg5W5+SwO3UWka+b+sy6howxJslZIDDGmCRngcAYY5KcBQJjjElyFgiMMSbJxS0QiMjDIrJNRD5t4nMRkbtEZL2IfCwi4+JVFmOMMU2L5xnBApwHfTdlKnCA+7oAuD+OZTHGGNOEuN1HoKqvi0hehEWm4zw0XIF3RaSbiPRT1WY9EtAYY9oTX52fap+f6to69+WnprqK2qoyfJU7qavaSV11OVpTDtXlULMTaitI8Vaws9MBOA/Za12JvKFsAE5+9YBCd16jQCAiF+CcNdC3b18KCgpatMOKiooWr9teWZ2Tg9W5Zfyq1NaB1w/eOqXW/ev1Q20d1PoVr/t5bXC+89fnqyO1rpJUXxVpdVWk11WS7q8iva6KTH8lmf5qMrWSTlpFllaRHXhJNZ2pIocqcqWK3lSRKbE9mvrpbufG5XduF3cWq+qDwIMAEyZM0JbeWWd3IiYHq3P7pKrUBI+U/dT4nL/BI+eQo+iaWj8ff/0Z+3TZO8yyfqp9ddTUhsxzP6/x1pLqqyTNV0F6bSWZfg+54jTKnaWKHKrJdd/nug11LlXkUk2uVJJDdfCzTuKNqV5eyaQmLQdvaja1aTn40nKpS+9DXXouJemdKc7MhczOkJFLSlYXUjp1ITWrM2mdupKe3ZWMnC5k5nQlo1MXerzxRlx+50QGgm+AvUOmB7rzjDEJpqrU1qnbgDoNb2gjW11bv5EONOA1Dbo8GjXgPv+uBrrBdmp8/qZKQydq3Ia5ul4D7fn8f+RKFZ2lmr6p1XRJqaGrVNE5ZVejnk0l2VpFJ38lWVq1a7PpTdffn5JOXXpn/Bm5aEYumtEZyeyHZHUhJctpqMnq4jTgwYa8sztdf15GahoZrfrrtL5EBoIlwMUisgiYCJTZ+IAx4YXrV3YaT+f9h9t8VH7yXYMG2D0KDu2L9jXdCFc3WM7fwocXikBWWgpd0uromV5D99QauqXW0Du1hq4p1XRJqaZLehW5GU5DnYPTfeI01JVk+SvJqPOQ4fOQXuchzedBtKkgEbrjVLcR7rKrIc7sDRm5IQ105+DRd6NlM3KD0ylpmUl1bX3cAoGILMQZ1eglIoXADbgxWFUfAJYCJwLrgUrg3HiVxZjW5PdrsJENexTcsFsi5Ki4XtdHSGPdeP363R2+WFrlDz4IOzsjLYWstBSy0lPdl/s+LZXOWWn07pxJVnoqmWkp5KTW0SWlhi7iHn1LFdlUka2VZLtH1Fl+D5l1laS7jXWaz0NqbQWp7oAm3nKkphz8PojaeyJhGueekLmv2yiHzs+tN2/Vp18w4bD8XeunZTlRyDRbPK8aOiPK5wr8Il77N8kh0K9cE3KUW1ju56MtpWH7lev1JUfpVw422qGNfq0fb10MR6dNSE8VstJSyQxtkNNTyEpLpVNGKt2zM5xGOaSxrrecOy8zPYXMwGepsP7Dd5k0+gA6+d2G2u8ho66StNoKUmo9UONefVJT4b4vB2+FM68iZF5dTYwVyanfBZLZGTr3Dts10mheZpddjXt6NqS07Ni74us66Ll/i9Y19bWLwWLTPkTrV64J2wCH71fe1QgHGuBm9iu/9VbU8qamSL0j5cy0lF0NtHu0HHoU7TS8znJZYRrycA14ZnpqvX2kprhHrH4/BBvoil0NtTekUQ5tsCvLw8+vKYfaSiYBhL11M0RaVoOj7y7QZUDjefUa8ibmpaQ255+G2cNZIOjAfHV+qnzKjoqaGLolGvQrhzbcERrhhg15S/uVgXpHvIH3gYa0W3bGrnmh3RzBxntXg/zVl58xfvTIRo12oCEPbCc9tZlHoqpQWxXSCJfWb8h3hjbkTTXuFbvWJ4YvKyW9cf92bh/osV+9eeu3bGXwsDFN9Hu7y6VGGB01Sc0CQRvx+zVsV0PUbokGDXDDBjzcIF+jfuX//rfZ5Y3Wr9wrNzPsEXHDBrzhEXHDxj5wBJ6RmoK0Uv9uQfl68of23TXDV7OrUa6oqH803ai7pEGDHdqFUlMBWhe9AJK6qz870BBndYOuAxt3jTTo927U6KdlxlTnwoICBo/Pb9kXZpKeBYIYfFtaxdJPvqPK2/gousYX+jd8/3Nr9ytnNmhYu3VKb7JfuXDzRoYfdGDI0XRol8auRjwztHsjLYWUlAQPutXVNu4CCdctEmb+hKLv4EPdNc9fG8MOpUFDnLvr6Lth10i9vu9w/d6dbNDStCsWCGLw19e+4pF3nKe8pQj1uiWCR79uQ9ort3G/cmaYI+aG/cqNGuhw/cotUFDwDfmH5bXSNxGFvy7GrpEm5oU28L7q2PaZnt3oqpPqrD7kDhgUw2WCIQ15ek6LBy2Nae8sEMRge0UN+/XOYdmlRzW/X3lPpwpeT4MukIZdI+G6S8obNOTlzuBnLFIzGxx5d4Eu/WPvLskIadBTG/8T/rQD3GVrTFuyQBCDYo+XXjmZe04QUHWOmJvsLtnV7z14wzoofbyJSwfd9zENWqY17hbJ7gXd85ro9244L6QRT9vT77M0JrlYIIhBscfLoF45u78hnzf6ZYJN9Hs36kKJadAyhb1SOkF5910NdJZ79B1Tv3fIvLRM6/c2poOyQBCDYk8t4/fNdK79/m41VJaE6S5p4jLB0Pl1sSWpqncUHXif07vxvLBdJiF94enZvPnaa9ZNYoyJyAJBFKpKSaWXHjnpsPYZeHJu+AXTOjXu9+62d5grUcJ0l4Q25Bm5NmhpjGlTFgii2Fnto86vdM/OgJJNzsxzXoBOPXY17hmdww5aGmNMe2CtVxTFHqc7p2duBny/zTmizzsiwaUyxpjWY30QUQQCQffsDKjY6txgZIwxHYgFgihK3EDQIycDKrZBbt8oaxhjTPtigSAKOyMwxnR0FgiiKK4MGSOwMwJjTAdkgSCKEo+XzLQUOlHj3BNgZwTGmA7GAkEURR4vPXIyEM92Z4adERhjOhgLBFGUuIGAim3ODAsExpgOxgJBFMWVgUCw1ZlhXUPGmA7GAkEUxR7vriuGwM4IjDEdjgWCKIrrdQ2Jk3rZGGM6EAsEEdTW+Smv9u3qGsrpZTmFjDEdjgWCCAJ3FXe3u4qNMR2YBYIIgjeT5dhdxcaYjssCQQT100vYGYExpmOyQBBBIBD0yE63MwJjTIcVdeRTRPoBhwN57qyvgTdV9bs4lmuPEBgj6JlW5Txm0s4IjDEdUJOBQESmAZcDR9D4zMEvIm8Af1LV5+JYvoQq9tQC0LWu2JlhgcAY0wFFOiNYDLwFXAOsBL4FBOgPHAyc4i6TGucyJkyxp4YuWWmkVwXyDFnXkDGm44kUCIar6row8z8DlgPzRWRofIq1ZyiurKVnbqblGTLGdGhNDhY3EQQQkQwR+UWkZUKWnSIin4vIehG5Oszn+4jIqyKyWkQ+FpETm1uBeCrxeOkeGCgGOyMwxnRITQYCEekjIneLyAsicr2IpInIhcAm4K5oGxaRVOBeYCowDDhDRIY1WOy3wOOqOhaYDdzXwnrERZEnJOFcagZkdUt0kYwxptVF6hr6O3CS+36K+5oIVAN/jmHbhwDrVXUDgIgsAqYDa0OWUaCL+74rzjjEHqPE42VE/y677iEQSXSRjDGm1UUKBEcAC3CO2qcCfwNeAc5W1a0xbHsAsCVkuhAnkISaB7wsIr8EcoAfhNuQiFwAXADQt29fCgoKYth9YxUVFTGvq6rsqKjGU7yV4up1pPmz+KCF+02k5tS5o7A6Jwerc+uJFAi6Ai+q6rcishgnENwVYxCI1RnAAlX9o4gcCvxTREaoqj90IVV9EHgQYMKECZqfn9+inRUUFBDrup4aH75lyxg9ZH96rPVBnwNiXndP0pw6dxRW5+RgdW490e4sfkREdgIbcbpxHheRnSJSFsO2vwH2Dpke6M4LdR7wOICqvgNkAXtEnufiegnn7K5iY0zHFemMYDNO4x9Q1MxtrwQOEJFBOAFgNnBmmH0cCyxwL0XNArY3cz9xEQgEvTqlgGeHXTpqjOmwmgwEqpq3OxtWVZ+IXAwsw7np7GFVXSMiNwGrVHUJcAXwNxG5DCfonKOq2vRW204g82iv1ApA7YzAGNNhRUox0R34A84A79fANar6SXM2rqpLgaUN5l0f8n4tTh6jPU5xhRsItMSZYWcExpgOKlLX0H3A6e77EcAoERmsqrXxL1bilbhnBF211JlhgcAY00FFGiw+HngK52awa3EGexveENZhFXu8pKUI2TU7nBnWNWSM6aAiBYLuwJOq+hnOpaPizksKxR4v3XMykECeoRwLBMaYjina8wiuFJGzgHScwdzfi8gOQFV1etxLl0DFHq/7iMptkNkFMrITXSRjjImLaIFgnPsKmOT+3SOu7Imnkkqv+4hKu4fAGNOxRQoEg9qsFHugIo+XoXt1sWcVG2M6vGgpJjap6s62KsyepCSQebR4K+w1ItHFMcaYuIk0WLwaOEFEeopInYgc01aFSrQ6v1JaVeuml7AzAmNMxxYpEIj7IuRvUiit9KIKvTProKbMxgiMMR1atKRz2sT7Di1wM9leaRXODLt01BjTgUW7augRwIcTBJ4XkTp3vqpq17iWLIGKPc7N033E7io2xnR8sWYfbW7m0Xat2FMDQPdgniE7IzDGdFxxyz7angXOCLr6LOGcMabji/Tw+hOirRzLMu1RYIwgp3YHIJCzRzwrxxhj4iJS19BSEdkCPI/zkJlvca4e6g9MAE7GSUQXbZyh3Smq8JKTkUpa5XbI7gmp6YkukjHGxE2kRnwc8H/AHODn7BovEKASJzPpH+NaugQpqfTaPQTGmKQRaYzgI+AnIpIJjAX2cT/aDKxW1Zo2KF9C7Eo4Z3mGjDEdX9RuHVWtEZEPgGpgo6rG8uD6dq2k0k0vUboNeg5OdHGMMSauot1QhoiMBTYAq4CDRWSdiPwt7iVLoKIKLz06pdsZgTEmKUQNBMA9QAXO2IAf+Bfwg3gWKtFKKr30y/JCXY2NERhjOrxYAsFoYEHI9LdAhz1Mrq6to9JbR/90N+mqBQJjTAcXy6WfhcDR7vtRwBnApngVKNGKPW6eoZRAIOiwMc8YY4DYAsEfgIfc93/E6SI6J14FSrRAIOiF5RkyxiSHWK4aelhEvgJOcme9oKqvxbdYiRO4q7i73/IMGWOSQyxXDS0HRFWvUtWrAI+IPBj/oiVG4Iygs68YUtKhU/cEl8gYY+IrlsHifKB3yPTBwHlxKc0eIBAIsr07nG4hSapn8hhjklCkpHM3uM8fUGCR+7jKOpzLSbe1VQHbWonHS4pARvUO6xYyxiSFSGME24B1wDCcK4fKcIJCCfCX+BctMYo8XrplZyAV26DrwEQXxxhj4i5SrqH7gftF5B/A/aq6ou2KlTjB9BIVW2HAuEQXxxhj4i6Wq4bOFZFDReQCICtk/l1xLVmCFHu89OyUCtt22KWjxpikEDUQiMh1wLwwH0UNBCIyBacbKRV4SFVvC7PMae72FfhIVc+Mtt14KvZ4Gd2tBtRvYwTGmKQQyw1lFwAvAVOAW3EeSLMs2koikgrcCxyHM8awUkSWqOrakGUOAH4DHK6qJSKS8Ja32FPL3n0qnAk7IzDGJIFYLh/tg/OUMoD3gPuAWTGsdwiwXlU3qKoXWARMb7DMT4F7VZ2nxKtqQq9GUlUn4Vyqm2nbAoExJgnEckawAyfraClwN5DpvqIZAGwJmS4EJjZY5kAAEXkLp/tonqq+1HBD7vjEBQB9+/aloKAght03VlFREXFdT61S51fY/iUA767ZSPWGqhbta08Rrc4dkdU5OVidW08sgeBOnEtHbwT+jBMUrmrF/R+Ac9PaQOB1ERmpqqWhC6nqg8CDABMmTND8/PwW7aygoIBI627c4YH/FTC0TzoUwaRjT4GMnBbta08Rrc4dkdU5OVidW08sVw3dHngvIg/hJJ3LjmHb3wB7h0wPdOeFKgTeU9VaYKOIfIETGFbGsP1WF7iruFtdCWR0bvdBwBhjYhFxjEBExovIj0Rkf3dWZ+B3OE8si2YlcICIDBKRDGA2sKTBMotxzgYQkV44XUWxbDsuAoEg11diVwwZY5JGk2cEInINcLM7WSciV+B0D3Wj8ZF9I6rqE5GLca4wSgUeVtU1InITsEpVl7ifHS8ia4E64EpVLdqtGu2GEjcQZNXYPQTGmOQRqWvoIuAL4J/AEThjBaXAr3D766NR1aXA0gbzrg95r8Dl7ivhit0U1BnV26Hv8ASXxhhj2kakQNAPOFdV/yki/XDOAi5T1Ufbpmhtr8TjJTMtBfFsg9xjEl0cY4xpE5ECQQpwpogcjJNaQoEZIjIB52D+V21RwLZU5PGyVzZIdZmNERhjkka0q4ZOcF8BM9y/itNF1KGUeLwM6lQJXmyMwBiTNCIFgsltVoo9RHGll9GZ5c6EBQJjTJKIlIa6wz6XuCnFHi8DugYCgXUNGWOSQyy5hpJGscfLXqk7nQk7IzDGJAkLBK7aOj/l1T56SykgkNMr0UUyxpg2YYHAFbiZrLuWQHZPSE1PcImMMaZtRA0EItJJRG4XkQ9F5HARuct9mEyHEriZrGtdiXULGWOSSixnBHcClwEjcdJPpwJXxrNQiRDIM5TjLbKBYmNMUoklEMwEbg+Zfh84KD7FSZxAIMis2WGBwBiTVGIJBH6c1NMBo4GK+BQncZwxAiW9arsFAmNMUonlwTQvsCsp3D+BvYCH4laiBCn21NKZKsRXbWMExpikEksguBTnjOAkIB14BPi/eBYqEYo9NQzKsofWG2OSTyyB4BjgAvcpYh1WcWWtEwiqsa4hY0xSiWWM4GngexG5X0QOj3eBEqXE42WfDMszZIxJPrEEgouBj4HzcR4uv0FEbo6yTrtT5PHSP80CgTEm+UQNBKp6n6pOBvoD9wP7AtfEu2BtrcTjpW9KGaSkQ1a3RBfHGGPaTNQxAhEZA8wCTsV5uDzAm/EsVFtTVYorvfTqUuqMD6RY5g1jTPKIZbD4A/fvGpwzgX+r6pb4Fantebx1eH1+uvlLbKDYGJN0YgkEtwOPqerH8S5MogQSznX2FUNuXmILY4wxbazJQCAiM4F3gfeAwSIyOPRzVX06zmVrM4H0EtneIsg9JMGlMcaYthXpjOAJ4AxgEc4zigPEnU6NY7naVLHHSwp+MmqK7IohY0zSiRQIbsIZF7ixjcqSMMUeLz0oR9RvgcAYk3QiPbP4RgARGQ+8pqpfu9M96WDZR0sqve6TybDBYmNM0onlOsl/AKEd58cBb8SnOIlR7PGyV0qZM2FnBMaYJBNpsHgaMANnTODnIjLV/WgsTkaeDqPY42XfrAqow84IjDFJJ9IYwVjgHJyB4aPdV8BjcSxTmyv2eJmYXu4EghwLBMaY5BIpEDyI8yyCFcC1wMs4QaFEVTe2QdnaTEmll36pOyEjFzJzE10cY4xpU5ECQQ3wIdCb+pePIiI9VLU4ngVrS0UeL32kzLqFjDFJKVIg2I5zH8HCMJ9plHXblRKPlx7ZJTZQbIxJSpEa89dxgsEbhL+hLCoRmQL8Befms4dU9bYmljsVeBI4WFVXxbLt1lLnV0qraumaWQK5o9ty18YYs0eIdB/BZPftqy3ZsIikAvfiXG5aCKwUkSWqurbBcp2BX+GksmhzpZVeVCG31u4qNsYkp6j3EYjIZSLySxHpJiKvi8iXInJyDNs+BFivqhtU1YuTqmJ6mOVuBuaToEtSSyq9ZFBLpq/cxgiMMUkp1ofX3wv8GOeS0jLgD8DzUdYbAISmqy4EJoYuICLjgL1V9QURubKpDYnIBcAFAH379qWgoCCGYjdWUVHRaN3Pi+vohXMz2WfflPJ9C7e9pwpX547O6pwcrM6tJ5ZA0Bf4Fuch9v/AeT7B/bu7YxFJAf6Ec69CRKr6IM7lrEyYMEHz8/NbtM+CggIarlv96Xf0XvkpAEPGH8WQA1u27T1VuDp3dFbn5GB1bj2xpJjYBvwfMA14H8gGymNY7xtg75Dpge68gM7ACKBARDYBk4AlIjIhhm23mmJPLb0lkF7CuoaMMcknlkBwCzAI2AA8DYwntlxDK4EDRGSQiGQAs4ElgQ9VtUxVe6lqnqrm4Tz7YFpbXzVUP+GcDRYbY5JP1K4hVf0r8FcRyXanz4tlw6rqE5GLgWU4l48+rKprROQmYJWqLom8hbZRVOFlQOpOZyKnd2ILY4wxCRDLw+sH4OQWOsKZlNeAn6jqN5HXBFVdCixtMO/6JpbNj6XAra2k0svI9HLI7Amp6YkogjHGJFQsXUP34gSBd4G3gaOAu+NZqLZU7PGyV2qZdQsZY5JWLIHgaOAaVT1CVY/ESUCXH9dStaFij5delmfIGJPEYgkElcBBIpImIunAgUBVfIvVdoo9Xnr4Lc+QMSZ5xRIIHgfOxWn8K3Gu+/9PHMvUpkoqa+hSV2xnBMaYpBXLDWW/xrlvIPCEsheA38etRG2ouraOFG8F6Sk1dkZgjElaEQOBiOQB44AlTV3t054Ve7whN5NZIDDGJKcmu4ZE5EfAZ8ATwHsicleblaqNFHu89CZwM5l1DRljklOkMYIbcTKCPgF8jfMA+/5tUqo24txVbGcExpjkFikQDAauVtXZwBR32f3bpFRtxOkasvQSxpjkFmmMIA3o6qaK7urOGyYiHgBV/SDehYu3QCDQlDQkq1uii2OMMQkR7aqh31P/CqH7Qt6ntn5x2laJx8veUgY5fSAllitpjTGm44kUCB5ps1IkSJHHyyGpOxEbKDbGJLFIzyw+ty0LkggllV76ppRB7gGJLooxxiRMUveHFHu89KTULh01xiS1pA4EpRVVdPWX2hVDxpikltSBoM5TQip+CwTGmKQWS64hRGQUzsPrlwADgE2quiWeBYs3VSW9ehukY11DxpikFssTymYD/wIE+Bj4DVAB/DC+RYuvnVU+eqrdTGaMMbF0Dd0I/C9k+nngsPgUp+0UV1qeIWOMgdgCQX9geci0D+gUn+K0Hcs8aowxjljGCD4Bznbf/wQn79BHcStRGwmkl6hLyyY1MzfRxTHGmISJ5YzgCmAvnDGCOTjDq/8Xz0K1hRL3jMCfY91CxpjkFvWMQFXfEZHBwKHurLdVA6Os7VdxpZdRlJLS2bqFjDHJLeoZgXvV0FTgVWA28ISITIp3weKt2OOlT0qZBQJjTNKLpWvoJpxnE5wJnAFMAu6JZ6HaQrHHSx8pQ2yg2BiT5GIJBHsDG4BDgIeBy4Fh8SxUWyivqKALFXbFkDEm6cUSCMqAc4ATgQ9wBo2r4limNlFXvs15Y/cQGGOSXCyB4AGc9BIpwNM4N5N9HM9CtYXUykAgsDMCY0xyi+WqoXkicidQoao+EbkY56aydi2jeofzxs4IjDFJrslAICJ3hZkXeKvAr+JUprirrfOTU1vsJpyzMwJjTHKLdEZwcYTPYgoEIjIF+AvO840fUtXbGnx+OXA+zhnGdmCuqn4dbbu7q8QTkmcop3e8d2eMMXu0SIFg8u5sWERSgXuB44BCYKWILFHVtSGLrQYmqGqliFwE/AE4fXf2G4viSueuYm9GNzLSMuK9O2OM2aNFembxa6HTIpIJZDZj24cA61V1g7v+ImA6EAwEqvpqyPLvAj9uxvZbLJBnyNepDxYGjDHJLpbnEQO9y40AACAASURBVBwB/B3nprIAjWHdAUDow2sKgYkRlj8PeLGJMlwAXADQt29fCgoKouw6vIqKCgoKCljxnY9jpIwyfxYrWrit9iJQ52RidU4OVufWE0v20TuBHJz7B94HRgKvRVyjmUTkx8AE4Ohwn6vqg8CDABMmTND8/PwW7aegoID8/Hy2vLOJ3utK6T7wSFq6rfYiUOdkYnVODlbn1hPLfQRDcQZ8FefpZNcC1TGs9w3OXckBA9159YjID9xtTlPVmhi2u9uKK5wxgvSu/dpid8YYs0eL5YzABxQBfpw01F2AH8Sw3krgABEZhBMAZuPkKwoSkbHAX4EpqrqtGeXeLRXlJWRLDVjCOWOMiSkQrAXygGXAWe68V6KtFHLz2TKcy0cfVtU1InITsEpVlwC3A7k4GU0BNqvqtGbXopl2pZewQGCMMbHcWXwogIjksOuqnsdi2biqLgWWNph3fcj7WM4sWp1UbHXe2F3FxhjT9BiBiBSLyHQR6SIiy4EDVfWv7quiDcvY6lIqtztv7IzAGGMiDhZ3w7lvIB3IB7q3RYHaQla1BQJjjAmIdtWQNvG+3VJVsmuLqJNU6NRhYpsxxrRYtDGCXwNzcYLALSLipuxEVXV6XEsWJx5vHT38pVR16kluSixXzxpjTMcWLRCMC3kf+pzidnt2UOKml/Bm9Up0UYwxZo8QKRAMarNStKFAniF/dl6ii2KMMXuESEnn4p4OOhGKPV6GSpndTNZO1NbWUlhYSHV1LDezO7p27cq6deviWKo9j9U5OcRS56ysLAYOHEh6enrM243lhrIOpbiiml6UUdFlr0QXxcSgsLCQzp07k5eXF/pgpIjKy8vp3LlznEu2Z7E6J4dodVZVioqKKCwsZNCg2Dt1km60tKpsG2niJ7O75RlqD6qrq+nZs2fMQcCYZCYi9OzZs1ln0JCEgaCm7HsAsrpZIGgvLAgYE7uW/H9JukCgO530EmLpJYwxBkjCQJDisYRzJjZFRUWMGTOGMWPGsNdeezFgwIDgtNfrjbjuqlWruOSSS5q9zw8//BAR4aWXXgrO27RpEyNGjKi33Lx587jjjjuC03fddRdDhgxhzJgxHHzwwTz66KPN3jfArbfeyuDBgznooINYtmxZ2GWWL1/OuHHjGDFiBHPmzMHn8wFOrvyuXbsGv6ObbropuM5LL73EQQcdxODBg7nttl2PLt+4cSMTJ05k8ODBnH766fW+18cff5xhw4YxfPhwzjxzV+Li1NRUxowZw+GHH860abtyVJ5zzjkMGjQouP8PP/ywXrlXrlxJWloaTz75ZHDe5s2bOf744xk6dCjDhg1j06ZNABx55JHB7fTv358ZM2YATh/8JZdcwuDBgxk1ahQffPBBcFtXXXUVw4cPZ+jQoVxyySWoOlfZT5kyhdGjRzN8+HB+9rOfUVdXF1zn7rvvZsiQIQwfPpyrrroq6u8wYsQIRo4cyZgxY5gwYULY36dFVLVdvcaPH68t9eqrr+ojt1+mekMX1eqdLd5Oe/Lqq68mugi7Ze3atc1eZ+fO1v9tb7jhBr399tvrzautrW31/Vx11VV6xBFH6Nlnnx2ct3HjRh0+fHiT5bn//vv1mGOO0bKyMlVVLSsr0wULFjR732vWrNFRo0ZpdXW1btiwQffbbz/1+Xz1lqmrq9OBAwfq559/rqqq1113nT700EOq6vxbO+mkkxpt1+fz6X777adfffWV1tTU6KhRo3TNmjWqqvqjH/1IFy5cqKqqF154od53332qqvrFF1/omDFjtLi4WFVVt27dGtxeTk6Oqjb+nefMmaNPPPFE2Lr5fD6dPHmyTp06td4yRx99tL788suqqlpeXq4ej6fRujNnztRHHnlEVVVfeOEFnTJlivr9fn3nnXf0kEMOUVXVt956Sw877DD1+Xzq8/l00qRJwf97gd/F7/frzJkzg/Vdvny5HnvssVpdXV2vjpF+h3322Ue3b98eto6hwv2/wcn6HLZdTbqrhrJqdlAjWWRm5Ca6KKaZbnxuDWu/3Rl1ubq6OlJTU2Pa5rD+XbjhlOExl+Gcc84hKyuL1atXc/jhhzN79mx+9atfUV1dTadOnfjHP/7BQQcdREFBAXfccQfPP/888+bNY/PmzWzYsIHNmzdz6aWXhj1bUFWeeOIJXnnlFY488kiqq6vJysqKWqbf//73PP/883Tp0gWALl26MGfOnJjrFPDss88ye/ZsMjMzGTRoEIMHD2bFihUceuihwWWKiorIyMjgwAMPBOC4447j1ltv5bzzzmtyuytWrGDw4MHst99+AMyePZtnn32WoUOHsnz5cv79738DMGfOHObNm8dFF13E3/72N37xi1/QvbuTBqZPn93ryr377rs59dRTWblyZXDe2rVr8fl8HHfccQDk5jZuE3bu3Mny5cv5xz/+ATjf0dlnn42IMGnSJEpLS/nuu+8QEaqrq/F6vagqtbW19O3r9DoEfhefz4fX6w324d9///1cffXVZGZm1qtjLL9Da0u6rqHc2iI86T3ABiBNCxUWFvL222/zpz/9iSFDhvDGG2+wevVqbrrpJq655pqw63z22WcsW7aMFStWcOONN1JbW9tombfffptBgwax//77k5+fzwsvvBC1LDt37qS8vDymSwVvv/32YHdH6CsQlL755hv23nvXQwUHDhzIN9/Uf6hgr1698Pl8rFq1CoAnn3ySLVt2PZr8nXfeYfTo0UydOpU1a9ZE3G5RURHdunUjLS2t0f6++OILvvjiCw4//HAmTZpUr6usurqaCRMmcMwxx7B48eJ65bv22msZNWoUl112GTU1NcH9P/PMM1x00UX1lv3iiy/o1q0bM2fOZOzYsVx55ZX1um0AFi9ezLHHHhtszJuqy6GHHsrkyZPp168f/fr144QTTmDo0KHB5U444QT69OlD586dmTVrVnD/b7zxBhMnTuToo48OBqlIv4OIcPzxxzN+/HgefPBBWktSnRH4VenqL6Eq09JLtEexHrnH+/ryH/3oR8EzjrKyMubMmcOXX36JiIRt4AFOOukkMjMzyczMpE+fPmzdupWBAwfWW2bhwoXMnj0bcI6aH330UU499dQmrwJp7tUhV155JVdeeWWz1gm3z0WLFgUb2uOPPz74XYwbN46vv/6a3Nxcli5dyowZM/jyyy9btB+fz8eXX35JQUEBhYWFHHXUUXzyySd069aNr7/+mgEDBvDxxx8zbdo0Ro4cyf7778+tt97KXnvthdfr5YILLmD+/Plcf/31XHrppcyfP5+UBrnFfD5fMIjvs88+nH766SxYsKDe2c3ChQs5//zzo5Z3/fr1rFu3jsLCQsA5U3rjjTc48sgjAVi2bBnV1dWcddZZLF++nOOOOw6fz0dxcTHvvvsuK1eu5LTTTmPDhg0R97Ns2TIOOuggtm3bxnHHHceQIUM46qijmvv1NpJUZwQVtdCbUmo79U50UUw7lpOTE3x/3XXXMXnyZD799FOee+65Jq/fDpz+gzPYGRhgDairq+Opp57ipptuIi8vj1/+8pe89NJLlJeX07NnT0pKSuotX1xcTK9evejSpQu5ubls3LgxarmjnREMGDCg3tF9YWEhAwYMaLSdQw89lDfeeIMVK1Zw1FFHBbuJAmUBOPHEE6mtrWXHjh1Nbrdnz56UlpYGv4vQ/Q0cOJBp06aRnp7OoEGDOPDAA4NBJbDMoEGDyM/PZ/Xq1QD069cPESEzM5Nzzz2XFStWAM7A/ezZs8nLy+PJJ5/k5z//OYsXL2bgwIGMGTOG/fbbj7S0NGbMmFFv8HfHjh2sWLGCk046KTivqbo888wzTJo0idzcXHJzc5k6dSrvvPNOve8tKyuL6dOn8+yzzwbrOHPmTESEQw45hJSUlIjfF0D//v0Bpxvphz/8YbCOuyu5AoFX6S1laI5dOmpaR1lZWfA/6YIFC1q8nf/973+MGjWKLVu2sGnTJr7++mtOPfVUnnnmGXJzc+nXrx/Lly8HnCDw0ksvccQRRwDwm9/8hiuuuIKdO53xk4qKirBXDV155ZV8+OGHjV533XUXANOmTWPRokXU1NSwceNGvvzySw455JBG29m2zbnyrqamhvnz5/Ozn/0MgO+//z54pcyKFSvw+/307NmTgw8+mC+//JKNGzfi9XpZtGgR06ZNQ0SYPHly8CqeRx55hOnTnaTGM2bMoKCgAHAa5C+++IL99tuPkpKSYJdPUVERb731FsOGDQPgu+++A5yxlsWLFwevtNq4cSObNm1i06ZNzJo1i/vuu48ZM2Zw8MEHU1payvbtzvNJli9fHtwWON1eJ598cr1xmmnTpvHoo4+iqrz77rt07dqVfv36sc8++/Daa6/h8/mora3ltddeY+jQoVRUVATL5fP5eOGFFxgyZEiwjq+++irgdBN5vV569erV5O/g8XgoLy8HwOPx8PLLLze6mqylkqpryFPtpbtUsNPyDJlWctVVVzFnzhx+97vf1TtybK6FCxfywx/+sN68U089lfvvv5+zzz6bRx99lF/84hdcfvnlANxwww3sv//+AFx00UUUFRVx8MEHk56eTnp6OldccUWzyzB8+HBOO+00hg0bRlpaGvfee2+w2+fEE0/koYceon///tx+++08//zz+P1+LrroIo455hjAaTjvv/9+0tLS6NSpE4sWLUJESEtL45577uGEE06grq6OuXPnMny40803f/58Zs+ezW9/+1vGjh0b7JY54YQTePnllxk2bBipqancfvvt9OzZk7fffpsLL7yQlJQUfD4fV199dbDxPuuss9i+fTuqypgxY3jggQci1jc1NZU77riDY489NnBFIj/96U+Dny9atIirr7663jonnngiS5cuZfDgwWRnZwcHkWfNmsXy5csZOXIkIsKUKVM45ZRT2Lp1K9OmTaOmpga/38/kyZODgXPu3LnMnTuXESNGkJGRwSOPPIKINPk7bN26lenTpwfrfuaZZzJlypRm/87hSCCCtxcTJkzQwEBVc/31n4u48KsL+fbI2+h/7EXRV+gACgoKyM/PT3QxWmzdunX1Bt1iYTlokoPVuWnh/t+IyPuqGvbmg6TqGpJqp581q3v/BJfEGGP2HEkVCFJrnECQ09MCgTHGBCRVIEh3A0GmJZwzxpigpAoEWb5S502OXT5qjDEBSRUIcnyl7JTOkJYZfWFjjEkSSRUIcn2llKf1SHQxjDFmj5JUgaCrluFJ75noYph2YnfSUINz6e7bb78dcZkZM2YwadKkevPOOeeceqmSoX5CtC+++IITTzyRAw44gHHjxnHaaacFb/JqjkgpoAO8Xi/nnnsuI0eOZPTo0cGbvKDp9MrXXXcdo0aNYsyYMRx//PF8++23ADz22GOMGjWKkSNHcthhh/HRRx8Ft1VaWsqsWbMYMmQIQ4cODd6VO2/evHrf+9KlS4Pluuiii8KW69prr2XvvfcOm0QO4KmnnkJEaHgZ+ubNm8nNza2X3jsvLy9s2ucnnniC4cOHk5KSUm87r7zyCuPHj2fkyJGMHz8+eBNgZWUlJ510UjDldOj9Ca+//jrjxo1rlCIbdqXcHjNmTL2U262uqbSke+prd9JQb7r+AP3kL7NavH57ZGmoW0e4NNS7u05JSYkOHDhQhwwZol999VVwfrh0yoHUy1VVVTp48GBdsmRJ8LNXX31V33333WaVTbXpFNCh7rnnHj3nnHNU1UmTPG7cOK2rq1PVptMrB+arqv7lL3/RCy+8UFWdVM2BtNJLly4NpnBWVT377LP1b3/7m6qq1tTUaElJiao2/R3ec889etZZZ4Ut1zvvvKPffvtt8DsLtXPnTj3yyCN14sSJunLlynqfnXrqqTpr1qx6+9t3333Dpn1eu3atfvbZZ3r00UfX284HH3yg33zzjaqqfvLJJ9q/f39VVfV4PLp8+fJg/Y444ghdunSpqjppxj/66CP9yU9+0uTvHlr+WFga6iZU19bRi1JKsm2guN168Wr4/pOoi3Wq80FqjP+09xoJU2+Lvpzr/fff5/LLL6eiooJevXqxYMEC+vXrx1133cUDDzxAWloaw4YN47bbbuOBBx4gNTWVf/3rX9x9993BBGQBTz/9NKeccgp9+/Zl0aJFTWYuDfXvf/+bQw89lFNOOSU4Lz8/P5h6IFaq2mQK6FBr164N3jncp08funXrxqpVqzjkkEOaTK8cmA9OKoTA/MMOOyw4f9KkScEEbWVlZbz++uvBFB0ZGRlkZGRELP/atWuDydYalqvhGVao6667jl//+tfcfvvt9eYvXryYQYMG1csjFUlTNzmOHTs2+H748OFUVVVRU1NDdnY2kydPDtZv3Lhxwfrn5eUBNEqK15aSpmuopKSEHKlB7clkpoVUlV/+8pc8+eSTvP/++8ydO5drr70WgNtuu43Vq1fz8ccf88ADD5CXl8fPfvYzLrvsMj788MNGQQCctBJnnHEGZ5xxBgsXLoypDJ9++injx4+Pulx5eXnYBHNjxoxh7dq1EVNAhxo9ejRLlizB5/OxceNG3n///XoJ0cKlV4Zd3TOPPfZYvSeVBfz9739n6tSpgNNF1bt3b84991zGjh3L+eefj8fjCS57zz33MGrUKObOnRtMvjd69GhefPHFJssVzgcffMCWLVsapQKpqKhg/vz53HDDDY3W2Z20z0899RTjxo2rl3AQnG6w5557jmOPPTbqNgIptydNmtQo5XZrSpozgvId39APSO1igaDdivHIvSpOqQdqamr49NNPgw8yqauro18/556UUaNGcdZZZzFjxozgYw0j2bp1K19++SVHHHEEIkJ6ejqffvopI0aMCJteurkppzt37tzoUY2hduzYEdN25s6dy7p165gwYQL77rsvhx12WL2H/oRLrwxwyy23cMstt3Drrbdyzz33cOONNwbXefXVV/n73//Om2++CThnFB988AF33303EydO5Fe/+hW33XYbN998MxdddBHXXXcdIsJ1113HFVdcwcMPP8zcuXP56KOPmixXQ36/n8svvzxsYsB58+Zx2WWXhR1TePPNNxkwYECz0z6vWbOGX//617z88sv15vt8Ps444wwuueSS4IN6Igmk3N6wYQPHHHMMzz77LKNHj466XnPFNRCIyBTgL0Aq8JCq3tbg80zgUWA8UAScrqqb4lGWymJnwMpuJjMtpaoMHz68UXphgBdeeIHXX3+d5557jltuuYVPPonchfX4449TUlISfKDMzp07WbhwIbfcckujtNOBlNPgdDe89tprUctaXl4e9iwEnO6loUOHBlNAp6WlNZlyOi0tjT//+c/B6cMOOyyYdjogNL1yIBAEnHXWWZx44onBQPDxxx9z/vnn8+KLL9Kzp3PhxsCBAxk4cCATJ04EnARugecaB57yBfDTn/6Uk08+OViu2267LRjww5Wr4ffx6aefBvNuff/990ybNo0lS5bw3nvv8eSTT3LVVVdRWlpKSkoKWVlZXHzxxcHvJDTtc7RAUFhYyA9/+EMeffTRYGLAgAsuuIADDjiASy+9NOI2AgL732+//cjPz+fjjz+OSyCIW9eQiKQC9wJTgWHAGSIyrMFi5wElqjoY+DMwP17l8ZY5qWA7WZ4h00KZmZls3749GAhqa2tZs2YNfr+fLVu2MHnyZObPn09ZWRkVFRV07ty5yb77hQsX8tJLLwXTI7///vssWrQIcPr8//Of/wSv4lmwYEGwf/nMM8/k7bffrvf0stdff521a9fW237gjCDca9iwYRFTQIeqrKwMdtO88sorwTGQSOmVQx9G8+yzzwbnb968mZkzZ/LPf/6zXqO91157sffee/P5558DTkruhqmlAZ555plg2uWmytWUrl27smPHjuD3PWnSJJYsWcKECRN44403gvMvvfRSrrnmGi6++OIWpX0uLS3lpJNO4rbbbuPwww+v99lvf/tbysrKuPPOOyNuIyA05faOHTt46623gt9lq2tqFHl3X8ChwLKQ6d8Av2mwzDLgUPd9GrADNyNqU6+WXjX0zr9/r3pDFy3Z9k2L1m+v7Kqh1hG4emX16tV65JFH6qhRo3TYsGH64IMPqtfr1cMPP1xHjBihw4cP11tvvVVVVT///HMdOXKkjh49Wl9//fXgtjZu3Kj9+/dXv99fbx9jx44NXv0zb948HTFihI4ePVpnzpyp27ZtCy63bt06PeGEE3Tw4ME6dOhQPf3003X9+vXNrtNXX32lBx98sO6///46a9as4EPUn332Wb3uuuuCZT3wwAN1yJAheuyxx+qmTZtUVfX777/XCRMm6MiRI3X48OF68cUXa21trao6D3sfPny4jhw5Uk8++WQtLCxUVdXzzjtPu3XrpqNHj9bRo0dr6P/l1atX6/jx43XkyJE6ffr04NVFP/7xj3XEiBE6cuRIPeWUU/Tbb78Nlmvw4MGNyqWqeuWVV+qAAQNURHTAgAF6ww03NKp7w6t9AkKvUvrqq6901KhRwd/6d7/7XXC5p59+WgcMGKAZGRnap08fPf7441VV9eabb9bs7OxgHUePHq1bt27VLVu2KKBDhgwJzg9cJbVixQodMGCAZmdna48ePXTYsGGq6lxlNWLECB01apSOGDFCH3roobhdNRS3NNQiMguYoqrnu9M/ASaq6sUhy3zqLlPoTn/lLrOjwbYuAC4A6Nu37/jAkVNzlK1/mz7f/Y+qw68hJSW2B5t3BBUVFU1eT90edO3alcGDBzdrneY8vL6jsDonh1jrvH79esrKyurNmzx5cpNpqNvFYLGqPgg8CM7zCFqUXz8/n4KCwzimHefmb4mO8DyC5g78Wp765GB1blpWVla9S1mjieflo98Ae4dMD3TnhV1GRNKArjiDxsYYY9pIPAPBSuAAERkkIhnAbGBJg2WWAHPc97OA5RqvvirTbtk/CWNi15L/L3ELBKrqAy7GGRBeBzyuqmtE5CYRCSTN+DvQU0TWA5cDV4ffmklWWVlZFBUVWTAwJgaqSlFREVlZWc1aL65jBKq6FFjaYN71Ie+rgR/FswymfRs4cCCFhYVs37495nWqq6ub/R+hvbM6J4dY6pyVlcXAgQObtd12MVhskld6enrwpqtYFRQUNGugrCOwOieHeNU5aXINGWOMCc8CgTHGJDkLBMYYk+TidmdxvIjIduDrFq7eCyeNRTKxOicHq3Ny2J0676uqYR/I0u4Cwe4QkVVN3WLdUVmdk4PVOTnEq87WNWSMMUnOAoExxiS5ZAsEzXvWXMdgdU4OVufkEJc6J9UYgTHGmMaS7YzAGGNMAxYIjDEmyXXIQCAiU0TkcxFZLyKNMpqKSKaI/Mf9/D0RyWv7UrauGOp8uYisFZGPReR/IrJvIsrZmqLVOWS5U0VERaTdX2oYS51F5DT3t14jIv9u6zK2thj+be8jIq+KyGr33/eJiShnaxGRh0Vkm/sEx3Cfi4jc5X4fH4vIuN3eaVPPsGyvLyAV+ArYD8gAPgKGNVjm58AD7vvZwH8SXe42qPNkINt9f1Ey1NldrjPwOvAuMCHR5W6D3/kAYDXQ3Z3uk+hyt0GdHwQuct8PAzYluty7WeejgHHAp018fiLwIiDAJOC93d1nRzwjOARYr6obVNULLAKmN1hmOvCI+/5J4FgRkTYsY2uLWmdVfVVVK93Jd3GeGNeexfI7A9wMzAeq27JwcRJLnX8K3KuqJQCquq2Ny9jaYqmzAl3c912Bb9uwfK1OVV8HiiMsMh14VB3vAt1EpN/u7LMjBoIBwJaQ6UJ3Xthl1HmAThnQs01KFx+x1DnUeThHFO1Z1Dq7p8x7q+oLbVmwOIrldz4QOFBE3hKRd0VkSpuVLj5iqfM84MciUojz/JNftk3REqa5/9+jsucRJBkR+TEwATg60WWJJxFJAf4EnJPgorS1NJzuoXycs77XRWSkqpYmtFTxdQawQFX/KCKHAv8UkRGq6k90wdqLjnhG8A2wd8j0QHde2GVEJA3ndLKoTUoXH7HUGRH5AXAtME1Va9qobPESrc6dgRFAgYhswulLXdLOB4xj+Z0LgSWqWquqG4EvcAJDexVLnc8DHgdQ1XeALJzkbB1VTP/fm6MjBoKVwAEiMkhEMnAGg5c0WGYJMMd9PwtYru4oTDsVtc4iMhb4K04QaO/9xhClzqpapqq9VDVPVfNwxkWmqeqqxBS3VcTyb3sxztkAItILp6toQ1sWspXFUufNwLEAIjIUJxDE/mzT9mcJcLZ79dAkoExVv9udDXa4riFV9YnIxcAynCsOHlbVNSJyE7BKVZcAf8c5fVyPMygzO3El3n0x1vl2IBd4wh0X36yq0xJW6N0UY507lBjrvAw4XkTWAnXAlarabs92Y6zzFcDfROQynIHjc9rzgZ2ILMQJ5r3ccY8bgHQAVX0AZxzkRGA9UAmcu9v7bMfflzHGmFbQEbuGjDHGNIMFAmOMSXIWCIwxJslZIDDGmCRngcAYY5KcBQKz20Rkk5vds+FrUxvt0yMib4jImN3YngayPYrIYSIyL3R7IlLgLtMqNyq52w+Uv8bNFHpKDOtlu+ue04x9XSIilSLSI9JvJSJ5DeYVi8giEenpbid03VIReV5E9nI/G+bOn9HiL8UkTIe7j8AkxC+BHOBk4CzgAeA1wBO6kIikubmdWksNTgqJkcA1wFPA/i3c1hlAIA3DYTjXbm8CPnTn3QT0AXa2cPtNuRMnu+Z84DER6a6qdRGWz3bL9hqwINrG3WSKVwBPq2qxiMTyW63Gue9kFnC6+9l57meB73wKzk2Z1wG/UNW1IvIWcDnOTW2mPUl0ylV7dZwX8H/suqEHnJtiFOcGmBXAKpxGRIH/c5d53p3Oc6fnAp/jND5vA+Oa2NcmoCJkequ7nV44t98vBkpwMlHeCWTinAE/gHMTYRWwFjjGXV+BT0PKHPrKAwpCtr8CJyVJmrvu40AtTqAYCryCEzC+pf8GswAABBNJREFUBi5rovzz3O3Ncqdfcaf7utNPuOWvdsv5w5B6h5ZtHk565jtw0gyUuuv2dpc/0l3utEi/lTsvz533vDs9NPC9NPzOcZKcKfBiyPq/BvzAgET/W7RX817WNWTawg+AZ4A/R1pIRPJx7vreBPwOJyPscyKSFWGdXiJyrLtsBU4j/xhwCvAHnDtSf4WTY2k0cCHwX+BnwLM0Pite664PTtA4g8bpCh4DegDHiEgnnLs8X3H3/SxOTvw/AO8Bf4rS5dNZRA7CaXQ3hexrJXAV8Bt3+lH3e7jGnV7nlu1Jd5krgOdwgt5Ut+wAR4RsL1bpItIbCHTzbA790O0eC2Q1XRHy0UqcHPmHN2NfZg9gXUOmLTyvqrcCROnbPsn9e7z7ChgGfBBm+Rx2NZyVwC9wuk6OBN5W1VtFJBM4G6dxvNddbgywDeeM49XQDarqNhH5EKfb5D1VXeSWO3SxRcAfcbpOOrvl+BdwELsSvN0csvxxOI10OA+7f6uBfFX1i0gqMBwn9UlGyLJ5wMvu+20hZfuHO+/CBvsE2Mf925wc/cfjfD/gnGVcE/JZ6Hf+HnBLyGeBfeQ1Y19mD2CBwLSF0EYo0P8d+LfXLczyVwAfu+9TgI1NbLca58jfA6xT1VIRyXU/a5Q7RVW3ishwnCPdiThH9sOA3zZctOmqBLfzX3c73XDORBYDg9xFluF01QR8H2FzN+N0K90E/MVNo3wcTvD6r7udX+IEyawIZfPh9PsHvt+GZ/vNefDSezhnUMXAWq2fqbYaOA24BOdM7zzg/hbsw+xBrGvItLVN7t+TROSnOOmhAwIPkDkD50h2InCXuk/bCqNOVf+rqu+om29fVStwHk05SZzn296H8+98qYgciNPdUoHT2AH0D7PdwP6mishpTez7MaA3zlnBM+o8/e1z4Euc7pixOGcIv8B57GBTPlbVm3G6ziayqzsGnKPvA6jf1bITpx9+sIicJc6zp5/HCaxzcL63Kew6Owh064SrZ1N2qOr/VHW1Nk5XXqeqzwE/wRlnuSGk6y6wj6+bsS+zB7BAYNram8BCnO6ZmTgDyACoagFOJsVcnG6cC3C6b5rrxziN49U4/fd3Ab/HOZodhzNW8Xt32/PDrL8EeB84FWjq4e/P4HQzCU63EOpcETUdeAvnLONmnK6jT2Io8zyco/1rccYbFuF8R7NxzjBw91GLc0VPN3e/RwK3uvOOBO7B6QZ7zV3lTfdvqz6HQVW/xzkT6IvzeMzAPjRkn6adsOyjxnRg7uWjG4E3VfXHcd7XmzhnDB366XcdkZ0RGNOBqXOk9ydgpoj0iNd+RGQYThdWxCvDzJ7JzgiMMSbJ2RmBMcYkOQsExhiT5CwQGGNMkrNAYIwxSc4CgTHGJLn/B0MkVlygFxEXAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TLiuU9NksHAs"
      },
      "source": [
        "###**10. Save Model using Joblib**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b_P3Ed8_emp0"
      },
      "source": [
        "from sklearn.externals import joblib \n",
        "\n",
        "# Save the model as a pickle in a file \n",
        "joblib.dump(xg, 'xgboost.pkl') \n",
        "joblib.dump(cat_clf, 'catboost.pkl') \n",
        "joblib.dump(lgbm, 'lgbm.pkl');"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uQieGdIur-o4",
        "outputId": "82155b2c-6f4a-41a4-b609-0abe46fca2c7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from prettytable import PrettyTable\n",
        "\n",
        "#If you get a ModuleNotFoundError error , install prettytable using: !pip install prettytable\n",
        "\n",
        "x = PrettyTable()\n",
        "x.field_names = [\"Model Used\", \"ROC-AUC\",\"Test Accuracy(%)\", \"Precision\"]\n",
        "\n",
        "x.add_row([\"CatBoost Classifier\",  93.37, 93.9, 0.834])\n",
        "x.add_row([\"XGBoost Classifier\",  89.98, 92.7, 0.844])\n",
        "x.add_row([\"LGBM Classifier\",  93.32, 93.9, 0.836])\n",
        "\n",
        "print(x)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "+---------------------+---------+------------------+-----------+\n",
            "|      Model Used     | ROC-AUC | Test Accuracy(%) | Precision |\n",
            "+---------------------+---------+------------------+-----------+\n",
            "| CatBoost Classifier |  93.37  |       93.9       |   0.834   |\n",
            "|  XGBoost Classifier |  89.98  |       92.7       |   0.844   |\n",
            "|   LGBM Classifier   |  93.32  |       93.9       |   0.836   |\n",
            "+---------------------+---------+------------------+-----------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NnOLPXE6tGSB"
      },
      "source": [
        "###**11. Conclusion**\n",
        "\n",
        "- CatBoost is the clear winner here and all the models have done a good job.\n",
        "- Performance can be increaded further by considering text features too which I had to ignore since it would make the dimensions huge and the number of data points were also large, the notebook was crashing.\n",
        "- So, even with these features and few of engineered features, the model is performing very good."
      ]
    }
  ]
}