Final_Pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Introduction\n",
    "\n",
    "In order to streamline and automate the data science lifecylcle, we have implemented a customized data pipeline that automates the below stages of the lifecycle:\n",
    "\n",
    "1. Preprocessing/Wrangling\n",
    "2. Feature Engineering\n",
    "3. Handling class imbalance\n",
    "4. Modelling/Making Predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Loading the required libraries\n",
    "\n",
    "Our pipeline uses the below libraries to preprocess the data and make predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Installing the required libraries\n",
    "\n",
    "To ensure that there no erros arise due to package version mismatch, it is highly recommended to run the below cell to install the libraries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "__NOTE:__ In order to run the below commands in the command line interface/terminal, please ensure the below:\n",
    "\n",
    "- You have python version 3.10 installed(https://www.python.org/downloads/release/python-3100/)\n",
    "- You have a standard pip accompanying it(https://pip.pypa.io/en/stable/installation/)\n",
    "- It is added to the classpath of your Operating System(https://geek-university.com/add-python-to-the-windows-path/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!pip install pandas==1.4.4\n",
    "!pip install category_encoders\n",
    "!pip install xgboost==1.6.2\n",
    "!pip install imblearn==0.9.1\n",
    "!pip install joblib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Importing the required libraries after installation\n",
    "\n",
    "Once the libraries are available to use, we need to load them in the current working environment. Please use the below cell to import all the required methods/classes from the libraries that will be used by the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.compose import ColumnTransformer\n",
    "# from sklearn.pipeline import Pipeline\n",
    "from imblearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from category_encoders.target_encoder import TargetEncoder\n",
    "from xgboost import XGBClassifier\n",
    "from sklearn.base import BaseEstimator, TransformerMixin\n",
    "from imblearn.over_sampling import SMOTE\n",
    "from joblib import dump, load\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Reading the training dataset\n",
    "\n",
    "We will be first reading the training dataset that will be used to train the pipeline.\n",
    "\n",
    "Please note that this training dataset csv file needs to included in the same path as the ipynb code for the below code to work as intended.\n",
    "Also, please note that the below cell works only if the dataset is a CSV file.\n",
    "\n",
    "Please provide the path to the training dataset here inside pd.read_csv(\"\"). Use the below code as reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 569,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    data = pd.read_csv(\"A2_change_job_labeled.csv\")\n",
    "    test_data = pd.read_csv(\"A2_change_job_submission.csv\")\n",
    "    kaggle_data = pd.read_csv(\"A2_change_job_kaggle.csv\")\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred while reading the dataset, {e}. Kindly google the error message to solve the error.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Verifying the data loaded\n",
    "\n",
    "Please run the below code that shows the first 5 rows of the dataset as a means of verification that the data is read properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 570,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cid</th>\n",
       "      <th>city.100</th>\n",
       "      <th>city.10</th>\n",
       "      <th>city.105</th>\n",
       "      <th>city.103</th>\n",
       "      <th>city.104</th>\n",
       "      <th>city.102</th>\n",
       "      <th>city.101</th>\n",
       "      <th>city.2119</th>\n",
       "      <th>city.114</th>\n",
       "      <th>...</th>\n",
       "      <th>education_level</th>\n",
       "      <th>academic_discipline</th>\n",
       "      <th>experience</th>\n",
       "      <th>company_size</th>\n",
       "      <th>company_type</th>\n",
       "      <th>last_new_job</th>\n",
       "      <th>city_development_index</th>\n",
       "      <th>relevant_experience</th>\n",
       "      <th>training_hours</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7124</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>15</td>\n",
       "      <td>100-500</td>\n",
       "      <td>NGO</td>\n",
       "      <td>1</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>21</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>4</td>\n",
       "      <td>50-99</td>\n",
       "      <td>Funded startup</td>\n",
       "      <td>1</td>\n",
       "      <td>0.219561</td>\n",
       "      <td>No relevant experience</td>\n",
       "      <td>92</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13137</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>High School</td>\n",
       "      <td>STEM</td>\n",
       "      <td>8</td>\n",
       "      <td>50-99</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>&gt;4</td>\n",
       "      <td>0.499002</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>21</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2769</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>2</td>\n",
       "      <td>50-99</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>never</td>\n",
       "      <td>0.351297</td>\n",
       "      <td>No relevant experience</td>\n",
       "      <td>114</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8374</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>PhD</td>\n",
       "      <td>STEM</td>\n",
       "      <td>7</td>\n",
       "      <td>5000-9999</td>\n",
       "      <td>Public sector</td>\n",
       "      <td>1</td>\n",
       "      <td>0.626747</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>24</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 73 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     cid  city.100  city.10  city.105  city.103  city.104  city.102  city.101  \\\n",
       "0   7124         0        0         0         0         0         0         0   \n",
       "1     51         0        0         1         0         0         0         0   \n",
       "2  13137         0        0         0         0         0         0         0   \n",
       "3   2769         0        0         0         0         0         0         0   \n",
       "4   8374         0        0         0         0         0         0         0   \n",
       "\n",
       "   city.2119  city.114  ...  education_level  academic_discipline  experience  \\\n",
       "0          0         0  ...    Undergraduate                 STEM          15   \n",
       "1          0         0  ...    Undergraduate                 STEM           4   \n",
       "2          0         0  ...      High School                 STEM           8   \n",
       "3          0         0  ...    Undergraduate                 STEM           2   \n",
       "4          0         0  ...              PhD                 STEM           7   \n",
       "\n",
       "   company_size    company_type  last_new_job  city_development_index  \\\n",
       "0       100-500             NGO             1                0.942116   \n",
       "1         50-99  Funded startup             1                0.219561   \n",
       "2         50-99         Pvt Ltd            >4                0.499002   \n",
       "3         50-99         Pvt Ltd         never                0.351297   \n",
       "4     5000-9999   Public sector             1                0.626747   \n",
       "\n",
       "       relevant_experience  training_hours  target  \n",
       "0  Has relevant experience              21      no  \n",
       "1   No relevant experience              92     yes  \n",
       "2  Has relevant experience              21      no  \n",
       "3   No relevant experience             114      no  \n",
       "4  Has relevant experience              24     yes  \n",
       "\n",
       "[5 rows x 73 columns]"
      ]
     },
     "execution_count": 570,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 571,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cid</th>\n",
       "      <th>city.100</th>\n",
       "      <th>city.10</th>\n",
       "      <th>city.105</th>\n",
       "      <th>city.103</th>\n",
       "      <th>city.104</th>\n",
       "      <th>city.102</th>\n",
       "      <th>city.101</th>\n",
       "      <th>city.2119</th>\n",
       "      <th>city.114</th>\n",
       "      <th>...</th>\n",
       "      <th>education_level</th>\n",
       "      <th>academic_discipline</th>\n",
       "      <th>experience</th>\n",
       "      <th>company_size</th>\n",
       "      <th>company_type</th>\n",
       "      <th>last_new_job</th>\n",
       "      <th>city_development_index</th>\n",
       "      <th>relevant_experience</th>\n",
       "      <th>training_hours</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>12321</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Masters</td>\n",
       "      <td>STEM</td>\n",
       "      <td>&gt;20</td>\n",
       "      <td>5000-9999</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>&gt;4</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>26</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>941</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Masters</td>\n",
       "      <td>STEM</td>\n",
       "      <td>&gt;20</td>\n",
       "      <td>100-500</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>2</td>\n",
       "      <td>0.834331</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>78</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>17715</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>&gt;20</td>\n",
       "      <td>Oct-49</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>2</td>\n",
       "      <td>0.922156</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>32</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6540</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>Business degree</td>\n",
       "      <td>11</td>\n",
       "      <td>5000-9999</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>2</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>7</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6760</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>5</td>\n",
       "      <td>100-500</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>never</td>\n",
       "      <td>0.351297</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>62</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 73 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     cid  city.100  city.10  city.105  city.103  city.104  city.102  city.101  \\\n",
       "0  12321         0        0         0         0         1         0         0   \n",
       "1    941         0        0         0         0         0         0         0   \n",
       "2  17715         0        0         0         0         0         0         0   \n",
       "3   6540         0        0         0         0         1         0         0   \n",
       "4   6760         0        0         0         0         0         0         0   \n",
       "\n",
       "   city.2119  city.114  ...  education_level  academic_discipline  experience  \\\n",
       "0          0         0  ...          Masters                 STEM         >20   \n",
       "1          0         0  ...          Masters                 STEM         >20   \n",
       "2          0         0  ...    Undergraduate                 STEM         >20   \n",
       "3          0         0  ...    Undergraduate      Business degree          11   \n",
       "4          0         0  ...    Undergraduate                 STEM           5   \n",
       "\n",
       "   company_size  company_type  last_new_job  city_development_index  \\\n",
       "0     5000-9999       Pvt Ltd            >4                0.942116   \n",
       "1       100-500       Pvt Ltd             2                0.834331   \n",
       "2        Oct-49       Pvt Ltd             2                0.922156   \n",
       "3     5000-9999       Pvt Ltd             2                0.942116   \n",
       "4       100-500       Pvt Ltd         never                0.351297   \n",
       "\n",
       "       relevant_experience  training_hours  target  \n",
       "0  Has relevant experience              26      no  \n",
       "1  Has relevant experience              78      no  \n",
       "2  Has relevant experience              32      no  \n",
       "3  Has relevant experience               7      no  \n",
       "4  Has relevant experience              62      no  \n",
       "\n",
       "[5 rows x 73 columns]"
      ]
     },
     "execution_count": 571,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 572,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cid</th>\n",
       "      <th>city.100</th>\n",
       "      <th>city.10</th>\n",
       "      <th>city.105</th>\n",
       "      <th>city.103</th>\n",
       "      <th>city.104</th>\n",
       "      <th>city.102</th>\n",
       "      <th>city.101</th>\n",
       "      <th>city.2119</th>\n",
       "      <th>city.114</th>\n",
       "      <th>...</th>\n",
       "      <th>education_level</th>\n",
       "      <th>academic_discipline</th>\n",
       "      <th>experience</th>\n",
       "      <th>company_size</th>\n",
       "      <th>company_type</th>\n",
       "      <th>last_new_job</th>\n",
       "      <th>city_development_index</th>\n",
       "      <th>relevant_experience</th>\n",
       "      <th>training_hours</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>&gt;20</td>\n",
       "      <td>500-999</td>\n",
       "      <td>Other</td>\n",
       "      <td>1</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>36</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>5</td>\n",
       "      <td>50-99</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>never</td>\n",
       "      <td>0.351297</td>\n",
       "      <td>No relevant experience</td>\n",
       "      <td>83</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>High School</td>\n",
       "      <td>No major</td>\n",
       "      <td>5</td>\n",
       "      <td>50-99</td>\n",
       "      <td>Funded startup</td>\n",
       "      <td>1</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>24</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Undergraduate</td>\n",
       "      <td>STEM</td>\n",
       "      <td>5</td>\n",
       "      <td>5000-9999</td>\n",
       "      <td>Pvt Ltd</td>\n",
       "      <td>1</td>\n",
       "      <td>0.942116</td>\n",
       "      <td>Has relevant experience</td>\n",
       "      <td>108</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>High School</td>\n",
       "      <td>STEM</td>\n",
       "      <td>5</td>\n",
       "      <td>&lt;10</td>\n",
       "      <td>Early stage startup</td>\n",
       "      <td>never</td>\n",
       "      <td>0.351297</td>\n",
       "      <td>No relevant experience</td>\n",
       "      <td>26</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 73 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   cid  city.100  city.10  city.105  city.103  city.104  city.102  city.101  \\\n",
       "0    1         0        0         0         0         1         0         0   \n",
       "1    3         0        0         0         0         0         0         0   \n",
       "2    7         0        0         0         0         0         0         0   \n",
       "3   12         0        0         0         0         1         0         0   \n",
       "4   15         0        0         0         0         0         0         0   \n",
       "\n",
       "   city.2119  city.114  ...  education_level  academic_discipline  experience  \\\n",
       "0          0         0  ...    Undergraduate                 STEM         >20   \n",
       "1          0         0  ...    Undergraduate                 STEM           5   \n",
       "2          0         0  ...      High School             No major           5   \n",
       "3          0         0  ...    Undergraduate                 STEM           5   \n",
       "4          0         0  ...      High School                 STEM           5   \n",
       "\n",
       "   company_size         company_type  last_new_job  city_development_index  \\\n",
       "0       500-999                Other             1                0.942116   \n",
       "1         50-99              Pvt Ltd         never                0.351297   \n",
       "2         50-99       Funded startup             1                0.942116   \n",
       "3     5000-9999              Pvt Ltd             1                0.942116   \n",
       "4           <10  Early stage startup         never                0.351297   \n",
       "\n",
       "       relevant_experience  training_hours  target  \n",
       "0  Has relevant experience              36      no  \n",
       "1   No relevant experience              83      no  \n",
       "2  Has relevant experience              24      no  \n",
       "3  Has relevant experience             108      no  \n",
       "4   No relevant experience              26      no  \n",
       "\n",
       "[5 rows x 73 columns]"
      ]
     },
     "execution_count": 572,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kaggle_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Extracting the features and target from the data\n",
    "\n",
    "After the data is read successfully, we need to extract the features __X__ which will be the input columns to the pipeline and the target __y__, which is the response/target variable we are trying to predict.\n",
    "\n",
    "Please run the below cell to perform the extraction.\n",
    "\n",
    "__NOTE:__ Please ensure that the target variable is named as \"target\" in your dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 655,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Please ensure that your target variable is named as \"target\"\n",
    "try:\n",
    "    final_columns = [] # Used for keeping track of the final columns in the output of the pipeline\n",
    "\n",
    "\n",
    "    X = data.drop([\"target\"],axis = 1) # Original feature data\n",
    "    X_mirror = data.drop([\"target\"],axis = 1) # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y = data.loc[:,\"target\"] # Target variable\n",
    "\n",
    "    # Doing the same for the test data\n",
    "    X_test = test_data.drop([\"target\"],axis = 1) # Original feature data\n",
    "    X_mirror_test = test_data.drop([\"target\"],axis = 1) # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y_test = test_data.loc[:,\"target\"] # Target variable\n",
    "\n",
    "    X_kaggle = kaggle_data.drop([\"target\"],axis = 1) # Original feature data\n",
    "    X_mirror_kaggle = kaggle_data.drop([\"target\"],axis = 1) # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y_kaggle = kaggle_data.loc[:,\"target\"] # Target variable\n",
    "\n",
    "\n",
    "except Exception as e:\n",
    "\n",
    "    print(f\"There was an error extracting features/target from the data. Error message: {e}. It could be possible that the target variable doesn't exist in the dataset. Attempting to extract based on the assumption that the last variable in the dataset is the target variable.\")\n",
    "\n",
    "    X = data.iloc[:,0:-1] # Original feature data\n",
    "    X_mirror = data.iloc[:,0:-1] # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y = data.iloc[:,-1] # Target variable\n",
    "\n",
    "    # Doing the same for the test data\n",
    "    X_test = test_data.iloc[:,0:-1] # Original feature data\n",
    "    X_mirror_test = test_data.iloc[:,0:-1] # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y_test = test_data.iloc[:,-1] # Target variable\n",
    "\n",
    "    X_kaggle = kaggle_data.iloc[:,0:-1] # Original feature data\n",
    "    X_mirror_kaggle = kaggle_data.iloc[:,0:-1] # Used as a version which will keep track of the final data coming from the pipeline\n",
    "    y_kaggle = kaggle_data.iloc[:,-1] # Target variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Pipeline\n",
    "\n",
    "We have defined a number of custom transformer classes extending the base classes (BaseEstimator, TransformerMixin) below. These classes extend the functionality of the base classes and implement additional logic on top of it catering to our specific needs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Please run all the cells below where the custom transformer classes are defined."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Custom Label Encoder\n",
    "\n",
    "We defined a custom label encoder which extends on the LabelEncoder functionality of scikit-learn. This allowed us to perform custom encoding of certain variables based on our domain knowledge of the categories in the variables. For instance, enrolled_university with categories is encoded as follows: \"No enrollment\": 0, \"Part time\":1, \"Full time\":2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 666,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# (BaseEstimator, TransformerMixin). This makes it compatible with scikit-learn’s Pipelines\n",
    "class CustomLabelEncoder(BaseEstimator, TransformerMixin):\n",
    "    # initializer\n",
    "    def __init__(self, columns = None):\n",
    "        # save the features list internally in the class\n",
    "        self.columns = columns\n",
    "\n",
    "    def fit(self, X, y = None):\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, y = None):\n",
    "        global X_mirror\n",
    "        global final_columns\n",
    "\n",
    "    # return the dataframe with the specified features\n",
    "        X[\"enrolled_university_encoded\"] = X.enrolled_university.replace({\"No enrollment\": 0, \"Part time\":1, \"Full time\":2})\n",
    "\n",
    "        X[\"relevant_experience_encoded\"] = X.relevant_experience.replace({\"Has relevant experience\": 1, \"No relevant experience\":0})\n",
    "\n",
    "        X[\"last_new_job_encoded\"] = X.last_new_job.replace({\"1\": 1, \"2\": 2, \"3\": 3, \"4\": 4, \">4\": 5, \"never\": 0})\n",
    "\n",
    "        X[\"experience_regrouped_encoded\"] = X_mirror.experience_regrouped.replace({\"<1\": 1,\"1-3\":2,\"3-7\":3,\"7-14\":4,\">20\":5 })\n",
    "\n",
    "        X_mirror[\"enrolled_university_encoded\"] = X[\"enrolled_university_encoded\"]\n",
    "        X_mirror[\"relevant_experience_encoded\"] = X[\"relevant_experience_encoded\"]\n",
    "        X_mirror[\"last_new_job_encoded\"] = X[\"last_new_job_encoded\"]\n",
    "        X_mirror[\"experience_regrouped_encoded\"] = X[\"experience_regrouped_encoded\"]\n",
    "\n",
    "        # Returning only the necessary columns from here\n",
    "        return_cols = []\n",
    "\n",
    "        for col in fs_cols:\n",
    "            if col in X.columns:\n",
    "                return_cols.append(col)\n",
    "\n",
    "\n",
    "        final_columns += return_cols\n",
    "\n",
    "        return X[return_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Custom One Hot Encoder\n",
    "\n",
    "We defined a custom one hot encoder which extends on the OneHotEncoder functionality of scikit-learn. This allowed us to perform one hot encoding of certain variables and then also set the required variable names for the one-hot encoded columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 667,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# (BaseEstimator, TransformerMixin). This makes it compatible with scikit-learn’s Pipelines\n",
    "class CustomOneHotEncoder(BaseEstimator, TransformerMixin):\n",
    "    # initializer\n",
    "    def __init__(self, columns = None):\n",
    "        # save the features list internally in the class\n",
    "        self.columns = columns\n",
    "\n",
    "    def fit(self, X, y = None):\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, y = None):\n",
    "        # return the dataframe with the specified features\n",
    "        X_copy = X.copy()\n",
    "        global X_mirror\n",
    "        global final_columns\n",
    "\n",
    "        # Creating a OneHotEncoder instance.\n",
    "\n",
    "        ohe = OneHotEncoder(handle_unknown = 'ignore', sparse = False)\n",
    "\n",
    "        # Fitting the encoder to the input list of features.\n",
    "        ohe.fit(X[list(self.columns)])\n",
    "        data_ohe = pd.DataFrame(ohe.fit_transform(X[list(self.columns)]), columns=ohe.get_feature_names_out())\n",
    "\n",
    "        # Resetting the index lost during transformation\n",
    "        data_ohe.index = X_copy.index\n",
    "\n",
    "        # Adding the newly generated one-hot encoded columns to the original data\n",
    "        X = pd.concat([X,data_ohe],axis=1)\n",
    "\n",
    "        # Replicating the same for the mirror data\n",
    "        X_mirror = pd.concat([X_mirror,data_ohe],axis=1)\n",
    "\n",
    "        # Returning only the necessary columns from here\n",
    "        return_cols = []\n",
    "\n",
    "        for col in fs_cols:\n",
    "            if col in X.columns:\n",
    "                return_cols.append(col)\n",
    "\n",
    "        final_columns += return_cols\n",
    "        return X[return_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Feature Engineering\n",
    "\n",
    "We defined a feature engineering transformer that performs __binning__ where the original categories of the categorical variable were regrouped based on the conditions(defined in the _transform_experience_ function) that seemed appropriate given the circumstances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 668,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class FeatureEngineering(BaseEstimator, TransformerMixin):\n",
    "    # initializer\n",
    "    def __init__(self, columns = None):\n",
    "        # save the features list internally in the class\n",
    "        self.columns = columns\n",
    "\n",
    "    def fit(self, X, y = None):\n",
    "        return self\n",
    "\n",
    "    def transform_experience(self, row):\n",
    "        \"\"\"\n",
    "        The feature experience is re-categorized in an exponential manner as it makes sense for\n",
    "        experience variable.\n",
    "        :param row:\n",
    "        :return:\n",
    "        \"\"\"\n",
    "        try:\n",
    "            if int(row) >= 1 and int(row) <= 3:\n",
    "                return '1-3'\n",
    "            elif int(row) > 3 and int(row) <= 7:\n",
    "                return '3-7'\n",
    "            elif int(row) > 7 and int(row) <= 14:\n",
    "                return '7-14'\n",
    "            elif int(row) > 14 and int(row) <= 20:\n",
    "                return '>20'\n",
    "\n",
    "        except ValueError:\n",
    "            if row == '<1':\n",
    "                return '<1'\n",
    "            elif row == '>20':\n",
    "                return '>20'\n",
    "\n",
    "    def transform(self, X, y = None):\n",
    "        # return the dataframe with the specified features\n",
    "        global final_columns\n",
    "        global X_mirror\n",
    "        final_columns = []\n",
    "\n",
    "        X['experience_regrouped'] = X.experience.apply(self.transform_experience)\n",
    "        X_mirror['experience_regrouped'] = X['experience_regrouped']\n",
    "\n",
    "        # Returning only the necessary columns from here\n",
    "        return_cols = []\n",
    "\n",
    "        for col in fs_cols:\n",
    "            if col in X.columns:\n",
    "                return_cols.append(col)\n",
    "\n",
    "        final_columns += return_cols\n",
    "        return X[return_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Custom Target Encoder\n",
    "\n",
    "We defined a custom target encoder which extends on the TargetEncoder functionality of category_encoders. This allowed us to perform target encoding of certain variables and then also set the required variable names for the target encoded columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 669,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# (BaseEstimator, TransformerMixin). This makes it compatible with scikit-learn’s Pipelines\n",
    "class CustomTargetEncoder(BaseEstimator, TransformerMixin):\n",
    "    # initializer\n",
    "    def __init__(self, columns, y):\n",
    "        # save the features list internally in the class\n",
    "        self.columns = columns\n",
    "        self.y = y\n",
    "\n",
    "    def fit(self, X, y = None):\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, y = None):\n",
    "        global final_columns\n",
    "        te=TargetEncoder()\n",
    "        for col in (self.columns + [\"experience_regrouped\"]):\n",
    "            X_mirror[col + \"_te\"]=te.fit_transform(X_mirror[col],self.y)\n",
    "            X[col + \"_te\"] = X_mirror[col + \"_te\"]\n",
    "\n",
    "        # Returning only the necessary columns from here\n",
    "        return_cols = []\n",
    "\n",
    "        for col in fs_cols:\n",
    "            if col in X.columns:\n",
    "                return_cols.append(col)\n",
    "\n",
    "        final_columns += return_cols\n",
    "        return X[return_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Custom Feature Selector\n",
    "\n",
    "We defined a feature selector transformer which selects features based on a pre-determined best feature list(generated based on 6 different feature selection techniques). There features will be later used for final modelling.\n",
    "\n",
    "Please refer to the report for in-depth explanation of how the feature selection process was employed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 670,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# (BaseEstimator, TransformerMixin). This makes it compatible with scikit-learn’s Pipelines\n",
    "class FeatureSelector(BaseEstimator, TransformerMixin):\n",
    "    # initializer\n",
    "    def __init__(self, columns = None):\n",
    "        # save the features list internally in the class\n",
    "        self.columns = columns\n",
    "\n",
    "    def fit(self, X, y = None):\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, y = None):\n",
    "        global X_mirror\n",
    "        X_mirror = pd.DataFrame()\n",
    "\n",
    "        # Returning only the specific features given to it\n",
    "        return_cols = []\n",
    "        global final_columns\n",
    "        for col in fs_cols:\n",
    "            if col in X.columns:\n",
    "                return_cols.append(col)\n",
    "\n",
    "\n",
    "        final_columns += return_cols\n",
    "        return X[return_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Pipeline Stage I: Column Transformer\n",
    "\n",
    "We bundled all the transformations that were applied on specific sets of features based on the custom classes(defined earlier) into a single ColumnTransformer unit for it be shipped as the first stage of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 690,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Defining the columns which will be passsed to specific stages of the Column Transformer\n",
    "le_cols = [\"enrolled_university\",\"relevant_experience\",\"last_new_job\"]\n",
    "ohe_cols = [\"gender\",\"academic_discipline\",\"company_type\",\"company_size\",\"education_level\"]\n",
    "fe_cols = [\"experience\"]\n",
    "te_cols = [col for col in X.columns if X[col].dtype == 'object']\n",
    "\n",
    "fs_cols = [\n",
    "           'city_development_index',\n",
    "           'enrolled_university_encoded',\n",
    "           'company_type_Pvt Ltd',\n",
    "           'relevant_experience_encoded',\n",
    "           'education_level_Undergraduate',\n",
    "            'company_size_10000+',\n",
    "           ]\n",
    "\n",
    "\n",
    "# Bundle preprocessing for all the transformations\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('feature_engineering', FeatureEngineering(),fe_cols),\n",
    "        ('custom_label_encoder', CustomLabelEncoder(columns=le_cols), le_cols),\n",
    "        ('onehot', CustomOneHotEncoder(columns=ohe_cols), ohe_cols),\n",
    "        ('feature_selector', FeatureSelector(columns=fs_cols), list(X.columns)),\n",
    "    ], remainder=\"drop\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Pipeline Stage II: Handling class imbalance using SMOTE\n",
    "\n",
    "The minority class in the target variable is oversampled using SMOTE algorithm. This is then shipped as the second stage of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 673,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "smote = SMOTE(random_state=1211)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Pipeline Stage III: Modelling using Extreme Gradient Boosting Tree algorithm\n",
    "\n",
    "The final stage of the pipeline uses XGBoost Classifier model with tuned hyperparameters based on HyperOpt's Bayesian Optimization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 691,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model = XGBClassifier(**{'max_depth': 16, 'learning_rate':0.13, 'n_estimators':350, 'objective':'binary:logistic',\n",
    "         'booster':'gbtree',\n",
    "         'n_jobs':1,\n",
    "         'nthread':None,\n",
    "         'gamma':7.2,\n",
    "         'min_child_weight':1,\n",
    "         'max_delta_step':0,\n",
    "         'subsample':1,\n",
    "         'colsample_bytree':0.7,\n",
    "         'colsample_bylevel':1,\n",
    "         'reg_alpha':0,\n",
    "         'reg_lambda':0.5,\n",
    "         'scale_pos_weight':1,\n",
    "         'base_score':0.5,\n",
    "         'random_state':0,\n",
    "         'seed':None,\n",
    "         'missing':1})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Bundling all the stages as a pipeline\n",
    "\n",
    "Finally, we are bundling all the stages as steps in the pipeline defined below. The stages/steps in the pipeline will be performed in the following sequence:\n",
    "\n",
    "Stages:\n",
    "1. preprocesser\n",
    "2. class_balancing_SMOTE\n",
    "3. model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 692,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Bundle preprocessing and modeling code in a pipeline\n",
    "pipeline = Pipeline(steps=[('preprocessor', preprocessor),\n",
    "                           ('class_balancing_SMOTE', smote),\n",
    "                           ('model', model)\n",
    "                           ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Fitting the pipeline on the given data\n",
    "\n",
    "Having defined the pipeline, we will now fit the pipeline on the given data. But first, we need to encode the target variable from \"yes\",\"no\" to 1 and 0 respectively. Please note that this couldn't be performed inside the pipeline due to a limitation of scikit-learn's library and hence had to be performed manually."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Performing 5fold Stratified Cross Validation and computing mean F1 Score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 693,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRAIN: [ 8118  2797  9401 ... 11355  1827  8332] TEST: [10648  2600  4735 ...   822  1627  9292]\n",
      "TRAIN: [ 742  711 1898 ... 2005 2262 5392] TEST: [ 9022  1821  4843 ... 10862  8175  2241]\n",
      "TRAIN: [ 5352  6183  7785 ...  3517  7811 10844] TEST: [ 5953 10250  9056 ...  5052  3837   397]\n",
      "TRAIN: [ 2561  1997 10476 ...  3330  1496 11030] TEST: [ 6188 10595  6015 ...  6682 11358 11032]\n",
      "TRAIN: [6739  845 1630 ... 2547 4452 1935] TEST: [ 616  512 2657 ... 3525 5929 3245]\n",
      "0.5475946136062944\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import StratifiedShuffleSplit\n",
    "import numpy as np\n",
    "\n",
    "nfold_f1 = []\n",
    "sss = StratifiedShuffleSplit(n_splits=5, test_size=0.2, random_state=0)\n",
    "\n",
    "for train_index, test_index in sss.split(X, y):\n",
    "    print(\"TRAIN:\", train_index, \"TEST:\", test_index)\n",
    "    X_train, X_valid = X.iloc[train_index,:], X.iloc[test_index,:]\n",
    "    y_train, y_valid = y.iloc[train_index], y.iloc[test_index]\n",
    "\n",
    "    y_train.replace({\"yes\": 1, \"no\":0}, inplace = True)\n",
    "    y_valid.replace({\"yes\": 1, \"no\":0}, inplace = True)\n",
    "    pipeline.fit(X_train,y_train)\n",
    "    nfold_f1.append(f1_score(pipeline.predict(X_valid),y_valid))\n",
    "\n",
    "print(np.mean(nfold_f1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Fitting pipeline on complete dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 683,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-51 {color: black;background-color: white;}#sk-container-id-51 pre{padding: 0;}#sk-container-id-51 div.sk-toggleable {background-color: white;}#sk-container-id-51 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-51 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-51 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-51 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-51 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-51 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-51 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-51 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-51 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-51 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-51 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-51 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-51 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-51 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-51 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-51 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-51 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-51 div.sk-item {position: relative;z-index: 1;}#sk-container-id-51 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-51 div.sk-item::before, #sk-container-id-51 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-51 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-51 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-51 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-51 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-51 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-51 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-51 div.sk-label-container {text-align: center;}#sk-container-id-51 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-51 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-51\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;feature_engineering&#x27;,\n",
       "                                                  FeatureEngineering(),\n",
       "                                                  [&#x27;experience&#x27;]),\n",
       "                                                 (&#x27;custom_label_encoder&#x27;,\n",
       "                                                  CustomLabelEncoder(columns=[&#x27;enrolled_university&#x27;,\n",
       "                                                                              &#x27;relevant_experience&#x27;,\n",
       "                                                                              &#x27;last_new_job&#x27;]),\n",
       "                                                  [&#x27;enrolled_university&#x27;,\n",
       "                                                   &#x27;relevant_experience&#x27;,\n",
       "                                                   &#x27;last_new_job&#x27;]),\n",
       "                                                 (&#x27;onehot&#x27;,\n",
       "                                                  CustomOneHotEncoder(columns=[&#x27;gender&#x27;,\n",
       "                                                                               &#x27;ac...\n",
       "                               gamma=0, gpu_id=-1, grow_policy=&#x27;depthwise&#x27;,\n",
       "                               importance_type=None, interaction_constraints=&#x27;&#x27;,\n",
       "                               learning_rate=0.01, max_bin=256,\n",
       "                               max_cat_to_onehot=4, max_delta_step=0,\n",
       "                               max_depth=3, max_leaves=0, min_child_weight=1,\n",
       "                               missing=nan, monotone_constraints=&#x27;()&#x27;,\n",
       "                               n_estimators=100, n_jobs=-1, num_parallel_tree=1,\n",
       "                               predictor=&#x27;auto&#x27;, random_state=0, reg_alpha=0,\n",
       "                               reg_lambda=1, ...))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-507\" type=\"checkbox\" ><label for=\"sk-estimator-id-507\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;preprocessor&#x27;,\n",
       "                 ColumnTransformer(transformers=[(&#x27;feature_engineering&#x27;,\n",
       "                                                  FeatureEngineering(),\n",
       "                                                  [&#x27;experience&#x27;]),\n",
       "                                                 (&#x27;custom_label_encoder&#x27;,\n",
       "                                                  CustomLabelEncoder(columns=[&#x27;enrolled_university&#x27;,\n",
       "                                                                              &#x27;relevant_experience&#x27;,\n",
       "                                                                              &#x27;last_new_job&#x27;]),\n",
       "                                                  [&#x27;enrolled_university&#x27;,\n",
       "                                                   &#x27;relevant_experience&#x27;,\n",
       "                                                   &#x27;last_new_job&#x27;]),\n",
       "                                                 (&#x27;onehot&#x27;,\n",
       "                                                  CustomOneHotEncoder(columns=[&#x27;gender&#x27;,\n",
       "                                                                               &#x27;ac...\n",
       "                               gamma=0, gpu_id=-1, grow_policy=&#x27;depthwise&#x27;,\n",
       "                               importance_type=None, interaction_constraints=&#x27;&#x27;,\n",
       "                               learning_rate=0.01, max_bin=256,\n",
       "                               max_cat_to_onehot=4, max_delta_step=0,\n",
       "                               max_depth=3, max_leaves=0, min_child_weight=1,\n",
       "                               missing=nan, monotone_constraints=&#x27;()&#x27;,\n",
       "                               n_estimators=100, n_jobs=-1, num_parallel_tree=1,\n",
       "                               predictor=&#x27;auto&#x27;, random_state=0, reg_alpha=0,\n",
       "                               reg_lambda=1, ...))])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-508\" type=\"checkbox\" ><label for=\"sk-estimator-id-508\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">preprocessor: ColumnTransformer</label><div class=\"sk-toggleable__content\"><pre>ColumnTransformer(transformers=[(&#x27;feature_engineering&#x27;, FeatureEngineering(),\n",
       "                                 [&#x27;experience&#x27;]),\n",
       "                                (&#x27;custom_label_encoder&#x27;,\n",
       "                                 CustomLabelEncoder(columns=[&#x27;enrolled_university&#x27;,\n",
       "                                                             &#x27;relevant_experience&#x27;,\n",
       "                                                             &#x27;last_new_job&#x27;]),\n",
       "                                 [&#x27;enrolled_university&#x27;, &#x27;relevant_experience&#x27;,\n",
       "                                  &#x27;last_new_job&#x27;]),\n",
       "                                (&#x27;onehot&#x27;,\n",
       "                                 CustomOneHotEncoder(columns=[&#x27;gender&#x27;,\n",
       "                                                              &#x27;academic_discipline&#x27;,\n",
       "                                                              &#x27;company_type...\n",
       "                                                          &#x27;company_size_10000+&#x27;]),\n",
       "                                 [&#x27;cid&#x27;, &#x27;city.100&#x27;, &#x27;city.10&#x27;, &#x27;city.105&#x27;,\n",
       "                                  &#x27;city.103&#x27;, &#x27;city.104&#x27;, &#x27;city.102&#x27;,\n",
       "                                  &#x27;city.101&#x27;, &#x27;city.2119&#x27;, &#x27;city.114&#x27;,\n",
       "                                  &#x27;city.2341&#x27;, &#x27;city.116&#x27;, &#x27;city.123&#x27;,\n",
       "                                  &#x27;city.128&#x27;, &#x27;city.2194&#x27;, &#x27;city.138&#x27;,\n",
       "                                  &#x27;city.134&#x27;, &#x27;city.136&#x27;, &#x27;city.143&#x27;,\n",
       "                                  &#x27;city.145&#x27;, &#x27;city.142&#x27;, &#x27;city.149&#x27;,\n",
       "                                  &#x27;city.3205&#x27;, &#x27;city.1532&#x27;, &#x27;city.958&#x27;,\n",
       "                                  &#x27;city.959&#x27;, &#x27;city.96&#x27;, &#x27;city.960&#x27;,\n",
       "                                  &#x27;city.1632&#x27;, &#x27;city.665&#x27;, ...])])</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-509\" type=\"checkbox\" ><label for=\"sk-estimator-id-509\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">feature_engineering</label><div class=\"sk-toggleable__content\"><pre>[&#x27;experience&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-510\" type=\"checkbox\" ><label for=\"sk-estimator-id-510\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FeatureEngineering</label><div class=\"sk-toggleable__content\"><pre>FeatureEngineering()</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-511\" type=\"checkbox\" ><label for=\"sk-estimator-id-511\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">custom_label_encoder</label><div class=\"sk-toggleable__content\"><pre>[&#x27;enrolled_university&#x27;, &#x27;relevant_experience&#x27;, &#x27;last_new_job&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-512\" type=\"checkbox\" ><label for=\"sk-estimator-id-512\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">CustomLabelEncoder</label><div class=\"sk-toggleable__content\"><pre>CustomLabelEncoder(columns=[&#x27;enrolled_university&#x27;, &#x27;relevant_experience&#x27;,\n",
       "                            &#x27;last_new_job&#x27;])</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-513\" type=\"checkbox\" ><label for=\"sk-estimator-id-513\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">onehot</label><div class=\"sk-toggleable__content\"><pre>[&#x27;gender&#x27;, &#x27;academic_discipline&#x27;, &#x27;company_type&#x27;, &#x27;company_size&#x27;, &#x27;education_level&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-514\" type=\"checkbox\" ><label for=\"sk-estimator-id-514\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">CustomOneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>CustomOneHotEncoder(columns=[&#x27;gender&#x27;, &#x27;academic_discipline&#x27;, &#x27;company_type&#x27;,\n",
       "                             &#x27;company_size&#x27;, &#x27;education_level&#x27;])</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-515\" type=\"checkbox\" ><label for=\"sk-estimator-id-515\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">feature_selector</label><div class=\"sk-toggleable__content\"><pre>[&#x27;cid&#x27;, &#x27;city.100&#x27;, &#x27;city.10&#x27;, &#x27;city.105&#x27;, &#x27;city.103&#x27;, &#x27;city.104&#x27;, &#x27;city.102&#x27;, &#x27;city.101&#x27;, &#x27;city.2119&#x27;, &#x27;city.114&#x27;, &#x27;city.2341&#x27;, &#x27;city.116&#x27;, &#x27;city.123&#x27;, &#x27;city.128&#x27;, &#x27;city.2194&#x27;, &#x27;city.138&#x27;, &#x27;city.134&#x27;, &#x27;city.136&#x27;, &#x27;city.143&#x27;, &#x27;city.145&#x27;, &#x27;city.142&#x27;, &#x27;city.149&#x27;, &#x27;city.3205&#x27;, &#x27;city.1532&#x27;, &#x27;city.958&#x27;, &#x27;city.959&#x27;, &#x27;city.96&#x27;, &#x27;city.960&#x27;, &#x27;city.1632&#x27;, &#x27;city.665&#x27;, &#x27;city.673&#x27;, &#x27;city.699&#x27;, &#x27;city.326&#x27;, &#x27;city.323&#x27;, &#x27;city.324&#x27;, &#x27;city.327&#x27;, &#x27;city.328&#x27;, &#x27;city.36&#x27;, &#x27;city.40&#x27;, &#x27;city.4615&#x27;, &#x27;city.45&#x27;, &#x27;city.46&#x27;, &#x27;city.50&#x27;, &#x27;city.57&#x27;, &#x27;city.698&#x27;, &#x27;city.64&#x27;, &#x27;city.65&#x27;, &#x27;city.67&#x27;, &#x27;city.70&#x27;, &#x27;city.798&#x27;, &#x27;city.73&#x27;, &#x27;city.74&#x27;, &#x27;city.75&#x27;, &#x27;city.76&#x27;, &#x27;city.83&#x27;, &#x27;city.89&#x27;, &#x27;city.90&#x27;, &#x27;city.998&#x27;, &#x27;city.97&#x27;, &#x27;city.98&#x27;, &#x27;city.99&#x27;, &#x27;gender&#x27;, &#x27;enrolled_university&#x27;, &#x27;education_level&#x27;, &#x27;academic_discipline&#x27;, &#x27;experience&#x27;, &#x27;company_size&#x27;, &#x27;company_type&#x27;, &#x27;last_new_job&#x27;, &#x27;city_development_index&#x27;, &#x27;relevant_experience&#x27;, &#x27;training_hours&#x27;]</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-516\" type=\"checkbox\" ><label for=\"sk-estimator-id-516\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FeatureSelector</label><div class=\"sk-toggleable__content\"><pre>FeatureSelector(columns=[&#x27;city_development_index&#x27;,\n",
       "                         &#x27;enrolled_university_encoded&#x27;, &#x27;company_type_Pvt Ltd&#x27;,\n",
       "                         &#x27;relevant_experience_encoded&#x27;,\n",
       "                         &#x27;education_level_Undergraduate&#x27;,\n",
       "                         &#x27;company_size_10000+&#x27;])</pre></div></div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-517\" type=\"checkbox\" ><label for=\"sk-estimator-id-517\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SMOTE</label><div class=\"sk-toggleable__content\"><pre>SMOTE(random_state=1211)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-518\" type=\"checkbox\" ><label for=\"sk-estimator-id-518\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">XGBClassifier</label><div class=\"sk-toggleable__content\"><pre>XGBClassifier(base_score=0.5, booster=&#x27;gbtree&#x27;, callbacks=None,\n",
       "              colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1,\n",
       "              early_stopping_rounds=None, enable_categorical=False,\n",
       "              eval_metric=None, gamma=0, gpu_id=-1, grow_policy=&#x27;depthwise&#x27;,\n",
       "              importance_type=None, interaction_constraints=&#x27;&#x27;,\n",
       "              learning_rate=0.01, max_bin=256, max_cat_to_onehot=4,\n",
       "              max_delta_step=0, max_depth=3, max_leaves=0, min_child_weight=1,\n",
       "              missing=nan, monotone_constraints=&#x27;()&#x27;, n_estimators=100,\n",
       "              n_jobs=-1, num_parallel_tree=1, predictor=&#x27;auto&#x27;, random_state=0,\n",
       "              reg_alpha=0, reg_lambda=1, ...)</pre></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('preprocessor',\n",
       "                 ColumnTransformer(transformers=[('feature_engineering',\n",
       "                                                  FeatureEngineering(),\n",
       "                                                  ['experience']),\n",
       "                                                 ('custom_label_encoder',\n",
       "                                                  CustomLabelEncoder(columns=['enrolled_university',\n",
       "                                                                              'relevant_experience',\n",
       "                                                                              'last_new_job']),\n",
       "                                                  ['enrolled_university',\n",
       "                                                   'relevant_experience',\n",
       "                                                   'last_new_job']),\n",
       "                                                 ('onehot',\n",
       "                                                  CustomOneHotEncoder(columns=['gender',\n",
       "                                                                               'ac...\n",
       "                               gamma=0, gpu_id=-1, grow_policy='depthwise',\n",
       "                               importance_type=None, interaction_constraints='',\n",
       "                               learning_rate=0.01, max_bin=256,\n",
       "                               max_cat_to_onehot=4, max_delta_step=0,\n",
       "                               max_depth=3, max_leaves=0, min_child_weight=1,\n",
       "                               missing=nan, monotone_constraints='()',\n",
       "                               n_estimators=100, n_jobs=-1, num_parallel_tree=1,\n",
       "                               predictor='auto', random_state=0, reg_alpha=0,\n",
       "                               reg_lambda=1, ...))])"
      ]
     },
     "execution_count": 683,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.replace({\"yes\": 1, \"no\":0}, inplace = True)\n",
    "pipeline.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Optional: Persisting the pipeline and loading it\n",
    "\n",
    "This optional code helps to persist the pipeline to a file which can be later loaded on any machine without having to retrain the pipeline on the given data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 684,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "dump(pipeline, 'pipeline.joblib') # Dump the pipeline to a file with extension .joblib\n",
    "pipeline_loaded = load('pipeline.joblib') # Load the stored pipeline using the filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "__Testing the loaded pipeline by calculating F1 score__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 685,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5540495867768594"
      ]
     },
     "execution_count": 685,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1_score(pipeline_loaded.predict(X),y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Making predictions on test dataset and exporting the predictions\n",
    "\n",
    "Having trained the pipeline on the train dataset, now we will make our predictions on the test dataset and ship it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Generating the predictions file from Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 695,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     cid target\n",
      "0  12321     no\n",
      "1    941     no\n",
      "2  17715     no\n",
      "3   6540     no\n",
      "4   6760    yes\n"
     ]
    }
   ],
   "source": [
    "predictions =  pd.concat([X_test.cid,pd.Series(pipeline.predict(X_test),name= \"target\").replace({1:\"yes\",0:\"no\"})],axis=1)\n",
    "\n",
    "print(predictions.head())\n",
    "\n",
    "predictions.to_csv(\"pred_labels.csv\",index= False, header = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Generating the predictions from Kaggle data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 694,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   cid  target\n",
      "0    1       0\n",
      "1    3       1\n",
      "2    7       0\n",
      "3   12       0\n",
      "4   15       1\n"
     ]
    }
   ],
   "source": [
    "predictions =  pd.concat([X_kaggle.cid,pd.Series(pipeline.predict(X_kaggle),name= \"target\")],axis=1)\n",
    "\n",
    "print(predictions.head())\n",
    "\n",
    "predictions.to_csv(\"pred_labels_kaggle_tuned.csv\",index= False, header = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# References\n",
    "\n",
    "- Creating custom scikit-learn Transformers. (n.d.). Andrew Villazon. Retrieved October 23, 2022, from https://www.andrewvillazon.com/custom-scikit-learn-transformers/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}