HOTH_6.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "HOTH-6.ipynb",
      "version": "0.3.2",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "XRO3lbOE6Ojb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# HOTH-6 Intro to ML Workshop\n",
        "\n",
        "Follow along at [https://github.com/uclaacmai/hoth-6-workshop](https://github.com/uclaacmai/hoth-6-workshop)!\n",
        "\n",
        "In this workshop, we'll go over the basic steps behind what a typical machine learning workflow looks like so that you can then use that knowledge to apply ML in your hack!\n",
        "\n",
        "![](https://cdn-images-1.medium.com/max/2000/1*KzmIUYPmxgEHhXX7SlbP4w.jpeg)"
      ]
    },
    {
      "metadata": {
        "id": "UWEkly3A615H",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Sample Workflow"
      ]
    },
    {
      "metadata": {
        "id": "KlEJSY2J64gt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let's think about what we need to do when approaching any machine learning competition/problem. Like we talked about during the slides, any problem involving some sort of prediction task creates the possibility for us to apply machine learning to it.  \n",
        "\n",
        "1) Determine your problem space. Do you have a classification problem, or a regression problem? \n",
        "\n",
        "2) Determine what model you want to use (Always good to start off with simple models).\n",
        "\n",
        "3) Load in and preprocess your dataset. Examine your database to see if there are any NULL or non-numeric values.\n",
        "\n",
        "4) Split up your dataset into training and testing components. \n",
        "\n",
        "5) Create your model. Depending on the libraries you are using, this could entail defining your function, your placeholders, the loss function, and the optimizer. \n",
        "\n",
        "6) Train, evaluate, and iterate on your model!\n",
        "\n",
        "7) Once you have a model that you've trained and that you're satisfied with, you can deploy it to a server! [Flask](http://flask.pocoo.org/) is often a good choice for serving ML models. "
      ]
    },
    {
      "metadata": {
        "id": "PNvTkhl_8Dgz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Using ML in the Titanic Competition"
      ]
    },
    {
      "metadata": {
        "id": "1E6ABo_t8P_M",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "To show everybody what this workflow looks like in practice, we'll do the classic Titanic Prediction problem that is hosted on [Kaggle](https://www.kaggle.com/c/titanic). The goal is to be able to predict who survived and who passed away during the Titanic tragedy, given information about the people involved."
      ]
    },
    {
      "metadata": {
        "id": "f6gtPtZr9TJx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Getting the Data\n",
        "\n",
        "You can download the data from the Kaggle website. The direct link is [here](https://www.kaggle.com/c/titanic/data)"
      ]
    },
    {
      "metadata": {
        "id": "5vJ3KnTy6PK6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import pandas as pd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Vb_rrq9n9S5P",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Use the Pandas read_csv() function to load in the train.csv\n",
        "titanicTrain = pd.read_csv('https://raw.githubusercontent.com/uclaacmai/hoth-6-workshop/master/train.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "sUGJKxcZ9YY7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Examining Data"
      ]
    },
    {
      "metadata": {
        "id": "tSuJ0Y4P8tnq",
        "colab_type": "code",
        "outputId": "37b8d5de-3b40-4176-f301-eaded939e73e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        }
      },
      "cell_type": "code",
      "source": [
        "# Figure out what the different column names are\n",
        "titanicTrain.columns.tolist()\n",
        "titanicTrain.describe()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PassengerId</th>\n",
              "      <th>Survived</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>Age</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Parch</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>891.000000</td>\n",
              "      <td>891.000000</td>\n",
              "      <td>891.000000</td>\n",
              "      <td>714.000000</td>\n",
              "      <td>891.000000</td>\n",
              "      <td>891.000000</td>\n",
              "      <td>891.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>446.000000</td>\n",
              "      <td>0.383838</td>\n",
              "      <td>2.308642</td>\n",
              "      <td>29.699118</td>\n",
              "      <td>0.523008</td>\n",
              "      <td>0.381594</td>\n",
              "      <td>32.204208</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>257.353842</td>\n",
              "      <td>0.486592</td>\n",
              "      <td>0.836071</td>\n",
              "      <td>14.526497</td>\n",
              "      <td>1.102743</td>\n",
              "      <td>0.806057</td>\n",
              "      <td>49.693429</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.420000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>223.500000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>20.125000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>7.910400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>446.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>14.454200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>668.500000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>38.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>31.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>891.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>80.000000</td>\n",
              "      <td>8.000000</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>512.329200</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
              "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
              "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
              "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
              "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
              "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
              "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
              "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
              "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
              "\n",
              "            Parch        Fare  \n",
              "count  891.000000  891.000000  \n",
              "mean     0.381594   32.204208  \n",
              "std      0.806057   49.693429  \n",
              "min      0.000000    0.000000  \n",
              "25%      0.000000    7.910400  \n",
              "50%      0.000000   14.454200  \n",
              "75%      0.000000   31.000000  \n",
              "max      6.000000  512.329200  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "metadata": {
        "id": "0G6aM6M3-tiP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Cleaning Data\n",
        "\n",
        "This is one of the most important parts of any machine learning pipeline. We want to make sure that the inputs we feed into any machine learning model are are valid, non-null, and are numerical values. To get you started with datapreprocessing, we'll show you one example of a column you may want to drop in this dataset "
      ]
    },
    {
      "metadata": {
        "id": "2hdUa3dp-p0F",
        "colab_type": "code",
        "outputId": "6bb13430-2b44-4255-cfe4-0ada58b74fba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        }
      },
      "cell_type": "code",
      "source": [
        "# Visualize the data we're working with\n",
        "titanicTrain.info()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 891 entries, 0 to 890\n",
            "Data columns (total 12 columns):\n",
            "PassengerId    891 non-null int64\n",
            "Survived       891 non-null int64\n",
            "Pclass         891 non-null int64\n",
            "Name           891 non-null object\n",
            "Sex            891 non-null object\n",
            "Age            714 non-null float64\n",
            "SibSp          891 non-null int64\n",
            "Parch          891 non-null int64\n",
            "Ticket         891 non-null object\n",
            "Fare           891 non-null float64\n",
            "Cabin          204 non-null object\n",
            "Embarked       889 non-null object\n",
            "dtypes: float64(2), int64(5), object(5)\n",
            "memory usage: 83.6+ KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "DtHBvwsF-z--",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "So, as you can see above, some of the people don't have values for the age and cabin attributes. There are ways we can deal with this (for example, replace the null values with the median of the other values, replace them with 0, etc), but a simple method is to just drop the column."
      ]
    },
    {
      "metadata": {
        "id": "Sc_DvXZR-w3V",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Drop the column\n",
        "titanicTrain.drop(['Cabin'], axis = 1, inplace = True)\n",
        "# alternatively, if you don't wish to modify the original data structure, you can re-assign the result of.drop().\n",
        "#titanicTrain_dropped = titanicTrain.drop(['Cabin'], axis = 1) # For axis number (0 for rows and 1 for columns)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ru49AFLJ-27Z",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Another column that needs processing is the age."
      ]
    },
    {
      "metadata": {
        "id": "ioPpsTmK-ykW",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Do the preprocessing\n",
        "medianAge = titanicTrain['Age'].median()\n",
        "titanicTrain['Age'].fillna(medianAge, inplace = True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "XIHOpscp-8Vr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now, try it on your own! The functions you will probably be using are (although you're not limited to just these!):\n",
        "- [drop()](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop.html)\n",
        "- [fillna](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html)\n",
        "- [get_dummies()](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html)\n",
        "- [dropna()](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.dropna.html)"
      ]
    },
    {
      "metadata": {
        "id": "zvl3_qGZ-6G1",
        "colab_type": "code",
        "outputId": "51c94c66-79fa-4a2b-dba4-10d9f03cab9f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "titanicTrain.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PassengerId</th>\n",
              "      <th>Survived</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>Name</th>\n",
              "      <th>Sex</th>\n",
              "      <th>Age</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Parch</th>\n",
              "      <th>Ticket</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Embarked</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Braund, Mr. Owen Harris</td>\n",
              "      <td>male</td>\n",
              "      <td>22.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>A/5 21171</td>\n",
              "      <td>7.2500</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
              "      <td>female</td>\n",
              "      <td>38.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>PC 17599</td>\n",
              "      <td>71.2833</td>\n",
              "      <td>C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>Heikkinen, Miss. Laina</td>\n",
              "      <td>female</td>\n",
              "      <td>26.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>STON/O2. 3101282</td>\n",
              "      <td>7.9250</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
              "      <td>female</td>\n",
              "      <td>35.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>113803</td>\n",
              "      <td>53.1000</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Allen, Mr. William Henry</td>\n",
              "      <td>male</td>\n",
              "      <td>35.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>373450</td>\n",
              "      <td>8.0500</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   PassengerId  Survived  Pclass  \\\n",
              "0            1         0       3   \n",
              "1            2         1       1   \n",
              "2            3         1       3   \n",
              "3            4         1       1   \n",
              "4            5         0       3   \n",
              "\n",
              "                                                Name     Sex   Age  SibSp  \\\n",
              "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
              "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
              "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
              "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
              "4                           Allen, Mr. William Henry    male  35.0      0   \n",
              "\n",
              "   Parch            Ticket     Fare Embarked  \n",
              "0      0         A/5 21171   7.2500        S  \n",
              "1      0          PC 17599  71.2833        C  \n",
              "2      0  STON/O2. 3101282   7.9250        S  \n",
              "3      0            113803  53.1000        S  \n",
              "4      0            373450   8.0500        S  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "metadata": {
        "id": "94qAkFdc-7jm",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# TODO Find the other attributes that may give us trouble later on! Once you find these\n",
        "# columns, figure out if you just want to drop the attribute altogether or replace with \n",
        "# median, or something else!\n",
        "\n",
        "# HINT: The name attribute is something you may want to look at. We don't want strings in our ML model!"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "hwq4SX27_QlB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now that you know a couple ways of dealing with null values and string values, feel free to be creative! The best way to get a more accurate machine learning model is to understand the best ways to visualize and clean your data! This is one of the most important steps in any ML pipeline. "
      ]
    },
    {
      "metadata": {
        "id": "8GigujTq_YQT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Creating Testing and Training Matrices"
      ]
    },
    {
      "metadata": {
        "id": "ijGfjUYD-60b",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "So, now that we've made our final changes to our dataframe, we want to convert it into a matrix of numbers. We want our Y Matrix to be filled with binary labels indicating whether the person survived or not. Our X Matrix should contain all of the features that represent each individual.  "
      ]
    },
    {
      "metadata": {
        "id": "FZ5eSgPQ_bEo",
        "colab_type": "code",
        "outputId": "b797093b-25db-475f-c434-f1575fc61143",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        }
      },
      "cell_type": "code",
      "source": [
        "titanicTrain.info()\n",
        "titanicTrain.head()\n",
        "mapping = {\n",
        "    'female': 1,\n",
        "    'male': 0\n",
        "}\n",
        "titanicTrain['Sex'] = titanicTrain['Sex'].map(mapping)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 891 entries, 0 to 890\n",
            "Data columns (total 11 columns):\n",
            "PassengerId    891 non-null int64\n",
            "Survived       891 non-null int64\n",
            "Pclass         891 non-null int64\n",
            "Name           891 non-null object\n",
            "Sex            891 non-null object\n",
            "Age            891 non-null float64\n",
            "SibSp          891 non-null int64\n",
            "Parch          891 non-null int64\n",
            "Ticket         891 non-null object\n",
            "Fare           891 non-null float64\n",
            "Embarked       889 non-null object\n",
            "dtypes: float64(2), int64(5), object(4)\n",
            "memory usage: 76.6+ KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "KVbSbH5x_fPv",
        "colab_type": "code",
        "outputId": "be5b450c-e8bf-43ef-baa0-8e56b51b1967",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Convert to matrices. \n",
        "# TODO Add/Remove columns as you see fit\n",
        "print(titanicTrain.columns)\n",
        "X = titanicTrain[['Pclass', 'Age', 'SibSp', 'Sex', 'Parch', 'Fare']].as_matrix()\n",
        "Y = titanicTrain['Survived'].as_matrix()\n",
        "Y = Y.reshape([Y.shape[0], 1]) # Reshaping from (891,) to (891,1)\n",
        "print (X.shape)\n",
        "print (Y.shape)\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp',\n",
            "       'Parch', 'Ticket', 'Fare', 'Embarked'],\n",
            "      dtype='object')\n",
            "(891, 6)\n",
            "(891, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "3vzleT1m_utG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Remember that whenever we have a dataset, it's good practice to seperate the dataset into 2 parts, one that we will use to train the model, and one that we will use to check how our model is doing as a test/validation set."
      ]
    },
    {
      "metadata": {
        "id": "5_gHZ90p_keg",
        "colab_type": "code",
        "outputId": "48324c95-7eaa-4758-df2a-a802034a57e9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, Y)\n",
        "print(X_train.shape)\n",
        "print(X_test.shape)\n",
        "print(y_train.shape)\n",
        "print(y_test.shape)\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(668, 6)\n",
            "(223, 6)\n",
            "(668, 1)\n",
            "(223, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "q4UBYGVv_yen",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Create Model\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "SmJAWx6B_x9u",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from sklearn import linear_model\n",
        "\n",
        "# Some other models you can import:\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.ensemble import AdaBoostClassifier\n",
        "from sklearn.ensemble import GradientBoostingClassifier\n",
        "from sklearn.neighbors import KNeighborsClassifier"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NMFdWqoX_rhI",
        "colab_type": "code",
        "outputId": "c8f2c783-23fc-4406-86fd-b8353f5c510f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        }
      },
      "cell_type": "code",
      "source": [
        "model = linear_model.LogisticRegression()\n",
        "\n",
        "# Comment out the below line if you want to try some other classifiers\n",
        "# model = RandomForestClassifier()\n",
        "# model = AdaBoostClassifier()\n",
        "# model = GradientBoostingClassifier()\n",
        "# model = KNeighborsClassifier()\n",
        "\n",
        "model.fit(X_train,y_train)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
            "  FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py:761: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "          intercept_scaling=1, max_iter=100, multi_class='warn',\n",
              "          n_jobs=None, penalty='l2', random_state=None, solver='warn',\n",
              "          tol=0.0001, verbose=0, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "metadata": {
        "id": "pGqbXILAABMo",
        "colab_type": "code",
        "outputId": "7e625374-3c47-4bf1-d70d-a07d64a5d578",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "predictions = model.predict(X_test)\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.metrics import accuracy_score\n",
        "print(\"The accuracy is:\", accuracy_score(y_test,predictions))\n",
        "print(classification_report(y_test,predictions))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The accuracy is: 0.7982062780269058\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.86      0.85      0.85       155\n",
            "           1       0.67      0.68      0.67        68\n",
            "\n",
            "   micro avg       0.80      0.80      0.80       223\n",
            "   macro avg       0.76      0.76      0.76       223\n",
            "weighted avg       0.80      0.80      0.80       223\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ugGqecr3BDw0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### How This Can Help With Your Hack!"
      ]
    },
    {
      "metadata": {
        "id": "bGDBopnZCf5k",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "So today we saw what a typical machine learning pipeline looks like. Machine learning models can be incredibly powerful given you have an appropriate problem space and relevant data + labels. In terms of bringing ML into your hack, try to think about if there is a prediction problem that you would like to automate (determine the sentiment of a text, if something is a hot dog or not, if you should recommend something to someone) and then determine if there is relevant data + labels for that problem. \n",
        "\n",
        "Once you've trained a problem, then you can look to deploying the model. Flask is a Python webserver that accomplishes this. You can take a look at the following links for more information: \n",
        "\n",
        "- http://flask.pocoo.org/\n",
        "- https://towardsdatascience.com/a-flask-api-for-serving-scikit-learn-models-c8bcdaa41daa \n",
        "- https://hackernoon.com/deploy-a-machine-learning-model-using-flask-da580f84e60c\n",
        "- https://medium.com/coinmonks/deploy-your-first-deep-learning-neural-network-model-using-flask-keras-tensorflow-in-python-f4bb7309fc49\n",
        "\n",
        "Come up to us afterwards if you have any questions!"
      ]
    }
  ]
}