Titanic_Iniciantes.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Titanic Iniciantes.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyM3qd1TuX91LARLFpJBvBts",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/josepefg/kaggle/blob/master/Titanic_Iniciantes.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7zyw0C-GjIKq",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://www.linkedin.com/in/josepegerent\"> <img border=\"0\" alt=\"Linkedin\" src=\"https://img.shields.io/badge/Autor-josepefg-red.svg\"> </a>\n",
        "# **Desafio do Titanic**  \n",
        "## **Solução básica**\n",
        "## [Titanic: Machine Learning from Disaster](https://www.kaggle.com/c/titanic)  \n",
        "Essa é a famosa competição do Kaggle sobre o acidente do Titanic. Ela é considerada a principal competição para iniciantes no mundo de Data Science. O objetivo dessa competição é simples, predizer os sobreviventes do acidente a partir de um modelo de Machine Learning.\n",
        "![Titanic](https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/RMS_Titanic_3.jpg/1280px-RMS_Titanic_3.jpg)  \n",
        "Abaixo segue uma maneira rápida, simples e direta para criar o seu primeiro modelo de machine learning para predição."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g4x6eGaNjS75",
        "colab_type": "text"
      },
      "source": [
        "## **Obtendo os dados**  \n",
        "Os dados foram divididos em dois grupos:\n",
        "*   Treino (train.csv)  \n",
        "*   Teste (test.csv)  \n",
        "Os dados de treino devem ser utilizados para a análise e criação dos modelos de Machine Learning, enquanto que os dados de teste servem para avaliar o modelo criado.  \n",
        "Para baixar os arquivos é necessário criar uma conta no Kaggle (não se preocupe, é de graça). Para facilitar eu já baixei esses arquivos e salvei no meu repositório.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QFSI8vcBi905",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# importando os pacotes necessarios\n",
        "# essa biblioteca é apenas para suprimir os warnings que possam aparecer\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\", category=Warning) \n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "# permitir visualizar todas as colunas\n",
        "pd.options.display.max_columns = None"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VXrgDMWdjG2F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train = pd.read_csv('https://raw.githubusercontent.com/josepefg/kaggle/master/data/titanic/train.csv')\n",
        "test = pd.read_csv('https://raw.githubusercontent.com/josepefg/kaggle/master/data/titanic/test.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kJEAN0H-ucpg",
        "colab_type": "text"
      },
      "source": [
        "## **Entendendo os dados**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dDFA46m1jc3k",
        "colab_type": "code",
        "outputId": "aed6ba11-c9c6-430c-8af4-73917a8954b2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "# Amostra do dataset\n",
        "display(train.head())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PassengerId</th>\n",
              "      <th>Survived</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>Name</th>\n",
              "      <th>Sex</th>\n",
              "      <th>Age</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Parch</th>\n",
              "      <th>Ticket</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Cabin</th>\n",
              "      <th>Embarked</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Braund, Mr. Owen Harris</td>\n",
              "      <td>male</td>\n",
              "      <td>22.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>A/5 21171</td>\n",
              "      <td>7.2500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
              "      <td>female</td>\n",
              "      <td>38.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>PC 17599</td>\n",
              "      <td>71.2833</td>\n",
              "      <td>C85</td>\n",
              "      <td>C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>Heikkinen, Miss. Laina</td>\n",
              "      <td>female</td>\n",
              "      <td>26.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>STON/O2. 3101282</td>\n",
              "      <td>7.9250</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
              "      <td>female</td>\n",
              "      <td>35.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>113803</td>\n",
              "      <td>53.1000</td>\n",
              "      <td>C123</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Allen, Mr. William Henry</td>\n",
              "      <td>male</td>\n",
              "      <td>35.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>373450</td>\n",
              "      <td>8.0500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   PassengerId  Survived  Pclass  \\\n",
              "0            1         0       3   \n",
              "1            2         1       1   \n",
              "2            3         1       3   \n",
              "3            4         1       1   \n",
              "4            5         0       3   \n",
              "\n",
              "                                                Name     Sex   Age  SibSp  \\\n",
              "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
              "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
              "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
              "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
              "4                           Allen, Mr. William Henry    male  35.0      0   \n",
              "\n",
              "   Parch            Ticket     Fare Cabin Embarked  \n",
              "0      0         A/5 21171   7.2500   NaN        S  \n",
              "1      0          PC 17599  71.2833   C85        C  \n",
              "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
              "3      0            113803  53.1000  C123        S  \n",
              "4      0            373450   8.0500   NaN        S  "
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zmLZTXLruqIl",
        "colab_type": "text"
      },
      "source": [
        "## **Preparação dos dados**  \n",
        "Para que os modelos possam ser treinados não podem haver dados categóricos nem dados faltantes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_n04iIH1lYyR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Feature Engineering\n",
        "# converter 'Sex' em 0 e 1\n",
        "train['Sex'] = train['Sex'].map({'male': 0, 'female': 1})\n",
        "test['Sex'] = test['Sex'].map({'male': 0, 'female': 1})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bGr05aI0lGMj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Excluir linhas categóricas do treino e do teste\n",
        "train.drop([\"Name\", \"Ticket\", \"Cabin\", \"Embarked\"],  axis=1, inplace=True)\n",
        "test.drop([\"Name\", \"Ticket\", \"Cabin\", \"Embarked\"],  axis=1, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nKeJTnBMj7hy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Salvar coluna Survived como alvo e depois excluir do treino\n",
        "target = train[\"Survived\"]\n",
        "train.drop(['Survived'], axis=1, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cf7zAP-Hl1nf",
        "colab_type": "code",
        "outputId": "99048383-f8f2-4530-c64d-5584337b04e3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        }
      },
      "source": [
        "# Valores faltantes nos datasets de treino e test\n",
        "print(train.isnull().sum())\n",
        "print(test.isnull().sum())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "PassengerId      0\n",
            "Pclass           0\n",
            "Sex              0\n",
            "Age            177\n",
            "SibSp            0\n",
            "Parch            0\n",
            "Fare             0\n",
            "dtype: int64\n",
            "PassengerId     0\n",
            "Pclass          0\n",
            "Sex             0\n",
            "Age            86\n",
            "SibSp           0\n",
            "Parch           0\n",
            "Fare            1\n",
            "dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nzfwp807tzpD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Excluir dados faltantes de \"Age\"\n",
        "train.drop([\"Age\"], axis=1, inplace=True)\n",
        "test.drop([\"Age\"], axis=1, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wOaVAhMIxVXJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Tratar dados faltantes de \"Fare\" no test\n",
        "fare_median = test['Fare'].median()\n",
        "test['Fare'].fillna(fare_median, inplace=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D_H7tpS-kMgd",
        "colab_type": "text"
      },
      "source": [
        "## **Construção e avaliação dos modelos**  \n",
        "Abaixo foram criados dois modelos simples de classificação para podermos comparar e escolher o de melhor acurácia."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GPOL0z8YkAd_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Importar bibliotecas dos modelos de Machine Learning\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.tree import DecisionTreeClassifier"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uASYWkczkPWM",
        "colab_type": "code",
        "outputId": "de260c78-9320-43cb-8c4e-b4a4fbcffb0e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Criar um modelo de Regressão Logística\n",
        "lr_model = LogisticRegression(solver='liblinear')\n",
        "lr_model.fit(train, target)\n",
        "\n",
        "# Verificar a acurácia do modelo\n",
        "acc_logReg = round(lr_model.score(train, target) * 100, 2)\n",
        "print(\"Acurácia do modelo de Regressão Logística: {}\".format(acc_logReg))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Acurácia do modelo de Regressão Logística: 79.57\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_scVOeFAkSDK",
        "colab_type": "code",
        "outputId": "1750e65c-79e5-48c5-d655-aec032ca9938",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Criar um modelo de árvore de decisão\n",
        "tree_model = DecisionTreeClassifier(max_depth=3)\n",
        "tree_model.fit(train, target)\n",
        "\n",
        "# Verificar a acurácia do modelo\n",
        "acc_tree = round(tree_model.score(train, target) * 100, 2)\n",
        "print(\"Acurácia do modelo de Árvore de Decisão: {}\".format(acc_tree))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Acurácia do modelo de Árvore de Decisão: 81.03\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9aF-o6DwvKCL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Salvar o melhor modelo para submissão\n",
        "y_pred = tree_model.predict(test)\n",
        "passengerId = test['PassengerId']\n",
        "\n",
        "submission = pd.DataFrame({\n",
        "    \"PassengerId\": passengerId,\n",
        "    \"Survived\": y_pred\n",
        "})\n",
        "\n",
        "# Gerar arquivo csv\n",
        "filename = './submission.csv'\n",
        "\n",
        "submission.to_csv(filename, index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eU_14Cdcy-Ku",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://raw.githubusercontent.com/josepefg/kaggle/master/resources/Screenshot_18.png)"
      ]
    }
  ]
}