Project-Predicting Bike Rentals with Ensemble Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project - Machine Learning:  Predicting Bike Rentals with Model Ensembles  (Linear Regression, Decision Trees, Random Forests)\n",
    "\n",
    "**Background and Aim**\n",
    "\n",
    "Many American cities have communal bike sharing stations where you can rent bicycles by the hour or day. Washington, D.C. is one of these cities. The District collects detailed data on the number of bicycles people rent by the hour and day. In this project, we'll try to predict the total number of bikes people rented in a given hour. We'll predict the `cnt` column using all of the other columns, except for `casual` and `registered`. To accomplish this, we'll create a few different machine learning models including liner regression, decision trees and random forests, and evaluate their performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataset\n",
    "\n",
    "Hadi Fanaee-T at the University of Porto compiled this data into a CSV file, which we'll be working with in this project. The file contains 17380 rows, with each row representing the number of bike rentals for a single hour of a single day. We can download the data from the [University of California, Irvine's website](http://archive.ics.uci.edu/ml/datasets/Bike+Sharing+Dataset). \n",
    "The dataset contains the hourly and daily count of rental bikes between years 2011 and 2012 in Capital bikeshare system with the corresponding weather and seasonal information.\n",
    "Here are the descriptions for the relevant columns:\n",
    "\n",
    "- `instant` - A unique sequential ID number for each row\n",
    "- `dteday` - The date of the rentals\n",
    "- `season` - The season in which the rentals occurred\n",
    "- `yr` - The year the rentals occurred\n",
    "- `mnth` - The month the rentals occurred\n",
    "- `hr` - The hour the rentals occurred\n",
    "- `holiday` - Whether or not the day was a holiday\n",
    "- `weekday` - The day of the week (as a number, 0 to 7)\n",
    "- `workingday` - Whether or not the day was a working day\n",
    "- `weathersit` - The weather (as a categorical variable)\n",
    "- `temp` - The temperature, on a 0-1 scale\n",
    "- `atemp` - The adjusted temperature\n",
    "- `hum` - The humidity, on a 0-1 scale\n",
    "- `windspeed` - The wind speed, on a 0-1 scale\n",
    "- `casual` - The number of casual riders (people who hadn't previously signed up with the bike sharing program)\n",
    "- `registered` - The number of registered riders (people who had already signed up)\n",
    "- `cnt` - The total number of bike rentals (casual + registered)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "# Setting pandas display options for large data\n",
    "pd.options.display.max_rows = 200\n",
    "pd.options.display.max_columns = 150"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_rentals = pd.read_csv(\"bike_rental_hour.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 17379 entries, 0 to 17378\n",
      "Data columns (total 17 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   instant     17379 non-null  int64  \n",
      " 1   dteday      17379 non-null  object \n",
      " 2   season      17379 non-null  int64  \n",
      " 3   yr          17379 non-null  int64  \n",
      " 4   mnth        17379 non-null  int64  \n",
      " 5   hr          17379 non-null  int64  \n",
      " 6   holiday     17379 non-null  int64  \n",
      " 7   weekday     17379 non-null  int64  \n",
      " 8   workingday  17379 non-null  int64  \n",
      " 9   weathersit  17379 non-null  int64  \n",
      " 10  temp        17379 non-null  float64\n",
      " 11  atemp       17379 non-null  float64\n",
      " 12  hum         17379 non-null  float64\n",
      " 13  windspeed   17379 non-null  float64\n",
      " 14  casual      17379 non-null  int64  \n",
      " 15  registered  17379 non-null  int64  \n",
      " 16  cnt         17379 non-null  int64  \n",
      "dtypes: float64(4), int64(12), object(1)\n",
      "memory usage: 2.3+ MB\n"
     ]
    }
   ],
   "source": [
    "bike_rentals.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>dteday</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>hr</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.24</td>\n",
       "      <td>0.2879</td>\n",
       "      <td>0.81</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.2727</td>\n",
       "      <td>0.80</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8</td>\n",
       "      <td>32</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.22</td>\n",
       "      <td>0.2727</td>\n",
       "      <td>0.80</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5</td>\n",
       "      <td>27</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.24</td>\n",
       "      <td>0.2879</td>\n",
       "      <td>0.75</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.24</td>\n",
       "      <td>0.2879</td>\n",
       "      <td>0.75</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant      dteday  season  yr  mnth  hr  holiday  weekday  workingday  \\\n",
       "0        1  2011-01-01       1   0     1   0        0        6           0   \n",
       "1        2  2011-01-01       1   0     1   1        0        6           0   \n",
       "2        3  2011-01-01       1   0     1   2        0        6           0   \n",
       "3        4  2011-01-01       1   0     1   3        0        6           0   \n",
       "4        5  2011-01-01       1   0     1   4        0        6           0   \n",
       "\n",
       "   weathersit  temp   atemp   hum  windspeed  casual  registered  cnt  \n",
       "0           1  0.24  0.2879  0.81        0.0       3          13   16  \n",
       "1           1  0.22  0.2727  0.80        0.0       8          32   40  \n",
       "2           1  0.22  0.2727  0.80        0.0       5          27   32  \n",
       "3           1  0.24  0.2879  0.75        0.0       3          10   13  \n",
       "4           1  0.24  0.2879  0.75        0.0       0           1    1  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bike_rentals.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAFmCAYAAACFsVhxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeyUlEQVR4nO3dfZBd9X3f8ffHggIFISkFzbpW3OLxhqjYGcJDJyIQuy4QiJvWKe0kPJshlAajFMqUjA3pKNPGJZBiASatkjJjqlSFNExdWU4AQUJxJYXaNC52LGBbE0KM9QRmDYqIQf31j3PWOlwtuneXK+3ub9+vmTN77znfe/acrx4+e875nbMppSBJkur0rpneAEmSdPAY9JIkVcyglySpYga9JEkVM+glSarYYTO9AdMxPj7urQKSpHlr0aJFGbTWI3pJkipm0EuSVDGDvjU2NjbTm1ANezk89nK47Ofw2MvhOdi9NOglSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWJ9gz7JnyYpk0xfbJcnyaokLybZk+SxJCf1rOOIJHcl2ZVkd5L1SZb11CxJsjbJeDutTbJ4uLsrSdL8MsgR/enAuzvTKUABfqddfiNwA7Cyrd0BbEyysLOO1cAFwIXAWcCxwIYkCzo169p1nw+c175eO629kiRJwAC/1KaUsrP7PsmVwHeB/5IkwHXALaWUB9rll9OE/UXAmiSLgCuBK0opG9uaS4HngbOBh5Ispwn3M0spm9uaq4EvJTmxlPLMUPZWkqR5ZkrX6NtgvxL47VLKXwAnACPAwxM1pZQ9wOPAGe2sU4HDe2peALZ2alYArwGbO99uE7C7UyNJkqZoqr+m9hyacP8P7fuR9uv2nrrtwHs6NXuBXZPUjHRqdpZSvv/rZ0spJcmOTs2khvmMYJ/dPDz2cnjs5XDZz+Gxl8MzSC9HR0ente6pBv1VwJdLKV/tmd/7++EzybxevTWT1fddz3R3vNfY2NjQ1jXf2cvhsZfDZT+Hx14Oz8Hu5cBBn2Qp8A+AT3Rmb2u/jgAvdOYvZd9R/jZgAXAcsLOn5vFOzdIkmTiqby8THM/+ZwsOitNOP/1QfJt3ZPyVV2Z6EyRJc8xUrtFfAfwlcF9n3nM0IX3OxIwkR9KMrJ+43v4k8EZPzTJgeadmC3AMzbX6CSuAo3nrdXtJkjQFAx3Rt0fXPw/cV0p5dWJ+ex19NXBTkqeBZ4GbaQbWrWtrxpPcA9zWXnN/CbgdeAp4pK3ZmuRBmlH6V9Gcsl8DbHDEvSRJ0zfoqfsPA+8HLp5k2a3AUcDdwBLgCeDc7g8EwPXAm8D9be2jwGWllL2dmouBO9k3On89cO2A2ydJkiYxUNCXUv6Q5ih7smUFWNVOb/f512keqLPyADUvA5cMsj2SJGkwPutekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVbKCgT/LuJPcm2Znk9STfSPKhzvIkWZXkxSR7kjyW5KSedRyR5K4ku5LsTrI+ybKemiVJ1iYZb6e1SRYPZ1clSZp/+gZ9G7SbgAAfBZYDK4EdnbIbgRva+ae3yzYmWdipWQ1cAFwInAUcC2xIsqBTsw44BTgfOK99vXY6OyZJkuCwAWpuBL5dSrmsM++5iRdJAlwH3FJKeaCddzlN2F8ErEmyCLgSuKKUsrGtuRR4HjgbeCjJcppwP7OUsrmtuRr4UpITSynPvLNdlSRp/hnk1P3HgCeS3J9kR5KvJrm2DXiAE4AR4OGJD5RS9gCPA2e0s04FDu+peQHY2qlZAbwGbO58703A7k6NJEmagkGO6N8HXAN8BrgFOBm4q132WZqQB9je87ntwHva1yPAXmDXJDUjnZqdpZQysbCUUpLs6NTsZ2xsbIBd6O+0oazl4BrWvh4Kc2lbZzt7OVz2c3js5fAM0svR0dFprXuQoH8X8JVSyifb93+cZBT4BE3QTyg9n8sk83r11kxWf8D1THfH56K5sq9jY2NzZltnO3s5XPZzeOzl8BzsXg5y6v7bwDd65m0F3tu+3tZ+7T3qXsq+o/xtwALguD41SzuXBCau/x/P/mcLJEnSAAYJ+k3AiT3zfohmIB00A/O2AedMLExyJM3I+onr7U8Cb/TULKMZwT9RswU4huZa/YQVwNG89bq9JEka0CCn7j8DbE5yE3A/8KPALwKfgu9fR18N3JTkaeBZ4GaagXXr2prxJPcAt7XX3F8CbgeeAh5pa7YmeZBmlP5VNKfs1wAbHHEvSdL09A36UsqXk3wM+DTwy8CftV9/o1N2K3AUcDewBHgCOLeU8mqn5nrgTZofFo4CHgUuK6Xs7dRcDNzJvtH564Frp75bkiQJBjuip5TyReCLB1hegFXt9HY1r9M8UGflAWpeBi4ZZJskSVJ/PutekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkivUN+iSrkpSeaVtnedqaF5PsSfJYkpN61nFEkruS7EqyO8n6JMt6apYkWZtkvJ3WJlk8vF2VJGn+GfSI/hng3Z3pg51lNwI3ACuB04EdwMYkCzs1q4ELgAuBs4BjgQ1JFnRq1gGnAOcD57Wv105xfyRJUsdhA9a9WUrZ1jszSYDrgFtKKQ+08y6nCfuLgDVJFgFXAleUUja2NZcCzwNnAw8lWU4T7meWUja3NVcDX0pyYinlmXeyk5IkzVeDHtG/L8m3kjyX5L4k72vnnwCMAA9PFJZS9gCPA2e0s04FDu+peQHY2qlZAbwGbO58z03A7k6NJEmaokGC/gng4zSn1K+iCfbNSf5a+xpge89ntneWjQB7gV19anaWUsrEwvb1jk6NJEmaor6n7kspv999n+SPgG8ClwN/NFHW87FMMq9Xb81k9X3XMzY21ufbDOa0oazl4BrWvh4Kc2lbZzt7OVz2c3js5fAM0svR0dFprXvQa/TfV0p5LcmfAKPA59vZI8ALnbKl7DvK3wYsAI4DdvbUPN6pWZokE0f17fX/49n/bMFbTHfH56K5sq9jY2NzZltnO3s5XPZzeOzl8BzsXk75PvokRwI/DHwbeI4mpM/pWX4W+663Pwm80VOzDFjeqdkCHENzrX7CCuBo3nrdXpIkTUHfI/okvw58AfgzmqPwX6YJ4HtLKSXJauCmJE8DzwI30wysWwdQShlPcg9wW5IdwEvA7cBTwCNtzdYkD9KM0r+K5pT9GmCDI+4lSZq+QU7dLwP+M/tOvf8R8GOllOfb5bcCRwF3A0toBu+dW0p5tbOO64E3gfvb2keBy0opezs1FwN3sm90/nrg2mnskyRJag0yGO/n+iwvwKp2erua12keqLPyADUvA5f02x5JkjQ4n3UvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqNuWgT/KpJCXJZzvzkmRVkheT7EnyWJKTej53RJK7kuxKsjvJ+iTLemqWJFmbZLyd1iZZPP3dkyRpfptS0Cf5MeAq4KmeRTcCNwArgdOBHcDGJAs7NauBC4ALgbOAY4ENSRZ0atYBpwDnA+e1r9dOZRslSdI+Awd9kkXAfwKuBL7TmR/gOuCWUsoDpZSvA5cDC4GLOp+9EvgXpZSNpZT/BVwK/AhwdluznCbc/0kpZXMpZQtwNfD3kpz4jvdUkqR5aCpH9L8J/G4p5Q965p8AjAAPT8wopewBHgfOaGedChzeU/MCsLVTswJ4DdjcWfcmYHenRpIkTcFhgxQluQp4P81ReK+R9uv2nvnbgfd0avYCuyapGenU7CyllImFpZSSZEenZj9jY2OD7EJfpw1lLQfXsPb1UJhL2zrb2cvhsp/DYy+HZ5Bejo6OTmvdfYO+PW3+aeCsUsr3DlBaet5nknn7rb6nZrL6A65nujs+F82VfR0bG5sz2zrb2cvhsp/DYy+H52D3cpBT9yuA44CvJ3kzyZvAh4Br2tcvtXW9R91L2XeUvw1Y0K7nQDVL22v+wPev/x/P/mcLJEnSAAYJ+s8DHwRO7kxfAe5rXz9LE9LnTHwgyZE0I+snrrc/CbzRU7MMWN6p2QIcQ/ODxYQVwNG89bq9JEkaUN9T96WUV4BXuvOS7AZebkfYk2Q1cFOSp2mC/2aagXXr2nWMJ7kHuK295v4ScDvNbXqPtDVbkzwIrGnHBARYA2wopTwzjJ2VJGm+GWgw3gBuBY4C7gaWAE8A55ZSXu3UXA+8Cdzf1j4KXFZK2dupuRi4k32j89cD1w5pGyVJmnemFfSllA/3vC/AqnZ6u8+8TvNAnZUHqHkZuGQ62yRJkvbns+4lSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFVsWPfR6xBYtHjxTG9CX+OvvNK/SJJ0yHhEL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIq1jfok3wiyVNJvttOW5J8tLM8SVYleTHJniSPJTmpZx1HJLkrya4ku5OsT7Ksp2ZJkrVJxttpbZLFw9tVSZLmn0GO6P8c+CXgFOA04A+Azyf5kXb5jcANwErgdGAHsDHJws46VgMXABcCZwHHAhuSLOjUrGu/x/nAee3rtdPbLUmSBHBYv4JSyn/rmXVTkl8AViT5GnAdcEsp5QGAJJfThP1FwJoki4ArgStKKRvbmkuB54GzgYeSLKcJ9zNLKZvbmquBLyU5sZTyzBD2VZKkeWdK1+iTLEjyc8AxwGbgBGAEeHiippSyB3gcOKOddSpweE/NC8DWTs0K4LV2nRM2Abs7NZIkaYr6HtEDJPkgsAU4kiaQf6aU8rUkEyG8vecj24H3tK9HgL3ArklqRjo1O0spZWJhKaUk2dGpkSRJUzRQ0APPACcDi2mutd+b5MOd5aWnPpPM69VbM1l93/WMjY31+TaDOW0oa9HEn8ew/lxkL4fNfg6PvRyeQXo5Ojo6rXUPFPSllO8B/6d9+5UkpwPXA7/azhsBXuh8ZCn7jvK3AQuA44CdPTWPd2qWJsnEUX2SAMez/9mCt5jujuvgGB0dZWxszD+XIbGXw2U/h8deDs/B7uV076N/F3AE8BxNSJ8zsSDJkTQj6yeutz8JvNFTswxY3qnZQnPdf0Xne6wAjuat1+0lSdIU9D2iT3IL8EWaI/aFNKPpPwx8tL2OvppmJP7TwLPAzTTX8dcBlFLGk9wD3NZec38JuB14Cnikrdma5EGaUfpX0ZyyXwNscMS9JEnTN8ip+xHgt9uv4zQBfX4p5aF2+a3AUcDdwBLgCeDcUsqrnXVcD7wJ3N/WPgpcVkrZ26m5GLiTfaPz1wPXTmOfJElSa5D76D/eZ3kBVrXT29W8TvNAnZUHqHkZuKTf9kiSpMH5rHtJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqNuivqZUGsmjx4ln9K3/HX3llpjdBkg4pj+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYga9JEkV6xv0ST6Z5MtJvptkZ5IvJPlAT02SrEryYpI9SR5LclJPzRFJ7kqyK8nuJOuTLOupWZJkbZLxdlqbZPFwdlWSpPlnkCP6DwO/AZwBfAR4E3gkyQ90am4EbgBWAqcDO4CNSRZ2alYDFwAXAmcBxwIbkizo1KwDTgHOB85rX6+d8l5JkiQADutXUEr5ye77JJcC48CPA19IEuA64JZSygNtzeU0YX8RsCbJIuBK4IpSysbOep4HzgYeSrKcJtzPLKVsbmuuBr6U5MRSyjPD2GFJkuaT6VyjX9h+7jvt+xOAEeDhiYJSyh7gcZqzAACnAof31LwAbO3UrABeAzZ3vtcmYHenRpIkTUHfI/pJ3AF8FdjSvh9pv27vqdsOvKdTsxfYNUnNSKdmZymlTCwspZQkOzo1+xkbG5vq9k/qtKGsRbPdsP6+HEpzcZtnM/s5PPZyeAbp5ejo6LTWPaWgT3I7cCbN6fW9PYtLb/kk8/ZbZU/NZPUHXM90d1zz01z7+zI2Njbntnk2s5/DYy+H52D3cuBT90k+QzOQ7iOllG92Fm1rv/YedS9l31H+NmABcFyfmqXtNf+J7xngePY/WyBJkgYwUNAnuYNmYN1HSilP9yx+jiakz+nUH0kzsn7ievuTwBs9NcuA5Z2aLcAxNNfqJ6wAjuat1+0lSdKA+p66T3I3cCnwMeA7SSaO3F8rpbzWXkdfDdyU5GngWeBmmoF16wBKKeNJ7gFua6+5vwTcDjwFPNLWbE3yIM0o/atoTtmvATY44l6SpOkZ5Br9Ne3XR3vm/wqwqn19K3AUcDewBHgCOLeU8mqn/nqae/Dvb2sfBS7rudZ/MXAn+0bnrweuHWRHJEnS/ga5jz4D1BSa0F91gJrXaR6os/IANS8Dl/T7fpIkaTA+616SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUsen8mlppzlq0ePFMb0Jf46+8MtObIKkiHtFLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLFDHpJkipm0EuSVDGDXpKkihn0kiRVzKCXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqNlDQJ/mJJOuTfCtJSfLxnuVJsirJi0n2JHksyUk9NUckuSvJriS72/Ut66lZkmRtkvF2Wptk8TveS0mS5qlBj+iPAb4O/DNgzyTLbwRuAFYCpwM7gI1JFnZqVgMXABcCZwHHAhuSLOjUrANOAc4Hzmtfrx10ZyRJ0lsdNkhRKeX3gN8DSPK57rIkAa4DbimlPNDOu5wm7C8C1iRZBFwJXFFK2djWXAo8D5wNPJRkOU24n1lK2dzWXA18KcmJpZRn3uG+SpI07wzjGv0JwAjw8MSMUsoe4HHgjHbWqcDhPTUvAFs7NSuA14DNnXVvAnZ3aiRJ0hQMI+hH2q/be+Zv7ywbAfYCu/rU7CyllImF7esdnRpJkjQFA526H1DpeZ9J5vXqrZms/oDrGRsbG2jj+jltKGuR3rlFi/eNP52Nfy+/8uUvz/QmvCPD+j9D9nKYBunl6OjotNY9jKDf1n4dAV7ozF/KvqP8bcAC4DhgZ0/N452apUkycVTfXv8/nv3PFnzfdHdc0vTM5X9zY2Njc3r7ZxN7OTwHu5fDOHX/HE1InzMxI8mRNCPrJ663Pwm80VOzDFjeqdlCM7p/RWfdK4Cjeet1e0mSNKCBjuiTHAO8v337LuC9SU4GXi6l/FmS1cBNSZ4GngVuphlYtw6glDKe5B7gtiQ7gJeA24GngEfamq1JHqQZpX8VzSn7NcAGR9xLkjQ9g566Pw34w877X2mne4GPA7cCRwF3A0uAJ4BzSymvdj5zPfAmcH9b+yhwWSllb6fmYuBO9o3OXw9cO/juSJKkrkHvo3+M5gj77ZYXYFU7vV3N6zQP1Fl5gJqXgUsG2SZJktSfz7qXJKliBr0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcyglySpYsP8NbWS5oHur9GdrcZfeWWmN0GaNTyilySpYga9JEkVM+glSaqYQS9JUsUMekmSKmbQS5JUMYNekqSKGfSSJFXMoJckqWIGvSRJFTPoJUmqmEEvSVLF/KU2kqrzdr9457RDvB0H4i/e0aHiEb0kSRUz6CVJqphBL0lSxQx6SZIqZtBLklQxg16SpIoZ9JIkVcz76CVpBrzdvf6zhff518MjekmSKuYRvSRpP/3OOMyGpwx61mEwHtFLklSxWRn0Sa5J8lyS15M8meSsmd4mSZLmolkX9El+FrgD+DTwo8Bm4PeTvHdGN0ySpDlo1gU98M+Bz5VSfquUsrWUshL4NvALM7xdkiTNObNqMF6SvwKcCvx6z6KHgTMO5vd2UIckaSaMjo4e1PXPtiP644AFwPae+duBkUO/OZIkzW2zLegnlJ73mWSeJEnqY1adugd2AXvZ/+h9KZ2j/EWLFuVQbpQkSXPVrDqiL6V8D3gSOKdn0Tk0o+8lSdIUzLYjeoDbgbVJ/iewCfinwF8H/v2MbpUkSXPQrDqiByil3A9cB9wMfBU4E/ipUsrzB+t7+oCeA0vyySRfTvLdJDuTfCHJB3pqkmRVkheT7EnyWJKTemqOSHJXkl1JdidZn2TZod2b2SXJp5KUJJ/tzLOXA0ry7iT3tn8vX0/yjSQf6iy3lwNIsiDJv+r8P/hckn+d5LBOjb18G0l+ot3Xb7X/nj/es3wovUuyJMnaJOPttDZJ/9+OVEqZ1xPws8AbwFXAcuAu4DXgvTO9bbNlAh4CrgA+AHwQ+K/ANuAHOjW/BLwKXNDW/Q7wIrCwU/Pv2nnnAKcAj9H8MLdgpvdxhvr6Y8BzwP8GPmsvp9y/xcA3gf8I/G3gBODvAsvt5ZR7+SngZeCngb8J/H3gO8Av28uB+vdTNA95+0fAXwAf71k+lN4Bvw/8Cc3t5iva11/ou30z3aCZnoAngN/qmTcG/JuZ3rbZOgHH0Aya/On2fWgeanRTp+ao9i/21e37RcD3gIs7NT8I/D/gJ2d6n2agh4uA/wt8pP0H/Vl7OeUefhrYdIDl9nLwXm4A7u2Zdy+wwV5OuZevdYN+WL2jORAtwI93as5s5514oG2adafuD6XOA3oe7ll00B/QM8ctpLns8532/Qk0d0p8v4+llD3A4+zr46nA4T01LwBbmZ+9/k3gd0spf9Az314O7mPAE0nuT7IjyVeTXJtk4q4cezm4/wH8nSQ/DJDkb9H8EPp77XJ7OX3D6t0Kmh8iugPTNwG76dPf2TgY71A60AN6zj70mzNn3EFzSmlL+37idsjJ+vieTs1emlsoe2vm1cOQklwFvB+4dJLF9nJw7wOuAT4D3AKcTHPpDeCz2Mup+DWaH+C/kWQvTTb8ainlN9rl9nL6htW7EWBnaQ/lAUopJckO+vR3vgf9BB/QM6Akt9OcLjqzlLK3Z/F0+jivep3kRJpTzmeV5nbSt2Mv+3sX8JVSyifb93+cZBT4BE3QT7CX/f0scBlwEc1135OBO5I8V0q5p1NnL6dvGL2brL7veub1qXsGfECPGkk+A1wIfKSU8s3Oom3t1wP1cRvN2ZPjDlAzH6yg6cHXk7yZ5E3gQ8A17euX2jp72d+3gW/0zNsKTPymS/9eDu424NdLKfeVUr5WSllLc6vzxA9R9nL6htW7bcDSzqUp2tfH06e/8zroiw/oGViSO2h+2v9IKeXpnsXP0fwlPKdTfyRwFvv6+CTN3Q3dmmU0A0zmU68/T3Pnwsmd6SvAfe3rZ7GXg9oEnNgz74eAiVtx/Xs5uL9Kc9DTtZd9GWEvp29YvdtCMxB6RWfdK4Cj6dffmR6hONMTzSmr7wE/3zb1DpoBD39jprdttkzA3cB3aQbnjHSmYzo1v9TW/EOa20fuY/LbR75FM/7hR4E/ZB7cejNAfx9j/9vr7GX/vp3e/ud4E82Yh38MjAOfsJdT7uXngD8HPkpze93PADuBf2svB+rfMez7wf0vgH/Zvn7vMHtHc3vd12huzV3Rvvb2ugH/kK4B/hT4S5qfrH5iprdpNk00138mm1Z1agKsojmd+jrw34EP9KznSJrBUi+1/xi+APzgTO/fTE/sH/T2cvDefZTmOQSv05wN+UUg9nLKfVwIrKY5G7KH5vkEnwaOtJcD9e/Db/N/5OeG2TvgB4Dfpvmh4bvt68X9ti/thyVJUoXm9TV6SZJqZ9BLklQxg16SpIoZ9JIkVcyglySpYga9JEkVM+glSaqYQS9JUsUMekmSKvb/AQsfejOO+fcLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Make Histogram of `cnt` column\n",
    "\n",
    "fig = plt.figure(figsize=(8, 6))\n",
    "plt.style.use('fivethirtyeight')\n",
    "plt.hist(bike_rentals[\"cnt\"], color='red')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant       0.278379\n",
       "season        0.178056\n",
       "yr            0.250495\n",
       "mnth          0.120638\n",
       "hr            0.394071\n",
       "holiday      -0.030927\n",
       "weekday       0.026900\n",
       "workingday    0.030284\n",
       "weathersit   -0.142426\n",
       "temp          0.404772\n",
       "atemp         0.400929\n",
       "hum          -0.322911\n",
       "windspeed     0.093234\n",
       "casual        0.694564\n",
       "registered    0.972151\n",
       "cnt           1.000000\n",
       "Name: cnt, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Determine correlation between `cnt` and various features\n",
    "# Using corr() automatically excludes non-numeric columns\n",
    "\n",
    "bike_rentals.corr()['cnt']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating Features\n",
    "\n",
    "It will be helpful to calculate features before applying machine learning models. Features can enhance the accuracy of models by introducing new information, or distilling existing information.\n",
    "\n",
    "The `hr` column in `bike_rentals`, for instance, contains the hours during which bikes are rented, from 1 to 24. A machine will treat each hour differently, without understanding that certain hours are related. We can introduce some order into the process by creating a new column with labels for morning, afternoon, evening, and night. This will bundle similar times together, enabling the model to make better decisions.\n",
    "\n",
    "Below, we will:\n",
    "Write a function called `assign_label` that takes in a numeric value for an hour, and returns:\n",
    "- `1` if the hour is from `6` to `12`\n",
    "- `2` if the hour is from `12` to `18`\n",
    "- `3` if the hour is from `18` to `24`\n",
    "- `4` if the hour is from `0` to `6`\n",
    "Use the `apply` method on series objects to apply the function to each item in the hr column.\n",
    "Assign the result to the `time_label` column of bike_rentals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def assign_label(hour):\n",
    "    ''' 1 = morning; 2 = afternoon; 3 = evening; 4 = night'''\n",
    "    if hour >=0 and hour < 6:\n",
    "        return 4\n",
    "    elif hour >=6 and hour < 12:\n",
    "        return 1\n",
    "    elif hour >= 12 and hour < 18:\n",
    "        return 2\n",
    "    elif hour >= 18 and hour <=24:\n",
    "        return 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply function\n",
    "bike_rentals[\"time_label\"] = bike_rentals[\"hr\"].apply(assign_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting Dataset into training and testing sets\n",
    "We will:\n",
    "- Select 80% of the rows in `bike_rentals` to be part of the training set using the `sample` method on `bike_rentals` and assign the result to `train`.\n",
    "- Select the rows that are in `bike_rentals` but not in `train` to be in the testing set and assign the result to `test`.\n",
    "\n",
    "We will use the function isin() to separate the two sets:\n",
    "- This line will generate a Boolean series that's False when a row in `bike_rentals` isn't found in `train`: `bike_rentals.index.isin(train.index)`\n",
    "- This line will select any rows in `bike_rentals` that aren't found in `train` to be in the testing set: `bike_rentals.loc[~bike_rentals.index.isin(train.index)]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = bike_rentals.sample(frac=.8)\n",
    "test = bike_rentals.loc[~bike_rentals.index.isin(train.index)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Error Metric\n",
    "We will use mean squared error (MSE) as the error metric, which works well for continuous variables and fits our data well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying Linear Regression\n",
    "\n",
    "Linear regression should work fairly well on this data, given that many of the columns are highly correlated with `cnt`.And linear regression works best when predictors are linearly correlated to the target and also independent -- in other words, they don't change meaning when we combine them with each other. The good thing about linear regression is that it's fairly resistant to overfitting because it's straightforward. It also can be prone to underfitting the data, however, and not building a powerful enough model. This means that linear regression usually isn't the most accurate option.\n",
    "\n",
    "We'll ignore the `casual` and `registered` columns because `cnt` is derived from them. We will also exclude `dteday` column as it is not a numeric column and also makes the data overly repetitious as each day is treated differently from the other. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove unwanted columns from the dataset\n",
    "predictors = list(train.columns)\n",
    "predictors.remove(\"cnt\")\n",
    "predictors.remove(\"casual\")\n",
    "predictors.remove(\"registered\")\n",
    "predictors.remove(\"dteday\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply Linear Regression\n",
    "from sklearn.linear_model import LinearRegression\n",
    "rntls = LinearRegression()\n",
    "\n",
    "rntls.fit(train[predictors], train[\"cnt\"])\n",
    "\n",
    "predictions = rntls.predict(test[predictors])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17910.058739423686"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "lr_mse = mean_squared_error(predictions, test['cnt'])\n",
    "lr_mse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Regression Error\n",
    "The error is very high, which may be due to the fact that the data has a few extremely high rental counts, but otherwise mostly low counts. Larger errors are penalized more with MSE, which leads to a higher total error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying Decision Trees\n",
    "Decision trees tend to predict outcomes much more reliably than linear regression models. Because a decision tree is a fairly complex model, it also tends to overfit, particularly when we don't tweak parameters like maximum depth and minimum number of samples per leaf. Decision trees are also prone to instability -- small changes in the input data can result in a very different output model.\n",
    "\n",
    "We can experiment with various parameters such as `depth` and `min_samples_leaf`. For this project, we will use `min_samples_leaf` = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predict for test set using decision trees algorithm\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "rntls = DecisionTreeRegressor(min_samples_leaf=5)\n",
    "\n",
    "rntls.fit(train[predictors], train[\"cnt\"])\n",
    "\n",
    "predictions = rntls.predict(test[predictors])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2472.8224087434137"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Error metrics\n",
    "dt_mse = mean_squared_error(predictions, test['cnt'])\n",
    "dt_mse\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decion Tree Error\n",
    "Decision tree error\n",
    "By taking the nonlinear predictors into account, the decision tree regressor appears to have much higher accuracy than linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying Random Forests\n",
    "\n",
    "now apply the random forest algorithm, which improves on the decision tree algorithm. Random forests tend to be much more accurate than simple models like linear regression. Due to the way random forests are constructed, they tend to overfit much less than decision trees. Random forests can still be prone to overfitting, though, so it's important to tune parameters like maximum depth and minimum samples per leaf.\n",
    "\n",
    "We can experiment with various parameters such as `depth` and `min_samples_leaf`. For this project, we will use `min_samples_leaf` = 5, and keep `n_estimators` at the default of `100` trees. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "rntls = RandomForestRegressor(n_estimators = 100, min_samples_leaf=5)\n",
    "rntls.fit(train[predictors], train[\"cnt\"])\n",
    "predictions = rntls.predict(test[predictors])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1602.2194476790592"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Error metrics\n",
    "rf_mse = mean_squared_error(predictions, test['cnt'])\n",
    "rf_mse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forests Error\n",
    "Random Forests error is a considerable improvement over both linear regression as well as the single decision tree as it gets rid of some of the sources of overfitting. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}