05-DSA-Cap09-ModeloGradientBoosting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='blue'>Data Science Academy - Machine Learning</font>\n",
    "\n",
    "# <font color='blue'>Capítulo 9 - Gradient Boosting</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gradient Boosting ou Gradient Boostted Regression Trees (GBRT) é uma técnica de aprendizagem estatística não-paramétrica usada para problemas de classificação e regressão. \n",
    "\n",
    "Gradient Boosting = Gradient Descent + Boosting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://deeplearningbook.com.br/aprendizado-com-a-descida-do-gradiente/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basicamente 3 etapas são realizadas na construção do modelo:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1- Gera um regressor\n",
    "\n",
    "2- Computa o erro residual\n",
    "\n",
    "3- Aprende a prever o resíduo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"imagens/GBRT1.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image\n",
    "Image(url = 'imagens/GBRT1.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"imagens/GBRT2.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image\n",
    "Image(url = 'imagens/GBRT2.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Como funciona o Gradient Boosting:\n",
    "\n",
    "\n",
    "1 - Realiza um conjunto de previsões (y)\n",
    "\n",
    "2- Calcula o erro das previsões (j)\n",
    "\n",
    "3- Tenta ajustar y reduzindo o erro (através de alpha)\n",
    "\n",
    "4- Para cada estimador base, é estimado o gradiente da função de perda\n",
    "\n",
    "5- Estimadores subsequentes estimam o erro residual dos estimadores anteriores\n",
    "\n",
    "6- Aplica o Gradient Descent para reduzir j\n",
    "\n",
    "7- Soma os resultados dos estimadores, dando peso a cada passo de acordo com o valor de alfa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://scikit-learn.org/stable/modules/ensemble.html#gradient-boosting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Boosting Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9117"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Imports\n",
    "from sklearn.datasets import make_hastie_10_2\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "\n",
    "# Define dados para x e y\n",
    "X, y = make_hastie_10_2(random_state = 0)\n",
    "X_train, X_test = X[:2000], X[2000:]\n",
    "y_train, y_test = y[:2000], y[2000:]\n",
    "\n",
    "# Cria o classificador\n",
    "clf = GradientBoostingClassifier(n_estimators = 100, learning_rate = 1.0, random_state = 0)\n",
    "\n",
    "# Treina o classificador\n",
    "clf.fit(X_train, y_train)\n",
    "\n",
    "# Calcula a acurácia (score)\n",
    "clf.score(X_test, y_test)                 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Boosting Regressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.009154859960319"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Imports\n",
    "import numpy as np\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.datasets import make_friedman1\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "\n",
    "# Define dados para x e y\n",
    "X, y = make_friedman1(n_samples = 1200, random_state = 0, noise = 1.0)\n",
    "X_train, X_test = X[:200], X[200:]\n",
    "y_train, y_test = y[:200], y[200:]\n",
    "\n",
    "# Cria o regressor\n",
    "est = GradientBoostingRegressor(n_estimators = 100, learning_rate = 0.1, max_depth = 1, random_state = 0, loss = 'ls')\n",
    "\n",
    "# Treina o regressor\n",
    "est.fit(X_train, y_train)\n",
    "\n",
    "# Calcula o erro médio ao quadrado\n",
    "mean_squared_error(y_test, est.predict(X_test))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Obrigado - Data Science Academy - <a href=\"http://facebook.com/dsacademybr\">facebook.com/dsacademybr</a>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}