1002_hyperparameter_tuning_cv_Search.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "1002-hyperparameter_tuning_cv Search.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/parekhakhil/pyImageSearch/blob/main/1002_hyperparameter_tuning_cv_Search.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8ntZ1AkXZIxY"
      },
      "source": [
        "\n",
        "\n",
        "This notebook is associated with the [Grid search hyperparameter tuning with scikit-learn ( GridSearchCV )](https://www.pyimagesearch.com/2021/05/24/grid-search-hyperparameter-tuning-with-scikit-learn-gridsearchcv/) blog post published on 2021-05-24.\n",
        "\n",
        "Only the code for the blog post is here. Most codeblocks have a 1:1 relationship with what you find in the blog post with two exceptions: (1) Python classes are not separate files as they are typically organized with PyImageSearch projects, and (2) Command Line Argument parsing is replaced with an `args` dictionary that you can manipulate as needed.\n",
        "\n",
        "We recommend that you execute (press ▶️) the code block-by-block, as-is, before adjusting parameters and `args` inputs. Once you've verified that the code is working, you are welcome to hack with it and learn from manipulating inputs, settings, and parameters. For more information on using Jupyter and Colab, please refer to these resources:\n",
        "\n",
        "*   [Jupyter Notebook User Interface](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html#notebook-user-interface)\n",
        "*   [Overview of Google Colaboratory Features](https://colab.research.google.com/notebooks/basic_features_overview.ipynb)\n",
        "\n",
        "\n",
        "Happy hacking!\n",
        "\n",
        "\n",
        "<hr>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NFhAzQB3aNMa"
      },
      "source": [
        "### Download the code zip file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7y0LG1EuaRlB"
      },
      "source": [
        "!wget https://pyimagesearch-code-downloads.s3-us-west-2.amazonaws.com/hyperparameter-tuning-cv/hyperparameter-tuning-cv.zip\n",
        "!unzip -qq hyperparameter-tuning-cv.zip\n",
        "%cd hyperparameter-tuning-cv"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_SgTVT3HagGZ"
      },
      "source": [
        "## Blog Post Code"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wcrOk6pURp50"
      },
      "source": [
        "### Import Packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VJaCNlDDRz6d"
      },
      "source": [
        "# import the necessary packages\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.model_selection import train_test_split\n",
        "from skimage import feature\n",
        "from imutils import paths\n",
        "import numpy as np\n",
        "import time\n",
        "import cv2\n",
        "import os"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZ3bW6aU2a6v"
      },
      "source": [
        "### Our Local Binary Pattern (LBP) descriptor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bS9nQNo02dIs"
      },
      "source": [
        "class LocalBinaryPatterns:\n",
        "\tdef __init__(self, numPoints, radius):\n",
        "\t\t# store the number of points and radius\n",
        "\t\tself.numPoints = numPoints\n",
        "\t\tself.radius = radius\n",
        "\n",
        "\tdef describe(self, image, eps=1e-7):\n",
        "\t\t# compute the Local Binary Pattern representation\n",
        "\t\t# of the image, and then use the LBP representation\n",
        "\t\t# to build the histogram of patterns\n",
        "\t\tlbp = feature.local_binary_pattern(image, self.numPoints,\n",
        "\t\t\tself.radius, method=\"uniform\")\n",
        "\t\t(hist, _) = np.histogram(lbp.ravel(),\n",
        "\t\t\tbins=np.arange(0, self.numPoints + 3),\n",
        "\t\t\trange=(0, self.numPoints + 2))\n",
        "\n",
        "\t\t# normalize the histogram\n",
        "\t\thist = hist.astype(\"float\")\n",
        "\t\thist /= (hist.sum() + eps)\n",
        "\n",
        "\t\t# return the histogram of Local Binary Patterns\n",
        "\t\treturn hist"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jppw5-Bd56H-"
      },
      "source": [
        "### Implementing our grid search for hyperparameter tuning"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "okM7Bpyeq8Kc"
      },
      "source": [
        "# # construct the argument parser and parse the arguments\n",
        "# ap = argparse.ArgumentParser()\n",
        "# ap.add_argument(\"-d\", \"--dataset\", required=True,\n",
        "# \thelp=\"path to input dataset\")\n",
        "# args = vars(ap.parse_args())\n",
        "\n",
        "# since we are using Jupyter Notebooks we can replace our argument\n",
        "# parsing code with *hard coded* arguments and values\n",
        "args = {\n",
        "\t\"dataset\": \"texture_dataset\"\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J43Rixrm3Duo"
      },
      "source": [
        "# grab the image paths in the input dataset directory\n",
        "imagePaths = list(paths.list_images(args[\"dataset\"]))\n",
        "\n",
        "# initialize the local binary patterns descriptor along with\n",
        "# the data and label lists\n",
        "print(\"[INFO] extracting features...\")\n",
        "desc = LocalBinaryPatterns(24, 8)\n",
        "data = []\n",
        "labels = []"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zD54-goR3IJV"
      },
      "source": [
        "# loop over the dataset of images\n",
        "for imagePath in imagePaths:\n",
        "\t# load the image, convert it to grayscale, and quantify it\n",
        "\t# using LBPs\n",
        "\timage = cv2.imread(imagePath)\n",
        "\tgray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
        "\thist = desc.describe(gray)\n",
        "\n",
        "\t# extract the label from the image path, then update the\n",
        "\t# label and data lists\n",
        "\tlabels.append(imagePath.split(os.path.sep)[-2])\n",
        "\tdata.append(hist)\n",
        "\n",
        "# partition the data into training and testing splits using 75% of\n",
        "# the data for training and the remaining 25% for testing\n",
        "print(\"[INFO] constructing training/testing split...\")\n",
        "(trainX, testX, trainY, testY) = train_test_split(data, labels,\n",
        "\trandom_state=22, test_size=0.25)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_A3Hycjs3NkY"
      },
      "source": [
        "# construct the set of hyperparameters to tune\n",
        "parameters = [\n",
        "\t{\"kernel\":\n",
        "\t\t[\"linear\"],\n",
        "\t\t\"C\": [0.0001, 0.001, 0.1, 1, 10, 100, 1000]},\n",
        "\t{\"kernel\":\n",
        "\t\t[\"poly\"],\n",
        "\t\t\"degree\": [2, 3, 4],\n",
        "\t\t\"C\": [0.0001, 0.001, 0.1, 1, 10, 100, 1000]},\n",
        "\t{\"kernel\":\n",
        "\t\t[\"rbf\"],\n",
        "\t\t\"gamma\": [\"auto\", \"scale\"],\n",
        "\t\t\"C\": [0.0001, 0.001, 0.1, 1, 10, 100, 1000]}\n",
        "]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E1iCNr2K3WEI"
      },
      "source": [
        "# tune the hyperparameters via a cross-validated grid search\n",
        "print(\"[INFO] tuning hyperparameters via grid search\")\n",
        "grid = GridSearchCV(estimator=SVC(), param_grid=parameters, n_jobs=-1)\n",
        "start = time.time()\n",
        "grid.fit(trainX, trainY)\n",
        "end = time.time()\n",
        "\n",
        "# show the grid search information\n",
        "print(\"[INFO] grid search took {:.2f} seconds\".format(\n",
        "\tend - start))\n",
        "print(\"[INFO] grid search best score: {:.2f}%\".format(\n",
        "\tgrid.best_score_ * 100))\n",
        "print(\"[INFO] grid search best parameters: {}\".format(\n",
        "\tgrid.best_params_))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NbkALIBP3Z-p"
      },
      "source": [
        "# grab the best model and evaluate it\n",
        "print(\"[INFO] evaluating...\")\n",
        "model = grid.best_estimator_\n",
        "predictions = model.predict(testX)\n",
        "print(classification_report(testY, predictions))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4ogkNauArL6u"
      },
      "source": [
        "For a detailed walkthrough of the concepts and code, be sure to refer to the full tutorial, [*Grid search hyperparameter tuning with scikit-learn ( GridSearchCV )*](https://www.pyimagesearch.com/2021/05/24/grid-search-hyperparameter-tuning-with-scikit-learn-gridsearchcv/) published on 2021-05-24."
      ]
    }
  ]
}