801_perceptron_neural_network.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "801-perceptron_neural_network.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/parekhakhil/pyImageSearch/blob/main/801_perceptron_neural_network.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j4RpK9pawQzP"
      },
      "source": [
        "# Implementing the Perceptron Neural Network with Python\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8ntZ1AkXZIxY"
      },
      "source": [
        "\n",
        "\n",
        "This notebook is associated with the [Implementing the Perceptron Neural Network with Python](https://www.pyimagesearch.com/2021/05/06/implementing-the-perceptron-neural-network-with-python/) blog post published on 2021-05-06.\n",
        "\n",
        "Only the code for the blog post is here. Most codeblocks have a 1:1 relationship with what you find in the blog post with two exceptions: (1) Python classes are not separate files as they are typically organized with PyImageSearch projects, and (2) Command Line Argument parsing is replaced with an `args` dictionary that you can manipulate as needed.\n",
        "\n",
        "We recommend that you execute (press ▶️) the code block-by-block, as-is, before adjusting parameters and `args` inputs. Once you've verified that the code is working, you are welcome to hack with it and learn from manipulating inputs, settings, and parameters. For more information on using Jupyter and Colab, please refer to these resources:\n",
        "\n",
        "*   [Jupyter Notebook User Interface](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html#notebook-user-interface)\n",
        "*   [Overview of Google Colaboratory Features](https://colab.research.google.com/notebooks/basic_features_overview.ipynb)\n",
        "\n",
        "\n",
        "Happy hacking!\n",
        "\n",
        "<hr>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NFhAzQB3aNMa"
      },
      "source": [
        "### Download the code zip file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7y0LG1EuaRlB"
      },
      "source": [
        "!wget https://pyimagesearch-code-downloads.s3-us-west-2.amazonaws.com/perceptron-neural-network/perceptron-neural-network.zip\n",
        "!unzip -qq perceptron-neural-network.zip\n",
        "%cd perceptron-neural-network"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_SgTVT3HagGZ"
      },
      "source": [
        "## Blog Post Code"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wcrOk6pURp50"
      },
      "source": [
        "### Import Packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VJaCNlDDRz6d"
      },
      "source": [
        "# import the necessary packages\n",
        "import numpy as np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBXf2pjTPi7C"
      },
      "source": [
        "### Implementing the Perceptron in Python"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lHIVGWIwPieH"
      },
      "source": [
        "class Perceptron:\n",
        "\tdef __init__(self, N, alpha=0.1):\n",
        "\t\t# initialize the weight matrix and store the learning rate\n",
        "\t\tself.W = np.random.randn(N + 1) / np.sqrt(N)\n",
        "\t\tself.alpha = alpha\n",
        "\n",
        "\tdef step(self, x):\n",
        "\t\t# apply the step function\n",
        "\t\treturn 1 if x > 0 else 0\n",
        "\n",
        "\tdef fit(self, X, y, epochs=10):\n",
        "\t\t# insert a column of 1's as the last entry in the feature\n",
        "\t\t# matrix -- this little trick allows us to treat the bias\n",
        "\t\t# as a trainable parameter within the weight matrix\n",
        "\t\tX = np.c_[X, np.ones((X.shape[0]))]\n",
        "\n",
        "\t\t# loop over the desired number of epochs\n",
        "\t\tfor epoch in np.arange(0, epochs):\n",
        "\t\t\t# loop over each individual data point\n",
        "\t\t\tfor (x, target) in zip(X, y):\n",
        "\t\t\t\t# take the dot product between the input features\n",
        "\t\t\t\t# and the weight matrix, then pass this value\n",
        "\t\t\t\t# through the step function to obtain the prediction\n",
        "\t\t\t\tp = self.step(np.dot(x, self.W))\n",
        "\n",
        "\t\t\t\t# only perform a weight update if our prediction\n",
        "\t\t\t\t# does not match the target\n",
        "\t\t\t\tif p != target:\n",
        "\t\t\t\t\t# determine the error\n",
        "\t\t\t\t\terror = p - target\n",
        "\n",
        "\t\t\t\t\t# update the weight matrix\n",
        "\t\t\t\t\tself.W += -self.alpha * error * x\n",
        "\n",
        "\tdef predict(self, X, addBias=True):\n",
        "\t\t# ensure our input is a matrix\n",
        "\t\tX = np.atleast_2d(X)\n",
        "\n",
        "\t\t# check to see if the bias column should be added\n",
        "\t\tif addBias:\n",
        "\t\t\t# insert a column of 1's as the last entry in the feature\n",
        "\t\t\t# matrix (bias)\n",
        "\t\t\tX = np.c_[X, np.ones((X.shape[0]))]\n",
        "\n",
        "\t\t# take the dot product between the input features and the\n",
        "\t\t# weight matrix, then pass the value through the step\n",
        "\t\t# function\n",
        "\t\treturn self.step(np.dot(X, self.W))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BiBcQuWWPrOM"
      },
      "source": [
        "### Evaluating the Perceptron on Bitwise Datasets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CgpGL440PmNB"
      },
      "source": [
        "# construct the OR dataset\n",
        "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n",
        "y = np.array([[0], [1], [1], [1]])\n",
        "\n",
        "# define our perceptron and train it\n",
        "print(\"[INFO] training perceptron...\")\n",
        "p = Perceptron(X.shape[1], alpha=0.1)\n",
        "p.fit(X, y, epochs=20)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bv0CVSwmPwep"
      },
      "source": [
        "# now that our perceptron is trained we can evaluate it\n",
        "print(\"[INFO] testing perceptron...\")\n",
        "\n",
        "# now that our network is trained, loop over the data points\n",
        "for (x, target) in zip(X, y):\n",
        "\t# make a prediction on the data point and display the result\n",
        "\t# to our console\n",
        "\tpred = p.predict(x)\n",
        "\tprint(\"[INFO] data={}, ground-truth={}, pred={}\".format(\n",
        "\t\tx, target[0], pred))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XoiD0LGbP6j9"
      },
      "source": [
        "# construct the AND dataset\n",
        "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n",
        "y = np.array([[0], [0], [0], [1]])\n",
        "\n",
        "# define our perceptron and train it\n",
        "print(\"[INFO] training perceptron...\")\n",
        "p = Perceptron(X.shape[1], alpha=0.1)\n",
        "p.fit(X, y, epochs=20)\n",
        "\n",
        "# now that our perceptron is trained we can evaluate it\n",
        "print(\"[INFO] testing perceptron...\")\n",
        "\n",
        "# now that our network is trained, loop over the data points\n",
        "for (x, target) in zip(X, y):\n",
        "\t# make a prediction on the data point and display the result\n",
        "\t# to our console\n",
        "\tpred = p.predict(x)\n",
        "\tprint(\"[INFO] data={}, ground-truth={}, pred={}\".format(\n",
        "\t\tx, target[0], pred))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lCBfCEp5P9kt"
      },
      "source": [
        "# construct the XOR dataset\n",
        "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n",
        "y = np.array([[0], [1], [1], [0]])\n",
        "\n",
        "# define our perceptron and train it\n",
        "print(\"[INFO] training perceptron...\")\n",
        "p = Perceptron(X.shape[1], alpha=0.1)\n",
        "p.fit(X, y, epochs=20)\n",
        "\n",
        "# now that our perceptron is trained we can evaluate it\n",
        "print(\"[INFO] testing perceptron...\")\n",
        "\n",
        "# now that our network is trained, loop over the data points\n",
        "for (x, target) in zip(X, y):\n",
        "\t# make a prediction on the data point and display the result\n",
        "\t# to our console\n",
        "\tpred = p.predict(x)\n",
        "\tprint(\"[INFO] data={}, ground-truth={}, pred={}\".format(\n",
        "\t\tx, target[0], pred))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4ogkNauArL6u"
      },
      "source": [
        "For a detailed walkthrough of the concepts and code, be sure to refer to the full tutorial, [*Implementing the Perceptron Neural Network with Python*](https://www.pyimagesearch.com/2021/05/06/implementing-the-perceptron-neural-network-with-python/) published on 2021-05-06."
      ]
    }
  ]
}