Learner Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 1: Introduction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 2: Dataset\n",
    "\n",
    "Real estate agent table:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|Area|Distance|Price|\n",
    "|---|---|---|\n",
    "|70|3|21200|\n",
    "|50|1|22010|\n",
    "|120|9|24305|\n",
    "|100|2|31500|\n",
    "\n",
    "You can write the relationship with a 2-variable linear equation:\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "y = b + w_1.x_1 + w_2.x_2\n",
    "\\end{equation}\n",
    "$\n",
    "\n",
    "In a vector form:\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "y = b + (w_1 w_2).\\binom{x_1}{x_2}\n",
    "\\end{equation}\n",
    "$\n",
    "\n",
    "Where\n",
    "$\n",
    "\\begin{equation}\n",
    "W = (w_1 w_2)\n",
    "\\end{equation}\n",
    "$\n",
    "and\n",
    "$\n",
    "\\begin{equation}\n",
    "X = \\binom{x_1}{x_2}\n",
    "\\end{equation}\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 3: Initialize Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The loss over **m** examples:\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "J = \\frac{1}{2m} \\sum_{i=1}^{m} (y - \\hat{y})^2\n",
    "\\end{equation}\n",
    "$\n",
    "\n",
    "The objective of the gradient descent algorithm is to minimize this loss value.\n",
    "\n",
    "Gradient Descent Objective is to \n",
    "$\n",
    "\\begin{equation}\n",
    "min(J)\n",
    "\\end{equation}\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 4: Forward Pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The gradient descent algorithm can be simplified in 4 steps:\n",
    "\n",
    "1. Get predictions y_hat for X with current values of W and b.\n",
    "2. Compute the loss between y and y_hat\n",
    "3. Find gradients of the loss with respect to parameters W and b\n",
    "4. Update the values of W and b by subtracting the gradient values obtained in the previous step\n",
    "\n",
    "Let's simplify our linear equation a bit more for an example:\n",
    "$\n",
    "\\begin{equation}\n",
    "y = wx\n",
    "\\end{equation}\n",
    "$\n",
    "\n",
    "Let's plot J as a function of w\n",
    "\n",
    "![Loss vs Param](JvsW.png)\n",
    "\n",
    "The gradients of loss with respect to w:\n",
    "\n",
    "\\begin{equation}\n",
    "\\frac{dJ}{dw} = \\frac{\\delta{J}}{\\delta{w}} = \\lim_{\\epsilon \\to 0} \\frac{J(w + \\epsilon) - J(w)}{\\epsilon}\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 5: Compute Loss\n",
    "\n",
    "The loss over **m** examples:\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "J = \\frac{1}{2m} \\sum_{i=1}^{m} (y - \\hat{y})^2\n",
    "\\end{equation}\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 6: Backward Pass\n",
    "\n",
    "The gradient of loss with respect to bias can be calculated with:\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "\\frac{dJ}{db} = \\frac{1}{m} \\sum_{i=1}^{m} (\\hat{y^{(i)}} - y^{(i)})\n",
    "\\end{equation}\n",
    "$\n",
    "\n",
    "$\n",
    "\\begin{equation}\n",
    "\\frac{dJ}{dW_j} = \\frac{1}{m} \\sum_{i=1}^{m} (\\hat{y^{(i)}} - y^{(i)}).x_j^{(i)}\n",
    "\\end{equation}\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 7: Update Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 8: Training Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task 9: Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}