03-autograd_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Traduction en français du notebook *03* du cours ***Deep Learning*** d'Alfredo Canziani, professeur assistant à la *New York University*  : \n",
    "https://github.com/Atcold/pytorch-Deep-Learning/blob/master/03-autograd_tutorial.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Autograd : différenciation automatique\n",
    "\n",
    "Le package ``autograd`` fournit une différenciation automatique pour toutes les opérations sur les tenseurs. Il s'agit d'un cadre défini par l'utilisateur, ce qui veut dire que votre toile de fond est\n",
    "défini par la façon dont votre code est exécuté, et que chaque itération peut être différent.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Créer un tenseur :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2.],\n",
      "        [3., 4.]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "# Créer un tenseur 2x2 avec des capacités d'accumulation de gradients\n",
    "x = torch.tensor([[1, 2], [3, 4]], requires_grad=True, dtype=torch.float32)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Effectuer une opération sur le tenseur :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.,  0.],\n",
      "        [ 1.,  2.]], grad_fn=<SubBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Déduire 2 de tous les éléments\n",
    "y = x - 2\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``y`` a été créé à la suite d'une opération, il a donc un ``grad_fn``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<SubBackward0 object at 0x000002F8E9782278>\n"
     ]
    }
   ],
   "source": [
    "print(y.grad_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Qu'est-ce qui se passe ici ?\n",
    "print(x.grad_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<SubBackward0 at 0x2f8e97820b8>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creusons un peu plus loin...\n",
    "y.grad_fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AccumulateGrad at 0x2f8e97826d8>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.grad_fn.next_functions[0][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 2.],\n",
       "        [3., 4.]], requires_grad=True)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.grad_fn.next_functions[0][0].variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 3.,  0.],\n",
      "        [ 3., 12.]], grad_fn=<MulBackward0>)\n",
      "tensor(4.5000, grad_fn=<MeanBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Faire plus d'opérations sur  y\n",
    "z = y * y * 3\n",
    "a = z.mean()  # moyenne\n",
    "\n",
    "print(z)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Visualisons le graphique de calcul ! (Alfredo tient à remercier ici @szagoruyko)\n",
    "from torchviz import make_dot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_dot(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradients\n",
    "\n",
    "Rétropropagons maintenant `out.backward()`. Cela équivaut à faire `out.backward(torch.tensor([1.0]))`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rétropropagation\n",
    "a.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Affichage des gradients $\\frac{\\text{d}a}{\\text{d}x}$.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.5000,  0.0000],\n",
      "        [ 1.5000,  3.0000]])\n"
     ]
    }
   ],
   "source": [
    "# Calculez-le à la main AVANT de l'exécuter\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vous pouvez faire beaucoup de choses avec autograd !\n",
    "> Avec une grande *flexibilité* vient une grande responsabilité"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([  12.5663, -995.8140,  188.6319], grad_fn=<MulBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Graphes dynamiques!\n",
    "x = torch.randn(3, requires_grad=True)\n",
    "\n",
    "y = x * 2\n",
    "i = 0\n",
    "while y.data.norm() < 1000:\n",
    "    y = y * 2\n",
    "    i += 1\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([5.1200e+01, 5.1200e+02, 5.1200e-02])\n"
     ]
    }
   ],
   "source": [
    "# # Si nous ne faisons pas le *backward* sur un scalaire, nous devons spécifier le *grad_output*\n",
    "gradients = torch.FloatTensor([0.1, 1.0, 0.0001])\n",
    "y.backward(gradients)\n",
    "\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8\n"
     ]
    }
   ],
   "source": [
    "# AVANT d'éxécuter la cellule, pouvez-vous dire ce qui sera affiché ?\n",
    "print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inférence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cette variable détermine l'étendue du tenseur en dessous\n",
    "n = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1.])\n",
      "tensor([1., 2., 3.])\n"
     ]
    }
   ],
   "source": [
    "# x et w permettent l'accumulation du gradient\n",
    "x = torch.arange(1., n + 1, requires_grad=True)\n",
    "w = torch.ones(n, requires_grad=True)\n",
    "z = w @ x\n",
    "z.backward()\n",
    "print(x.grad, w.grad, sep='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n",
      "tensor([1., 2., 3.])\n"
     ]
    }
   ],
   "source": [
    "# Seulement w permet l'accumulation du gradient\n",
    "x = torch.arange(1., n + 1)\n",
    "w = torch.ones(n, requires_grad=True)\n",
    "z = w @ x\n",
    "z.backward()\n",
    "print(x.grad, w.grad, sep='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RuntimeError!!! >:[\n",
      "element 0 of tensors does not require grad and does not have a grad_fn\n"
     ]
    }
   ],
   "source": [
    "x = torch.arange(1., n + 1)\n",
    "w = torch.ones(n, requires_grad=True)\n",
    "\n",
    "# Indépendamment de ce que vous faites dans ce contexte, tous les tenseurs de torch n'auront pas d'accumulation de gradient\n",
    "with torch.no_grad():\n",
    "    z = w @ x\n",
    "\n",
    "try:\n",
    "    z.backward()  # PyTorch va renvoyer une erreur ici, puisque z n'a pas de grad accum.\n",
    "except RuntimeError as e:\n",
    "    print('RuntimeError!!! >:[')\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plus de choses\n",
    "\n",
    "La documentation relative au paquet de différenciation automatique se trouve à l'adresse suivante\n",
    "http://pytorch.org/docs/autograd."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}