PyProb - Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import pyprob\n",
    "from pyprob import util, Model\n",
    "from pyprob.distributions import Normal, Uniform, Categorical\n",
    "\n",
    "pyprob.set_verbosity(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PyProb "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **PyProb** is a PyTorch-based probabilistic programming language\n",
    "* It supports **inference compilation** with minimal intervention\n",
    "* It uses a protocol (PPX) that allows **distributed** training and inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Primitive Stochastic Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Built on top of **PyTorch's** distributions library\n",
    "* API is, however, slightly different than Pyro's"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Drawing sample from unit Normal "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "mu = 0.\n",
    "sigma = 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "normal = Normal(mu, sigma) # create a normal distribution object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample tensor(-0.3993)\n",
      "log prob tensor(-0.9986)\n"
     ]
    }
   ],
   "source": [
    "x = normal.sample() # draw a sample from N(0,1)\n",
    "print(\"sample\", x)\n",
    "print(\"log prob\", normal.log_prob(x)) # score the sample from N(0,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Just like in the Pyro examples, these distributions and sampling functions are backed by the underlying PyTorch implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sampling "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7073)\n"
     ]
    }
   ],
   "source": [
    "x = Normal(mu, sigma).sample()\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* No need to explicitly pass a name to the random variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Ice-Cream Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s suppose we have a bunch of data with daily mean temperatures and cloud cover. We want to reason about how temperature interacts with whether it was sunny or cloudy. A simple stochastic function that does that is given by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('cloudy', tensor(52.4443))\n",
      "('cloudy', tensor(54.9226))\n",
      "('cloudy', tensor(49.1204))\n"
     ]
    }
   ],
   "source": [
    "def weather():\n",
    "    cloudy = Uniform(0, 1).sample()\n",
    "    cloudy = 'cloudy' if cloudy < 0.3 else 'sunny'\n",
    "    mean_temp = {'cloudy': 55.0, 'sunny': 75.0}[cloudy]\n",
    "    scale_temp = {'cloudy': 10.0, 'sunny': 15.0}[cloudy]\n",
    "    temp = Normal(mean_temp, scale_temp).sample()\n",
    "    return cloudy, temp\n",
    "    \n",
    "for _ in range(3):\n",
    "    print(weather())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ice_cream_sales():\n",
    "    cloudy, temp = weather()\n",
    "    expected_sales = 200. if cloudy == 'sunny' and temp > 80.0 else 50.\n",
    "    return Normal(expected_sales, 10.0).sample()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(215.8908)"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ice_cream_sales()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference: One Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GaussianUnknownMean(Model):\n",
    "    def __init__(self):\n",
    "        super().__init__(name=\"Gaussian with unknown mean\") # give the model a name\n",
    "        \n",
    "        # prior mean and std\n",
    "        self.prior_mean = 1\n",
    "        self.prior_stdd = math.sqrt(5)\n",
    "        self.likelhood_stdd = math.sqrt(2)\n",
    "\n",
    "    # Needed to specifcy how the generative model is run forward\n",
    "    def forward(self):\n",
    "        # sample the (latent) mean variable to be inferred:\n",
    "        mu = pyprob.sample(Normal(self.prior_mean, self.prior_stdd))\n",
    "\n",
    "        # define the likelihood\n",
    "        likelihood = Normal(mu, self.likelhood_stdd)\n",
    "\n",
    "        # observation \"placeholders\"\n",
    "        # these can have concrete values associated with them later when\n",
    "        # conditioning on data.\n",
    "        pyprob.observe(likelihood, name='observation-1')\n",
    "        pyprob.observe(likelihood, name='observation-2')\n",
    "\n",
    "        # return the latent quantity of interest\n",
    "        return mu\n",
    "\n",
    "# create an instance of the model\n",
    "gum = GaussianUnknownMean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The probabilistic program above models **Gaussian with Unknown Mean**: our goal is to infer the expected value of `mu` above.\n",
    "* PyProb supports multiple inference algorithms, including inference compilation (discussed later)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean: tensor(7.1918)\n",
      "stddev: tensor(0.6880)\n"
     ]
    }
   ],
   "source": [
    "posterior = gum.posterior_distribution(\n",
    "    num_traces=1000, # number of samples to be used\n",
    "    inference_engine=pyprob.InferenceEngine.IMPORTANCE_SAMPLING, # the inference algorithm\n",
    "    observe={'observation-1': 9, 'observation-2': 8} # conditioning on some observations\n",
    ")\n",
    "\n",
    "# inferring the mean and standard deviation of our `mu` latent variable\n",
    "print(\"mean:\", posterior.mean)\n",
    "print(\"stddev:\", posterior.stddev)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* PyProb provides other inference algorithms too:\n",
    "    * Importance sampling\n",
    "    * Lightweight Metropolis-Hastings\n",
    "    * Random Walk Metropolis-Hastings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference Compilation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyProb supports **inference compilation**. Let's apply that to the _Gaussian with Unknown Mean_ example we used in the last section. Recall that we defined a model for it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.GaussianUnknownMean at 0x7f77d81d1e10>"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyProb provides an inference engine to use in combination with a **learned inference network**.\n",
    "\n",
    "First, we need to learn the inference network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating new inference network...\n",
      "Initializing inference network...\n",
      "Observable observation-1: observe embedding not specified, using the default FEEDFORWARD.\n",
      "Observable observation-1: embedding depth not specified, using the default 2.\n",
      "Observable observation-2: observe embedding not specified, using the default FEEDFORWARD.\n",
      "Observable observation-2: embedding depth not specified, using the default 2.\n",
      "New proposal layer for address: 16__forward__mu__Normal__1\n",
      "Total number of parameters: 5,834\n",
      "Train. time | Trace     | Init. loss| Min. loss | Curr. loss| T.since min | Traces/sec\n",
      "0d:00:00:02 | 1,024     | +2.19e+00 | +2.11e+00 | \u001b[32m+2.12e+00\u001b[0m | 0d:00:00:00 | 548.0                              \n"
     ]
    }
   ],
   "source": [
    "gum.learn_inference_network(\n",
    "    num_traces=1000, # how many traces (executions of the generative model) to use to train the inference network\n",
    "    observe_embeddings={\n",
    "        'observation-1': {'dim': 10},\n",
    "        'observation-2': {'dim': 10}\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the inference network is **trained** with the traces produced by the generative model, we can use it to perform **inference**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean: tensor(7.3547)\n",
      "stddev: tensor(0.7760)\n"
     ]
    }
   ],
   "source": [
    "posterior = gum.posterior_distribution(\n",
    "    num_traces=1000,\n",
    "    inference_engine=pyprob.InferenceEngine.IMPORTANCE_SAMPLING_WITH_INFERENCE_NETWORK,\n",
    "    observe={'observation-1': 9, 'observation-2': 8.4}\n",
    ")\n",
    "\n",
    "# inferring the mean and standard deviation of our `mu` latent variable\n",
    "print(\"mean:\", posterior.mean)\n",
    "print(\"stddev:\", posterior.stddev)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference Compilation: Solving (Mini) Captchas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* A more compelling example of inferene compilation is in tasks such as **captcha solving**\n",
    "\n",
    "* For simplicity, our captchas are composed of _only one letter_\n",
    "    * Any letter of the alphabet can be chosen\n",
    "    * Different fonts are used\n",
    "    \n",
    "* First, let's define our generative model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from PIL import Image, ImageDraw, ImageFont\n",
    "\n",
    "class MiniCaptcha(Model):\n",
    "    # possible letters used in our captcha\n",
    "    _alphabet = [\n",
    "        'A', 'b', 'C', 'd', 'E', 'f', 'G', 'h', 'I', 'j', 'K', 'l', 'M',\n",
    "        'n',  'O', 'p', 'Q', 'r', 'S', 't', 'U', 'v', 'X', 'w', 'Y', 'z'\n",
    "    ]\n",
    "    \n",
    "    # possible fonts used in the captcha generation\n",
    "    _fonts = [\n",
    "        'Ubuntu-B.ttf',\n",
    "        'FFF_Tusj.ttf',\n",
    "        'LobsterTwo-Regular.otf',\n",
    "        'SHPinscher-Regular.otf'\n",
    "    ]\n",
    "\n",
    "    def __init__(self, alphabet=_alphabet, fonts=_fonts, noise=0.1):\n",
    "        super().__init__('MiniCaptcha')\n",
    "        self._alphabet = alphabet\n",
    "        \n",
    "        # all letters and fonts have the same probability\n",
    "        self._letter_probs = [1/len(alphabet) for i in range(len(alphabet))]\n",
    "        self._font_probs = [1/len(fonts) for i in range(len(fonts))]\n",
    "        \n",
    "        # add some variance to the observations\n",
    "        self._noise = noise\n",
    "\n",
    "    # Builds an image canvas of the given text with the given font\n",
    "    def render_canvas(self, text, font, size=18, height=28, width=28, x=6, y=6):\n",
    "        pil_font = ImageFont.truetype(font, size=size)\n",
    "        text_width, text_height = pil_font.getsize(text)\n",
    "        canvas = Image.new('RGB', [height, width], (255, 255, 255))\n",
    "        draw = ImageDraw.Draw(canvas)\n",
    "        draw.text((x, y), text, font=pil_font, fill='#000000')\n",
    "        return canvas\n",
    "\n",
    "    # encodes the canvas into a tensor\n",
    "    def render(self, text, font, size=18, height=28, width=28, x=6, y=6):\n",
    "        canvas = self.render_canvas(text, font, size, height, width, x, y)\n",
    "        return torch.from_numpy(1 - (np.asarray(canvas) / 255.0))[:, :, 0].unsqueeze(0).float()\n",
    "    \n",
    "    # convenience method: runs the generative model once but instead and returns the\n",
    "    # actual image generated. Used in order to allow us to see some of the possible\n",
    "    # captchas we generate\n",
    "    def forward_canvas(self):\n",
    "        letter_id = Categorical(self._letter_probs).sample()\n",
    "        font_id = Categorical(self._font_probs).sample()\n",
    "        \n",
    "        image = self.render_canvas(self._alphabet[letter_id], self._fonts[font_id])\n",
    "        return image\n",
    "\n",
    "    # the generative model:\n",
    "    def forward(self):\n",
    "        \n",
    "        # sample a letter and a font from a categorical distribution (where every letter\n",
    "        # and every font have the same probability)\n",
    "        letter_id = pyprob.sample(Categorical(self._letter_probs))\n",
    "        font_id = pyprob.sample(Categorical(self._font_probs))\n",
    "        \n",
    "        # render the image\n",
    "        image = self.render(self._alphabet[letter_id], self._fonts[font_id]).view(-1)\n",
    "        \n",
    "        # define our likelihood to be a Normal distribution with mu = the calculated\n",
    "        # value for the image above\n",
    "        likelihood = Normal(image, self._noise)\n",
    "        \n",
    "        # observe some image (to be given later)\n",
    "        pyprob.observe(likelihood, name='query_image')\n",
    "        \n",
    "        # return the latent variable on which we want to do inference: in this\n",
    "        # case, the letter id in our alphabet.\n",
    "        return letter_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that our model is defined, let's create an instance of it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "captcha_model = MiniCaptcha()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now able to generate some of our captchas (for illustration purposes):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAA0UlEQVR4nO2VMQqEMBBFx2Wr1IJFGgs9gWcICFZa5AQeI5WlHsHeyjPEUhCPYRHIFYTZYsEqYUmI1frahEfm/4FEiAiheQU3PtJH+s/St+1g27Z1XbMsq6rK2YomOOd5ngNAkiRlWWqtjddsmKVxHAshEHGeZwA4jsNJ+iPTNE3btiWEOE1vzfRLURTjODoZ4ab2zZk2TXONzBgLU9TFLUX5YZYOw7AsS2Bp3/dSSm+pdaWmadr3XSnlITW/VGvddd15nnVdIyKl1Eka4fPvh+YDbkFSgTnCEf0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAB4ElEQVR4nO3VP8uqUBwH8KveeDTMUqIcKlKnXkEFtUVtzg0tvYB8GREFDb2ElqB3ULi0uFZjSNYSiJAN/ssMzx2Ch5Cw6+WB+wzPdzy/cz78/IHnQACAX18d+MvFH/QH/QaoZVlv9/wOqZ3P58VioSgKRVEfHx8cx2Wz2clkIghCqVSKjDqOMxgM9vt9MpmEYRgAsN/vV6uVLMsMw1AUFblTx3EEQWBZtlgschzX7XYf67PZLJfLaZoWi8XC0eBMr9drr9er1+v5fN6yrE6n81lqt9u6rpMkmUgkoqGSJGEYhqKoLMvVavW5KQiCeJ7fbDamaYajwc/fbrfpdPp0OimK8tzmI7VaDUEQ27ZJkoyAuq6byWTi8bjruiiKBqoEQbRarfA2X6AIgmiaRhAERVG32+3t+ZcJzrRQKBiGAQAgCGK9Xr88Y5qmYRgR0EqlIsuyqqo4js/nc9/3AxvG43G/379cLhFQHMebzSYEQRiG6bouiuJzdblc7nY7z/Nomg5BocDDZxgGjuOj0cjzPACAbduP+SIIcjgcjscjTdPD4RCGwy6NIPoZSZKm0+n9fmcYJpVK6bquaVqj0eB5HoKgEDEMfUQURVVVfd9nWbZcLr/9Qf8K/bd8j0v6v6F/AEdzyiq/uFK6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAABKklEQVR4nO2Vv6qDMBSHT41QRHGQgmPBpSCKeyd3wb6GkEfxHcShoy8gglNewI51ULooFNq5QZI7FMpFBGtv7tZvyh/OxyG/kKw45yAaSbjxK/1Kv9IZTqfT4XAwDMN1XcbYfAGfI89z27brutY0DQCGYZgtmZFeLhfDMAghnHNh0iAI9vv9cyxGSggBgCzLlkrHQd1utyiKttvter32fR8hRCm93++L0pdHc4xxWZZJkjiO43leGIaWZWVZ1vf94/F41zrqPI5jVVWv12tRFACQpulrS8CZYowB4Hw+i5TudjtFUX6vfB7UE8ZY0zSmaS7K58W0tOs6SqlgKaUUAGR5fDfeZMWn/n3G2Gaz0XW9bdsPpNOdSpJ0PB4RQlVVfSCd7vSP/Msj/QOATj/rUQ6FYwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAAZUlEQVR4nO3VsQ2AQAiFYXEWxqBhoxuPjoYx2AXbizExciRqwj/A17ziQURs1e3lYqONNnpujAFXIWIeXewGVdWYcvcCNNcbKDPPQ5lZAZrr2VBEVIDm+sBQACAiq2gu6Iv+B3oAMq4twT/DSREAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAABIElEQVR4nO2Ur4qEUBSHz1k1islsE4zjE1hkTCbBYDBPUsYX8AUGDFab4NjsFpPV6AOoRWFeQPBusCyLqLMYZsEvHe49fPd3/3CREAJH83W48ZSe0lP6G9d1ERERi6KYR+q6FgQBERmGSdN0XUrvWbmua0VR+r6nKCpJEsMw/pL0J03TqKra9z0iRlG0adyWvl6v6/XadR0ABEFg2/amEQCALOE4zjzL8/xc+L6/2LnIhnTmcrnsNxJCNrYvyzIAVFV1u93e+M7Xk+Z5rmnaXHued0xSmqafz6coigDweDzCMNwTdPtJcRyXZRnLsgBwv9/LsjxACgCSJMVxjIjjOJqmOQzDAVIA0HV9Pui2bS3LmqZppRnfuNPd/Pev7xOl33pVFfKnonw/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAA8ElEQVR4nO3VoaqEQBSA4TNrFBFFUYPZarYZTT6Dj2A0mn0Oq82iNh/AIDbRYBJkBgSj44aFDfd6WdidGxb80wwcPoaBYdBxHMC6G3PxQi/0Qi/0n9Gu6xBCQRA8tqZpGoYBAFmWaZpWVdU76F9JkqQoSpIkLFHXdYui6PueJQoAgiBwHMcYxRirqsoApZQ+103T2Lb9DsrzPABM0wQAhJB5ngkh+74DQJqmnue9RNHv35RSquv6uq5RFOV5vizLOI5hGG7b1rZtXdevr/U4qyxLy7JkWY7jGGPs+74oio7jDMNwOv+jk5N+3le//c+7Az28j+wtgsVuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAA20lEQVR4nO3VMQqEMBAF0J81jVoI6iUstLW31CtYBA8heADBG3kLG/UcopBCEbOFYCOyWTbb5XeB4QX+FEOEEFCdl3JRoxrVqEb/jI7jSAhhjJVl6ft+GIZt236niluGYQDgeV6SJFVVUUodx1nX9T75lEfUdV3OuRAijmMAXdfJo4+dpmlqWRaAIAgAcM5/6vQMpfTqB8D5wa/olX3fAZimCaDvezWoYRgAtm2bpqmuaxmUfpyIoghA0zTzPOd5LoM+bp8xdj6XZcmyzLbtoiiO45DZPhH67qvOGxog+wFlxD36AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAAeUlEQVR4nGP8//8/A7UBE9VNHDV01NBRQ0cNRQFbtmyJiorS09MTFhZmZ2dXUVE5ePAgCab+xwacnZ3RlN28eROrSqyABY99vLy8T548YWFhuXv3rpqaGvEOxWcoExMTHx8fAwODrq4u8SYyDP3YHzGGMo5W0UPDUADYn1WY01idHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAAxElEQVR4nO3UoQ2EMBiG4a+XmwBSSQiqEwBDdAbWYAB898DgSBGEGRAoDKnrBiiSnjh15cTRa1B9bZOnrfh/YoyB7x7exYAGNKAfLcuSZVnTNJ5RpdQ4js4ozKnjOKSUWuvz0Y+Rm7bUuq6EkKqqfKL/F9Db0bZth2HwjE7TVJalN3TfdyFEkiRRFF1Fv4yplBIAY6zve7cxfVp31HW9bRuAPM8555ffCMD6ftd1cRy/lx6l1E20UaVUmqbzPAMoisIZfQE/xtAHJatSxgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAIAAAD9b0jDAAAAlElEQVR4nO2UMQoEIQxFM6u9lrY2oo2llTfyShYewPuIt0htk72Ay+KOLAz4ygQeIfnkIiLYzWu78UiP9EiPdCd8WkVEzjkRjTEYY1LKJek1/afOuRgjItZarbWttQ2TAkDOOaVUShFCLBkBAGiGtTaE0Hufdr/y8VDee2PM8owA8KRI/VGqtVZK/Syd5/QmT9/pTd4nDHBSxbZ0NAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import IPython\n",
    "import io\n",
    "\n",
    "for _ in range(10):\n",
    "    canvas = captcha_model.forward_canvas()\n",
    "    bytes = imgByteArr = io.BytesIO()\n",
    "    canvas.save(bytes, format='PNG')\n",
    "\n",
    "    IPython.display.display(IPython.display.Image(bytes.getvalue()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Since this is a simple example, we are able to compute the posterior distributions ourselves.\n",
    "    * We can then compute the KL-divergence between this and the inferred posterior distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Categorical(num_categories: 26, probs:tensor([1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e+00,\n",
      "        1.0000e-08, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e+00, 1.0000e-08])), Categorical(num_categories: 26, probs:tensor([1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08, 1.0000e-08,\n",
      "        1.0000e-08, 1.0000e+00]))]\n"
     ]
    }
   ],
   "source": [
    "epsilon = 1e-8\n",
    "\n",
    "true_posteriors = [\n",
    "    Categorical(util.one_hot(len(MiniCaptcha._alphabet), i) + epsilon) for i in range(len(MiniCaptcha._alphabet))\n",
    "]\n",
    "\n",
    "print(true_posteriors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, it's time to train the inference network using our generative model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating new inference network...\n",
      "Initializing inference network...\n",
      "New proposal layer for address: 12__forward__letter_id__Categorical(len_probs:26)_...\n",
      "New proposal layer for address: 28__forward__font_id__Categorical(len_probs:4)__1\n",
      "Total number of parameters: 455,138\n",
      "Train. time | Trace     | Init. loss| Min. loss | Curr. loss| T.since min | Traces/sec\n",
      "0d:00:00:09 | 1,024     | +4.64e+00 | +4.62e+00 | \u001b[32m+4.63e+00\u001b[0m | 0d:00:00:01 | 109.0                              \n"
     ]
    }
   ],
   "source": [
    "# Training the inference network\n",
    "\n",
    "captcha_model.learn_inference_network(\n",
    "    num_traces=1000, # use 1000 traces from the generative model\n",
    "    observe_embeddings={'query_image': {'dim': 32,\n",
    "                                        'reshape': [1, 28, 28],\n",
    "                                        'embedding': pyprob.ObserveEmbedding.CNN2D5C}}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once our network is trained, we can do inference with it. In the code below, we calculate:\n",
    "\n",
    "* How accurate our posterior is in determining what letter corresponds to the captcha we generate\n",
    "* What is the KL-divergence between the true posterior and the inferred posterior"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 1.0\n",
      "KL-divergence of all posterior distributions: 9.536744300930877e-07\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "alphabet = MiniCaptcha._alphabet\n",
    "choose_font = lambda: MiniCaptcha._fonts[random.randint(0, len(MiniCaptcha._fonts) - 1)]\n",
    "\n",
    "\n",
    "# generate a test image for each letter in the alphabet\n",
    "test_images = [captcha_model.render(letter, choose_font()).view(-1) for letter in alphabet]\n",
    "\n",
    "posteriors = []\n",
    "estimates = []\n",
    "\n",
    "# for each letter, compute the posterior distribution (by condition on a captcha of that letter)\n",
    "# and record the what letter in the alphabet is inferred.\n",
    "for i in range(len(alphabet)):\n",
    "    posterior = captcha_model.posterior_distribution(\n",
    "        1000,\n",
    "        pyprob.InferenceEngine.IMPORTANCE_SAMPLING_WITH_INFERENCE_NETWORK,\n",
    "        observe={'query_image': test_images[i]}\n",
    "    )\n",
    "    \n",
    "    posteriors.append(posterior)\n",
    "    estimates.append(alphabet[int(posterior.mode)])\n",
    "\n",
    "correct_estimates = sum([1 if estimates[i] == alphabet[i] else 0 for i in range(len(alphabet))])\n",
    "accuracy = correct_estimates / len(alphabet)\n",
    "\n",
    "kl_divergence = float(sum([pyprob.distributions.Distribution.kl_divergence(\n",
    "    util.empirical_to_categorical(posterior, max_val=len(alphabet) - 1), true_posterior)\n",
    "                           for (posterior, true_posterior) in zip(posteriors, true_posteriors)]))\n",
    "\n",
    "\n",
    "\n",
    "print(\"Accuracy:\", accuracy)\n",
    "print(\"KL-divergence of all posterior distributions:\", kl_divergence)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}