Convolutional VAE Style Transfer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional VAE Style Transfer\n",
    "\n",
    "This project is an experimental attempt to transfer artistic style learned from a series of paintings onto a video sequence by fitting a variational autoencoder with 512 codes to both paintings and video frames, isolating the mean feature-space embeddings and modifying the video's embeddings to be closer to the painting's.\n",
    "\n",
    "Because the general quality of the VAE is relatively low, a convolutional post-processing network based on residual connections was trained with the purpose of making the resulting image less similar to the VAE's generated output and more similar to the original input images.\n",
    "\n",
    "This makes the processing pipeline consist of a encoder, variational embedding, decoder and a generator; sort of a three-quarter deep convolutional VAEGAN architecture minus the adversarial training. No cross-validation, dropout or regularization procedure has been used in order to get the networks to as closely fit the inputs as possible. \n",
    "\n",
    "Image frame size was limited to 320x180 because of memory limitations; The VAE uses 6 layers with increasing feature map sizes in an attempt to make up for this. Training the whole network took about three days on an nVidia GTX 980TI.\n",
    "\n",
    "## Training inputs\n",
    "\n",
    "The paintings and videos used are:\n",
    "\n",
    "* [Leonid Afremov](http://leonidafremov.deviantart.com/gallery/)'s DeviantArt gallery\n",
    "* [Disclosure - Magnets](https://www.youtube.com/watch?v=b_KfnGBtVeA) music video\n",
    "* [Wim - See You Hurry](https://vimeo.com/22328077) music video\n",
    "\n",
    "Finally, the trained VAE was used on an unrelated video,\n",
    "\n",
    "* [Pentatonix - Daft Punk](https://www.youtube.com/watch?v=3MteSlpxCpo)\n",
    "\n",
    "## Resulting media\n",
    "\n",
    "The script creates video-only MP4 files to which I added back the music from the original videos using `ffmpeg`.\n",
    "\n",
    "The Pentatonix video can be found in the zip file. Because of the general file sizes though, the remaining videos have been uploaded to Vimeo instead. For copyright reasons, the videos are protected with the password\n",
    "\n",
    "    cadl\n",
    "\n",
    "You can find the videos here\n",
    "\n",
    "* [CADL VAE Style Transfer on Pentatonix - Daft Punk](https://vimeo.com/202984113)\n",
    "* [CADL VAE Style Transfer on Wim - See You Hurry](https://vimeo.com/202979720)\n",
    "* [CADL VAE Style Transfer on Disclosure - Magnets](https://vimeo.com/202991439)\n",
    "\n",
    "of which the Pentatonix one is by far the most interesting, because the network had to make things up on its own.\n",
    "For See You Hurry and Magnets, movement is a bit choppy since the network was only trained on every tenth frame. It can also be seen that faster motion tends to correlate with more colorful rendition, whereas the fog in the See You Hurry video doesn't do the video any favor at all.\n",
    "\n",
    "In order to get a cleaner outcome, I assume a real VAEGAN approach might be more fruitful.\n",
    "\n",
    "## Training process\n",
    "\n",
    "Data is extracted using `extract_tiles.py` and written to `.tfrecord.gz` files for later usage. The `preview_tiles.py` script is used to validate correctness.\n",
    "\n",
    "`train_vae.py` performs the actual training based on the TFRecord files. The network was pre-trained using See You Hurry and the Afremov paintings, to which I later added the Magnets video frames (that part went well). Learning that VAE is _very_ slow, although adding another training set did not appear to make it worse. I stopped training the VAE after approximately two days and left the refining network running for about 16 hours, at which point improvement was noticeable, yet very subtle.\n",
    "\n",
    "The `export_graphs.py` script takes the network checkpoints produced by TensorFlow's Supervisor and exports them as reusable protocol buffer files. The `evaluate*.py` load these files in order to perform inference and some tests on the latent embedding vectors.\n",
    "\n",
    "Finally, `mogrify_video.py` is used to process videos using the network.\n",
    "\n",
    "### Impressions from the training process\n",
    "\n",
    "After the VAE training reached a certain point, convergence slowed down. This is a screenshot about eight hours into the learning process ...\n",
    "\n",
    "![VAE training](doc/preview-20170205-231757.jpg)\n",
    "\n",
    "... while this is about four hours later.\n",
    "\n",
    "![VAE training, four hours later](doc/preview-20170206-013356.jpg)\n",
    "\n",
    "A video of the learning progress  over about 9000 batch iterations is assembled here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkz\nODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2MBERISGBUYLxoaL2NCOEJjY2NjY2NjY2Nj\nY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY//AABEIAWgB4AMBIgACEQED\nEQH/xAAbAAACAwEBAQAAAAAAAAAAAAAAAwIEBQEGB//EAEYQAAICAQMCAwQHBAgEBQUBAAECAAMR\nBBIhBTETQVEUImFxBiMygZHR4RWhscEzQkNSYpKT8BY0U/EkRIOi4iVyc4KjY//EABkBAQEBAQEB\nAAAAAAAAAAAAAAABAgMEBf/EACcRAAICAwACAgIDAQADAAAAAAABAhEDEiETMUFRBCIUMmFxI0JS\n/9oADAMBAAIRAxEAPwD5/CEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEI\nAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBC\nEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQg\nBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCPXSWM20YnX0jpjLIc+hM3p\nL6M7IrwjDSwbbkZnBUT5gfOZ1ZbIQjhprCFPHvdp2zSWVoHcqATgcyULQiEmtTOwC4OY06N17sgP\npmVJv0G0ivCMFDHsROpp3dsAiNWLQqEedK4XO5flJV6K2wKQVG7tk95dJfRNkVoRzaZwQAysT5Dy\nk20VqkfZOfMZ4jSX0NkVoS0mgudsDaPmYV6C6wsF2+73yZrxz+hvH7KsJY9it3Y4kx0+3j36xnty\nfyk8c/om8fsqQlp9Dandk/GcXRWN5r95l8U/obx+ytCWl0NrHgp+JjF6TqTaK22IxGfeMnjl9F2R\nRhLdvT7KzgWVv67SePxEiNFafNfxl8c/obx+ytCWhoLD/XT8TOp06522gpnGe8jhJfAUk+FSEt2a\nB0bHi1N8VJ/KKNBAJ8RMg4xz+UzRbEwluvQWWLlXr+RJz/CQfR2IfeZf3zWkvom69FeEZ4LeokvZ\nn9VmC2hMI72Z/wC8sl7JZnGVihaK8JZ9is/vJ+MBobSftJ+JlobIrQlwdNuI4ZPxP5Tn7Ouz9pPx\nMUTZFSEujpd5ON1f4n8o2vomosYKtlOT5ZP5RQ2Rmwmw30b1a2ms3afIOM7jj+EaPoprmr3i7TY/\n+9uP3RQ2RhQm3/wxrf8Aq6f/ADN+U5/wxrf+rp/8zflFDZGLCbJ+jWtH9pR/mP5SP/Dms/6lH+Y/\nlFDZGRCa5+jurH9pR/mP5SH7B1Wf6Sn8T+UUNkZcJonoupBwWq/E/lIt0m5RnxKj8ifyihsihCaV\nXRNTbUbFeraDjkn8opel3vaaw1eR55P5RTGyKUI86SwHBKy/R9HdZqPC2PT9apYZJ4A8zx8YouyM\nmE2tR9GNbp03Pbp8fBj+UrU9F1VxIQ18c8k/lFMmyM6E0f2LqtxXdUcd+T+UUem3hc5XHrkxTGyK\ncJZ9hs495PxPEn+zrecPWcehP5RTLsizS7b1qTALHuZMe8g2+89f2hiIqcpYtmOV5Ev9MUC2x7wN\nudrTu304qJX8AWWbggXI3Zi9RTmz3tqgADiXHo1Fd5ZMlVyePT/ZkrbVuait6wHAzu8sfKWUkRKS\nI6TQvYh8NS61Jkn0/wB4mf7Jb4nvodgPJM9DTqF6fqH2IXJUDB8zH9f0TOj3VqBgguq8eeJ54uUp\nN1w0vX+nnfDFafVIDu85FMFiGHv+seSNqDgbTOgDK7hhScnjynXZL0YvnSmyKbz3wRJCpUXAPfzj\n/C8W0spx5cjtJ2aZKq8q6H4ZBMjkroturFtpQaPEyScx1bMCVz7mzGPjH1qTRhiMDgRboBtYnhc9\np1j74cNr4yK0iy0nA977RlnU4NK7UBbOFA+GI7RIQzF1AB7CWxpGNVjEMqqcbvj6Tsq+TDlJMV06\nrTKPEvVywHCgZlO4Ku+4IC1rnAxwBNXpoqJO9gM8HPkJX19tb6my0qBuPuoB2EkL2f0abqJj+DZd\nlgoGTxDUUNWVxhviO4k7mtO4oCFPpGdPUW7lcncTEsmstWbSk1a+CvXp9z4Y8ZktW5LeXHaXzR64\n2evnEamlWwKwSQc/dOvH6Mpu+lalxWdyHJ/vY7RW92D2ZOfMy2EqNOCMOMkn1lGrdht5K1k5KiZv\n6OqiOrrRQpubYp7HGZHxGr3VjYRnuPOWBbpXpKkeWcxGK9wONvric5ySkulStPhxQxbAjDU/2gJE\nMMe6eYxz9WgFmMd5NlqKoSqAE7h3MnqPDqIsrrHu4HMt0Lp9pFjZZtvP93J5z90TqkRbFZsCsckn\n0+Xz/lPOsq9JG0m5WyL4KJelZrRh5nP3xN6NWUbghxwT5fCL1mq8S1q6G3Vj7P4SAqeypQ7AlfIn\nOJ1Um/RtxSdiwAe87jngSPbykhxPMQl5dpNG45kOxk6lzgdsykJAyYM4UKsAZIKM98CUyOST2gkY\niN2M4M6jtnvxBC1sBziXNIffUggHPf0lJT7wKjylqoupBG0iCF/V/wDNXnggse38ZGpmxuIOPnFv\nYzuSwGTH6fK5Axgj8IBJWJjh2iVIWNrPBgARwREt3j7ARtI9MxBgpFhkCVgPfEt4ysQF5gEbafEX\nHYj0lTU1JXWwB57Ylx968iUdUxIAx3PeUFuhSvTBx3BMoaILuudyQFrJ4EvWMy9PwOAEA7TPQKuh\n1TlsHAUD1yYCM0nJnreg2qdEdS4IFdYrHyH6zyQGSAOSe09TodDdbWNLhtLXThrDu3Mx7geg9fwg\nrF63U+JYyaolV7rX+frO6KuxlPhabdz3dgomm+g02mQ2pSbH+J3E/OVaGsVSKlcL/WI5xKYKN+l1\nfhbilQUksVUmVXtDIKrE2+hPnNkasVrsvIYgYAA7mV2o0upUlB7xOeYBjtpgHySNpHAhdpSjFwcD\n+73lptNssybCp/uuM4iNXqHA2KMHPDLyDBbMlCFsRm5UDJmnR4OrR0RwjmzxFLDjBEyRW4P2cjsR\nNDp2mYutte2xl7o04Zpr2evHj34a+mpK8eIGA4by9ZS6xozUK9ZW2No2FfjkzWS3dWpetVRhyEnL\nRYmmY1YekjPvczyYfyLk/IzrP8eS4jO6TqKbC1mpIsCgcH0/SI13UrL2eimzNORg+Zx6yvVQrahk\nY7Qc4PlLg6VgKygqoXJz6iemWaMeHKOF3wj0zSVXWfXOo8wpPJk+qCuh1qX3jjjHkM/lIV6Jxy4J\nGcS81a2ahrQCp2bSuPh/2nllNb7Wd44rjo1RjO3hMRnnHPzkbLVtr4AU+s5rFYXPu43YkKqyBk+v\naezH3p4skVH0WqbsAe5ye0t16U3KGYgD0nEw3gqEUKNpI+feWtKr2IFUYGMzost/NHjkqfCdNSoj\n2m0Ar2xznGCf3RtNluoHh2tlVJIA8sxnTtKKXdrsZv2qB6DPM1dmkpFjIibsMT8cE4nfyV1nVYtl\nxmbRTp6HHuhV74lLqemWu5RSxYMM89x6zX1FAtsTCBUI7nkmZ2q09Wmt9xmYt5kzrjq7T6cZ2uNe\nikul3gpuwSO0KdL4QznJWWtOy+IFRCXJwCT3jbti2sHIYg4ODxNNRm05LppqWJcfsSW3Ls2428ZP\nnIGupa8WsM+SjuJ220hClK5Y+fmJQuPgrhjiw9hPA8mTdt8SPowxQ16WLUprNLooKl8MD3xK+sbT\nJYVrrUBhuGY3T1m6tmuGMDdnPEybWLuR2yf3Tf8AKT4iPB8kr6XZGatgSoyR8IzTBUrYv75ZD5Zw\neJWRWJIxnMuLpNTUgZRtLKe3lyJwy5FLlnSOFpiqRvxzj5x+oqVM1Iwct5iUjW1dhRn5B5mto9EG\n09tjuCF43Z8vWWedRSa9COG2VE07Vq3uk4XLfCV7G35BJ94cibVZVNKj7wHZSBvHkT2lLT1I4JGG\nszjJnKOercjUfx9pcKukrrrsX3lUscFm8oxlNd7om0gng+slq9MaMOc8ngeslp3oV69+FXu5Pn8J\n0WZ/2Xo34Ver9mYp3LxOkEcj8IvIA4gp9SROh4RofuCI5DxK3lnIjVYDGOZURlhjwJwEwB3RgEGS\nIw3zj1Qbcr3kEX1HMegH3ykJVDPy9JZCZOQcGLXAMeGEEGKPj2lmnbsPODKi89jLlFJNZcdh5wDg\nU9zGL2nRXupL+YkR7qwU7k9pAyRPM5AItkDEgF5k25gE3ZgCid2VGMyrrxveisDBI/eZppSqjsMn\nkygw8XrNa/3SP3cygl1JGq0GDyC4X90xNQWGnQf1WY/ux+c2Ouki5E8sZmFc24geQ7QEGkIXV0sw\nLAOpIHnzPfadNmn97BdyXcj1M8f9H9OLupoT2rG/7/KexZuIQkFrsaztXce2O0oaWxw5HhhVY9gc\n4+cthsRF1taBjwADzgymBjaKl8cYx8e8RrdLXSh8GtVb1DYImedQ9epLk7iPjxLF+uVdP4VdPvHk\n5PY/CAZ1itZaqvkuxxKl9LVu2/ujHEa1j9yTH0Ml1mLj7x7sWxgQUzum3G5TpSqLuG0Ps5jUHsdx\n0tSmy1TwV8x8pm03Np7Rag95cd5c6Zra69e9+oOSeMz5+SDVuuH1cGRcPf8A0d0lHsK2uiM7E9xn\nER1+nwmRaSE8TPAmNo+peJqNmmchCeV3lQ0qdR1GpGpCW3tdngLuyV+R/Ocv1lDWqZ0uszldoo6m\ntGtZQ21t+CcSXT2sqsuo1DlXAI2MeAcQ1ITTVfXI62Mfcb4/GWqrdKNG511TvY5ydUjblY/E+XpO\nlPT0YlkW6aPRdD02n1jva6hlTGBmXOq6Kiqg21oqkfvnl9Jr7dJSx0DFSe7nkAS3T1o6lT7XqSc9\nlfAx+E4ySWJqum5Nyy2nwzdbSr2bigBY4AxM3qDWG+1ztBOMhe0t9Q1NY12+ots2+vnKusdtUj2I\nijA5+c64HKNMfkRxzj66K0d7ncgPn+6eh6Qa6iWucDcCAPhPL6MtXYWwSB3muNet4ryuxKxxtHJn\nt1Xs+VKFdNKxbNRrMf1Rxmal2jX2Kt1xlRyPURHSDSujZg/iuRlsDkE+UlffcNSUsI2hQAAO2Z5v\n5f7tV6N48TjG2Ou1OnqoVWf6wcBRMph4+oJ+1jtIarS/WNbSzsd2MN8pq9F0j48PUoC23K44z859\nbE4qG6+Tx5dpzoVXpADv2AEDHAlS3RKlqsCQhOSJv6wCqpAAFycbZl66wJoycHduwJ4llniyN+7O\n/i8iSKLXIh2hVUZ8vOUClOp1INg25bA5j6NKmo1G+zJPznNZp6KbyMF1HxxzOudylCvR3w49Z37L\nNKaeyw6etQ39UgmUdToSrWVrSV2kjtkn4Sehc6e3xjURuOA38pZGqNfUme9cA4bnyOJ8dxnCb19H\n0ovnUN6b0t7qUu9mLAdmxDqQRVVEU8ZDAjBm90vrvThoaarL1qtVACrcZwPKZXU9VTqNY1lJVht4\nGOTGXHqlKy48jlJpo84aMcvSG4O7I/CWtBoq9QXqvVlBUFSPKVNVqm1Go2DKgrgY4xLug1HtF+EJ\nX3Bgnz/3manuoWaioOWtlO3SlrdhY7UYgfjGV6TF7NU5A8sGb3RdCl3VW9pAKr2B8zNvq3T9GdI7\nipEZFzuUYImoqU4WmZ8kceRJo8VqVV6q67bCx5CKoyfnM7TGkMyXDnOPiJo6yyrSeAcZYE7fXEz9\na2mscWopV2PvD+c1iTlz4Z0zUm38oz1r934yYU4AzOrOOxXGPxnuPinSoYEzoAHdSPSAPHJjh25l\nIdQHAj4oHAnQTBkcJJW5i8ya8ykHoxyJYQ5HMqqY+thBCyvaXqrQNOa8HJHBEzwxAjK7ciAXQ31J\nGcY/fF8kSG7idD+hgpPtCVb9Qa2AHOe8tVkNg+RgEfM/ONpHJkGADHElV3gFjbg4mX08eJ1e1+4G\n4/vmoThC2ewzMzoYzbfZ8APxgFbrlgbXFR/VUA/xmG/LnM1eptu11p/xSjrtvtdgXsDj8JSo1/oq\ng3ah/P3R/Gb77hz5TB+jyeBSbtjMbDgY7YE2n1OWxkKD694MP2VNTe2z3MkeeJwN4tRU1hSee2Z2\n1C527e3n5SK7d31f218yJohw6alKybATYeVPYfhFX0Op3OpAA4yZZF16v7ybl9MSrdrTfZ4e3cSe\nPPEAQaQ1eAMEc8jykaVzYrY3N2CsMgQuuaqxkqZgM4JHELtU5rBdt57qRwc/H8IKYToVRt2PfXK4\n+EjpjWdTvdlVARnPoJO+46mgEBQEOB8ZXVAXYqcqMEj+U5zgn6PRF17N672RWa2khguFXyzKlfi2\nXsyf0YOCf9/GKr1FVemvVlU2MOG9OPzlWvWvVRsTj4zyRhR33bPRUX26nTazT1OHqckF/MHA7RvS\nKRR06lRYcWKHOT5k9pk9Aptu8TD4rX7S+v3TXosS3o1NC+46g+9nnGeAJ3WNatL5Od9KXUupal9m\nl4UE/ecGZl9zPYodcMODxLAqU0XX2W5szgbjzkx1fTrbFFm7BxwT5zzfpA9EZS4jq9Ks3rZY+ayu\ncqMkSVOjGLHrXlBkA9iPjLeke0oVtLAHy7Ezd19vSK9D/wCEvrV1HCggk/Oc4Tl1v4O048X+mUmi\n0uq0YqFgqexSzeH3wPh85QbpLV6ZWU9hlgfKL0FjrqGsU7mJGSfPJnqWu6celLlgNQFxjdhiZvLO\nTlUXVHNQj6kjG6MG04HOC3G0y9cLNT7+CGyO0zrdyufDyduCfhLFOuK1qpyN5wJMUk3t8s5ZsDi6\nfotLVYyjIOTnBx3l0LZUVtJbCkZlTxittKkjABZSPMTQvvqspHvgncMj15nolkyRpfBwjCLK12pt\nv1TWMMIBx6ASlqtcg0rFMOTxxziautZKqwABll7funnNbWKiqVhtzN72BmdXjjlanLlGklErpdYl\nxC8ECcvc7VLNnbk4PnLz001U7XuXe4BCqcn9JmJpbLXG9tqDnJ9Mzvkpqoo1GSTLWg6shsCX1g7Q\nceX++8r230O9jMwCkcY9YjT6OzqNhooAGH+0e5EZ1jol3TbFW07lYZBAnz50z24516FDqCVcV5Kg\n5I9f98Rd2o3g2M4HkoHp8Ze6b0g60rU2Sx7Y4GPWWuodGbpdtRKqVJwPjxMJxsTdu2ZLVNcvuagB\n2GVQLyT5cxmjr9nYCu5QxHG7kH75f11K39IuvbBdGBUrxj7ImVqLBo7kdavq9nA9c4naUG4cPPut\nvRqVanqd9beHUilP7QfyibdX1LxT4+utav7u87pdfbYl3sx2KWGMDsMekshqf2fm5wSp3Bm78zyW\n8dqvZ1Utmn7owte7PqKqzuyRxnzz2j7dS9HTU0Y02zk77GHLSjqiH1xG/eOwMs6jUNfStNl2fDG1\ndx8v5+k9kLglRzySc5tszgD3Bkve8zmRzxxxOKQp5nY8Q4MMZzzJh+PylfI3ekaiAHIJgNFhTnny\nkx2kF7SQMpkmJMGLkhKQaG9I9XJx5Y8xKy5jVOB3ghZDxlZEqqcmPQgY5ghZDSYMQGnd0ARe4ez5\ncS5p7Q6YHlxM9hzn1Ms6X3c/EZgpcMZVE54k62gE9W+zR2n/AAzH0tj1odrEc+Ut9Sv+pNY8+8oV\nnFJMoFOxe0seSTmZ5OTk8zS0pB1abu2ZnMNrFT5HEFR6botJTRI6O2X7qTx90ukll2NTj94Ezel3\nldLSu7aB8O8vrqlJO3n0PkYMP2MqBAIszuHb4iRuGaspxt54kVvD8EYYciGosC0Mw817SkGWXg14\nUjbjBHmJh2MEvJ7EHIx5RzajapAwcnOQTxKZyx45zKUkDus9ZFxu/CP8Faqy7upbb9nPYyo7b2gp\njAkYGfdBGY6qrNtiqRgc5Pwl79mVV/Rx9e+orNxcBagQTg/zmZUVdlFrEKWG5vQTndnoocoXnjd5\n5k9NpzqtZVUvAYnn5DMuUU6L9uGqjxNVpBjG33SwwM9/jNHW0J0jXftTSVMtIOBXb/VyMeUzUmnR\nq0hXQwV0dtlQ97BzkcESo17ey0bWAUpgn0M2+j6rTH6O6lGQ+PcljggcA4JitBqKtH0TR2BA5s3b\nwMZBBnCeWUFxWdIx70wClhNwx9j3pa0D2V6rKgkA9s+cua3UaQW3WadiF1FONuOzAj8/3Su7jTUL\ntsHLAg454/UyuW0UmjmtlK0xut17eKu1TWcbf3yGpy+9tq7n5JxEam8WBADkI2Q2O/nOJaPDfecs\nRxJHAvg6eaUr29jSGqYhlI24GQec+k0NPbSWRXPl7x8x8ZnPYnglRk7Tn7z/ANpFLAjBskEDyMzl\nxXGxDI1I3+rL06sodDabVZTv2tn74/oui0eqVrNY7KlR+qQt3+MxNHWmq1GBjA95iPITV1ttGksq\nFag1j+qB6Tzp1PiO0r16xHWNQqaoLRnav7pX0mq8K1CfeJI4++O6vqNPq767KKimQNwkLjpFdGpr\nKEDkme2E6XUeeUbPQ65/F0pZWy/LAYGVE85qLmGoTazNjjnEgbnvTc7E+hirtzICATzPXhSo8mZt\nMbbRezm5FLcdhyRKg1dl7V6etVUqMM2e8cNbZ7M9WSd3c+ePSR6ZoU1WuKWXqgC/aA7TnmmoxqJ1\nhLZpr0Xx0i7R9Jt16XZycjacRehTU9UFddd4stHIW/n98VoB4mobT6jVYq3cNk7D6cQ8E3aqxNPZ\ng1EgFR7pAM+e3Xs9sejUs1WjNVtdi7lHcDGPzitR1G/qJpr1V29xZ3HGO+JZUq3SkZ++wDPqcTL6\ntQKbE1FeNrHAwZzg7ep1aqOxdRdug1lDfb4b7sj85V19Nnh6FrAoSysMOf8ADn+ctspHTrbLcq77\nRyfUCV+o2NZptAjDCqUVW/w4x/KenFkdJfR5skLf/TLV7adQ6jCE87QeO36wvewqQX4AziPaoL1O\nxQSQqjJ9OJWY7mB7gmacoyl6LHG4xv5KrNhwee0lWdz5PeQt4bHpkRmlr8VvtKoGSSZv4Mtu+kBD\nHl5xfaSGB3nQ841FwBmMJzwIoHjvGJ2gg1TJ5ixJgymRoks4igZIHMEGqZ0HJi1aSBlA9GwQfSOU\n55lTxAveMR2PwEELYMlniIVsCTD5gEMyxUeFb0OIhgAQY2rGwCAWd04bgixZfAzKtr7jmAR1Lll5\n7kxYbFPMjc/aQZvqwJQS0/8ATSlYc2sR6mWq84sx32yjBUbvTl36MBhwYxQ1eSp7HykdHj2Svn+r\nHKcDmUwxqOdgAUAxRWwqd7duMSYdcYkLbAoYMe+DBCu4LNsGOTF27UGwdxOWW++TX+MQSSSTz8YN\nHTyM+UsabTDUOQ52gDsJFacVqx7nnEt1e44YDy5GYB5EPtJ44IxiSqQuDhchVLGFanHOAPUySkgb\nR6EcefMy0eknT77ArxjH3TXvd26NdVbY7sMFdxJ/rD+UzNMuGJGOBLw8V9FYjNxsPOPTn+UsGkma\nlG0jQ6VYE6G9fvb3rsx6D3f0kdPomfpARVJfLZ/ExWjruXpbXEr4aIyqM45bj+cZ0/VOK1R7yGWw\nllxwec9/vnFI6uS4jN6oyp7LUhB2Vndj1yfyEUPHs0wfBNQO3PoZZ6tZVfqq7kUDd9rHzMlXWi1Z\nUZBPbM3/AGdnOKStFVgQq/A8xiVWGl7BWWRftMBwvznbAfDwT3sAlpUFVFiYOXGPnIZUbsqldtJy\nO+Dn74M62KqjHu8SV4Za0+KDPHl/syVfTdWcstLkHkRrdE70vaAkae7wsA7eT/KKsZ24ILe9JKup\n0GjtayllXj3vTmFVN2oY+GjHJ+HJmccGpNyRHT4WtPX9QhdQNz4Hr2MoWpdXbYljDKnBxzzmO1Gm\n12nFO4MpL4UL3zgznsl60JuQ+Nbb4a7m+GTx/OaqvZJX8MWr212r4TbwO/pOtdYw2DsCe5i6FJFn\nihztBKlRwSJLp9RtfFowDk5PaddU6PNK6pi7az4ZPbEqivwxv3Hv+M1raR4ClSG3d1ORgyq+guK7\nlBP3Tnkjfo6YpKPsg9wFJxtOQFCyGn12oo2IPsZPGMZkqKNTVbvaktt8jIbcVksp3YzPO4pXZ64T\nLFd5t0yoTtCJ694vXmy0VDgIrhVEjoKrH0z2r2UbWzOaxXCKc8BgfvnJJbnbb9C7qtTYekXox95N\nuDjzBA/OJ1N5fp+jJXG3Ycn4f95Oyiz2Tx1TxAHG+onG6N1VW76OI5Ubm98Y/qjPaWLSj/1mKuX/\nAAo025uvbKgu3cziUK2p8MWKw7bh2lWlT4Qyv2iZYXTMm4MSnG4czTXTezqylraWp1JRufMEdiPW\nRILJj+Eg5O8ck+kYEYV57ZM7fBx9uxHcd51uMTgGTJHDCbOIJ6GNG4SIAyDJA4MpljN5wMSaniJB\n8z3jEPqYINHM727yGRk4kXsAIyZSDxOeITwvPx8hFKS/fhf3xowBxAJrwc8k+pjVeJBzJCUg8NGI\nZWBjFeAWGPb5xpOMY4lXxMfOdazPnAGWWeWYlmnGaLYwCNpyZAngQc8iQYwUdQcbj8JRzLlTe6fj\nKZ4aAjY01hXT1r8I42cSlQ/1K/KM3ymB/ilWBzxIaq0OFx3ii8UzbmlFDFGVPoP3yVpHuqMducRR\nbA4nFbuT3MFH+KcYH4Zkl1LAEcFwPlEbwvJ84l3zIDPBLNk/Z85HI8Unyl5goTw/DA3DvK50+3UI\nDyGH8Jz3vh6KLGl072BmTBAHPPab2n0Y9g3lkBtrYL73wmTtqStmXAJGJa06ai2itKCXKltw2jjI\nxjn4SN0VSLFVmkp+jrW2tcA67SqY95s8dxMimkmnavFfdWPnmaGv0ep0/wBHBXcFC0tzg5PJkOm0\nKNCajcFY4sUN/WHI/lIn8mdijfWaxWo5GeTNDTUbkyfdGOc8Tj6O+0qEpYgEHkeU19viVj2mtQV7\nZEWa8iXTI1ejdNKzgZwwIwPjLq6VnrJ4ZQOZZL7sgU2OB5oBiITVlbMNVZWp9EPM2rZz8ysqW1C/\nqVVB+wEPun7poNU9NYGQqjgZY8TI1NznqgurADDgY8paOsseoG9zgnja2T+E3q7J5eHdZX7RUUaz\nPpl4mpH3t4dagKu4ndLVns9OkJ8YF2QkBTyPgZzpNZuou8J0RG9xwRzzM2vZlyk+DbNJp77NFv3k\n2jk4PBlHrdd2m62xry4CFwQTwoGTNDqN6jp2g9n3ht42sCf4SXUBu60q2gLnQMpAOcnPP7hI3YiV\nNLZVX0euq5FS633q22ZG0nPf5RHT6ku1TJlVKZPPbv3lvWV1t0PQhGq8ZFHdgAOOR+IlShLaWrew\nVNhmJAYcnHA/fNJ0ZkrZN7EqqFlZFn1jBTgc4Pn92Pxk9JfdqLcBDjGcVpyZQpa7wvBaxM7wxIZc\nfGP6e/h6pkOoQsQVGW3D/vMSlZdaLvik0ksjjBxuIlW1M6Q4sGQp4IlyrQaVtP41usJVs8rk4MhZ\no9L4f/h21F37h/CeST+EztHnwZ2jD+zkqSBuPuzt1NtgG0bmJ4A85dr0vh/2Vg9QJrdPr04ZW8Mh\n/RhODyU7PVH+vUZvXhbR06i/IT6wAoR3bEpILD0vUra4Krv2gdgR3mv9Nnz0zTKQBm7PA+Ey9FUu\no6VqNp9ytvDwPM4GTO0X/wCNM5p/sZ2lqL7C3CZxmWtaVfavicEYJkaqHrrKlW7+UW6nBBU5HqDN\nbX6Oqi6VlHVIqlNpGOZapr8WxFI4BycGVNQckSzp7hXyOD64nT4MfLM0njE6k4eIL6zseYnmSzFi\nSEpBgks5ixDcT9mCEy2Ow5goOdzd5HPkPvMlniAMDcSYbESDiSDCUg8NJBonMkGglDg3MkDzE7ue\n8lulA3PxgWit0N0Anv4nC8UWnMwDrHJnCZycMFJq2BK7HLH5xvG3OYjzgIv1HFaj4SW6Vg5AElvz\nBmhpecDReYZlAwtDdF5hmAMLxbGcJzI5gtFt7K7VArAwB3IxiJtsqwMAKR55lBS1mFXz9I1KfDsI\nuYqFPIxzOeiNuTHVnc2V3GXaLlFqBamXB7hu8ztReQfDG7A/vCdo1NdLhhuJHyhxtGbkbn0g1AHS\nvDDMTYQcHyxKvSuqeFrioQFTUioCO2B+pmZq9Vbr7VHJA7CMVdTp9atiVlgqheV8sTKhyjVnsPE1\nLYAvqA+Ldv3SNlTuQDqKiT62j+YmfRqmsUjUIqgf1hWxz+EbZoumON9jMSewFNkLG/s5XfwW20i0\n0FmtpC+ZL4/hJDSGujxRbVsAzuZsiYGp6fQ3Oka0DzHhn+Znoul6Ip0Fn1CWLXnJsUDOOB2JMqlX\nyXT/AAyuo606cBV1GnsBGdtdK5x84avSlek16puQxBKgFfL4Y/jF6phc9gtZttVeykldpx5ZnpOr\nBKfopUuoYBmrChgM+X6TPkbfDfjMLrenBrR6amO6sHcPX45lXoyFadRbYK8rgoLOOR6cTR1WuRtH\nXXWux3rCKAuAw9f9+k59Hlovp1SvZb4iJjaq5z3Bz6zipSdpnZ448aErq7VKhPAZVQt2+zj05nOs\ndRut6ijNYhsWogGvHGc8S70zTr+0rEpVSopY4K+fHf4TJ1Oi1Fes8V9Kqr4Tdm7Af9xOsZcOenCP\nieN0+mo017lA94kg4nQ+ncr4wbIHHEbX0/WNp0PhNtAypyB8j3iWq2GsWs/ubtu495jZv2dNFdot\n+Do6kry4QvggFM8evb4S9RZo1bc+pRv7pFe0j90q6rSUUtU9+qNa2fYBO73fXjyGZOnTaGwhKNdX\nYfL3Pzh89mXA0F9hGh2bHso35xsPf/eZwjRWVrWhdccj6rgCQt6dq66dqtQyA7vsiVtPXqUtLWV1\nkEHGNqnP3zk2pBJos+HVjKtXx695BWrqfep348gM/wAo2g3bfrUY48lIx/GdsIwd2nbHz5/jPE7i\n+HrhKMl0xPpd1JdZTpKkrYOjFmGMA+mP3zv0WsFXt+mvQbWKPg+Xf9Iy6jT6m0u1z1k8D6vtFnTt\np7PHo1dlt+3A+rwMfGe2OSOurObxy2tFzqAqo95Xx8MTLe++04qAOfjO2W9U1AKXWrt/xoJTPTru\n/tFYPwMVBfJ2ipfRG/SW28lRkHPeQNZRPfr+/MYdCyV/bBb5jEr2VheGcTad8suldZQIwMmczAji\nE9J886OZOLHElzBCXeG49gJESQMoOjiSEjkQzBCeZ3ykMwz+EAYDJgxa/MTuZQNzOhosHtJZ5ghP\ndDMhDOZQdJhmckSeeIBLM5mRzkTp4kAHsYqTPaLgpYUjAnT8IoHiG4wSiecQ3GRyIZEoJbp0NITh\ngEy0jukczhMFHUUPSwatsEdmxGPXdcMvee+e3nOhnHHkZJA2TzmdaXoz32VGpdiV8VmkVoCn3szU\nRAATjDGLejldx7mHCi7or11aZSC1dpPqtmP5Sy2qqqwqJaT6teT/AClrTaLxtyJgnEqPobKrDvE5\nNq6FL2ya9QsZgEVlH/3kxi9QvQsrkspGDzK71bQCsE3Zwwl1Xpmf9RFtXbjaNn+Xn+M1x9JX/ZJ0\nPgPyu0v4vH4bZiuuGxidFfHaFBWaulZJN6IS1rM3G0k/Zm7qeuXazpHsXgbWHa0NkfhiY66c7A0k\nljKdoE2sUPoy8jLmu6pZqdFpNOtCpZpxjxc53fdEaPV6jSPfZ9pr++ONvOeJKusuM47Rrrmv7HI8\n5vwQftGPM0O6Z1K3R623ULUbA9ZQIzfZz93PaZwNltztaWbJJxuIxNTQFUQsa8mVb1JsLKu3mSOD\nG+Ua8812xSaiqs4amwjyBtP5Rza2sou7T1YHYYzK3hMzZJnLKwWAVpf42P6Hnn9lvUazS3ov/gdr\nKNoIs8vwmayVb9yV2Vn4WfpLCV4PvH8Iq1TnIj+PBdoqzSfLJLYx7Wan/XMYrIxG4WEjuWtJzEqw\nAPEgDuMP8fH9E3kaNWuq054qZv8A1P0jD1c/1aj97fpMzbxkiRB97ic5fi4W+xOsc2RL2aX7SyP6\nBc+uf0i/2jcoOxFH3AyoBz3jGrYAYI5mX+Lh/wDkv8jJ9kn6lqdv9Kc/BV/KVrb7rs77Hx8OI4V7\nftYMizjcFUDMeDGvgvnmyt7P4n27LPxkDolHd2l5lYc8CJYnPeb8cKM7zv2Zop/xfunfA/xD8IxT\nJZxOJLYn2c/3/wB0kulLf2n/ALYwR1UEcmJXQk/2n/t/Wd9hOceL/wC39ZbVsZnc5MtGdmVV0BP9\nqB/+v6xg6YT/AG4/yfrLIjVzmCbMpjpTE/03/s/WPp6E1o/5oL/6f6y5V9rmX6QF5EtEc5GT/wAO\ntkj2sf6X6ya/RtiP+cH+l+s2WznMkCcRRN2Y4+jLH/zo/wBL/wCUD9G3H/nB/pfrN1cgRi5IyZaL\nszz4+jTk860D/wBH/wCU430cZf8AzgP/AKX/AMp6M4zFN37SUNmee/4ffP8Azf8A/L9ZH9gNux7U\nP9L9ZvMDIgkMIobMw/8Ah5gM+1j/AEv1ibOkMp51I/0/1npWRivaU7hjvFE2ZmL9Hmakv7WBgdvC\n/WZ37Of3iLc7f8P6z1Dsw0px2xMqtwtNpxzLRpSZlJpWYgG0DP8Ah/WbGk+i51Kg+3Bc/wD+Of5y\nhSwazB9Z6vQ2baBniRISk0YOo+jJobB16k//AIv1il+j7Ef80P8AT/Wb9yNfdlRmTSpgpBEtGd2e\nYfozJ/5kf5P1iT00/wDXH+T9Z6W3TgnkyhfRtbiWhuzFbQsP7YH/APX9ZxdGT/aj/L+s0GrOTOCv\nAko0pMoguSJc0XDHfgDMqUWAN7/aXtI1V/ubSDnvMyya9Omt8GXaV3t3UnIhZZ9ZXXYgwO5mhXQK\nuNxxiVuoaVfZRdUxZs+UxHN5HwPG4Lo2q81agrQoIlnqmmNujWxCucc8zN6bq6qQ/iqd2POI1esb\nUZCMwX0zLSXSatimYLtXGTOOSO6kfOWumV0FgbW5+Md1PabFSsLz5zn5/wBtTp4f1szRXu5JAjXq\nVVHvZkLVWnhyCZzf4g4HadFO+o5OLXGWK1xwxM6K/eOJysO2AQZbr0z+nE9CmjzP2c0oc5GRzLRp\n21neeJCpAK22faEKPFt/pCxHpidVJmWh2jausYHMqat91xAOBNBDTVhWUg/GUtUtfjZTzmoLtlk+\nGZcrAZ3cRmlKA4PJl1Klf3WWcXSV0sT3MkoycrRqM1VMgUJJITETqEOO4lo2WFtoXC/GM20EZfBP\npOb/ACEnqzvH8aTWyMxCApGzJMUE2HlTNPUNWNpRQoiNXfWfcXGcd5vdey6NCPaFWrBWV/FUtwJJ\nwpXlpCraTicp5HZ0hBUNVlMnbsYDnEhWqKcsZJ9jn3RMymkjOjb4C+FkdzIsaUt3dx6RgpZE3AE/\ndKzKGPPeclJM2oyRy+43We6MD0gtbY7RlNSbveOIzH1hAJImlNLhrRtWZSmT7xYMlmYORNY1IpcR\nqymGNWdHeAncSmRyRgODxFJ3jMSmR1f2uJbpcjiVKhzmXaUlIWFfOMxyc8yvtwRzLKgbfSASJk6z\nlYtgMTqnAgo0+sU2e+JMNxJd1zAK1h47TgB7xrrk4kCpDd8CAHKrkyjeWL5I4zNVgChxM24EWhTy\nIBC8t4JA9JiO+Eces3dWUr02T3M87ewwRmLNRRPRgvqEUDOTPa6bQ7UQsfLtPI9Kz7VVj+9PelSK\n1zLEk30rbVryFEq4Jcy3bnI+MXtAfntNHOyraBjMr2VrjcRLzBCeJW1QHg+7IEYupPv8RO/jBj7U\nw2Ys1ZkNoy0qZjkCXunpsywhoiWfYCBmWa38K01gA8zxZJt8PqYoJO2aFK2uucZEi9LppioOJ6Lo\nYoOlyxAPnmVesrUpGxhg98Tz6yhHZM6bRnPVo8glW3UnxW4lmrQ1uC6uPunNWtbOwG75xWjxWcsz\nbflOjlKUbCxxUqLOn6e4YHbuxLnsxbJNeCB5za6ItLrvBB+ctdVrQ07lAB+E4tTlHdkbjGeqPnur\nRhc4K+fEgqumCcCbOpFZu2g+9MrWKVtY8kT14cm3Dl+RgaWxdF+QmMAyzpBdaxAfiZGmbewzPQaI\n1acAMcE85na2mfPcEvYzR1JRc3jP38ppNfp0T3NuflMtaTqdTuDArmaVunrrqByOJq0n7EF/hX1K\nGywMQMTP1laVkFRlpet6lQqhFBZpnhDfcTtOJ7MTdWzjk1ukc09h8T3lA4kt+XO0r3loaIhScdxF\ntoqqQLAfPtOccqcmmblCkmhTktleMyjcorPJ5Pxlqy+sXYlXdUdQTapInDNCEeo92DLOX6skF3Vh\nmGZn3Vs1jEDE3tJ4Fw2Vr2i7tIfEdeCfSfPWdqXT3PGmjEr09rdlzLHsFij3lP3T0eh6bY1AY1gR\nOvV0bwwmDEs079EhCHo8w9TK+D2mhp66Fq948ztmnKsA5GTLGl0uXIYcYmsmW4+xHFT9EK76lqZS\nQZW0yJZvziMOjR9Qyrnv2EcNAKs+64EjkkjUYXIq6nTeGVcMOZCm7w7M8NLGtWvYmM5EqI61Wcrx\n8ZqD2j0uSGrMkCSnJ2ew+QyaRqxSCNUQZY1YwRacxyiaMsmq8x6LmLTiMB5lMj61AMtKQO0pq8fU\n3rBCwp3GWkXKysuMZAlhG92ATPpADAyRxOFvPM5vyBKBi4xxOngSCNiDHIkBxjzxOHDSAJLdoxQc\ndoBZCDw5QUbtTgDzmgygVZzziUdEu69yfIwCn1pRWgUnE81eR4mJu9ds3Xhc5GZhWj66RnWBu/Rv\nTrbcDjlTPYE8AdxPNfRhdxY9pvN7pPM3H0cZvpO3bjI8pRvt54nbbDjgxOM+cpkV4h5xEajUMa9u\nJbFYAiL6vdOZDSMpySY1BheZx05++SA93kwUxEtFdoIMs6LWeHqSzKCDKtlYUBp3Tuq3Kz/ZnmnB\nNHvhkaPQ0dQycBdin4xGs8RrwVfdn4xYvoehtg7StTu8XxCxCieZRpcO/k7Y+1rq9O7mrcPXEfo9\nUx6cB7KHye+IaNvadNeguG3HnLOiA0+gVRZxn0m/H+vUcpZntwT4mqq052IaxJabqIRcai92J8iZ\nT1mtvdtgsBTMpXuHsVRj5zCxtrVnbZN7FjV6pDrN1YOJXtc2g4Yy4Om4rFm7dOV6YmtiybcdpqDg\nuL4N5HJoo0q9fvbePWWjrrbAFZAQPOalK6W/R+CeLAIi3QJXWNuR8Z0lljF99nlWBzRodKetqcAY\nPzk9Q7ByCcr85n6BLNOrEsMGXfCF1W9n5nBQuTlZlppa0VbNMHYNWQs0+m0FCRYQYhdNvxk8S0tR\nT7AOcT6EcyUVE8rwO9mP1q7VVU4z6TN1a/V7QSTLd7XhAWXgTNsudlyuZwyJuapHoxR51lPTV1Nq\nc2EnEdq9iWZqrDSqVsLkjgxVpCj7bb/LBnpyR4bgulzR1aipi/h7Q0n7Q1Wsc4yfQyvoupWV5FuS\nPImQt1YOoaw7cn0nz8kLdntxd4z0uk+kGnXTLXahV+3aUOo66qyzcpmDZriWxgESD2eIVxnJhxc0\nkzKjHHJtDdVeXbf6S70vU73IPORM2zT1ivL2MD6Szp1qrx4dhDYmZwi4UVZGpdPQ9K0yjWNbYF2n\n1mz1JdM2mbGzt5TxtCazUBkF5VPWIarUVH3tTY4zzzJB1FxbMPs00S11qU2AAZAMqamw3JvFRA9c\nReuf69QpyY/U3ahdIlLKAp9BO2OCSVms2a20jGkhOTonpPmE0jliQYxTKZY5Y1YkGNQymSwBxOiQ\nDSQMEJgyzU0qgx1ZghfQ/GTQnPeVK7CDGJZjvKQuMRicyNkTvkt2V4gEfFKuBLIOVJmXdbh8y5RY\nSkFHo3OPONBwJXU+9HAwQde31fBxKmksWpmLHOZPWW405x3mal2EOZC/BV6qwfVEjtmZFh3X8S9q\niC7YJmevNo+cG4nruip7PUj47iadlu4mUNDkaOsAZOBLLPjibRxZxsdpDYflGquDk+cHx5ygWxwZ\nR1N5GeZaucDgTH1bHPEhUcazJzniSL+53lYHJxGeUGjNusXwsA8xCHkSMmuw4ycTDdnpSotLqBXW\nyjzil1Nmwru4iywa3C9o3S6ZrrWUTjSXTr7ovdEqNiXnJwB5TVVq/wBmeGzYye8zun1WV6TUlWK4\nHOIg+Jbo69hc88y7qjCh3hwpXtsAfOPWM0/TktG7fk+gleqjIsDBgQIzRi6tty5A+JnKcXTcWdVN\nXUjT0oNFyrbnZ6TS1vUumNpjUgO/HkJg6vU3tcoYAD5zhUKc7hOUItf2+TtKUX6+COk/pjYAcZnq\nb9Zo30C1lRv+U8nYLEr8RSCufKaOl1mnDIXGeOcy5FJegskWQtQbwFYgSwmpalFTbu5jOrazR3Kg\n0yDd8BH9JOm8PN9ZLeQxJG41ZJtSVna7grqWYAHyl06mp2CoQT8Ji9U1IGoC1ptERp9QUuTbnvyZ\n2cFNWzz24npdUQKcETE1agbQhxky/qdSj1HL84mJdaxtUBuMz1YonKUqZYsfTBCA/vYlKrTDeXLZ\nHeOs0rFGtBEqVWNbaKzZtEs3SdnSMraobpqF1btWz7Fz3k+p9KTSsuy3eDLj6HTVdOd11C7/AEBl\nbR+HqFUWWAEeZnh2+j0pk+k9LGqYK4x84/q3TF6eUeth8pWtW6lfEpvbC+YiLtW+pKLZYWPxnNW3\nZ0dt/wCF3UGm/pbuygvMrUA6d6iq9xLnhbdDeoY/CI11bKmlZm+0Mcz0RlFqjzSi4tsnpNU9hZd4\nXMui+lNKwZgWzPPhWF7bSO87YxUcnM5zwqyxm/Z3WWK+pyDgSdlpdVXfuxKjHe2SBOo208CdlGkT\n2yuDJgxYkgZs8xMHmMUxQjFlMscDxGIYoGTUymSwpkt0WpnScYgyMDHMYj/GVwcxgMAt1mPU5lOt\no4PxKQtgkjygSREq3xkt2c8yoFe0EtLenyFlVj7/AHj6293iClg8v6RobAzEeWcyDuR5wQNZaTWR\nKJPuydlueDmK3g8QEV7E3kmZuSLPvmoxKscHiZZ5tx8ZGbiet0Go/wDB1+uJZ9oUnnGZn6EKNGo8\n8SRB3TRzZpm0EcGQe7iVFJ4wZxicnJ4lsycvt5lO4bo6zB5zKjufMyGkcCwPMB84w15XMFKfgaL2\nE2b82Dymfp3qFy+KuUzIge6fSQA5nOj1o1KLdMOpKaqx4fo01epVJplfUU7BuHlPPVIHcKBzNG1v\n/p7KwOZqMU/YlaqjU6VqVHR9RuTJZTzK2k1CU9NVyv8AWnem3KvSrFK91kdNpvG6aBkD3s4nmcFN\nUzt/XpDU6ir3n243CKsKrpVIbvF9VTw7VQccRPgs1AbPAmvHVUc127O6i4uF+HnBLVC4ZcxdgO0D\nEklOVzOiSMdGizdpmQDzka28I+8MyOdinid357rmSStUWLpmhoVrttDekvanVDTldgzMzR58Jio5\nkWNjrz6zzrDtLp1ll56Lmq1C6h0fZ27zr21bcKvMjQreyhNyhifOU7M12EE8idFH4MPJQ8MSCWPE\ngyErleYoFgu4GQFjt5z0wuPDzZHsxi6lkqZCT8Yzpleje0+K+CZXsqxTu7kxIrAUODyPKYy9RvD7\nNGlKq9TarMWTykSiWsxrACgSq1pZSFB7eUQlllfZiAZ5XBs9amkblLVroQrD4Sh1GlK/CdOCTOU2\n79IVOScyGqAKJljkTlCGsjs5XAvvlOnuSO8r60s9Ok8QYXynb7m9iKbsxV1ztoKC/IQzpjicZvop\na09qcKMgRThXcgLzOpqBvYjjMlVsazlwM+c327ZpVRRu918TqBdvPeS1a4vOGDD1ka+3JnVeji+M\nqjvJ4kR3kpo4HRGLFiTBlMsaDGIYgGMUyog8NJg+sQGk90pkaCJ3MWGnd2RBBytgx6NmUwY5GxIC\n4GGJMMDKu/iMRvOVEJMvvRq+6IotloFjKCw1gA4iHfJkGs+MUz8wU5YeeJENzOFosNzAJ/3jMxBu\nvGfWX3swplFB9YJGaiel0xC0AD0nWc7x6SrTZ9UF+EYX+M0c2WUcAyFr5zEtZ2xEPYSYFDGsxxEO\nMyJbnvOZJkKMRY/fwFlVbCpkvG5lBkL5iR8+0mVIPaRxk95iz1UO0/FmR3ltVsbSsdxxmI09W7kO\nBNiqqsdLclgT6SWdLRUpSwdMewWcdsTmi1KpWQ7NgeUs+LXX0JgE94nGZmU02PSbBjbIq+SOQzW3\npfeGUHHxk1UGge990RZXjaQc5mjRQioC48ovhU1ZnvkryJZCbKY7VUKKga4+mhig3DPwhEtIzLR2\nA7mWKem22pncq/MydlSv1BUxjEu3UIqcsfuM0znZQejUaRDllx8DO6eq2/gMBmTtUW1Ec8Tmnrba\ndh7SUHK0Gp09lCrudTk+sXdTtvCMRkjMvX01j2U2qSGaV+tUFeqbaVI9zMj/AMIu+xFakVPlMqPO\nc0QVrDu4EuJb7P0oCyvO495W0ao13IIBm0zlJDLlV021jzibNA7LlfSTssNIbbjAM7o7rbWbBHA7\nGJMRIVaTUIvCqcxXhMtTBwOJf9oPgEMQDmVbjWaWwx3Tyyl09MHwhpafqWbcBmQ1K5Ayc4ktIjMh\nBJjzpHcBV7zndStnp9w4L1VBr0auGxmOuUN0RMEE48pPrdD16GksfulSiv8A+muzMflL/wCqf+mE\n1dFCpSRH26Q1/bbuOJLSUeIpsJ4BlrUvU5ABPadHPo1Mc8NiM2cDAnb0UP7vaWKKtwzmacuGdO0Z\nM7mRzAGdDzDBJAyAnQYIMBk1MVmSDSkoaD8ZMGJBksxZmhoMlmKBksyihoaTBiVMYDKQcrRgfHaV\ngcSW6CFoPz3kmb3ZVDyRsyIKSZpDIMgWzOZgEiRFk4MCeIsmQpJjlZVQneI5m4xFL9rMFRpVvhRz\nHBwRzKKNgRofiaMNDzZx3imsiy8izZkFHd+TJBosGGZSjGb4yBYjzkS0iWghIZzjbkSNle1wdvBj\nzqKtmMRNl6PgKTOKtHqcrGr4OOxzG01G5diZGfUysqlVzmPosw45iV+zGxb12jfTdJG9vPsDIdJ0\n5tqdShbjMl1m5Toq1BOfnKvS9ZfXqgqEgEYkTbiVou/svUDa4TAzLj1uiAWYxiNstOFFlxUZiLrd\nObdp1XHxhP7ObFstmw7UyJGt9WLB7nEs+NpFo/5sDHlJ+FSNL46akt8p1jJMjTRlXVamzWlthnWd\nkqJcMSDiT1+qajBSxiT3Ms6nSMnRq9UTkswPMrkkKYiyyqvTkPlGI4kemBHpvfecr5Sz1mh7aarU\nQABeZU6QijTagk4JnHyWjppRc1zvbRokrryd4kupBqeqZfv4Uq13W2XVrVdwnPaL6jrDdrid5JC4\nmlLhmu0WdWyP0ekFsEmZ4BpGRYJ1rWbSVqedpkksDOPErBEjlw3pTEYNgILcmN0tTVX4fdtI7iXL\nG0oVdyY+Uu06jR15LbicSJ2jLjRTr02lXSmx1sYZnW0+msp+pof5mao1NHsIK1EqT2kfaaWQKKmX\n4CcJWjcemdTp3CL7gABmjpyviABORO1hCnuhl+cgz+G2RzPE5tume1R5wrfSvPsVR+MpdMpOp0Fy\n+QEj1/WnUJXUAfd5Ml9Gb/DN6PyGHE9qjWPhwVqYurSsibV4MhZRYoyRNXXA1171WZRuttOC+Pvm\nI2+s9PDPv+3yMSS2HgCOvqH98ExXhkD0E7Jpoxq7syoCEJ3PCSnQZASWeIBLM6DIgwBgg0GdzICd\nEpkaDO5i8yQglDVMYGiQZLdKQbuyIZi8zoaAMBhui8zuYKT3ZnN0hmczAJluIpjOk8SDGAcYyAM7\nmcgo9H4k98QO0lmUyM3Q34i8yJaBQ4NAtEBoZgUMLSJeRzOSGqOjc/AEZWoQ5sEoDWWDsFnG1Vjd\n8Rw3TLtlxJwucSdOo2nlSZne0N6LOjVOP6qyDUv22WXsAxO3MnWly3Bqj2lBdc64+qqOPUH8409V\nszkUUL8g35wmGmej072suLlDSxYukFYdtIW+U8m3VdQwxtQfLP5wXq2pVduEI+Ofzi/owsbs2r9N\nTaC1dZQ+hM9DotGKeiFmrDY5ngm6le3ko+Wfzl7/AIn1/sJ0m2nYfPDZ/jM9Ojialj16ixgVwO09\nD1c16f6L1BV3dp87HULh2C/vl+76T66/QLo3Snw17EKc/wAZnV30r9cPQ6vVO+kpPhlVYYkegmnZ\nqkevLYnnb/pHrL9MlDJSFTsQpz/GV9N1fU6Z2ZNhLd8g/nJ40vRtSv2eu6SobX2qqqMDzlDV0WDU\n2EPVxnzmJp+u6vT3Nai1FmGDkH84g9SuLlilZJ78H85dScs9HXpd2mVjcmT5AxK1LWxDtxMZer2o\nQVpoGPgfzjG67qmGDXT/AJT+czqzeyPRXroENfiuwHeMWzpjOqVXY4855u36Qai4KLNPpm29sqfz\nlZupMX3ezacfAKfzjRszw9tdTQah4etQDHbMr1FkOW1VZnkV6iynPs9B+Yb85P8Aa9vGKKBj4H85\nnxEs9vS+8n69GnSSoPIM8bV9INRV9nT6b/K35yZ+k2tJ/oqP8p/OeeX40m7R3jkS9no7FptB3pky\nuuirTLUMVYzD/wCI9Z/06P8AKfzkD9INYT2qHyU/nNLDkXo15IG0+j1TjD6g7fnEN05VGWvwfnMk\n9b1bDB2fgfzlduoXP32/vnRQyfLHkhRrnT1qP6cHEVcygDD5mWdXYfJfwnPan9F/CdFB/LJ5l8I6\nO0IrxW+EPEb4TqeShs7E+K3oIeK3oIFDszoiPFb0E74zeggUWQZ2VfGb0EPGb0EE1LQkgZU8dvQT\nvtL+iwNWXAZIGURqXHks77U/ostk1ZdzJAyh7XZ6LD2uz0WLJqy/mS3TO9rs9FnfbLPRfwiy6s0M\nyOZR9ts9F/CHtlnov4S2NWXSeJEyp7XZ6L+E57W/osWNWWSeYCVfaX9Fh7S/osWNWXQZwmU/an9F\nh7S/osWNWXMzhMqe0v6LD2l/RYsastgzkq+0v6LD2h/RZLGpZJhmVfaH9Fh7Q3oJbLqxUIQmTYQh\nCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIA\nQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCE\nIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgB\nCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQ\ngBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAEIQgBCEIAQhCAE\nIQgH/9k=\n",
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"400\"\n",
       "            height=\"300\"\n",
       "            src=\"https://www.youtube.com/embed/dTUmzAW4t3A\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.YouTubeVideo at 0x7f4a247e5780>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import YouTubeVideo\n",
    "YouTubeVideo('dTUmzAW4t3A')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Impressions from the refinement network\n",
    "\n",
    "The below screenshot shows the output of the VAE on the top and the refined images on the bottom; note that the images appear to feature sharper edges and smoother areas.\n",
    "\n",
    "![Refinement network after 10 hours](doc/preview-refine-20170207-144602.jpg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}