6 - Makemore 5 - WaveNet/6 - Makemore #5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3109d51b",
   "metadata": {},
   "source": [
    "# Makemore 5\n",
    "\n",
    "[Video](https://www.youtube.com/watch?v=t3YJ5hKiMQ0)<br>\n",
    "[Repository](https://github.com/karpathy/makemore)<br>\n",
    "[Eureka Labs Discord](https://discord.com/invite/3zy8kqD9Cp)\n",
    "\n",
    "## Table of Contents\n",
    "\n",
    "- [Network Setup](#network-setup)\n",
    "- [Addressing the noisy loss graph](#addressing-the-noisy-loss-graph)\n",
    "- [Improving the Forward Pass](#improving-the-forward-pass)\n",
    "- [More simplifications](#more-simplifications)\n",
    "- [Going deeper](#going-deeper)\n",
    "- [Dilated Causal Convolutional Layers](#dilated-causal-convolutional-layers)\n",
    "- [BatchNorm Bug?!](#batchnorm-bug)\n",
    "- [Going wider](#going-wider)\n",
    "- [Development Process of Deep Neural Networks](#development-process-of-deep-neural-networks)\n",
    "    - [Rough Roadmap](#rough-roadmap)\n",
    "    - [Challenge for now](#challenge-for-now)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff38178e",
   "metadata": {},
   "source": [
    "Over the last few lectures we've built up processing and training structures to close in on this approach:<br><br>\n",
    "\n",
    "![](./images/bengioetal03.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a83acf84",
   "metadata": {},
   "source": [
    "This particular setup receives three characters. From these, it predicts the fourth character using a single hidden layer and a softmax output. \n",
    "\n",
    "**There's a problem with that**.\n",
    "\n",
    "If we'd just input more characters, the input layer would get wider and wider, while the hidden layer would turn more and more into a bottleneck.<br>Think of it as a glass ceiling for scalability and capability.<br><br>However, to further the model's capabilities, we'll have to input more characters and deepen the model.<br>We could address our problem by more progressively fusing the individual inputs, instead of doing that all at once between input layer and hidden layer.\n",
    "\n",
    "![](./images/convolutions.png)\n",
    "\n",
    "> Too many individual inputs into a single layer are just not that scalable. We want to avoid that from now on.\n",
    "\n",
    "We replace the wide-input-to-hidden bottleneck with multiple layers, progressively fusing the inputs.<br>\n",
    "This scaling approach was also described in the paper about [WaveNet](https://arxiv.org/abs/1609.03499), an audio-based language model.<br>Fundamentally though, the structure still remains the same as with [Bengio et al. 2003](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "443fa6db",
   "metadata": {},
   "source": [
    "For a start, we'll take some code from Makemore 3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fcc59144",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn.functional as F\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7cb41500",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word Count: 32033\n",
      "Max Word Length: 15\n",
      "['emma', 'olivia', 'ava', 'isabella', 'sophia', 'charlotte', 'mia', 'amelia']\n"
     ]
    }
   ],
   "source": [
    "# Read in all the words\n",
    "words = open('../names.txt', 'r').read().splitlines()\n",
    "print('Word Count:', len(words))\n",
    "print('Max Word Length:', max(len(w) for w in words))\n",
    "print(words[:8])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "97c88455",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{1: 'a', 2: 'b', 3: 'c', 4: 'd', 5: 'e', 6: 'f', 7: 'g', 8: 'h', 9: 'i', 10: 'j', 11: 'k', 12: 'l', 13: 'm', 14: 'n', 15: 'o', 16: 'p', 17: 'q', 18: 'r', 19: 's', 20: 't', 21: 'u', 22: 'v', 23: 'w', 24: 'x', 25: 'y', 26: 'z', 0: '.'}\n",
      "{'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 6, 'g': 7, 'h': 8, 'i': 9, 'j': 10, 'k': 11, 'l': 12, 'm': 13, 'n': 14, 'o': 15, 'p': 16, 'q': 17, 'r': 18, 's': 19, 't': 20, 'u': 21, 'v': 22, 'w': 23, 'x': 24, 'y': 25, 'z': 26, '.': 0}\n",
      "27\n"
     ]
    }
   ],
   "source": [
    "# Build a vocabulary of characters map them to integers\n",
    "chars = sorted(list(set(''.join(words))))  # set(): Throwing out letter duplicates\n",
    "\n",
    "stoi = {s:i+1 for i,s in enumerate(chars)} # Make tupels of type (char, counter)\n",
    "stoi['.'] = 0                              # Add this special symbol's entry explicitly\n",
    "itos = {i:s for s,i in stoi.items()}       # Switch order of (char, counter) to (counter, char)\n",
    "\n",
    "vocab_size = len(itos)\n",
    "\n",
    "# Showing the two mappings, they really just are mirrors of one another\n",
    "print(itos)\n",
    "print(stoi)\n",
    "print(vocab_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ef90dd5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Randomize the set of words (with reproducibility)\n",
    "import random\n",
    "random.seed(42)\n",
    "random.shuffle(words)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fd87ed2",
   "metadata": {},
   "source": [
    "We will first apply our usual `block_size` of $3$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9beb0853",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([182625, 3]) torch.Size([182625])\n",
      "torch.Size([22655, 3]) torch.Size([22655])\n",
      "torch.Size([22866, 3]) torch.Size([22866])\n"
     ]
    }
   ],
   "source": [
    "# Build the dataset\n",
    "block_size = 3 # Context length: How many characters do we take to predict the next one?\n",
    "\n",
    "def build_dataset(words):  \n",
    "  X, Y = [], []\n",
    "  \n",
    "  for w in words:\n",
    "    context = [0] * block_size\n",
    "    for ch in w + '.':\n",
    "      ix = stoi[ch]\n",
    "      X.append(context)\n",
    "      Y.append(ix)\n",
    "      context = context[1:] + [ix] # Crop and append\n",
    "\n",
    "  X = torch.tensor(X)\n",
    "  Y = torch.tensor(Y)\n",
    "    \n",
    "  print(X.shape, Y.shape)\n",
    "  return X, Y\n",
    "\n",
    "# These are the \"markers\" we will use to divide the dataset\n",
    "n1 = int(0.8 * len(words))\n",
    "n2 = int(0.9 * len(words))\n",
    "\n",
    "# Dividing the dataset into train, dev and test splits\n",
    "Xtr, Ytr = build_dataset(words[:n1])     # 80%\n",
    "Xdev, Ydev = build_dataset(words[n1:n2]) # 10%\n",
    "Xte, Yte = build_dataset(words[n2:])     # 10%"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8eeb560e",
   "metadata": {},
   "source": [
    "Given a triplet (`block_size`$=3$) of characters, the fourth character is the one we want to predict:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6c1692d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "... --> y\n",
      "..y --> u\n",
      ".yu --> h\n",
      "yuh --> e\n",
      "uhe --> n\n",
      "hen --> g\n",
      "eng --> .\n",
      "... --> d\n",
      "..d --> i\n",
      ".di --> o\n",
      "dio --> n\n",
      "ion --> d\n",
      "ond --> r\n",
      "ndr --> e\n",
      "dre --> .\n",
      "... --> x\n",
      "..x --> a\n",
      ".xa --> v\n",
      "xav --> i\n",
      "avi --> e\n"
     ]
    }
   ],
   "source": [
    "# Show the dataset's layout -> Three inputs, one expected output (given the three, expect the fourth)\n",
    "# We have (as seen above) 182625 of these 'three, give fourth' sets of characters\n",
    "for x, y in zip(Xtr[:20], Ytr[:20]):\n",
    "    print(''.join(itos[ix.item()] for ix in x), '-->', itos[y.item()])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03156ba4",
   "metadata": {},
   "source": [
    "## Network Setup\n",
    "\n",
    "With Makemore 3, we started to develop the structure of Linear Layers and applied Batch Normalization to them.<br>\n",
    "The code below is the same, but now compartmentalized.<br><br>\n",
    "**This is on purpose.**<br><br>\n",
    "Think of class instances as building blocks making up a network structure.<br>\n",
    "This is easier to handle in the long run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b3aec68b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Near copy/paste of the layers developed in Makemore 3\n",
    "# -----------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.linear\n",
    "class Linear:\n",
    "  \n",
    "  # fan_in: Number of input dimensions\n",
    "  # fan_out: Number of output dimensions\n",
    "  # bias: Whether to add a bias term or not\n",
    "  def __init__(self, fan_in, fan_out, bias=True):\n",
    "    self.weight = torch.randn((fan_in, fan_out)) / fan_in**0.5  # Kaiming init\n",
    "    self.bias = torch.zeros(fan_out) if bias else None\n",
    "  \n",
    "  # Performs a forward pass if called (syntax: l = Linear(x))\n",
    "  def __call__(self, x):\n",
    "    self.out = x @ self.weight\n",
    "    if self.bias is not None:\n",
    "      self.out += self.bias\n",
    "    return self.out\n",
    "  \n",
    "  # Returns a list of the parameters (syntax: l.parameters())\n",
    "  def parameters(self):\n",
    "    return [self.weight] + ([] if self.bias is None else [self.bias])\n",
    "\n",
    "\n",
    "# Mimicks torch.nn.batchnorm1d\n",
    "class BatchNorm1d:\n",
    "  \n",
    "  # dim represents the dimension along which the normalization is applied\n",
    "  # PyTorch's BatchNorm is also performed along just one dimension\n",
    "  def __init__(self, dim, eps=1e-5, momentum=0.1):\n",
    "    self.eps = eps           # Small constant for numerical stability\n",
    "    self.momentum = momentum # Running mean/variance momentum\n",
    "    # Batchnorm1d behaves differently during training and production\n",
    "    self.training = True\n",
    "    # Parameters (trained with backprop)\n",
    "    self.gamma = torch.ones(dim) # Scaling\n",
    "    self.beta = torch.zeros(dim) # Shifting\n",
    "    # Buffers (trained with a running 'momentum update' outside backprop)\n",
    "    self.running_mean = torch.zeros(dim) # Batchnorm mean\n",
    "    self.running_var = torch.ones(dim)   # Batchnorm variance\n",
    "  \n",
    "  # Forward pass (syntax: bn = BatchNorm1d(x), where x is a batch of inputs)\n",
    "  def __call__(self, x):\n",
    "    # If we're in training mode, calculate the batch mean and variance (for active normalization)\n",
    "    # Otherwise, use the running mean and variance (for passive normalization)\n",
    "    if self.training:\n",
    "      xmean = x.mean(0, keepdim=True) # Batch mean\n",
    "      xvar  = x.var(0, keepdim=True)  # Batch variance\n",
    "      with torch.no_grad():\n",
    "        self.running_mean = (1 - self.momentum) * self.running_mean + self.momentum * xmean\n",
    "        self.running_var  = (1 - self.momentum) * self.running_var + self.momentum * xvar\n",
    "    else:\n",
    "      xmean = self.running_mean # Use pre-determined running mean/variance\n",
    "      xvar  = self.running_var  # (pre-determined = calculated during training)\n",
    "    \n",
    "    xhat = (x - xmean) / torch.sqrt(xvar + self.eps) # Normalize to unit variance\n",
    "    self.out = self.gamma * xhat + self.beta # Scale and shift (gamma and beta are learned)\n",
    "    return self.out\n",
    "  \n",
    "  # Returns the list of this layer's parameters (syntax: bn.parameters())\n",
    "  def parameters(self):\n",
    "    return [self.gamma, self.beta]\n",
    "\n",
    "\n",
    "# Mimicks torch.nn.tanh\n",
    "class Tanh:\n",
    "  def __call__(self, x):\n",
    "    self.out = torch.tanh(x)\n",
    "    return self.out\n",
    "\n",
    "  def parameters(self):\n",
    "    return []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "460a9721",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42); # Seed for reproducibility"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50715e8c",
   "metadata": {},
   "source": [
    "With the above code, we can now build the actual network structure. We'll do so just like before, with \n",
    "- an embedding layer with of size `(27, 10)`, \n",
    "- a hidden layer of size `(30, 200)`,\n",
    "- a Batch Normalization layer across the hidden layer output's second dimension,\n",
    "- a Tanh activation function,\n",
    "- a linear layer of size `(200, 27)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5a483989",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12097\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10    # Dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "C = torch.randn((vocab_size, n_embd))  # Random but unique n_embd-dimensional embedding vectors\n",
    "layers = [\n",
    "    Linear(n_embd * block_size, n_hidden, bias=False), \n",
    "    BatchNorm1d(n_hidden), \n",
    "    Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "]\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident (in beginning)\n",
    "\n",
    "# Print the number of parameters in the model\n",
    "parameters = [C] + [p for layer in layers for p in layer.parameters()]\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a82a6d8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.2844\n",
      "  10000/ 200000: 2.3177\n",
      "  20000/ 200000: 2.4191\n",
      "  30000/ 200000: 1.8858\n",
      "  40000/ 200000: 1.9077\n",
      "  50000/ 200000: 1.5355\n",
      "  60000/ 200000: 1.6192\n",
      "  70000/ 200000: 2.5743\n",
      "  80000/ 200000: 2.0838\n",
      "  90000/ 200000: 2.1183\n",
      " 100000/ 200000: 2.2409\n",
      " 110000/ 200000: 2.4485\n",
      " 120000/ 200000: 2.1119\n",
      " 130000/ 200000: 1.8765\n",
      " 140000/ 200000: 2.1631\n",
      " 150000/ 200000: 1.9839\n",
      " 160000/ 200000: 2.3134\n",
      " 170000/ 200000: 1.8970\n",
      " 180000/ 200000: 1.7244\n",
      " 190000/ 200000: 1.8793\n"
     ]
    }
   ],
   "source": [
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    emb = C[Xb]                         # Embedding characters into vectors\n",
    "    x = emb.view(emb.shape[0], -1)      # Concatenate the vectors, \"stretch them out\"\n",
    "    for layer in layers:\n",
    "        x = layer(x)\n",
    "    loss = F.cross_entropy(x, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (mini-batch SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "59944841",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABe2klEQVR4nO3deVxUVf8H8M8AMoDKosgqCohKLoCKEu7mJJi/0qwnNEujsnKpfKg0WjS1wszHbDHtsVzKSqvHbNGoJMlU1ETJnRRF3ABBWRVQ5v7+IEYGZoa5s9/h83695vWCO+eeOXcGZr5zlu+RCYIggIiIiEhCHKzdACIiIiKxGMAQERGR5DCAISIiIslhAENERESSwwCGiIiIJIcBDBEREUkOAxgiIiKSHAYwREREJDlO1m6AKSiVSly8eBFt27aFTCazdnOIiIhID4IgoLy8HAEBAXBwENenYhcBzMWLFxEUFGTtZhAREZEBzp07h44dO4o6xy4CmLZt2wKoewLc3d2t3BoiIiLSR1lZGYKCglSf42LYRQBTP2zk7u7OAIaIiEhiDJn+wUm8REREJDkMYIiIiEhyGMAQERGR5DCAISIiIslhAENERESSwwCGiIiIJIcBDBEREUkOAxgiIiKSHAYwREREJDkMYIiIiEhyGMAQERGR5DCAISIiIsmxi80czeVmrRKvbzkOAHhxdDhcWjlauUVEREQEsAdGJ6UArN2di7W7c1FTq7R2c4iIiOgfDGCIiIhIchjAEBERkeQwgNGTIFi7BURERFSPAYwOMpm1W0BERESaMIAhIiIiyWEAQ0RERJLDAEZfnANDRERkMxjASMwbW44h7p0duFZz09pNISIishoGMDo0nMNbfbPWau1oaNUfZ5BdUI7/Hbhg7aYYZfn2U4h7ZwdKrtVYuylERCRBBgUwy5cvR3BwMFxcXBATE4N9+/ZpLbt27VrIZDK1m4uLi1oZQRAwd+5c+Pv7w9XVFQqFAidPnjSkaWYz5v2d1m6CGqVS2mNab/+cjeyCcnz8xxlrN4WIiCRIdACzceNGJCUlYd68eThw4AAiIyMRFxeHwsJCree4u7vj0qVLqtvZs2fV7l+8eDHee+89rFy5Env37kXr1q0RFxeHqqoq8VdkJpfLq63dBLt0Q8ktGqREEAS8lXoC32VJuweQiKRPdACzdOlSTJ06FYmJiejRowdWrlwJNzc3rF69Wus5MpkMfn5+qpuvr6/qPkEQsGzZMrzyyisYO3YsIiIi8Omnn+LixYvYvHmzQRdlLj/8ddHaTSAbIggChBaW4XDXqWKsSM/BsxuyrN0UImrhRAUwNTU1yMzMhEKhuFWBgwMUCgUyMjK0nldRUYHOnTsjKCgIY8eOxdGjR1X3nTlzBvn5+Wp1enh4ICYmRmud1dXVKCsrU7uZg6xRJrunvzxolsch6REEAQ98lIH7V2a0qCCmuJI9kURkG0QFMEVFRaitrVXrQQEAX19f5Ofnazyne/fuWL16Nb777jusX78eSqUSAwcOxPnz5wFAdZ6YOlNSUuDh4aG6BQUFibkMIqOVXLuBP3OvIvPsVRRVcCIyEZGlmX0VUmxsLCZPnoyoqCgMGzYMmzZtQocOHfDRRx8ZXGdycjJKS0tVt3PnzpmwxdLQkr7126KGzz63nLB/giBgx9+XcbHkurWbQkT/cBJT2NvbG46OjigoKFA7XlBQAD8/P73qaNWqFfr06YNTp04BgOq8goIC+Pv7q9UZFRWlsQ65XA65XC6m6UREBkvPvozEtX8CAHIXjbFya4gIENkD4+zsjH79+iEtLU11TKlUIi0tDbGxsXrVUVtbi8OHD6uClZCQEPj5+anVWVZWhr179+pdp7nwi7UF6OhI2nu6GMmbDqH0+g3LtYdIg4zTxdZuAjXCXmgSPYSUlJSEVatWYd26dTh+/DimTZuGyspKJCYmAgAmT56M5ORkVfkFCxbgl19+wenTp3HgwAE89NBDOHv2LB5//HEAdRNlZ82ahddffx3ff/89Dh8+jMmTJyMgIADjxo0zzVW2ELaSbM9UEv67B1/uO4fFqSes3ZQWqaiCE3bJNqUeyUfk/F+Qnq09fQfZP9EBTEJCApYsWYK5c+ciKioKWVlZSE1NVU3CzcvLw6VLl1Tlr169iqlTp+K2227DXXfdhbKyMuzevRs9evRQlZk9ezaefvppPPHEE+jfvz8qKiqQmpraJOGd1KQeycfrPx4z+JtC9c1a/HWuRGfSuvr7/pd5Ht1fScXX++1vPlDelWtqv5/IL0MxP1zN6t1tJxH9+jas3cVEg2Qeb6WewOTV+1BrQFLOp9ZnoqzqJh5Z86cZWkZSYdAk3pkzZ+Ls2bOorq7G3r17ERMTo7ovPT0da9euVf3+zjvvqMrm5+djy5Yt6NOnj1p9MpkMCxYsQH5+PqqqqrBt2zZ069bNsCuyIU+tz8THO89g2voDBp0/4/MDGLt8F1buyNF4/7vbTqLv67/ibHElnvv6LwDAC98cAmB/vTH1/i4oR/yyP9Dv9W3Wbopde2fb3wCA1344pna8cWoBa/hibx4GLfoNpworrN0UMsKK9Bzs+Psydpy8bO2mkERxLyQDbT18Cdn55Vrv//3vW/+UqUc1Lwdvzrbjdd2ja3blNrlPQN2HTMm1G3j752y1+04WlKP7K6lI3nTYoMe1NRk5t+Yf7D1zxYotIUu7WavEo2v/xPtpdVuL/HWuBC99exgXSq7j5W/t4++7pbtxk9m4yTAMYHTQ9GVz27ECPPVZJqZ/fgBxy3ZoPXfKau37Q5nbh+l1PTZf7suzyuNfLLmOxakncKnUNEtOb0p83ycy3C/HCvDbiUL859e/UVhWhbHLd6nuM2TogbQ7d+Ua/rsjBxXV3OmepEHUMmoCHv90v0nrq75Zi7Ef7EJkR0+8dX+EQXWcv2pbuSmmrN6Hk4UV+O1EIVJnDbV2c0jCrtfcGgo922guFJnWXe/9gfKqmzhVWIHF90dauzlEzWIPjJGe/2fuiaF+O16IE/nl2GjE5NuscyVGtUGbiuqb+OVoPqpuiJtPc/KfuQkndAyxafL1/nO498NdKCy3nU08WyKu7FD3aUYuvj1o/5tXllfV9bzszuGScZIGBjBG+ibzPF7ZfFhtpVDpNf3ylnz15zlM+9ywCb6WMG19Jp74LBPzG03kNKWGgwAvfHMIB/NK8HZqttbyZH62tLLD2oNEq3eewdzvjorajb6oohpPfrafgSCRmTGA0UHfFRfr9+Qh7UTdm9Xy7acQueAXvc6b/b9DepW7XF6Nf63cjSuV+u2584uBk4Yb++NkEQBg45+WnUtzrcb6K6iuVNbg/hW7tV67OZNo1SoFPL5uP97+mflvrG3Bj+KD94U/HsPPRwtsKhAkskcMYEykPlts4xVB9QRB+OeD6U8s0VJGlz9zr2Lpr7fOSz2iPUipNHMAkJFTjIN5V836GNb2zq9/Y//Zq5jzv8PNBiv1YW5RRbVJhr/2nC7GtuMFWL5d8/J5sm35pbY9BCo2+D535Rqz3pJNYgBjITO/PIidp4qw7XghPth+yqA6KqpurQ4w53LiK5U1eEtL9tsrlTWYuGoP7v1wt1515VyWZq6O8qpbw4Cj3/0DT395UGf5WqWA6Ne3YcAbaaLnDDVWY8PLSq2fBYaMMfe7IxixJB2Veq40+viP0xiyeDsW/njczC0jEo8BjAml/KT9n3zLoUuobvDBlltU2aSMsR98pjL7m0NYka7527/YDLjXbWA4yFgn8svxw18XdZZpGHSImS9BujX85n/+KlchGevTjLPILb6GTXpOSk75qe6LzGpmZBZFV/Z0Mh0GMCb00e+n9S5bv7NtQ+GvpqqStiV9lWWqZqncqNXvm/0BOx4eulmrxAe/nbTra7RX9pKYsbGv95/Dgh8M33LEIBwS0qnmphIFZYYNBb7w9V8Y/NZvar24ZB4MYKzkjIYeGACYuGoPqm7UYtOBpt+QjHnLmf/DUYS/morTFh7SeWOLbXU9f7EvD0t++Rvj9RwCM4cjF0rx6No/cSK/zGptkIovGiRjrLqhHoA3nGN/sqBc79V/+rJUQPHCN4ewetcZ7DxVZJHHszW2sD1FY/d8sBMxb6bh6MVS0ed+nXkeF0ur8F2W7l5bazhbXIlvMs/bTRJIBjAmYso3O1O+bxZVVON6TS3W7MpFrVIweP6NoTJOF+Ohj/fiz1zNc3YsPTnwZIH5A7hnNhzUeV3jV+zGbycK8eCqvSZ7TH26rOf/cFTr3CZDmHvIM+dyBQ7mlTRb7kR+Ge58Zwei3/jVpI//741ZJq2vOfULAcj66nNYfd/M0LHUDHs7Hc9//Re+2HvW2k0xCQYwdi769W3ou1DcG7up563sPFWEf63M0Fnmqp5LxKXgYF4Jjl7U3rtSP1/mSmWNScbKn/nyIO74T7rOgCK/tAprduViRXqOyV7f3q/9bNY5TvqmDdj5z3L/G7WmDYY36/EN+kxRJW42MzRbVFGNx9b+iV+PFZiqaURG2ZdrH0PoDGBM5IAe3xSt5XqDD7ZThRXNfmheN+M36y2HLuHeD3c1OT5w0W/6VWCjY/f1uzfXq9FzvtGyRucZ4vu/LiK3+JrOxGkN5z8JJkoPd6NWwOELdV3sl8ursXrnGZRcs+1ANCOnGClbj5tkt/ZvD57HiCXpePiTfViRnoOzxZqHhd/YchxpJwox1YBtSP4uKMeSn7NRJnI+RWFZVbOBFZHUMYAxEWttnCjWofOlCH1pq+jzNL0ZGjL8M+OLA2rDApuzLuJMUaVZgyZL+O8O/SdwN7TSwPM02XP6Cq7X1GL59lM4WaC+jYPSRIGftukKiWv3YcGPxzDLwsMuYk1ctQcf7TiNdbtz1Y7XT7gsvX4Da3ad0Sufz6oddStzMk4X463UExj1jubNXQ1ZlXbuyjUMXbwdo97ZgQ+2n8JCEdmwD50vwYA303B/M72etqpuDuB5HLlQiv/8ko1CAyfTkv3jZo4SYq3OB6UAdH81Fesfi4F3G2fV8Zg30/Cv6I54IS5cv3o09PxcLq/GiCXp+jfGBif8mZyBl5h6JB9yJwd8tOM03v45G7mLxgCo+zAc9na6qtzh86V4c+txJN91G24PbW+CBgNHLtQNmaVnXzZJfeaWW3xrSfbSX7Lx3m+n8OGkvth04AK2HS/Ahn3i9yarNmH+nje3Hkdeg80r63u69PH1/vMAzLdHmrm9vuUY1u+59YVw16kibJo+yIotIlvFHhgbZKoufl3EvrnVKoUmkxoLy6tFZYut326hWSaIUc5duYbH1+3XOnm4OVsPN810XHNTieOXymw6K6mm13XZtpNqv0/6eC/+Ol+KCf/dg6c+yzTrioT80iqDlpOKSebXcKPFbw+eF/1Y7/1WN7F97ndHkHaibp5KdoG4jUi10TSsNOOLA0hcs0/n35E1kxla6zuCUlmXrbzx/55Vh+dt91+dwACmxRq3vOk8FHPTd5XFlkOX8OwG3ZlvmzPzy4PYdryg2cnD2miawzJtfSZGv/sH1u85a7L3NWt8UN1sELCkHs3H1sOXzPI4heVVuD0lDb1f029vsHpzvjmEbq/8hHNXmk9cd/7qNbUJ0//eaNzu8Kb20rfquWuqbtRiy6FL2J59GeevXle7T/ZP5P7a90f1D/b1oGm+jzF/v2+lnsC8744YUUNTgiDg7g92YviS7Zy784+zxZW4VqNfxuSWigGMBRmb76AlfRkwNofCBQOztlbfrNW6l039h8onO9WzktpiHgsxGk5A1pQh2lB/nROfQwMANu6vG77RJ/vr1UrbXnrcOHeNPtY2mp9jjNOXK9D9lVTM+Ua/jWObU3NTiRXpOViXcRYXS643f4KelAJw9GIZzl25jrIq+/jQNub9+siFUgx7O11t6FessqobGLd8F9KO2+/qNwYwNihumebJgOYcuii9dgOKpb/jnV+NXxUjZf/33k7cnpJm7WZY1OnLt4IWsRmgxcRuxy+VYcSSdPx4yPq5Nb7Ym4cikdtiSFH95PL6oFBfv50o0Lhha8PJ4DebWbZ+7so1i2yPkrL1OEa987ve+zuZy5XKGoMS32nyy9G6YTRjtiUZvewPZJ0rwWPrxK9+kwoGMBbUuDtZm3NXNH+zMXUStqQGc1pe2nwYpwor8G7aSa3l88uqMPt/pvkmJ9YDKzMMTv//6uYjSD2i3zDJyULrbD55o1aJr/af02vYRJeGMa5SKeDXYwWiUqLrm3vFEDO/OIAzRZWY+YV+w4P1E4O1kRk5WWq5hZM6amKLy87PXbmOR9fu13vDVk2OXizFkMXbMfI/vzdb1tj+y492nMbfBRX4SmSQZmr9Xv8VY97badU2NHTBhD1ktooBjAUZu8mfqSYW1mu4oduWQ/p9wGvKjHq5vBpbDl3Se68lQ+zLvaI1/X9heRXi3tmBNbvOYHdOEZ758iCKG3wQf7bnLJ5afwBHLpTisz2myUD50e/6T17Wx+qdZzD7m0MYsni7zjf0vOJrOHKhVONzfVMpYF+DScvPbszC1E/320yKelvb2LO5HgRLaLhliL69WVU3ag0KfEz9/OtabFD/QW7JD1FzzEUXU6UNz+23W1xG3cJdrazB9M8PGFXHmPf+QGF5NWbHd8f04WEmapn+3vn1JLILyjG/mVwZ//e+ab4d1QoCVv1xa37GlcqmganYN7Pd/2ziqcuVyhoMfXs7AKCHvzu2PjtE7f7GQyLN7aBtbddqbmL9nrMY1cMPwd6tTVavUinAwUH89/qiihqTrMBpOMySefYqAj1dja+0gZg301B6/QY+nhyNP89ewb8V3eDSyrHZ8+b/cNSk7fg+6yKeHtm1yfEKKw/l2AxGNGbHHpgW7p1tfyPjdPMfnroU/tOztM1KqdJNkVVVjMbvS4qlTecsnSosx+TV+0z6nDTciPPYJdNuBLny9xzUKgW1nqvGvsu6oPfwjz7e+ukE3tx6Anf8J91kdQJ1XfnvbtM+FKqLsfHLqh2n8WejNO369kLo+9j1q/ke/3Q/Pvr9ND7Ucyhsc1bTDWKN8Z9f/8Z2Ddmfa0X2bH3/10WbTk1Q769zJZj/w1G72LNK37+1mptKnDdwQYQlMIBp4cpM+M9o+29B6hannsBxEwcC9eb87zB2/H0Zj+uRPn7WhoNqEybFrGoydF5QY4t+OoH7VuxGuY4VIM9uyBJVZ3OrVPaeqRvuUgp1KyYO5l01yQfZ1Ws3mmztoC9jhyHe2Gr53dd35RTju6wLGocVG/YGmXqvKABIXPOn0XV8/9dFxL/7hwlaY15jl+/Cml25WPST5V/jxs4WVxo9X04f96/cjcFvbcdeI7/kmgsDGDIZCXyJUvNheg5G28Ab5+asiwYvHX1w1R6TtcOQzK26Qq3G9RU16t2p3/EXAEYt3YF7P9wtasPDWqn9wZlJ5tmreHZDVpPl/YD6cI4hCQvNmeSwoVNWmjxviL8tsKO9rmf9ek0thr2djiGLt4uad2jIl4ND5+tWVX2TKT5BpCUwgCGTyTpXojPrqrbN7kRp8E9YqxRw5EKpVbOWWtP27EKD8oyIZejy1IYZcgH1pH295v2sdl/+PyulUo82zYCsTebZqzhdJJ0PPqBuiMlcdG3mSdJQXnUDE/6bgfU6FhsUN5hzp+/2FYIgYNLHe41un63hJN4WbrORCeMa+2q/9kj9/d+MX7baMIvsm1uPa/zWaahD50v0Kif2i0zVjVq0cnTAntPFeo09NxzWqqi+iXW7c3FXb/8m5UzRfa+PO5f+jgIRK+iOXCiFp1srtWONg0xTTfTUNawlCILNJRn8j4XzLGnaEgOoW/U2OTZY9XvD+VXafPzHaTx0e2et9xdVVOO3E4UY2rWD6HbaKmOGNA058+M/zmDPacO2P9Hl+o1avRYKSA0DGDKpT/4w3zdMAGorjUwVvNQqBTg6yHDPB+bZXiFi/i9wlMn03nF70U8nVD+/seUYvtx3Dkt//Rsbn7jdLO1rzkUtmYm10bTaq9/rv+p1rinnabyy+QjeuLe3yeozN11Xrk8gJuaDr+FmlkqlgDv0yNfy+pbjkOtY7ZTwUQZyLldiTIR6sL3pwHn0D26HoHZuerfPGD8fzUd+aRWmDAy2yOOZkqW3Dqi6UYvzV68jzKdNk/sa5tWx1cFaDiGRSYn9sLMFG/80bwKsmptKvYOXxvb9M9HVUnMRzEXX5OCGTLn0+/O9eTrvN3TT1B8PXcSFkusm3/Rw+ucHtM5DulpZg3e3nWx2RUixiOzCwS9uQfCLW7BCRE6jVzdr3wMp55+Mzj8fUe/1SfrqLwxZvF3vxzBU/cvx5GeZmPf9UbNN0DeFWqWAOd8cwoZ9uv9GNTF0i46Gvv/n/+yBjzKgWPp7k9WSl0qvY7aJtp8wJ4MCmOXLlyM4OBguLi6IiYnBvn379Dpvw4YNkMlkGDdunNrxRx55BDKZTO0WHx9vSNOIRNuVYxuJ3jTJuWy6vYnINGZ+cRCDFv2mtomkoRrPC9O2iWl+WRXe2fY3Br+lOxC4aUCg+/bP2aLPkYLiCtvLclzvl6P52Lj/HF7cpF929oZmfGFc3q6G6ifpNtxq4tyVa3hwlTTmy4gOYDZu3IikpCTMmzcPBw4cQGRkJOLi4lBYqHsCWW5uLp5//nkMGTJE4/3x8fG4dOmS6vbll1+KbRqRRbSEFN36srEpJpKzLkN9subZYtvNuaFN4yW2hgRRUpJ59ioulV7H6csV+EFHDpvG8752/H0ZL397GNdram0yl0x+aRU++j0HD32yF2dMuKmrOYkOYJYuXYqpU6ciMTERPXr0wMqVK+Hm5obVq1drPae2thaTJk3C/PnzERoaqrGMXC6Hn5+f6ubl5SW2aUQG0XcbBZKmrHMlGrfxsPYqbGs/vqkk/Nd0S/ltgT6rGkct3YE7/vM7nv7yIH7RsvS/8aKFyav34fO9efjvjtM2GcA8uGoPUn46oTGItotl1DU1NcjMzIRCobhVgYMDFAoFMjIytJ63YMEC+Pj44LHHHtNaJj09HT4+PujevTumTZuG4mLtM6arq6tRVlamdiMiy7tvRYZNL989mHcV45bvQv83tjX50DA2AzXdYuvbB5y7cg0fpp9CmY40D/V+O3Hr71lbkFne4Hr/0jJvSVuQsvXwJaQ0mKivjzNFlRqTVpoyg/FpifS6NCRqFVJRURFqa2vh6+urdtzX1xcnTmh+QXbu3IlPPvkEWVlZWuuNj4/H+PHjERISgpycHLz00ksYPXo0MjIy4OjYdNZ7SkoK5s+fL6bpRDpZejsCe/KIhZZzG6Lhjsrxy9S3fCiuqMHHZl41Z2lidh43pcZ5fTS5VHod/h51+0Lps2zblOonEWfnl+PdCX10lm2YvfhKZQ3OFleic3vT7dVlyKa8I5akA4DGlYhVN2pxUyng2wPn4ezkgIT+nYxtomSYdRl1eXk5Hn74YaxatQre3t5ay02YMEH1c+/evREREYEuXbogPT0dI0eObFI+OTkZSUlJqt/LysoQFBRk2sZTi5L01V/WbkKzrJGm3p5carRCrvT6Dby+xb6e0z9OFiHXFAkjzSA25Tfsf0WBa9W1ei3bNofvsi6qBTA3a5VY8svfiO3SHsO6Nc1fs+ngBWw6eAH7Xm76OWQpDVcgahqui1rwi1pCy3v7dISzk/rgyvFLuoMmqQ5nigpgvL294ejoiIIC9TG/goIC+Pn5NSmfk5OD3Nxc3H333apjSmXdE+3k5ITs7Gx06dKlyXmhoaHw9vbGqVOnNAYwcrkccrlcTNOJdJLCPJiDeSXWbgKZkKHLuJvzx0nbXVV3+5tp8HV3Mfj8WqVg0pQCX2eex8rfc7Dy9xzkLhqjtVx2vvYA4MP0HPz+92WN910ur8bbP5/Av6IN/4L9yBrdq3wbZ+NWaohG7luxu8kxeyAqgHF2dka/fv2QlpamWgqtVCqRlpaGmTNnNikfHh6Ow4fVl4m98sorKC8vx7vvvqu11+T8+fMoLi6Gv3/T7KNERPbgyc8yTV7n1sO2HYjfVApGreK7c+nvOndMF+vCVf3aktfMxonaltTf8Z90lFfd1Jmh/EatEq0c63pMNCUsFBuQrt9zFt1824o6R6pEDyElJSVhypQpiI6OxoABA7Bs2TJUVlYiMTERADB58mQEBgYiJSUFLi4u6NWrl9r5np6eAKA6XlFRgfnz5+O+++6Dn58fcnJyMHv2bISFhSEuLs7IyyMisk0ndHyrN5QpctPYMl0TTcUs6b9cXo0ObfXvxX/5W+0J/HTRJ4HjgDe2ISN5JFx0ZDkWw7BhUWmOIYkOYBISEnD58mXMnTsX+fn5iIqKQmpqqmpib15eHhwc9F/c5OjoiEOHDmHdunUoKSlBQEAARo0ahYULF3KYiIiITGrpr3/jvbSTmH9PTxSWWz9z+NVrN3AwrwSxXdpbrQ3n9eyJsjUGTeKdOXOmxiEjoG45tC5r165V+93V1RU//9z8DHYiIiJ9Hcy7iuLKaoyNClQ7/l7aSQDAvO+PWqNZJmXMBo17GqQRMEdvoCVwM0ciIrI79Tt/d/QSt4mklFbkGDOPaoIdJCDkZo5ERGS3Xvja9lMk2GJmXilgAENERDZNzC7bjembYbbqRi0EQTDb8nZdnlqfiZu1zW9hQOo4hERERDat3+vb8N2MQc2WO5Fv+Cqs8FdTDT7XFK7fYDZwsdgDQ0RENm/s8l3NlvnvDmlvDcHN3cVhAENERGRlz0lgOxNbwyEkIiKSPKn3XvxyrKD5QqSGPTBERCR5O08ZnhOFpIk9MEREJHnbjrMHw5wKy6rgY8RGnObAHhgiIiLSqazK9nLVMIAhIiIiyWEAQ0RERDr9d8dpXK+xrVw1DGCIiIhIp6/2n8d/fsm2djPUMIAhIiKiZv159qq1m6CGAQwRERFJDgMYIiIikhwGMERERNSsGzdta8dsBjBERETUrGOXDN/t2xwYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkx6AAZvny5QgODoaLiwtiYmKwb98+vc7bsGEDZDIZxo0bp3ZcEATMnTsX/v7+cHV1hUKhwMmTJw1pGhEREbUAogOYjRs3IikpCfPmzcOBAwcQGRmJuLg4FBYW6jwvNzcXzz//PIYMGdLkvsWLF+O9997DypUrsXfvXrRu3RpxcXGoqqoS2zwiIiJqAUQHMEuXLsXUqVORmJiIHj16YOXKlXBzc8Pq1au1nlNbW4tJkyZh/vz5CA0NVbtPEAQsW7YMr7zyCsaOHYuIiAh8+umnuHjxIjZv3iz6goiIiMj+iQpgampqkJmZCYVCcasCBwcoFApkZGRoPW/BggXw8fHBY4891uS+M2fOID8/X61ODw8PxMTEaK2zuroaZWVlajciIiJqOUQFMEVFRaitrYWvr6/acV9fX+Tn52s8Z+fOnfjkk0+watUqjffXnyemzpSUFHh4eKhuQUFBYi6DiIiIJM6sq5DKy8vx8MMPY9WqVfD29jZZvcnJySgtLVXdzp07Z7K6iYiIyPY5iSns7e0NR0dHFBQUqB0vKCiAn59fk/I5OTnIzc3F3XffrTqmVCrrHtjJCdnZ2arzCgoK4O/vr1ZnVFSUxnbI5XLI5XIxTSciIiI7IqoHxtnZGf369UNaWprqmFKpRFpaGmJjY5uUDw8Px+HDh5GVlaW63XPPPRgxYgSysrIQFBSEkJAQ+Pn5qdVZVlaGvXv3aqyTiIiISFQPDAAkJSVhypQpiI6OxoABA7Bs2TJUVlYiMTERADB58mQEBgYiJSUFLi4u6NWrl9r5np6eAKB2fNasWXj99dfRtWtXhISE4NVXX0VAQECTfDFEREREgAEBTEJCAi5fvoy5c+ciPz8fUVFRSE1NVU3CzcvLg4ODuKk1s2fPRmVlJZ544gmUlJRg8ODBSE1NhYuLi9jmERERUQsgEwRBsHYjjFVWVgYPDw+UlpbC3d3dpHUHv7jFpPURERFJVe6iMSatz5jPb+6FRERERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkGBTALF++HMHBwXBxcUFMTAz27dunteymTZsQHR0NT09PtG7dGlFRUfjss8/UyjzyyCOQyWRqt/j4eEOaRkRERC2Ak9gTNm7ciKSkJKxcuRIxMTFYtmwZ4uLikJ2dDR8fnybl27Vrh5dffhnh4eFwdnbGjz/+iMTERPj4+CAuLk5VLj4+HmvWrFH9LpfLDbwkIiIisneie2CWLl2KqVOnIjExET169MDKlSvh5uaG1atXayw/fPhw3HvvvbjtttvQpUsXPPvss4iIiMDOnTvVysnlcvj5+aluXl5ehl0RERER2T1RAUxNTQ0yMzOhUChuVeDgAIVCgYyMjGbPFwQBaWlpyM7OxtChQ9XuS09Ph4+PD7p3745p06ahuLhYaz3V1dUoKytTuxEREVHLIWoIqaioCLW1tfD19VU77uvrixMnTmg9r7S0FIGBgaiuroajoyM+/PBD3Hnnnar74+PjMX78eISEhCAnJwcvvfQSRo8ejYyMDDg6OjapLyUlBfPnzxfTdCIiIrIjoufAGKJt27bIyspCRUUF0tLSkJSUhNDQUAwfPhwAMGHCBFXZ3r17IyIiAl26dEF6ejpGjhzZpL7k5GQkJSWpfi8rK0NQUJDZr4OIiIhsg6gAxtvbG46OjigoKFA7XlBQAD8/P63nOTg4ICwsDAAQFRWF48ePIyUlRRXANBYaGgpvb2+cOnVKYwAjl8s5yZeIiKgFEzUHxtnZGf369UNaWprqmFKpRFpaGmJjY/WuR6lUorq6Wuv958+fR3FxMfz9/cU0j4iIiFoI0UNISUlJmDJlCqKjozFgwAAsW7YMlZWVSExMBABMnjwZgYGBSElJAVA3XyU6OhpdunRBdXU1tm7dis8++wwrVqwAAFRUVGD+/Pm477774Ofnh5ycHMyePRthYWFqy6yJiIiI6okOYBISEnD58mXMnTsX+fn5iIqKQmpqqmpib15eHhwcbnXsVFZWYvr06Th//jxcXV0RHh6O9evXIyEhAQDg6OiIQ4cOYd26dSgpKUFAQABGjRqFhQsXcpiIiIiINJIJgiBYuxHGKisrg4eHB0pLS+Hu7m7SuoNf3GLS+oiIiKQqd9EYk9ZnzOc390IiIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwEMERERSQ4DGCIiIpIcBjBEREQkOQxgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIiIgkhwFMM3oGuFu7CURERNQIA5hmyGTWbgERERE1xgCGiIiIJIcBDBEREUkOAxgiIiKSHAYwREREJDkMYIiIiEhyGMAQERGR5DCAISIiIslhAENERESSwwCmGW/dF2HtJhAREVEjBgUwy5cvR3BwMFxcXBATE4N9+/ZpLbtp0yZER0fD09MTrVu3RlRUFD777DO1MoIgYO7cufD394erqysUCgVOnjxpSNNMrmeAh7WbQERERI2IDmA2btyIpKQkzJs3DwcOHEBkZCTi4uJQWFiosXy7du3w8ssvIyMjA4cOHUJiYiISExPx888/q8osXrwY7733HlauXIm9e/eidevWiIuLQ1VVleFXRkRERHZLJgiCIOaEmJgY9O/fHx988AEAQKlUIigoCE8//TRefPFFvero27cvxowZg4ULF0IQBAQEBOC5557D888/DwAoLS2Fr68v1q5diwkTJjRbX1lZGTw8PFBaWgp3d9Nvvhj84haT10lERCQ1uYvGmLQ+Yz6/RfXA1NTUIDMzEwqF4lYFDg5QKBTIyMho9nxBEJCWlobs7GwMHToUAHDmzBnk5+er1enh4YGYmBitdVZXV6OsrEztRkRERC2HqACmqKgItbW18PX1VTvu6+uL/Px8reeVlpaiTZs2cHZ2xpgxY/D+++/jzjvvBADVeWLqTElJgYeHh+oWFBQk5jKIiIhI4iyyCqlt27bIysrCn3/+iTfeeANJSUlIT083uL7k5GSUlpaqbufOnTNdY4mIiMjmOYkp7O3tDUdHRxQUFKgdLygogJ+fn9bzHBwcEBYWBgCIiorC8ePHkZKSguHDh6vOKygogL+/v1qdUVFRGuuTy+WQy+Vimm6UL6bG4MFVey32eERERKSbqB4YZ2dn9OvXD2lpaapjSqUSaWlpiI2N1bsepVKJ6upqAEBISAj8/PzU6iwrK8PevXtF1WlOA7t4W7sJRERE1ICoHhgASEpKwpQpUxAdHY0BAwZg2bJlqKysRGJiIgBg8uTJCAwMREpKCoC6+SrR0dHo0qULqqursXXrVnz22WdYsWIFAEAmk2HWrFl4/fXX0bVrV4SEhODVV19FQEAAxo0bZ7orJSIiIrshOoBJSEjA5cuXMXfuXOTn5yMqKgqpqamqSbh5eXlwcLjVsVNZWYnp06fj/PnzcHV1RXh4ONavX4+EhARVmdmzZ6OyshJPPPEESkpKMHjwYKSmpsLFxcUEl0hERET2RnQeGFtk7jwwAHPBEBERSTYPDBEREZEtYABDREREksMAhoiIiCSHAQwRERFJDgMYIiIikhwGMERERCQ5DGCIiIhIchjAEBERkeQwgCEiIiLJYQBDREREksMAhoiIiCSHAQwRERFJDgMYIiIikhwGMERERCQ5DGCIiIhIchjAEBERkeQwgCEiIiLJYQBDREREksMAhoiIiCSHAQwRERFJDgMYIiIikhwGMERERCQ5DGCIiIhIchjAEBERkeQwgCEiIiLJYQCjp+HdO2g83t23rYVbQkRERAxg9LTmkf4ajy+f1NfCLSEiIiIGMHqSyWToH+zV5HiYTxsrtIaIiKhlYwAjwqrJ0dZuAhEREYEBjCiebs4YGxVg7WYQERG1eAYFMMuXL0dwcDBcXFwQExODffv2aS27atUqDBkyBF5eXvDy8oJCoWhS/pFHHoFMJlO7xcfHG9I0s3t3Qh9rN4GIiKjFEx3AbNy4EUlJSZg3bx4OHDiAyMhIxMXFobCwUGP59PR0TJw4Edu3b0dGRgaCgoIwatQoXLhwQa1cfHw8Ll26pLp9+eWXhl0RERER2T3RAczSpUsxdepUJCYmokePHli5ciXc3NywevVqjeU///xzTJ8+HVFRUQgPD8fHH38MpVKJtLQ0tXJyuRx+fn6qm5dX0wmzRERERIDIAKampgaZmZlQKBS3KnBwgEKhQEZGhl51XLt2DTdu3EC7du3Ujqenp8PHxwfdu3fHtGnTUFxcrLWO6upqlJWVqd2IiIio5RAVwBQVFaG2tha+vr5qx319fZGfn69XHXPmzEFAQIBaEBQfH49PP/0UaWlpeOutt/D7779j9OjRqK2t1VhHSkoKPDw8VLegoCAxl0FEREQS52TJB1u0aBE2bNiA9PR0uLi4qI5PmDBB9XPv3r0RERGBLl26ID09HSNHjmxST3JyMpKSklS/l5WVMYghIiJqQUT1wHh7e8PR0REFBQVqxwsKCuDn56fz3CVLlmDRokX45ZdfEBERobNsaGgovL29cerUKY33y+VyuLu7q91siWsrR2s3gYiIyK6JCmCcnZ3Rr18/tQm49RNyY2NjtZ63ePFiLFy4EKmpqYiObj4Z3Pnz51FcXAx/f38xzSMiIqIWQvQqpKSkJKxatQrr1q3D8ePHMW3aNFRWViIxMREAMHnyZCQnJ6vKv/XWW3j11VexevVqBAcHIz8/H/n5+aioqAAAVFRU4IUXXsCePXuQm5uLtLQ0jB07FmFhYYiLizPRZRIREZE9ET0HJiEhAZcvX8bcuXORn5+PqKgopKamqib25uXlwcHhVly0YsUK1NTU4P7771erZ968eXjttdfg6OiIQ4cOYd26dSgpKUFAQABGjRqFhQsXQi6XG3l5REREZI8MmsQ7c+ZMzJw5U+N96enpar/n5ubqrMvV1RU///yzIc0gIiKiFop7IREREZHkMIAxA5nM2i0gIiKybwxgiIiISHIYwBAREZHkMIAxgVE96lZgxff0w7akoVZuDRERkf2z6FYC9mrlQ/1Qev0GvFo7W7spRERELQJ7YEzAwUHG4IWIiMiCGMAQERGR5DCAMQMvN/bGEBERmRMDGDP4eEo0IoM8rd0MIiIiu8UAxgxu83fHdzMGWbsZREREdosBDBEREUkOAxgiIiKSHAYwREREJDkMYIiIiEhyGMAQERGR5DCAISIiIslhACNRPm3lRp3/6KAQE7WEiIjI8hjASJRMZu0WEBERWQ8DGIkSBGu3gIiIyHoYwNi4h2/vbO0mEBER2RwGMAYY3zcQAHB/v44G1zFteBe9yvXp5GnwYwDAjhdGaDwuwPpdOOnPD7d2E4iISKKcrN0AKXrz3t4Y36cj+od4WbUdEwcE4dD5Uhy9WKa1TKf2bk2O2cpGk8Hera3dBCIikij2wBjApZUjBnf1htzJUWc5ZyfzPr0p4yOw5Zkhos97dmSYzvsfiDa8Z4mIiMgSGMCY0U/PDsGTw0Ix7+4e+GO25qEca3B21B14vXlvbwu1hIiIyDAMYMyoS4c2SB59GxIHhSCoXdOhHH1oWy5tzhksTo7S+7OY0D/I2k2wCX2NnDNFRCQV0vuksmOz47sjzKcNPFxbmf2xuAzb/iSPDkcrCQafRESG4LudjQj1bo3pw8OwLWkY2rV2tnZzSIK8+HdDRC0IAxgbMTYq0NpNkDR77FEa3zcQ4/vw74LEmRMfbu0mEFkEAxgb5+hg/EsU0mi5so+7cfsokXmtfiQaiYOC8dZ9ERqXwRMRkYEBzPLlyxEcHAwXFxfExMRg3759WsuuWrUKQ4YMgZeXF7y8vKBQKJqUFwQBc+fOhb+/P1xdXaFQKHDy5ElDmmZ34nr6ajzeXsRwwRdTY/Dcnd2w9IFIvHVfb3TzbWuq5qmJ7+lnlnr1Yaq9oWaM0C/BoDndEe6LeXf35HwWIiIdRL9Dbty4EUlJSZg3bx4OHDiAyMhIxMXFobCwUGP59PR0TJw4Edu3b0dGRgaCgoIwatQoXLhwQVVm8eLFeO+997By5Urs3bsXrVu3RlxcHKqqqgy/Mhun75CH3MkRXz8Vq/o9JqQdBoS0w/JJffV+LH8PVzw9sivG9+2IhP6dxDbVIjoYubu2vs9nREcPLBzXS+v9sxTdjGqHtQ3v7mPtJhAZJJi9jSSS6ABm6dKlmDp1KhITE9GjRw+sXLkSbm5uWL16tcbyn3/+OaZPn46oqCiEh4fj448/hlKpRFpaGoC63pdly5bhlVdewdixYxEREYFPP/0UFy9exObNm426OKlq3JnQP7id6uchXb3x1ZOx6NKhjWUbZSZ/vqzAj08PRkSgh0Ue770JfbTuL7VofG9J93r4e7jg8SEhepfv6nPrb2j+PT3N0SSyAqnuVP+fByKx/rEYazeDJETUu3VNTQ0yMzOhUChuVeDgAIVCgYyMDL3quHbtGm7cuIF27eo+lM+cOYP8/Hy1Oj08PBATE6O1zurqapSVlandyLTGRPhb5HE6tJWjl4WCl+YM697B2k0wyJpH+mN2fHcMDvM2OACL6+mHcD/zDC0S6UPuVJfhnEhfot7tioqKUFtbC19f9XkZvr6+yM/P16uOOXPmICAgQBWw1J8nps6UlBR4eHiobkFB0kpitmCs7X/bdWvVNFuvdxvt827cXay7rZahm1M2XLEh1ZVMI8J9MH14GGRGfvVur+P1JTJEJwMTeJJtGhluW0PUFu0vX7RoETZs2IBvv/0WLi4uBteTnJyM0tJS1e3cuXMmbKX5+bk3vXav1oYnr3v6jrq9jcb09scLcd0xtJt5ehLG99W8R1Kndm74adZQo+r+953i5p5MHGCauTxtrBx4WVMbFyf0D/ZCZJAnfNrK0UZumefiiaGhFnkcWzJVxNCe1Hz66ABrN4Es5A0b22ZG1DuWt7c3HB0dUVBQoHa8oKAAfn66V6AsWbIEixYtwrZt2xAREaE6Xn9eQUEB/P1vDVsUFBQgKipKY11yuRxyufSWAq98qC/+Ol+KO3v44kBeier42KgATNBzcq2mLQmS7uyGsVGB6NKhNWQyGf4uOGhQ+565Iwy3h7YXfd5Dt3dCoKerQY9Zr1egB04sjEcrRwd0eWlrs+V9uRTcaDIAXz1ZN0Hc2N4bsY9r76bEdsa6jLOq342dpC4lAR4uuFgqfgEGUwbYPhNk9TApUc1xdnZGv379VBNwAagm5MbGxmo9b/HixVi4cCFSU1MRHR2tdl9ISAj8/PzU6iwrK8PevXt11ilF8b38MSc+vMmHxbsT+jS7c/Xnj8fg34puuDsioMl9MpkMYT5tVPX2DHBvti2ChvGSuyMDMDBMvzHoF+K661Wu3gcP9mm2jEsrRzg6GP7xdntou+YLNdLWQr0Otkgmk6luUjbcBucudfFpg7Y6evd+elb8LvJS8cr/9VD9rM+f1pje/liWEAV3F/NvoUL2RXQ8lZSUhFWrVmHdunU4fvw4pk2bhsrKSiQmJgIAJk+ejOTkZFX5t956C6+++ipWr16N4OBg5OfnIz8/HxUVFQDq3kRnzZqF119/Hd9//z0OHz6MyZMnIyAgAOPGjTPNVdqBQWHeeFbRFQ56fMA/MjAEc+LD8ePTg0U9RnNTQB6IrptrNLBLe8wYESaq7v/TEHiZUqd2bljzyAB8PDm6+cIAFo7rhfv7dcQdt9nWmK6bs/rcI+825vvm3vgvKcRbfWWbMcGkJU0bZv3cPWK5aphj1lIt+VckxjHjNBlAdACTkJCAJUuWYO7cuYiKikJWVhZSU1NVk3Dz8vJw6dIlVfkVK1agpqYG999/P/z9/VW3JUuWqMrMnj0bTz/9NJ544gn0798fFRUVSE1NNWqeTEvm7OSAacO7mHx1T5hPG/w1b5RNLnVU9PCFq7MjFD189eqBevj2zljyr0g4NPiKaItzeBMsuMv2MyPD8OigW3M1BgSL79GiWxL+CfgN6Rk01jN3iPuCYQyJd+CRhBnUfz5z5kzMnDlT433p6elqv+fm5jZbn0wmw4IFC7BgwQJDmkMmos8qHEvslK3J+L6B2HTggtb7ZQ36E8T0HGgq2aGtHJfLqwEAbeROqKi+qXd9ptbK0QFz/68HFvx4zOyP5ebshLl398DqXWfM/lj1wv3a4kR+udkfp63cCeUWfh1nx4djYFh7DAhpjwNnr1r0scOMzLadOCgYa3blGnSuSysbmyhBdot/aWSz7olsMOykI7gaENxOLSmbsRxt7Cvlw7GaE+9ZS+NhLmN8OfV2k9TTXBZXRQ/NW3IY690JUVrvc3ZywB3hvmgjd8IQE+c36WzmCa/399O84lAfw7rdGpbV5z/J0BQIRAxgWqj6pdbaOisM+Qzv29nT8AZpoG9Q8tVTsXrNDTKEpZYW62Ku7MC2EKd5idjTS5cXR1tnB2Ztu8i7NJrj0niitLHP/aQY824JYszfXMMeUKlPECfbxgDGSqz9reOOcB988XgM9rw00mR1Jg4KwUIbSNLXcJjL2OyyER098NDtTT8s0p8fblS9UtBaR/BmVMAg8jOtfi6JNgIAuY5JsabKGSTGOC2BjSW0czM+KDRV2GGtIeeGRtjgKjWpOPDqnWqb9LaVW//1bIgBTAslk8kwMMwbPm1NN1G6laMDHo4NNll9hnrz3t7o28kTKx/Sf8NLbSKDPDXm6HGTm2YYZXxf9Q86feYhyWQwOu+OLvPv6YknhoY2mQzt9M8369v83S22F5eXWyuM7SNuBdvxBfGqnz94sA9Sxvc2a+6Z72cOanKsubQI5nRv30D0D/ay2uMDwAPRdUNQzyq6WrUdALAmkYn2DNWutbPa37KrCYePTYEBDNmdoHZu2DR9EOJ7+aOjl/5zBTTlxjG3//wrUvQ5v8waKmo38oaWP9j8eVMGBuOlu25rMsyx9dkhmBTTSa+cPqaSkaxfD2HDob6Gb7IyDaGLk4mHGyM6eooq72PmpHatHB3g72G+ALcxTc/xW/dF4OCrd2KECXdH56o4aowBDFnci6PDseOFERZ5rEcHBxt1vuI280z+rGfIHAFjphUYs0lnN9+2eOPe3hb5cFzzSH/8MXtEk7kk2kR39sKkmE54tUESNW2yXx9tbPOauFePPCbbnx+OJ4eGYssz9pvErp5MJjPZ/KZ6Hm7WH76w1lwr0owBDJnVS3eFY/MM9S7220PbWyxtuNxJ9wegpmAgrmdd0OLs6IDufm01T2jUo7Pm4dt1rx6S8vxGQzfv/N80/bJrjwj30bhthjYymQxv3Nsbjw1ufs8hcyTo06fOEO/WSL7rNpNsKzCwS/OrmjY8IW6F1+he2reDsWTiPVvOjv2UBJMm2jMGMDaofty48fwIczPHxOInhnZBVJCnQeea4gM+oqMHAjzEzfNJvus2vH1/BHa9eAcAoJtvG/xfhL+oLmy5k0Oz30DbOBv+Rq0rTb2+9N1/SxN9e0Ya69dZ/DCApiEKfVkqmbClRx/1SVIpdl+zD3QMLwZ7txZVlzFS7jN8w8C054aZsCVk62w31G3BxkYFon9wO427VtsisROBLdnx0MrRAWnPDcdtc1P1PsellSP+1WDli0wmwwcP9sWRC6X4v/d3mqOZonXp0AZJd3ZD+zaGd9P3DGw+Y7FUTY7tjKxzJRj5zxBgoJfl5oTowxT7/jg5yHBTqX/kpLjNF9uOF2i9v3GwJ3dyQPVNpcUT07XTEvjrEyRaanI52Qb2wNioAE9Xs+U2MZWPHu6HxweH4O5I8+5zVC/sn7wwYpPWtXK89Ty2MUHPha14ZmRXTIppPsndOwniJwpb03N3djO6jgVje+H7mYNVKyimDe+CiQOCsO5R869IcdYjh4pXa2eTrJIT4+5IcfOf/jdtIIZ09cY3Tw3UWa6HHlt3iKIjUGm4+u5TC7yWhujX2ctsS7cXjTe8d8pQtpxm0H7ezcmk9Om2j+vph7ie2sfNB4WJ68LWRQCw7tEB+DQjF5Njg/H1/nN6n+vk6ID/PtwP1TeVZt0c0Va1sbHcDc15eqT2pbfZr8ejvOompn9+APvOXNG7TjdnJ6SMjzCoPaEdWuOpYV2w6KcTuFJZ02z5L6bqt1dYfC/DJ1QDwEO3d8ba3bl6lxf7t98r0AOf6dj37PBro3C9plZrj0m9br5t8HdBBdYm9keHtnKMec80vZi2+r/8v2l1AV/wi1tMXret9SRaG3tgyGzGRpp2Dk+gpyuSR99mUA6UUT39mu0pEjOPQZ+irW0sZ4KtMGZOi9zJEd5t5PjqSf0mAxtrWLcO+O254XggOgiZryj0OifaQst9X7rrNlHlB3Yx/AvFq2OaPlZbl1bw+WeY21PHCqEfnh6MP2aPwPDuPjYbdDSWOCjY6DrWJvY3viGkEwMYsjhb6pI05sNUl0cGBhu0h5EpJlLf1ftWr5hLKwft20UYce2W/CYot5HNAW0tLb4+yfISBwXD112Og6/eaVT7B4Z54/iCePTw1zxcpCu3jtzJUeeKMpdGKwX1/Q/wdTdfMDTvbuMzig83YQ6c5jQ3bOnTVm7W5JfWYhvvDC2RLX2K68GY/CEtzYjuHfB8XHe4GbHKyBj/+VcUlvwrEo8OCsHWZ4ZgaLcOiAzyNOn+Od5t5Phh5mBR58SKXBVTr0+QJ+6JDMDMEWEGnW8vnh9VNzdodnx3vc+Zd3dP7EkeaZKcLObKwmroXL/2beT4+qlYvBCn//NhqEBPV/xbIX5ulrExr7eGSfpbnlH/v5s5IgyfPBKts55g79bNphi4x0JzGU2Jc2CoWR9PjsagMNPupmvPbg9tb/BmeKZYjuvq7Fi3m3C/W8e+m9E03b2xbendsfmlvPX++3A/xBo4hCGTyfDexKbZf+tX4dympVdACvYkj0Tm2auY8cWBZss+MbQLEvp3Ep1HxtZ6jpqjvblN/yD7B7dDRfVNtWM/Pj0Yc/53CEcvlpmsTTvnjND4PA4IaYfcokoUlleb7LEamjEiDPN/OKZ2rPEXo+fjuuNkQbnRj+XkKK2/E4ABDDWg7QNL0cO82WhbgoiOHnB2dMD+s1eNrsuSaeJNZZSOyd6GOvxaHKpu1NrEhoGG8vNwQWcRSR1NkQTP1jk5GDcw0CvQA1ueGYIjF0pN9nxpCwK/ejIWQxdvN8ljAMD4PoF4emRXjFiSbrI6DdW4p8cWcQiJyALevLc3RvU0PhBck9hf5y7RYohJhrf0Adtbiu3q7GiydPX6Lnt9WcvE2VmKrmgrd8K04czUqot3Gzn8m0ksaao9yXoFesDXwrm0xkapD8OIvZQuPm0QYsGkgfU0DVX1DKjrYbXGHnH6YgBDpKeGy0UN+Z+ODVUfhvs/A8acfU2we/i7E6Lwxr29ECBiUt/4vh2Nflxb1qGtHP/qV3eNuoKZqUND8dHD/ZocD2rnhqx5ozAn3rb3yrF07pnGHB1k+GP2CDw6qPktH6To3QnGbXTqoKGnR9/tN8R6ZGAwfntuGFJnDcH254eb5THMjUNIZDb6rJKwtoZJ7tybGYoI8HTFBw/2MTiLau+OHvh+5iC4OTvi6MUynTl0GgvzaYOiimpVMj9jjI2y7BYVUrFwXC8oevhicJg3es77WWs5bX/Xhu6xVP+aurs4oazqZjOljdM498x9fTvifwfOm/UxG3NydDB4vkVruXETiSODPOHl1grp2ZeNqsfUHrq9E3bnFGPS7U0n2huy/YY+XrtHfaWVPgkYbQ0DGNLImHl//1Z0w77cYtzV2/ZXLjk5OmDT9IG4cVOp11yK/4uo6zXJL60y6PEiOnoCAMJ82t5qgx4p4X+eNRS1SkESQWFzvGxgV2FNXFo5igoqTfm4xxbEwdFBhu6v6L/lhbHuiQzAuD4BogIYNyvnNkq6szueWp8p+ry2Lk4or7qJYV29cehCqRlaJl7D4HF2fLhJtpcwxr/v7IYNf+qfINQWSP/dkGzOs4qu+Pzx2yXzYdu3kxdiDFziqy9dQ04/6jFZztFBJpnnszlTBgYjrqcv3r7fsMy4pqAr8RoAdPetCzDv7WOZ3io3Z6dmd043NbmTA7zcNM8h0pZw7q37IxDm0wbLEqLM1i5t/yqCYPgk5l/+PRRv3x+BGXc0vxS/Pkgz9150DZd/28I0E193F5Mk8LMk9sCQzZLJZJjQPwgb/jxn19vYh/tJdxmwIVxaOeKjh3XnrTC3zu1bo+Raidb7N00fiFOFFYjQsFQ8zAIbBj4yMBj7z17BkQumWwqsSa9AD7wQ1x2Bnq6QyWT4YeZgVN2s1To5ukuHNtiWZL0dn6OCPNGnkyeCvPRfuQXUrdxruEFrQ70C3VXP88O3d8aUgXUJKB8dHIw3t56A4jbdCekMST750l3h6r3cRgYwpop/pLbKjQEM2bRF90XgtXt6wqWVbaXlN0XGXM31tmzm2gRPrNZyJ0QGeWq8L6idGzbPGARPCS/fbmhGgwSBYnL7WIOjgwzfThef00iXQE9XVQCzcFwv1fGpQ0IRG+qN7n5ttZ1qkPv6dsQTQ7ugsMywYWhzenRQCE4WVEBx260Vk7b8nmQffdJklHsiAxAb2l7VbW5rbC14aeyTKdF4vcEbHxnO2j0z+ooK8kSwGZa79gqs640b37fp0JWudP1SY8y+TKb049OD4arl/UUmk6F3Rw+TD92K2RrD0q+4SytHvJMQpZ553YYjGPbAkMYsp6S/ASHt0NalFV7ZfMTaTZE8e5nnY6hvpw9CcUUN/BrlSjk6P87glPu2aFi3Dvj88RiNq+osOR+kV6Bt9zhpYshE6sZ7WC2+z3rzz0ypZb9bWNHEAXXL5Rp21bUU9vM2rL/mJo0SAUArR4cmwQsAkyUvtBUymQyDwrybTTT34aRbeWs8tUw4bmnEJuebOSKsyX5RD/TXPB9Iauzrv0JCgr1b49iCOK3dl2Qflj/YF1ev1ejcjZeszxZWgUjZsoQ+eOiTvXjt7h4mrfeu3v74cFJffLkvD8l3mSZJ4L8V3ZCefRmTDdgt3lzMNafO112O5y2w2aW1MICxImvtVqwPqW0AZ6vsfRfvhOggbNx/Dm3kTk021dOXh2srlF6/oZr/QdIzuKs3Tr4x2uBNTHW5q7e/SXNKRQZ54viCeJPtrm1w8GvAW2xHkauv7J3tfoKSVT07siu2nyhUDXWRuoZd+uZ405aKheN64d6+gcgvrcKsjVlq90V09MCh880nDds0fSDW7c616FL5PkGe+OtcicUeryUw1f+BJZbymip4sZTvZgxCcWW1VfZJsmUMYEgjPw8XZCTfwZ4YLdxdWuHjydFwcpSpVkn5ustRUFaNkS1oXpOzkwNuD22PLYcuGVxHlw5tsGCsZVdxzY7vjrW7cy36mKSfMJ82eOu+3vAxwb5f+jDne1xkRw/8pUcQ32w9Wpb0zxihf9Bvj2/lBoXMy5cvR3BwMFxcXBATE4N9+/ZpLXv06FHcd999CA4Ohkwmw7Jly5qUee211yCTydRu4eG2vSlaS8DgRTdFD18M734rydXvL4xARvIdJtmviMzLzdnJYll2bZUt/3sn9O+EEeG6E8hJwUoNG3+aUnMTemUmWDJhrvk5piA6gNm4cSOSkpIwb948HDhwAJGRkYiLi0NhYaHG8teuXUNoaCgWLVoEPz/t+4z07NkTly5dUt127twptmlEVuXSyhH+Hvrv8KxJSHv76SLuoCUdPVFL4e/hiscGa955u2FwwUnkhhE9hLR06VJMnToViYmJAICVK1diy5YtWL16NV588cUm5fv374/+/fsDgMb7VQ1xctIZ4BA1pG0PF6n68enBWPF7DmbbwYqBb6cPRGV1LX7466K1m0JkUzxcW2HmP5mPvds4Y1BYe8ggY5oFA4kKYGpqapCZmYnk5GTVMQcHBygUCmRkZBjVkJMnTyIgIAAuLi6IjY1FSkoKOnXiBFLSbMKAIOw/exXDu9lG6nlj9Qr0wPIH+zZf0Eb1bpAQrE8nLwBgAEMtgpjek4Ov3qlKSCiTybD+sRjVz4YI+KfHt6X24IgKYIqKilBbWwtfX/VJir6+vjhx4oTBjYiJicHatWvRvXt3XLp0CfPnz8eQIUNw5MgRtG3bNL19dXU1qqurVb+XlZl3wzOyDl3/03InR7zPDMJmNSCknd5lO7V3w8+zhsKrNb9JEmnTOJuyoYHLp48OwOELpRjZzEaT9s4mViGNHj1a9XNERARiYmLQuXNnfPXVV3jsscealE9JScH8+fMt2USyoCeHheLnI/l46HbbSTTVEg0K88anjw5AaAf95uWYetM7IktwlOAWDUO7dcBQO+l9NoaoSbze3t5wdHREQUGB2vGCggKTzl/x9PREt27dcOrUKY33Jycno7S0VHU7d+6cyR6brC959G1If2EE3F34bd7ahnbrwORZZNc0bZxpK/p29jLqfHufWyMqgHF2dka/fv2QlpamOqZUKpGWlobY2FiTNaqiogI5OTnw99ecfVEul8Pd3V3tRkREJJa5M6I7OYrv4dn94h34+qlYRHT0NOqxTZFfyZbn14h+5ZKSkjBlyhRER0djwIABWLZsGSorK1WrkiZPnozAwECkpKQAqJv4e+zYMdXPFy5cQFZWFtq0aYOwsLrZ2M8//zzuvvtudO7cGRcvXsS8efPg6OiIiRMnmuo6iYjsWvrzw+1u00d78NTQLvjteCHGicg7FODpigBP41IyAIB/g41BpTdQ1jzRf+0JCQm4fPky5s6di/z8fERFRSE1NVU1sTcvLw8ODrc6di5evIg+fW5NtlyyZAmWLFmCYcOGIT09HQBw/vx5TJw4EcXFxejQoQMGDx6MPXv2oEMHjvERkfnZ8JdMvQUzzbxN8mrtjF+Thlm7GXbJoHB95syZmDlzpsb76oOSesHBwRCa6YPasGGDIc0gIiKiBj54sA9mfnHQ2s2wiJa7Cx1Zjb+nZfY4ISLtzD33QyoCTTBUY0tGhrecvdj4F0wW89OzQ3Ct5qbFNmkjoqZSxvfGN5nn8ezIrtZuik3o0FaO72YM4vwhCeIrRhZzmz9Xi5HtCG2hc0YmDuiEiQOY5bwhbbs9k23jEBIRtUhTh4ZauwkkQW1d6r73d/Syr6EnbVydHa3dBK3YA0NELZJLK9t9Yyb9WXrX803TBuLD9Bw8888QnFfrVrhQct2ibWisjYv5PsrnxIfj74JyTIqxvczoDGCIiEiyegV64NX/62GxybhdfdvinYQo1e/vTeiD2d8cwow7wizy+JoEerpi/j090dbFCYKJkwL4urvgx6eHmLROU2EAQ0REkvbY4BCrPXZohzb4ZtpAqz1+vSkDgwEA12puqo413CvSww63FWAAQ0REZOfujgjAHyeLECNil3lbxwCGiIjIzjk5OmDpA1HWboZJcRUSERERSQ4DGCIyi2Hd6/YykzvxbYaITI9DSERkFqN7+WH9YzHo7tfW2k3RKtDTFRdKrmNMb39rN4WIRGIAQ0RmIZPJMLirt7WbodPP/x6K3KJK9AxglmgiqWEAQ0QtVhu5E3oFeli7GURkAA5OExERkeQwgCEiIiLJYQBDREREksMAhoiIiCSHAQwRERFJDgMYIiIiOyHDrR0cHRru5miHuIyaiIjITrg6O+L+fh1RdaMW/h4u1m6OWTGAISIisiNL/hVp7SZYBIeQiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAIaIyAb1CuAmk0S6cBUSEZENemnMbfB0c8Y9kQFay0R39sL+s1cxpKu3BVtGZBtkgiAI1m6EscrKyuDh4YHS0lK4u7tbuzlERBZxpbIGPx66iHsiA+Dp5mzt5hCJZsznN3tgiIgkql1rZ0yODbZ2M4isgnNgiIiISHIMCmCWL1+O4OBguLi4ICYmBvv27dNa9ujRo7jvvvsQHBwMmUyGZcuWGV0nERERtWyiA5iNGzciKSkJ8+bNw4EDBxAZGYm4uDgUFhZqLH/t2jWEhoZi0aJF8PPzM0mdRERE1LKJnsQbExOD/v3744MPPgAAKJVKBAUF4emnn8aLL76o89zg4GDMmjULs2bNMlmdACfxEhERSZExn9+iemBqamqQmZkJhUJxqwIHBygUCmRkZIh6YGPqrK6uRllZmdqNiIiIWg5RAUxRURFqa2vh6+urdtzX1xf5+fkGNcCQOlNSUuDh4aG6BQUFGfTYREREJE2SXIWUnJyM0tJS1e3cuXPWbhIRERFZkKg8MN7e3nB0dERBQYHa8YKCAq0TdM1Rp1wuh1wuN+jxiIiISPpE9cA4OzujX79+SEtLUx1TKpVIS0tDbGysQQ0wR51ERERk30Rn4k1KSsKUKVMQHR2NAQMGYNmyZaisrERiYiIAYPLkyQgMDERKSgqAukm6x44dU/184cIFZGVloU2bNggLC9OrTiIiIqKGRAcwCQkJuHz5MubOnYv8/HxERUUhNTVVNQk3Ly8PDg63OnYuXryIPn36qH5fsmQJlixZgmHDhiE9PV2vOomIiIga4maOREREZBUWywNDREREZAvsYjfq+k4kJrQjIiKSjvrPbUMGg+wigCkvLwcAJrQjIiKSoPLycnh4eIg6xy7mwCiVSly8eBFt27aFTCYzad1lZWUICgrCuXPn7HJ+jb1fH2D/18jrkz57v0Z7vz7A/q/RXNcnCALKy8sREBCgtgBIH3bRA+Pg4ICOHTua9THc3d3t8o+ynr1fH2D/18jrkz57v0Z7vz7A/q/RHNcntuelHifxEhERkeQwgCEiIiLJYQDTDLlcjnnz5tnt3kv2fn2A/V8jr0/67P0a7f36APu/Rlu8PruYxEtEREQtC3tgiIiISHIYwBAREZHkMIAhIiIiyWEAQ0RERJLDAKYZy5cvR3BwMFxcXBATE4N9+/ZZu0lISUlB//790bZtW/j4+GDcuHHIzs5WKzN8+HDIZDK121NPPaVWJi8vD2PGjIGbmxt8fHzwwgsv4ObNm2pl0tPT0bdvX8jlcoSFhWHt2rVN2mPq5+i1115r0vbw8HDV/VVVVZgxYwbat2+PNm3a4L777kNBQYEkrg0AgoODm1yfTCbDjBkzAEjztduxYwfuvvtuBAQEQCaTYfPmzWr3C4KAuXPnwt/fH66urlAoFDh58qRamStXrmDSpElwd3eHp6cnHnvsMVRUVKiVOXToEIYMGQIXFxcEBQVh8eLFTdry9ddfIzw8HC4uLujduze2bt0qui1iru/GjRuYM2cOevfujdatWyMgIACTJ0/GxYsX1erQ9LovWrTI5q8PAB555JEmbY+Pj1crY8uvnz7XqOl/UiaT4e2331aVseXXUJ/PBVt679SnLc0SSKsNGzYIzs7OwurVq4WjR48KU6dOFTw9PYWCggKrtisuLk5Ys2aNcOTIESErK0u46667hE6dOgkVFRWqMsOGDROmTp0qXLp0SXUrLS1V3X/z5k2hV69egkKhEA4ePChs3bpV8Pb2FpKTk1VlTp8+Lbi5uQlJSUnCsWPHhPfff19wdHQUUlNTVWXM8RzNmzdP6Nmzp1rbL1++rLr/qaeeEoKCgoS0tDRh//79wu233y4MHDhQEtcmCIJQWFiodm2//vqrAEDYvn27IAjSfO22bt0qvPzyy8KmTZsEAMK3336rdv+iRYsEDw8PYfPmzcJff/0l3HPPPUJISIhw/fp1VZn4+HghMjJS2LNnj/DHH38IYWFhwsSJE1X3l5aWCr6+vsKkSZOEI0eOCF9++aXg6uoqfPTRR6oyu3btEhwdHYXFixcLx44dE1555RWhVatWwuHDh0W1Rcz1lZSUCAqFQti4caNw4sQJISMjQxgwYIDQr18/tTo6d+4sLFiwQO11bfg/a6vXJwiCMGXKFCE+Pl6t7VeuXFErY8uvnz7X2PDaLl26JKxevVqQyWRCTk6Oqowtv4b6fC7Y0ntnc23RBwMYHQYMGCDMmDFD9Xttba0QEBAgpKSkWLFVTRUWFgoAhN9//111bNiwYcKzzz6r9ZytW7cKDg4OQn5+vurYihUrBHd3d6G6uloQBEGYPXu20LNnT7XzEhIShLi4ONXv5niO5s2bJ0RGRmq8r6SkRGjVqpXw9ddfq44dP35cACBkZGTY/LVp8uyzzwpdunQRlEqlIAjSfu0EQWjy4aBUKgU/Pz/h7bffVh0rKSkR5HK58OWXXwqCIAjHjh0TAAh//vmnqsxPP/0kyGQy4cKFC4IgCMKHH34oeHl5qa5REARhzpw5Qvfu3VW/P/DAA8KYMWPU2hMTEyM8+eSTerdF7PVpsm/fPgGAcPbsWdWxzp07C++8847Wc2z5+qZMmSKMHTtW6zlSev20XWNjY8eOFe644w61Y1J5DQWh6eeCLb136tMWfXAISYuamhpkZmZCoVCojjk4OEChUCAjI8OKLWuqtLQUANCuXTu1459//jm8vb3Rq1cvJCcn49q1a6r7MjIy0Lt3b/j6+qqOxcXFoaysDEePHlWVaXj99WXqr9+cz9HJkycREBCA0NBQTJo0CXl5eQCAzMxM3LhxQ+0xw8PD0alTJ9Vj2vq1NVRTU4P169fj0UcfVduIVMqvXWNnzpxBfn6+2mN5eHggJiZG7TXz9PREdHS0qoxCoYCDgwP27t2rKjN06FA4OzurXVN2djauXr2q13Xr0xZTKC0thUwmg6enp9rxRYsWoX379ujTpw/efvttta55W7++9PR0+Pj4oHv37pg2bRqKi4vV2m5Pr19BQQG2bNmCxx57rMl9UnkNG38u2NJ7pz5t0YddbOZoDkVFRaitrVV7IQHA19cXJ06csFKrmlIqlZg1axYGDRqEXr16qY4/+OCD6Ny5MwICAnDo0CHMmTMH2dnZ2LRpEwAgPz9f47XV36erTFlZGa5fv46rV6+a5TmKiYnB2rVr0b17d1y6dAnz58/HkCFDcOTIEeTn58PZ2bnJB4Ovr2+z7baFa2ts8+bNKCkpwSOPPKI6JuXXTpP6Nml6rIbt9fHxUbvfyckJ7dq1UysTEhLSpI76+7y8vLRed8M6mmuLsaqqqjBnzhxMnDhRbdO7Z555Bn379kW7du2we/duJCcn49KlS1i6dKnNX198fDzGjx+PkJAQ5OTk4KWXXsLo0aORkZEBR0dHu3r9AGDdunVo27Ytxo8fr3ZcKq+hps8FW3rv1Kct+mAAI3EzZszAkSNHsHPnTrXjTzzxhOrn3r17w9/fHyNHjkROTg66dOli6WaKMnr0aNXPERERiImJQefOnfHVV1/B1dXVii0zvU8++QSjR49GQECA6piUX7uW7saNG3jggQcgCAJWrFihdl9SUpLq54iICDg7O+PJJ59ESkqKTaVn12TChAmqn3v37o2IiAh06dIF6enpGDlypBVbZh6rV6/GpEmT4OLionZcKq+hts8Fe8MhJC28vb3h6OjYZFZ0QUEB/Pz8rNQqdTNnzsSPP/6I7du3o2PHjjrLxsTEAABOnToFAPDz89N4bfX36Srj7u4OV1dXiz1Hnp6e6NatG06dOgU/Pz/U1NSgpKRE62NK5drOnj2Lbdu24fHHH9dZTsqvXcM26XosPz8/FBYWqt1/8+ZNXLlyxSSva8P7m2uLoeqDl7Nnz+LXX39V633RJCYmBjdv3kRubq7OtjdstzWvr6HQ0FB4e3ur/U1K/fWr98cffyA7O7vZ/0vANl9DbZ8LtvTeqU9b9MEARgtnZ2f069cPaWlpqmNKpRJpaWmIjY21YsvqltjNnDkT3377LX777bcmXZaaZGVlAQD8/f0BALGxsTh8+LDam079m26PHj1UZRpef32Z+uu31HNUUVGBnJwc+Pv7o1+/fmjVqpXaY2ZnZyMvL0/1mFK5tjVr1sDHxwdjxozRWU7Krx0AhISEwM/PT+2xysrKsHfvXrXXrKSkBJmZmaoyv/32G5RKpSqAi42NxY4dO3Djxg21a+revTu8vLz0um592mKI+uDl5MmT2LZtG9q3b9/sOVlZWXBwcFANvdjy9TV2/vx5FBcXq/1NSvn1a+iTTz5Bv379EBkZ2WxZW3oNm/tcsKX3Tn3aohe9p/u2QBs2bBDkcrmwdu1a4dixY8ITTzwheHp6qs3QtoZp06YJHh4eQnp6utpyvmvXrgmCIAinTp0SFixYIOzfv184c+aM8N133wmhoaHC0KFDVXXUL5cbNWqUkJWVJaSmpgodOnTQuFzuhRdeEI4fPy4sX75c43I5Uz9Hzz33nJCeni6cOXNG2LVrl6BQKARvb2+hsLBQEIS65XedOnUSfvvtN2H//v1CbGysEBsbK4lrq1dbWyt06tRJmDNnjtpxqb525eXlwsGDB4WDBw8KAISlS5cKBw8eVK3CWbRokeDp6Sl89913wqFDh4SxY8dqXEbdp08fYe/evcLOnTuFrl27qi3DLSkpEXx9fYWHH35YOHLkiLBhwwbBzc2tyRJVJycnYcmSJcLx48eFefPmaVyi2lxbxFxfTU2NcM899wgdO3YUsrKy1P4n61du7N69W3jnnXeErKwsIScnR1i/fr3QoUMHYfLkyTZ/feXl5cLzzz8vZGRkCGfOnBG2bdsm9O3bV+jatatQVVUlidevuWusV1paKri5uQkrVqxocr6tv4bNfS4Igm29dzbXFn0wgGnG+++/L3Tq1ElwdnYWBgwYIOzZs8faTRIAaLytWbNGEARByMvLE4YOHSq0a9dOkMvlQlhYmPDCCy+o5RIRBEHIzc0VRo8eLbi6ugre3t7Cc889J9y4cUOtzPbt24WoqCjB2dlZCA0NVT1GQ6Z+jhISEgR/f3/B2dlZCAwMFBISEoRTp06p7r9+/bowffp0wcvLS3BzcxPuvfde4dKlS5K4tno///yzAEDIzs5WOy7V12779u0a/yanTJkiCELd0tBXX31V8PX1FeRyuTBy5Mgm115cXCxMnDhRaNOmjeDu7i4kJiYK5eXlamX++usvYfDgwYJcLhcCAwOFRYsWNWnLV199JXTr1k1wdnYWevbsKWzZskXtfn3aIub6zpw5o/V/sj63T2ZmphATEyN4eHgILi4uwm233Sa8+eabagGArV7ftWvXhFGjRgkdOnQQWrVqJXTu3FmYOnVqk0DXll+/5q6x3kcffSS4uroKJSUlTc639dewuc8FQbCt90592tIc2T8XTkRERCQZnANDREREksMAhoiIiCSHAQwRERFJDgMYIiIikhwGMERERCQ5DGCIiIhIchjAEBERkeQwgCEiIiLJYQBDREREksMAhoiIiCSHAQwRERFJDgMYIiIikpz/BwaQOfQjUV/EAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(lossi);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd22e888",
   "metadata": {},
   "source": [
    "Let's also determine the train and val losses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9e9fc166",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into eval mode (needed for batchnorm ESPECIALLY)\n",
    "for layer in layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "00474949",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 2.0587270259857178\n",
      "val 2.1071507930755615\n"
     ]
    }
   ],
   "source": [
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]   # That's a switch!\n",
    "    \n",
    "    emb = C[x] # (N, block_size, n_embd)\n",
    "    x = emb.view(emb.shape[0], -1) # Concatenate into (N, block_size * n_embd)\n",
    "    \n",
    "    for layer in layers:\n",
    "        x = layer(x)\n",
    "    \n",
    "    loss = F.cross_entropy(x, y)\n",
    "    print(split, loss.item())\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "744e468b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "damiara.\n",
      "alyzah.\n",
      "fard.\n",
      "azalee.\n",
      "sayah.\n",
      "ayvi.\n",
      "reino.\n",
      "sophemuellani.\n",
      "ciaub.\n",
      "alith.\n",
      "sira.\n",
      "liza.\n",
      "jah.\n",
      "grancealynna.\n",
      "jamaur.\n",
      "ben.\n",
      "quan.\n",
      "torie.\n",
      "coria.\n",
      "cer.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the model\n",
    "for _ in range(20):\n",
    "    \n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all ...\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      emb = C[torch.tensor([context])] # (1,block_size,n_embd)\n",
    "      x = emb.view(emb.shape[0], -1)   # Concatenate the vectors, \"streching them out\"\n",
    "      for layer in layers:\n",
    "            x = layer(x)\n",
    "      logits = x\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "819fda0a",
   "metadata": {},
   "source": [
    "Admittedly, up until now we just refactored the Makemore 3 approach.<br>\n",
    "It's still working exactly as before.<br>\n",
    "**No changes in function.**\n",
    "\n",
    "If you take a look at the loss graph, the graph really looks crazy. It's too noisy.<br>\n",
    "The main cause for this is the batch size.\n",
    "\n",
    "$32$ is too small, because per batch we only have $32$ predictions.\n",
    "\n",
    "This is not enough to get a good average loss value, representing nearly any relationship with performance on whole dataset.<br>Even worse, small batch sizes can make for very lucky or unlucky batches, which can skew the loss graph even more.\n",
    "\n",
    "**Let's fix this horrible loss graph.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60b01d0b",
   "metadata": {},
   "source": [
    "## Addressing the noisy loss graph\n",
    "\n",
    "`lossi` is our accumulated list of float values, representing the loss values per each batch.<br>\n",
    "To make the graph more readable, we can go and average the loss values over a certain number of batches.<br>\n",
    "In this context, recall that we can 'malform' any $m$-dimensional array into an $n$-dimensional array.<br>\n",
    "This will be helpful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d031778a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
      "tensor([[0, 1, 2, 3, 4],\n",
      "        [5, 6, 7, 8, 9]])\n",
      "tensor([[0, 1],\n",
      "        [2, 3],\n",
      "        [4, 5],\n",
      "        [6, 7],\n",
      "        [8, 9]])\n",
      "tensor([[0, 1, 2, 3, 4],\n",
      "        [5, 6, 7, 8, 9]])\n"
     ]
    }
   ],
   "source": [
    "print(torch.arange(10))             # An array with numbers 0-9 -> (10,)\n",
    "print(torch.arange(10).view(2, 5))  # Same array, but transformed -> (2,5)\n",
    "print(torch.arange(10).view(5, 2))  # Same array, but transformed differently -> (5,2)\n",
    "print(torch.arange(10).view(2, -1)) # -1 as joker, infers one dim automatically, given input dimensions -> (2,5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "73bbeeb6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200000\n"
     ]
    }
   ],
   "source": [
    "print(len(lossi))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18f64443",
   "metadata": {},
   "source": [
    "We can mend `lossi` more easily if we turn it into a tensor.<br>\n",
    "`lossi` has a length of $200,000$. We reshape it to be `(200, 1000)`.<br>\n",
    "For each row (carrying $1,000$ values), we calculate the average.<br>\n",
    "With all these individual losses ($=200,000$) in total, we are now down to $200$ averaged losses.<br>\n",
    "This will be more readable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "cf3f71cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([200, 1000])\n",
      "torch.Size([200])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfOElEQVR4nO3deXhU1f0/8Pes2Sch+56wJ2FJgEAEZVGDoLQgLqVuIFWsFdQaay1asfq1xWprcaHgz6UoaF0q4oIiEGQPWyCsIRAg+56QTNaZZOb+/rgzN5nsA0nuJLxfz5OnZO6SczPCvHvO55yjEARBABEREZEDU8rdACIiIqKuMLAQERGRw2NgISIiIofHwEJEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PDUcjegJ5jNZhQUFMDDwwMKhULu5hAREVE3CIKA6upqBAcHQ6nsvA9lQASWgoIChIWFyd0MIiIiugK5ubkIDQ3t9JwBEVg8PDwAiA+s0+lkbg0RERF1h16vR1hYmPQ53pkBEVisw0A6nY6BhYiIqJ/pTjkHi26JiIjI4TGwEBERkcNjYCEiIiKHx8BCREREDo+BhYiIiBweAwsRERE5PAYWIiIicngMLEREROTwGFiIiIjI4TGwEBERkcNjYCEiIiKHx8BCREREDm9AbH7YW4xNZvx9y1k0msx4fk40nNQquZtERER0TWIPSxc+2HsJH6dkw9BklrspRERE1ywGlk5oVM3bXTcysBAREcmGgaUTCoUCaqUYWhpNgsytISIiunYxsHRBoxJ/RY0m9rAQERHJhYGlC9ZhISMDCxERkWwYWLqgVYu/oiYOCREREcmGgaULHBIiIiKSHwNLF6yBhUNCRERE8mFg6YK1hoXTmomIiOTDwNKF5iEh1rAQERHJhYGlC9aiW9awEBERyYeBpQvWheNYw0JERCQfBpYucJYQERGR/BhYusAhISIiIvkxsHRB6mFpYtEtERGRXBhYuiBNazazh4WIiEguDCxdaO5hYWAhIiKSyxUFltWrVyMyMhLOzs5ISEjAoUOHunXdZ599BoVCgdtvv93mdUEQsGLFCgQFBcHFxQWJiYk4f/78lTStx2m5DgsREZHs7A4sn3/+OZKSkvDiiy/i6NGjiI2NxaxZs1BSUtLpdVlZWfjDH/6AqVOntjn22muv4a233sLatWtx8OBBuLm5YdasWWhoaLC3eT2OS/MTERHJz+7A8sYbb2DJkiVYvHgxYmJisHbtWri6uuLDDz/s8BqTyYT77rsPL730EoYMGWJzTBAErFq1Cn/+858xb948jB07Fh9//DEKCgqwadMmux+op2nUlhoWBhYiIiLZ2BVYjEYjUlNTkZiY2HwDpRKJiYlISUnp8LqXX34Z/v7+eOihh9ocu3TpEoqKimzu6enpiYSEhE7v2Ve4DgsREZH81PacXFZWBpPJhICAAJvXAwICcPbs2Xav2bt3Lz744AOkpaW1e7yoqEi6R+t7Wo+1ZjAYYDAYpO/1en13H8Fu3EuIiIhIfr06S6i6uhoPPPAA3nvvPfj6+vbYfVeuXAlPT0/pKywsrMfu3Zp1WrORs4SIiIhkY1cPi6+vL1QqFYqLi21eLy4uRmBgYJvzL1y4gKysLPzyl7+UXjNb1jNRq9XIyMiQrisuLkZQUJDNPePi4tptx/Lly5GUlCR9r9frey20cEiIiIhIfnb1sGi1WkyYMAHJycnSa2azGcnJyZg8eXKb86OionDy5EmkpaVJX3PnzsWNN96ItLQ0hIWFYfDgwQgMDLS5p16vx8GDB9u9JwA4OTlBp9PZfPUWBhYiIiL52dXDAgBJSUlYtGgR4uPjMWnSJKxatQq1tbVYvHgxAGDhwoUICQnBypUr4ezsjNGjR9tc7+XlBQA2r//+97/HK6+8guHDh2Pw4MF44YUXEBwc3Ga9FjlwHRYiIiL52R1YFixYgNLSUqxYsQJFRUWIi4vDli1bpKLZnJwcKJX2lcb88Y9/RG1tLR555BFUVlbihhtuwJYtW+Ds7Gxv83qctDQ/e1iIiIhkoxAEod93Hej1enh6eqKqqqrHh4c+OZiN578+hVmjAvDuA/E9em8iIqJrmT2f39xLqAuc1kxERCQ/BpYuaFl0S0REJDsGli5IewlxHRYiIiLZMLB0Qc2iWyIiItkxsHSB05qJiIjkx8DSBS4cR0REJD8Gli5IewkxsBAREcmGgaULGjV7WIiIiOTGwNIFaw1LE2tYiIiIZMPA0gXWsBAREcmPgaULUg0L12EhIiKSDQNLF7g0PxERkfwYWLqgZdEtERGR7BhYuqBWikNCTWYBZjN7WYiIiOTAwNIF67RmAGg0s5eFiIhIDgwsXbBOawZYx0JERCQXBpYuaFoGFs4UIiIikgUDSxdUSgUsZSwsvCUiIpIJA0s3SFObWXRLREQkCwaWbrDWsXBIiIiISB4MLN3ADRCJiIjkxcDSDdLy/AwsREREsmBg6QYuz09ERCQvBpZu4I7NRERE8mJg6QbrkBCLbomIiOTBwNIN1h4W1rAQERHJg4GlG1jDQkREJC8Glm7QsoaFiIhIVgws3aBRW2pYGFiIiIhkwcDSDRwSIiIikhcDSzdwWjMREZG8GFi6gTUsRERE8mJg6QZpaX6uw0JERCQLBpZuULOGhYiISFYMLN3AGhYiIiJ5MbB0g1bFac1ERERyYmDpBi7NT0REJC8Glm7QqC1DQk2sYSEiIpIDA0s3WHtYmszsYSEiIpIDA0s3sIaFiIhIXgws3SDVsHBIiIiISBYMLN3Aac1ERETyYmDpBqnoloGFiIhIFgws3aBRsoaFiIhITgws3dC8DgtrWIiIiOTAwNINzeuwsIeFiIhIDgws3cBpzURERPJiYOkGzhIiIiKSFwNLNzQHFtawEBERyYGBpRvYw0JERCQvBpZu0KpZw0JERCQnBpZu4JAQERGRvBhYukGttK7Dwh4WIiIiOTCwdAOHhIiIiOTFwNIN0pAQF44jIiKSBQNLN7CGhYiISF4MLN3QvJeQGYLA0EJERNTXGFi6Qatq/jU1mRlYiIiI+hoDSzdoLEW3AAtviYiI5MDA0g2aFj0srGMhIiLqewws3aBWsoeFiIhITgws3aBQKKQ6FgYWIiKivsfA0k1qlWXxuCYOCREREfW1Kwosq1evRmRkJJydnZGQkIBDhw51eO7GjRsRHx8PLy8vuLm5IS4uDuvXr7c5p6amBsuWLUNoaChcXFwQExODtWvXXknTek3Lqc1ERETUt9T2XvD5558jKSkJa9euRUJCAlatWoVZs2YhIyMD/v7+bc739vbG888/j6ioKGi1Wnz//fdYvHgx/P39MWvWLABAUlISduzYgQ0bNiAyMhJbt27FY489huDgYMydO/fqn7IHaDgkREREJBu7e1jeeOMNLFmyBIsXL5Z6QlxdXfHhhx+2e/6MGTMwf/58REdHY+jQoXjyyScxduxY7N27Vzpn//79WLRoEWbMmIHIyEg88sgjiI2N7bTnpq9pVdxPiIiISC52BRaj0YjU1FQkJiY230CpRGJiIlJSUrq8XhAEJCcnIyMjA9OmTZNenzJlCr799lvk5+dDEAT8/PPPOHfuHG655ZZ272MwGKDX622+eptGzR4WIiIiudg1JFRWVgaTyYSAgACb1wMCAnD27NkOr6uqqkJISAgMBgNUKhX+/e9/Y+bMmdLxt99+G4888ghCQ0OhVquhVCrx3nvv2YSallauXImXXnrJnqZfNamGhUW3REREfc7uGpYr4eHhgbS0NNTU1CA5ORlJSUkYMmQIZsyYAUAMLAcOHMC3336LiIgI7N69G0uXLkVwcLBNb47V8uXLkZSUJH2v1+sRFhbWq8/AGhYiIiL52BVYfH19oVKpUFxcbPN6cXExAgMDO7xOqVRi2LBhAIC4uDikp6dj5cqVmDFjBurr6/Hcc8/h66+/xpw5cwAAY8eORVpaGv7xj3+0G1icnJzg5ORkT9OvmrWGpcnMwEJERNTX7Kph0Wq1mDBhApKTk6XXzGYzkpOTMXny5G7fx2w2w2AwAAAaGxvR2NgIpdK2KSqVCmYHCgfNQ0KO0yYiIqJrhd1DQklJSVi0aBHi4+MxadIkrFq1CrW1tVi8eDEAYOHChQgJCcHKlSsBiPUm8fHxGDp0KAwGA3744QesX78ea9asAQDodDpMnz4dzzzzDFxcXBAREYFdu3bh448/xhtvvNGDj3p13JzEX5W+oUnmlhAREV177A4sCxYsQGlpKVasWIGioiLExcVhy5YtUiFuTk6OTW9JbW0tHnvsMeTl5cHFxQVRUVHYsGEDFixYIJ3z2WefYfny5bjvvvtQUVGBiIgI/PWvf8Wjjz7aA4/YMzxdNAAAfX2jzC0hIiK69igEQej30170ej08PT1RVVUFnU7XKz/jhU2nsP5ANh6/aRievmVkr/wMIiKia4k9n9/cS6ibrD0sVexhISIi6nMMLN3k5crAQkREJBcGlm7SsYeFiIhINgws3cQhISIiIvkwsHQTAwsREZF8GFi6SQosdQwsREREfY2BpZta9rAMgJngRERE/QoDSzdZZwk1mQXUGU0yt4aIiOjawsDSTS4aFTSWDRBZx0JERNS3GFi6SaFQsPCWiIhIJgwsduBaLERERPJgYLEDe1iIiIjkwcBiBwYWIiIieTCw2IFrsRAREcmDgcUOXuxhISIikgUDix04JERERCQPBhY7cJYQERGRPBhY7MAeFiIiInkwsNiBgYWIiEgeDCx2sAYWPQMLERFRn2JgsYOnK3tYiIiI5MDAYgcvFy0AoLK+EYIgyNwaIiKiawcDix2sQ0Ims4Bao0nm1hAREV07GFjs4KxRQqsSf2UcFiIiIuo7DCx2UCgUzWuxcHl+IiKiPsPAYidPFzUA9rAQERH1JQYWO3EtFiIior7HwGInrsVCRETU9xhY7OTlKk5tZg8LERFR32FgsZO1h6Wy3ihzS4iIiK4dDCx28rKsdpt/uV7mlhAREV07GFjsNGWoLwBge3oJ6rl4HBERUZ9gYLFTfMQghA5yQY2hCdvSi+VuDhER0TWBgcVOSqUCd4wLAQBsPJonc2uIiIiuDQwsV2D++FAAwJ7zZSipbpC5NURERAMfA8sVGOzrhnHhXjCZBXybViB3c4iIiAY8BpYrZB0W+vY4AwsREVFvY2C5QrNGBQIATuZXQd/AReSIiIh6EwPLFfLXOSPSxxWCAKRmXZa7OURERAMaA8tVmBjpDQA4lFUhc0uIiIgGNgaWqzBpsCWwXGJgISIi6k0MLFfBGlhO5FWioZGr3hIREfUWBparEO7tCn8PJzSaBBzLqZS7OURERAMWA8tVUCgUUi/LYdaxEBER9RoGlqvEOhYiIqLex8BylayB5WjOZTSazDK3hoiIaGBiYLlKI/w94O2mRZ3RhL2ZZXI3h4iIaEBiYLlKSqUCc2ODAQD/O8Ldm4mIiHoDA0sPuDte3L1525liVNYZZW4NERHRwMPA0gNGBXsiJkgHo8mMb7h7MxERUY9jYOkh1l6WL1NzZW4JERHRwMPA0kPmxYVAo1LgVL4e6YV6uZtDREQ0oDCw9BBvNy0SowMAAF+y+JaIiKhHMbD0IOuw0Ka0fBibuCYLERFRT2Fg6UHThvvB38MJFbVG7DhbIndziIiIBgwGlh6kVikxf3wIAOB/LL4lIiLqMQwsPezuCWEAgJ8zSlFS3SBza4iIiAYGBpYeNszfHePDvWAyC9h0LF/u5hAREQ0IDCy94PZx4rDQrnOlMreEiIhoYGBg6QXjwwcBAE7l6yEIgsytISIi6v8YWHrBiAAPaFVKVNU3IreiXu7mEBER9XsMLL1Aq1YiOsgDAHAiv1LexhAREQ0ADCy9ZHSIJwDgZH6VzC0hIiLq/64osKxevRqRkZFwdnZGQkICDh061OG5GzduRHx8PLy8vODm5oa4uDisX7++zXnp6emYO3cuPD094ebmhokTJyInJ+dKmucQxoZaAkseAwsREdHVsjuwfP7550hKSsKLL76Io0ePIjY2FrNmzUJJSfsru3p7e+P5559HSkoKTpw4gcWLF2Px4sX46aefpHMuXLiAG264AVFRUdi5cydOnDiBF154Ac7Ozlf+ZDIbE+IFQOxhYeEtERHR1VEIdn6aJiQkYOLEiXjnnXcAAGazGWFhYXj88cfxpz/9qVv3GD9+PObMmYP/+7//AwD8+te/hkajabfnpTv0ej08PT1RVVUFnU53RffoaY0mM0a9+BOMTWbs/MMMRPq6yd0kIiIih2LP57ddPSxGoxGpqalITExsvoFSicTERKSkpHR5vSAISE5ORkZGBqZNmwZADDybN2/GiBEjMGvWLPj7+yMhIQGbNm3q8D4GgwF6vd7my9FoVErEBIm//BOsYyEiIroqdgWWsrIymEwmBAQE2LweEBCAoqKiDq+rqqqCu7s7tFot5syZg7fffhszZ84EAJSUlKCmpgavvvoqZs+eja1bt2L+/Pm44447sGvXrnbvt3LlSnh6ekpfYWFh9jxGnxljLbzNq5S3IURERP2cui9+iIeHB9LS0lBTU4Pk5GQkJSVhyJAhmDFjBsxmMwBg3rx5eOqppwAAcXFx2L9/P9auXYvp06e3ud/y5cuRlJQkfa/X6x0ytIwJ5UwhIiKinmBXYPH19YVKpUJxcbHN68XFxQgMDOzwOqVSiWHDhgEQw0h6ejpWrlyJGTNmwNfXF2q1GjExMTbXREdHY+/eve3ez8nJCU5OTvY0XRbWHpbTlhVvFQqFzC0iIiLqn+waEtJqtZgwYQKSk5Ol18xmM5KTkzF58uRu38dsNsNgMEj3nDhxIjIyMmzOOXfuHCIiIuxpnsMZ6ucOjUqBakMT8iu54i0REdGVsntIKCkpCYsWLUJ8fDwmTZqEVatWoba2FosXLwYALFy4ECEhIVi5ciUAsd4kPj4eQ4cOhcFgwA8//ID169djzZo10j2feeYZLFiwANOmTcONN96ILVu24LvvvsPOnTt75illolUrMdTPHWeLqnG2sBqhg1zlbhIREVG/ZHdgWbBgAUpLS7FixQoUFRUhLi4OW7ZskQpxc3JyoFQ2d9zU1tbiscceQ15eHlxcXBAVFYUNGzZgwYIF0jnz58/H2rVrsXLlSjzxxBMYOXIkvvrqK9xwww098Ijyig7SiYGlSI/EmICuLyAiIqI27F6HxRE54josVu/uuoCVP57FL8YG4Z17x8vdHCIiIofRa+uwkP1GBoqbIJ4tqpa5JURERP0XA0svi7YsHnextAYNjSaZW0NERNQ/MbD0Mn8PJwxy1cAsAJklNXI3h4iIqF9iYOllCoUCUYFiLwuHhYiIiK4MA0sfiAqy1LEUOt6eR0RERP0BA0sfiGLhLRER0VVhYOkD1iGh9EI9dmaU4IvDuTA0sQCXiIiou/pk88Nr3YgADygUQHmtEQ/+5zAAwGAy44Hr+vfWA0RERH2FPSx9wEWrwpShPgAAZ434K2c9CxERUfexh6WPrFs8CVX1jfj5bAme+d8JZJfXyd0kIiKifoM9LH1Eo1LC190Jkb5uAICs8lqZW0RERNR/MLD0sQgfccfmgsp6GJvMMreGiIiof2Bg6WN+7k5w1apgFoC8yxwWIiIi6g4Glj6mUCgQ7i32srCOhYiIqHsYWGQQ6cM6FiIiInswsMjAWsfCHhYiIqLuYWCRQYSlhyWbPSxERETdwsAig0j2sBAREdmFgUUGEZa1WHIv18FkFmRuDRERkeNjYJFBoM4ZWpUSjSYBBZX1cjeHiIjI4TGwyEClVCDM2wUAh4WIiIi6g4FFJpzaTERE1H0MLDIJtxTeHrpUgX2ZZSjRN8jcIiIiIsfF3ZplYu1h+fZ4Ab49XgA/Dyfs/MMMuDnxLSEiImqNPSwymT06EPERgzDUzw3uTmqUVhvw30M5cjeLiIjIITGwyCRA54z//W4Kkp+egRd+EQ0AeHf3RTQ0mmRuGRERkeNhYHEA88eFItjTGaXVBnyZmid3c4iIiBwOA4sD0KqV+O30oQCAtTsvILOkWuYWERERORYGFgexYGIY/DyckF9Zj8Q3duP21fuQWVIjd7OIiIgcAgOLg3DWqLBu8UQkRgdApVQgLbcSyz49ikaT2ea89SlZuOPf+1BabZCppURERH2PgcWBjAr2xPuL4rHnjzdikKsGZ4uq8e6uC9Jxs1nAqu3ncTSnEt8eL5BezyqrRVVdoxxNJiIi6hMMLA4o2MsFL/5yFADgreRMaWjodIEe5bVGAMCBi+UAgLNFesz81y4s++9ReRpLRETUBxhYHNS8uGDcONIPRpMZL313GgCw61yJdPzQpQqYzQK+P16IRpOA1OzLEATu/ExERAMTA4uDUigUeHneaCgVwJ7zZcgsqcauc6XS8ar6RqQX6bE9vRgAUGc0oazGKFdziYiIehUDiwML83bFzJgAAMA7OzJxNKcSADAywAMA8OWRPJwtap4CnVPBjRSJiGhgYmBxcAsnRwIANqUVwGQWMNTPDfPHhwAANhzItjk3p6Kur5tHRETUJxhYHNyUoT4Y6ucmfT99hD+uG+IDAGgyizUrWrX4NmaXM7AQEdHAxMDi4BQKBRZNiZS+nz7SD6ODdXDTqqTX5sUGA2APCxERDVwMLP3AHeNDEaBzQqDOGQmDvaFWKTFxsDcAYJi/O6aN8AMA5LCHhYiIBii13A2grrk7qfHT76cBEFfEBYA5Y4KwM6MUd08IRYSPKwD2sBAR0cDFwNJPeLlqbb6/a0IoEgb7IMzbBVX14iq3JdUG1BtNcGkxXERERDQQcEion1IoFAj3cYVCoYCXqxY6ZzF75l5mLwsREQ08DCwDRLhlWKirmUJ5l+twPLeyD1pERETUcxhYBohw767rWEqqG/DLt/fijjX7kdvOefqGRhibzO1cSUREJC/WsAwQ4d7iWi055e2vdisIAp7beBKXLbs6p+VWIszbFQ2NJry/5yK2nSnGifwqRAXqsPnxG6BUKvqs7URERF1hD8sA0bKH5b3dF3Hbm3uQVdYcXr46mo/t6c2bJ6YX6gEA36YV4B9bz+F4XhUEQXz9VEFV3zaeiIioCwwsA4R1avO+zHL89Yd0nCnUY83OCwCAshoDXvpW3PF5RIA7AEh7EB3JrgAg7g49Y6S4nsv2M8V92nYiIqKuMLAMENYeFqOpuQblm+P5qKprxHt7LqLa0ITRITq8PG80gOYeluO5Ym/KnDFBmGtZMXcrAwsRETkYBpYBIsjTGc4a8e1cMnUwogI90NBoxvt7L2J9irhJ4lOJIxATrAMAFFY1IL+yHudKxJ6WuHAv3DjSH0qF2PvSXlEuERGRXBhYBgi1Som37xmPV24fjedui8Z910UAAN7ekYk6owmjgnW4KcofOmcNQge5AAC+OJwLQQBCvFzg7+GMQW5axEeKS/4np7OXhYiIHAcDywAyMyYA918XAYVCgfnjQmw2SHzi5uFQKMSZP9FBYi/Ll0dyAQCxYZ7SebfEBACATYEuERGR3BhYBih3JzXuGB8KAIgK9MDM6ADpWHSgBwCgoKoBABAX5iUdu9ly3oGL5dA3iFOgBUHA1tNFOJZzGYIg9EXziYiIbHAdlgHsqZkjoFAA90wKt1lXxdrDYhUb6iX9ebCvG4b4uuFiWS0OXaxAYkwAdp0rxSPrUwEAo0N0WHbjMMweHdQnz0BERASwh2VA83bT4uV5o9sElJbfq5QKjAn1tDkeF+4FADiZL84gOnCxQjp2Kl+PZZ8ekzZcJCIi6gsMLNegcG9XuFrqW0YEeMBVa9vRNjZEDDDWwHIirxIA8MfZIxHi5YIms4A07kdERER9iIHlGqRUKjDSUscSF+bZ5vgYyxDRibwqmM0CTuaJwWX6CD9MjBwEADiafblvGktERAQGlmvWTMtsoFvbqUWJCdJBpVSgrMaAlIvlqDY0wVmjxIgAD4yPsASWHDGwVDc04sO9l1BZZ+y7xhMR0TWHRbfXqEenDcV9kyLg6appc8xFq8Jwf3ecLarGhgPionOjgj2hUSkxPlwMLGm5lTCbBfxz6zms25+F1JzLWH3v+D59BiIiunawh+UapVQq2g0rVmMthbjWZfqt30cFesBFo0J1QxPSi/TYlJYPAPjxZCGyO9gpGgAqao0wmzklmoiIrgwDC7XLWsdisoQM61otapVSCi//2nYelXXibCGzALy/5xIaGk34/WfH8Nv1R9Bo2ddo84lCjP+/bViz68IVtaVY34BV289xZhIR0TWMgYXaZZ0pJH3fYq2WcZZhoe2W5fvHW6ZBf5mai9+sO4xNaQX46XQxvjteAEEQ8GbyOQBicLHafKIQq3/O7Favy183p2PV9vP4f7uvLPAQEVH/x8BC7YoK8oBGJS42p3NWI9LHVTpmDShWr9w+BmNCPNHQaMb+C+XS6+/uuoi9mWU4V1wDADhbpEeNoQkNjSYkfZGG13/KwO7zpZ22o9Fkxs8Z4jYBh7M4M4mI6Fp1RYFl9erViIyMhLOzMxISEnDo0KEOz924cSPi4+Ph5eUFNzc3xMXFYf369R2e/+ijj0KhUGDVqlVX0jTqIU5qlTT1OTbMS9qHCIA0UwgQa1pignX47fQhAABnjRLvPjABbloVMoqr8ez/TkjnmgXgWM5lHLpUAUOTOFz0TVpBm599Mq8KF0vFkHMk6zKqG5oAiOvBWIeZWssoqsaSj4/gdEHV1Tw2ERE5KLsDy+eff46kpCS8+OKLOHr0KGJjYzFr1iyUlLS/WZ63tzeef/55pKSk4MSJE1i8eDEWL16Mn376qc25X3/9NQ4cOIDg4GD7n4R6XHyEt83/Wvm6OyHcW+xxuWN8CABgzpgg/PPuWHz1uymYNSoQ9yaEAxD3K1IomntlDmddxu5zzb0qW04VodbQJH1/qawW8/+9D/P/vR+VdUbsONu8a3RDoxlnC6vbbetHKVnYdqYYGw7kXOVTExGRI7I7sLzxxhtYsmQJFi9ejJiYGKxduxaurq748MMP2z1/xowZmD9/PqKjozF06FA8+eSTGDt2LPbu3WtzXn5+Ph5//HF88skn0Gg6nr1CfeepxBF45fbReGTakDbHnp8TjbsmhOLehAgAgEKhwJ0TQjEqWKx9+c0Ng6UhpZtG+ksbMaZmV2DP+TLLNUB9ownbzjSHki+O5KLJLKCqvhHv7bmI5LNiEHbRiCvzWtd/ae1MgR4AkFtRd9XPTUREjseuwGI0GpGamorExMTmGyiVSExMREpKSpfXC4KA5ORkZGRkYNq0adLrZrMZDzzwAJ555hmMGjWqy/sYDAbo9XqbL+p5nq4a3H9dBFwsy/i3NGtUIP5xdyzcndpfyifI0wWLrx8MrVqJpTcNQ7xlhdzDWZeRUVwNhQJYeJ0Ydr4+Jk6NbjKZ8VVqnnSP9/ZcwsXSWqiVCtxn6bFpL7CYzAIyisSel+yKjqdWExFR/2VXYCkrK4PJZEJAQIDN6wEBASgqKurwuqqqKri7u0Or1WLOnDl4++23MXPmTOn43//+d6jVajzxxBPdasfKlSvh6ekpfYWFhdnzGNRHlt8ahTMvzcL48EEY4e8BD2c1jJbalbEhnnjw+sEAgD3nS1FabcDOjFKUVBvg46bFqGCddO6kwd6YPtIPQPuBJau8FvWNJgBAQWVDh3UuRETUf/XJLCEPDw+kpaXh8OHD+Otf/4qkpCTs3LkTAJCamoo333wT69atsyns7Mzy5ctRVVUlfeXm5vZi6+lKKRQKqFXif2JKpQITWhTrTh3uh8G+bogL84JZAJZ+ehT/2X8JADB/XAj+MGukdO5NUf6IC/OCQgHkVtSjtNpg83Osw0GA2NtSUFlvc7y6oRHHOhhKsmpoNEm9NERE5HjsCiy+vr5QqVQoLi62eb24uBiBgYEd/xClEsOGDUNcXByefvpp3HXXXVi5ciUAYM+ePSgpKUF4eDjUajXUajWys7Px9NNPIzIyst37OTk5QafT2XyR45sY2Vy8O3W4LwDgudui4e6kxqFLFdiXKU6JXjAxDDNG+OHGkX7wcFbjtjFB8HDWYIS/OGupdS/LmULbIcGcVnUsT39xHPP/vR97LbUz7fnn1gzMWrUbW04VdngOERHJx67AotVqMWHCBCQnJ0uvmc1mJCcnY/Lkyd2+j9lshsEg/r/kBx54ACdOnEBaWpr0FRwcjGeeeabdmUTUf1kDi7uTWpoaPWmwNzYtnSKt8zIu3AvDAzygUCjw3sJ4pP55JoK9XAAA4yO8AACv/ngWk/66Hfe/fxAms4D0VoElu7w5sJTVGKTC3ZSLHQeWQ5Y1Xn481fHQZmd2ZpTgle/PwNBkuqLrrX44WYh5q/chq4y1OERELdm9+WFSUhIWLVqE+Ph4TJo0CatWrUJtbS0WL14MAFi4cCFCQkKkHpSVK1ciPj4eQ4cOhcFgwA8//ID169djzZo1AAAfHx/4+PjY/AyNRoPAwECMHDkSNHBMjByEP90ahWF+7tComrPyMH8PbFp6PT49lINbYpp76tQq2zw9IcIb/z2Ui0uWD/OSagN+PlsiDQnFhnnheG6lzUyhH08WStsLtBw6ai3Hsg/S/gvlEAShy+HJhkYTNColVEoFBEHA8o0nUVjVgBEBHvjVxCuvqVq3LwvHcyvxUUoWXvxl1wXoRETXCrsDy4IFC1BaWooVK1agqKgIcXFx2LJli1SIm5OTA6Wy+YOmtrYWjz32GPLy8uDi4oKoqChs2LABCxYs6LmnoH5BoVDg0elD2z3m5arFYzOGdXr93NhgXCytgatWhQultfj6WD5WJZ9DSbUBCgVwS0wAjudW2vSwfHu8eWG69A7WcKmqb8Rly55IpdUGZJbUYHiAR4ftOFdcjdve3IP7r4vAX+aOwsWyWhRWNQAAfjxVeMWBRRAEnCsR27g9vRgrfhHT7bouIqKBzu7AAgDLli3DsmXL2j1mLaa1euWVV/DKK6/Ydf+srKwraRYNcFq1En+cHQVAXG9lU1o+TuWLvSaDfdwQZVmZ11rDkl9Zj8NZl6FQAIIAFOkbUF5jgI+7k819c8pta172XyjvNLDsyyxDk1nAZ4dz8MfZI21qY/ZmlkHf0Aids/1rCZVWG6TNJHMr6nG+pAYjOmkHEdG1hHsJUb8U5u2Km6Oap9dHB+kQYamDyamogyAI2HxC7F2ZFOkt1ci018vSeu2W/Rc6rnUBxDABiCvv7sooxd7M5vMbTQKS04uRX1mPX61Nwft7Lra4rg4bj+ZJQ1StWfdcsmq5oB4R0bWOgYX6rUVTIqQ/xwTrEDpIDCU1hiZcrmuUhoN+GRuM6CBxJtmZwrZ7DVmHkIb4uQEAUi6UdxgqANtZSN+dKMABy4aPMyxrxWw+UYinPkvDoawKrN11EYIg3uu5r08i6Yvj2HG2/W0szhWLYcq6QrB1N+zO5FfWX3WhLxFRf8DAQv3WDcN8MSLAHQAwPnwQnDUqBOqcAYir557K10OjUuDW0YGIsQQWaw9LdUOjtMBctqXg9hdjguDhpIa+oanTAt28y82B5YeTRag2NMHLVYM/3CIWiW9PL8GhrAoA4iylwqoGNJnMOGKZiWTd2LG185b6lfnjxP2Z0nIr26w509LhrArc8Pcd+Mu3pzs8h4hooGBgoX5LoVBg3eJJ+GBRPCYPFWeaWTdl/Ne2cwCAO8aFwsfdCTHBlh6WAj0yS6ox5dUd+M26wwCALKmHxR0JQ8Sp1x0NCwmCIPWwOGua//pMGeqDUcE66ecDzfsfncirxLnimhar8doubGdlHRK6YbgfxoZ6QhBgs/ljaz+cLIQgAFtPF0u9OEREAxUDC/VrwV4uuDm6uZYl3Kd5WEihAB6ZLm7caA0smaU1ePHb06huaMKe82Uo0TdIRbcRPq64fpi4oN3HKdmoshTAtlRRa0Sd0QSFArhrQqj0+g3D/KBQKHC7pXdkbmww5sWJu44fz6tCWm6ldG5+ZUOb+wqCIA0JjQhwR6Llmf66OR0/nW5/bZgUy1BUea3RZmbU1fjueAH+s+9Sj9yLiKgnMbDQgNKyh+PW0YEY6icOGQXqnDHIVQOTWZBW1AWAbenFKNKLASLCxw13x4chwscV+ZX1+ONXx9v0XFh7VwI8nHF7XIj0+g2WoLPsxmFY/9Ak/PNXsRgb6gVA7GFJy21enbe9HpZivQHVDU1QKRUY7OuGRVMiERfmBX1DE367PhVvbM2wOb+8xoCzLbYSSM3ufOuB7rhca8RTn6fhpe/OILOk/WErIiK5MLDQgGKdKQTAZs0XhUIhFd4CgJerOO34kwM5AAAPZzUGuWrg7qTGO/eMh0alwE+ni/FxSrbN/XMvi2Ej3NsV48MH4Z5JYXjohsFSz45WrcTU4X7QqJQYG+oJADiRV4VjOZXSPQqr2gYWa+9KpI8rnNQqeLpo8MVvJ2PJVHGDyLd2ZOKwpS4GAA5eqrC5vr1NIe310+kiNFmKjVuvHkxEJDcGFhpQJg32hruTGnNjg6UeDitr4a23mxav3xULoHkfoggfV2mRtjGhnlh+azQAYOWP6Sipbh7Csa6iG+rtAqVSgZV3jMULv4hpty0jAz2gVStR3dCE8y16LC7XNaLO2ARAXLSu0WRuMRzUvO6KVq3E83NicM8kcSG6Fd+clmYvWYeDQgeJ2xZ0p4flze3n8dfNZzqsd9l8snkfJWt7iIgcBQMLDShBni5IWzETqxbEtTl2d3wYRofo8Pc7x2L6CD+4aVXSsQgfN5tzF18fiXHhXmhoNOPfP1+QXrcGlpZDTx3RqJRSSAKAEC8XeDiJazUWVDagqKoBCX/bjpv+uRNbLWuutLdg3TOzouDpokF6oR6fHhR7fFIuioHlt5ZepIzialQ3tK25saqsM+Jf28/hvT2X2q2Jqag1Yv+F5qGys53sXF1raLIJcd1RWm3AXzefwc6M9qd0ExF1hYGFBhy1Sgmlsu2S9iMDPfD941MxMyYAWrVSmlkEABGtAohCocAzs8Rpyp8czJamMltrWMIGdR1YACDWMiwEAHHhXtJGjgWV9Th4qRwNjWbkVtTjkGWIxzpNuyVvNy3+cMsIAMDrP2Vg84lCZJbUQKEAfjk2CKGDXCAIwPFccY0Z63Ttli6UNi+O969t52Futc7M1tNFMJkFOKnFfxIyOggsDY0m3L56H6a/trPNCsEdOXCxHLe9tQfv7bmEhz46wgXxiOiKMLDQNWvqcD/pzy1rX6ymDPXF9cN80GgS8Ob28wCAXEtwCetGDwsAm2GpcWFeCPYS14kpqKyXhqOsvS4AEBWoQ3vuTYhAbKgn9A1NWPrpUQBAdKAOXq5aTLDsfL09vRiLPjyE4c//iNEv/oRZ/9qNY5balkstdn/OKK62Gf4BmoeD7r9OXIwvp6JOGrZq6c3k8zhfIk7R/vxITpfPn5xejHvfO4DSagPcndQwmQUs/fQo9md2vpqwVZ2xCanZlzltm4gYWOjaNW1Ey8Di1u451sXgvjqah4yiahRYpiR3Z0gIAGLDmntYxrXqYbEuTven26Lwz7tjseIXMRjm37aHBQBUSgU2PJyAu1tMpbb2EFkDy7r9Wdh1rhSAOK07o7ganx3KBQBcKhNraFwtw2Crtp+T6mHKagzScNAD10XA110LoO1WAacLqvD/djdvNfBVan6nKwID4vRwsyDO2Nq//CbcEhMAY5MZv92QCn0nQ1jWds17Zx/uXLMfyendG0q6kmBT3dCI0wVtV0AmIsfCwELXrEgfV0yMHARfd620Tktr48IHYWZMAMwC8OxXJ2AyC9CqlfD3cGr3/NaG+LojOkiHSB9XjAr2lAJLXovAMirYE3dOCMVvbhjc6b08nDV4/e5YrL1/An4xNggPTokEIK7yaxXu7Ypvl12P/7t9NAAgvUj8GdYelkemDYGniwYXSmvx6SGxh+SdHZkwmQXEhnkh0tcNIy2bSGYUNc8UMpkFLN94EiazgMToAHi5alCkb8Ce86Udtreh0YSDl8Qg9GTicOicNXjrnnEY5u+O6oYmbEzN6/DashoD7vl/B6Ri5QMXyzs81+pEXiWmvLoDf950Ek3tDIt15OkvjmPOW3ul3qjWz1BraNvTdKUaGk3Yf6GszZCc3ExmAQ98cBCPfHyEvVnksBhY6JqlUIi9FnufvanT3ZWfvmUEFApIi7+FDnJpt0amPUqlAt8uux5bn5oOZ40KIZbAcjy3EuW1RigVwEg7d2SePToQ79w7XhqWig7SYdaoAMweFYhvll6PsaFe0rowGUXVaDKZcdFSwxIb6oWnEocDAP7+41nszyzDhgNiIe8fLTU7IwN0lmube1i2ni7CibwqeDir8bf5o6U1aL480nHoOJp9GQ2NZvh5OEnP6KxRYeFkcdhpw8GcDj8cf7s+FedLamD9NZ/qogdEEAS89N0ZFFY1YMOBHDzx2TEYm5pDS2WdER+nZLU7zHUqX7y3deuElvec+85eTPrrdnx+uOO2fnE4Fyt/TO/WB/1byedx73sH8aGDLc53qawWe86XYeuZYmnHcCJHw8BC1zQntQrOGlWn50QF6jAvNlj6vrsFt1YalRJaSzGrtYfFWgQ7xM8dLtrOf35XVEoF3n0gHmsfmIBBbuJwToS3K1y1KhiazLhQWiv1sAzxc8MDk8UZUDWGJiz88BCazAJujvKXVvkdGSgOS2UUiz0sgiDgXctQ0INTIuGvc8bd8eLQ1LYzxcgpr2v3w3r3ebFOZepwX2nKOCDuleSqVSGzpAYHLla0ua66oVGapv3mr8cBAE7n6zvtldhxtgSp2ZehVSuhVSnxw8kiPPV5mnT8H1szsOKb0/jbD+k21xmbzCi0LBzYeip33uV6nCuuQa3RhGe/Oonfrk+1CUEAUG804c+bTuHdXRdxPK/rYSXrxpdfH8vv8tzu2nyiECe78bM70/LZ8y63v3UEkdwYWIi64feJI6C2/N/9MG+XK76PtejWquW0556kVCoQZRna2XG2BIYmMzQqBUK8XKBSKvD3O8dCo1KgySxApVRg+W3R0rUjA217WFKzLyMttxJalRILJ0cCEIexYoJ0MJrMmPb6z4h/ZTve2JphM0PJOlw0dbivTds8nDXSFgbW3p2WrNsM+LhpMXt0ILQqJaoNTVLBc2tms4DXfxJXAl48JRLvL4qHUiEWEltXFbbW9nyVmm+z5UJhVT2sWetcq9V9rT0vni4aaFQKbD1TjG/SbIPG4awKGC3PfKGL1YEv1xql6eKnC/TSFPmrcTTnMpZ+ehSPbki9qvu0nBWW18HvmUhuDCxE3RDp64YHLEMZ8RHeV3yfAJ0zWnQ2dFg70xOsK/tuPlkAQKxvUavEv/IjAjzw+0RxqvSDUyJtin2tU6vLagworzFIhbZ3jA+BX4vanWdvjcIQPzcoFOJ+Rm/tyMRda1OQVVaL8hoDTltqdKw9Ny3dnyD+Ln86XYTzrXo2rFPHw31coVEpERUkBq9T+e2vvrspLR9ni6rh4aTGo9OHYtoIP8SGeQEQQ1NuRR1yK8TgUt9owmeHm2c3texNOF9cbdOLYx2GunV0oBTUzrRaAXhfi9lOF8tsA4vZLOC1LWelvZkOZdn2JnW0R5Q9ktPFKeL5lfUoq+l4Z++uWHcKt96LyBExsBB10wtzYrDtqWmY22J4yF4alRIBHs29LL3VwwI0hyHrB/0QP9sZSI/NGIptT03D8y16VwDAVauWZkEt/fQotlk+FB+ealsUPH2EH3Y8PQNnXpqNN38dB52zGsdzKzHnrT14+fszAMTQ5O9h26tkbVvCYG80mQXMW70Pm1oMkWSVi8NXkZaZW6OCxZlW7c3kOZ5biee/PgUA+O30IdKQ2DTLlPXd58qkVYE1KjEpfpySLRXltuxNqDOabD6sT1p+b6NDPFsUItuGq30tdvW+UFJrcyw15zL+vfMCXvruDLLLa3HQMvzl4SxOY7cGlrIaA0qr2w8bGUXVqKwztnsMAHZmNBc9ny3senXiPedL8f6ei22G8Gx7WBhYyDExsBB1k1KpwPAAj24X3Hak5bBQdC8Gltb3HuJrO3Vboej4ecaFewEADlysgCAAidEBGObffnGwi1aFeXEh+PH30zBpsDdqjSZ8kyb26rQeDmrp7XvG4boh3qgzmvD7z9Pw5RFxCrZ1QTpraBplDV4Ftr0b2eW1+M26w6hvNGHqcF9p1V8AmD5SDCx7zpdit2Vo6sEpkfB20yK/sh7bLSHM2vNiZe1pEAQBpy1DQqNDPKWi4Za1HhW1RqkXCWjbw9JygbzPDudKM6aW3TgMAHAk+zLWp2Rhyqs7MHvVblTU2gaTw1kVmP3mbixed7jdGqGS6gabn3/WMqurztiEI1kVba7Jr6zHko+P4JXN6dh5rjnoNDSakNViEUAOCZGjYmAh6mPWwlt/DyebIZaeFhXoYTP8NNi3/bVm2rPiFzH4+51j8Mrto/H3O8fg9bvGdnlNiJcL/rvkOjwzayRUlhA0Y6Rfh+f765zxycPX4Z5J4QCAHyyL10k9LL5iYBkdYulhya+y+RB+8rM0lNcaMSpYhzX3T4BG1fzPWWyoFzxdNNA3NOHHU2JPxo1R/rjX8rM+OSgOC7X+cLauPVNY1YDyWiNUllqg4QHuUCiAshqjNPSScqEcggAMsmykmVVWJ/XcCIJgG1gO5UjDSbePC0FsmBcEAXjhm9MwNplRXmvEu7svtGwK3tt9EYIAHMupbHc4bPc528X3rPf/v+/P4K61KVJotPrbD+loaBTb99Op5uGoi6W1NuvpsIeFHBUDC1Efs05t7s36FUAc2hncYkE8ewKLj7sTFkwMx/3XRWDBxHBpqKUrKqUCS28chu8fvwHv3DsOk4f4dHn+XZbF8E7m6yEIglR0G+4ttjcq0AMqpQLltUYUWWb01BqacDyvEgDw7gMT4N5itWDrfW+w9O5YtxwYHz4Ic8YGARBDgNksSB/O1gJlaw+KteB2uL87nDUqm2Gyc5bhE+tw0Ly4EDiplTCazNL9LpTW4FJZLbQqJXzdnXC5rhGCIK79E6BzxqxRAVJbp1gWAPx4f7Y0NJRbUSf1AgFod1Vh675M1ranF4o1OD+dFq/7ocVqxikXyrH5RPP3284USyHF+szWBQPzL9dDEATUG034Ji0fDY2mNj+7Myaz4HDrzNDAwMBC1MduivKHr7sT5ltmyvSm6BahqHUNS6/+3CAdfjE22GY6c0dignRQKRUoqzEgp6IOhVViKIm0bJfgrFFhuKUo2NrTkFFcDUEQe6lCO5hmPr3F1gvxkYOk+zhrlKgxNOFiWY0UMG6K8gcAnLf0sFgDy5iQ5pWKrTtpZ1g+4K0FtzcM85XCoHVYyBoapgzzkXbbBoCEwWI4WRAfhqnDffHnOdH45OEExIV5ob7RhDU7xV6W9QfEFYIDdeLw4TfHClBvbA4OTSYz9limjf9uhjgUlllSjbS8Smloaf+FcjSazDCZBbz03WkAwK8nhsHTRYPyWqM0ddz6PNNHiL+DakMT9PVNWLMzE09+lobXtmS0+/ttjyAIuP/9g7j+7zs63YyT6EowsBD1sYQhPjjy50TMi+v9wGIt6vVwUkv/D9rRuGibA4l1TyN3JzW8W/TqtC68tRaYRnVSA9Ry64UpQ8XeFrVKidGWex3Ouoxiy67TUmApEXsprPUyo1sElqgWhbfZ5bXILq+DSqlAwhBvDLW031p4ax0OmhkTgF9PCpcWwEsYIs4w83F3wvqHEvDw1CFQKBRIminO2NpwMBvv7DiPzyyrEL9y+2iEebug2tCEH08195Ck5Vaiqr4ROmc15owJgoeTGo0mAev2ZUnn1BiacCynEjszSnC2qBo6ZzWenR2Fmy3Pai36tfYYxYV5Sv+N5FXWYY8lkH2Tlt9m5eBGk7ndWpes8jqkXCxHYVUDjuVUtvOuEF05BhaiAcxaPBsdrOtWb4dcrMHg++Pih3KEj6tNe8eEiMHE+iFoLTCNDux4leBAT2fERwyCSqnAzdH+0uvWDSl/PFUEQQBcNCrEhXlBq1aKu2dfrsPJFgW3Vi17WKwr/E4Z6gMPZw2GtuhhKdE3SKsiJ0YHIMTLBUumDkFsqCdujm4eCmpp6nBfTB3uC2OTGf/Yeg76hiZE+Ljipih//GqC2EPzcUo2SvQNOHSpAr/7RNwAc/pIf6hbTP22Bj5njfhP++5zpVK9zoKJYRjkpsUtowIBAFtOFUEQBKmHZUSAB0IsvVWZJTVSL1N5rRH7LjRvjdDQaML97x/E1Nd+xtZWU7N3tyjmbT0FnOhqMbAQDWCTh/hgzX3j8c+7Y+VuSqesQy/WD7nWu2dPHCz2TBzJqkCTydyih6XzbQ3eWxiPLU9OtdkF27ohpXXH6DBvF6hVSgy1DJmtTxFrSZQK22nnUp1LUTU+t8xoshYMt+xh2WiZoh0b5oUAy5DO8tui8c2yG+Dp0v4WEAqFAu8tjMdrd43FlKE+cFIrkTRzBJRKBe6KD4XSsjXEpL8l49f/LwWl1QYM93eXtlOwzgiz1qU8fMMQAOIaNT9bal3utax9M32EH5w1SuRX1uOn08XSsNiIAA+EWuqrfjxZhEZTcx3Kt5YCXrNZwDP/O4GDl8TZY3/59rTNdgctA8vpAgYW6lkMLEQDmEKhwK1jgqR9hxxVy54MoO3u2dGBOni6aFBrNOFkfpW0qWPLINKeQW5aDG+1V5O1h6XJ8uFurYGxLpj3/l5xobfoIJ3NtgmRvm7QqBSoNZpQWm2Ar7sWiZYeE2vYySiuxtpdYh3KA9dFdOPJmzlrVPhVfBg+XXIdMl65VRoyDPJ0wb/vGy8thmcWgLmxwdi09HrpfW35ewjzdpEWOcy7LK7k27LOxkWrktapsa6Q6+fhhEFuWoQOEgOLdQsB66rOP50uQp2xCa9uOYvvjhdArVTA190JBVUNeCs5E4C4zUFKi00qrcN3Zwr0mLwyGV8czrXr90HUmrrrU4iIeldMkA5KhfhhDIh7IbWkVCowabA3tp0pxtfH8lHd0ASNSiEFBXtE+rhC56yGvkHsGbB+SE8f4Ydv0gowyFWDX4wNxuLrI22u01h6YazL698dHybtEWUNA1X1YqHpMH/3Hi2qnj06CLNHB6GkugEFlQ2IDfW0GTKLbtHTNHW4HwJ0zogK9JDael9CuM39/jh7JJrMAvZmlsHYZMakSLEHy/q7sG43sGhyJP6zLwv5lfW47c090notr945Fp4uGiz5+Aje33MRd00IQUm1AXVGEzyc1Kg2NOFSWS1qDU3YcDAbhVUN+Ol0EX41MQxEV4qBhYhkJxbeekj1FK17WAAgwRJY/pcq1o8M9XOXAoM9FAoFxoZ6Ya91SMjSwzJ/XAgmDfZGgM7ZZk2Xlka2CAH3TGwOAW5OagR5OksznP5wywhpLZqe5O/h3O7KwSMta+4IAjDNMp172gg/nC2qhp+HExJjbGtnhvl74MMHJ6LG0ISTeVXS0FfrGVcTI71RXmvEmp0XkFVeBxeNCn+ZGyNNRb85yh/JZ0vw6IajmGgJPYkxAdiXWYaSagPOFunxs6W3ptrQdqdsIntwSIiIHELLYaHWNSwAcJ1lTZc6y/Teq1kleGxo88+y9iooFAqEDnLtMKwAkGYYTR3ui/BWbRziJ4asMSGemGUpbO0rrlo17hwfithQT0y1DPfclxCOUcE6PHdbVIfP5O6kxuShPvC0LH4XMqh5Y08XjQoxwTr8emIYvN20GB/uhc1P3IAFLYLaX+ePQaDOGZklNfivZWbTtBG+0urEXx3Nl0JcdQMDC10d9rAQkUMYG+qJr47mQatWSuuPtBQdpLMZyonqZIZQ1z/LS/pzR+u4tOe+68LRaDbjjnGhbY7dOT4U2eV1+MvcUbLMyPpHq8LqCB83bH5iql33sC5qCABxYV7QqJSI8HHD4ecT2+0xCvR0xscPTcJda/ZL78vU4X64UFKLnzNK8T/LbCoAqDFwXRa6OuxhISKHMMkyEygmSNfu/kYqpQKTBjevnNvZGixdsc4UApoLS7vDVavGYzOGIdCzbaC6Y3wo9j57EyZEDLridsnNrcX6NxMjm5+js+GtEQEe+ODBiXB3UmP6CD/4ujtJPSzGFuu31LCHha4Se1iIyCFEB+nwv0cnd9rjcd0Qb2nJ+s7WYOlKkKcLnrYs1ubl6pgL6sllVLAOe86XYeqIjveBam1ipDcOPX8znNUqyz0825xTY2iCIAgOvR4QOTYGFiJyGPGWws2OWPcHCvZ0vuqNIx+/efhVXT9Q/ePuWGSV1UpFtN3lqm3+OAnzdoGHsxrVDU0I8XJBfmU9Gk0CDE1mOGtUndyFqGMcEiKifiMqUIcPH4zHe4vi+f/Ue0mAzhkJXWxa2RWFQiHNPPrF2CBp13AW3tLVYGAhon7lpqiAdoccyLEsu2kYbo7yx+LrB8Pd0vtSw6nNdBU4JERERD1u6nA/aYq1u7O4mBwLb+lqsIeFiIh6lbuT+P+Nqzm1ma4CAwsREfUqD2dLYGEPC10FBhYiIupV7s7iSrocEqKrwcBCRES9ysOJRbd09RhYiIioV7kzsFAPYGAhIqJe5W6pYdE3sOiWrhwDCxER9Spr0S1rWOhqMLAQEVGv4pAQ9QQGFiIi6lXsYaGewMBCRES9yt1JnNbMdVjoajCwEBFRr5IWjuOQEF0FBhYiIupV1llCNVyan64CAwsREfUqaeE4DgnRVWBgISKiXtXcw9IEQRBkbg31VwwsRETUqzwsewk1mgQYmswyt4b6KwYWIiLqVa4aFRQK8c+cKURXioGFiIh6lVKpgLuWi8fR1WFgISKiXufOxePoKjGwEBFRr7Muz1/dzgaIFbVG/GbdYWw5VdjXzaJ+RC13A4iIaODrbPG4H04WYsfZElTUGjF7dFBfN436CfawEBFRr3O3zBRqb0goo6gaAJBbUdenbaL+hYGFiIh6nUcnOzZnFIuBpbzWiFoW5VIHGFiIiKjXuXcQWARBkHpYACD3MntZqH0MLERE1OusNSz6VkW3JdUGVNU3v5ZbUd+n7eprZrOAYzmX0dBokrsp/Q4DCxER9bqOpjWfbdG7AgA5A7yO5bsTBZj/7/14bUuG3E3pdxhYiIio13U0JHSuVWAZ6IW3+zLLAABnCqtkbkn/w8BCRES9zqODHhZrwW2AzgnAwA8sJ/LEoFJQ2SBzS/qfKwosq1evRmRkJJydnZGQkIBDhw51eO7GjRsRHx8PLy8vuLm5IS4uDuvXr5eONzY24tlnn8WYMWPg5uaG4OBgLFy4EAUFBVfSNCIickDuTuK05tZ7CVkLbm+ODgAwsItu64xNOGcJaIVV9TCbuXO1PewOLJ9//jmSkpLw4osv4ujRo4iNjcWsWbNQUlLS7vne3t54/vnnkZKSghMnTmDx4sVYvHgxfvrpJwBAXV0djh49ihdeeAFHjx7Fxo0bkZGRgblz517dkxERkcNob+E4k1nA+RLxA3ymNbBU1EMQBuYH+ZkCPawZpdEkoLTGIG+D+hm7V7p94403sGTJEixevBgAsHbtWmzevBkffvgh/vSnP7U5f8aMGTbfP/nkk/joo4+wd+9ezJo1C56enti2bZvNOe+88w4mTZqEnJwchIeH29tEIiJyMNai29JqA744kgtvVy2G+LmhodEMZ40Sk4f6QKEA6htNKKsxws/Dya776xsaobMsTueojufZ1q3kV9YjQOcsU2v6H7t6WIxGI1JTU5GYmNh8A6USiYmJSElJ6fJ6QRCQnJyMjIwMTJs2rcPzqqqqoFAo4OXl1e5xg8EAvV5v80VERI5LZwksZTUG/PF/J/Dwx0fw8EdHAADD/T3grFEhyPLhbe9MoR9OFmLsX7biqc/TYGwy92zD7VSsb+iwh+hEXqXN9wWVA3sKd0+zK7CUlZXBZDIhICDA5vWAgAAUFRV1eF1VVRXc3d2h1WoxZ84cvP3225g5c2a75zY0NODZZ5/FPffcA51O1+45K1euhKenp/QVFhZmz2MQEVEfG+Lrjl/GBmNUsA5Th/tCq1LiYlktAGBEgAcAINTbFQCQZ2cdy+aT4qaJXx/Lx8MfH8G3xwvw5GfHsPTTo9hxthimPqoV2XQsHwl/S8a7uy+2e9xacOvrrgXAwGKvPtn80MPDA2lpaaipqUFycjKSkpIwZMiQNsNFjY2N+NWvfgVBELBmzZoO77d8+XIkJSVJ3+v1eoYWIiIHplQq8PY946TvT+VXYdmnR5FVXofxEV4AgHBvVxy6VIGc8u4HFkEQcOhShfgzFMDuc6XYfa5UOr75RCHCvV3x/qJ4KRhZNZrMaGg0waOHhpK+PS5OFvnPvktYMnUIVEqFdKyqvhGXLAFtZkwA/nsot89mChmaTFj6yVFU1Tfio99Mgqu2f+57bFerfX19oVKpUFxcbPN6cXExAgMDO7xOqVRi2LBhAIC4uDikp6dj5cqVNoHFGlays7OxY8eODntXAMDJyQlOTvaNbxIRkeMYHeKJH56ciuO5VZgYOQiAGFgA+2YKZZXXobTaAK1KiY8fmoSkz9PgrFXhlphANJnM+DI1DzkVdVj6yVF89/gNcNaopGt/s+4wjudW4vvHpyLcx/WqnqfRZMbBi+UAgGK9AXvOl2LGSH/p+Kl8sXclzNsFMcGeAHKRd7lvelhe+T4d29PFiTEf7LmEx28e3ic/t6fZNSSk1WoxYcIEJCcnS6+ZzWYkJydj8uTJ3b6P2WyGwdBcHW0NK+fPn8f27dvh4+NjT7OIiKgfctWqMXmoD9Qq8aMozNsFgH3L8x+6JIaEuDAvXDfEB/uX34wdT8/An26Nwp9/EYPkp6fD190J50tq8Lcf0qXrKmqN2HO+DPqGJnywt/0hnJZK9A3459YMlHUws+dEXiVqjc3L7f8vNc/m+HFL/crYUC+EeIm1OtYhoar6RhRVdd7bUlBZj9Tsii7b2dpXqXlYfyBb+n7trgsdPkNrhiYT1uy8gMyS6q5P7gN2T2tOSkrCe++9h48++gjp6en43e9+h9raWmnW0MKFC7F8+XLp/JUrV2Lbtm24ePEi0tPT8c9//hPr16/H/fffD0AMK3fddReOHDmCTz75BCaTCUVFRSgqKoLRaOyhxyQiIkdn7WGxp+j2oGU4aNJg73aP+7o74Z+/igUAfJySjZ/Pij0Nh7OaP/y/OJKHyrrOP29e+u4M3t6RiRXfnGr3+P5MMTgN83cHAGw9U4yquuY9ko5mVwIAxoZ4IthLDGYFVeIU7gc+OIib/rkTJfr2Q8vW00WY+cYu3LkmBRuP5rV7TntyK+rw3NcnAQBP3DwcY0M9UWs04c3t57t1/acHc/D3LWdx//uHuvz99AW7A8uCBQvwj3/8AytWrEBcXBzS0tKwZcsWqRA3JycHhYWF0vm1tbV47LHHMGrUKFx//fX46quvsGHDBjz88MMAgPz8fHz77bfIy8tDXFwcgoKCpK/9+/f30GMSEZGji/Bxg0IhTvddu+tCt6451EVgAYDpI/yw+PpIAMDqnzNtrgPEqdSfHMzp8PqS6gb8dFqcWPLjqSKcL27b47Dvgrjk/qIpkYgK9ICxyYxvT4g1LXXGJuzNFOtqbhjuixBLYKmsa8SZQj1O5FWhzmjCyfy2y/X/e2cmHlmfKvXevPjN6TZFyeeKq3G2qO1s2U8P5cDQZMakSG/8/ubheO62aOl1az1NZ7adEcs/ivQNeP7rU7Kvj3NFK90uW7YM2dnZMBgMOHjwIBISEqRjO3fuxLp166TvX3nlFZw/fx719fWoqKjA/v37sWDBAul4ZGQkBEFo96t1US4REQ1cvu5OWHajWO/46o9n8dTnaVjxzSk8+dmxdocl8ivrkXe5HiqlAuMjBnV670emDQEApOZcRrG+QephmTHSDwCwbn8WDE3t76D85ZE8NFlmGglCc+ixqjeapB6U64f64K4JoQCAzw/nQBAE7MwoRUOjGRE+rogJ0sHDWSMtpPf54VzpPq1DxLniammTxEWTIzAhYhCqDU1I+uK4NPNJ39CI+av34bY39+DDvZekUNFoMuPLI2JvzG9uiIRSqcB1Q3wwdbgvTGYBW051PLPXel9rqFMpFdh8shBfHc3v9Jrexr2EiIjIYTx9y0g8d1sUAHGa8scp2fgmrQAPfXQEVfXiEIuhyYSyGoNUvzI6WCdtrtiRIE8XjAv3giAAG4/mS0WwL88djQCdE0qrDdjYzgeyySzgU0vvi7WX5tvjBTieW4kjWRU4W6THkewKGE1mBHk6Y7CvG+aPC4GzRolT+Xrsv1COHyzTrmePDoRCIc4csvayfH2s+WdmldsGFuuxxOgAvDRvNN74VSzctCoculSB/x4S27QzoxS1RhPMAvDy92ewfONJNJrMSE4vRlmNAb7uTtK2BwCkQuAjWZ3Xw+zKKEWTWcBQPzckzRwBAHjxm1Oy7vXUP+c2ERHRgPXItKGI9HHD9vRi+Hk4YdOxAmSX1+GZL4/j5mh//O2Hs6iqb5SmDXc2HNTSbaODcCynEv/emQmzAIQOckG4jysemTYU//f9Gfxz6zn8YmyQzTTn3edLkV9ZD08XDZ6dHYWsslr8nFGKeav3See4acWZR+JqvQr4uDthQXwYPkrJxpvbz+NUQZX0862CvVxwtqjaZm+lrLLmMGA2C/jGEljuGB8CQBwye2rmCLyyOR3/PZSD+6+LkIZtRgXrkF6ox2eHc1FWY0B9o9hbdNeEUGhUzX0T1hlZR7Ivw2wWoGwx9bql5HTxvonRAXh0+lDsPleK2DAv+Ovkm6HLHhYiInI4t4wKxGt3xeKZWVFYc/94aFVKbD1TjGe/Oin1tFiHRW6KCujsVpLZo8XlN6whwRp0HrguAoN93VBWY8A7P2dCEARsO1OMP311An/83wkAwJ3jQ+GsUSFp5khoLQEg2NMZWrVSqi+ZOtxX+llLponrsBzKqkCd0YQQLxeMDfWUjgd7tV2Sv+WQ0OGsChRUNcDDSY2bopqnR98xPhQalQKnC/Q4lV+FnZYi4pfnjcZ7C+PhpFZie3oJ9lmKgH890XaNspggHVy1KlTVN+J8SU27v6cmkxk/Z4g1NzdHB0ClVGDDwwl47rZoOKlV7V7TF9jDQkREDm1sqBdW/DIGf950Ci4aFZ6+ZQTumRSOnIo6mMwCRod4dn0TAGHerhgT4ikVt06KFAOLVq3En+dE46GPjuA/e7OQcqFcWpUWEHtQFk6OAACMCfXE4ecToVYp4Oakhr6hEVtPF+NyrRFzY0Oka0IHuWJebDA2WnpJWg4HAZBmCgFi0NlzvgwFVfVoaDTBWaPCpjTxulvHBNqsHePtpsWNI/2x9Uwxnvv6JKoNTfB1d8K4MC8olQr858GJePjjI6gzmjB5iA8ifd1sfgdqlRLjwr2wL7Mch7MqMDKweTE9QRCgb2jCibxKVNU3wstVg/HhXgBg00sjFwYWIiJyePdfF4GYYB1CvFykDQOjgzpeYLQjt44JbA4sLYaSboryx7QRfth9rhQn8qrgqlVhwcQwTB3ui/hIb5uNFT1dm/+sc9ZIRbatPTpjqBRYbh1tu7hqSIvAMn9cCI7lVKLG0ITcijqE+7hi8wmx7uX2uBC0dueEUGw9UyyFqsRof2loZ8owX3zycALe3XURj904tN12xUd4Y19mOY5kVeD+68QgZjYLuO/9g0ixLH4HADeO9JfWyHEEDCxERNQvjA/vfCZQd/xiTDBWbT+PEC8XDG7R+6BQKPB/80bh6S+OIyZYh8dvGm73jtGtjQjwwP/dPhrlNQZMaDWLKcSmh8UPkb6uOJWvx6WyWlwqq4W+oQmBOmckDGm7kOqNI/0xyFWDy5Z1XmbG2A6JjQsfhLUPTOiwXRMtPUuHsy5Lr313osAmrGjVSiyY6Fhb3jCwEBHRNSPcxxU/PHED3J00NkM0gFjU+r/fTenRn/eApQejtdEhnhgb6omRAR7w83BCpI8bTuXrkVVei0xLbcns0YE2+xFZadVKzI0Nxkcp2XDRqHD9MN8253QmLtwLKqUC+ZX1KKish4+7Vpo+nTRzBJZMHQKlErLWq7SHgYWIiK4pw/w9uj6plzlrVPh22Q3S99benktltdhxVix4TYzuuJj4gcmR+OZ4gVQMbA93JzVignQ4mV+FvZllqKwzIr+yHgE6JyyZOgQuWscKKlYMLERERDKL9BEDy9bTxSivNcJNq+p0uvYwf3ekrbjlin9efOQgnMyvkmZBAcDTM0c6bFgBOK2ZiIhIdtbZPOW14p4900b4QavuvY/oO8eHInRQcx3N2FBP3NlB8bCjYA8LERGRzAa3mn7ccu2V3jA6xBN7n70JxiYzymsN8HFzardexpGwh4WIiEhmg1w10Fn2F1IompfQ721atRJBni692pvTUxy/hURERAOcQqGQelliQ72uekr1QMTAQkRE5ACiAsWF8Fqvq0Ii1rAQERE5gKdnjUBsmFeHK+de6xhYiIiIHIC/hzPuTQiXuxkOi0NCRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOb0Ds1iwIAgBAr9fL3BIiIiLqLuvntvVzvDMDIrBUV1cDAMLCwmRuCREREdmruroanp6enZ6jELoTaxyc2WxGQUEBPDw8oFAoevTeer0eYWFhyM3NhU6n69F7O4qB/owD/fkAPuNAMNCfD+AzDgQ9/XyCIKC6uhrBwcFQKjuvUhkQPSxKpRKhoaG9+jN0Ot2A/I+vpYH+jAP9+QA+40Aw0J8P4DMOBD35fF31rFix6JaIiIgcHgMLEREROTwGli44OTnhxRdfhJOTk9xN6TUD/RkH+vMBfMaBYKA/H8BnHAjkfL4BUXRLREREAxt7WIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOj4GlC6tXr0ZkZCScnZ2RkJCAQ4cOyd2kK7Jy5UpMnDgRHh4e8Pf3x+23346MjAybc2bMmAGFQmHz9eijj8rUYvv95S9/adP+qKgo6XhDQwOWLl0KHx8fuLu7484770RxcbGMLbZPZGRkm+dTKBRYunQpgP75/u3evRu//OUvERwcDIVCgU2bNtkcFwQBK1asQFBQEFxcXJCYmIjz58/bnFNRUYH77rsPOp0OXl5eeOihh1BTU9OHT9G5zp6xsbERzz77LMaMGQM3NzcEBwdj4cKFKCgosLlHe+/9q6++2sdP0r6u3sMHH3ywTdtnz55tc05/fg8BtPv3UqFQ4PXXX5fOceT3sDufD9359zMnJwdz5syBq6sr/P398cwzz6CpqanH2snA0onPP/8cSUlJePHFF3H06FHExsZi1qxZKCkpkbtpdtu1axeWLl2KAwcOYNu2bWhsbMQtt9yC2tpam/OWLFmCwsJC6eu1116TqcVXZtSoUTbt37t3r3TsqaeewnfffYcvv/wSu3btQkFBAe644w4ZW2ufw4cP2zzbtm3bAAB33323dE5/e/9qa2sRGxuL1atXt3v8tddew1tvvYW1a9fi4MGDcHNzw6xZs9DQ0CCdc9999+H06dPYtm0bvv/+e+zevRuPPPJIXz1Clzp7xrq6Ohw9ehQvvPACjh49io0bNyIjIwNz585tc+7LL79s894+/vjjfdH8LnX1HgLA7Nmzbdr+3//+1+Z4f34PAdg8W2FhIT788EMoFArceeedNuc56nvYnc+Hrv79NJlMmDNnDoxGI/bv34+PPvoI69atw4oVK3quoQJ1aNKkScLSpUul700mkxAcHCysXLlSxlb1jJKSEgGAsGvXLum16dOnC08++aR8jbpKL774ohAbG9vuscrKSkGj0Qhffvml9Fp6eroAQEhJSemjFvasJ598Uhg6dKhgNpsFQej/7x8A4euvv5a+N5vNQmBgoPD6669Lr1VWVgpOTk7Cf//7X0EQBOHMmTMCAOHw4cPSOT/++KOgUCiE/Pz8Pmt7d7V+xvYcOnRIACBkZ2dLr0VERAj/+te/erdxPaC951u0aJEwb968Dq8ZiO/hvHnzhJtuusnmtf7yHgpC28+H7vz7+cMPPwhKpVIoKiqSzlmzZo2g0+kEg8HQI+1iD0sHjEYjUlNTkZiYKL2mVCqRmJiIlJQUGVvWM6qqqgAA3t7eNq9/8skn8PX1xejRo7F8+XLU1dXJ0bwrdv78eQQHB2PIkCG47777kJOTAwBITU1FY2OjzfsZFRWF8PDwfvl+Go1GbNiwAb/5zW9sNvzs7+9fS5cuXUJRUZHNe+bp6YmEhATpPUtJSYGXlxfi4+OlcxITE6FUKnHw4ME+b3NPqKqqgkKhgJeXl83rr776Knx8fDBu3Di8/vrrPdrV3tt27twJf39/jBw5Er/73e9QXl4uHRto72FxcTE2b96Mhx56qM2x/vIetv586M6/nykpKRgzZgwCAgKkc2bNmgW9Xo/Tp0/3SLsGxOaHvaGsrAwmk8nmlw8AAQEBOHv2rEyt6hlmsxm///3vcf3112P06NHS6/feey8iIiIQHByMEydO4Nlnn0VGRgY2btwoY2u7LyEhAevWrcPIkSNRWFiIl156CVOnTsWpU6dQVFQErVbb5kMgICAARUVF8jT4KmzatAmVlZV48MEHpdf6+/vXmvV9ae/voPVYUVER/P39bY6r1Wp4e3v3y/e1oaEBzz77LO655x6bjeWeeOIJjB8/Ht7e3ti/fz+WL1+OwsJCvPHGGzK2tntmz56NO+64A4MHD8aFCxfw3HPP4dZbb0VKSgpUKtWAew8/+ugjeHh4tBlu7i/vYXufD93597OoqKjdv6vWYz2BgeUatHTpUpw6dcqmvgOAzZjxmDFjEBQUhJtvvhkXLlzA0KFD+7qZdrv11lulP48dOxYJCQmIiIjAF198ARcXFxlb1vM++OAD3HrrrQgODpZe6+/v37WusbERv/rVryAIAtasWWNzLCkpSfrz2LFjodVq8dvf/hYrV650+CXgf/3rX0t/HjNmDMaOHYuhQ4di586duPnmm2VsWe/48MMPcd9998HZ2dnm9f7yHnb0+eAIOCTUAV9fX6hUqjZV0MXFxQgMDJSpVVdv2bJl+P777/Hzzz8jNDS003MTEhIAAJmZmX3RtB7n5eWFESNGIDMzE4GBgTAajaisrLQ5pz++n9nZ2di+fTsefvjhTs/r7++f9X3p7O9gYGBgmyL4pqYmVFRU9Kv31RpWsrOzsW3bNpvelfYkJCSgqakJWVlZfdPAHjRkyBD4+vpK/10OlPcQAPbs2YOMjIwu/24CjvkedvT50J1/PwMDA9v9u2o91hMYWDqg1WoxYcIEJCcnS6+ZzWYkJydj8uTJMrbsygiCgGXLluHrr7/Gjh07MHjw4C6vSUtLAwAEBQX1cut6R01NDS5cuICgoCBMmDABGo3G5v3MyMhATk5Ov3s///Of/8Df3x9z5szp9Lz+/v4NHjwYgYGBNu+ZXq/HwYMHpfds8uTJqKysRGpqqnTOjh07YDabpcDm6Kxh5fz589i+fTt8fHy6vCYtLQ1KpbLNUEp/kJeXh/Lycum/y4HwHlp98MEHmDBhAmJjY7s815Hew64+H7rz7+fkyZNx8uRJm/BpDd8xMTE91lDqwGeffSY4OTkJ69atE86cOSM88sgjgpeXl00VdH/xu9/9TvD09BR27twpFBYWSl91dXWCIAhCZmam8PLLLwtHjhwRLl26JHzzzTfCkCFDhGnTpsnc8u57+umnhZ07dwqXLl0S9u3bJyQmJgq+vr5CSUmJIAiC8Oijjwrh4eHCjh07hCNHjgiTJ08WJk+eLHOr7WMymYTw8HDh2WeftXm9v75/1dXVwrFjx4Rjx44JAIQ33nhDOHbsmDRD5tVXXxW8vLyEb775Rjhx4oQwb948YfDgwUJ9fb10j9mzZwvjxo0TDh48KOzdu1cYPny4cM8998j1SG109oxGo1GYO3euEBoaKqSlpdn83bTOrNi/f7/wr3/9S0hLSxMuXLggbNiwQfDz8xMWLlwo85OJOnu+6upq4Q9/+IOQkpIiXLp0Sdi+fbswfvx4Yfjw4UJDQ4N0j/78HlpVVVUJrq6uwpo1a9pc7+jvYVefD4LQ9b+fTU1NwujRo4VbbrlFSEtLE7Zs2SL4+fkJy5cv77F2MrB04e233xbCw8MFrVYrTJo0SThw4IDcTboiANr9+s9//iMIgiDk5OQI06ZNE7y9vQUnJydh2LBhwjPPPCNUVVXJ23A7LFiwQAgKChK0Wq0QEhIiLFiwQMjMzJSO19fXC4899pgwaNAgwdXVVZg/f75QWFgoY4vt99NPPwkAhIyMDJvX++v79/PPP7f73+WiRYsEQRCnNr/wwgtCQECA4OTkJNx8881tnr28vFy45557BHd3d0Gn0wmLFy8WqqurZXia9nX2jJcuXerw7+bPP/8sCIIgpKamCgkJCYKnp6fg7OwsREdHC3/7299sPvDl1Nnz1dXVCbfccovg5+cnaDQaISIiQliyZEmb/9PXn99Dq3fffVdwcXERKisr21zv6O9hV58PgtC9fz+zsrKEW2+9VXBxcRF8fX2Fp59+WmhsbOyxdiosjSUiIiJyWKxhISIiIofHwEJEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOj4GFiIiIHB4DCxERETm8/w+uHxf0TSnAcQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(torch.tensor(lossi).view(-1, 1000).shape)         # Each row has 1000 consecutive losses\n",
    "print(torch.tensor(lossi).view(-1, 1000).mean(1).shape) # Row-wise mean -> 200 means for 200 rows\n",
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));   # Mean on every Row -> 200 means as array"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bfef403",
   "metadata": {},
   "source": [
    "This is a very simple approach to make the loss graph more readable. Elegant nonetheless.<br>\n",
    "For example, we can now clearly see where the learning rate was decreased, and that this indeed resulted in a finer approach of the \"perfect\" local minimum i.e. less error overall."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a2e47e4",
   "metadata": {},
   "source": [
    "## Improving the Forward Pass\n",
    "\n",
    "The next thing we want to improve is the forward pass itself. <br>\n",
    "It's too bloated, too many lines of code.<br>\n",
    "Also the embedding table `C` is special-cased outside of the 'regular' layers.<br>\n",
    "The `emb.view` also happens outside of the layers as an additional step.<br>\n",
    "That is costly, because it's not parallelized.\n",
    "\n",
    "*In simpler terms:* There's this part in the forward pass code that is `A` to `Z` unnecessary:\n",
    "\n",
    "```python\n",
    "emb = C[Xb]\n",
    "```\n",
    "\n",
    "It's a lookup into the embedding table using the input character's integer representations as indices.<br>\n",
    "We should incorporate this extra operation differently.<br>\n",
    "\n",
    "The exact same goes for the vector flattening right after that:\n",
    "\n",
    "```python\n",
    "# Concatenate vectors\n",
    "# emb.shape[0] is batch size, \n",
    "# -1 for automatic inference of block_size * n_embd\n",
    "x = emb.view(emb.shape[0], -1)\n",
    "```\n",
    "\n",
    "We did organize some layers within the `layers` list already (Linear, BatchNorm1d, Linear),<br>\n",
    "so we could just treat those two operations as layers too and add them. One will be the `Embedding` layer,<br>\n",
    "the other one will be called `Flatten`.\n",
    "\n",
    "Let's begin with the Embedding Layer. There is a functional equivalent in PyTorch, called `torch.nn.embedding`.<br>The layer looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "671c0e17",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Does the indexing operation\n",
    "# Mimicks torch.nn.embedding\n",
    "class Embedding:\n",
    "  \n",
    "  def __init__(self, num_embeddings, embedding_dim):\n",
    "    # What was previously C (lookup table) is now the weight in this layer\n",
    "    self.weight = torch.randn((num_embeddings, embedding_dim)) # Behaves just like C did\n",
    "\n",
    "  # This is the forward pass (syntax: embedding(IX))   \n",
    "  def __call__(self, IX):\n",
    "    self.out = self.weight[IX]\n",
    "    return self.out\n",
    "  \n",
    "  # Returns list of all layer parameters (syntax: embedding.parameters())\n",
    "  def parameters(self):\n",
    "    return [self.weight]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "edd30a45",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Does the vector flattening\n",
    "class Flatten:\n",
    "  # This is the forward pass (syntax: flatten(x))\n",
    "  # x is a tensor of shape (N, block_size, n_embd), where N is the batch size, n_embd is the embedding dimension\n",
    "  def __call__(self, x):\n",
    "    # Concatenate the vectors, \n",
    "    # x.shape[0] is the batch size, \n",
    "    # -1 means \"infer the rest\" (in this case, block_size * n_embd)\n",
    "    self.out = x.view(x.shape[0], -1)\n",
    "    return self.out\n",
    "  \n",
    "  # Returns list of all layer parameters (none in this case, syntax: flatten.parameters())\n",
    "  def parameters(self):\n",
    "    return []"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb31ab39",
   "metadata": {},
   "source": [
    "This approach greatly improves readability and streamlines the layer processing, as we can now add these layers to the old `layers` list and plainly iterate over them as well.<br> Same function, better understanding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "c594ccab",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "391a6ebf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12097\n"
     ]
    }
   ],
   "source": [
    "n_embd   = 10  # Dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "layers = [\n",
    "    Embedding(vocab_size, n_embd), # NEW: previously C = torch.randn((vocab_size, n_embd))\n",
    "    Flatten(),                     # NEW: previously x = emb.view(emb.shape[0], -1)\n",
    "    Linear(n_embd * block_size, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "]\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# OLD: parameters = [C] + [p for layer in layers for p in layer.parameters()]\n",
    "parameters = [p for layer in layers for p in layer.parameters()]\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e732993",
   "metadata": {},
   "source": [
    "Not only the design, but also the interaction through forward pass and backward pass get simplified through our changes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "c1f8bc13",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.2966\n"
     ]
    }
   ],
   "source": [
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    x = Xb # batch input later gets embedded into vectors\n",
    "    for layer in layers:\n",
    "        x = layer(x)\n",
    "    loss = F.cross_entropy(x, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the gradients\n",
    "    \n",
    "    # Update (mini-batch SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    break # We see if this works first"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e0feda1",
   "metadata": {},
   "source": [
    "## More simplifications\n",
    "\n",
    "Let's now simplify our ... simplification.<br>\n",
    "We store the layers in a list called `layers`.<br>\n",
    "This is not be the most effective way to store them.<br>\n",
    "Let's instead build something like a `container`.\n",
    "\n",
    "Just like in `torch.nn` this should be an organizer for our layers. The particular PyTorch idea that we'll copy is the `Sequential` container.<br>It is able to - on its own - pass input though all the layers that it contains. Just what we need! \n",
    "\n",
    "Let's build this `Sequential` container!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "e3cd4b32",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mimicks torch.nn.Sequential\n",
    "class Sequential:\n",
    "  \n",
    "  def __init__(self, layers):\n",
    "    # Psst, this is basically the same list structure as before\n",
    "    self.layers = layers\n",
    "  \n",
    "  # Simply pass input x through every layer in self.layers\n",
    "  def __call__(self, x):\n",
    "    for layer in self.layers:\n",
    "      x = layer(x)\n",
    "    self.out = x\n",
    "    return self.out\n",
    "  \n",
    "  # Get parameters of all layers, stretch them out into one list\n",
    "  def parameters(self):\n",
    "    return [p for layer in self.layers for p in layer.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccc89bb1",
   "metadata": {},
   "source": [
    "Once again, this moving of processes to classes has positive implications for our network design and interactivity:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "186f5475",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "9bb7a468",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12097\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10    # Dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd), # previously C = torch.randn((vocab_size, n_embd))\n",
    "    Flatten(),                     # previously x = emb.view(emb.shape[0], -1)\n",
    "    Linear(n_embd * block_size, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# Parameters looked like this before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "62cbcf7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.4915\n",
      "  10000/ 200000: 2.2179\n",
      "  20000/ 200000: 2.3681\n",
      "  30000/ 200000: 2.1342\n",
      "  40000/ 200000: 2.4067\n",
      "  50000/ 200000: 2.2406\n",
      "  60000/ 200000: 1.9608\n",
      "  70000/ 200000: 1.9236\n",
      "  80000/ 200000: 2.6587\n",
      "  90000/ 200000: 2.0502\n",
      " 100000/ 200000: 2.2596\n",
      " 110000/ 200000: 1.6270\n",
      " 120000/ 200000: 2.1705\n",
      " 130000/ 200000: 2.2806\n",
      " 140000/ 200000: 2.1980\n",
      " 150000/ 200000: 1.8434\n",
      " 160000/ 200000: 1.8250\n",
      " 170000/ 200000: 2.3077\n",
      " 180000/ 200000: 2.0817\n",
      " 190000/ 200000: 2.1585\n"
     ]
    }
   ],
   "source": [
    "# Same optimization as last time (just written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb) # our new Sequential container in action\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51d3aebd",
   "metadata": {},
   "source": [
    "These simplifications carry their positive effect on readability through all of the pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "d22eea74",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([200000])\n",
      "torch.Size([200, 1000])\n",
      "torch.Size([200])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfKUlEQVR4nO3deVzUdf4H8Nfc3CD3DYIo4gEqikceJWbmZttpZmlWdh8bbdvP2mpr26WtttzaNts2O7Tt2rTTLCVv8US8RUROuQSE4Z5h5vv7Y2a+MAIyw/Ud8PV8PHg84Dvf+c7ny6jz8vN5fz4fmSAIAoiIiIgcmFzqBhARERF1hYGFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIoenlLoBvcFoNKK4uBju7u6QyWRSN4eIiIhsIAgCamtrERwcDLn80n0ogyKwFBcXIywsTOpmEBERUTcUFhYiNDT0kucMisDi7u4OwHTDHh4eEreGiIiIbKHVahEWFiZ+jl/KoAgslmEgDw8PBhYiIqIBxpZyDhbdEhERkcNjYCEiIiKHx8BCREREDo+BhYiIiBweAwsRERE5PAYWIiIicngMLEREROTwGFiIiIjI4TGwEBERkcNjYCEiIiKHx8BCREREDo+BhYiIiBzeoNj8sK/oWoz428ZT0BuMeHb+SGiUCqmbREREdFliD0sXPtiZi0/S89HcYpS6KURERJctBpZLUClat7vWM7AQERFJhoHlEmQyGZRyU2jRGwSJW0NERHT5YmDpgkph+hXpDexhISIikgoDSxcsw0IMLERERNJhYOmCWmnpYeGQEBERkVQYWLrAISEiIiLpMbB0wRJYdAwsREREkmFg6YJYw8JpzURERJJhYOlC65AQa1iIiIikwsDSBdawEBERSY+BpQuWISHWsBAREUmHgaUL7GEhIiKSHgNLFyzrsLSwhoWIiEgyDCxd4LRmIiIi6TGwdIFL8xMREUmPgaULYg0L12EhIiKSDANLF9Rch4WIiEhy3Qos77zzDiIjI+Hk5ISkpCTs27fPpud9/vnnkMlk+O1vf2t1XBAEPP/88wgKCoKzszOSk5ORnZ3dnab1OiWnNRMREUnO7sDyxRdfICUlBS+88AIyMjIQHx+PuXPnory8/JLPy8vLw+9//3tMnz693WOvvvoq3nrrLaxatQp79+6Fq6sr5s6di6amJnub1+s4rZmIiEh6dgeWN954A8uXL8eyZcsQFxeHVatWwcXFBatXr+70OQaDAYsXL8aLL76IqKgoq8cEQcDKlSvxxz/+Eddffz3Gjh2LTz75BMXFxfjmm2/svqHexsBCREQkPbsCi06nw8GDB5GcnNx6AbkcycnJSE9P7/R5L730Evz9/XHPPfe0eyw3NxelpaVW1/T09ERSUlKn12xuboZWq7X66iuWdVhYw0JERCQduwJLRUUFDAYDAgICrI4HBASgtLS0w+fs3LkTH3zwAd5///0OH7c8z55rpqamwtPTU/wKCwuz5zbswmnNRERE0uvTWUK1tbW488478f7778PX17fXrrtixQrU1NSIX4WFhb127YtxSIiIiEh6SntO9vX1hUKhQFlZmdXxsrIyBAYGtjs/JycHeXl5uO6668RjRqPpg1+pVCIrK0t8XllZGYKCgqyumZCQ0GE7NBoNNBqNPU3vttZ1WDgkREREJBW7eljUajUmTJiAtLQ08ZjRaERaWhqmTJnS7vzY2FgcPXoUmZmZ4teCBQtw5ZVXIjMzE2FhYRg6dCgCAwOtrqnVarF3794Or9nf1OxhISIikpxdPSwAkJKSgqVLlyIxMRGTJk3CypUrUV9fj2XLlgEAlixZgpCQEKSmpsLJyQmjR4+2er6XlxcAWB3/3e9+h5dffhkxMTEYOnQonnvuOQQHB7dbr0UKXIeFiIhIenYHloULF+L8+fN4/vnnUVpaioSEBGzcuFEsmi0oKIBcbl9pzB/+8AfU19fjvvvuQ3V1Na644gps3LgRTk5O9jav17GGhYiISHoyQRAGfHGGVquFp6cnampq4OHh0avX/nJ/If7w9RFcFeuP1XdN7NVrExERXc7s+fzmXkJdUCk5rZmIiEhqDCxd4JAQERGR9BhYuqDibs1ERESSY2DpAqc1ExERSY+BpQuWHhZdCwMLERGRVBhYusC9hIiIiKTHwNIFJWtYiIiIJMfA0gXWsBAREUmPgaULXIeFiIhIegwsXWDRLRERkfQYWLpgGRJqMbKGhYiISCoMLF3gSrdERETSY2DpQuu0ZgGDYJ9IIiKiAYmBpQsqZeuviFObiYiIpMHA0gWVvG1g4bAQERGRFBhYumAZEgIYWIiIiKTCwNIFhVwGmTmz6BhYiIiIJMHA0gWZTNZmphBrWIiIiKTAwGIDcXl+Lh5HREQkCQYWG1jqWFqMDCxERERSYGCxQevy/BwSIiIikgIDiw242i0REZG0GFhsoFYysBAREUmJgcUGSrmphoXTmomIiKTBwGIDTmsmIiKSFgOLDSz7CXFaMxERkTQYWGygFndsZmAhIiKSAgOLDcQhISOHhIiIiKTAwGIDFVe6JSIikhQDiw24DgsREZG0GFhsoFayhoWIiEhKDCw2UMrNS/NzWjMREZEkGFhswCEhIiIiaTGw2EAcEmLRLRERkSQYWGzAHhYiIiJpMbDYwBJYWMNCREQkDQYWG1gCSwt7WIiIiCTBwGIDLs1PREQkLQYWG3BIiIiISFoMLDYQd2tmDwsREZEkGFhsoJRzSIiIiEhKDCw2ULOHhYiISFIMLDYQa1haWMNCREQkBQYWG3DhOCIiImkxsNhAxWnNREREkmJgsYFaXDiOQ0JERERSYGCxQes6LOxhISIikgIDiw24DgsREZG0GFhsoOI6LERERJJiYLFBaw8La1iIiIikwMBig9Z1WNjDQkREJAUGFhtwWjMREZG0GFhsoObCcURERJJiYLGBiuuwEBERSYqBxQaWoluuw0JERCQNBhYbsIaFiIhIWgwsNrDUsBgFwGDksBAREVF/Y2CxgVLR+mtiLwsREVH/Y2CxgWVICGAdCxERkRQYWGygkrfpYeHicURERP2OgcUGcrkMSnE/IdawEBER9TcGFhupuHgcERGRZLoVWN555x1ERkbCyckJSUlJ2LdvX6fnrlu3DomJifDy8oKrqysSEhKwZs0aq3Pq6urwyCOPIDQ0FM7OzoiLi8OqVau607Q+w6nNRERE0lHa+4QvvvgCKSkpWLVqFZKSkrBy5UrMnTsXWVlZ8Pf3b3e+t7c3nn32WcTGxkKtVuOHH37AsmXL4O/vj7lz5wIAUlJS8Ouvv2Lt2rWIjIzEL7/8goceegjBwcFYsGBBz++yF6i5YzMREZFk7O5heeONN7B8+XIsW7ZM7AlxcXHB6tWrOzx/1qxZuOGGGzBy5EhER0fj8ccfx9ixY7Fz507xnN27d2Pp0qWYNWsWIiMjcd999yE+Pv6SPTf9jUNCRERE0rErsOh0Ohw8eBDJycmtF5DLkZycjPT09C6fLwgC0tLSkJWVhRkzZojHp06diu+++w7nzp2DIAjYsmULTp8+jauvvtqe5vUppXlIiNOaiYiI+p9dQ0IVFRUwGAwICAiwOh4QEIBTp051+ryamhqEhISgubkZCoUC//rXvzBnzhzx8bfffhv33XcfQkNDoVQqIZfL8f7771uFmraam5vR3Nws/qzVau25jW4Re1g4rZmIiKjf2V3D0h3u7u7IzMxEXV0d0tLSkJKSgqioKMyaNQuAKbDs2bMH3333HSIiIrB9+3Y8/PDDCA4OturNsUhNTcWLL77YH00XqRWsYSEiIpKKXYHF19cXCoUCZWVlVsfLysoQGBjY6fPkcjmGDRsGAEhISMDJkyeRmpqKWbNmobGxEc888wzWr1+P+fPnAwDGjh2LzMxMvP766x0GlhUrViAlJUX8WavVIiwszJ5bsRtrWIiIiKRjVw2LWq3GhAkTkJaWJh4zGo1IS0vDlClTbL6O0WgUh3T0ej30ej3kcuumKBQKGI0dhwONRgMPDw+rr75mmdbczCEhIiKifmf3kFBKSgqWLl2KxMRETJo0CStXrkR9fT2WLVsGAFiyZAlCQkKQmpoKwDR8k5iYiOjoaDQ3N2PDhg1Ys2YN3n33XQCAh4cHZs6ciaeeegrOzs6IiIjAtm3b8Mknn+CNN97oxVvtGWe1AgDQqG+RuCVERESXH7sDy8KFC3H+/Hk8//zzKC0tRUJCAjZu3CgW4hYUFFj1ltTX1+Ohhx5CUVERnJ2dERsbi7Vr12LhwoXiOZ9//jlWrFiBxYsXo6qqChEREfjLX/6CBx54oBdusXd4OasBADUNeolbQkREdPmRCYIw4KtItVotPD09UVNT02fDQyvWHcVn+wrwRPJwPJ4c0yevQUREdDmx5/ObewnZyNNZBQCobtRJ3BIiIqLLDwOLjbxcTIGlppFDQkRERP2NgcVGlh4WLQMLERFRv2NgsZE4JMSiWyIion7HwGIjL2cOCREREUmFgcVGHgwsREREkmFgsZGl6LaagYWIiKjfMbDYyFLDomsxoklvkLg1RERElxcGFhu5aZRQyE37CXFYiIiIqH8xsNhIJpNxphAREZFEGFjs4MnCWyIiIkkwsNiBM4WIiIikwcBiBy9xSIj7CREREfUnBhY7cEiIiIhIGgwsduB+QkRERNJgYLEDF48jIiKSBgOLHTgkREREJA0GFjtwlhAREZE0GFjs4MWF44iIiCTBwGIHFt0SERFJg4HFDl4uagAsuiUiIupvDCx2aFt0KwiCxK0hIiK6fDCw2MESWAxGAfU6g8StISIiunwwsNjBSSWHWmn6lXF5fiIiov7DwGIHmUzGtViIiIgkwMBiJwYWIiKi/sfAYifLWiw1XIuFiIio3zCw2Ik9LERERP2PgcVODCxERET9j4HFTp7csZmIiKjfMbDYycfVtNptTnmdxC0hIiK6fDCw2Gn2yAAAwK+nylFR1yxxa4iIiC4PDCx2GhnkgfgwL7QYBazLKJK6OURERJcFBpZuuG1iGADg8/2F3FOIiIioHzCwdMN18cFwUStw9nw99uddkLo5REREgx4DSze4aZS4bmwwAODzfQUSt4aIiGjwY2DpplsnhgIANp0oQ4vBKHFriIiIBjcGlm5KCBsCT2cVaptbcLioRurmEBERDWoMLN2kkMswbZgPAGDXmQqJW0NERDS4MbD0wLRhvgCAndkMLERERH2JgaUHpg/zAwBkFFxAXXOLxK0hIiIavBhYeiDcxwVh3s5oMQrYl1spdXOIiIgGLQaWHrrC3Muyg8NCREREfYaBpYemx7COhYiIqK8xsPTQ1GgfyGRAdnkdii40SN0cIiKiQYmBpYe8XNSYPNQ0vXldxjmJW0NERDQ4MbD0gpsnmFa9/d/BIhiN3AyRiIiotzGw9IJ5YwLhplGioKoB+/KqpG4OERHRoMPA0gtc1ErMHxMEAPjqQJHErSEiIhp8GFh6yS2JpmGhDUdLuIgcERFRL2Ng6SUTIoYgytcVjXoDvjnE4lsiIqLexMDSS2QyGe6YHAEAeH/HWRhYfEtERNRrGFh60W2TwuDlokJ+ZQN+OlYidXOIiIgGDQaWXuSiVuKuqZEAgHe35kAQ2MtCRETUGxhYetnSKZFwVilwvFjL/YWIiIh6CQNLLxviqsZtk8IAAF8cKJS4NURERIMDA0sfuCrWHwBwolgrcUuIiIgGBwaWPjAyyAMAkFdZj3quyUJERNRjDCx9wNdNA393DQQBOFVaK3VziIiIBjwGlj4SF2zqZTlRwmEhIiKinmJg6SOWYaGTDCxEREQ9xsDSR+LMgYWFt0RERD3HwNJHLD0sp0q1XKafiIioh7oVWN555x1ERkbCyckJSUlJ2LdvX6fnrlu3DomJifDy8oKrqysSEhKwZs2aduedPHkSCxYsgKenJ1xdXTFx4kQUFBR0p3kOYaivK5xUcjTpjcirrJe6OURERAOa3YHliy++QEpKCl544QVkZGQgPj4ec+fORXl5eYfne3t749lnn0V6ejqOHDmCZcuWYdmyZfj555/Fc3JycnDFFVcgNjYWW7duxZEjR/Dcc8/Bycmp+3cmMYVchthADgsRERH1Bplg54Y3SUlJmDhxIv75z38CAIxGI8LCwvDoo4/i//7v/2y6xvjx4zF//nz8+c9/BgDcdtttUKlUHfa82EKr1cLT0xM1NTXw8PDo1jX6wop1R/HZvgI8OCsaT18TK3VziIiIHIo9n9929bDodDocPHgQycnJrReQy5GcnIz09PQuny8IAtLS0pCVlYUZM2YAMAWeH3/8EcOHD8fcuXPh7++PpKQkfPPNN51ep7m5GVqt1urLEVmmNnOmEBERUc/YFVgqKipgMBgQEBBgdTwgIAClpaWdPq+mpgZubm5Qq9WYP38+3n77bcyZMwcAUF5ejrq6Orzyyiu45ppr8Msvv+CGG27AjTfeiG3btnV4vdTUVHh6eopfYWFh9txGv4kLcgfAISEiIqKeUvbHi7i7uyMzMxN1dXVIS0tDSkoKoqKiMGvWLBiNRgDA9ddfjyeeeAIAkJCQgN27d2PVqlWYOXNmu+utWLECKSkp4s9ardYhQ8vwAFNgKa9tRk2DHp4uKolbRERENDDZFVh8fX2hUChQVlZmdbysrAyBgYGdPk8ul2PYsGEATGHk5MmTSE1NxaxZs+Dr6wulUom4uDir54wcORI7d+7s8HoajQYajcaepkvC3UmFIE8nlNQ04cz5WkyI8Ja6SURERAOSXUNCarUaEyZMQFpamnjMaDQiLS0NU6ZMsfk6RqMRzc3N4jUnTpyIrKwsq3NOnz6NiIgIe5rnkIb5uwEAssvqJG4JERHRwGX3kFBKSgqWLl2KxMRETJo0CStXrkR9fT2WLVsGAFiyZAlCQkKQmpoKwFRvkpiYiOjoaDQ3N2PDhg1Ys2YN3n33XfGaTz31FBYuXIgZM2bgyiuvxMaNG/H9999j69atvXOXEorxd8eO7AqcZmAhIiLqNrsDy8KFC3H+/Hk8//zzKC0tRUJCAjZu3CgW4hYUFEAub+24qa+vx0MPPYSioiI4OzsjNjYWa9euxcKFC8VzbrjhBqxatQqpqal47LHHMGLECHz99de44ooreuEWpTU8wNzDUs5dm4mIiLrL7nVYHJGjrsMCAAfzq3DTu+kI8nRC+orZUjeHiIjIYfTZOixkv2F+pplCJTVNqG3SS9waIiKigYmBpY95uqjg726a0XSmnHUsRERE3cHA0g8s67FkM7AQERF1CwNLP2id2szCWyIiou5gYOkHMeJMIVMPS4vBKGVziIiIBhwGln4Q49+6p9D9aw4g7oWfsetMhcStIiIiGjgYWPpBjHlIqLy2GT8fL4OuxYjt2eclbhUREdHAwcDSD4a4qhHs6QQAcFUrAAAFlQ1SNomIiGhA6Zfdmgl4Z/F4nCmvg4ezCvevOYh8BhYiIiKbMbD0k3HhQzAufIi4Fkt+ZT0EQYBMJpO4ZURERI6PQ0L9LMzbGTIZUK8zoLJeJ3VziIiIBgQGln6mUSoQ7OkMwNTLQkRERF1jYJFAuLcLALCOhYiIyEYMLBKI8GFgISIisgcDiwQifFwBcEiIiIjIVgwsEhB7WKrYw0JERGQLBhYJWGpYuHgcERGRbRhYJGDpYams16G2SS9xa4iIiBwfA4sE3J1U8HFVA2DhLRERkS0YWCQSbu5lKWAdCxERUZcYWCQSaZ4plMeZQkRERF3iXkISsRTe/u9AEbJKazEt2he3TgyTuFVERESOiYFFIiODPAAAZyvqcbaiHt9mFiM2yB1jQ72kbRgREZED4pCQRObEBeCtRePw/G/iMD3GFwDw0vcnIAiCxC0jIiJyPAwsElHIZVgQH4y7rxiK126Oh7NKgQP5F/DDkRKpm0ZERORwGFgcQKCnEx6cFQ0A+OuGk/j6YBHKtU0St4qIiMhxMLA4iOXToxDi5YySmiY8+dVhTHnlV3x3uFjqZhERETkEBhYH4axW4MsHpuChWdGIDXSHwSjgT98dh5Yr4RIRETGwOJIQL2f84ZpYfP/oFYj2c0VVvQ5vp2VL3SwiIiLJMbA4IJVCjj/+Jg4A8NHuPORWtC4ut/dsJV756RR0LUapmkdERNTvGFgc1JUj/DFrhB/0BgGpG04CAJpbDHjks0NYtS1HrG+pa27B3R/tx+qduVI2l4iIqE8xsDiwP84fCbkM+OVEGY6dq8H3h0twvrYZALDnbCUAYOOxUvx6qhyrtuVI2VQiIqI+xcDiwIb5u2NBfDAAYOXmbPxnx1nxsb25psCyI/s8AKC8thlNekP/N5KIiKgfMLA4uEeuioFcBmw+WYZTpbVwUSugkMtQWNWIogsN2HWmQjy36EKjhC0lIiLqOwwsDm6Yv5vYywIAtyaGYXSIJwDg4915qKjTiY8VXmjo9/YRERH1BwaWAeDR2aZeFrkMWDYtEpOjvAEAn6TnW51XVMXAQkREgxN3ax4Aov3csPbeJEAAInxcMXmoD97bdhbN5qnNbhol6ppbUMghISIiGqTYwzJATI32xdRhpl2dEyOHQC5rfez6BNOQUSF7WIiIaJBiYBmA3J1UYh1LgIcGV47wBwAUMLAQEdEgxcAyQE2NNvW2zBzuhzBvFwDsYSEiosGLNSwD1MNXRsPLRYVbJoTCSaUAAGibWlDTqIens0ri1hEREfUu9rAMUO5OKjwwMxo+bhq4apTwcVUDYC8LERENTgwsg0SoeVio6BJrsRiNAtZlFOE/O85CEIT+ahoREVGPcUhokAgb4ozDhdUorOp4avOJYi2e/eYoDhVUAwAmDfXG2FAv8fHKumY8u/4YInxcsOLakf3QYiIiItuxh2WQEAtvLzTgQr0Om0+Uib0ohVUNuOnd3WJYAYCM/Avi94VVDbh5VTo2Hi/Fe9vPihssEhEROQoGlkEibIgpsJwuq8Vt/96Dez85gM/2FQIAvjpQiEa9AWNCPLE4KRwAcLioBgBQXN2IG9/djdyKevFalp2giYiIHAUDyyARbu5h2XO2CllltQCA93echd5gxFcHiwAA98+Mwpy4AABAZmE1AODz/YU4X9uMmDZ7Fu3OYWAhIiLHwhqWQSLM21n8XiYDnFUK5FbU48Xvj6OkpgleLirMiQtAo84AAMitqEd1gw5bs8oBAMtnRMHHVY3vDhcjPaeiw9cgIiKSCntYBolgL2cozev1P3ZVDJZMiQQArN1TAAD4bUIINEoFvFzUGOrrCgDYfLIcR8xDQ7NG+GHSUG8o5DLkVTbgXDX3JSIiIsfBwDJIqBRyvHj9KDx8ZTQemx2DpVMjxAADALcmhonfx4ealvX/56/ZAIAxIZ7wd3eCu5MKY8xL/qd3Miy0M7uCNS5ERNTvGFgGkcVJEXhqbiwUchmCPJ1xnbkmZUyIJ+KCPcTzEsK8AAB5laY1W64c4Sc+NjXaBwCw+6JhIYNRwF83nMQdH+zF0tX70KBr6ctbISIissLAMog9NXcErosPxp8WxFkdTwgfYvXzlbH+4veWPYrScyrFadF6gxH3fXIA/95+FgDQ3GJEdlldXzadiIjICgPLIBbs5Yy3F43DhAhvq+Mjg9yhVpjeem9XtdUCchMihkClkKGkpknsgdlwtARpp8qhUcoR5OkEAMgqre2fmyAiIgIDy2VJo1RgpHmIaNZwPyja1Lo4qxUYZ+6BsdSx7DlbBQC4c3IErhkdCADi1GkiIqL+wMBymbplQihc1Qrcbl5Irq2L61j25ZqCS1KUD2ID3QGYFqgjIiLqLwwsl6k7Jkfg+EvXIDHSu91jbetYztc2I+d8PWQyYGLkEAwPMAUWDgkREVF/YmChdhLCvOCkkqOyXodP9+YDAEYEuMPLRY0Yc2Apr23GhXod6ptbsCY9DzWN+k6v16Q34N/bc1BSw7VdiIioexhYqB21Uo6J5p6XD3bmAgCShpp+dtMoxVV1s8pq8drPWXju2+P48w8nOr3ee9vO4q8bTiF1w6k+bjkREQ1WDCzUIcuwUG2Tab2VSUN9xMdGmHtZjp2rwbeZ5wAAPx4pQW1Tx70sW8zL/+/Pq+qz9hIR0eDGwEIdshTeWkwc2rp2i6WO5cNdebjQYAopjXoDfjxS0u46F+p1OFxUDQAoqWlCcTeW/DcaBeRX1ovrwhAR0eWHgYU6NCrYA+5Opr0xo3xd4e/uJD42wjxTyLLfkK+bGgDw5YFCCIKAtXvy8cHOXAiCgB1nKtA2ZxzMv2B3W97dloOZr23Ft5nF3b0dIiIa4BhYqENKhRxJ5mGgSUOtZxJZAovFmwsToJDLkFFQjfvWHMQfvzmGP/9wAr+cKMO2rPOm65nXerEElmfXH8XC99JR13zpJf6NRgGf7jEV/qadKu/5jRER0YDUrcDyzjvvIDIyEk5OTkhKSsK+ffs6PXfdunVITEyEl5cXXF1dkZCQgDVr1nR6/gMPPACZTIaVK1d2p2nUix6+MhpTo31w7/ShVsejfN3EADIq2APTY/zE/Yg2nSgTz/vbxlPYnm0KLDdPCAUAHCq4gBPFWny6twB7c6vwxf5Cq2vrWox4+NMM/Om74xAEAYcKL6C4pgkAcNQ8tNQRg1HA/rwq6A3Gnt00ERE5JLsDyxdffIGUlBS88MILyMjIQHx8PObOnYvy8o7/9+vt7Y1nn30W6enpOHLkCJYtW4Zly5bh559/bnfu+vXrsWfPHgQHB9t/J9TrxoUPwX+XT8Ywf+seFbVSjig/VwDAjeNNQcSyG7RaIcdrN4/FEBcVzp6vx/naZjirFLh/ZjQA4HixFu/vOCte68NduTAYW8eMfj5eih+PluCj3XnYdaYS3x9urYvJq2zodPr0h7tyccuqdHFWExERDS52B5Y33ngDy5cvx7JlyxAXF4dVq1bBxcUFq1ev7vD8WbNm4YYbbsDIkSMRHR2Nxx9/HGPHjsXOnTutzjt37hweffRRfPrpp1CpVN27G+o3z1w7EkumRGDRJFNQmRMXgFdvGov/PTgFtySG4dGrYsRzp0b7INLHBYEeTmgxClh/yDSzSK2Uo+hCI345Xiqe+9+9BeL3r/18ChuOmgKLzLx7wPFzNR22Z9tpU0/OAc5EIiIalOwKLDqdDgcPHkRycnLrBeRyJCcnIz09vcvnC4KAtLQ0ZGVlYcaMGeJxo9GIO++8E0899RRGjRrV5XWam5uh1Wqtvqh/zRrhj5euHw0XtakwVyaT4daJYeJGiosnhyN0iLP5XD/IZDJMiGidaRQb6I77pkcBAP5j7hXJOV+H9LOVkMsAZ5UCh4tqUF7bDHcnJWbHBgAAjnQQWARBwFHz8TPl3EWaiGgwsiuwVFRUwGAwICAgwOp4QEAASktLO3kWUFNTAzc3N6jVasyfPx9vv/025syZIz7+t7/9DUqlEo899phN7UhNTYWnp6f4FRYWZs9tUD/QKBX4YOlE/P7q4Vg40bRf0fg2geXOKRFYMiUCKoUMB/MvYMPREnxm7l25coS/Vd3M3FGBYtg52kFgKahqQLV5enVBVQOa9IY+uy8iIpKGsj9exN3dHZmZmairq0NaWhpSUlIQFRWFWbNm4eDBg/jHP/6BjIwMyGSyri8GYMWKFUhJSRF/1mq1DC0OaESgu9WMIstque5OSvw2IQSuGiVunhCKz/YV4qFPM6BWmPLz7UnhSIz0xifp+ahp1OO6+GCxyPdoUfvAkllYLX5vFID8yoZ2M5mIiGhgsyuw+Pr6QqFQoKyszOp4WVkZAgMDO32eXC7HsGHDAAAJCQk4efIkUlNTMWvWLOzYsQPl5eUID2/dNdhgMODJJ5/EypUrkZeX1+56Go0GGo3GnqaTAxgd4om3Fo1D2BBnuGpMf/ReuG4UnFVKrN6VC53BiGBPJ8wa4Q+FXIZP7p6E7PI6zIjxhbbRNP3Z1Juig5eLWrzukYtCzJnyOrsDy6YTZXhvWw5evyUekb6uPbxTIiLqbXYNCanVakyYMAFpaWniMaPRiLS0NEyZMsXm6xiNRjQ3NwMA7rzzThw5cgSZmZniV3BwMJ566qkOZxLRwLYgPhjjwluHhpxUCjx/XRzW3DMJ02N88dL1o6Ew96bEh3nh5gmhkMlk8HRRIcLHBQBw7Jx1zdIR83RnV7UCQPfqWD5Jz8OB/Av4d5sZTJdSUddsNbupt3y6Nx9Jf92MU6WsyyIiasvuWUIpKSl4//338fHHH+PkyZN48MEHUV9fj2XLlgEAlixZghUrVojnp6amYtOmTTh79ixOnjyJv//971izZg3uuOMOAICPjw9Gjx5t9aVSqRAYGIgRI0b00m2So5se44c19yQhOS6g03PGhHgCAI6cqxaPtRiMYoCZNyYIgKl4t62307Ix+a9pOHu+8yCTV1kPAPjpaEmXa7kcKarGxL9sxjPrjorH3tlyBvPf2oEybdMln9uVz/cVokzbjK8OFPXoOkREg43dgWXhwoV4/fXX8fzzzyMhIQGZmZnYuHGjWIhbUFCAkpLWtTPq6+vx0EMPYdSoUZg2bRq+/vprrF27Fvfee2/v3QVdFiyBZdOJMjToTENE2eV1aNQb4KZRYo457LTtYckuq8XKtGyUapvE6dQX07UYce6CaZuBCw167M6pvGQ7MvIvQBCAtFNlEAQBgiDgPzvO4nixFh/uyuv2/elajMgqrQUApHfRBiKiy023im4feeQRPPLIIx0+tnXrVqufX375Zbz88st2Xb+juhWi2SMD8PdfTuNQQTVufS8d7y9JFIeDRod4IMbfDQBwtqIORqMAuVyGl388KQ7d7MiuwJNXt++1O1fdiLajO98fLsbM4X6dtqPIHG4q6nQoutCIFqMgbgL55YFCPDEnBhqlAi0GI5QK2/9PkF1eC525d+dkqbZdrQ4R0eWMewnRgDHM3w2fLk+Ct6sax85pMfPVrXj9l9MAgPhQL4R7u0CtkKNJb8S56kZszSrHttPnxRlGR4qqO1wp1zIcpFaa/jr8fKwUzS2dT422BBbANEMpo82GjlX1Omw8Vop9uVUY/+dNWNFm2Kgrx9vU5ggCsDeXi+AREVkwsNCAMjHSG98+PA1jQjyhMxhxvtZUvD0u3AtKhRyRvqbC3OPFNfjzDycAAHdNjUSUnyuMQsdDLfkVpsAyc7gfAj2cUNvcIm7a2BHLLtWAObAUmAKLu3nm07+25OC+NQegbWrBt5nn0GLuNdmfV4UXvz/e6Toxx4pNs50ss/ttGRY6kFfV6XYFRESDCQMLDThh3i747pFp2JwyA89cG4sn5wzHnDjTtPph5mGhZ9YfQ875evi6qfHoVTGYPswXALDzTPsgkl/VAACI8nXF/LGmwt2vDnZe9Fp0oUH8/lDBBRwqqAYApFw9HAq5DFllteJCdg06A44Xm3pO/rj+GD7clYcfj5S0uybQuije1eZanD1nLx1Yfj1VhptXpePJLw9f8jwiosGAgYUGJJlMhmH+7rhvRjQenR0jToWO9jMFlqp6HRRyGd5eNB6eLipMMweWXWc66GGpNAWQcB8X3DYxDDKZqbC3o0Xq6ptbxHoVADhWrBWnIM8bHSSGjRAvZ3F13v15VSipaURWmamg9uJZTIBpttPJEtN17rnCtGXBqdJaVNY1d/o7sOyMvSWrHBfqdZ2eR0Q0GDCw0KBi6WEBgD/MHYEp0T4AgMnRPlDIZcitqEdGwQX8Z8dZ7DdvlGipYYn0cUVMgDuujzftFv73TVntrm8ZDvJwUsLLRQVdixFGAQjydEKgpxOeuXYklk6JwJp7JomzlvblVmH76daeHcvrtXW2oh5NeiNc1QokRgzB8ADTfVyqjsUSvgxGQQwvPXWo4AK2ZHW88zoRkZQYWGhQmTbMF+HeLlg0KQz3zYgSj3s4qRAfapoWfeO/duPlH0/ivk8OoElvQKF5SMiyMN3vkk1DO1uzzrfb/dkyHBTm7YKEMC/x+HjzYnhh3i548frRiPJzw8RI07ED+RfE3aQB4Oz59oHlmHk4KC7YA3K5DFOiTEHr7V/PYGtWOQTBepG6wqoGFFS1Dk1tONbxMJM96ppbcMd/9uKej/ajuE2dDhGRI2BgoUHF102D7X+4Eqk3jm23N9WsEf7i93KZac2V9YfOQW8QoFLIEORp2l060tcVtyaGAgD+bp6FZGGZIRTi5WwVWMaFe+FiY0K8oFHKUVWvs+oByaush/GiVXIti9+NNq81c0tiGDRKOU6WaHHXh/tx27/3oKrNsM/unAoAQLCnEwBg15kK1DT0rPh247FS1OsMMAoQh6+IiBwFAwtdNu6bEYUXrovD1w9OxaJJpr2r3tuWA8DUM2KpgwGAR6+KgVIuQ/rZSrG2BGgNLKFDXC4KLK3bDViolXLxHL1BgIeTEkq5DE16I8pqm2A0Cnjlp1N48svD+OWEabfz0cGmwDI6xBNbfj8L91wxFM4qBfbmVuGGf+0SV+u1DAfdPCEUIwLcoTcI2Hyy62Ghi3tq2vqmzcJ6uR30AknhfG0z1h8q6pNtEIhoYGFgocuGk0qBZdOGYkLEEMwbbZoNlGcuuI30sd7wMNjLGVePMtWg/HdvgXj8nBhYnDEubAjcNEoMcVFhVLBHh685ybxDNWDafiDM2zTslHu+HjvPVGDVthx8nVEkBqGx5mErSxue+00cvntkGkKHOCO/sgE3/Gs3jp2rEVfjnTrMF/PGmGZIrTtUhEZdx1OmjUYBC99Lx2/e3tnhOaU1Tdhl7rUBgNyK3g0sgiBg++nzKK2xfesCQRBw7ycH8MQXh/Hj0Z4PeRHRwMbAQpelpChvDHFRiT9b6lfaWpwUAQBYf+gc6ptNWwFYalhChjjD00WFrx+civ89OBVOKkWHrzMxsjWwzBzuh6HmnaBzK+uxz1xQGx/mhXuuGIo/Xz8KMQHtd5mOCXDH+oemIT7MCzWNetz6Xjoq6prhpJJjXLgXrjXvobTrTCUmvLwJj39+CLtzKqx6U4ouNGJvbhWOF2vxcXpeu9f47vA5CIJpqAwwrRbcmf/sOIuULzPtWv/l64xzWLJ6H+75eP8le3na2nb6PA4XVgNorfEhossXAwtdllQKuTiLBwAivNsHlqnRPojydUVdcwu+zSwG0HZIyFTvMiLQXZxK3ZHxEUOgUcqhkMswo21gOd8aWG6fFIbnfhOHO6dEdnodP3cN1twzCfGhnmgw95BMjPSGRqnA8AB3/PWGMQjzdkaDzoBvM4tx+/t7cdXft4kf9Cfb7P68alsOtE3WYWP9IdP9/XZciNi+jnx9sAgv/3gS6zLO4d6P93e6CF5bDboWvLrxFADgeLEW6V2sLwOYelfe/vWM+LOtO3DnV9Z3OHwkCEKPa3yISFoMLHTZsuzuDAARvq7tHpfJZLg9yVTr8unefDTqDKg0F76GDmkfcDriplHik7sn4aNlExHo6YRI8+ucKq1Fprn3YNJQH5uu5eGkwid3J4nDT7NjW4uIb08Kx/anrsT6h6ZicVI43DRK5FbUY016vun1SlqLaKsb9PhgR67484liLU6WaKFSyPDoVTEAgOKaJnGDSYus0lo8+41pqwG5DNifdwGP/DejyyDw3razKK9tXU/Glg0i03MqcbDNlge2BJaPd+dh5mtbsfg/e8TVhS3+u68A8S/9gm8zO94AszNV9Tq7hrEGssKqBpTUcHYYOS4GFrpsTYv2hY+rGkq5DCM6GIoBgJvGh0KtlON4sRZr9uQBANydlPB0VnV4fkeSonwwPca0mWKUObDszqmAzmCEn7sGkR0MR3XG00WFL++fgg+XTcTiyRFWj8lkMowLH4K/3DAGby5MAAAcKjR96FsKhxPNi9n9Z8dZcdbRp3tNoWZOXACG+rrCyzxUllfROm26SW/Ag58eRJPeiOkxvvjv8snQKOXYfLIc41/ehFtXpYs9Rm2V1jThve2mwubfXz0cALD5ZBnyO1iLpi1L78p8c6gsvNBwyd6cQwUX8PKPpq0Y9pytwt83Wc/u2myepfXpnoJ2z92SVY4V6462q+0xGgX89p1dmP33rZes6dmfV2VzEDpdVouULzKttndwBPXNLZj/1g5c9/audmGPyFEwsNBlS62U4/P7JuO/yycj2Mu5w3OGuKqxdIopGPxto2khuZBOzrWFZUjIMmoxaah3u+nXXXHVKHHlCH+oLrETtGWadXZ5HbRNenE13seTYzAq2AP1OgP+sfk0apv04uygO8wBSBy2avMh/dOxEpw9Xw8/dw1WLkzA5CgfvL8kETH+bjAYBezLq8JLPxxv145/pJ1Gk96IxIghePjKYZg53A+CAHy8O7/TtjfqDNiTaxo2+r95sfB0VkEQOl6/BgCqG3R45L+HoDcIiAsy9T69uzUHv55qnTV1qtTUw7Q/v0rcf8ri9Z+z8Nm+Avx8vNTqeG5lPQqqGlCvM+DF7493WHujazHino/24/HPM23qBfpHWjbWHTqHf2w+3eW5tiqobEB1Q89WOj5RooW2qQUVdc0o1V4ePUo08DCw0GUtJsDdaiZPR1LmjECkj4tYG2HrcFBHAj2coFG2/rVL6uK1u8vXTYNwbxcIArD7TKW4X1JckAeevXYkAGDt3gK8/nMW6nUGRPu5iovVRfmaanLOttlCwNIzsWRyBHzcNACAGcP9sCllJjY9MQMymWktmbI2H3bnqhvxP/OeTE/Pi4VMJsOyaZEAgM/2FXS6uWNuRT0EAfB0ViF0iDNizKsXn+lgSwMA+PMPJ3GuuhGRPi744v7JuGuq6TWe/voojEYB1Q06lJiHdQQBVtO/BUFAgXmm2NGLCnstBb8AsDXrfIerCR/MvwBtk2no7PRFa9cIgoA3N50We7AEQcB+cy/ULyfKoO+Fnoz8ynrMeXMbln20v0fXOd7m3tsuSEjkSBhYiLrgrFbgbzeNFX+2FNx2h1wuE3swAHQZlnpivLmX5fP9BRAEU+Guj5sGU4f5Yk5cAAxGAR+ba1zumBwh9vRE+Vn3sJwuq8WB/AtQyGVYODGs3evEBLgjPtT0WltOtS7r/+9tOdAbBEyO8hZnS82I8cO0YT5o1BuwdPW+DjeCtMxQivJzNe8ZZQ4sHfRg5FbUY/0hUyh6Y2EC3J1UWHFtLFzVCpyvbcbp8lqcLLEOEm17Umoa9ag1zwC7eCbSEfNeUu5Opl24X/z+RLtho62nW+/34mGjA/kX8I+0bDz3zTFU1etQWNUo1vJUN+ix92zn2y5YnCzRXnKfqM0ny9HcYsShguoe7dp9rLi1KLuQgYUcFAMLkQ2Sonxwv3mp/8lRPQsZlsDi6azCcP+Oa2d6g2UxO8u2ACODWteKeebakVApTAHFWaXAjeNDxccsdTY55g9gyzo0c0YGwN/DqcPXuspcAPyrObCUa5vw2f5CAMBj5kJewBTYPlg6EXNHBUBnMOKRzzKw8Zj1UIxl6MfS09MaWNqvvvv2r9kwCqYCZMv2CBqlAuPFjScviMNh0eYgtutMhThLqrCqtZbkRLHWagXiw0XV4u8q2NMJ56obseGi9WC2ZXW+5cLP5vsyCsC20+Xi3lUWXW2ncDC/Cte+tQN3fdT5VPAd2a2vf7y4+1O/j1sFFseqryGyYGAhstGKa0fi4B+Tcc3ooK5PvgRLYEmMGAK53L76FXtYPsAtn3UjA1vD0VBfV9w9bSgA4MbxIVZFxEMtPSzn69CoM+DrDFMPhmXGVEcsgWXnmQo0txiwattZ6FqMGB/uJW5AaeGkUuBfiydgYWIYBAF48stMZLcZTrH0VFh6eqI76WHJragX628eT46xeiwxwhQqD+RViTOk5o8JQrSfK/QGQewJKrzQ2ptQ29wiDofoDUacMH+IT47yEde6aTtsVFrTJNbGmNrT2j5BELCxTU/Or6fO40C+KbBYZnn9crz0kiv4rknPhyCYhqYyCi60e7y5xYA9baaId7VWjdEo4MkvD+OaldutdgFvbjFY/f45JESOioGFyA6W+o2euGNyBK4ZFYgn5gzvhRZ1LjbIHU4qudXPbf3hmlh8fPckPPebOKvjkT6ukMkAbVMLfvfFIdQ2tSDc2wVXDPPt9LVGBXsgwEODBp0Br/x0Ch/uNk2bfnR2TIdFxQq5DC/fMBqTo7xRrzPgvjUHxSENS+2MpUdkmHmdm9yKeqsZLG17V8aah6QsEi0bT7bpYYkN8sDcUaZVgX85bqpHuXj4wxJIskpr0dxihLuTEpE+LuIeT21DgWUHbkvYazskdLxYK67ZAwDbssqxxzwE9MiVw+DprEJFnQ6/nipH2skyvPzDCcx/aweuen0rzp6vQ3WDDhva9Dyt7WB208G8C2jSt/4+jp7Ttjunrb9vysLXGUU4VVprdb3TpXVoaROc2oY4IkfCwELUz4K9nLHqzgnih2BfUSnkGBviJf7cdkgIMIWGmcP92q3S66RSINi8EeTPx8sglwG/nzvikr1BMplM7GX5cFceBAFYNCkcs4b7XbJ979w+HiFezsitqMc7W85AEITWISFzUAnxcoazSgG9QRD/928wCvjpqOkD/aErh7W7dkKYFxRyGc5VN4r1GbGB7uIGmJYei4s/nI+Zh1Us9StjQz0hk8kwOsT0uzterBV7RSz1K5aNMi806MV6k1/MvSvJIwPg7aqGtqlFDDRJUT642rxo4fJPDuCejw/gPztzcbxYi7MV9Xjqf0fwdcY56FqM8HVTAwB+PFJitfklAOw4Y9pKIci8AaYlTJXWNOHj3XnQtbSGme8PF+OdLTniz2v35ouPW+7Zx9X0Wj2tYTleXINdZyq6PpHITgwsRIOYZXqzSiETa0JsYRmG8XIxLVa3ID64y+dc2WY37KSh3nhxwagup2z7uGnwjHnW0q+nynG+rhm1zS2Qy1q3S5DLZeLwkGVY6Oz5OjTqDXBRK6w2obRw1SjFoReDUYCTSo4IH1fEBXtAJgNKappQVa9Dgblew/J7Om7upThirl+x9NwM9XWDs0qBRr0BuRV1aDEYsSPb9KE8b0wQAs21Pbnm9WUsw0HXjgnErBGtoW2Yvxu8XdW4aUJrzdBQX1csmhSOv900Bm4aJQ7mX8DfzCsDPzY7BmNCPKEzGPHVgUKre7TUryyfbqqtyq2oh7ZJjye/ysQL3x3H6l2mXq7Kumb84X9HAAB3TxuKAA8Nztc24ydzDY2l9sWyd1ZFnQ4NuhaknSzDmD/9jJ862MeprrnFalNQC12LEXf8Zy+WrN7ncGvN0MDHwEI0iFlmIY0M8oBaaftf95Q5w3Hn5Ah8/8gVuCKm86Ggtq6I8UWIlzOG+bvh3Tsm2Px6VwzzhVxmCiOWqc6hQ1ygUbb2/Ay7aGqzpUh0ZJCH1S7bbVnqWABgRIA7FHIZ3DRKcaPL48U1KDL3JswbbRoqOlZcA0EQcNjcwxJv3oxSIZchzhyAjp3TYl9eFWqbWuDlokJ8qJfVlgtnz9fhdFkdlHIZZscGYHZs6xYQE81DVZOjfLA5ZQbSV1yFLb+fhdQbx2DhxHAxvOlajNAo5bg+PgR3TDbVDn2Sno88cy9NZV0zjpnD1XXxweLaQD8eKRF38rYUCG84WoJGvQEjgzzw7PyRuMO8R9Zq84rDlutMifaFh3lGVGFVIz7bV4Daphb8fdPpdkW/T311GPP+sQMbLyoc3ptbiQsNehiMglgDRNRbGFiIBrGrYv3xt5vG4LWb4+16XkKYF/7829Hi7tK2cFErse2pWfjp8enwNg8v2MLTRYUx5p6Mj3fnAWgtuLWwrMViKaC1DH+M7mSXbKA1HABAbGDreZbgcfRcjVhnMntkAFQKGaob9Mg5XyeuqdK2NmaMeQjv6Lka/GCejj03LhAKuay1ULmiXgwKU6J94OmiwvThvlCaQ1XbEDXM3x1BntZT5BdNCsO0YaYi5WvHBMHTRYUF8SHwddPgXHUj5ry5Db/7/BCWrN4HwBTY/Nw14pDVaz9nidc6UlSDogsN+N7c1hvHhUAhl2FRUjjUCjkOF1ZjXUaRWOMzKtgD4eZerdyKeuw1rxlzprzOav+nC/U6cU2at9LOWIWZtJOt07yzO5jVRdQTDCxEg5hMJsPCieEYEdh306fbUirkl1yBtzPTzQW9GQXVANBu+MoyTXl3TiUEQRDrLkZdog5oQpvAMrJNwbFlqGjrqfPQGYxQyGWI8HbBcPP2DIv/sxcGowBfN41YH9L2eZmF1eJU7N/EB5nbawosZyvqsN48c8kyjObhpMIdkyMwzN9NrPPpjEwmw9uLxuP/5sXij/NNvS3OagW+emAKZgz3g94g4JvMYrGH6abxps0qRwebfg+WOhfL2jEf784Tp1PPH2tqq6+bBjeYN7lM+fIwmvRGuKoVGOrjinBzQP3lRClqm1r3krLsSQWY1rGxFOmeKNGKQ2OCIFgtymcZviu60IDb399jtUYPUXcopW4AEdH0GF/8c0vr7swX97BMiBgCZ5UCFXXNOFGiFT+wLR/UHfF3d8LwADecLqtDQnhreBllfs5+8zTjYC8nKBVyjAnxxPFiLcq0zXBRK/Dcb0Za1eBYiqQtmzL6uKrF1YEtQ0LbT1egrrkFGqUc15iHmQDgTwtG2fy78HZV44GZ0VbHhvq64uNlE7Elqxx7z1ZheIA7JkQMETfTHB3a+nsI9HDCvdOH4uUfT+KDnbkQBNMU+rbbT7ywIA5erip8sjsfjXoD4sO8IJfLEGZexdnSSxTl54qz5+vxy4kylNY0IdDTCT+aH/N2VaOqXodV23IwY7gfssvrrGZG5ZgDy/8OFmF3TqVpS4kuAhvRpTCwEJHkxoUPgYtagQbzSrIXBxaNUoHJUd7YknUe/91rqq1QK+SICbh0IfG/Fk9AbkW9VWGupafEMpJh+ZC+Y3IEjp6rwdRoH9w/Mxq+F01hH+bvBrVSLs6umTcmEEpzb5JlRlOdedXcOXEBcHeyfYNMW5hmYgXgqjY1MRZtg9utE8Pwm7HBePnHk+KeVb8Za712kItaiRXzRmL59Cj8crxMHIayDAFapksvmhiOTSfKsC+vCmv35GPZtEjsNtcZ/fP2cbjzg33YnVOJzMJq7M4x9bRE+rggr7IBZ8rrIAgCDpl7zeqarHf/JrIXh4SISHJqpRyTo1oXmIv2ax9EZpinSH9l3p9oRKB7l8NPw/zdMCfO+gPe102DAI/WMGIJLKNDPPHjY9Px7Py4dmEFME3Dbjs1/LqxrTOnQoc4i3UqAMQhl/7i565BfJgXPJyUuG1iGAI9ncSZT3IZcO3Yjhc79HXT4PakcESYC5EvrlmaHOWDJVNNRbr/2noGKV8ehsEoYEyIJ6ZG+4rDXktX7xP3m7praiSUchnqdQYU1zQh07wnU21z97cOIAIYWIjIQVgWpnNVK+Dv3j4wTI8xBRZLD4el0LQ7RrXpkQjztn1vKEuRb4CHRtwfCTCFGUv9xxAXlRiu+tNny5Ow7akrxaGf+ebVeadE+8DfveMtFS4W3iawuDspERfsgWtHB+G2iWHmLQZMU6kt9TBPXxOL0SEeqGnUi9OY544OFIeqNh0vFRcEZA8L9RQDCxE5hGtGB8LLRYU5cQEdrt8S7eeKYKsi2O4vvDeqzewie2ZCWepS7po6tN1CepY6lt+MDe5W4XFPuaiVGNJmdtbSqZF4ccEoq407uxLs5QTLrz5pqDcUchnkchlSbxyDJ5JNKzPLZK1hKNDTCesfmoan5o6AWiFH8kh/BHk6i6sTf3mgSLy2ZbiMqLtYw0JEDiHYyxkHnk3udF0VmUyGGcP98Ll5U8WerBTcNrCEDrE9sEyP8cPxF+fCRa1o99hDV0bDSa3Awx2svCsFlUKOpVMj7XqORqlAkIcTimuarIboZDIZHk+OQXyYJwxGwSrkqRRyPHzlMHEBOwCICXDDxuOmWUQWtexhoR5iYCEih6HsomdieowpsCjkMsT2YKp2d4eEANMquh2ZEOGNCW3WWRmo5o8NwvpDxZg3pn3dy6wRnc/yabtQoGWhv7aaW4zQtRjtWsCQqC0GFiIaMGaN8MOEiCEYFezRbg8ke4QOccYt5uXx/XphQ8vB5Nn5cXh2flzXJ15CR0XTAFDf3AK10vZFBYnaYmAhogHDVaPE1w9O7fF1ZDIZXrvFvtV/yXbRfm6QyUxTxwM9nFDTqEej3oC65harOhsie7BvjoiIepWzWoHQIaahtnHhXnAzr7zLOhbqCQYWIiLqdSMCTIXN48OHiFsFcKYQ9QSHhIiIqNf937wRGBnkjjsmR+CHI8UAgDouHkc9wMBCRES9bpi/O568egQAcEiIegWHhIiIqE+5aTgkRD3HwEJERH3KTWPaCJLL81NPMLAQEVGfYtEt9QYGFiIi6lOWISHWsFBPMLAQEVGfcmMPC/UCBhYiIupTYtEte1ioBxhYiIioT1lqWGq5Dgv1AAMLERH1KfawUG9gYCEioj4lFt2yhoV6gIGFiIj6lFh0yx4W6gEGFiIi6lPuloXj2MNCPcDAQkREfcrSw9KgM8BgFCRuDQ1UDCxERNSnXDUK8Xv2slB3MbAQEVGf0igVUCtNHzcMLNRdDCxERNTn3Dm1mXqIgYWIiPpc6/L8XDyOuoeBhYiI+tylNkCsadDj4U8zkHayrL+bRQOIUuoGEBHR4CeudttBDcuGYyX48WgJSmoaMXtkQH83jQYI9rAQEVGfc7/E4nFnyusAAEUXGvu1TTSwMLAQEVGfc3fqfPE4S2Apr21Gc4uhX9tFAwcDCxER9blL1bBYAgsAFFc39VubaGBhYCEioj7XOkvIOrA06gw4V906FFR0oaFf20UDBwMLERH1udYeFutpzTnn66x+Psc6FuoEAwsREfU59056WC4OLIO98NZgFPBt5jlU1jVL3ZQBh4GFiIj6XGc1LDnm+hWZzPRz2+GhwWjtnnw8/nkmXv8lS+qmDDjdCizvvPMOIiMj4eTkhKSkJOzbt6/Tc9etW4fExER4eXnB1dUVCQkJWLNmjfi4Xq/H008/jTFjxsDV1RXBwcFYsmQJiouLu9M0IiJyQJ2tw3LG3MMSH+oFYPDXsGw/fR4AcKKkVuKWDDx2B5YvvvgCKSkpeOGFF5CRkYH4+HjMnTsX5eXlHZ7v7e2NZ599Funp6Thy5AiWLVuGZcuW4eeffwYANDQ0ICMjA8899xwyMjKwbt06ZGVlYcGCBT27MyIichhunazDklNeDwCYOdwPQM+HhA4VXEBJjTS9NDuzKxD73E/45tC5Dh83GAXsy60CABRU1vdn0wYFuwPLG2+8geXLl2PZsmWIi4vDqlWr4OLigtWrV3d4/qxZs3DDDTdg5MiRiI6OxuOPP46xY8di586dAABPT09s2rQJt956K0aMGIHJkyfjn//8Jw4ePIiCgoKe3R0RETkEd037dVhaDEbkVpg+uGeNMAWWMm0TdC3Gbr3G+kNFuOFfu3H7+3thMAo9bHF3Xv8cmvRGfJKe1+HjJ4q1qDXf/4UGPbRN3FfJHnYFFp1Oh4MHDyI5Obn1AnI5kpOTkZ6e3uXzBUFAWloasrKyMGPGjE7Pq6mpgUwmg5eXV4ePNzc3Q6vVWn0REZHj8nFTAzAFkoyCCwBMvSk6gxEapRxjQ72gUcphFIDSGvvXYjlZosWKdUcBALkV9fjleKn4mCD0T3g5eq4aAJBZWI2ahvZhZG9updXPBZWDe/irt9kVWCoqKmAwGBAQYL3XQ0BAAEpLSzt5limAuLm5Qa1WY/78+Xj77bcxZ86cDs9tamrC008/jUWLFsHDw6PDc1JTU+Hp6Sl+hYWF2XMbRETUz4K9nHFdfDCMAvD454egbdKLC8ZF+blBIZchdIgzAPvrWGoa9Xhg7UE06Y1ircz7O84CAP628RTG/ukXHMy/0It30159c4t4P0YB2Hmmot05e85eFFiqGFjs0S+zhNzd3ZGZmYn9+/fjL3/5C1JSUrB169Z25+n1etx6660QBAHvvvtup9dbsWIFampqxK/CwsI+bD0REfWGv9wwGqFDnFFY1YiHP83A5/tN/3YP83cDAIQMcQEAFNk5U+jj3XnIr2xA6BBnrH9oKtQKOTIKqvHM+qN4d2sOaptb8MpPJ9v1tJwo1uLn46W90gNzokSLtqNQluJaC4NRwF5z/UqUnysAIJ89LHaxK7D4+vpCoVCgrMx6C/CysjIEBgZ2/iJyOYYNG4aEhAQ8+eSTuPnmm5Gammp1jiWs5OfnY9OmTZ32rgCARqOBh4eH1RcRETk2DycV3lo0Dgq5DDuyK7D5pOmzZESAKbC09rDYF1gs4eCRK4chJsAd1ycEAwD+u7e1DnJ/3gWk57T2cBiNAu7+aD/uX3MQG452PkJgqyNFNQAALxdTrc727PNWQehkiRa1TS1w1yhx7eggAEBBVdeFt/XNLWjS93x/pV+Ol2L9oaJ+Gx7rC3YFFrVajQkTJiAtLU08ZjQakZaWhilTpth8HaPRiObm1kVzLGElOzsbmzdvho+Pjz3NIiKiAWJ8+BC8dds43DguBLcmhuL+mVFYnBQBAN0aEqpt0uNQYTUA4IoYXwDAPdOHio9fFx+MOyebrr8yLVs8fqq0FqVaU63MX348gQZd+z2O7HHsnCmw3D4pHBqlHCU1Tchus0eSZTgoMXIIhvqaelgsQ0KvbjyFpav3tdv4sbZJj5mvbcHM17Yg03yP3fHPX7Nx35qDeOKLw/gms+MZTAOB0t4npKSkYOnSpUhMTMSkSZOwcuVK1NfXY9myZQCAJUuWICQkROxBSU1NRWJiIqKjo9Hc3IwNGzZgzZo14pCPXq/HzTffjIyMDPzwww8wGAxiPYy3tzfUanVv3SsRETmA+WODMH9sULvjIV6mwGLP8vx7z1bBYBQQ6eOCUPOQUmygB36XHIPCqka8/NvRqG7U4Yv9hdiXW4X0nEpMifbBzjOtQzbFNU3415Yc/H7uiG7f05GiagDAxKHeOFasxfbT57H99HkMD3AH0FrTkhTlg3AfUzvzKxtQ06DHqm05MApAZkE1kqJa/8O+60wlKup0AIBb30vHazePxfUJIXa167WfT+GdLTniz89/cxwTI73F39VAYncNy8KFC/H666/j+eefR0JCAjIzM7Fx40axELegoAAlJSXi+fX19XjooYcwatQoTJs2DV9//TXWrl2Le++9FwBw7tw5fPfddygqKkJCQgKCgoLEr927d/fSbRIRkaOzfIjaMyRkCQLThvlaHf9d8nD8/dZ4OKsVCPJ0xsKJpskZ/9p6BgCwI9v0vCvMz/v39rPIq+h8iKZM24Q3N53G6bL2C77VNulx1vzcMSGemGHu6dmaZQpFNQ167DK3c3asPyK8TfdZXN2IrafLxdqXwovue3u26fnuGiV0LUY8/nkmNp+wLsm4lP15VWJY+b95sRgX7oXa5hY8+eVhm6d951bUQ2/o3jTz3tatottHHnkE+fn5aG5uxt69e5GUlCQ+tnXrVnz00Ufizy+//DKys7PR2NiIqqoq7N69GwsXLhQfj4yMhCAIHX7NmjWr2zdGREQDS7j5g/xcdSNSN5yE3mDEvtwqvLs1B1X1ug6fYwks02N8O3zcYvn0KMhkpqByqlSL/XmmAtjnfhOHGcP9oDMY8XEn66cIgoBH/3sI/0jLxrX/2IHUn06ivLZJrAc5XqyFIJh6iHzdNJg90vQf+N05FSisasAvJ0qhNwgYEeCOmAB3+Llr4KxSwCgAn+1rrbNpO2tIEASxNufNhQlYNMkUuJ7++gjO19q2D5Flavf1CcF4YGY03rw1AS5qBfbmVllN++7Mr6fKcOXrW/G7zzNter2+xr2EiIjIIfi5a/DY7BgAwHvbz2LcS5tw63vp+NvGU3j880PtCkZLa5pwprwOMhkwJerSgSXcxwVXjvAHADz11RE06Y3wd9dgeIAblphrXDYcLYHRaPoP80e7crEmPQ9Go4DvDhdjX14V5DKgxSjgvW1nMekvaRj3502456P9YugYE+IJABjq64rpMb4wCsCaPfn44Yhp1OE35mEwmUwmhrM9Z6vENha1CSy5FfUoutAItUKOqcN88KcFoxAb6I7Keh2e+t/hdr8LQRCwJj0PKV9mimvApJ0yrUB/dZxpUkykrytumRAKwFSE3JVP95ju68ejJWIPkZQYWIiIyGGkzBmOfy0eD1e1AnXNLXBVK6BWyLEjuwLfHymxOtfyITo2xBOe5tk5l3LnFFMwOWoukL1imC9kMhmmD/eFu5MSZdpmHCy4gJ1nKvCn70/guW+P48FPD+KvG06KbfvPkkTEBrpDJgOqG/RIO1WObzNNe9+NCfUUX+uuqZEATD0olnZe26ZuJ8y7fQ1JYZtiY0vvSmLkELioldAoFXhr0TholHJszTqPdRmtxbNNegOe/Oownvv2ONZlnMO/d+Qgt6IeZ8/XQymXYcbw1jAXH+YFADhsrrnpTFW9DtvaTM3+8w8nJFk9uC27i26JiIj60rVjgjAmxBPHi7WYMdwX72/PxZubT+Ol708gyNMJO7IrcLSoWpxKfHH9SmdmxvghwsdFXP/EMqtIo1Tg6rhAfJ1RhB8OF+NUaWudys/HTTUj4d4uuHd6FJxUCiTHBaBJb0B2WR2+P1KMLw8Uoq6pRezBAYArR/hbvdbIIA9E+7mJj0f4tAYWXzc1Kup0VkNC2801NjPMeywBwPAAdzw4KxorN2fj+yPFuGlCKARBwD0f78euM61TttfuKYCTUgEASIryhrtTa5izBJbjxTXQG4xQKVr7LXQtRjS3GODupMKPR0vQYhQQ7eeKijodTpXW4ov9hbg9Kdym33VfYA8LERE5nDBvF1wzOhAuaiUemBWFKD9XVNQ145ZV6XgrLRtbss6jsl4HlUKGa8e0n3HUEblcJk5xBloLboHW4ZovDxRhb24VVAoZ3l40Dt6uashkwAvXxcFJpRDPd1IpMCbUE89cOxJ7VszG/meTERfsYfVaS6ZEtru+RdvAcmuiqT6lTNuMJr0BzS0Gcc2YGTF+Vs+bZ17DJT2nEk16AzIKqrHrTCWcVHJ8cvckDPV1RU2jHm//aiouvirWemX6oT6ucNco0aQ3WhUQC4KAhf9Ox+S/pmH3mQp8a97A8baJ4Xgi2TRM9/dfsiTd/4g9LERE5NA0SgX+esMY3PnBXijkMswc7ocrhvki2t8NIwLc4eOmsflatySG4euMcxge4AZ/Dyfx+LRhvvB0VqGm0fSBfPOEUFwXH4wrhvmiVNuEkUGdL1DqpFJYhZnW1wrFys2n0agztAssbYeErk8Iwce781CvM6DoQiPKa5vQqDfAz12DkUHuVs8bHuCGIE8nlNQ0If1sJXacNvXEXDMqEDOG++HuK4biuW+OQWee2TM71t/q+XK5DGPDPLHrTCWOFNVgVLBpGOtEiRaHCqoBAHd/vB9NeiNkMtM6Nj5uaqzdW4AxIZ7Qd3Njyt7AwEJERA5vcpQP0lfMhqtaCWd1+3BgK09nFX56fHq742qlHHNHBeDLA0WQy4D7Z0QDAIa4qjHEtXvrgXk4qfC/B6airlmPCB9Xq8dGBXnASSVH2BAXDA9wQ5i3C06V1qLwQgMOmfc9mhbtA5lMZvU8mUyGWSP88dm+Amw5VY5fzENW88eaVve9eXwo3vglCxca9Ijyc0Wkr/XrAsDYUC/sOlOJw4XVWDTJNMTz/WFTfZBKIUOT3hRKpkT5INDTFOq+e2QaXNTSRgYOCRER0YDg66bpUVjpyu1JEVApTMNGHX3Qd8eIQHdMiPBud9zfwwkbH5+B/y6fDJlMJva4FFU1YJ95ynXbReTamjXCNEz05YFClGqb4K5RioW1zmoF7p0eBQBYEB/c4fPjQ70AAIfNNUCCIOD7w6bC4VdvHosrzde/o83wmdRhBWAPCxEREQAgIcwLR/80F2pF//xfvm0oskxzzjlfLw7NTBraPugApuGrtj0hc0YFQKNsDXIPzYrGFcN8rWpq2ooPMw0DnS6rRaPOgJOlWpyrboSrWoFrRgXh+vgQnKtu7HAmk5TYw0JERGTmpFJALpd1fWIvCzPvo/TTsRI0txjh66ZGVCe9PG4aJSZGtoaZ68Za96TIZDLEh3lZzQBqK9DDCf7uGhiMAo4X1+A787TsOXEBcFab7t/RwgrAwEJERCQ5S0Ao05pWsZ0Y6d2ufqUty7CQp7PK5mndFjKZDGPNw0Jv/3oG35mHg67rZAjJUTCwEBERSSz8oh6Ntj0oHblpfCiShnrj93NHQK20/6N8XLgXAGDb6fOoqtfBy0WF6RdNoXY0rGEhIiKS2MW7J3dWv2Lh46bBF/dP6fbr3TE5ArVNLWhuMcBFrcBVsQHdCj79iYGFiIhIYs5qBfzcNThf2wx3jfKS6770Bk9nFf5vXmyfvkZvc+w4RUREdJmwFN5OiBwChQSFv46OgYWIiMgBjAg0rWo7Ldq+ItrLBYeEiIiIHMATc4ZjTIgXbp4QKnVTHBIDCxERkQPwd3eSdDdkR8chISIiInJ4DCxERETk8BhYiIiIyOExsBAREZHDY2AhIiIih8fAQkRERA6PgYWIiIgcHgMLEREROTwGFiIiInJ4DCxERETk8BhYiIiIyOExsBAREZHDY2AhIiIihzcodmsWBAEAoNVqJW4JERER2cryuW35HL+UQRFYamtrAQBhYWESt4SIiIjsVVtbC09Pz0ueIxNsiTUOzmg0ori4GO7u7pDJZL16ba1Wi7CwMBQWFsLDw6NXr+0oBvs9Dvb7A3iPg8Fgvz+A9zgY9Pb9CYKA2tpaBAcHQy6/dJXKoOhhkcvlCA0N7dPX8PDwGJR/+Noa7Pc42O8P4D0OBoP9/gDe42DQm/fXVc+KBYtuiYiIyOExsBAREZHDY2DpgkajwQsvvACNRiN1U/rMYL/HwX5/AO9xMBjs9wfwHgcDKe9vUBTdEhER0eDGHhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2Ng6cI777yDyMhIODk5ISkpCfv27ZO6Sd2SmpqKiRMnwt3dHf7+/vjtb3+LrKwsq3NmzZoFmUxm9fXAAw9I1GL7/elPf2rX/tjYWPHxpqYmPPzww/Dx8YGbmxtuuukmlJWVSdhi+0RGRra7P5lMhocffhjAwHz/tm/fjuuuuw7BwcGQyWT45ptvrB4XBAHPP/88goKC4OzsjOTkZGRnZ1udU1VVhcWLF8PDwwNeXl645557UFdX1493cWmXuke9Xo+nn34aY8aMgaurK4KDg7FkyRIUFxdbXaOj9/6VV17p5zvpWFfv4V133dWu7ddcc43VOQP5PQTQ4d9LmUyG1157TTzHkd9DWz4fbPn3s6CgAPPnz4eLiwv8/f3x1FNPoaWlpdfaycByCV988QVSUlLwwgsvICMjA/Hx8Zg7dy7Ky8ulbprdtm3bhocffhh79uzBpk2boNfrcfXVV6O+vt7qvOXLl6OkpET8evXVVyVqcfeMGjXKqv07d+4UH3viiSfw/fff46uvvsK2bdtQXFyMG2+8UcLW2mf//v1W97Zp0yYAwC233CKeM9Dev/r6esTHx+Odd97p8PFXX30Vb731FlatWoW9e/fC1dUVc+fORVNTk3jO4sWLcfz4cWzatAk//PADtm/fjvvuu6+/bqFLl7rHhoYGZGRk4LnnnkNGRgbWrVuHrKwsLFiwoN25L730ktV7++ijj/ZH87vU1XsIANdcc41V2z/77DOrxwfyewjA6t5KSkqwevVqyGQy3HTTTVbnOep7aMvnQ1f/fhoMBsyfPx86nQ67d+/Gxx9/jI8++gjPP/987zVUoE5NmjRJePjhh8WfDQaDEBwcLKSmpkrYqt5RXl4uABC2bdsmHps5c6bw+OOPS9eoHnrhhReE+Pj4Dh+rrq4WVCqV8NVXX4nHTp48KQAQ0tPT+6mFvevxxx8XoqOjBaPRKAjCwH//AAjr168XfzYajUJgYKDw2muviceqq6sFjUYjfPbZZ4IgCMKJEycEAML+/fvFc3766SdBJpMJ586d67e22+rie+zIvn37BABCfn6+eCwiIkJ48803+7ZxvaCj+1u6dKlw/fXXd/qcwfgeXn/99cJVV11ldWygvIeC0P7zwZZ/Pzds2CDI5XKhtLRUPOfdd98VPDw8hObm5l5pF3tYOqHT6XDw4EEkJyeLx+RyOZKTk5Geni5hy3pHTU0NAMDb29vq+KeffgpfX1+MHj0aK1asQENDgxTN67bs7GwEBwcjKioKixcvRkFBAQDg4MGD0Ov1Vu9nbGwswsPDB+T7qdPpsHbtWtx9991WG34O9PevrdzcXJSWllq9Z56enkhKShLfs/T0dHh5eSExMVE8Jzk5GXK5HHv37u33NveGmpoayGQyeHl5WR1/5ZVX4OPjg3HjxuG1117r1a72vrZ161b4+/tjxIgRePDBB1FZWSk+Ntjew7KyMvz444+455572j02UN7Diz8fbPn3Mz09HWPGjEFAQIB4zty5c6HVanH8+PFeadeg2PywL1RUVMBgMFj98gEgICAAp06dkqhVvcNoNOJ3v/sdpk2bhtGjR4vHb7/9dkRERCA4OBhHjhzB008/jaysLKxbt07C1touKSkJH330EUaMGIGSkhK8+OKLmD59Oo4dO4bS0lKo1ep2HwIBAQEoLS2VpsE98M0336C6uhp33XWXeGygv38Xs7wvHf0dtDxWWloKf39/q8eVSiW8vb0H5Pva1NSEp59+GosWLbLaWO6xxx7D+PHj4e3tjd27d2PFihUoKSnBG2+8IWFrbXPNNdfgxhtvxNChQ5GTk4NnnnkG8+bNQ3p6OhQKxaB7Dz/++GO4u7u3G24eKO9hR58Ptvz7WVpa2uHfVctjvYGB5TL08MMP49ixY1b1HQCsxozHjBmDoKAgzJ49Gzk5OYiOju7vZtpt3rx54vdjx45FUlISIiIi8OWXX8LZ2VnClvW+Dz74APPmzUNwcLB4bKC/f5c7vV6PW2+9FYIg4N1337V6LCUlRfx+7NixUKvVuP/++5GamurwS8Dfdttt4vdjxozB2LFjER0dja1bt2L27NkStqxvrF69GosXL4aTk5PV8YHyHnb2+eAIOCTUCV9fXygUinZV0GVlZQgMDJSoVT33yCOP4IcffsCWLVsQGhp6yXOTkpIAAGfOnOmPpvU6Ly8vDB8+HGfOnEFgYCB0Oh2qq6utzhmI72d+fj42b96Me++995LnDfT3z/K+XOrvYGBgYLsi+JaWFlRVVQ2o99USVvLz87Fp0yar3pWOJCUloaWlBXl5ef3TwF4UFRUFX19f8c/lYHkPAWDHjh3Iysrq8u8m4JjvYWefD7b8+xkYGNjh31XLY72BgaUTarUaEyZMQFpamnjMaDQiLS0NU6ZMkbBl3SMIAh555BGsX78ev/76K4YOHdrlczIzMwEAQUFBfdy6vlFXV4ecnBwEBQVhwoQJUKlUVu9nVlYWCgoKBtz7+eGHH8Lf3x/z58+/5HkD/f0bOnQoAgMDrd4zrVaLvXv3iu/ZlClTUF1djYMHD4rn/PrrrzAajWJgc3SWsJKdnY3NmzfDx8eny+dkZmZCLpe3G0oZCIqKilBZWSn+uRwM76HFBx98gAkTJiA+Pr7Lcx3pPezq88GWfz+nTJmCo0ePWoVPS/iOi4vrtYZSJz7//HNBo9EIH330kXDixAnhvvvuE7y8vKyqoAeKBx98UPD09BS2bt0qlJSUiF8NDQ2CIAjCmTNnhJdeekk4cOCAkJubK3z77bdCVFSUMGPGDIlbbrsnn3xS2Lp1q5Cbmyvs2rVLSE5OFnx9fYXy8nJBEAThgQceEMLDw4Vff/1VOHDggDBlyhRhypQpErfaPgaDQQgPDxeefvppq+MD9f2rra0VDh06JBw6dEgAILzxxhvCoUOHxBkyr7zyiuDl5SV8++23wpEjR4Trr79eGDp0qNDY2Che45prrhHGjRsn7N27V9i5c6cQExMjLFq0SKpbaudS96jT6YQFCxYIoaGhQmZmptXfTcvMit27dwtvvvmmkJmZKeTk5Ahr164V/Pz8hCVLlkh8ZyaXur/a2lrh97//vZCeni7k5uYKmzdvFsaPHy/ExMQITU1N4jUG8ntoUVNTI7i4uAjvvvtuu+c7+nvY1eeDIHT972dLS4swevRo4eqrrxYyMzOFjRs3Cn5+fsKKFSt6rZ0MLF14++23hfDwcEGtVguTJk0S9uzZI3WTugVAh18ffvihIAiCUFBQIMyYMUPw9vYWNBqNMGzYMOGpp54SampqpG24HRYuXCgEBQUJarVaCAkJERYuXCicOXNGfLyxsVF46KGHhCFDhgguLi7CDTfcIJSUlEjYYvv9/PPPAgAhKyvL6vhAff+2bNnS4Z/LpUuXCoJgmtr83HPPCQEBAYJGoxFmz57d7t4rKyuFRYsWCW5uboKHh4ewbNkyoba2VoK76dil7jE3N7fTv5tbtmwRBEEQDh48KCQlJQmenp6Ck5OTMHLkSOGvf/2r1Qe+lC51fw0NDcLVV18t+Pn5CSqVSoiIiBCWL1/e7j99A/k9tHjvvfcEZ2dnobq6ut3zHf097OrzQRBs+/czLy9PmDdvnuDs7Cz4+voKTz75pKDX63utnTJzY4mIiIgcFmtYiIiIyOExsBAREZHDY2AhIiIih8fAQkRERA6PgYWIiIgcHgMLEREROTwGFiIiInJ4DCxERETk8BhYiIiIyOExsBAREZHDY2AhIiIih8fAQkRERA7v/wEonNsd8hj5IAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(torch.tensor(lossi).shape)                        # (200000,)\n",
    "print(torch.tensor(lossi).view(-1, 1000).shape)         # Make each row carry 1000 consecutive elements\n",
    "print(torch.tensor(lossi).view(-1, 1000).mean(1).shape) # Row-wise (1) mean --> 200 rows/means\n",
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));   # Mean on every Row -> 200 means"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f427f59",
   "metadata": {},
   "source": [
    "Let's also determine the train and val losses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "b6c696d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into eval mode (needed for batchnorm especially)\n",
    "# (Accessing Sequential's layers directly like this is bad, we'll fix this later)\n",
    "for layer in model.layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "602a000d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train\t2.058220863342285\n",
      "val\t2.1056692600250244\n"
     ]
    }
   ],
   "source": [
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]   # That's a switch!\n",
    "    \n",
    "    logits = model(x) # This got simplified\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(f'{split}\\t{loss.item()}')\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "cdbbae50",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ivon.\n",
      "fanili.\n",
      "thoommestenell.\n",
      "mattevyn.\n",
      "alana.\n",
      "joleshaun.\n",
      "siah.\n",
      "prus.\n",
      "carleen.\n",
      "jah.\n",
      "jorrena.\n",
      "joriah.\n",
      "jas.\n",
      "vishylaharia.\n",
      "juna.\n",
      "vio.\n",
      "orven.\n",
      "mina.\n",
      "laylee.\n",
      "esteffead.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the model\n",
    "for _ in range(20):\n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all ...\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      logits = model(torch.tensor([context]))\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6f2276",
   "metadata": {},
   "source": [
    "We see that the losses on training and validation set are very similar to one another.<br>\n",
    "This hints at the model being able to generalize pretty well.\n",
    "\n",
    "**It does not overfit. That's good.**\n",
    "\n",
    "This is a good basis for trying to increase the model size<br>\n",
    "to deepen the understanding and further reducing the loss we have."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10671ab2",
   "metadata": {},
   "source": [
    "## Going deeper\n",
    "\n",
    "Ok, back to the beginning.<br>\n",
    "Right now, we use a (more elegantly implemented) architecture to take in three letters and predict the fourth.<br>\n",
    "The problem is that while we are able to in theory \"just add more layers\", in the beginning we effectively crush all the input characters into one layer.<br>This is the architecture's core problem/bottleneck. The [WaveNet](https://arxiv.org/abs/1609.03499) approach addresses this:\n",
    "\n",
    "![](./images/convolutions_1.png)\n",
    "\n",
    "The different characters are fed in, and not just crushed into a single layer at once.<br>\n",
    "This processes of crushing now happens much less harshly across several layers.<br>\n",
    "This **lowers the risk of information loss**.\n",
    "\n",
    "In the first layer, we \"fuse\" two characters. Then, we \"fuse\" the fusions of those two characters (now considering $4$ characters in total) and so on. A tree structure emerges, which we can scale in depth (therefore we can scale the context size) to our liking.\n",
    "\n",
    "> This exact concept of stacking up layers and \"tree-branch-style\" merging inputs is called<br>\n",
    "> **\"dilated causal convolutional layers\"**\n",
    "\n",
    "**In short:**<br>\n",
    "We're actually going to make the network deeper, but mostly regarding the input process.<br>\n",
    "This will already add more information for us to learn from,<br>\n",
    "as we can now take characters into account in a more \"contextual\" way."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "490d5f18",
   "metadata": {},
   "source": [
    "First, let's increase the `block_size` from $3$ to $8$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "d3817d55",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([182625, 8]) torch.Size([182625])\n",
      "torch.Size([22655, 8]) torch.Size([22655])\n",
      "torch.Size([22866, 8]) torch.Size([22866])\n"
     ]
    }
   ],
   "source": [
    "# Build the dataset\n",
    "block_size = 8 # Context length: How many characters do we take to predict the next one?\n",
    "\n",
    "def build_dataset(words):  \n",
    "  X, Y = [], []\n",
    "  \n",
    "  for w in words:\n",
    "    context = [0] * block_size\n",
    "    for ch in w + '.':\n",
    "      ix = stoi[ch]\n",
    "      X.append(context)\n",
    "      Y.append(ix)\n",
    "      context = context[1:] + [ix] # Crop and append\n",
    "\n",
    "  X = torch.tensor(X)\n",
    "  Y = torch.tensor(Y)\n",
    "    \n",
    "  print(X.shape, Y.shape)\n",
    "  return X, Y\n",
    "\n",
    "# These are the \"markers\" we will use to divide the dataset\n",
    "n1 = int(0.8 * len(words))\n",
    "n2 = int(0.9 * len(words))\n",
    "\n",
    "# Dividing the dataset into train, dev and test splits\n",
    "Xtr, Ytr = build_dataset(words[:n1])     # 80%\n",
    "Xdev, Ydev = build_dataset(words[n1:n2]) # 10%\n",
    "Xte, Yte = build_dataset(words[n2:])     # 10%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "3516d514",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "........ --> y\n",
      ".......y --> u\n",
      "......yu --> h\n",
      ".....yuh --> e\n",
      "....yuhe --> n\n",
      "...yuhen --> g\n",
      "..yuheng --> .\n",
      "........ --> d\n",
      ".......d --> i\n",
      "......di --> o\n",
      ".....dio --> n\n",
      "....dion --> d\n",
      "...diond --> r\n",
      "..diondr --> e\n",
      ".diondre --> .\n",
      "........ --> x\n",
      ".......x --> a\n",
      "......xa --> v\n",
      ".....xav --> i\n",
      "....xavi --> e\n"
     ]
    }
   ],
   "source": [
    "# Show the dataset's layout -> Eight inputs, one associated/expected output\n",
    "for x, y in zip(Xtr[:20], Ytr[:20]):\n",
    "    print(''.join(itos[ix.item()] for ix in x), '-->', itos[y.item()])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7e53add",
   "metadata": {},
   "source": [
    "The model architecture remains for now. The number of parameters has increased by $10,000$. This is due to the larger context size. \n",
    "\n",
    "**The question for now is:**<br>\n",
    "Does plainly increasing the context size yield a lower loss, all while still crushing everything into the first layer?\n",
    "\n",
    "Let's try that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "10b83be5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22097\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10    # Dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# Setting up the network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd), # previously C = torch.randn((vocab_size, n_embd))\n",
    "    Flatten(), # previously C.view(-1, n_embd * block_size)\n",
    "    Linear(n_embd * block_size, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "4e283119",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.6036\n",
      "  10000/ 200000: 2.1418\n",
      "  20000/ 200000: 2.2006\n",
      "  30000/ 200000: 2.4524\n",
      "  40000/ 200000: 2.2123\n",
      "  50000/ 200000: 1.8461\n",
      "  60000/ 200000: 2.5121\n",
      "  70000/ 200000: 2.1705\n",
      "  80000/ 200000: 2.3022\n",
      "  90000/ 200000: 2.2967\n",
      " 100000/ 200000: 1.9490\n",
      " 110000/ 200000: 2.0411\n",
      " 120000/ 200000: 2.8504\n",
      " 130000/ 200000: 1.5823\n",
      " 140000/ 200000: 2.0511\n",
      " 150000/ 200000: 2.0658\n",
      " 160000/ 200000: 2.0859\n",
      " 170000/ 200000: 1.7717\n",
      " 180000/ 200000: 1.4267\n",
      " 190000/ 200000: 2.0282\n"
     ]
    }
   ],
   "source": [
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "5eeab59c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfV0lEQVR4nO3deVzUdf4H8Nd3Zpjhvu/7UvECVJTwtkhtLa22stbSqO2+qbbcfunW1tK9bmVabad2WJvdZRlemXiBeIsKIgJyywznMMx8f3/MzBdGQBiuAXw9Hw8eK9/5fr98vkw6r/183p/PRxBFUQQRERHRACazdQOIiIiIOsPAQkRERAMeAwsRERENeAwsRERENOAxsBAREdGAx8BCREREAx4DCxEREQ14DCxEREQ04Cls3YDeYDAYUFxcDBcXFwiCYOvmEBERUReIooiamhoEBgZCJrtwH8qQCCzFxcUICQmxdTOIiIioG86cOYPg4OALnjMkAouLiwsA4wO7urrauDVERETUFRqNBiEhIdLn+IUMicBiHgZydXVlYCEiIhpkulLOwaJbIiIiGvAYWIiIiGjAY2AhIiKiAY+BhYiIiAY8BhYiIiIa8BhYiIiIaMBjYCEiIqIBr1uBZeXKlQgPD4e9vT0SExOxe/fuLl33+eefQxAEXH311RbHRVHEsmXLEBAQAAcHByQnJ+PEiRPdaRoRERENQVYHlnXr1iE1NRXLly9HVlYW4uLiMGfOHJSVlV3wuvz8fDz22GOYNm1am9deeuklvP7661i9ejV27doFJycnzJkzB42NjdY2j4iIiIYgqwPLa6+9hjvuuAMpKSkYNWoUVq9eDUdHR7z//vsdXqPX67Fo0SI888wziIyMtHhNFEWsWLEC//d//4cFCxYgNjYWH3/8MYqLi/HNN99Y/UBEREQ09FgVWJqampCZmYnk5OSWG8hkSE5ORkZGRofXPfvss/D19cXtt9/e5rVTp06hpKTE4p5ubm5ITEzs8J5arRYajcbii4iIiIYuqwJLRUUF9Ho9/Pz8LI77+fmhpKSk3Wu2b9+O9957D++++267r5uvs+aeaWlpcHNzk764UzMREdHQ1qezhGpqanDLLbfg3Xffhbe3d6/dd+nSpVCr1dLXmTNneu3erWmb9fjnD0ew7NtD0Dbr++RnEBERUees2q3Z29sbcrkcpaWlFsdLS0vh7+/f5vzc3Fzk5+fjqquuko4ZDAbjD1YokJOTI11XWlqKgIAAi3vGx8e32w6VSgWVSmVN07vtve2nAACPzh4BlULeLz+TiIiILFnVw6JUKjFhwgSkp6dLxwwGA9LT05GUlNTm/JiYGBw8eBDZ2dnS1/z58zFr1ixkZ2cjJCQEERER8Pf3t7inRqPBrl272r1nf7KTtfx6dHqDDVtCRER0cbOqhwUAUlNTsWTJEiQkJGDSpElYsWIF6urqkJKSAgBYvHgxgoKCkJaWBnt7e4wZM8biend3dwCwOP7www/jueeew7BhwxAREYGnn34agYGBbdZr6W8ymQA7uQCdXmRgISIisiGrA8vChQtRXl6OZcuWoaSkBPHx8diwYYNUNFtQUACZzLrSmL/97W+oq6vDnXfeierqakydOhUbNmyAvb29tc3rdXZyGXR6PZqaGViIiIhsRRBFUbR1I3pKo9HAzc0NarUarq6uvXrvuGd+hbpBh99SpyPa16VX701ERHQxs+bzm3sJdUKpMP6KmpoHfa4jIiIatBhYOqGUmwILa1iIiIhshoGlE3ZyAQBnCREREdkSA0snzENCOhbdEhER2QwDSyfsTENCWvawEBER2QwDSyfMgYU9LERERLbDwNIJaUhIz1lCREREtsLA0omWWULc/JCIiMhWGFg6Ic0S4josRERENsPA0glp4TgW3RIREdkMA0snzEW33EuIiIjIdhhYOmGuYeHCcURERLbDwNKJlllCDCxERES2wsDSCQ4JERER2R4DSyekwMJ1WIiIiGyGgaUTHBIiIiKyPQaWTihN67BwSIiIiMh2GFg6YcdZQkRERDbHwNIJLhxHRERkewwsneAsISIiIttjYOmEHYtuiYiIbI6BpRMqqYaF05qJiIhshYGlE3YKzhIiIiKyNQaWTrQsHMfAQkREZCsMLJ3g5odERES2x8DSCXPRLYeEiIiIbIeBpRPsYSEiIrI9BpZOtOwlxFlCREREtsLA0gkuHEdERGR7DCydsDNvfsghISIiIpthYOmEiivdEhER2RwDSyc4JERERGR7DCydsOMsISIiIptjYOlE61lCosiZQkRERLbAwNIJcw8LwMJbIiIiW2Fg6YSyVWDhWixERES2wcDSCfOQEADoWHhLRERkEwwsnZDLBMiMS7FwSIiIiMhGGFi6gFObiYiIbIuBpQuUXDyOiIjIphhYusBceMshISIiIttgYOkCafG4Zs4SIiIisgUGli4wDwmxh4WIiMg2GFi6QNqxmUW3RERENsHA0gVKhRwAi26JiIhshYGlC5SmHhYGFiIiIttgYOkCrsNCRERkWwwsXcCiWyIiIttiYOkCaVozNz8kIiKyiW4FlpUrVyI8PBz29vZITEzE7t27Ozx3/fr1SEhIgLu7O5ycnBAfH481a9ZYnFNbW4v7778fwcHBcHBwwKhRo7B69eruNK1PcEiIiIjIthTWXrBu3TqkpqZi9erVSExMxIoVKzBnzhzk5OTA19e3zfmenp546qmnEBMTA6VSiR9++AEpKSnw9fXFnDlzAACpqanYtGkT1q5di/DwcPz666+49957ERgYiPnz5/f8KXtIxaX5iYiIbMrqHpbXXnsNd9xxB1JSUqSeEEdHR7z//vvtnj9z5kxcc801GDlyJKKiovDQQw8hNjYW27dvl87ZsWMHlixZgpkzZyI8PBx33nkn4uLiLthz05/sOEuIiIjIpqwKLE1NTcjMzERycnLLDWQyJCcnIyMjo9PrRVFEeno6cnJyMH36dOn45MmT8d1336GoqAiiKGLz5s04fvw4Zs+e3e59tFotNBqNxVdfMg8JaTkkREREZBNWDQlVVFRAr9fDz8/P4rifnx+OHTvW4XVqtRpBQUHQarWQy+V46623cPnll0uvv/HGG7jzzjsRHBwMhUIBmUyGd9991yLUtJaWloZnnnnGmqb3CHdrJiIisi2ra1i6w8XFBdnZ2aitrUV6ejpSU1MRGRmJmTNnAjAGlp07d+K7775DWFgYtm3bhvvuuw+BgYEWvTlmS5cuRWpqqvS9RqNBSEhIn7W/ZZYQAwsREZEtWBVYvL29IZfLUVpaanG8tLQU/v7+HV4nk8kQHR0NAIiPj8fRo0eRlpaGmTNnoqGhAX//+9/x9ddfY968eQCA2NhYZGdn45VXXmk3sKhUKqhUKmua3iPSOiwcEiIiIrIJq2pYlEolJkyYgPT0dOmYwWBAeno6kpKSunwfg8EArVYLANDpdNDpdJDJLJsil8thMAyMgKDkOixEREQ2ZfWQUGpqKpYsWYKEhARMmjQJK1asQF1dHVJSUgAAixcvRlBQENLS0gAY600SEhIQFRUFrVaLn376CWvWrMGqVasAAK6urpgxYwYef/xxODg4ICwsDFu3bsXHH3+M1157rRcftfukdVg4JERERGQTVgeWhQsXory8HMuWLUNJSQni4+OxYcMGqRC3oKDAorekrq4O9957LwoLC+Hg4ICYmBisXbsWCxculM75/PPPsXTpUixatAhVVVUICwvD888/j7vvvrsXHrHn7BTGac0cEiIiIrINQRTFQT/OodFo4ObmBrVaDVdX116//39/z8NzPx7FgvhA/OfGcb1+fyIioouRNZ/f3EuoCzitmYiIyLYYWLqAewkRERHZFgNLFyilottBP3pGREQ0KDGwdIGdeUiIPSxEREQ2wcDSBUrT5oec1kxERGQbDCxdwKJbIiIi22Jg6QIW3RIREdkWA0sXcKVbIiIi22Jg6QIOCREREdkWA0sXSJsfNnNaMxERkS0wsHQBh4SIiIhsi4GlC5Rch4WIiMimGFi6wI7rsBAREdkUA0sXKFsNCQ2Bza2JiIgGHQaWLjAPCYkioDcwsBAREfU3BpYuMBfdAoCOGyASERH1OwaWLmgdWLjaLRERUf9jYOkCc9EtwMJbIiIiW2Bg6QJBEFoWj2NgISIi6ncMLF0kTW3mkBAREVG/Y2DpIu4nREREZDsMLF3E5fmJiIhsh4Gli6TAwiEhIiKifsfA0kUqaUiI67AQERH1NwaWLrLjLCEiIiKbYWDpIjsFZwkRERHZCgNLF5nXYdEysBAREfU7BpYu8nRSAgDKahpt3BIiIqKLDwNLF0X7ugAATpTW2rglREREFx8Gli4a7ucMADheWmPjlhAREV18GFi6aJi5h6WMPSxERET9jYGli6J9nSEIQFVdEyprtbZuDhER0UWFgaWLHJRyhHg4AgCOs46FiIioXzGwWGGYr7GO5UQZ61iIiIj6EwOLFYb5caYQERGRLTCwWIEzhYiIiGyDgcUKnClERERkGwwsVmg9U6iCM4WIiIj6DQOLFVrPFGIdCxERUf9hYLESZwoRERH1PwYWK3GmEBERUf9jYLFSlI8TACCvgoGFiIiovzCwWCnSxzgklFdeZ+OWEBERXTwYWKwU6W3sYTmrbkR9U7ONW0NERHRxYGCxkoeTEh6OdgCAUxXsZSEiIuoPDCzdEGHqZeGwEBERUf9gYOkGcx0Le1iIiIj6BwNLN0SaZwqVc6YQERFRf2Bg6QZz4S17WIiIiPpHtwLLypUrER4eDnt7eyQmJmL37t0dnrt+/XokJCTA3d0dTk5OiI+Px5o1a9qcd/ToUcyfPx9ubm5wcnLCxIkTUVBQ0J3m9bnWU5tFUbRxa4iIiIY+qwPLunXrkJqaiuXLlyMrKwtxcXGYM2cOysrK2j3f09MTTz31FDIyMnDgwAGkpKQgJSUFv/zyi3RObm4upk6dipiYGGzZsgUHDhzA008/DXt7++4/WR8K9XSEIAA12maUcxNEIiKiPieIVnYRJCYmYuLEiXjzzTcBAAaDASEhIXjggQfw5JNPduke48ePx7x58/DPf/4TAHDjjTfCzs6u3Z6XrtBoNHBzc4NarYarq2u37mGtaS9twpmqBqy78xIkRnr1y88kIiIaSqz5/Laqh6WpqQmZmZlITk5uuYFMhuTkZGRkZHR6vSiKSE9PR05ODqZPnw7AGHh+/PFHDB8+HHPmzIGvry8SExPxzTffdHgfrVYLjUZj8dXfIr1Nw0KsYyEiIupzVgWWiooK6PV6+Pn5WRz38/NDSUlJh9ep1Wo4OztDqVRi3rx5eOONN3D55ZcDAMrKylBbW4sXXngBc+fOxa+//oprrrkG1157LbZu3dru/dLS0uDm5iZ9hYSEWPMYvaJlLRbOFCIiIupriv74IS4uLsjOzkZtbS3S09ORmpqKyMhIzJw5EwaDAQCwYMECPPLIIwCA+Ph47NixA6tXr8aMGTPa3G/p0qVITU2VvtdoNP0eWsybIHKmEBERUd+zKrB4e3tDLpejtLTU4nhpaSn8/f07vE4mkyE6OhqAMYwcPXoUaWlpmDlzJry9vaFQKDBq1CiLa0aOHInt27e3ez+VSgWVSmVN03tdBIeEiIiI+o1VQ0JKpRITJkxAenq6dMxgMCA9PR1JSUldvo/BYIBWq5XuOXHiROTk5Ficc/z4cYSFhVnTvH4V4G6cwVSu4SwhIiKivmb1kFBqaiqWLFmChIQETJo0CStWrEBdXR1SUlIAAIsXL0ZQUBDS0tIAGOtNEhISEBUVBa1Wi59++glr1qzBqlWrpHs+/vjjWLhwIaZPn45Zs2Zhw4YN+P7777Fly5beeco+4ONi7OGp0TajoUkPB6Xcxi0iIiIauqwOLAsXLkR5eTmWLVuGkpISxMfHY8OGDVIhbkFBAWSylo6buro63HvvvSgsLISDgwNiYmKwdu1aLFy4UDrnmmuuwerVq5GWloYHH3wQI0aMwFdffYWpU6f2wiP2DReVAiqFDNpmA8prtAj1crR1k4iIiIYsq9dhGYhssQ4LAEx9cRMKzzXgq3uSMCHMs99+LhER0VDQZ+uwkCVf07BQeQ3rWIiIiPoSA0sP+DCwEBER9QsGlh4wB5YyBhYiIqI+xcDSAz7OpqnNDCxERER9ioGlB3xdOSRERETUHxhYesDH2RRYahlYiIiI+hIDSw+w6JaIiKh/MLD0QOvAYjAM+uVsiIiIBiwGlh7wNg0JNRtEVDfobNwaIiKioYuBpQeUChk8HO0AcFiIiIioLzGw9BDrWIiIiPoeA0sPtSwe12jjlhAREQ1dDCw9JE1tZg8LERFRn2Fg6SFfV652S0RE1NcYWHqIi8cRERH1PQaWHpJqWDQMLERERH2FgaWHpFlC7GEhIiLqMwwsPcRpzURERH2PgaWHfE2BRd2gw39/z8P6rELouUw/ERFRr1LYugGDnZuDHRzs5GjQ6fHcj0cBAA06PRYlhtm4ZUREREMHe1h6SBAEvHJ9HK4dF4RJ4Z4AgK8yC23cKiIioqGFPSy9YF5sAObFBqBM04hL0tKRVVCN/Io6hHs72bppREREQwJ7WHqRr6s9pg7zAQB8k11k49YQERENHQwsveyacYEAgK/3FUEUWXxLRETUGxhYetnsUf5wsJPjdGU99p2ptnVziIiIhgQGll7mpFJg7hh/AMC3+zgsRERE1BsYWPrAFabAsu1EhY1bQkRENDQwsPSBS6K8IBOAUxV1KKpusHVziIiIBj0Glj7gam+HuBB3AMAfJ9nLQkRE1FMMLH1kSpQ3AAYWIiKi3sDA0kemRJsDSyWnNxMREfUQA0sfGR/mDns7GSpqtTheWmvr5hAREQ1qDCx9RKWQY6JpbyEOCxEREfUMA0sfmhrNOhYiIqLewMDSh8x1LL+frECJutHGrSEiIhq8GFj60OhAVySEeaCp2YA3N5+wdXOIiIgGLQaWPiQIAh6bMwIA8PnuMyiorLdxi4iIiAYnBpY+dkmkF6YN80azQcSK9OO2bg4REdGgxMDSDx6bbexl+XpfEYq5VD8REZHVGFj6QVyIO0YGuEIUgcPFGls3h4iIaNBhYOkn0b7OAIDcci4iR0REZC0Gln4S5eMEAMgtY2AhIiKyFgNLP4nyYQ8LERFRdzGw9JOWwFLHzRCJiIisxMDSTyK8nSAIgLpBh8q6Jls3h4iIaFBhYOknDko5gtwdALCOhYiIyFoMLP3IPCyUV1Fn45YQERENLgws/UiqY2EPCxERkVUYWPpRlK9parNpptCRYg3qtM22bBIREdGg0K3AsnLlSoSHh8Pe3h6JiYnYvXt3h+euX78eCQkJcHd3h5OTE+Lj47FmzZoOz7/77rshCAJWrFjRnaYNaK1nCn26qwB/ev13PPv9ERu3ioiIaOCzOrCsW7cOqampWL58ObKyshAXF4c5c+agrKys3fM9PT3x1FNPISMjAwcOHEBKSgpSUlLwyy+/tDn366+/xs6dOxEYGGj9kwwC5sBy5lw90n4+CgDYX1htwxYRERENDlYHltdeew133HEHUlJSMGrUKKxevRqOjo54//332z1/5syZuOaaazBy5EhERUXhoYceQmxsLLZv325xXlFRER544AF88sknsLOz697TDHDezkq42isgikBNo3EoqIibIRIREXXKqsDS1NSEzMxMJCcnt9xAJkNycjIyMjI6vV4URaSnpyMnJwfTp0+XjhsMBtxyyy14/PHHMXr06E7vo9VqodFoLL4GA0EQEGXaU0guEwAYg4umUWfLZhEREQ14VgWWiooK6PV6+Pn5WRz38/NDSUlJh9ep1Wo4OztDqVRi3rx5eOONN3D55ZdLr7/44otQKBR48MEHu9SOtLQ0uLm5SV8hISHWPIZNxQW7AwBunxoBd0djT1Ixe1mIiIguSNEfP8TFxQXZ2dmora1Feno6UlNTERkZiZkzZyIzMxP/+c9/kJWVBUEQunS/pUuXIjU1Vfpeo9EMmtDy6OzhmBXji2nR3vjjZAWq63UoOteAGH9XWzeNiIhowLIqsHh7e0Mul6O0tNTieGlpKfz9/Tu8TiaTITo6GgAQHx+Po0ePIi0tDTNnzsTvv/+OsrIyhIaGSufr9Xo8+uijWLFiBfLz89vcT6VSQaVSWdP0AcPF3g4zhvsAAALdHXC4WMM6FiIiok5YNSSkVCoxYcIEpKenS8cMBgPS09ORlJTU5fsYDAZotVoAwC233IIDBw4gOztb+goMDMTjjz/e7kyiocS8VD8DCxER0YVZPSSUmpqKJUuWICEhAZMmTcKKFStQV1eHlJQUAMDixYsRFBSEtLQ0AMZ6k4SEBERFRUGr1eKnn37CmjVrsGrVKgCAl5cXvLy8LH6GnZ0d/P39MWLEiJ4+34AW7GEKLOcYWIiIiC7E6sCycOFClJeXY9myZSgpKUF8fDw2bNggFeIWFBRAJmvpuKmrq8O9996LwsJCODg4ICYmBmvXrsXChQt77ykGqUD2sBAREXWJIIqiaOtG9JRGo4GbmxvUajVcXQdP8er+M9VYsPIP+LmqsOvvydiSU4as0+fw4GXDoJBz1wQiIhrarPn87pdZQtS+INOQUFmNFtpmPR778gAqarWI8nXGgvggG7eOiIho4OD/jbchLyclVAoZRBHYfKwMFbXGQuQfD5y1ccuIiIgGFgYWGxIEQZop9PmeM9LxLcfLUcPVb4mIiCQMLDZmLrzderxcOtbUbED60fY3kyQiIroYMbDYmLmHxVz6PG9sAADgx4McFiIiIjJjYLExc+EtAAS62eOBy4wrAm/NaX9YKPN0FV79NQc6vaHf2khERGRrDCw2Zh4SAoBpw3wwws8FUT5OaNIb8NvR0jbn/983h/HGppP4fn9xfzaTiIjIphhYbCyodWAZ7g1BEPAn07DQ+XUs9U3NyCnRAAD2nj7Xf40kIiKyMQYWGwv1cgQACAIwJcobAKTNEbefrIDe0LKu3+FiDczfZjGwEBHRRYQLx9lYkLsDnr5yFFztFfBwUgIA4kLc4aJSoLpeh4NFasSHuAMwroxrllNag5pGHVzs7WzQaiIiov7FHpYB4PapEbg+IUT63k4uw+Ro44aQv7ea7nygUC39WRSB7FYBhoiIaChjYBmgppuGhbadaB1YqgEAvi4qAEAmh4WIiOgiwcAyQE0fZgwsWQXV0DTqoK7XIb+yHgBw8yVhAIyBpanZgLe35mJXXqXN2kpERNTXGFgGqBBPR0R6O0FvELHjZCUOFFUDAEI9HXFpjC8AILugGv/66SjSfj6GpesP2rC1REREfYuBZQAzDwv9drRUql+JDXZDjL8LHJVy1Gib8eGOfADAqco6NOr0tmoqERFRn2JgGcCSR/oBAP6XWYj3tp8CAMQFu0Mhl0kzh8xEEThtGjIiIiIaahhYBrAp0V54OHkYAKCqrgmAsYcFAJIijbOIJoZ7YEyQKwAgt7zWBq0kIiLqe1yHZQATBAEPJw+Hn6s9nvr6IBzs5BgTZAwsf50WiWBPB1w20g//+PYwDhVpkMfAQkREQxQDyyBw06RQxAW7QxAAJ5XxLXNQynHNuGAAQJSvMwAgt7zOZm0kIiLqSwwsg8SoQNcOX4v0dgIA9rAQEdGQxRqWIcDcw5JXXgdRFDs5m4iIaPBhYBkCwrwcIROAGm0zymu0tm4OERFRr2NgGQJUCjmCPYy7PueW10GnN6DwHKc4ExHR0MHAMkRE+RjrWHLLa/G3/x3A1Bc345t9RTZuFRERUe9gYBkiIn2MdSzf7y/G16ag8vS3h3BW3WDLZhEREfUKBpYhIsoUWHadqgIACAJQ09iMJ786yEJcIiIa9BhYhohI05AQAChkAt5bkgClQoatx8vx6Bf7sftUFQwGBhciIhqcGFiGiNaB5fqEEFwa44cn5sYAANbvK8INb2fg7rWZ7G0hIqJBiYFliPBxVmGYrzPcHe3w4GXRAIDbp0bg0zsScd2EYNjJBfx6pBS7TUNGREREgwlXuh0iBEHA1/dNQVOzAZ5OSun45ChvTI7yhkohwye7CvD2tjwkmjZOJCIiGizYwzKEOKsUFmGltTumRUIQgE3HypBTUnPB+zTq9Kis5QJ0REQ0cDCwXCTCvZ1wxRh/AMA72/I6PE8URdz6wW5MeXET9yYiIqIBg4HlInLX9CgAwDfZRfjXT0dxrq6pzTk786qwM68KjToDvt9/tr+bSERE1C4GlotIXIg7rh0XBL1BxDvb8jD95c04VKS2OOe/v7f0vvx6pKS/m0hERNQuBpaLzKs3xOGDWydihJ8Lahqb8e+Nx6XXTpbVIP1YGQTBuPDc4WIN9yQiIqIBgYHlIiMIAmbF+GLVzeMBAJtyynC6sg4A8N72UwCAy0f6YWKYJwDgtyOlFtc36vRo1On7scVEREQMLBetSB9nzBzhA1EEPs44jQOF1fgqy7gH0Z3TIzF7tB8A4NdWgSW3vBbTXtqMGS9vRn5FnXRczxV0iYiojzGwXMRunRwOAPhizxnc9uEeNDUbMGuEDyaEeeDyUcbAsutUFarrm1CqacTi93ajvEaLUo0WSz7YjfyKOixdfwAjl23At9ncGZqIiPoOF467iE0f5oNIbyfkVdShRguMDnTF6zeNgyAICPNyQoy/C46V1ODPq3agUWdAUXUDwr0c0WwQcbqyHrNe3QLzSv8bj5RiQXyQbR+IiIiGLPawXMRkMgEpU8IBACGeDvggZSJc7O2k16+bEAwAyC2vQ1F1A7ydVfj4tkR8mDIJbg52EEXARWXMvHnldW3uT0RE1FvYw3KRW5QYBndHJRIjPeHrYm/x2l+nReLSGF/kV9bhrLoRU6K8EerlCABYf+9kZOafw9hgN1zxn99xqqIOBoMImUywxWMQEdEQx8BykZPJBFwVF9jh65E+zoj0cW5zPMrHGVE+ztDpDVDIBDTo9CjRNCLQ3aEvm0tERBcpDglRj9jJZVKvi3lYyGAQUatttmWziIhoiGEPC/VYpLcT8srrkFdRiynRXpi/cjsOFWng72qPuBA3pF0b2+GmjERERF3BHhbqMfOQUV55HU6W1eJQkQYAUKJpxC+HS/HzIe5JREREPcPAQj0W6e0EAMirqMPOvEoAwKQIT1xvnmVUxhlERETUMwws1GMtPSy1yDAFlunDvDEu1MN4vKLWZm0jIqKhgTUs1GORPsYelqLqBqnY9pJILzSbluznGi1ERNRT7GGhHvNyUsLFXgFRBKrrdXCwkyM22F0KMmfO1XPDRCIi6pFuBZaVK1ciPDwc9vb2SExMxO7duzs8d/369UhISIC7uzucnJwQHx+PNWvWSK/rdDo88cQTGDt2LJycnBAYGIjFixejuLi4O00jGxAEwWKtloRwDygVMvg4q6Qgc7qy3oYtJCKiwc7qwLJu3TqkpqZi+fLlyMrKQlxcHObMmYOysrJ2z/f09MRTTz2FjIwMHDhwACkpKUhJScEvv/wCAKivr0dWVhaefvppZGVlYf369cjJycH8+fN79mTUr6JMhbeAcTgIsAwyeeXdr2M5VKTGk18dwKEidc8aSUREg5Ygiubt67omMTEREydOxJtvvgkAMBgMCAkJwQMPPIAnn3yyS/cYP3485s2bh3/+85/tvr5nzx5MmjQJp0+fRmhoaKf302g0cHNzg1qthqura9cfhnrNm5tO4JVfjwMAvrpnMiaEGQtuU9dlY/2+Ijw+ZwTumxVt9X0LKutx9Vt/oKquCSqFDM9fM1ba44iIiAY3az6/rephaWpqQmZmJpKTk1tuIJMhOTkZGRkZnV4viiLS09ORk5OD6dOnd3ieWq2GIAhwd3e3pnlkQ+aeFEelHLHBbq2OG3tecstr0ajT48HP9uHtrbnS65mnz+HRL/bjRGlNm3vWNOpw+0d7UFXXBEelHNpmAx77cj/++3teHz8NERENNFYFloqKCuj1evj5+Vkc9/PzQ0lJSYfXqdVqODs7Q6lUYt68eXjjjTdw+eWXt3tuY2MjnnjiCdx0000dpi2tVguNRmPxRbY1JdobccFuuGNaJOzkLf9ZmYNMbnkdvttfjO/2F+OlX3JQUasFADzz/WF8lVWIBSv/wA8HWuqWDAYRD3+ejRNltfBzVeG31Bm4a0YkAGDtztP9+GRERDQQ9MssIRcXF2RnZ2PPnj14/vnnkZqaii1btrQ5T6fT4YYbboAoili1alWH90tLS4Obm5v0FRIS0oetp65wc7DDt/dPxSOXD7c4HtWqhuWLPWcAAHqDiO+yi5FTUoMDhca6lPomPe7/dB9e3HAMBoOI97afQvqxMigVMry7OAGB7g64Z0YUACC/sh7qel0/Ph0REdmaVeuweHt7Qy6Xo7S01OJ4aWkp/P39O7xOJpMhOtpYvxAfH4+jR48iLS0NM2fOlM4xh5XTp09j06ZNFxzLWrp0KVJTU6XvNRoNQ8sAFeblCEEAahqbsff0Oen4V1mFKNE0AgAui/FFtJ8z3t6ah1VbcnH0rAbbT1QAAJZdOQqxwe4AAHdHJcK8HHG6sh4HiqoxbZiPxc+q0zbDScWlhYiIhiKreliUSiUmTJiA9PR06ZjBYEB6ejqSkpK6fB+DwQCtVit9bw4rJ06cwG+//QYvL68LXq9SqeDq6mrxRQOTvZ0cwR4O0vcJYR6wkws4XKzBJ6ahnYUTQ7D0ipF4+bpYKGQCtuSUo9kg4k9j/bEo0bLo2hxezD0zZu9uy8Po5b9gTUZ+nz4PERHZhtVDQqmpqXj33Xfx0Ucf4ejRo7jnnntQV1eHlJQUAMDixYuxdOlS6fy0tDRs3LgReXl5OHr0KF599VWsWbMGN998MwBjWLnuuuuwd+9efPLJJ9Dr9SgpKUFJSQmampp66THJliK9W9Zo+eu0CFwa4wsAqGvSw8tJiVmm769PCMF7t06Ei70C0b7OSLs2FoIgWNwrNshY0Lv/TLV0bGdeJdJ+PgoAeHXjcdQ0criIiGiosbr/fOHChSgvL8eyZctQUlKC+Ph4bNiwQSrELSgogEzWkoPq6upw7733orCwEA4ODoiJicHatWuxcOFCAEBRURG+++47AMbhotY2b95sMWxEg1OUjzO2Hi+Hl5MSl8b4QRAE/HLYOKy4ID7Iokh3xnAf7HkqGYIAqBTyNvcyz0Ay97CU12jxwGf7YBABQTCutPvRjnzcf+mwfngyIiLqL1avwzIQcR2WgW37iQrc/N4uPHlFDO6eEYWmZgOmvLgJlbVa/PjgNIwM6Pp7Vqdtxth//AKDCOz++2VYuv4g0o+VYbifM26bEoEn1x+Eq70Cr94Qj5WbT0KpkOHj2ybB3s4YfqrqmuDuYAeZTOjkJxERUV+z5vObgYX6RaNOD5VCJg3x5JXXoqquCQnhnlbfa/a/t+J4aS1umxKB9/84BYVMwM8PTUOkjzPmrtiGE2WWq+o+M380lkwOx7o9BXjiq4N4Ym4M7pkZ1SvPRURE3ddnC8cRdZe9ndyiHiXSx7lbYQVoKbx9/49TAIBFiaEY5ucCuUzAo7ON06plAjAx3Lja7qotuSiubsBzPxrrXH4+dLa7j0FERDbCOaA06MQFu+F/mYUAACelHA9c1lKvMndMANbcPgkBbvYI8XTEzJe34Ky6EdevzkBNYzMA4HCxhlOgiYgGGfaw0KBj7mEBgLtmRMHbWWXx+rRhPoj2dYFKIZeGfoqqGwAYA47eIFrMMupMUXUDN14kIrIxBhYadEYGuCLMyxGRPk64fWrEBc+9ISEEvi7GQHP9hGBpCnXrRexaq2nU4dnvj2BNRj6q6prw39/zMOuVLbjqze3YmVfZuw9CRERdxqJbGpSa9QboRbHdqc/n25FbgZ8OnsXjs2PwTXYRln93GNOH++Dj2ya1Ofe1jcfxevqJdu8zLtQd6++ZDEEQcKaqHgFu9lDIL5z5RVHEsm8PQ9usxwvXxnJ2EhFRKyy6pSFPIZd1KawAwOQobzx39Vi4OdphQpixEDfr9DnoDZZZXRRFaQNGf1d7AMbdp5deEQN7Oxn2FVTjt6NlWPHbcUx7aTNu/2gvOsv7mafPYc3O0/hibyFyy2sveC4REXWMVYd0UYnxd4GTUo5abTNySmowKrAl0R89W4O88jooFTJsTJ2OUk0jXB3s4Otij+oGHVZtyUXqumzUaI3Fu1uPl+PHg2dxZWwg9AYRtdpmuDnYWfy8Na12lj5QqMYwP5d22/XmphMoqm7Ec1ePgbxVL0xTswH3fpIJD0clls8fDWcWChPRRYo9LHRRUchlGBdq7GXJPF2FUk0jzlTVA4DUuzJrhA9c7O0Q7esCXxdjT8td0yPhYq+Qwoq5p+b5H4/iUJEa817/HQnPbcTJVmvAVNRq8dPBlinUBwqr223Tmap6vPLrcXy2uwDZZyxrazJPn8NvR8vwZWYhrn3rD+RX1F3w+f698TgWvLkd5TXaC55HRDTYMLDQRSfBtD7LaxuP45K0dMx6ZQt+PngW35sCy1VxgW2ucXdU4ul5o+DppMQ/F4zGJ39NRJC7A86qG3HlG9txrKQGOr2I9KMtO5l/sfcMdHoRdnJjj8mBDmYafZVVKP0563S1xWtZBS0B5nhpLa5dtQPV9e3vsZVXXos3Np3A/kI1Pttd0IXfBBHR4MHAQhediaYF687V6yCKQLNBxL2fZuFMVQMc7OTS5oznu2FiCDL/Lxm3JIXD3k6Op68cKb3m7mgcCtp9qgoAoDeI+GSnMTTcOT0SAHCkWAOd3oBabTPe2ZaLs+oGGAyiZWApaNvDAgB3zYhEkLsDquqakJHb/mylNzefhLks53+ZhTAYBn09PRGRhIGFLjpJkV548NJoPHhpNNIfnYFrxwXBXDt72UhfOCo7rhNpvVrvnNH+WHblKPzfvJF4/9aJAIDd+VXQG0RsP1mBouoGuDnY4f5Zw+Bir4C22YDjpTVYsfE4/vXTMSz67y78drQUZ6oapHtmFZyTCnkNBlEKMH8aE4Dpw30AAPsL2/bU5FfU4dtsYw+RUiFDQVU9dudX9eC3REQ0sDCw0EVHJhOQOnsEUmePQJSPM16+Pg43TQqBUi7DLZeEdfk+giDgtqkR+Ou0SMQGucFZpUBNYzOOlWjwbXYRAGB+XCAclHJpl+m9+efwP1OPSl55He77NAsAsCA+EHKZgFKNFsXqRuPrFXWortfB3k6GUYGuiJN2qq5u05a3tpyE3iBi1ggfXBMfBAD4cm9hm/OIiAYrBha66MllAtKujcXhZ+cgMdKrW/dQyGVSIe624xX49bCxlmV+vLEeZmyQOwBg5eaTqK7XwctJCXs7GXR6Y2/KLZeEYWSAcQZRlmkYyPy/scHusJPLMNYUWA4WqS2Ge8prtFifZQxID1w2DNcnBAMAfjp4FrWmImEiosGOgYXIxK6TReA6MynCWBvz9rZc1GqbEehmjwmmGUnm3pEy0+ydmy8Jw6vXxwMARvi5YEKYB8abzjUPA5nrV8xBaLifC1QKGWoam5Ff2TJbaMPhEjQbRMQFu2F8qAcmhHkg0tsJDTo9fjQVErfnTFU9ln17CNtPVFzwuQrP1bdZs4aIqL8xsBD1kkRTYKmu1wEwzjYyr2xr7h0BjDtJ3zAxBPNiA7Dxken49I5ECILQKrBUAwAyTcHFHHrs5DKMNq0bc6BVHYs5lFwZa+zNEQQB1yeEAADW7ixod3G7mkYdUj7cg48zTuPm93bhwc/2oaK27VToDYdKMPXFzfjnD0e68yshIuo1DCxEvSQ22B0qRctfKfNwEAAEuTvAy0kJAJgx3AdB7g4AgGF+LvAybd5oDixHitUo1TRKa7qMN/WwmH8G0BJYymoascs0M+mKsf7SeQsnhkCpkOFgkbrNzCODQUTqF/txsqwWLvYKyATgu/3FuHtNZptn+mTXael/z6ob2rxORNRfGFiIeolSIZNCR5SPE0YFtKyiKwgCZo7whSAAKVPa37AxxNMB3s5K6PQi/vLuTgBApLcTPE1BB4BUvGsuvN1wqASiaNznKNjDUTrP00mJq02B6YM/8i1+ztvb8rDxSCmUchk+vm0Svr53CgTBuCFkiangFzCGoT9OGoeLdHoR728/1W67RVHEzrzKdteHOVBYjRvfyUBOSU271xIRdRUDC1Ev+lNsAABjEW3rKdAA8NzVY7DlsZnS9OTzCYKAS0xFv7nlxhqVmSMs14Qx97AcKlajWW/ADweMK+nOGxvQ5n5LJocDMIYacxBpajbgnW25AIDl80dhXKgH4kLcEWe679bjZdL1P+w/C4MIabuBT3cVQG0a7mrt9fSTuPGdnfjb/w60ee2dbXnYmVcl9dR0plGnb3cIq1Gnxye7TlsEKlvbfKyMQYyoHzGwEPWimxND8fvfZklhoTUHpRxhXk4XvH7ZVaPw/DVj8OZfxuGreybjqXkjLV6P9HaCs0qBRp0B7/5+CntMa638qZ3AMjrQDZPCPdFsELHWtKfR5pwynKvXwcdFhYWmOhcAmDnCGKK2Hi+Xjn2731gb80jyMMT4u6CuSY81O/MtfsbvJ8qxIv24dO+aRstAs89Uj5NXfuEtBQDgZFkNEv+Vjvs/29fmted+PIKnvj6EV3/N6fQ+7TmrbsALPx9DVV37qwRb66eDZ5Hy4R7c9uGeTjfAJKLewcBC1IsEQUCIp2Ob3pWu8nWxx6LEMFwZG4gJYR4WGyECxjVkxgQZh5pe3HAMoggkhHkg0FQTc75bp4QDAD7akY/yGi2+yjSuzXLtuCAoWs2KmmHq9fn9RAWa9QbkV9Rh/5lqyGUC5sUG4u4ZUQCA97afgrrBGErOqhvw0OfZ0qJ7Or2ILTktgadM04iiamPdS14Xdqp+cUMO1A06bDxSiqZmg3T8SLEGn+4yrhp8vLR7PRr/+O4wVm/NxevpJ7p1fWvqBh2Wf3cYAFBU3YAC015URNS3GFiIBplrxwVDKZchxt8Ft04Ox8vXx3V47tzR/ogNdkONthlPfX0Qm44Zh3z+PCHY4rzYYHd4ONqhprEZ+85US9sFTIn2ho+LClfGBiDa1xnn6nV4Pf0EdHoD7v90H6rqmjA60BW3mnqUfj3SspfSvjPV0p+L1Y2ob+p4TZi9+VXYaLq2ybQiMGCsj3n2h8PSlgOnuxEOKmu1SD9qfO5tJ8o7ObtzL244ZrG55N78cxc4e+ir1Tazl4n6BQML0SBzw8QQ5Dw3Fxseno5/zB+NCO+Oh5lkMgHLrxoNwBgmmg0iYoPdMNzPxeI8uUzAtGHGXpZXf83BW1uMdS5/Hm9cNVchl+HpK0cBMPbWPLwuG5mnz8HFXoG3Fo2XZkRtPlYGbbMeQMtwkNmpDnaaFkURL244ZnHsoGmjyJ8PlWBnXpU0+6q6XtemjkanN+BIsabDD83v9hej2ZR48srrpF6f7tiTXyX19kw0baK59/TFG1g2HyvDmOW/4MMd+bZuCl0EGFiIBiFrhpwmhHngmnFB0vd/Hh/c7nnmYaGdecb9kK4dF4T5rXaunjHcB5fF+KLZIOJHU7HvK9fHIczLCfHB7vBxUaFW24ydeca6mn3nTac2B5b3t5/Cf3/Pk45vySnHnvxzUClkUjvN07bfM81Mumt6JLxN079PV1kGnye+OoA/vf47Pu1gh2rzFgXm0bXt3exl0TbrsXT9QQDAjRND8Ndpxk0tsy7iwLLRtDt56w08ifoKAwvRReCJuTFwUSngolJYhJDWWs9emjbMGy9eF9smGD01byTs5MZjd0yLwJzRxrVfZDIBl4/yAwD8ergEzXqDFDriQtwBtPRuPPvDETz341EUnjMO72w4VAIAuGlSqHSPg0XVqKprktaQuSkxFOFexmnbpytbhoX25FdJ2xK8uemk1LtjdrhYjSNnNVDKZVicFA4A2NbJyr4AUFGrhU5vsDi2aksuTpbVwttZhaVXjJSmsB8vq5Hqei42x02zpA4Xa6Ser//75iBueW8XGnX6C11KZDUGFqKLgL+bPX5+eBp+emgaPFqt69Kaj4sK98+KxpWxAVh184R2tyqI9HHGvxfG48HLhuFvc2MsXpttChsbDpVgR24lGnR6uKgU0vG88lrsONkSFswr+ppDyZRob4wNMq4zk1NSg18PG9eYGRXgigA3B4SaAou5yFVvEPEPU/ErAJxVN0rhxeyrTOP3l4/yw1VxxplUf5ysuOBWAwcL1Zj64ibMfHmLtN5NTkkN3tpsHCb7x/xRcHO0g4+LCuFejhBFtFmc72IgiiJypFojYNepSuRX1GHtzgL8fqJCCqJEvUVh6wYQUf9ovbBcRx6bM6LTc8xbAJxvcpQ3IrydcKqiDnevNa6aGxfijigfZwDG3adlrWY9ZZ0+hxnDfXDCtKLvuFB3eDkp4eFoh3P1Ory9zThsdNlI41o0YZ7GWp3Tpn2Uvtx7BoeLNXCxV2BxUhhWbs7FW1tO4voJwdIMqF+PGD80rxkXhLhgd7ioFKiu1+FQkVrq+Tnfaxtz0KgzoKi6AdetysDECA9pmOzSGF+LNW/Gh3kgv7IeWafPYdZ5a+Z0RhTFbs8mGwhKNVrUNLYUUmfkVUrvJWAcJrq61VAkUU+xh4WIeoVSIcN/lyTAzcEO9U3G4YBxoe6I9DEGjbzyOmTkVkrnZ54+J9W5hHk5wttZBUEQMNa0iJ255uXSGF/pHADIr6yHKIrSFOWHLhuG+2cNg5eTEmeqGvBttnH9mOr6JhSeMxbYTozwhEIuQ1KUcWG+3zuoY8k+U43NOeWmImRvNOkN+ONkJfQGEZMiPPGva8ZahIyEMOP+Ua1nClXWavHs90ewM6+yzf3NduZVYvw/N+JfPx298C91AMs5b4p5Rm6ltJAhAGw/WTGgFvqjwY+BhYh6TZSPM1bfPEGqcxkf5oEwL0fIBOP017PqRqn49chZjbT0v7keBABig1o2ivRyUkqr8JoDS0FlPU6U1aJY3QiVQoabLwmDg1KO26Yatzz4Yu8ZAMa6CvN15tV6p5nqdLafbL+O5T+/GRfBu2ZcED6+bRKev2YMHrpsGNIfnYEv7kqCv5u9xfkJpplC2WeqodMbIIoiHv1yP97/4xQWv7+73dCSW16Lu9Zk4ly9Du9vPzVoP9RPmAKLeZfyYyU1OHpWA4VMwMgAV4gi8PW+ogvdgsgqDCxE1KuSorzwwa2T8Njs4ZgxzAcqhdxiOGpiuCcC3OyhN4hYt8cYLsaHukuvt97ZelaMrzSMZF4luETTKK3ZkhjpBXs7OQBIBcDZZ6rR1GzAIdPUaPMO10DLjtoHCtVt6lh25VVKvSv3z4qGIAhYlBiGRy4fLg1rnS/axxkejnZo0Onx6Bf78XHGaWnxvKZmA+74aK/UDsDY+5LywR6pSLfZIOLjjHwAwJacMqT9dBSaxs4LeEVRxPHSGhguUIvTXcXVDdh+XmHynvwqlGosg5V5W4LJUV4Y5tvy+5kS7Y0lSWEAjMNCXKOFegsDCxH1uqnDvHH/pcOksGEeFgKMtS4TTDtQa0w1EONa97C0CiyXxbTUhXg42sFFZSy7+8w0hXn6MG/p9SgfJ3g42kHbbMChYrXUwzI60K3VOc5wUspR36SXdsMGgF8Ol+DWD/YAMK4CHH6BtW1ak8kEpF07FgqZgO/2F0sr4P5t7ggkRniiRtuMWz/YjVMVdWjU6XHHx3tRUFWPUE9HvHDtWADAJ7sKsPlYGf760V68vS0PD3y674JFwQDwVVYRZv97G574qu3+TT318OfZuPm9Xfg229g78vPBs7h+dQbuPG83b/PifsP9XKShNgC4MjYAf4oNgEohw8myWvxyuBREvYGBhYj6XKR3y/8DnxztJQUWAHBUyhHj37KQnb+rPSaGeyDU01EawgGMa8+YZwqZa1OmtgosgiAgIdxcU1KFQ8XGno0xrYaY5DJB6sHZb1qJ94s9Z3D32kw06PSYNswbT181yqpnmzsmAO8sngClaXG7yVFeuHt6FN5dkoBRAa6oqG3Czf/dhQc+24esgmq4Odjh/Vsn4vqEEIR7OULdoMNtH+2RFrfberwcL/x84dqWdXuMge3LzEJsPlYGvUHEG+knsOzbQ1KvS622GS9uOCY9Z3vO1TXhXz8dxV7TnlSNOr004+mlDTmo1TbjBdOifvvPVEsFzwaDKBXYDvdzQZJp0047uYDZo/zham+HBabFBO9em4ml6w+iTtvxSsdEXcHAQkR9ztzD4mAnR1ywu0VgiQ12s9jXSBAEfHFXErY8NhPOKsuJjOGtNo/0cVFhxHkr9ppXn92SUy4V7bYeEgJa1oXJLqyGwSDi5V9zIIrAXxJD8cGtE+Fqb2f1810a44fP7rgEd0yLwH9uHAeZTICrvR0+um0Swr0cUVTdgI1HSmEnF7D65gmI9nWGXCZIdTeiaPw9vGLaZuHd309J+z4BgKZRh7Ia45BMiboRe1oV+f7964O4Z20mXt14HB9nnJY2xPxsVwFWbcnFje/stJhOblZeo8VN7+7EO9vy8PS3xp6hQ0VqKTgVVTfgpnd2Wqx788vhEum1+iY9lHIZwr0cMSvGF38a64/H54yAm6Px9/eP+aOloaHPdhdghak+iKi7GFiIqM9NifaGUiHDgvhAKBUyjAxwhb2d8Z+f1gW3ZoIgWEyBNjP3sADGxe3OnxZs7mHZkVsJUTT21phXyDWLNxXx7j9TjezCapTXaOGiUuAfV422CE7WmhDmgafmjYKPS8vP83FRYc3tifBzNR574dpYi+GT6yYEI9jDAcEeDnh3cQKumxCMBy6NBmAMIkeKNdhXcA5TXtiEy17ZitOVdfjxoHEmTmywG0I9HXFW3Wixh1OGqdD3j1xjSGnQ6ZHy4R5sNu0jBQBnqupx4zsZOGaqQzl6VoOquiZkm3pjXO2NQdG8RcI4U42ReW0Vc/1KpI8TFHIZ7O3keGvRBNw5PUr6GY5KBZ5ZMEba0uHo2e5tXElkxnVYiKjPRXg74cDy2VCYQoidXIap0T747WiptCVAV4R5tgSW6cPaXjcm0A0qhQxa027P5p2tWzP3sBwrqcF3pinQM2N8pSGd3hbi6YiNqTNQpmlEtK9lj5CjUoFNj86ECBEqhbF4+JHk4ThQqMbW4+W44+O90DToUGMaTvnHd4dRbSrYvXZcEIb7ueDm93bBWaXAnNH++DKzEBm5lbh3pgG7Txl7WmKD3XCgUI2UD/cgeaQvwr2c8PHO02hqNiDQzR4ymYDCcw3YmVcpbVh55/RIfL//LHJKazDM1xkr/zIek1/YhKyCapRqGnG8zBg+RvhbPk97Ik31QNUNTT3/ZdJFjT0sRNQv7O3kFj0Yr1wfi2/vm4LESK8LXGWpdQ/LlGjvNq8rFTLEt1oQrnXBrVmAmz18XFTQG0Rp/6E5o/263IbucLW3axNWzJQKmRRWAGMh74qF8Qhyd0BRdQNqtM2ID3GHUi7D5pxy7CuohiAAfxobgMnR3vj1kRnY8vgs3DvL2DOzr6Aau09Vob5JD08nJb64KwmLEkMhE4Dfjpbhv9tPoanZgKRIL6y7KwnJI43P/sfJCmSbVh8eH+aBl6+PxZRoL7x8fRwC3R2kYbzv9xdL09HP30SzPeYhour6i3P7Auo9DCxEZBPujsoOV5vtSFywO4b7OWNBfKDF0EtrE03DQkDb+hXAONxkXtulqdkApVyGmVauUtvXPJyUWHXzeHg7KzFrhA8+vSMRd82IlF6fFO4JX1fjmjDRvs7wdFIi3MsR/q72aNIb8MYm46J6SaZp389fMxYbU2dgflwgxoW6470lCfj0jkSEeDpKwe+XwyUoqm6AIACxwe6IDXbHJ3+9RAqAc03Txp/78Sj+OGkcdpoc1XnYdDetgXP+LttE1uKQEBENGk4qBX59ZMYFzzEv5gZYzhBqLT7EDb+ZdhqeEu3Vprh3IIgNdseuvydDbhpGu29WNL7JLsKZqgZc1c4GloIgICnKC1/vK8Iu03BQ63qZKB9nvH7TuDbXTYrwhEwAKmqNQzbDfV3a/X3MGe2P500r83o7K5F2bazFdPSOuDsa966q0TZDpze0u0cVUVcMvL+lREQ9MDHcE0HuDvB2ViLgvJVpzVr37JgXnBuI5K0Kj+3t5PgoZRK2Hi/HjRND2j3fHFjM2hs2O5+bgx3GBrlhv2l37fgOer1CvRzx5BUxKFE34oFLo+Hl3H4P1/nMBbwAoGnQdfk6ovMxsBDRkOKkUiD90RmQy4QONxeMDXaHUiGDKIpIHtW39Su9KdLHGZEdrLoLQFoPBTDW6oR7db7hJQAkRXm3BJZWqw6f7+4ZUR2+1hGFXAYXewVqGptRzcBCPcC+OSIacuzt5BccenBzsMPHt03C2tsT20x7HsxCPB0R7OEAwNjb0tXdoFvXonTUw9IT7iy8pV7AwEJEF6VLIr2smqE0WMw31bfMb6fOpSPmYbRIH6cuzfyxlruDsY5FzanN1AMcEiIiGkJSLx+OJZPD4efafv1OexyUcvz6yHQIgmXdTG9hDwv1BgYWIqIhRCGXWRVWzJz6cKaUmwMDC/Uch4SIiKhPST0sDQws1H0MLERE1KekGpZ61rBQ9zGwEBFRn2IPC/UGBhYiIupTrGGh3sDAQkREfcq8PD97WKgnuhVYVq5cifDwcNjb2yMxMRG7d+/u8Nz169cjISEB7u7ucHJyQnx8PNasWWNxjiiKWLZsGQICAuDg4IDk5GScOHGiO00jIqIBxjwkxBoW6gmrA8u6deuQmpqK5cuXIysrC3FxcZgzZw7KysraPd/T0xNPPfUUMjIycODAAaSkpCAlJQW//PKLdM5LL72E119/HatXr8auXbvg5OSEOXPmoLGxsftPRkREA4IHa1ioFwiiKIrWXJCYmIiJEyfizTffBAAYDAaEhITggQcewJNPPtmle4wfPx7z5s3DP//5T4iiiMDAQDz66KN47LHHAABqtRp+fn748MMPceONN3Z6P41GAzc3N6jVari6tt1OnoiIbKe8RouJz/8GQQByn/8TZH2wOB0NTtZ8flvVw9LU1ITMzEwkJye33EAmQ3JyMjIyMjq9XhRFpKenIycnB9OnTwcAnDp1CiUlJRb3dHNzQ2JiYpfuSUREA5u56FYUgZrGZhu3hgYrq5Y2rKiogF6vh5+f5e6mfn5+OHbsWIfXqdVqBAUFQavVQi6X46233sLll18OACgpKZHucf49za+dT6vVQqvVSt9rNBprHoOIiPqRUiGDk1KOuiY9qhua4GYaIiKyRr/MEnJxcUF2djb27NmD559/HqmpqdiyZUu375eWlgY3NzfpKyQkpPcaS0REvU6aKcSpzdRNVgUWb29vyOVylJaWWhwvLS2Fv79/xz9EJkN0dDTi4+Px6KOP4rrrrkNaWhoASNdZc8+lS5dCrVZLX2fOnLHmMYiIqJ9Ja7Gw8Ja6yarAolQqMWHCBKSnp0vHDAYD0tPTkZSU1OX7GAwGaUgnIiIC/v7+FvfUaDTYtWtXh/dUqVRwdXW1+CIiooGrZcdmTm2m7rF6e87U1FQsWbIECQkJmDRpElasWIG6ujqkpKQAABYvXoygoCCpByUtLQ0JCQmIioqCVqvFTz/9hDVr1mDVqlUAAEEQ8PDDD+O5557DsGHDEBERgaeffhqBgYG4+uqre+9JiYjIZqS1WNjDQt1kdWBZuHAhysvLsWzZMpSUlCA+Ph4bNmyQimYLCgogk7V03NTV1eHee+9FYWEhHBwcEBMTg7Vr12LhwoXSOX/7299QV1eHO++8E9XV1Zg6dSo2bNgAe3vrt0gnIqKBx82BNSzUM1avwzIQcR0WIqKB7cUNx7BqSy5umxKBZVeNsnVzaIDos3VYiIiIusNdKrplDQt1DwMLERH1uZaiWw4JUfcwsBARUZ9rqWFhDwt1DwMLERH1OXdugEg9xMBCRER9zhxYztWxh4W6h4GFiIj6XKinIxQyAefqdThTVW/r5tAgxMBCRER9zlGpQFyIOwAgI6/Sto2hQYmBhYiI+kVSpBcAYGcuAwtZj4GFiIj6RVKUMbBk5FViCKxZSv2MgYWIiPrF+FAP2MkFnFU34nQl61gGqma9AUXVDbZuRhsMLERE1C8clHKMC/EAAOxkHcuA9Xr6CUx5YRM255TZuikWGFiIiKjfXBLpCYCFtwPZL4dLAQC78qps3BJLDCxERNRvLjHXseSyjmUgqmnU4XhZDQDgzLmBNWzHwEJERP1mfKgHlAoZymq0OFSksXVz6DwHCtUw58jCcwOrjoWBhYiI+o29nRyzR/kBAP7109FOe1lOVdThya8O4HRlXbd+nrpBh805ZTAY+q83p6FJj9s+3IN/fHcY6kG2FcG+gnPSn4vYw0JERBezJ+bGQKmQISOvUqqXaI/BIOKRddn4fM8ZLF1/0OqfU9Oow/WrdyDlgz344eDZnjTZKjtPVWLTsTJ8uCMfl726Fb8cLum3n91T+wqqpT9X1DahvqnZdo05DwMLERH1qxBPR9w5LRIA8PxPR1DT2H4vxHf7i5F9phoAsCO3EhlWLDjXrDfg/k/34XhpLQBYdW1PnSpv6Q2qqNXi/k+zcFZtHF4pqm7Akvd3Y8fJin5rT1eJooh9pt+3WdEAGhZiYCEion53z8wo+LmqcKaqAeP/uRHXr96BjzPyUac1/j/6+qZmvPDzMQCAn6sKAPDvjcfbHUI6V9eENzedQHGrtUPSfj6GrcfLpe+zz/sg7kv5puGr26dGIMbfBTq9KE3j/mhHPrYeL8eLG471W3vMRFHEHycr8HFGPvTtDJEVVNWjqq4JSrkM0b7OAAZWHQsDCxER9TsnlQL/XhiPYA8H6PQi9uSfw7JvDyMpLR1//WgvbnpnJ0o0jQhyd8AXdyVBqZBhd34Vtp/XM2EwiLj/syy88utx/OunowAAdb0OH+7IBwAsv2oUAOB4aU27wxvNegP+l1mIpesPoEzT2OZ1nd7QYQ9QR05VGAPLCH8XTB/uAwDYfco4RXhHrrH9+wvVFgGrr2UVnMO1q3Zg0X93Ydm3h7E+q7DNOebhoNFBroj0dgIwsGYKMbAQEZFNTI7yxu9/m4Vtj8/CsitHIdzLEZrGZvx2tBT7C9UAgL//aSTCvJywKDEUAPD21jyLe3ywIx9/nDT2XmzJKYe2WY8tx8ugN4gY4eeClCkR8HNVQW8QpVlJ2mY99uZXYdWWXFz66lY89uV+fLb7DF75Ncfi3qIo4rYP92Di87/hpGmqb1eYA0uEtxMmhRvXndl1qgrV9U04XNwyM+rXXqptEUURzXpDh6/r9Abc/uEei/qU9nqczMfGhXgg2MMRwMDqYVHYugFERHTxEgQBoV6OuG1qBG6dHI7fT1ag6FwDFDIB/m72Ug/FosRQfPBHPvaeroJOb4CdXIackhppaEUuE1CrbcbOvCpsPGIs5E0e5QsAiAt2x69HSrH/TDWG+Tpj7n+2oVSjldrgYq9ATWMzvs0uxlN/GgU3RzsAxhqa308Ye0TWZJzGMwvGtPsMjTo9KuuaEOTugEadXlrWPsLbCXYyGQQByCuvw/cHzqL1iNaGwyW4dUrEBX8/pZpG5FfUQQQQ7OEgBQmzZr0BV735h7Gn6J7JcHOwa3OPY2drcK5eB1d7BR65fDie+f4IDhWp25yXZZohFB/qjspa4++nkD0sRERElmQyATOG++AviaG4YWKIFFYAINLbGS72CjTqDMgpMfZ2vPDzUTQ1GzBrhA9uSAgGAPx88Cy25hhrV5JHGqdPx4e6AzD2IHy2pwClGi1c7BWYM9oPzy4YjV1/vwwx/i7QNhvwZeYZAECdtlkaYgKAr/cVoVGnb7fdS9cfxLQXNyH7TDXOVNVDFAEXlQJeTkq4OdphhJ8LAGDV5pMAgEtjjEFq96kqVNU1dfj7qKprQvKrW7HwnZ248Z2dmP7SZnybXWRxztGzNTh6VoMTZbV49vsj7d4n+4w5iHhg1gjjzz5aUgNdq16ZvPJaHChUQxCAieEtPSxnqozh65VfcrDpWOkFe3L6GgMLERENeDKZgLhgdwDG4NGo02OHaebPk1eMxOxR/gCALzMLUaNthrezSjo/PsT4v1kF57A24zQAYPlVo/H2LQlYnBQOR6UCi5PCAQBrd56GwSDirS0nUarRIsTTAUHuDtA0NmPDobZDOM16A349XAKDCPx86CzyTMNB4d5OEAQBAHBJpHF132K1sUbmhoQQjApwhUEEfjvS8bTu7/cXo0bbDGeVAsEeDjCIwGNf7se2VsXEe/Jbls//KqtQ6l1qzTwUFB/ijlBPR7ioFGhqNuBkWa10zpqdxt/LpSN8EeDmgGAPBwDGHpbc8lq8ufkk7vg4E1X1HQesvsbAQkREg4I5eGSfqUZWwTlomw3wdVFhuJ8zkqK84KiUS7NfLovxhUxmDAxjg9wgCMBZdSOK1Y3wclLiytgAi3tfPS4QLioF8ivrsWDlH1i5ORcA8H/zRuGGhBAAwOd7Ctq06VhJDeqajD0vO3Mrkd+qfsVsUoSn9GdBAJIivTB3jDFgXWiNFnNh7KOzh2Pb47MwPy4QOr2Iu9dm4uhZYy3M3tPGwGKeSbV0/cE2xcNSbUqoO2QyAaMCXQFAGhaq0zbjf5nGn7V4cjgASIHlXL0Oa0whb/owb/i62HfY3r7GwEJERINC68Cyw1RoOznKC4IgwN5OjhmthpCSTavpAoCLvR2GmabpAsBNk0Jhbye3uLejUoE/TzAOKx0sUkMmALdODsfsUX64PiEYggDszKvCV5mF2H6iQpp+vbdVD8fBIjUOmIqFw1sFlonhLYFlTKAb3BztMGe0MbD8fqICtdq2s5dyy2uxv1ANuUzAVXGBkMkEvHJ9HKZEe6G+SY+3tuRCFEXsPmUc7nn1+nhE+zqjolaLhe/slGYgVdc3Sb0+8aYep7FBbgBaAss32UWoaWxGhLcTpkV7S78zd1Mtz2e7jUHN/PuxFQYWIiIaFMy1KLnltfj1iLFnYrLpAxYALjeFFJVChqmtjgOQhofkMgGLLglt9/73zozCpTG+uG1KBLY+Pgv/mD8agiAg0N1BCkOPfrkfN7+3C3evzQQA7DndspS9QYQ0JBPZKrD4uKgQ6WP8frJp88fhfs6I8HZCk96ALTllbdrydZaxVmXGcB94Oxt7T5QKGZ6YGwPAOMPocLEGFbVaKOUyJIR74P0lExHs4YBTFXW4fnUGzlTVS70rEd5O8HBSAgDGmANLsQaiKOLjHcYelFsuCZN6pYCWXhZtswEu9gqpJshWGFiIiGhQ8HZWIdjDAaIIaQXbKa2CyRVjAjAvNgBPzI2Bg9KyB2XGCGPgWBAXiAA3h3bv7+tqj/dvnYhlV41CiKflbJwn5sbg0hhfTAjzgEww9oycLKuVeljMC601mYpSW/ewAMbemiB3B1xvKg4WBAGzRxsDwPm1MQaDiK/3GQPLNeOCLF4bG+SGKB8naJsN+OcPxiLbuBA32NvJEerliC/uSkKktxOKqhvw8LpsZLWqXzEbE2QcEjpSrME32UXIKa2Bg528TQ9KSKsZSVfGBrbplepvDCxERDRotP7gDfdyRJB7S/hwUMqx8i/jcdvUtlOF540NwFf3TMa/rh3brZ87MsAV7986EV/dMxmXxhiDxsu/HEOpRgs7uYA7p0danB/hZRlYFieF448nL0W0r4t0bK5pWGjzsTI06vTQNOrw9tZcLPrvLhRVN8BFpZB6jcwEQZBCzC7TYnQJrYacAt0dsOaviXBSypF5+hze334KgLF+RWqbtzMclXI06PTSHk33zoxqMyXa3MMCANdNsAxOtsDAQkREg0brwDL5vGGfCxEEARPCPHqll+AvicYiXPPGjWOC3DBzREv9jKdpOnNn4oLd4eeqQl2THpuPleGW/+5C2s/HkGFaxv8viW1rbQBgQbxleJgY7mHxfZC7Ax6fMwIApPqY1r83uUzAqABjL0ujzoCRAa64e2ZUm58TauplivB2wvhQjzav9zcGFiIiGjRaf/BOiep6YOlNM4b7IsCtZbbMxHBP+LrYS8NC4V6OHV1qQSYTpOLbR7/cj/2Farg72mHZlaOw8ZHpePKKmHavC/F0lFbQFQRgQqhnm3NuSQpHnOl3pVTIEOPvavG6uY5FLhPw8nWxsJO3jQPz44JwVVwgnrt6jDRF25YYWIiIaNAYE+QGV3sFHOzkSDIVsPY3uUzAwokh0vcJYcbeB3NBbaSPc7vXtcccWOqb9JAJwJs3GYe0hvm5XDAkXDve2MsS4+/abm+OXCbghWvHws3BDleM8YdSYflx/6exAbC3k+HR2cOl8HI+N0c7vHHTOIs6IVvi0vxERDRo2NvJ8eXdk6HTG+BpmvViCzckhOCNTSchE4AJpsBy36xoaHUG3DkjspOrW0yK8ISXkxKVdU34+59GYuqwroWD6yYEQ9OoQ1Jkx+ePDHDF7qcug7Kd3pNJEZ448sxci1lBA50gtrdX9yCj0Wjg5uYGtVoNV1fXzi8gIiLqoT9OVsAgipg2zKfzky9g/5lqnK6qx1WxAQNi6KU/WfP5zR4WIiKibuitoZK4EHep3oQ6xhoWIiIiGvAYWIiIiGjAY2AhIiKiAY+BhYiIiAY8BhYiIiIa8BhYiIiIaMBjYCEiIqIBj4GFiIiIBjwGFiIiIhrwGFiIiIhowGNgISIiogGPgYWIiIgGPAYWIiIiGvCGxG7NoigCMG5TTURERIOD+XPb/Dl+IUMisNTU1AAAQkJCbNwSIiIislZNTQ3c3NwueI4gdiXWDHAGgwHFxcVwcXGBIAi9em+NRoOQkBCcOXMGrq6uvXrvgWKoP+NQfz6AzzgUDPXnA/iMQ0FvP58oiqipqUFgYCBksgtXqQyJHhaZTIbg4OA+/Rmurq5D8j++1ob6Mw715wP4jEPBUH8+gM84FPTm83XWs2LGolsiIiIa8BhYiIiIaMBjYOmESqXC8uXLoVKpbN2UPjPUn3GoPx/AZxwKhvrzAXzGocCWzzckim6JiIhoaGMPCxEREQ14DCxEREQ04DGwEBER0YDHwEJEREQDHgNLJ1auXInw8HDY29sjMTERu3fvtnWTuiUtLQ0TJ06Ei4sLfH19cfXVVyMnJ8finJkzZ0IQBIuvu+++20Yttt4//vGPNu2PiYmRXm9sbMR9990HLy8vODs7489//jNKS0tt2GLrhIeHt3k+QRBw3333ARic79+2bdtw1VVXITAwEIIg4JtvvrF4XRRFLFu2DAEBAXBwcEBycjJOnDhhcU5VVRUWLVoEV1dXuLu74/bbb0dtbW0/PsWFXegZdTodnnjiCYwdOxZOTk4IDAzE4sWLUVxcbHGP9t77F154oZ+fpH2dvYe33nprm7bPnTvX4pzB/B4CaPfvpSAIePnll6VzBvJ72JXPh678+1lQUIB58+bB0dERvr6+ePzxx9Hc3Nxr7WRguYB169YhNTUVy5cvR1ZWFuLi4jBnzhyUlZXZumlW27p1K+677z7s3LkTGzduhE6nw+zZs1FXV2dx3h133IGzZ89KXy+99JKNWtw9o0ePtmj/9u3bpdceeeQRfP/99/jyyy+xdetWFBcX49prr7Vha62zZ88ei2fbuHEjAOD666+Xzhls719dXR3i4uKwcuXKdl9/6aWX8Prrr2P16tXYtWsXnJycMGfOHDQ2NkrnLFq0CIcPH8bGjRvxww8/YNu2bbjzzjv76xE6daFnrK+vR1ZWFp5++mlkZWVh/fr1yMnJwfz589uc++yzz1q8tw888EB/NL9Tnb2HADB37lyLtn/22WcWrw/m9xCAxbOdPXsW77//PgRBwJ///GeL8wbqe9iVz4fO/v3U6/WYN28empqasGPHDnz00Uf48MMPsWzZst5rqEgdmjRpknjfffdJ3+v1ejEwMFBMS0uzYat6R1lZmQhA3Lp1q3RsxowZ4kMPPWS7RvXQ8uXLxbi4uHZfq66uFu3s7MQvv/xSOnb06FERgJiRkdFPLexdDz30kBgVFSUaDAZRFAf/+wdA/Prrr6XvDQaD6O/vL7788svSserqalGlUomfffaZKIqieOTIERGAuGfPHumcn3/+WRQEQSwqKuq3tnfV+c/Ynt27d4sAxNOnT0vHwsLCxH//+99927he0N7zLVmyRFywYEGH1wzF93DBggXipZdeanFssLyHotj286Er/37+9NNPokwmE0tKSqRzVq1aJbq6uoparbZX2sUelg40NTUhMzMTycnJ0jGZTIbk5GRkZGTYsGW9Q61WAwA8PT0tjn/yySfw9vbGmDFjsHTpUtTX19uied124sQJBAYGIjIyEosWLUJBQQEAIDMzEzqdzuL9jImJQWho6KB8P5uamrB27VrcdtttFht+Dvb3r7VTp06hpKTE4j1zc3NDYmKi9J5lZGTA3d0dCQkJ0jnJycmQyWTYtWtXv7e5N6jVagiCAHd3d4vjL7zwAry8vDBu3Di8/PLLvdrV3te2bNkCX19fjBgxAvfccw8qKyul14bae1haWooff/wRt99+e5vXBst7eP7nQ1f+/czIyMDYsWPh5+cnnTNnzhxoNBocPny4V9o1JDY/7AsVFRXQ6/UWv3wA8PPzw7Fjx2zUqt5hMBjw8MMPY8qUKRgzZox0/C9/+QvCwsIQGBiIAwcO4IknnkBOTg7Wr19vw9Z2XWJiIj788EOMGDECZ8+exTPPPINp06bh0KFDKCkpgVKpbPMh4Ofnh5KSEts0uAe++eYbVFdX49Zbb5WODfb373zm96W9v4Pm10pKSuDr62vxukKhgKen56B8XxsbG/HEE0/gpptusthY7sEHH8T48ePh6emJHTt2YOnSpTh79ixee+01G7a2a+bOnYtrr70WERERyM3Nxd///ndcccUVyMjIgFwuH3Lv4UcffQQXF5c2w82D5T1s7/OhK/9+lpSUtPt31fxab2BguQjdd999OHTokEV9BwCLMeOxY8ciICAAl112GXJzcxEVFdXfzbTaFVdcIf05NjYWiYmJCAsLwxdffAEHBwcbtqz3vffee7jiiisQGBgoHRvs79/FTqfT4YYbboAoili1apXFa6mpqdKfY2NjoVQqcddddyEtLW3ALwF/4403Sn8eO3YsYmNjERUVhS1btuCyyy6zYcv6xvvvv49FixbB3t7e4vhgeQ87+nwYCDgk1AFvb2/I5fI2VdClpaXw9/e3Uat67v7778cPP/yAzZs3Izg4+ILnJiYmAgBOnjzZH03rde7u7hg+fDhOnjwJf39/NDU1obq62uKcwfh+nj59Gr/99hv++te/XvC8wf7+md+XC/0d9Pf3b1ME39zcjKqqqkH1vprDyunTp7Fx40aL3pX2JCYmorm5Gfn5+f3TwF4UGRkJb29v6b/LofIeAsDvv/+OnJycTv9uAgPzPezo86Er/376+/u3+3fV/FpvYGDpgFKpxIQJE5Ceni4dMxgMSE9PR1JSkg1b1j2iKOL+++/H119/jU2bNiEiIqLTa7KzswEAAQEBfdy6vlFbW4vc3FwEBARgwoQJsLOzs3g/c3JyUFBQMOjezw8++AC+vr6YN2/eBc8b7O9fREQE/P39Ld4zjUaDXbt2Se9ZUlISqqurkZmZKZ2zadMmGAwGKbANdOawcuLECfz222/w8vLq9Jrs7GzIZLI2QymDQWFhISorK6X/LofCe2j23nvvYcKECYiLi+v03IH0Hnb2+dCVfz+TkpJw8OBBi/BpDt+jRo3qtYZSBz7//HNRpVKJH374oXjkyBHxzjvvFN3d3S2qoAeLe+65R3RzcxO3bNkinj17Vvqqr68XRVEUT548KT777LPi3r17xVOnTonffvutGBkZKU6fPt3GLe+6Rx99VNyyZYt46tQp8Y8//hCTk5NFb29vsaysTBRFUbz77rvF0NBQcdOmTeLevXvFpKQkMSkpycatto5erxdDQ0PFJ554wuL4YH3/ampqxH379on79u0TAYivvfaauG/fPmmGzAsvvCC6u7uL3377rXjgwAFxwYIFYkREhNjQ0CDdY+7cueK4cePEXbt2idu3bxeHDRsm3nTTTbZ6pDYu9IxNTU3i/PnzxeDgYDE7O9vi76Z5ZsWOHTvEf//732J2draYm5srrl27VvTx8REXL15s4yczutDz1dTUiI899piYkZEhnjp1Svztt9/E8ePHi8OGDRMbGxulewzm99BMrVaLjo6O4qpVq9pcP9Dfw84+H0Sx838/m5ubxTFjxoizZ88Ws7OzxQ0bNog+Pj7i0qVLe62dDCydeOONN8TQ0FBRqVSKkyZNEnfu3GnrJnULgHa/PvjgA1EURbGgoECcPn266OnpKapUKjE6Olp8/PHHRbVabduGW2HhwoViQECAqFQqxaCgIHHhwoXiyZMnpdcbGhrEe++9V/Tw8BAdHR3Fa665Rjx79qwNW2y9X375RQQg5uTkWBwfrO/f5s2b2/3vcsmSJaIoGqc2P/3006Kfn5+oUqnEyy67rM2zV1ZWijfddJPo7Owsurq6iikpKWJNTY0NnqZ9F3rGU6dOdfh3c/PmzaIoimJmZqaYmJgourm5ifb29uLIkSPFf/3rXxYf+LZ0oeerr68XZ8+eLfr4+Ih2dnZiWFiYeMcdd7T5P32D+T00e/vtt0UHBwexurq6zfUD/T3s7PNBFLv272d+fr54xRVXiA4ODqK3t7f46KOPijqdrtfaKZgaS0RERDRgsYaFiIiIBjwGFiIiIhrwGFiIiIhowGNgISIiogGPgYWIiIgGPAYWIiIiGvAYWIiIiGjAY2AhIiKiAY+BhYiIiAY8BhYiIiIa8BhYiIiIaMBjYCEiIqIB7/8BxepgOGNlUO4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1)); # Mean on every Row -> 200 means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "44cedf7b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into evaluation mode\n",
    "for layer in model.layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "b575808a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train\t1.9186382293701172\n",
      "val\t2.0330464839935303\n"
     ]
    }
   ],
   "source": [
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split] # That's a switch!!\n",
    "    \n",
    "    logits = model(x)\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(f'{split}\\t{loss.item()}')\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfa37920",
   "metadata": {},
   "source": [
    "We actually see a **good improvement** concerning the loss here.<br>\n",
    "And this is only due to making the batch size larger.<br>\n",
    "Nothing else, no structural changes yet.\n",
    "\n",
    "Performance log:\n",
    "|                                      |train|val  |\n",
    "| ------------------------------------ | --- | --- |\n",
    "| original (3 characters, 12K params)  |2.058|2.105|\n",
    "| context: 3 -> 8 (22K params)         |1.918|2.033|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "ab022272",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mahriah.\n",
      "catheoj.\n",
      "demanui.\n",
      "briuja.\n",
      "alim.\n",
      "devon.\n",
      "alunya.\n",
      "duriyan.\n",
      "amer.\n",
      "catriana.\n",
      "aryshail.\n",
      "arpod.\n",
      "nalynn.\n",
      "elynn.\n",
      "ozep.\n",
      "lavade.\n",
      "rudiel.\n",
      "beitum.\n",
      "micbelly.\n",
      "karlene.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the 22K model\n",
    "for _ in range(20):\n",
    "    \n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all ...\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      logits = model(torch.tensor([context]))\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ae1935c",
   "metadata": {},
   "source": [
    "This looks good.<br>\n",
    "But let's make the model more scalable through the above discussed 'dilated causal convolutional layers' approach."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d783d16",
   "metadata": {},
   "source": [
    "## Dilated Causal Convolutional Layers\n",
    "\n",
    "Let's build up some intuition on how this multilayer-input construct could be built.<br>\n",
    "Below we can see the shape of the tensors throughout the network's layers.\n",
    "\n",
    "**This is just for visualization:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "e441233c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 8])\n",
      "tensor([[ 0,  0,  0,  0,  0,  0,  0,  8],\n",
      "        [ 0,  0,  0,  0,  5, 12, 12, 19],\n",
      "        [ 0,  0,  0,  0,  0,  0,  0, 12],\n",
      "        [ 0,  0,  0, 13,  1,  5,  2, 18]])\n"
     ]
    }
   ],
   "source": [
    "# Let's look at what a batch of just 4 examples does\n",
    "ix = torch.randint(0, Xtr.shape[0], (4,))\n",
    "\n",
    "# Take these 4 random rows from the training set\n",
    "Xb, Yb = Xtr[ix], Ytr[ix]\n",
    "logits = model(Xb) # Run them through the model\n",
    "print(Xb.shape)    # Total input set dimensions, 4x8; 4 is batch size, 8 is block size\n",
    "print(Xb)          # Let's see the actual input batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "9db027e4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 8, 10])\n"
     ]
    }
   ],
   "source": [
    "# Each character is represented by a 10-dimensional vector that we want to learn (we chose that)\n",
    "print(model.layers[0].out.shape) # 4x8x10 -> 4 is batch size, 8 is block size, 10 is embedding size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "41000422",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 80])\n"
     ]
    }
   ],
   "source": [
    "# Output of the Flatten Layer (compresses 8 and 10 together into one big concatenation per input)\n",
    "print(model.layers[1].out.shape) # 4x80 -> 4 is batch size, 80 is inferred from 8x10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "c7dbd943",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 200])\n",
      "torch.Size([4, 200])\n"
     ]
    }
   ],
   "source": [
    "# Output of the Linear Layer (200 expected outputs created per input)\n",
    "print(model.layers[2].out.shape) # 4x200 -> 4 is batch size, 200 is number of neurons in the hidden layer\n",
    "\n",
    "# Digression/Refresher: How come [4, 80] turns into [4, 200]? See below for explanation\n",
    "print((torch.randn(4, 80) @ torch.randn(80, 200) + torch.randn(200)).shape)\n",
    "# We obtain 4 (batch size) activation sets of 200 activations each, just as planned\n",
    "# The + torch.randn(200) is the bias"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4061b163",
   "metadata": {},
   "source": [
    "Interestingly, this last step can work no matter what dimensionality the input has.<br>\n",
    "It always 'attaches' to the last dimension, ignoring $n \\geq 0$ prior listed matrix dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "e981da25",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 5, 6, 200])\n"
     ]
    }
   ],
   "source": [
    "# Just a sidenote, nothing structural\n",
    "print((torch.randn(4, 5, 6, 80) @ torch.randn(80, 200) + torch.randn(200)).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc454c0c",
   "metadata": {},
   "source": [
    "We can use this property to our advantage.<br>\n",
    "We have `block_size=8` many characters:<br>\n",
    "`1 2 3 4 5 6 7 8`\n",
    "\n",
    "We embed each character into a $10$-dimensional vector. (Just as we always did.)\n",
    "\n",
    "We now want to apply step-wise grouping over the embedded (now $10$-dimensional) characters:<br>\n",
    "`(1 2) (3 4) (5 6) (7 8)`\n",
    "\n",
    "We also want to do this in a way as to process the above emerging $4$ subgroups efficiently, in parallel.\n",
    "\n",
    "Let's modify our previously applied matrix multiplication to be able to account for that.<br>\n",
    "We only want $2$ characters to be fused in our first layer.\n",
    "\n",
    "These are the $4$ times (batch size) $4$ groups of $2$ characters,<br>\n",
    "`(1 2) (3 4) (5 6) (7 8)`, where each character itself is $10$ dimensional<br>\n",
    "(per group: $2$ characters $\\times 10$ embeddings $= 20$):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "073353e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4, 200])\n"
     ]
    }
   ],
   "source": [
    "# n_embd = 10    # Character embedding dimensionality\n",
    "# We want this behavior for our new linear layer, it's just a draft for now:\n",
    "print((torch.randn(4, 4, 20) @ torch.randn(20, 200) + torch.randn(200)).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5be5835d",
   "metadata": {},
   "source": [
    "The `Flatten()` layer should therefore be adapted<br>\n",
    "to not output $[4, 80]$, but $[4, 4, 20]$ (again $4 \\times 4$ groups of $2$ characters, each of $10$ dimensions)<br>\n",
    "so that it correctly feeds into the new linear layer which already accounts for the grouped input approach now.\n",
    "\n",
    "We currently have an input of dim $(4, 8, 10)$ (batch size, block size, embedding size).<br>\n",
    "**Goal:** Get this to be $(4, 4, 20)$ with $20$ being $2$ consecutive $10$-dim vectors concatenated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "b3ad4c7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 80])\n"
     ]
    }
   ],
   "source": [
    "e = torch.randn(4, 8, 10) # Mocked input to the flatten layer\n",
    "# Currently, Flatten() creates just this reshaped version:\n",
    "print(e.view(4, -1).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14ef5d2e",
   "metadata": {},
   "source": [
    "To solve the re-arrangement task, we can utilize a concept from the python list API to reshape exactly how we want it right now.<br>\n",
    "This is the concept we want to use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "735bab7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 2, 4, 6, 8]\n",
      "[1, 3, 5, 7, 9]\n"
     ]
    }
   ],
   "source": [
    "print(list(range(10))[::2])\n",
    "print(list(range(10))[1::2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b726d26f",
   "metadata": {},
   "source": [
    "From every input in the batch (batch size $4$), we now pick only every second $10$-dimensional character embedding vector.<br>\n",
    "Given `e` of shape $(4, 8, 10)$, we therefore have to plug out a shape $(4, 4, 10)$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "5671822f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4, 10])\n",
      "torch.Size([4, 4, 10])\n"
     ]
    }
   ],
   "source": [
    "print(e[:, ::2, :].shape)  # Even character representations (zeroth, second etc.)\n",
    "print(e[:, 1::2, :].shape) # Odd character representations (first, third etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8146e57c",
   "metadata": {},
   "source": [
    "If we now concatenate these two tensors along the third dimension (zero indexed in Python, so there is a $2$ in the code),<br>\n",
    "we re-combine every even character embedding with just its following odd character embedding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "d9aae00b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4, 20])\n"
     ]
    }
   ],
   "source": [
    "explicit = torch.cat([e[:, ::2, :], e[:, 1::2, :]], dim=2)\n",
    "print(explicit.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a597d82",
   "metadata": {},
   "source": [
    "A remarkable move.<br><br>\n",
    "We have a $[4, 8, 10]$ input. We splice this one tensor into two, both of size $[4, 4, 10]$.<br>\n",
    "The first tensor contains all the even number embeddings, and the second tensor contains all the odd number embeddings of the groups of embeddings.<br>\n",
    "We then concatenate the tensors on their third dimension.<br>\n",
    "We thus always paste the $10$-dimensional embedding of a second character to the back of the embedding of a first character.\n",
    "\n",
    "In fact, we just did that operation very explicitly.<br>\n",
    "PyTorch's `.view()` has our back in this case, we don't need to be as verbose.<br>\n",
    "But its always good to see how this is done.\n",
    "\n",
    "The simplified code utilizing `.view()` for the same task:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "3f1144ff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(True)\n"
     ]
    }
   ],
   "source": [
    "print((e.view(4, 4, 20) == explicit).all())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "040a21ef",
   "metadata": {},
   "source": [
    "This can be generalized.<br>\n",
    "If a flattening layer was to receive a tensor of shape $[B, T, C]$ (batch size, characters, embedding channels),<br>\n",
    "and it received an $n$ to group the characters into $n$ groups,<br>\n",
    "we can generalize the reshaping to $[B, T//n, C \\times n]$ (batch size, characters divided by $n$, embedding channels times $n$)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6c54e78",
   "metadata": {},
   "source": [
    "Let's now propagate our findings to the model architecture and specifically to the implementation of `Flatten()`.<br>\n",
    "These classes remain untouched:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "d7ad3727",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mimicks torch.nn.linear\n",
    "\n",
    "class Linear:\n",
    "  \n",
    "  def __init__(self, fan_in, fan_out, bias=True):\n",
    "    self.weight = torch.randn((fan_in, fan_out)) / fan_in**0.5  # Kaiming init\n",
    "    self.bias = torch.zeros(fan_out) if bias else None\n",
    "  \n",
    "  # Performs a forward pass if called (syntax: l = Linear(x))\n",
    "  def __call__(self, x):\n",
    "    self.out = x @ self.weight\n",
    "    if self.bias is not None:\n",
    "      self.out += self.bias\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.weight] + ([] if self.bias is None else [self.bias])\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.batchnorm1d\n",
    "class BatchNorm1d:\n",
    "  \n",
    "  def __init__(self, dim, eps=1e-5, momentum=0.1):\n",
    "    self.eps = eps\n",
    "    self.momentum = momentum\n",
    "    # Batchnorm1d behaves differently during training and production\n",
    "    self.training = True\n",
    "    # Parameters (trained with backprop)\n",
    "    self.gamma = torch.ones(dim)\n",
    "    self.beta = torch.zeros(dim)\n",
    "    # Buffers (trained with a running 'momentum update' outside backprop)\n",
    "    self.running_mean = torch.zeros(dim)\n",
    "    self.running_var = torch.ones(dim)\n",
    "  \n",
    "  def __call__(self, x):\n",
    "    # Calculate the forward pass\n",
    "    if self.training:\n",
    "      xmean = x.mean(0, keepdim=True) # Batchnorm mean\n",
    "      xvar = x.var(0, keepdim=True) # Batchnorm variance\n",
    "    else:\n",
    "      xmean = self.running_mean\n",
    "      xvar = self.running_var\n",
    "    xhat = (x - xmean) / torch.sqrt(xvar + self.eps) # Normalize to unit variance\n",
    "    self.out = self.gamma * xhat + self.beta\n",
    "    # Update the buffers\n",
    "    if self.training:\n",
    "      with torch.no_grad():\n",
    "        self.running_mean = (1 - self.momentum) * self.running_mean + self.momentum * xmean\n",
    "        self.running_var = (1 - self.momentum) * self.running_var + self.momentum * xvar\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.gamma, self.beta]\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.tanh\n",
    "class Tanh:\n",
    "  def __call__(self, x):\n",
    "    self.out = torch.tanh(x)\n",
    "    return self.out\n",
    "\n",
    "  def parameters(self):\n",
    "    return []\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.embedding, does the indexing operation\n",
    "class Embedding:\n",
    "  \n",
    "  def __init__(self, num_embeddings, embedding_dim):\n",
    "    # What was previously C (lookup table) is now the weight in this layer\n",
    "    self.weight = torch.randn((num_embeddings, embedding_dim))\n",
    "    \n",
    "  def __call__(self, IX):\n",
    "    self.out = self.weight[IX]\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.weight]\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.Sequential\n",
    "class Sequential:\n",
    "  \n",
    "  def __init__(self, layers):\n",
    "    self.layers = layers      # This is basically the same list structure as before\n",
    "  \n",
    "  def __call__(self, x):      # Simply pass input x through every layer in self.layers\n",
    "    for layer in self.layers:\n",
    "      x = layer(x)\n",
    "    self.out = x\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    # get parameters of all layers, stretch them out into one list\n",
    "    return [p for layer in self.layers for p in layer.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb8512f1",
   "metadata": {},
   "source": [
    "This one is new:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "ec9e1bcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# NOT ANYMORE mimicking torch.nn.flatten\n",
    "class FlattenConsecutive:\n",
    "    \n",
    "  def __init__(self, n):  # n is the number of consecutive elements to be concatenated\n",
    "    self.n = n\n",
    "    \n",
    "  def __call__(self, x):\n",
    "    B, T, C = x.shape # 4, 8, 10 in example before\n",
    "    x = x.view(B, T//self.n, C*self.n) # Flattening in groups, now 4, 4, 20 for n=2 characters\n",
    "    if x.shape[1] == 1:\n",
    "        x = x.squeeze(1) # Takes out the dimension 1, as it only has value 1 (irrelevant)\n",
    "    self.out = x\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "cfa75a7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42); # For reproducibility"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51c9f782",
   "metadata": {},
   "source": [
    "Let's now set up the thinned down and deepened model architecture:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "f69ab3ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22397\n"
     ]
    }
   ],
   "source": [
    "n_embd   = 10 # Dimensionality of the character embedding vectors\n",
    "n_hidden = 68 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd),   # Previously C = torch.randn((vocab_size, n_embd))\n",
    "    # The new layer at work now (block_size was 8 before)\n",
    "    FlattenConsecutive(2), Linear(n_embd * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "3063259a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 8])\n",
      "tensor([[ 0,  0,  0, 13,  9, 12,  9,  1],\n",
      "        [ 0,  0,  0,  1, 22,  1, 12, 25],\n",
      "        [ 0,  0,  0,  0,  0,  0,  0, 10],\n",
      "        [ 0,  0,  0,  0, 11,  8, 25, 14]])\n"
     ]
    }
   ],
   "source": [
    "ix = torch.randint(0, Xtr.shape[0], (4,)) # Looking again at a random batch of 4 examples\n",
    "Xb, Yb = Xtr[ix], Ytr[ix]\n",
    "logits = model(Xb)\n",
    "print(Xb.shape)\n",
    "print(Xb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "2281e378",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embedding : (4, 8, 10)\n",
      "FlattenConsecutive : (4, 4, 20)\n",
      "Linear : (4, 4, 68)\n",
      "BatchNorm1d : (4, 4, 68)\n",
      "Tanh : (4, 4, 68)\n",
      "FlattenConsecutive : (4, 2, 136)\n",
      "Linear : (4, 2, 68)\n",
      "BatchNorm1d : (4, 2, 68)\n",
      "Tanh : (4, 2, 68)\n",
      "FlattenConsecutive : (4, 136)\n",
      "Linear : (4, 68)\n",
      "BatchNorm1d : (4, 68)\n",
      "Tanh : (4, 68)\n",
      "Linear : (4, 27)\n"
     ]
    }
   ],
   "source": [
    "# This is neat, it shows the shape of the output of each layer\n",
    "for layer in model.layers:\n",
    "    print(layer.__class__.__name__, ':', tuple(layer.out.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b35ad78",
   "metadata": {},
   "source": [
    "We now have built what was set out in the paper:<br>\n",
    "\n",
    "![](./images/convolutions_1.png)\n",
    "\n",
    "But not quite. The total receptive field size of the paper's model is $16$. We have $8$.<br>\n",
    "So it's one squashing layer less, basically.<br>\n",
    "With the above architecture, we actually shrunk `n_hidden` from $200$ down to $68$.<br>\n",
    "We now have a very different NN architecture, but roughly the same amount of parameters in total.<br>\n",
    "We can now see whether our architectural changes optimized the use of the parameter amount or not.\n",
    "\n",
    "**Let's train!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "ed16a7c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.5555\n",
      "  10000/ 200000: 2.2055\n",
      "  20000/ 200000: 2.4019\n",
      "  30000/ 200000: 2.1561\n",
      "  40000/ 200000: 2.1377\n",
      "  50000/ 200000: 2.2740\n",
      "  60000/ 200000: 2.0626\n",
      "  70000/ 200000: 1.8016\n",
      "  80000/ 200000: 2.0158\n",
      "  90000/ 200000: 2.0814\n",
      " 100000/ 200000: 1.6752\n",
      " 110000/ 200000: 2.3915\n",
      " 120000/ 200000: 2.3702\n",
      " 130000/ 200000: 2.1049\n",
      " 140000/ 200000: 2.2882\n",
      " 150000/ 200000: 1.7862\n",
      " 160000/ 200000: 2.0824\n",
      " 170000/ 200000: 2.2584\n",
      " 180000/ 200000: 2.0024\n",
      " 190000/ 200000: 1.7854\n"
     ]
    }
   ],
   "source": [
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "033a8351",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfPElEQVR4nO3deXhU1f0/8Pedmcxkn+x7yEoIawIEYkABNWzSuitaKxjrCq5Rq2jVtlZjbb+Wqgj9WbeCVWzFpYqghEWWsIYYwhJICITsCySTdSaZub8/7sxNhiRkAklmEt6v58nTZObeO+dmJPPuOZ9zjiCKoggiIiIiB6awdwOIiIiIesPAQkRERA6PgYWIiIgcHgMLEREROTwGFiIiInJ4DCxERETk8BhYiIiIyOExsBAREZHDU9m7Af3BZDKhrKwMHh4eEATB3s0hIiIiG4iiiIaGBoSEhEChuHAfyrAILGVlZQgPD7d3M4iIiOginDlzBmFhYRc8ZlgEFg8PDwDSDXt6etq5NURERGQLnU6H8PBw+XP8QoZFYLEMA3l6ejKwEBERDTG2lHOw6JaIiIgcHgMLEREROTwGFiIiInJ4DCxERETk8BhYiIiIyOExsBAREZHDY2AhIiIih8fAQkRERA6PgYWIiIgcHgMLEREROTwGFiIiInJ4DCxERETk8IbF5ocDRd9uxBsb8tFmNOGFBaOhUSnt3SQiIqLLEntYevH+jiL8K+s0WttM9m4KERHRZYuB5QKcFB2/njYjAwsREZG9MLBcgEIhwEkpAGBgISIisicGll44KaVfkaGdgYWIiMheGFh6YQks7GEhIiKyHwaWXqhVlh4W0c4tISIiunwxsPRCbRkSYg8LERGR3TCw9IJFt0RERPbHwNILy5BQG4tuiYiI7IaBpReWols9e1iIiIjshoGlF/IsIfawEBER2Q0DSy/kISEjZwkRERHZCwNLLzpmCRnt3BIiIqLLFwNLL+RZQlyHhYiIyG4YWHohLxzHolsiIiK7YWDpBfcSIiIisj8Gll6ouZcQERGR3TGw9KJjlhADCxERkb0wsPSCQ0JERET2x8DSCzmwcB0WIiIiu2Fg6QWHhIiIiOyPgaUXau7WTEREZHcMLL1gDQsREZH9MbD0ggvHERER2R8DSy/k3ZpZdEtERGQ3DCy9cLL0sLRz80MiIiJ7YWDphYY9LERERHbHwNILJxVnCREREdkbA0svLDUses4SIiIishsGll5w80MiIiL7Y2DphRNXuiUiIrI7BpZeqLlwHBERkd0xsPSiYy8hzhIiIiKyFwaWXnBpfiIiIvtjYOmFk3nzQy7NT0REZD8XFVhWrFiByMhIODs7Izk5GXv37u3x2HXr1iEpKQleXl5wc3NDYmIiVq9ebXVMY2MjHnnkEYSFhcHFxQVjxozBqlWrLqZp/U7DolsiIiK7U/X1hLVr1yI9PR2rVq1CcnIyli9fjrlz5yI/Px8BAQFdjvfx8cELL7yA+Ph4qNVqfPvtt0hLS0NAQADmzp0LAEhPT8fmzZuxZs0aREZG4ocffsCSJUsQEhKC66+//tLv8hLIewlxSIiIiMhu+tzD8uabb+L+++9HWlqa3BPi6uqKDz74oNvjZ82ahZtuugmjR49GTEwMHn/8cUyYMAE7duyQj9m1axcWL16MWbNmITIyEg888AASEhIu2HMzWOQaFvawEBER2U2fAovBYMCBAweQmpracQGFAqmpqcjKyur1fFEUkZmZifz8fMyYMUN+fNq0afjmm29QWloKURSxZcsWHD9+HHPmzOn2Onq9HjqdzuproHSeJSSKnClERERkD30aEqqpqYHRaERgYKDV44GBgTh27FiP59XX1yM0NBR6vR5KpRLvvvsuZs+eLT//9ttv44EHHkBYWBhUKhUUCgXee+89q1DTWUZGBv7whz/0pekXzdLDAkihRW3eW4iIiIgGT59rWC6Gh4cHcnJy0NjYiMzMTKSnpyM6OhqzZs0CIAWW3bt345tvvkFERAR++uknLF26FCEhIVa9ORbLli1Denq6/LNOp0N4ePiAtF3dKbAYjCa5x4WIiIgGT58Ci5+fH5RKJSorK60er6ysRFBQUI/nKRQKxMbGAgASExNx9OhRZGRkYNasWWhpacHzzz+PL7/8EgsWLAAATJgwATk5OfjrX//abWDRaDTQaDR9afpF6xxQ2tpNwOC8LBEREXXSp+4CtVqNyZMnIzMzU37MZDIhMzMTKSkpNl/HZDJBr9cDANra2tDW1gaFwropSqUSJpP9C12VCgEK8ygQpzYTERHZR5+HhNLT07F48WIkJSVh6tSpWL58OZqampCWlgYAWLRoEUJDQ5GRkQFAqjdJSkpCTEwM9Ho91q9fj9WrV2PlypUAAE9PT8ycORPPPPMMXFxcEBERgW3btuFf//oX3nzzzX681YvnpFRA326CnlObiYiI7KLPgWXhwoWorq7GSy+9hIqKCiQmJmLDhg1yIW5xcbFVb0lTUxOWLFmCkpISuLi4ID4+HmvWrMHChQvlYz777DMsW7YMd911F86ePYuIiAi8+uqreOihh/rhFi+dWiUFFvawEBER2YcgDoO5ujqdDlqtFvX19fD09Oz3609+5UfUNhmw8YkZGBXk0e/XJyIiuhz15fObU15swA0QiYiI7IuBxQaWmUJc7ZaIiMg+GFhsYNmxmTUsRERE9sHAYgMOCREREdkXA4sNNPJ+QgwsRERE9sDAYgNLDwsDCxERkX0wsNjAEli4cBwREZF9MLDYQC0PCQ35JWuIiIiGJAYWG3BIiIiIyL4YWGygVknTmjlLiIiIyD4YWGygZg8LERGRXTGw2EBeh4WBhYiIyC4YWGzgpOLCcURERPbEwGIDDgkRERHZFwOLDTitmYiIyL4YWGxg2fyQQ0JERET2wcBiA7VSCYBFt0RERPbCwGIDJ/M6LG3sYSEiIrILBhYbqDmtmYiIyK4YWGzQUXTLwEJERGQPDCw2kBeOa+csISIiIntgYLEBh4SIiIjsi4HFBpaVbll0S0REZB8MLDZQm9dhYQ0LERGRfTCw2MBSdMshISIiIvtgYLFBR9EtAwsREZE9MLDYwImbHxIREdkVA4sNOCRERERkXwwsNrBMa27jOixERER2wcBiAw4JERER2RcDiw3kISEW3RIREdkFA4sNnMzrsLCGhYiIyD4YWGyg5pAQERGRXTGw2MAyJGQSgXaGFiIiokHHwGIDS9EtALQZOVOIiIhosDGw2KBzYGEdCxER0eBjYLGBpegW4EwhIiIie2BgsYEgCCy8JSIisiMGFhtZelkYWIiIiAYfA4uNuHgcERGR/TCw2MhSeMuiWyIiosHHwGKjjv2EOK2ZiIhosDGw2EjDISEiIiK7YWCxEXdsJiIish8GFhs5qbgBIhERkb0wsNjIsg4Lh4SIiIgGHwOLjdw0KgBAk77dzi0hIiK6/DCw2MjbVQ0AONtksHNLiIiILj8MLDbycWNgISIishcGFhtZAsu5ZgYWIiKiwcbAYiNvc2CpbWRgISIiGmwMLDbycWUPCxERkb0wsNiINSxERET2w8BiIwYWIiIi+2FgsZElsNS1tMFo4gaIREREg+miAsuKFSsQGRkJZ2dnJCcnY+/evT0eu27dOiQlJcHLywtubm5ITEzE6tWruxx39OhRXH/99dBqtXBzc8OUKVNQXFx8Mc0bEF6uTgAAUQTqW9rs3BoiIqLLS58Dy9q1a5Geno6XX34Z2dnZSEhIwNy5c1FVVdXt8T4+PnjhhReQlZWF3NxcpKWlIS0tDRs3bpSPKSwsxJVXXon4+Hhs3boVubm5ePHFF+Hs7Hzxd9bPnJQKeDpLq92ebdLbuTVERESXF0EUxT6NbyQnJ2PKlCl45513AAAmkwnh4eF49NFH8dxzz9l0jUmTJmHBggV45ZVXAAB33HEHnJycuu15sYVOp4NWq0V9fT08PT0v6hq2uPqvW1FU04TPH0zB1CifAXsdIiKiy0FfPr/71MNiMBhw4MABpKamdlxAoUBqaiqysrJ6PV8URWRmZiI/Px8zZswAIAWe7777DnFxcZg7dy4CAgKQnJyMr776qsfr6PV66HQ6q6/B4G0eFmLhLRER0eDqU2CpqamB0WhEYGCg1eOBgYGoqKjo8bz6+nq4u7tDrVZjwYIFePvttzF79mwAQFVVFRobG/H6669j3rx5+OGHH3DTTTfh5ptvxrZt27q9XkZGBrRarfwVHh7el9u4aJwpREREZB+qwXgRDw8P5OTkoLGxEZmZmUhPT0d0dDRmzZoFk8kEALjhhhvw5JNPAgASExOxa9curFq1CjNnzuxyvWXLliE9PV3+WafTDUpo4fL8RERE9tGnwOLn5welUonKykqrxysrKxEUFNTjeQqFArGxsQCkMHL06FFkZGRg1qxZ8PPzg0qlwpgxY6zOGT16NHbs2NHt9TQaDTQaTV+a3i+82cNCRERkF30aElKr1Zg8eTIyMzPlx0wmEzIzM5GSkmLzdUwmE/R6vXzNKVOmID8/3+qY48ePIyIioi/NG3CW5fkZWIiIiAZXn4eE0tPTsXjxYiQlJWHq1KlYvnw5mpqakJaWBgBYtGgRQkNDkZGRAUCqN0lKSkJMTAz0ej3Wr1+P1atXY+XKlfI1n3nmGSxcuBAzZszA1VdfjQ0bNuB///sftm7d2j932U9Yw0JERGQffQ4sCxcuRHV1NV566SVUVFQgMTERGzZskAtxi4uLoVB0dNw0NTVhyZIlKCkpgYuLC+Lj47FmzRosXLhQPuamm27CqlWrkJGRgcceewyjRo3CF198gSuvvLIfbrH/sIaFiIjIPvq8DosjGqx1WLKLz+Hmd3ch1MsFO5+7ZsBeh4iI6HIwYOuwXO582cNCRERkFwwsfWCZJdRsMKK1zWjn1hAREV0+GFj6wEOjgpNSAMDCWyIiosHEwNIHgiDAm1ObiYiIBh0DSx9xphAREdHgY2DpI67FQkRENPgYWPqIy/MTERENPgaWPrIsz3+OgYWIiGjQMLD0ka+7FFgqdK12bgkREdHlg4Glj+ICPQAAR8p1dm4JERHR5YOBpY/Gh2oBAPkVDdC3c/E4IiKiwcDA0kdh3i7QujihzSjieEWjvZtDRER0WWBg6SNBEORelkOl9XZuDRER0eWBgeUijGNgISIiGlQMLBfB0sOSx8BCREQ0KBhYLoIlsByr0LHwloiIaBAwsFyEcB8W3hIREQ0mBpaLIAgCxoV6AmAdCxER0WBgYLlILLwlIiIaPAwsF8lSx7K3qBZGk2jn1hAREQ1vDCwXaVqMH9w1KhRWN+GTPaft3RwiIqJhjYHlIvm4qfHsvFEAgDc25KO8vsXOLSIiIhq+GFguwV3JEZg0wguN+na8+NVhezeHiIho2GJguQQKhYDXb5kAlULApqOVOHO22d5NIiIiGpYYWC5RXKAH4gI9AADHKhrs3BoiIqLhiYGlH8QFugMAjlcysBAREQ0EBpZ+MNLcw3KCgYWIiGhAMLD0g5EBUg/LiSou009ERDQQGFj6gaWGpaCqkYvIERERDQAGln4Q7uMKjUoBfbuJM4WIiIgGAANLP1AqBMT4c1iIiIhooDCw9BPOFCIiIho4DCz9hDOFiIiIBg4DSz/hTCEiIqKBw8DSTzrPFCqqacIr3x7BsQqdnVtFREQ0PKjs3YDhovNMoXnLf4K+3YQKXStW/GqSvZtGREQ05LGHpZ90nimkbzcBAErPtdizSURERMMGA0s/uiY+AAoBuG58EACgvJ6BhYiIqD9wSKgfPT13FB6eFYMmQzvWH6pAVYMebUYTnJTMhURERJeCn6T9zE2jgp+bBk5KAaIIVDfo7d0kIiKiIY+BZQAoFAICPZ0BcFiIiIioPzCwDJBgrSWwtNq5JUREREMfA8sACdK6AAAqGFiIiIguGQPLAAkx97CU1TGwEBERXSoGlgESZA4sFTrWsBAREV0qBpYBwhoWIiKi/sPAMkCCzTUs5RwSIiIiumQMLAPE0sNS1dCKdqPJzq0hIiIa2hhYBoivuwYqhQCTCFQ3cvE4IiKiS8HAMkCUnRaP40whIiKiS8PAMoAsw0Jci4WIiOjSMLAMoCAtl+cnIiLqDwwsAyjEyzxTiD0sREREl4SBZQAFeXJIiIiIqD8wsAygYA4JERER9YuLCiwrVqxAZGQknJ2dkZycjL179/Z47Lp165CUlAQvLy+4ubkhMTERq1ev7vH4hx56CIIgYPny5RfTNIcSbB4SKq1jYCEiIroUfQ4sa9euRXp6Ol5++WVkZ2cjISEBc+fORVVVVbfH+/j44IUXXkBWVhZyc3ORlpaGtLQ0bNy4scuxX375JXbv3o2QkJC+34kDig1wh0ohoFKnR3Fts72bQ0RENGT1ObC8+eabuP/++5GWloYxY8Zg1apVcHV1xQcffNDt8bNmzcJNN92E0aNHIyYmBo8//jgmTJiAHTt2WB1XWlqKRx99FJ988gmcnJwu7m4cjLtGhUkjvAEAP52otnNriIiIhq4+BRaDwYADBw4gNTW14wIKBVJTU5GVldXr+aIoIjMzE/n5+ZgxY4b8uMlkwt13341nnnkGY8eO7fU6er0eOp3O6stRzYjzAwBsZ2AhIiK6aH0KLDU1NTAajQgMDLR6PDAwEBUVFT2eV19fD3d3d6jVaixYsABvv/02Zs+eLT//5z//GSqVCo899phN7cjIyIBWq5W/wsPD+3Ibg+qqkf4AgF0FtWjjnkJEREQXZVBmCXl4eCAnJwf79u3Dq6++ivT0dGzduhUAcODAAfz973/HRx99BEEQbLresmXLUF9fL3+dOXNmAFt/acaFauHt6oQGfTt+PlNn7+YQERENSX0KLH5+flAqlaisrLR6vLKyEkFBQT2/iEKB2NhYJCYm4qmnnsKtt96KjIwMAMD27dtRVVWFESNGQKVSQaVS4fTp03jqqacQGRnZ7fU0Gg08PT2tvhyVUiFgeqw0LPTTiRo7t4aIiGho6lNgUavVmDx5MjIzM+XHTCYTMjMzkZKSYvN1TCYT9HppB+O7774bubm5yMnJkb9CQkLwzDPPdDuTaCiaYR4W+uk461iIiIguhqqvJ6Snp2Px4sVISkrC1KlTsXz5cjQ1NSEtLQ0AsGjRIoSGhso9KBkZGUhKSkJMTAz0ej3Wr1+P1atXY+XKlQAAX19f+Pr6Wr2Gk5MTgoKCMGrUqEu9P4dwlbnwNrekDmebDPBxU9u5RURERENLnwPLwoULUV1djZdeegkVFRVITEzEhg0b5ELc4uJiKBQdHTdNTU1YsmQJSkpK4OLigvj4eKxZswYLFy7sv7twcMFaF4wL9UReqQ4f7SxC+pzhEcSIiIgGiyCKomjvRlwqnU4HrVaL+vp6h61n+f5QOR7+JBvuGhV2PHs1vFzZy0JERJe3vnx+cy+hQTJ3bBBGB3uiUd+O97aftHdziIiIhhQGlkGiUAh4MnUkAODDnadwtslg5xYRERENHQwsg2j2mECMD9Wi2WDEp3uL7d0cIiKiIYOBZRAJgoC7kkcAAL7PK7dza4iIiIYOBpZBNntMIBQCkFeqw5mz3MGZiIjIFgwsg8zXXYOpUT4AgA150v5L5fUtqGnU27NZREREDo2BxQ7mjwsGIA0LHSw+h1l/2YobV+yEyTTkZ5gTERENCAYWO5g7Vtp3Kbu4Dg+sPgB9uwkl51pQoWu1c8uIiIgcEwOLHQRpnTFxhBcAoLqhYyiooKrRTi0iIiJybAwsdjJ/nNTL4qFRYXyoFgADCxERUU/6vJcQ9Y+7r4hEbaMB88YFYdPRShwqrUdBNQMLERFRdxhY7MRFrcSy60YDAE7VNgEACtnDQkRE1C0OCTmAGH93AEAhe1iIiIi6xcDiACyBpabRgLpm7jFERER0PgYWB+CmUSFE6wyAhbdERETdYWBxEDEBHcNCuSV1SPtwLwqqGuzcKiIiIsfAwOIgLMNCJyobsWzdIWzJr8Y/txfZuVVERESOgbOEHESsuYfly4OlqG2S6lj2njprzyYRERE5DPawOAhLYLGEFQA4Wd2EWm6KSERExMDiKCxDQgCgVioQbC7C3XfqnL2aRERE5DAYWByEn7saWhcnAMAtk0NxdXwAAGB/D8NCRTVNuO/jfcgrrR+0NhIREdkLA4uDEAQBN00MRYjWGUtmxWJqpA8AYJ85sHy86xTW7D4tH///firEpqNVeG39Ubu0l4iIaDCx6NaB/P76sXj5l2MgCAIEQXosr0yHLw6U4OVvDgMAZsb5I9zHFbtPSkEm62QtSutaEOrlYq9mExERDTj2sDgYwZxUwrxdEaJ1htEk4tkvcuXntx6vRkV9K4pqpP2HRBH46mCpXdpKREQ0WBhYHNiUKGlYqN0kyj0u2/KrsKeoFgCgMD/2RXYJRFG0RxOJiIgGBQOLA0sy17EoBODVG8cDAHYV1mLb8WoAwMIp4XB2UuBkdRN+LmHxLRERDV8MLA7slxOCMT3WF7+/fizunBqOAA8Nmg1GfJNTBgBIHR2IuWODAADrskvs2VQiIqIBxcDiwLxc1fjkviuwKCUSgiBgZpw/AGmISCFIPTA3TwoDAHyXWw6jicNCREQ0PDGwDCEzR/nL348N0ULr4oRpMb7wdFahtsmAg8VcZI6IiIYnBpYh5KpYf7nQ9opoqb7FSamQF5n78Uhlt+fp2434zUf78H8/5A9KO4mIiPobA8sQonV1wpUjpV6Wa+ID5cdnj5G+7ymw7Cs6h8xjVVi1rRD6duPAN5SIiKifceG4IebvCxNxqrYJE0d4y4/NjPOHk1LAyZomFFQ1yhspWhwuk2YQtRlFnKhsxLhQ7aC2mYiI6FKxh2WI8XZTW4UVAPBwdsIV0b4AgE1Hu/ayHC7Tyd8f4t5DREQ0BDGwDBNzLjAsdKS8I7Bws0QiIhqKGFiGiVRzYDlw+hweXnMAJyobAAAtBiNOVjfKx+V16m0hIiIaKljDMkwEa12wZFYMVm4rxPd5FfjxSCXWPpgCQQBMIuCkFNBmFHG0XIc2owlOSimr1re0IWP9UeSV1eN0TTOuivPDu3dNtvPdEBERWWMPyzDy23nx2PjEDEyN8kG7ScSHO4twxNyjckW0L9w1KhjaTSjs1OPyZXYJPtt3BnmlOjTo27H+UAVaDJxJREREjoWBZZiJC/TAiwvGAAB+OFKJ3SeljRLHhWoxJsQTAJBX2jEstP+0tNjcnVPD4aGROtxK65oHs8lERES9YmAZhsaFeiIu0B2GdhO+O1QOABgb4onx5unMnQtvDxbXAQB+MSEEYT6uAIAz51oGt8FERES9YGAZhgRBwC3mPYZE8/ZCY0O0GBcq9bBY1mWp1LWitK4FCgFICPdCmLcLAKCEgYWIiBwMA8swdePEUHkZfze1EhE+rhgXIvWwHC7TwWQSkW0eDhoV5Al3japTYOGQEBERORYGlmEq0NNZXsZ/dLAnFAoB0f7ucHFSotlgRG5pPbLNmyVOGuEFAAjzloaE2MNCRESOhoFlGHtwRjTUSgV+MSEYAKBUCJg7Vlqv5b3tJ3HgtCWwSCvnckiIiIgcFddhGcamx/rh2CvzoLCMDQF4cGYMvsopw/eHyqE0Pz4pwjqwlHJIiIiIHAx7WIa5zmEFkIaHrh7lD5MobYbo46ZGpK80FGQZEqppNHAtFiIicigMLJehh2bGyN9PGuEFQZBCjdbFCR7OXIuFiIgcDwPLZWhqlI9caJsU6WP1nKWXpfNaLG1GEz7Y0bFqLhER0WBjYLkMCYKAv98xEU/NjsOilAir57orvP3n9iL88dsj+M3H+9DaJg0VNerb5fVciIiIBhoDy2Uq3McVj147Eq5q67rr89diqWs24N2tBQCA8vpWfLTrFJr07bjl3V1Y8NYO7D91dnAbTkRElyXOEiIr56/FsmJLARpa2+GuUaFR3453txTgYPE55Fc2AAA25FV0GVYiIiLqb+xhISudh4RKzjXj412nAQBv3ZmI+CAP6FrbsfFwpXz8tuPVdmknERFdXhhYyIolsBTXNiH9859hMJqQEu2Lq0cF4Nn58fJxj10TC4UAnKhqRFkdF5ojIqKBxcBCVixDQuea27C36Cw8NCr84YaxEAQBs+L88czcUXhqdhyenB2HieYVcn9iLwsREQ0wBhay0nktFrVSgf+3KAlxgR4ApNlFS6+OxaPXjoQgCJgZJ+1VxGEhIiIaaAws1MWkEd5QKgQsvyMRKTG+PR43wxxYdhTUoErXirczT2BvEWcNERFR/7uowLJixQpERkbC2dkZycnJ2Lt3b4/Hrlu3DklJSfDy8oKbmxsSExOxevVq+fm2tjY8++yzGD9+PNzc3BASEoJFixahrKzsYppG/eAfd0/GrueuwXXjgy943PhQLbxdndDQ2o4Zf9mC//vxOB5ecwBtRhMA4KOdRbjm/7biWMXFLThXVNOErflVF3UuERENL30OLGvXrkV6ejpefvllZGdnIyEhAXPnzkVVVfcfLD4+PnjhhReQlZWF3NxcpKWlIS0tDRs3bgQANDc3Izs7Gy+++CKys7Oxbt065Ofn4/rrr7+0O6OL5uykRKCnc6/HKRUCrhop9bK0tkkhpbbJgB0FNWhtM+LNH4/jZHUTnl93CCaT2Kc2GNpN+NV7u3HPh/uQV8oF6oiILneCKIp9+iRJTk7GlClT8M477wAATCYTwsPD8eijj+K5556z6RqTJk3CggUL8Morr3T7/L59+zB16lScPn0aI0aM6PV6Op0OWq0W9fX18PT0tP1m6JIdLdfh5W8OY86YQJyqbcKa3cW4ITEEM+P8kf75z/Jxf70tAb+YEIxvcsoQF+SBxHCvC17365xSPP5ZDgDgD9ePxeJpkQCA4tpm+Hto4KJWDtAdERHRYOnL53efelgMBgMOHDiA1NTUjgsoFEhNTUVWVlav54uiiMzMTOTn52PGjBk9HldfXw9BEODl5dXt83q9HjqdzuqL7GN0sCc+fzAF910VjVsmhQEAfjhciQ93ngIARPu7AQAy1h/F/L9vx2+/yMXd7+9Bk779gte1nA9A3gLgwOlzmPXXLfjdV3n9fyNEROTQ+hRYampqYDQaERgYaPV4YGAgKioqejyvvr4e7u7uUKvVWLBgAd5++23Mnj2722NbW1vx7LPP4s477+wxbWVkZECr1cpf4eHhfbkNGiCJ4V6I9HVFS5sRh0rroVQI+Ne9UxHt54baJgOKapoAAA2t7fjyYGmP1zlYfA45Z+rkn/NKpUD645FKmERgV2HNgN4HERE5nkGZJeTh4YGcnBzs27cPr776KtLT07F169Yux7W1teH222+HKIpYuXJlj9dbtmwZ6uvr5a8zZ84MYOvJVoIg4IbEUPnn1NEBCPN2xRu3TkC0vxvumRaJp2bHAQD+lXUK549GVjW04nBZPVZuLQQApERLM5SOVzZA327EPvO+ReX1rWg8r4em2dCOf+8pRkNr24DdHxER2U+f9hLy8/ODUqlEZWWl1eOVlZUICgrq8TyFQoHY2FgAQGJiIo4ePYqMjAzMmjVLPsYSVk6fPo3NmzdfcCxLo9FAo9H0pek0SG6cGIq/Z54AAPwqWdoJOinSB5ufmgUA0LW2YeW2QhyvbETWyVpMi/EDAPz3QAme+yIX7Z2Kc19YMBq/fn8P6prbkFtSj9ySOvm5wqpGJHSqg/l75gn8Y9tJnDnXjGfndazIS0REw0OfeljUajUmT56MzMxM+TGTyYTMzEykpKTYfB2TyQS9Xi//bAkrJ06cwKZNm+Dr2/PaH+TYovzckD47DvdMi8RVsX5dnvd0dsLNk6RemI93nQIA/HymDs+vO4R2kwhfNzVCvVywOCUC40K1GBeiBQB8svs02owdYaagqtHquluPSYvXHSrhjCIiouGoz7s1p6enY/HixUhKSsLUqVOxfPlyNDU1IS0tDQCwaNEihIaGIiMjA4BUb5KUlISYmBjo9XqsX78eq1evlod82tracOuttyI7OxvffvstjEajXA/j4+MDtVrdX/dKg+Sxa0de8PlFKZFYs7sYGw9X4lfv7UZRTRMMRhNSRwfi/909GQqFIB87NtQTOwpq8N2hcqtrFFZ3BJaqhlZ59+jj5v8lIqLhpc+BZeHChaiursZLL72EiooKJCYmYsOGDXIhbnFxMRSKjo6bpqYmLFmyBCUlJXBxcUF8fDzWrFmDhQsXAgBKS0vxzTffAJCGizrbsmWL1bARDQ9xgR6478oovL+zCLsKawFIs4neXJhgFVYAyD0slt6VaH83nKxusuph2VnQUYRb1aBHXbMBXq4MukREw0mf12FxRFyHZWg6c7YZ/zlQgqPlOiybH49of/cuxxTVNOHqv26Vf/7dgtH403dHEe3vJtfFPPX5z/giu0Q+5vMHUzA1ymegm09ERJdowNZhIepP4T6uSJ8dh/cWJXUbVgAgwscV7hqpI9DTWYX55u0CTtc2w9BugiiK2FFQLT8PcFiIiGg4YmAhh6ZQCBgTLKXupEgfhGid4aZWwmgSUXy2CYXVjajU6aFWKXCzeeG6E70EFlEU8eYP+dh4uOe1g4iIyLEwsJDDmxEnzTaaMyYQgiAgJkDqjSmoasT2E1L9ytRIH4wLlepd8s8LLKIowthpunTOmTq8tbkAz6871GUtGCIickx9LrolGmwPzozBNfGBGB3sAQCI9XdHbkk9CqoasbNAKtqdHuuHUYHS8ycqpYLcrw6W4tO9xTharoNJBL5aOg2xAR44VSutuFvbZEBVg96mjR6JiMi+GFjI4TkpFRgT0lGMZelh+WDnKZxtMsBJKWDu2EAEa10gCFIQ2VVQgyfW5lhdZ2t+NWIDPFBc2yI/drRch0BPZ+wqrMF7P53EmBBPzBjpjymRPl1mLF1IeX0Limqa5IXwiIiof3FIiIacGHOB7tkmAwDg+etGI9rfHS5qJcK9XQEAv/0iFwAwa5S/vCmjpWel+GyzfK1jFdLw0fJNJ7AlvxorthRi4f/bjT9vPHbBNlTqWqFvN8o/P/5pDn713h7sKuA+R0REA4GBhYac2ICOGUXzxwXhnmmR8s9xgdJzJeekXpRl80fjimhpivOpGimonDnXEViOluugbzfiZ/Nmi1MivQEAu0+e7fH1s4vPISUjEy9/fRgA0NpmxIHicwCA7/NYyEtENBAYWGjIifR1RUK4FyaEafHnWydAEDqGbkaa61gA4LrxQRgV5IEoPzcAkHeLPtO5h6W8AXmlOujbTfBxU+MP148DAJwyH9udDXkVMIlSODGZRBwuq5eLejOPVl5SIW99cxu2n6iGycRiYCKizhhYaMhRKRX4eul0fL10OjydnayeG9UpsFi2CIg0B5ay+hboWttQoWuVjymsbkRWoTSMMznCWw439S1tOGcecjrfnpO18jEnqhqRc6Zj/6Ky+lZ5mKk3bUYTlq07hLX7iuXHfv+/w7j7/b14bf1Rm65BRHS5YGChIatzz4rFVSP9EOXnhnunRyE+SCrU9XVTw0OjgigCuwpqIYqAq1oJT2cV2k0i1u4/A0AaDnJRKxFknjVUVNu1l6WhtQ2HSjsCyt5TZ+XhJIvNx6oASNOuda1tPbZ/+4lqfLq3GL//5gj07UYYTaJ87j93FGFDXnmP5xIRXW4YWGhY8XXXYMvTs/DSL8fIjwmCgCh/qefkpxPSqrgjfFwRb16Q7sxZqd4lKVKqdYn0kwp3uxsW2n/6HDqP1uwrOoufS+oASENQALDpaCVWZ51C6pvbsPST7B7bmn1aOq+lzYj9p87hSJkO9S0dAeeZ/+TKw1gA8M7mE3jgX/vR2mY8/1JERMMeAwtdFiJ9pcCyLV8KLOE+rhgd1DF8pFEp5I0Wo/ykwt2ibgLLbvNwUISvFGq2n6jG6VqpJubxa+MAAAeL6/DSN1JB7o6Cmh6HlrLNhboAsO14NXaah6ZmxvljSqQ3GvTt+L35OmV1LXjzx+P44UglssxtICK6nDCw0GXBUsdSWif1poR7d/SwAEBiuBfUKumfQ5S5h6W7wLLHPHvovquioVIIONfcZj7HDaOCPDAuVLqmKAJKhQBR7OjV6cxoEq2GkrblV8u7Ts+I88dfb0uAIEhBpqCqAZ/tOyP37Bw2D0mJooivc0pRUd96/uWJiIYdBha6LESbA4vFCB8XxHfqYZkS2bG7s6U35tR5NSyN+na5fuXqUf4Ya94KAAASwqTvb0wMNf9vCO6dHgmgo1dH19om96ocr2xAk8EIFyclBEHaTsAShq6M9UOErxtmjw4EALz3U5FVYW5eqQ4A8G1uOR7/LAf3/WsftxggomGPgYUuC5HnBxZfV8QFesBSt5tkXn8FgDxT6FRNs1UQOHD6HIwmEWHeLgjzdsWUiI5zEsK9AABp06Pw45Mz8LeFibg6PgCA1EtiNIm498N9uPndXfgut1wOLpMivJAQJp1rMJrg566R15L5zZVRAIC1+8+gUqeX25pXJoWm7eaem7xSHXZwwToiGuYYWOiyEOV7fg+LK9w0KixMCkdShDeSo3zl58J9XCEIUo9KTWNH/ckG86JwV0RLx06J6uiVsQQWpULAyEAPCIKApAgfuGtUqG0y4LX1R7H/tBRS3so8gQPm7yeN8MbMOH/5OtNifOXZT1OjfDC205YEd04dAUBaFK+u2YA9RR2L2/1j28mL/M0QEQ0NDCx0WdC6OsHbtWPNljDzEv6v3zIB/314GlzUSvk5ZyclQr1cAHTUsWzIK8ene6VhmV8mhACQdoh2dlLA01mFMZ3qYSzUKgWmx0rh5v0dRfLj+ZUN+PZnacryxBFemDmqI7BYjgek2U2WXhYAeGhGjFzsu+loFU7XNkMhSCFpR0ENDpV0TLcmIhpuGFjosmEZ6gn01MDZSWnTsadqmlBQ1Yin/yPtTXT/VVFyj4i3mxr/fWgaPn8opcfrzRoVIH8f6uUibyNgMJoAABPDvZEQ5oVQLxdoVApcNdLf6vxfTAjBHVPC8dt5ozDC11WeyWQJQONCtfjlhGAAwF9/yEdNo962XwYR0RDDwEKXDUsdywgf196PNQ8h5ZTU4YHV+9Gob0dylA+enRdvddy4UK28QF13ZnXqPXlufjyWXB0jz0aK9nODt5saSoWAzx9Kwf8evRIh5p4dC7VKgddvmYAls2IBAGPNs5COlkuFt8lRPnhgRgwAqVbmitcy8einBy96rZb65jZU6fpv1tGRMh3u+3g/TlTatvovEVFPGFjosjEyQJoVFHVeAW53LOHm33uKcbK6CcFaZ7z9q4lQKfv2TyZY64LfLRiNx64diV9MCEaAhzMWJoUDsJ6ZFOrlgrhO2wr0xNLDYpEc5YsxIZ5Y9etJSAjTot0k4n8/l2HHid6LcDfklWPqq5vw3wMlAKSanQVvb8c1/7fNar+lnnyXW47pr2/GsnW5OFah6/aYj3YVYdPRSmR8f+Hdr4mIesPAQpeNX00dgfTZcXj0mpG9Htt5GrS7RoUP06YgwMP5ol73vquikT47Ti6mff660fjjDWPx9NxRfb5W5yJcQego/J03LhhfP3IlFpiHhwqqGy94nUMl9Xj8sxxUNejx0td5KK1rwVuZJ1ByrgWN+na8vflEr21ZvfsUSuta8OneM5i3fDs+3FnU5RhLDdCW/CqbQtCl2lt01mq1YCIaPhhY6LKhdXXCY9eORLgNQ0IjzVOLlQoB79416YLDPn3lolZiUUok/D00fT7X112DEK0UnMYEe0LrYr35Y5y5F6mwqmtgOVndiO9yy7HpSCXu/9d+6NtNUCkENBuMWPpJNj7oVBj8RXYpTl4g9BhNorwezBTzlPAPug0sUkgRReCzTmvJDISvc0px+z+y8OJXeQP6OkRkHwwsRN0I83bFql9Pxqf3X4EZcf69nzCILAvWdZ6KbRETIPUMFZ4XNr48WII5f/sJS/+djfv+tR8VulbEBrjjPw+lQK1UIOdMHdpNIlJHB+La+AAYTSL+ntlzL8vJ6kY06tvhqlbi/XumQKkQcOZsC0rOdfSiNLS2WRUBr913BoZ20yXd+4V8skcKRFvzq2AycSE9ouFGZe8GEDmqeeOC7N2Ebj1ydSyUgoDfXBXV5bkYf6lnqLC6CaIoQhAE/GNboVxDEh/kAYUgwNlJgTdvT0SknxuWXh2Lv206DrVKgZd+MQa61jZkHqvCNz+XwdtVjRlxfhgZ4IEgrTOczDU8OeZtBcaFauHp7IQJYVocLK7D7pNncetkqQfLsseSt6sTnJQKVDXo8X1eOW4wrwbcn07XNmGveV0aXWs78isbMLqbqeZENHQxsBANMQnhXlh19+Run4vyc4MgAPUtbahtMuBUTZMcVu6/KgrL5o+GQiFYnfPwrBgYRRHjQjwxwrzOy80TQ7HuYCk+2nUKH+06BUCqmblpYijevD1R3qE60bxg3hXRvubAUotbJ4cB6KhfifZ3x/QYX7y1uQBPrM3B+zuKcHtSOH59RUS//U6+yC61+nnPyVoGFqJhhkNCRMOIs5MS4eZF8QqrGrHVvI/RgvHBeGHBmC5hBZCmTqfPjsOcsR09Sm/cOgEr75qEO6eGI8rPDWqlAqIIrMsuxfHKBvx8RlqkzrKtgGX1392ddpI+ZQ4skb5uSJseheQoH4gikFtSj999lXfBGpnvD5VjddYpm/ZIMplEfGGe6WRZwG/vqbMXOsVKQVUDPt1bfNHDSB/uLMLzXx7iMBTRAGNgIRpmYvwtdSxNcoDovJquLVRKBeaPD0bGzROw5elZyP/TPMwZI23G+NGuU/I6MAnhUj1NUoQ3VAoBJeda5NlARbWWwOIKbzc11j6Ygr3PX4up5plNXx4sPf9lAUjTqx/77CBe/PowsjoFoJ7sPlmL0roWeDir8Px1owFIs4Vs3RBy2bpDWLbuELZ1s6v2u1sL8NLXeWg3dl9709pmxGvrj+Lfe4rljTGJaGAwsBANM5Y6lkOl9fLQTUp01wLdvhAEAXcmS3sZfba3GO0mEX7uankLAzeNChPMO1ZbQpKlhqXzxpMBns7yUNC67NJueyX2FZ1Fm1F6/P3tXWcene+bn8sASFsmTInyhkalQE2jAYXVHbttN7S24fN9Z7osqGcyiThcJoWv8xe3q2s24I0N+fhX1mm8u7Ww29fOK62X21pyrqXXtnbnaLkO+RVcWI+oNwwsRMNMTIAUWL79uQxtRhGhXi4I83bp5azezRjpj1AvF1gyRkKYl7y2DNB5WEgajrEMCZ2/UN+cMYHw0KhQWtfS7dDNzk47T2ceq+oy4+l8lp2vrx4VAI1KiYkjvABALsIFgL9uzMdvv8jFu1sKrM4trWtBs0EKMZYp2BaWwmJA2rAyr5seFMtrA7CaIWUrXWsbbluVhVtX7UKzob3P5xNdThhYiIYZSw9Lg176AEyO8rEKFhdLqRBwu3mVXqBjh2oLS2DJKqyRi34B6x4WQKqzuW68tMDduuySLq+zq1DqobGsMdN548gmfTvufn8PXlt/VP65wLzmTIK5h2eqebr33qKO4aRtx6Xhns35VVav1bln43Rtk9Vz2cV1AACFALSbRKR/ntOlh8ay6zZwcT0seSX1aNS3o6G1HQfNr2cvBVUNcqE0kSNiYCEaZiw1LBZXXOJwUGe3TwmDpW73/MAyJdIH7hoVyupb8dHOUwAAP3cN3DVdJyPeYp5JtP5QBVoMHSHgbJMBR8z1Ma/dNB4A8MWBEpw1h5912SXYfqIG/9x+EvUtbThcpoNJBII8nRHgKS2ol2yukdl9UqpjKa1rwSnz8NThMp18LUDaOdvi1Hkf1gfNvSePXxsHP3c1jlc2YuPhCvl5URTlUANIvTV9ldup12ZfHwqF+1tdswE3rtiF21bt6rFeh8jeGFiIhhkfNzW8XDtWwO3PwBKsdcHTc0fhFxOCu9TFuKiVuMtc57JiqzT0EuXX/arCSRHeCPdxQaO+HZ/sOS0/bql/GRXogevGB2F8qBb6dhP+sa0QoihizW5pcTiTKB2ba67RsdTPAMCkEd5wVStRoWvFwTN1yCrs6GkRReshp851K2X1rXIPiskkykNC144OwFzzDKrjnY4vOdeC6gZ9p5+th4TONhlw07s78fhnB+XHahv1+Of2k2gy934dKulbYDlWocO7WwsuenNLURS7LUbOKqxFo74dNY0GVDZwx29yTAwsRMOMIAiINQ8LhWidEe5z6fUrnS2ZFYt3fjVJ3nW6s3uvjIJaqZBXtLXsen0+hULAUvMO1G/+eFzunbCEiWmxvhAEAU/OlvZ9+nDXKXzzc5lVj8jOghr8bP7A79zb46JWygHjq4Ol2FUoXdPS3s4bQ+ZXWtfHFJtnOBVWN6KhtR0uTkrEB3nIdTidh0ws9Ss+bmoAUoCxhIF2owmP/DsbB4vr8HVOmRxmXlt/DH/67ijeMq8ibCmKBoCDxXVo66V349kvDuGNDfldanFs9einBzH1tUzUNlqHkh2dQlzZRfQUEQ0GBhaiYchSx3JFtG+/1K/YKtDTGTdN7FjJ9vz6lc5uTwpHUoQ3mg1G/P6bwwA66lemxfgBkAppr4j2gaHdhKc+/xkA5JlJOwpquu1hAYAbEkMAAN/mlsshaJF5dtKOghqIooh2o0nec8kSOizDQpYwMiFMC5VSIf8+T3aaeZRtrl+xrIjcbDCirlnaePHPG47J9wIAPx2vgdEkYvOxSgDAhsMVONtkkOte3NRKNBuM8nTx7lTqWvGzudfng52ncK7T0JYtTlQ24NvcclQ36K3aBlj3Op0fWGoa9fjl2zuw4iJDElF/YWAhGobumR6JmXH+eGhWzKC/9gMzo2HJSOfPEOpMoRDw2s3joVII+PFIJa7882YU1TRBIQDJ0VIdiiAIWDZfWlul3Tw96Y1bJ0AhSOHBMnV6QqiX1bWvjPWDn7saZ5sMqNTpoVYqsOTqWKiVCpTWtaCopgmnzzbDYDTBxUkpD29ZrmcpgJ0U4W11H0U1TfJU7APmUJMS7StvZFlyrgVZhbV4zzwde2qkdB8/Ha9Gzpk6nDMHmtO1zfJid9F+bkiOthQKS7tNf7zrFL7LLUd+RYPca7PpaKV8f436dvzjp5MXfB/O9++9HZtP5pV1DEWVnGuWa3yArrU43+SU4VBpPT42r3hMZC8MLETD0OhgT3x871TEBXoM+mvH+Lvj/quiMSrQA9NiLlw/ExfogYfNocrS2zAzzh+ezh01OAnhXvjFBGlWUUKYFtNj/TDevMIuIC1Mp3W13rVapVTgFxNC5J8njvCCj5sakyKk83YU1OC4eYZQXKA7os2FypbF7iw9LBPNQ01h3i5wUgrQt5tQVt+CZkM7jpZL50+K8JanjZeca5aDxc0TQ/H8Ails7SyswY9HOgIHAKzcJq3tMj5MiynmYLOzoAb3frQPL39zGEv/nY25y6UNK0VRxCbz+ZaF9z7edcpqc8kLaW0zYl2n7QsOl3b05OwqsO5tOb+HZat5hlVVg77PvTpE/YmBhYj63fPXjcbGJ2fAy1Xd67FPpsbhi4dT8MXD07Dhiavw3qKkLsf8/vqxWJQSgddulmYOXRnbEYQmdAovnXUemrIMMV01Ulrx97vcchyTA4sHIsy1Nqdrm6BrbcMJ81CRpYdFpVRghI9UQFxU04ScM3UwmkQEeTojROssD1OVnGvBfnPx7MxR/hgfqoWXqxMaWtuxZrdUXGyZxWSZrTQ+VIupUdLrbMmvxoHT5+ChUWHiCC+oFALWH6rAlwdLsdM8jPOnG8chIUyLljYjXv3uqE0r+n6fV476ljY4O0l/8vPK6uXzLPUrQeZZVmV1rfJ5LQaj1XYLxyu5wB3ZDwMLEdmVQiFgcoQPJkd4Iz7IEypl1z9Lfu4a/PGGcRgbItWqTI/1k587v36l8+PxQR4QBOCa+AAAwPxxQVArFdhTdFZe3yUu0EOezXSqphlbjlVBFIERPq7wc9fI14vyk+pYimo6doaeal7jJsy8f9PxygbkmVfOTYr0gVIh4EpzWxv17RAEKXB03tJpQpgXxoVq5aJglULAP+6ejC+XTMdDM6Xep+e+OARDuwkRvq4YGeCOZdeNhkKQtjd4Z3PvtSWf7jkDALj/qmg4KQXUNbehtK4FJpMo16/cMlkKeJ17WHafrJULqC33R2QvDCxENORMGuEt9xacvx6MhSAI+Pjeqfji4WkYbw410f7ueOXGsQCkAAEAcUEdPSxl9S14d4s0VGPZddrCMmx0sto6sACQh4Q2Hq6A0SStLmzpdZkR17GPU0KYF0YGeiApQjpPIQBjQzyhUSkx03zcazeNxzRzyHl4VgwCPDQwmGcPpY4OhCAIuCLaF3+8YRwA4P9+PI6vetiXCQC+zinF3lNnoVQI+PUVEfIwYV6pDvmVDahtMsDFSYkF46UhtNJOC+BtNS+0ZwlY+QwsZEcMLEQ05Dg7KfHnWybgsWtikWQetulOoKczJo2wfn7hlBHyejGAVMPi66aGh0YFUZQ+lF2clLjbPKvIItpceJtf0SDXuCSfF1h0rVIISorseM0ZIzsCi6WnZ85YaSPJ2AB3uJkX1vv7HYnY9sws3D6lYzVhN40Kz82Pl39OHR0of//rKyLwwIxoAMBvv8iVtw4oqGrEii0F+PlMHQ6cPotn/psLALjvqigEejpjnLmX6nBZvRx0psX4ItLcy9Sgb4euVSoOttSvzB8n1RAdr7jwNglEA6nrEpREREPADYmhvR/Ug5d/ORYtBiNc1EoEeTpDEARE+Lkiz1yMunBKOLzdrOtvLDOF9hTVwiRKU6Fjzfs2nb9Xk6WIFgCCtM6YGumDnDN1uG58kHz9YxUN8s8A4KpWIcK365/kGxNDsaOgBs16I6ZEWoev5+bFo7CqEZnHqvDg6gN4em4cfvdlHpoMRvxlYz4UgrTIXuroQPx2rhR8xoV6Yu1+aaE6S+HwnVNHwFWtgrerE841t6GsrgW1KgNO1zbDSSngnumR+O5QOfIrpVlLdc1tOFbRgBRzUbXJJOL9HUWIC/KQe4qI+hsDCxFddtQqBd5cmGj1WKSvG/JKdVAqBPzmyqgu50Sb12KxbP44JdJbXuMm1Mt6Rd/OgQUA3lucBF1LG8LNhbsezk74620JNrVVoRDw5u2JPT+3MBHXv7MDp2ub8eTan81tdUNZXQta20wYE+yJv9+RCKV5XGdsqGVXbWlYK8zbBVebe35CvFzkwFJsnuqcFOGD8aFaKBUC6lvaUNWgx1Of/4wdBTV4686JuD4hBBsPV+DV9UfhrlFh7wvXwlXNjxbqfxwSIiICMCbEEwDwywnBcrDozM9dGjaysGyyCEir6/q5Sz0yWhcnjDT3vFhoXZy6vWZ/0Lo4YdWvJ8s1PXdODceGx2dg/+9m48O0KVj74BXysBMAjA7ytCr6XZQSIYeZEHPdTWldqzwcNGuUP5ydlIj0ldr/XW65PLPon9tPSlsmmLdXaNS347vc8gG5TyLGYCIiAPdMi0Sgh7O8cu35BEFAlL8bcs3bAVjqVyxCvV1R02hAUoQ3FIrBW10YkNbd+WrpdFTq9Jgx0g+CIECtUuDqUQFdjnVRKxEb4I7jlY1wdlJY7cBtKRQ+Wd0o78E0y3yNUUEeKKxuwvJNx+Xjc0vqpSnXndZy+Xz/GdzW6ZpE/YU9LEREkGpIbpkcZtUbcT5L4a2HRoXRwZ5Wz1n2b0rpZbG8gRIf5ImZcf42bcVgWbvmhoRQq7VyQryktVj+93M59O0mBGudERco3ZdldpGlsNgya+q5Lw4BkGZrKQRg36lzKKxuRGubEUfKdDatE0NkC/awEBHZyLKn0ORIb3kYxeLpuXFICNfijikjujvVoTyROhLBWucutTqWISHLCrqzRnUEoFGdVk0O83bB8oWJuP6dnfKU68euicUne4qx+VgVMtYfQ36lDmfOtuDWyWHIuHk8nLpZX4eoLxhYiIhs9KvkESg+24y06V2LcoO1LliUEjn4jboIYd6ueGrOqC6PWwKLxcy4jiGluKCOwPKr5BGYEOaFpAhv7D99DqFeLpg1KgBtRhGbj1VZ7Xv03wMlqNS14t27JsHD2XoLBaK+YOQlIrKRr7sGf7ktQS7QHW5COwUWlULA9E5bIET4uCLI0xkeGhVumyzVqKTPiYO3qxOenB0HpULAtaMDEO4jXePXV4zAu3dNgqtaie0navB/PxwH0aVgDwsREQEA/N01cFIKaDOKmBzhbdUjolIq8PUj09FmNMm7U0+L8cPBl+bIxzgpFfhyyXQ06dvl1YObDUY8/Z+f5YXtiC4WAwsREQGQ1nUJ0jrjzNkWeXZQZ4HmDRIvxM9dY7UHU4y5OPf8XaCJ+opDQkREJLs+IQRh3i64ITGkX65nqYup0LWi3Wjq5WiinrGHhYiIZM/Mjcczc+N7P9BGnYeZqhr0XQp7iWzFHhYiIhowlmEmgMNCdGkYWIiIaEAFay1L/jOw0MVjYCEiogFlmS5dVtdq55bQUHZRgWXFihWIjIyEs7MzkpOTsXfv3h6PXbduHZKSkuDl5QU3NzckJiZi9erVVseIooiXXnoJwcHBcHFxQWpqKk6cOHExTSMiIgdjWfK/vJ49LHTx+hxY1q5di/T0dLz88svIzs5GQkIC5s6di6qqqm6P9/HxwQsvvICsrCzk5uYiLS0NaWlp2Lhxo3zMG2+8gbfeegurVq3Cnj174Obmhrlz56K1lWmciGioC5F7WBhY6OIJYh93pkpOTsaUKVPwzjvvAABMJhPCw8Px6KOP4rnnnrPpGpMmTcKCBQvwyiuvQBRFhISE4KmnnsLTTz8NAKivr0dgYCA++ugj3HHHHb1eT6fTQavVor6+Hp6ew3MFSiKioWrLsSqkfbQPo4M98f3jV9m7OeRA+vL53aceFoPBgAMHDiA1NbXjAgoFUlNTkZWV1ev5oigiMzMT+fn5mDFjBgCgqKgIFRUVVtfUarVITk7u8Zp6vR46nc7qi4iIHJOlh4VDQnQp+hRYampqYDQaERgYaPV4YGAgKioqejyvvr4e7u7uUKvVWLBgAd5++23Mnj0bAOTz+nLNjIwMaLVa+Ss8PLwvt0FERIPIUsNS19yGJn27nVtDQ9WgzBLy8PBATk4O9u3bh1dffRXp6enYunXrRV9v2bJlqK+vl7/OnDnTf40lIqJ+5eHsBA+NtE5pd70sRpOILw+WoORc82A3jYaQPq106+fnB6VSicrKSqvHKysrERQU1ON5CoUCsbGxAIDExEQcPXoUGRkZmDVrlnxeZWUlgoODra6ZmJjY7fU0Gg00Gk23zxERkeMJ8XJBfmUDSutaERvgYfXcP34qxBsb8jFxhBe+XDLdTi0kR9enHha1Wo3JkycjMzNTfsxkMiEzMxMpKSk2X8dkMkGv1wMAoqKiEBQUZHVNnU6HPXv29OmaRETkuOSpzefNFKrSteKdzQUAgIPFdThR2TDobaOhoc97CaWnp2Px4sVISkrC1KlTsXz5cjQ1NSEtLQ0AsGjRIoSGhiIjIwOAVG+SlJSEmJgY6PV6rF+/HqtXr8bKlSsBAIIg4IknnsCf/vQnjBw5ElFRUXjxxRcREhKCG2+8sf/ulIiI7Kbz1OYjZTocr2xA6phAvLExH80Go3zcf7NLsGz+aHs10+F8suc0fFzVmD8+uPeDh7k+B5aFCxeiuroaL730EioqKpCYmIgNGzbIRbPFxcVQKDo6bpqamrBkyRKUlJTAxcUF8fHxWLNmDRYuXCgf89vf/hZNTU144IEHUFdXhyuvvBIbNmyAs3PvW5kTEZHjswSW7QU1eH9HEZoMRriplWhuk8LKQzNjsGpbIb7MLsUzc0ZBpeRC7JW6VrzwZR7USgVmjvKHq/ry3q+4z+uwOCKuw0JE5Ni+PFiCJ9f+LP/s4qREizms3JgYgjduTcAVGZk422TAh/dMwdXxAfZqqsPILj6Hm9/dBQD49/3JmBbjZ+cW9b8BW4eFiIjoYoSYN0AEgGg/N2Qtuwb/vi8Zv503Cn+8cRzUKgVuSAwBAPz3QMklv15VQytO1zZd8JgjZTr8cLjnJTnsrUrXsdr7/lPn7NgSx8DAQkREAy42wB1qlQJaFye8f88UeLmqMS3WD0tmxcLT2QkAcOvkMADAj0cqrT6s+6qqoRXX/X075i7/qcftANqNJiz+cC8eWH0AOWfqLvq1BlJVg17+fv9pBhYGFiIiGnC+7hqsf+wq/PjkDET5uXV7zNgQLZIivGEwmvD+zqKLeh1RFLHsi0OoaTSgtc2EL3rordl36hyqzYHgxyN972URRRErthTg3a0F8mMnqxux+IO9OFh84XBhMtlWiVHZKbRlnz4Ho43nDVcMLERENChiA9wR4HnhyRQPz4oBAPx7dzF0rW02XXdXQQ3G/34j7vlwL17fcAyZxzo24/1vdgm6K9XckFcuf7/5WLVNr9NZVmEt/rIxH29syJeDxZrdxdh2vBp/+N+RHs/7Lrcc43+/ER/aEMgqdR09LI36dhyr6H4bmhOVDZfFdHAGFiIichhXjwpAXKA7GvTtWLP7tE3nfLTrFBpa27E1vxr/2HYSAPBE6ki4qZU4XduMfafOobC6EU98dhBZhbUwmURsPNyxAOrRcl2fdpIWRRF/23Rc/vlEZSMAoKBa+t+cM3XIK63vct6hknqkf56DJoMRX2T3XqdjCUKCIP3cXR1Lo74dN6/chVtW7kKzYXhve8DAQkREDkOhEPDQTKmX5YMdp9DaZrzg8a1tRmw/UQMA+MWEYDg7KXD1KH88es1ILJggrV3yj22FuPufe/BVThkeXL0fPxypQIWuFW5qJcaFSjNTtuRX9fga59tVWIt9ncJDQZXUu1HQqZfj33uLrc6pbtDjgdX7oW83AZAKfht72VepytzDMiXCB0D3dSz7T51FQ2s7dK3tOG4OTsMVAwsRETmUXyaEINTLBTWNemzIu3B9SVZhLVrajAjROuPtOyfiyB/m4f3FU6BUCLh1srQxbuaxKpTVS70VutZ2PPZpDgDgmtGBmD9OCjVbjl04sJTVteDV745g1bZC/HnDMQDS1GwAOFHViEZ9u/waAPD1wVKrQPLHb4+gvL4V0f5uCPJ0hklEr7UuVQ3S9a4bL21hs6/obJfhrd0nz8rfHyvvfsjIViu2FOCRf2f3GhLthYGFiIgcipNSIc8Y+iqn9ILH/nhUGtq5dnQgBEGAQiF9AcCUSG+M8HEFAAR5OuNf906FRqWAwSj1cswbG4SrR0nrvewoqLngB/XyTcfx3vYivP79MeSW1EOjUuDx1JEApMBSWCX1bvi6qRHt74YmgxFfHZTaXt/Sho3m6dN/uz0RV0RLPSb7LjBVWd9uxLlmqYZnztggqBQCKnStKD1v6Gr3yVr5+2MVF1/Hom83Yvmm4/g2t1xuq6NhYCEiIodz48RQAMD2EzXybJ7zmUwiMs2BJXVMYJfnBUHAi78Yg5lx/vjXb6ZiRpw/fjsvHgCgUSkwa5Q/Rgd7IFjrjNY2E7I6ffh3JoqiPOw0PdYXCWFa/G7BaEw3L+RWWNWIAnNgGRnojruSIwAAH+86BaNJxIa8chjaTRgV6IEJYVokRZqHeE6d7ebVJJbhILVKgWCtM8aFagEAezr1qDTp23GoU63M0fN6WM6cbcaCt7bjqc9/Rovhwr0mR8sb0GaUem++PGgdEkVRRPraHPx7T7FdZyoxsBARkcOJ8nNDYrgXjCYR//u5rNtj8srqUanTw02tlHstzjd7TCA+vncq4gKlHaLTpkXi+evi8dadE+GmUUEQBMwy97L8dLz72UKF1U0or2+FWqnAPxdNwdePXIm7UyIREyBNz65tMmBvkRQkYgPccevkMHg6q3CiqhFf55TKAeDGiaEQBAFTzIEl50wd2sy9PeezDAcFeGggCAKuiPYFAKtQtd881Vlt3sbgWEWDPGTUZjThsc8O4nCZDl9kl+CO/5clX7M7uSV18vfnh8St+dVYd7AUL3+T16fi5P7GwEJERA7p5klSL8tXOaWob27Dx7tOWfUibDoi9a7MiPOHRqW06ZoKhYAHZsRg7tgg+bGrRko9JTvMvSgA0GIwykNEOwukx5MiveGi7ngdV7UKoeY9kixDUyMDPKB1ccJD5unZr39/TK4zsazkOzLAHZ7OKjQbjF16RSwsU5oDzdPAU2LMgaWwVg4le8zhZe64ICgVAupb2lBhnln0VuYJHCyug4dGBW9XJ/xcUo+pr2Yi7oXvcdUbm1FUY70K8M9nOnpqjCYR3+ZKIbHNaMIr30nTtO+dHoVw8xCbPTCwEBGRQ1owPhgqhYDcknpc+cZmvPzNYdz1zz0422RAfXMbPt13BgCQOrrrcFBfTIvxhSBItSiVulboWttw9V+34rq3tqPFYOw0HNR1L5+Rge4AgLNNBgBSDwsApE2LQoCHRl6t9opoH3kDSIVCkIeFeqpjsaz0G+ipAQAkRXhDpRBQWteCM2elXg5L/cqMkX6I8Zd6e46VN2D/qbN4Z4u0oN1rN4/Hl0umI87cToPRhDNnW+ShNAtLD4slvFl6hVZnncbJ6ib4uqmx9JrYXn+XA4mBhYiIHJKvuwYz4/wBAA2t7VAqBJxtMuBP3x3B6xuOobpBj2h/N/wiIfiSXsfLVY3x5hqRnQU1+CanDBW6VpysbsK7WwvkYGD5MO9spDmgnP+zi1qJx64dKT9+k7kmxyIp0huANPOnO5XmoBPgIfWwuGlUSAz3AgBknaxBs6EduSVSr8gV0b6ID5KmZx+t0OFvm45DFKUeql8mhCDSzw0bHp+B/b9Lxb3TowDAqoelUd8uryHzwoLRUJpD4rP/zcVy83ozT80ZJW+hYC8MLERE5LCemx+P+eOC8NfbEvD5g1dAEIB12aX41LzOScZN420eDroQS+/JjhM1+Hz/Gfnxd7YUoFHfDi9XJ4wN0XY5b2SAh/y9h7MK/h4a+eeFU8KREKZFiNYZ88dbhypLHcuGwxV4eM0BeS0XC8uicQGeHdfrPCy0Ia8C7SYRoV4uCPN2QXyw1I5vcsqws6AWSoWAp+aMks9VKAT4uWswNkQKNp0DS15pPUQRCNE6Iz7IE7PMIXHt/jPQtbYjPsgDC6eEX/gXOAhU9m4AERFRT0YGemDlryfLPy9OicRHu04BAO6cGo5kczHqpboq1g8rtxbi+7wKtLQZ4aQUMCrIA3mlUo3J9Bg/KM3TpTuL6dTDMjLAHYLQcYyTUoEvHp4GQRC6nJsU4Y1FKRFYvfs0vs+rwI4TNfgxfSaCtFKPimWWUKBHx1YGKdG+eHtzAbafqMFWc4HwbUlhEAQBo809LJapzdeND5brazqLNO/j1DmwWIaDEsw9OK/cOA4JB0qkgl6VAtcnhHR774ONPSxERDRkPDN3FEYGuCPC1xXPzRvdb9edFOENjUqBFnOh7ewxgXj1xvHy893VrwAdNSvnf2+hUiq6/bAXBAF/vGEcNjw+A9H+bmjQt2NTp7qSSrmGpSOwTIrwhlqpQG2TAXXNbRgfqsXSq6W6EksPi8V9V0Z1295oc2Apr2+VpzpbCm4nhHkBAEK8XPDYtSPx5Ow4LL061q6Ftp0xsBAR0ZDhplFhwxMzkJk+E1rX/qupcHZSYmpUx9To25PCkRDuhceuHYlJI7zk1WbPp3VxkgtjOw8P2WpUkAduNte3bOs0rdpSrBvYaUjI2UmJiSO8AEjryPxtYQKczFOagzydoXWRfh9TI33k3pLzebup4WX+vZ2qlXpZfrb0sIR1HfJyJAwsREQ0pCgVAlTK/v/4utLcixKidcZVI6U6jvTZcVi3ZDq8XNU9nmepR7EU0vbVzDhpHZiswloY2k1obTOivkVa5TbAw3p361smh0GlEPD768citlNAEgQBKebhsYevjrng60X6dgwLnW0yoOScNOtonIMHFtawEBERAbhjygjklelw88TQPtVs/PW2BDw1ZxSizMMtfTU2xBO+bmrUNhmQXXwOIVqp9kSjUsDTxfpj+vakcNyYGAq1qmtge+O2CXhydhxGBV24pyfazw05Z+pQVNMEy11Ka8PYdxZQb9jDQkREBEDr6oS375yIq+MD+nSes5PyosMKIM3gsUyZ/ul4NSobOupXOhfxWnQXVgDA09mp17ACQG5rUU0TtpsXxbuymynbjoY9LERERHY2c5Q/vsopw7bj1VCZe3c616/0p84zhSzL9Xe3xoyjYWAhIiKyM0vNzOEyHQ6XSVOp54zpvtD3Ull6WA6V1MNgNMFJKSA5qn+mhw8kDgkRERHZmZ+7BuNCPeWfn5k7Cvdd1f3U5EtlCSwG88aLk0Z4w03j+P0XDCxEREQO4J5pUQjydMabtydg6dWx3dav9Ac3jQoBnVbkHQrDQQCHhIiIiBzCrZPDcOvksEF5rSg/N3mtlyvNw1GOjj0sREREl5lo8+7OWhcneeNHR8fAQkREdJkZHSzVy8yI83eIfYJswSEhIiKiy8wdU0ZAIQiYN25gZiINBAYWIiKiy4xapcCvr4iwdzP6hENCRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOb1js1iyKIgBAp9PZuSVERERkK8vntuVz/EKGRWBpaGgAAISHh9u5JURERNRXDQ0N0Gq1FzxGEG2JNQ7OZDKhrKwMHh4eEAShX6+t0+kQHh6OM2fOwNPTs1+v7SiG+z0O9/sDeI/DwXC/P4D3OBz09/2JooiGhgaEhIRAobhwlcqw6GFRKBQICwsb0Nfw9PQclv/xdTbc73G43x/AexwOhvv9AbzH4aA/76+3nhULFt0SERGRw2NgISIiIofHwNILjUaDl19+GRqNxt5NGTDD/R6H+/0BvMfhYLjfH8B7HA7seX/DouiWiIiIhjf2sBAREZHDY2AhIiIih8fAQkRERA6PgYWIiIgcHgNLL1asWIHIyEg4OzsjOTkZe/futXeTLkpGRgamTJkCDw8PBAQE4MYbb0R+fr7VMbNmzYIgCFZfDz30kJ1a3He///3vu7Q/Pj5efr61tRVLly6Fr68v3N3dccstt6CystKOLe6byMjILvcnCAKWLl0KYGi+fz/99BN++ctfIiQkBIIg4KuvvrJ6XhRFvPTSSwgODoaLiwtSU1Nx4sQJq2POnj2Lu+66C56envDy8sJvfvMbNDY2DuJdXNiF7rGtrQ3PPvssxo8fDzc3N4SEhGDRokUoKyuzukZ37/3rr78+yHfSvd7ew3vuuadL2+fNm2d1zFB+DwF0++9SEAT85S9/kY9x5PfQls8HW/5+FhcXY8GCBXB1dUVAQACeeeYZtLe391s7GVguYO3atUhPT8fLL7+M7OxsJCQkYO7cuaiqqrJ30/ps27ZtWLp0KXbv3o0ff/wRbW1tmDNnDpqamqyOu//++1FeXi5/vfHGG3Zq8cUZO3asVft37NghP/fkk0/if//7H/7zn/9g27ZtKCsrw80332zH1vbNvn37rO7txx9/BADcdttt8jFD7f1rampCQkICVqxY0e3zb7zxBt566y2sWrUKe/bsgZubG+bOnYvW1lb5mLvuuguHDx/Gjz/+iG+//RY//fQTHnjggcG6hV5d6B6bm5uRnZ2NF198EdnZ2Vi3bh3y8/Nx/fXXdzn2j3/8o9V7++ijjw5G83vV23sIAPPmzbNq+6effmr1/FB+DwFY3Vt5eTk++OADCIKAW265xeo4R30Pbfl86O3vp9FoxIIFC2AwGLBr1y58/PHH+Oijj/DSSy/1X0NF6tHUqVPFpUuXyj8bjUYxJCREzMjIsGOr+kdVVZUIQNy2bZv82MyZM8XHH3/cfo26RC+//LKYkJDQ7XN1dXWik5OT+J///Ed+7OjRoyIAMSsra5Ba2L8ef/xxMSYmRjSZTKIoDv33D4D45Zdfyj+bTCYxKChI/Mtf/iI/VldXJ2o0GvHTTz8VRVEUjxw5IgIQ9+3bJx/z/fffi4IgiKWlpYPWdludf4/d2bt3rwhAPH36tPxYRESE+Le//W1gG9cPuru/xYsXizfccEOP5wzH9/CGG24Qr7nmGqvHhsp7KIpdPx9s+fu5fv16UaFQiBUVFfIxK1euFD09PUW9Xt8v7WIPSw8MBgMOHDiA1NRU+TGFQoHU1FRkZWXZsWX9o76+HgDg4+Nj9fgnn3wCPz8/jBs3DsuWLUNzc7M9mnfRTpw4gZCQEERHR+Ouu+5CcXExAODAgQNoa2uzej/j4+MxYsSIIfl+GgwGrFmzBvfee6/Vhp9D/f3rrKioCBUVFVbvmVarRXJysvyeZWVlwcvLC0lJSfIxqampUCgU2LNnz6C3uT/U19dDEAR4eXlZPf7666/D19cXEydOxF/+8pd+7WofaFu3bkVAQABGjRqFhx9+GLW1tfJzw+09rKysxHfffYff/OY3XZ4bKu/h+Z8Ptvz9zMrKwvjx4xEYGCgfM3fuXOh0Ohw+fLhf2jUsNj8cCDU1NTAajVa/fAAIDAzEsWPH7NSq/mEymfDEE09g+vTpGDdunPz4r371K0RERCAkJAS5ubl49tlnkZ+fj3Xr1tmxtbZLTk7GRx99hFGjRqG8vBx/+MMfcNVVVyEvLw8VFRVQq9VdPgQCAwNRUVFhnwZfgq+++gp1dXW455575MeG+vt3Psv70t2/QctzFRUVCAgIsHpepVLBx8dnSL6vra2tePbZZ3HnnXdabSz32GOPYdKkSfDx8cGuXbuwbNkylJeX480337Rja20zb9483HzzzYiKikJhYSGef/55zJ8/H1lZWVAqlcPuPfz444/h4eHRZbh5qLyH3X0+2PL3s6Kiott/q5bn+gMDy2Vo6dKlyMvLs6rvAGA1Zjx+/HgEBwfj2muvRWFhIWJiYga7mX02f/58+fsJEyYgOTkZERER+Pzzz+Hi4mLHlvW/999/H/Pnz0dISIj82FB//y53bW1tuP322yGKIlauXGn1XHp6uvz9hAkToFar8eCDDyIjI8Phl4C/44475O/Hjx+PCRMmICYmBlu3bsW1115rx5YNjA8++AB33XUXnJ2drR4fKu9hT58PjoBDQj3w8/ODUqnsUgVdWVmJoKAgO7Xq0j3yyCP49ttvsWXLFoSFhV3w2OTkZABAQUHBYDSt33l5eSEuLg4FBQUICgqCwWBAXV2d1TFD8f08ffo0Nm3ahPvuu++Cxw3198/yvlzo32BQUFCXIvj29nacPXt2SL2vlrBy+vRp/Pjjj1a9K91JTk5Ge3s7Tp06NTgN7EfR0dHw8/OT/7scLu8hAGzfvh35+fm9/tsEHPM97OnzwZa/n0FBQd3+W7U81x8YWHqgVqsxefJkZGZmyo+ZTCZkZmYiJSXFji27OKIo4pFHHsGXX36JzZs3IyoqqtdzcnJyAADBwcED3LqB0djYiMLCQgQHB2Py5MlwcnKyej/z8/NRXFw85N7PDz/8EAEBAViwYMEFjxvq719UVBSCgoKs3jOdToc9e/bI71lKSgrq6upw4MAB+ZjNmzfDZDLJgc3RWcLKiRMnsGnTJvj6+vZ6Tk5ODhQKRZehlKGgpKQEtbW18n+Xw+E9tHj//fcxefJkJCQk9HqsI72HvX0+2PL3MyUlBYcOHbIKn5bwPWbMmH5rKPXgs88+EzUajfjRRx+JR44cER944AHRy8vLqgp6qHj44YdFrVYrbt26VSwvL5e/mpubRVEUxYKCAvGPf/yjuH//frGoqEj8+uuvxejoaHHGjBl2brntnnrqKXHr1q1iUVGRuHPnTjE1NVX08/MTq6qqRFEUxYceekgcMWKEuHnzZnH//v1iSkqKmJKSYudW943RaBRHjBghPvvss1aPD9X3r6GhQTx48KB48OBBEYD45ptvigcPHpRnyLz++uuil5eX+PXXX4u5ubniDTfcIEZFRYktLS3yNebNmydOnDhR3LNnj7hjxw5x5MiR4p133mmvW+riQvdoMBjE66+/XgwLCxNzcnKs/m1aZlbs2rVL/Nvf/ibm5OSIhYWF4po1a0R/f39x0aJFdr4zyYXur6GhQXz66afFrKwssaioSNy0aZM4adIkceTIkWJra6t8jaH8HlrU19eLrq6u4sqVK7uc7+jvYW+fD6LY+9/P9vZ2cdy4ceKcOXPEnJwcccOGDaK/v7+4bNmyfmsnA0sv3n77bXHEiBGiWq0Wp06dKu7evdveTbooALr9+vDDD0VRFMXi4mJxxowZoo+Pj6jRaMTY2FjxmWeeEevr6+3b8D5YuHChGBwcLKrVajE0NFRcuHChWFBQID/f0tIiLlmyRPT29hZdXV3Fm266SSwvL7dji/tu48aNIgAxPz/f6vGh+v5t2bKl2/8uFy9eLIqiNLX5xRdfFAMDA0WNRiNee+21Xe69trZWvPPOO0V3d3fR09NTTEtLExsaGuxwN9270D0WFRX1+G9zy5YtoiiK4oEDB8Tk5GRRq9WKzs7O4ujRo8XXXnvN6gPfni50f83NzeKcOXNEf39/0cnJSYyIiBDvv//+Lv+nbyi/hxb/+Mc/RBcXF7Gurq7L+Y7+Hvb2+SCKtv39PHXqlDh//nzRxcVF9PPzE5966imxra2t39opmBtLRERE5LBYw0JEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeAwsRERE5PAYWIiIiMjhMbAQERGRw2NgISIiIofHwEJEREQOj4GFiIiIHB4DCxERETk8BhYiIiJyeP8f23VYvFs9T2oAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "589726a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into evaluation mode (needed esp. for batchnorm)\n",
    "for layer in model.layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "46c11ba2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 1.9428514242172241\n",
      "val 2.0290133953094482\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the loss\n",
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]\n",
    "    \n",
    "    logits = model(x)\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(split, loss.item())\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27b52be0",
   "metadata": {},
   "source": [
    "Performance log:\n",
    "\n",
    "|                                      |train|val  |\n",
    "| ------------------------------------ | --- | --- |\n",
    "| original (3 characters, 12K params)  |2.058|2.105|\n",
    "| context: 3 -> 8 (22K params)         |1.918|2.033|\n",
    "| flat -> hierarchical (22K params)    |1.942|2.029|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "700a8542",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "zhlene.\n",
      "jafar.\n",
      "aleala.\n",
      "zyanna.\n",
      "sarman.\n",
      "gidella.\n",
      "lucove.\n",
      "wilvi.\n",
      "jerediah.\n",
      "yanua.\n",
      "keerthaus.\n",
      "bryrio.\n",
      "loursten.\n",
      "aahli.\n",
      "janaisha.\n",
      "leynn.\n",
      "zahmia.\n",
      "ellam.\n",
      "breck.\n",
      "kendrix.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the model\n",
    "for _ in range(20):\n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all zeros (representing the special '.' token)\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      logits = model(torch.tensor([context]))\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1781ed69",
   "metadata": {},
   "source": [
    "## BatchNorm Bug?!\n",
    "\n",
    "We still have the bug with our `BatchNorm1d` layer.<br>\n",
    "It runs, but does it even do the right thing right now?<br><br>\n",
    "Let's see:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "0aa8e50d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embedding : (1, 8, 10)\n",
      "FlattenConsecutive : (1, 4, 20)\n",
      "Linear : (1, 4, 68)\n",
      "BatchNorm1d : (1, 4, 68)\n",
      "Tanh : (1, 4, 68)\n",
      "FlattenConsecutive : (1, 2, 136)\n",
      "Linear : (1, 2, 68)\n",
      "BatchNorm1d : (1, 2, 68)\n",
      "Tanh : (1, 2, 68)\n",
      "FlattenConsecutive : (1, 136)\n",
      "Linear : (1, 68)\n",
      "BatchNorm1d : (1, 68)\n",
      "Tanh : (1, 68)\n",
      "Linear : (1, 27)\n"
     ]
    }
   ],
   "source": [
    "for layer in model.layers:\n",
    "    print(layer.__class__.__name__, ':', tuple(layer.out.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9e82f42",
   "metadata": {},
   "source": [
    "The first `BatchNorm1d` receives a $[32, 4, 68]$ input.<br>\n",
    "This is three-dimensional. **But our current BatchNorm1d only accounts for 2D inputs.**\n",
    "\n",
    "**What? How did we get an output then anyway?**\n",
    "\n",
    "Everything broadcasts properly across the model, but the BatchNorm1d layer just does ... something. Anything.<br>\n",
    "We still have a learning effect that somewhat compensates this rogue behavior, though.<br>\n",
    "This is what keeps BatchNorm1d seem to run smoothly. **But it doesn't.**\n",
    "\n",
    "**Let's think our way through BatchNorm1d:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "90473e8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([32, 4, 68])\n"
     ]
    }
   ],
   "source": [
    "# Our faulty BatchNorm1d works like this:\n",
    "e = torch.randn(32, 4, 68)      # 32 examples, 4 characters, 68 embeddings\n",
    "emean = e.mean(0, keepdim=True) # (1, 4, 68); mean ONLY over the very first dimension\n",
    "evar = e.var(0, keepdim=True)   # (1, 4, 68); variance ONLY over the very first dimension\n",
    "ehat = (e-emean) / torch.sqrt(evar + 1e-5) # (32, 4, 68); normalize over the first dimension\n",
    "print(ehat.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "460d4642",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 4, 68])"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.layers[3].running_mean.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37144f68",
   "metadata": {},
   "source": [
    "The means are only taken over $32$ inputs of the first dimension. Each input is now itself a vector $[4, 68]$. That's the point.<br>\n",
    "The `running_mean` this way becomes $[1, 4, 68]$ instead of the expected $68$.<br>\n",
    "This tells us that the current implementation 'batch-normalizes' in parallel over $4$ times $68$ channels,<br>\n",
    "instead of only $68$ channels total.<br>\n",
    "\n",
    "> We normalize over groups of character embeddings, instead of over the character embeddings themselves.<br>\n",
    "> Small groups, so admittedly its not too bad, but it still distorts the normalization process.\n",
    "\n",
    "So, we actually want to average over the $32\\times 4$ embeddings, each with $68$ channels.<br>\n",
    "\n",
    "> We want the $68$ channel wide average of all the embeddings.\n",
    "\n",
    "To achieve this, we can do something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "id": "d9c88427",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 4, 68])\n",
      "torch.Size([1, 1, 68])\n"
     ]
    }
   ],
   "source": [
    "# Just to get an idea of what's going on:\n",
    "emean_alt = e.mean(0, keepdim=True) # Mean across the batch dimension\n",
    "print(emean_alt.shape)\n",
    "emean_alt = emean_alt.mean(1, keepdim=True) # Mean across the character dimension\n",
    "print(emean_alt.shape) # That looks good, but can we do that better?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "64094087",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([32, 4, 68])\n"
     ]
    }
   ],
   "source": [
    "e = torch.randn(32, 4, 68)\n",
    "emean = e.mean((0,1), keepdim=True)        # 1, 4, 68 ; mean ONLY over the very first dimension\n",
    "evar = e.var((0,1), keepdim=True)          # 1, 4, 68\n",
    "ehat = (e-emean) / torch.sqrt(evar + 1e-5) # 32, 4, 68\n",
    "\n",
    "print(ehat.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "id": "ed9b678a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 1, 68])\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "print(emean.shape)                             # This is what we want\n",
    "print((emean.all() == emean_alt.all()).item()) # We get the same result, but much more efficiently"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6143f61",
   "metadata": {},
   "source": [
    "Now that we theoretically addressed the 'wrong mean' problem,<br>\n",
    "let's add a little bit of logic to the `BatchNorm1d` layer to practically address it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "id": "b4544ee0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mimicks torch.nn.linear\n",
    "class Linear:\n",
    "  \n",
    "  def __init__(self, fan_in, fan_out, bias=True):\n",
    "    self.weight = torch.randn((fan_in, fan_out)) / fan_in**0.5  # Kaiming init\n",
    "    self.bias = torch.zeros(fan_out) if bias else None\n",
    "  \n",
    "  # Performs a forward pass if called (syntax: l = Linear(x))\n",
    "  def __call__(self, x):\n",
    "    self.out = x @ self.weight\n",
    "    if self.bias is not None:\n",
    "      self.out += self.bias\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.weight] + ([] if self.bias is None else [self.bias])\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# NOT ANYMORE Mimicking torch.nn.batchnorm1d\n",
    "class BatchNorm1d:\n",
    "  \n",
    "  def __init__(self, dim, eps=1e-5, momentum=0.1):\n",
    "    self.eps = eps\n",
    "    self.momentum = momentum\n",
    "    # Batchnorm1d behaves differently during training and production\n",
    "    self.training = True\n",
    "    # Parameters (trained with backprop)\n",
    "    self.gamma = torch.ones(dim)\n",
    "    self.beta = torch.zeros(dim)\n",
    "    # Buffers (trained with a running 'momentum update' outside backprop)\n",
    "    self.running_mean = torch.zeros(dim)\n",
    "    self.running_var = torch.ones(dim)\n",
    "  \n",
    "  def __call__(self, x):\n",
    "    # Calculate the forward pass\n",
    "    if self.training:\n",
    "      # THIS IS THE NEW PART! IT BREAKS WITH PYTORCH LINGUA\n",
    "      # Addressing the 'wrong mean' problem from above\n",
    "      if x.ndim == 2:\n",
    "        dim = 0 # average just over the batch dimension\n",
    "      elif x.ndim == 3:\n",
    "        dim = (0,1) # average over both the batch and character dimensions\n",
    "      xmean = x.mean(dim, keepdim=True) # Batchnorm mean\n",
    "      xvar = x.var(dim, keepdim=True) # Batchnorm variance\n",
    "    else:\n",
    "      xmean = self.running_mean\n",
    "      xvar = self.running_var\n",
    "    xhat = (x - xmean) / torch.sqrt(xvar + self.eps) # Normalize to unit variance\n",
    "    self.out = self.gamma * xhat + self.beta\n",
    "    # Update the buffers\n",
    "    if self.training:\n",
    "      with torch.no_grad():\n",
    "        self.running_mean = (1 - self.momentum) * self.running_mean + self.momentum * xmean\n",
    "        self.running_var = (1 - self.momentum) * self.running_var + self.momentum * xvar\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.gamma, self.beta]\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.tanh\n",
    "class Tanh:\n",
    "  def __call__(self, x):\n",
    "    self.out = torch.tanh(x)\n",
    "    return self.out\n",
    "\n",
    "  def parameters(self):\n",
    "    return []\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.embedding, does the indexing operation\n",
    "class Embedding:\n",
    "  \n",
    "  def __init__(self, num_embeddings, embedding_dim):\n",
    "    # What was previously C (lookup table) is now the weight in this layer\n",
    "    self.weight = torch.randn((num_embeddings, embedding_dim))\n",
    "    \n",
    "  def __call__(self, IX):\n",
    "    self.out = self.weight[IX]\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return [self.weight]\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Modified from torch.nn.flatten\n",
    "class FlattenConsecutive:\n",
    "    \n",
    "  def __init__(self, n):  # n is the number of consecutive elements to be concatenated\n",
    "    self.n = n\n",
    "    \n",
    "  def __call__(self, x):\n",
    "    B, T, C = x.shape # 4, 8, 10 in example before\n",
    "    x = x.view(B, T//self.n, C*self.n) # Flattening in groups, now 4, 4, 20 for n=2 characters\n",
    "    if x.shape[1] == 1:\n",
    "        x = x.squeeze(1) # Takes out the dimension 1, as it only has value 1 (irrelevant)\n",
    "    self.out = x\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    return []\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------\n",
    "\n",
    "# Mimicks torch.nn.Sequential\n",
    "class Sequential:\n",
    "  \n",
    "  def __init__(self, layers):\n",
    "    self.layers = layers      # This is basically the same list structure as before\n",
    "  \n",
    "  def __call__(self, x):      # Simply pass input x through every layer in self.layers\n",
    "    for layer in self.layers:\n",
    "      x = layer(x)\n",
    "    self.out = x\n",
    "    return self.out\n",
    "  \n",
    "  def parameters(self):\n",
    "    # get parameters of all layers, stretch them out into one list\n",
    "    return [p for layer in self.layers for p in layer.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4709ae93",
   "metadata": {},
   "source": [
    "With the sneaky bug in the `BatchNorm1d` layer addressed and fixed, let's see how the model performs now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "id": "06c1dec8",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42); # For reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "id": "5714a10d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22397\n"
     ]
    }
   ],
   "source": [
    "n_embd   = 10 # Dimensionality of the character embedding vectors\n",
    "n_hidden = 68 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd),   # Previously C = torch.randn((vocab_size, n_embd))\n",
    "    # The new layer at work now (block_size was 8 before)\n",
    "    FlattenConsecutive(2), Linear(n_embd * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Shrink output layer weights to avoid saturation\n",
    "\n",
    "# before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "id": "26ddea9b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.3012\n"
     ]
    }
   ],
   "source": [
    "# Training Sanity Check\n",
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "id": "e6c17ea9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 1, 68])"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.layers[3].running_mean.shape # Just like we wanted (1, 1, 68)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "id": "6ddd06ff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embedding : (32, 8, 10)\n",
      "FlattenConsecutive : (32, 4, 20)\n",
      "Linear : (32, 4, 68)\n",
      "BatchNorm1d : (32, 4, 68)\n",
      "Tanh : (32, 4, 68)\n",
      "FlattenConsecutive : (32, 2, 136)\n",
      "Linear : (32, 2, 68)\n",
      "BatchNorm1d : (32, 2, 68)\n",
      "Tanh : (32, 2, 68)\n",
      "FlattenConsecutive : (32, 136)\n",
      "Linear : (32, 68)\n",
      "BatchNorm1d : (32, 68)\n",
      "Tanh : (32, 68)\n",
      "Linear : (32, 27)\n"
     ]
    }
   ],
   "source": [
    "for layer in model.layers:\n",
    "    print(layer.__class__.__name__, ':', tuple(layer.out.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "id": "80be5411",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.2207\n",
      "  10000/ 200000: 2.1758\n",
      "  20000/ 200000: 2.4535\n",
      "  30000/ 200000: 2.0300\n",
      "  40000/ 200000: 2.0540\n",
      "  50000/ 200000: 2.2182\n",
      "  60000/ 200000: 2.1820\n",
      "  70000/ 200000: 1.9922\n",
      "  80000/ 200000: 2.2610\n",
      "  90000/ 200000: 1.5608\n",
      " 100000/ 200000: 2.3285\n",
      " 110000/ 200000: 1.6706\n",
      " 120000/ 200000: 2.0902\n",
      " 130000/ 200000: 2.3220\n",
      " 140000/ 200000: 1.8958\n",
      " 150000/ 200000: 1.5260\n",
      " 160000/ 200000: 2.2711\n",
      " 170000/ 200000: 1.8425\n",
      " 180000/ 200000: 2.0662\n",
      " 190000/ 200000: 2.0791\n"
     ]
    }
   ],
   "source": [
    "# Full Training\n",
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "id": "c321036b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABiv0lEQVR4nO3deXhU1f0G8HfWTNYJ2fc9ELYkQEgE2dSwWCq4I6UFo9Uqro21Fv0J1qWxai0tInRzg1apFRUVQQmbQNgSQhBCCJCd7Ps+k5n7++PO3GRIIAtJJgnv53nyPMmde++cywh5Ped7zpEJgiCAiIiIaAiTW7sBRERERN1hYCEiIqIhj4GFiIiIhjwGFiIiIhryGFiIiIhoyGNgISIioiGPgYWIiIiGPAYWIiIiGvKU1m5AfzAajbh06RIcHR0hk8ms3RwiIiLqAUEQUF9fDx8fH8jlV+9DGRGB5dKlS/D397d2M4iIiKgPCgoK4Ofnd9VzRkRgcXR0BCA+sJOTk5VbQ0RERD1RV1cHf39/6ff41YyIwGIeBnJycmJgISIiGmZ6Us7BolsiIiIa8hhYiIiIaMhjYCEiIqIhj4GFiIiIhjwGFiIiIhryGFiIiIhoyGNgISIioiGPgYWIiIiGPAYWIiIiGvIYWIiIiGjIY2AhIiKiIY+BhYiIiIa8EbH54UDRtRnxxx1noTcY8cLCsbBRKqzdJCIiousSe1i68a8DOfgoJQ+tbUZrN4WIiOi6xcByFSpF+3bXegYWIiIiq2FguQqZTCaFFr1BsHJriIiIrl8MLN1QKcQ/Ih17WIiIiKyGgaUbUmAxMLAQERFZCwNLN8yBRc/AQkREZDUMLN1QSzUsDCxERETWwsDSDbWSPSxERETWxsDSjfaiW84SIiIishYGlm6whoWIiMj6GFi6oeKQEBERkdUxsHSDRbdERETWx8DSDfOQEPcSIiIish4Glm6017Cw6JaIiMhaGFi6waJbIiIi62Ng6YZayRoWIiIia2Ng6Yaamx8SERFZXZ8Cy/r16xEUFASNRoO4uDgcPXr0iudu3boVMTExcHZ2hr29PaKjo7Fp0yaLcxoaGvD444/Dz88Ptra2GDduHDZu3NiXpvU71rAQERFZn7K3F2zZsgWJiYnYuHEj4uLisHbtWsyfPx9ZWVnw8PDodL6LiwteeOEFREREQK1W4+uvv0ZCQgI8PDwwf/58AEBiYiJ2796NzZs3IygoCN999x1WrlwJHx8fLFq06Nqf8hpwHRYiIiLr63UPy9tvv42HHnoICQkJUk+InZ0d3nvvvS7PnzNnDu644w6MHTsWoaGheOqppxAZGYkDBw5I5xw6dAgrVqzAnDlzEBQUhIcffhhRUVFX7bkZLGoW3RIREVldrwKLTqdDamoq4uPj228glyM+Ph4pKSndXi8IApKTk5GVlYVZs2ZJx6dPn45t27ahqKgIgiBgz549OHfuHObNm9flfVpbW1FXV2fxNVBUpoXjWMNCRERkPb0aEqqoqIDBYICnp6fFcU9PT5w9e/aK19XW1sLX1xetra1QKBR49913MXfuXOn1devW4eGHH4afnx+USiXkcjn+8Y9/WISajpKSkvD73/++N03vM2nzQ/awEBERWU2va1j6wtHREenp6WhoaEBycjISExMREhKCOXPmABADy+HDh7Ft2zYEBgZi//79eOyxx+Dj42PRm2O2atUqJCYmSj/X1dXB399/QNrOdViIiIisr1eBxc3NDQqFAqWlpRbHS0tL4eXldcXr5HI5wsLCAADR0dHIzMxEUlIS5syZg+bmZjz//PP4/PPPsXDhQgBAZGQk0tPT8dZbb3UZWGxsbGBjY9ObpveZ2lx028ZZQkRERNbSqxoWtVqNKVOmIDk5WTpmNBqRnJyMadOm9fg+RqMRra2tAAC9Xg+9Xg+53LIpCoUCRqP1ezVU3PyQiIjI6no9JJSYmIgVK1YgJiYGsbGxWLt2LRobG5GQkAAAWL58OXx9fZGUlARArDeJiYlBaGgoWltbsX37dmzatAkbNmwAADg5OWH27Nl49tlnYWtri8DAQOzbtw8fffQR3n777X581L5Rs4aFiIjI6nodWJYsWYLy8nKsXr0aJSUliI6Oxo4dO6RC3Pz8fIveksbGRqxcuRKFhYWwtbVFREQENm/ejCVLlkjnfPLJJ1i1ahWWLVuGqqoqBAYG4rXXXsMjjzzSD494bbgOCxERkfXJBEEY9sUZdXV10Gq1qK2thZOTU7/e+7/HC/Db/2Xg5ggPvHf/1H69NxER0fWsN7+/uZdQN7hwHBERkfUxsHTDPK25lQvHERERWQ0DSzc4S4iIiMj6GFi6waJbIiIi62Ng6YZUw8KF44iIiKyGgaUbXJqfiIjI+hhYumFemp8LxxEREVkPA0s3WHRLRERkfQws3Whfh4U1LERERNbCwNINqYaF67AQERFZDQNLN8zTmls5JERERGQ1DCzd6FjDMgK2XSIiIhqWGFi6Ya5hEQTAYGRgISIisgYGlm6Ya1gAFt4SERFZCwNLNzoGFq7FQkREZB0MLN0w17AAXIuFiIjIWhhYuiGTyTqsxcLAQkREZA0MLD0gzRTiBohERERWwcDSAyppPyGDlVtCRER0fWJg6QFz4a2OPSxERERWwcDSA6xhISIisi4Glh7gjs1ERETWxcDSA9KQEAMLERGRVTCw9IC0YzNXuiUiIrIKBpYeUJtmCenb2MNCRERkDQwsPcCiWyIiIutiYOkBlVIsumUNCxERkXUwsPRA+zosDCxERETWwMDSAyy6JSIisi4Glh5gDQsREZF1MbD0ABeOIyIisi4Glh7gwnFERETWxcDSAyppHRbWsBAREVkDA0sPsIaFiIjIuhhYekBa6ZaBhYiIyCoYWHrAXHTLGhYiIiLrYGDpAS4cR0REZF0MLD2gYg0LERGRVTGw9ICaK90SERFZFQNLD7CGhYiIyLoYWHqgfR0WBhYiIiJrYGDpAdawEBERWRcDSw/YKFnDQkREZE0MLD3AvYSIiIisi4GlB7gOCxERkXUxsPSAeZYQa1iIiIisg4GlB7j5IRERkXUxsPSAikW3REREVsXA0gOsYSEiIrIuBpYeYA0LERGRdTGw9ABrWIiIiKyLgaUH1KxhISIisioGlh7gwnFERETWxcDSAx2LbgWBvSxERESDrU+BZf369QgKCoJGo0FcXByOHj16xXO3bt2KmJgYODs7w97eHtHR0di0aVOn8zIzM7Fo0SJotVrY29tj6tSpyM/P70vz+p25hgUA2owMLERERIOt14Fly5YtSExMxJo1a5CWloaoqCjMnz8fZWVlXZ7v4uKCF154ASkpKcjIyEBCQgISEhKwc+dO6ZwLFy5gxowZiIiIwN69e5GRkYEXX3wRGo2m70/Wj1RKmfQ9C2+JiIgGn0zo5RhHXFwcpk6dinfeeQcAYDQa4e/vjyeeeAK/+93venSPyZMnY+HChXjllVcAAPfddx9UKlWXPS89UVdXB61Wi9raWjg5OfXpHlejNxgR/sK3AICTq+dBa6fq9/cgIiK63vTm93evelh0Oh1SU1MRHx/ffgO5HPHx8UhJSen2ekEQkJycjKysLMyaNQuAGHi++eYbjB49GvPnz4eHhwfi4uLwxRdfXPE+ra2tqKurs/gaSEp5ew8LC2+JiIgGX68CS0VFBQwGAzw9PS2Oe3p6oqSk5IrX1dbWwsHBAWq1GgsXLsS6deswd+5cAEBZWRkaGhrw+uuvY8GCBfjuu+9wxx134M4778S+ffu6vF9SUhK0Wq305e/v35vH6DWZTMa1WIiIiKxIORhv4ujoiPT0dDQ0NCA5ORmJiYkICQnBnDlzYDSKAWDx4sX49a9/DQCIjo7GoUOHsHHjRsyePbvT/VatWoXExETp57q6ugEPLSqFDDoDAwsREZE19CqwuLm5QaFQoLS01OJ4aWkpvLy8rnidXC5HWFgYADGMZGZmIikpCXPmzIGbmxuUSiXGjRtncc3YsWNx4MCBLu9nY2MDGxub3jT9mqmVcjTqDAwsREREVtCrISG1Wo0pU6YgOTlZOmY0GpGcnIxp06b1+D5GoxGtra3SPadOnYqsrCyLc86dO4fAwMDeNG9Ata/FwmnNREREg63XQ0KJiYlYsWIFYmJiEBsbi7Vr16KxsREJCQkAgOXLl8PX1xdJSUkAxHqTmJgYhIaGorW1Fdu3b8emTZuwYcMG6Z7PPvsslixZglmzZuGmm27Cjh078NVXX2Hv3r3985T9gKvdEhERWU+vA8uSJUtQXl6O1atXo6SkBNHR0dixY4dUiJufnw+5vL3jprGxEStXrkRhYSFsbW0RERGBzZs3Y8mSJdI5d9xxBzZu3IikpCQ8+eSTGDNmDD777DPMmDGjHx6xf7TvJ8TAQkRENNh6vQ7LUDTQ67AAwLw/78O50gb855dxmB7mNiDvQUREdD0ZsHVYrmccEiIiIrIeBpYeUknrsAz7DikiIqJhh4Glh+xtFACA2ma9lVtCRER0/WFg6aFgN3sAwIXyBiu3hIiI6PrDwNJDYe4OAIDsUgYWIiKiwcbA0kPhno4A2MNCRERkDQwsPRTmIfaw5FU2okVvsHJriIiIri8MLD3k4WgDR40SRgHIrWy0dnOIiIiuKwwsPSSTyaRelvNlHBYiIiIaTAwsvcDCWyIiIutgYOmFcE9TDwsLb4mIiAYVA0svSENC7GEhIiIaVAwsvRDuIU5tzqloRBv3FCIiIho0DCy94OtsC41KDp3BiPyqJms3h4iI6LrBwNILcrkMoe6cKURERDTYGFh6SapjYeEtERHRoGFg6SXz1GYW3hIREQ0eBpZe4tRmIiKiwcfA0ksdV7s1GgUrt4aIiOj6wMDSS4Gu9lDKZWjSGVBc12Lt5hAREV0XGFh6SaWQI8jNHgCQXVpv5dYQERFdHxhY+iCcmyASERENKgaWPjDXsVxg4S0REdGgYGDpA3Ng4a7NREREg4OBpQ+kwFLWAEHgTCEiIqKBxsDSB6HuDpDJgNpmPSoadNZuDhER0YjHwNIHGpUC/qPsALDwloiIaDAwsPRR+wJynNpMREQ00BhY+ohTm4mIiAYPA0sfhXLXZiIiokHDwNJHEV6OAIAfi+pg4J5CREREA4qBpY/GeTvB0UaJ2mY9zlyqs3ZziIiIRjQGlj5SKuSIC3EBABy6UGHl1hAREY1sDCzXYHqoGwDg4IVKK7eEiIhoZGNguQbTw1wBAMdyqqBrM1q5NURERCMXA8s1GOPpCDcHNZr1BpzIr7Z2c4iIiEYsBpZrIJPJMM00LHSIw0JEREQDhoHlGt0YKg4LsfCWiIho4DCwXKMbw8QelhP5NSivb7Vya4iIiEYmBpZr5O9ih4m+WrQZBfz2fychCFxEjoiIqL8xsPSDt+6Jglopx56scmw6nGft5hAREY04DCz9YIyXI56/NQIA8No3mcipaLRyi4iIiEYWBpZ+smJ6EGKDXdDaZsR3p0us3RwiIqIRhYGln8hkMswwFeCeK+UOzkRERP2JgaUfjfZ0AACcK623ckuIiIhGFgaWfjTa0xEAkF1WD6ORs4WIiIj6CwNLPwp0tYdaKUeL3oiC6iZrN4eIiGjEYGDpRwq5DGHu4rBQVgmHhYiIiPoLA0s/G+NlHhZi4S0REVF/YWDpZ+Ge7GEhIiLqbwws/WyMqfCWM4WIiIj6DwNLPzPPFLpY3gi9wWjl1hAREY0MDCz9zNfZFnZqBXQGI/IquUQ/ERFRf2Bg6WdyuQzhpl6WrBIW3hIREfWHPgWW9evXIygoCBqNBnFxcTh69OgVz926dStiYmLg7OwMe3t7REdHY9OmTVc8/5FHHoFMJsPatWv70rQhYQxXvCUiIupXvQ4sW7ZsQWJiItasWYO0tDRERUVh/vz5KCsr6/J8FxcXvPDCC0hJSUFGRgYSEhKQkJCAnTt3djr3888/x+HDh+Hj49P7JxlCOq54S0RERNeu14Hl7bffxkMPPYSEhASMGzcOGzduhJ2dHd57770uz58zZw7uuOMOjB07FqGhoXjqqacQGRmJAwcOWJxXVFSEJ554Av/+97+hUqn69jRDRLCbPQAgt4Kr3RIREfWHXgUWnU6H1NRUxMfHt99ALkd8fDxSUlK6vV4QBCQnJyMrKwuzZs2SjhuNRvziF7/As88+i/Hjx3d7n9bWVtTV1Vl8DSVB5sBS2QhB4J5CRERE16pXgaWiogIGgwGenp4Wxz09PVFSUnLF62pra+Hg4AC1Wo2FCxdi3bp1mDt3rvT6H//4RyiVSjz55JM9akdSUhK0Wq305e/v35vHGHD+o+wglwFNOgPK61ut3RwiIqJhb1BmCTk6OiI9PR3Hjh3Da6+9hsTEROzduxcAkJqair/85S/44IMPIJPJenS/VatWoba2VvoqKCgYwNb3nloph+8oWwBAbiWHhYiIiK6Vsjcnu7m5QaFQoLS01OJ4aWkpvLy8rnidXC5HWFgYACA6OhqZmZlISkrCnDlz8MMPP6CsrAwBAQHS+QaDAc888wzWrl2L3NzcTvezsbGBjY1Nb5o+6IJc7VFQ1YzcikbEBrtYuzlERETDWq96WNRqNaZMmYLk5GTpmNFoRHJyMqZNm9bj+xiNRrS2ikMlv/jFL5CRkYH09HTpy8fHB88++2yXM4mGiyBXsY4lh4vHERERXbNe9bAAQGJiIlasWIGYmBjExsZi7dq1aGxsREJCAgBg+fLl8PX1RVJSEgCx3iQmJgahoaFobW3F9u3bsWnTJmzYsAEA4OrqCldXV4v3UKlU8PLywpgxY671+axGKrytYGAhIiK6Vr0OLEuWLEF5eTlWr16NkpISREdHY8eOHVIhbn5+PuTy9o6bxsZGrFy5EoWFhbC1tUVERAQ2b96MJUuW9N9TDEHBbnYAWMNCRETUH2TCCJh3W1dXB61Wi9raWjg5OVm7OQCAC+UNuOVP+2CnVuD07+f3uKCYiIjoetGb39/cS2iAdJzaXMapzURERNeEgWWAqJVy+I0yDQuxjoWIiOiaMLAMoEBXcx0LAwsREdG1YGAZQOY9hXK4pxAREdE1YWAZQOa1WPLYw0JERHRNGFgGUJBpanNmcR3qW/RWbg0REdHwxcAygMb7aGGjlCO3sgm3/Gkftp8qtnaTiIiIhiUGlgHk6aTBP1fEIMjVDmX1rXji4xOcMURERNQHDCwDbGa4O3Y8PQvjfZxgMApIL6ixdpOIiIiGHQaWQaBRKTApwBkAcLak3rqNISIiGoYYWAZJhJe45PDZkjort4SIiGj4YWAZJBFejgCALPawEBER9RoDyyAZbQosxbUtqGnSWbk1REREwwsDyyBx0qjg62wLgHUsREREvcXAMojGenNYiIiIqC8YWAYRC2+JiIj6hoFlEI0x1bFwSIiIiKh3GFgGUcchIaNRsHJriIiIhg8GlkEU5GoPtVKOJp0BBdVN1m4OERHRsMHAMoiUCjnCPRwAAG/szMJ7B3JQVt9i5VYRERENfQwsgyza3xkA8E1GMV7++gx+99kp6zaIiIhoGFBauwHXm9/Oj8AYL0ecL2vARyl5OHC+Ak26Ntip+VEQERFdCXtYBpnWToXl04Lw+0Xj4etsC12bESkXKq3dLCIioiGNgcVKZDIZ5oxxBwDszSq3cmuIiIiGNgYWK7ppjAcAYO+5MggCpzkTERFdCQOLFU0Pc4VaIUdBVTMuVjRauzlERERDFgOLFdmplYgNdgEA7DlbhqpGHYpqmq3cKiIioqGHgcXKzHUsf9mVjZhXv8ecN/cgs5h7DREREXXEwGJlN0WIdSz1rW0wCoDeIGDLsQIrt4qIiGhoYWCxslB3B6xdEo3/WzgWr90xAQDw1clL0BuMVm4ZERHR0MHAMgTcPskXv5wZgntj/OFqr0Zlow4HzldYnHP5LKI2gxG7zpSiSdc2mE0lIiKyCgaWIUSlkOO2KB8AwBcniqTjLXoDbl9/EHe8exBtpp6X9w7m4JcfHcfGfRet0lYiIqLBxMAyxNw+yRcAsPN0CRpaxd6TLccKcLKwFifya3C2pB4AsP+c2ANz5hILdImIaORjYBliovy0CHazR4veiP8dL0CL3oB3956XXj+WWwWDUcCJ/GoAQGF1k7WaSkRENGgYWIYYmUyGn8UGAABe+SYTif9NR2ldq/T68dxqZJXUo1FnAAAUVTdzlVwiIhrxGFiGoAdnBOPOyb4wGAVsP1UCAFgcLda2HM2tQmpelXRufWsb6ppZeEtERCMbA8sQJJfL8MZdkVgw3gsA4Otsi1dvnwC1Qo7y+lZs7VCQCwAFHBYiIqIRTmntBlDXlAo5/rI0GluOFWBqkAscNSpM9NMiNa8aJ/JrxHPkMrQZBRRWN2OCr9a6DSYiIhpA7GEZwmyUCiyfFoSx3k4AgJigUdJrMhkwPcwNAAtviYho5GNgGUZig1yk78d4OmKstyMAoLCaGyYSEdHIxsAyjEwJHGXxvd8oOwAMLERENPIxsAwjznZqRHiJvSoxQaPgN8oWAFBUw8BCREQjG4tuh5nX7piIfefK8dNIH+RVNgJgDQsREY18DCzDzJTAUdLQkK+zOCRU39KG2mY9tLYqazaNiIhowHBIaBizVSvgaq8GwF4WIiIa2RhYhjlzHcvVCm91bUa8tO00thzLH6xmERER9SsGlmGuJzOFPk0twAeHcvHStjMwGrnvEBERDT8MLMNcew9L10NCujYj3t1zAQDQrDcgv0o8b/fZUty94RAOZFcMTkOJiIiuAQPLMGcOLD9kV+BXm45j5b9T8UN2ubSD8+cnCi2mPZ8tqQcA/OtADo7nVWPF+0ex6XDe4DeciIioFzhLaJgzDwmdL2vA+bIGAMD2UyUI93DAggle+CJd3CjRXq1Ao86ArJJ6zB/viYzCWgCAwSjgxS9+hK7NiAdnBFvnIYiIiLrBHpZh7oYQV8we7Y4ZYW747YIxuH96EOzUCmSXNWDd7vMoqGqGq70aD88KBQBkldYht7IJ9S1tUCvleGS2eHwze1mIiGgIYw/LMGerVuDDB2Itjv167mh8d7oE+86V41RRLX4dPxqjTNOfz5bUI6OwBgAw3scJK28KxT9+uIicikYUVjdJPTZERERDCQPLCKS1VeGeGH/cE+MvHSutawEA5FY04mhOFQAg0lcLJ40KUX5apOXX4EB2Be6LDbBKm4mIiK6GQ0LXCQ9HGzjbqWAUgK8zigEAkX7OAIAZ4e4AgAPnOWOIiIiGpj4FlvXr1yMoKAgajQZxcXE4evToFc/dunUrYmJi4OzsDHt7e0RHR2PTpk3S63q9Hs899xwmTpwIe3t7+Pj4YPny5bh06VJfmkZXIJPJMMZT3DixtlkPAIjy1wIAZoa7AQAOXajkOi1ERDQk9TqwbNmyBYmJiVizZg3S0tIQFRWF+fPno6ysrMvzXVxc8MILLyAlJQUZGRlISEhAQkICdu7cCQBoampCWloaXnzxRaSlpWHr1q3IysrCokWLru3JqBPzTs8A4GCjRIibAwAg2t8Z9moFqhp1OFNcZ63mERERXZFMMC/Y0UNxcXGYOnUq3nnnHQCA0WiEv78/nnjiCfzud7/r0T0mT56MhQsX4pVXXuny9WPHjiE2NhZ5eXkICOi+pqKurg5arRa1tbVwcnLq+cNcZ/5zJB/Pf34KAHBDiAs+eXia9NqDHxxD8tky/O7WCGnmEBER0UDqze/vXvWw6HQ6pKamIj4+vv0Gcjni4+ORkpLS7fWCICA5ORlZWVmYNWvWFc+rra2FTCaDs7Nzl6+3trairq7O4ou6N6ZDD0uUqX7FbIZpWIgr3xIR0VDUq8BSUVEBg8EAT09Pi+Oenp4oKSm54nW1tbVwcHCAWq3GwoULsW7dOsydO7fLc1taWvDcc89h6dKlV0xbSUlJ0Gq10pe/v3+X55GljoEl8vLAEiYGlmO5VdC1GS1eK6ltwS8/PI4vTYvQERERDbZBmSXk6OiI9PR0HDt2DK+99hoSExOxd+/eTufp9Xrce++9EAQBGzZsuOL9Vq1ahdraWumroKBgAFs/cjjYKBEX7AKtrQpxIS4Wr4V5OMBJo0RrmxHnSuul420GI578+AR2ZZbi7e/PDXaTiYiIAPRyHRY3NzcoFAqUlpZaHC8tLYWXl9cVr5PL5QgLCwMAREdHIzMzE0lJSZgzZ450jjms5OXlYffu3Vcdy7KxsYGNjU1vmk4mHz4Qi9Y2I7S2KovjMpkMUf7O+CG7AicLazDBV5xBtG73eRzNFddtyatsQkVDK9wc+GdPRESDq1c9LGq1GlOmTEFycrJ0zGg0Ijk5GdOmTbvKlZaMRiNaW1uln81hJTs7G7t27YKrq2tvmkW9oFEpOoUVs0g/MaScLKgBIA4PrdudDQCwVSkAACfyawa8jURERJfr9Uq3iYmJWLFiBWJiYhAbG4u1a9eisbERCQkJAIDly5fD19cXSUlJAMR6k5iYGISGhqK1tRXbt2/Hpk2bpCEfvV6Pu+++G2lpafj6669hMBikehgXFxeo1er+elbqhrmuxbwx4t/2XYRRAO6c5AuVQo4txwuQll+NueM8r3IXIiKi/tfrwLJkyRKUl5dj9erVKCkpQXR0NHbs2CEV4ubn50Mub++4aWxsxMqVK1FYWAhbW1tERERg8+bNWLJkCQCgqKgI27ZtAyAOF3W0Z88ei2EjGljR/s4AgHOl9Sivb8X+7HIAwMOzQ5BRUIstxwuQmlcNAGjStaG0rhXBbvbWai4REV1Her0Oy1DEdVj6T9wfdqG0rhX3Tw/CB4dyEeRqhz2/mYML5Q2If3s/NCo5Tr00H/e/fxSHLlTi85U3SkEns7gOHo42cGWNCxER9cCArcNCI595WOg/R/IBAPMneEEmkyHETZxF1KI34l8HcnDwfCUEAdh5Why+yyiswcK//oBH/51mraYTEdEIxsBCFsy9JTqDuBbLgvHi7C+5XIZJAaMAAG/tzJLOP2jaMPGbjGIYBbFQt65FP4gtJiKi6wEDC1kwzxQCAC8njcWKuFMCxcDSZhSgVoj/6ZwqqkV1ow7JZ8W9pAQBUp3LpZpmfJZayA0ViYjomjGwkIVIX2fp+/njPSGXy6SfJ5t6WABgaaw/Rns6QBCAT44V4HxZg/TacdO6LYn/Tcczn57E1hNcIZeIiK4NAwtZ0NqpMNZbLHxaGOlj8Vp0gDMcbJTQqOR4ZE4objQt579+z3kAgMIUbo7lVKOktgWHL4rBJTnTcqFBIiKi3mJgoU42/nwyNj0Yi9hgy+X7HWyU2LpyOrY9PgPeWltp/6GG1jYAwH1TxT2d0gtr8EWHfYcOnK9Am8Fyf6LLFdU04453D2LbyUv9+ShERDRCMLBQJ4Gu9pgZ7t7la6M9HTHaU9xEMS7EFcoOQ0a/nBkCNwc1dG1GbNh7QTpe39KGdNPquVfyeVohTuTX4A/fZMLAmhciIroMAwv1mYONEpMCnAEAIe72CHazR0yg2CtT26yHTAbEBok/7ztXftV7nSoSV9ctqWvBAdPMo8Qt6Zj5xm6U1bcM0BMQEdFwwcBC12TBBG8AwOIoXwBATFB7Ye7UQBfcE+MHANh/rhwGo4CPUnKxxzSjqKMfi+qk7/+XWoj958qx9UQRCqqa8enxwoF8BCIiGgZ6vTQ/UUcJ04MwKcAZkabdnTvWvSyM9Mbs0eLQUkZRLX616Th2ZYph5aGZwXhuQQSUCjmqG3UoqmmWrtt5ugRni9sDzGephVg5JxQyWfvwExERXV/Yw0LXRC6XYXLAKChN67KM83aCp5MN7NUK3DrRCx5OGkR4OUIQgF2ZZVLNyz9+yEHCB8fQZjDix0vicFCgqx0ivByhazMiu6wBjholbFUKXKxoxIluamCIiGhkY2ChfqVUyLF15Y34+smZ8HDUAABmjxF7WWyUcvxzRQzeXTYZtioFfsiuwJ6scmk4aIKvFndP8ZPu9dhNYbh1grjS7v9Sux4WOpZbhb/syu6yUPfx/6Rh2T8Ps4iXiGgE4JAQ9TtfZ1uLnx+cEYy65jbcPcUXU0xFual51fjXgRx8mV4Ec5yY4KPFHZN8sX7PebjYq3H/9CCk5lVj64kifHXyElb/dBw0KoXFvZ/fegrZZQ0Y7emAWyd6S8dLalvwdUYxAKCgqglB3FWaiGhYYw8LDTgPRw2S7pwohRUAuD1aLNL9/kwpUnPFpfwn+mrh6mCDvc/ehK+emAGNSoFpIa7w0WpQ39ImBRCz2mY9sk0r7F4+bfpkYfvPlzrUxxAR0fDEwEJWMcHXCSFu9mhtM6KkTpy2PN5HXGFXa6uCnVrs/JPLZVgaGwAAePWbMyiubQ8fpwprpe8zOnwPACc7BJhLtZwWTUQ03DGwkFXIZDIsNvWyAOIw0ih7dZfn/mp2KCb6alHTpMdTn6RLNSkde1F+LKq12GSxY4ApZg8LEdGwx8BCVrM4un2vogm+Tlc8T62UY93SSbBXK3A0pwob9op7F3UcBqpvbUNOZSMAQBAEZHQcEqq9cmB5/D9pmPv2PtS36Pv4FERENBgYWMhqgtzsEe3vDECsX+nu3N8vngAA+NeBHLS2GaTAYq8WC3HNQ0S5lU2oa2mTrr1U0/WQkK7NiG9OFSO7rAF7sq6+Ei8A5FY04vVvz6KiobXbc4mIqH8xsJBVvXr7BCyN9cfPbwjs9tw7JvnCy0mD6iY9NqXkoby+FQq5DItMPTXmISJz/Yp5zZfiK/SwFNU0QzCNIvVkR+kNey9g474L+Cglr9tziYiofzGwkFVN8NUi6c5IONt1Xb/SkUIuw11TxLqXtbuyAQARXo6YatqvyNzDYg4u00JdAVy5hyW/qkn6fm9Webc7SpuHnM5cqrvqeURE1P8YWGhYuWeKPwCgoVUc8onyd0aknzMA4PSlOrQZjFLB7QLTonMNrW2o66JGpWNgqW3WIzWv+qrvXWA6/2xJ/weWNoMRrW2Gfr8vEdFIwcBCw0qQm73FfkXRfs4IcbOHvVqBZr0BZ0vq8aNp5+cbQlyhtVUBAIq76GUp6BBYAGB3F5symrW2GaTp14XVzV0GoGvxs38ewew39qK2icW/RERdYWChYefeGH/p+yh/Z8jlMkwwFe0mfHAMrW1GONooEexqDx/TqrtdLR6XXykGlqmmHaaTrxJYLtW0SPUuAJBVUn/Nz2FW3ajD0ZwqlNS1YO+5K7eBiOh6xsBCw85PJnoh0NUO4R4OCPNwAACpjqW8XpzBc/skX8jlMvhoxf2MLtU2S9OdzUMv5iGhn98QCKVchvNlDcgz1alc7vLeGPNu0p8eL8DnJ7re56inMjvsTL3nKqGJiOh6xr2EaNixUyux8+lZUMhlUJhmAj08OwQeTjbwcNQgyl8Lb63Ys+LtLAaW4poWfHKsAKu2nsIjs0Px3IIxUggZ76PF1CAXpFysxFvfncNf74uGTCazeM/8ywJLZkk9zpfV49n/ZQAA5KaF8L5ML8K7ey5gzW3jMD3MrUfPk9mht2Z/dgUMRkF6LiIiErGHhYYljUoBlaL9P18njQrLpwVhwQQvKawAaB8Sqm3G1jSxJ2RXZilqmvSoNxXu+o2yxa/njoZSLsNXJy/h3b0XOr1fQbUYWNwcbACIvSLb0i9Jrz/3WQZe//YsnvokHVml9VhvWtyuJzr2sFQ16iwWvSMiIhEDC41oPqbwcqqwFsdNs4DOlzXgRIH4vZeTBhqVArHBLvj94vEAgDd3ZnVal6WwSqyBmTvOA4BYw7LtpBhYPJ1s0KI3YuO+9qBz6EIlyup6toeRObA4acQOz709WMSOiOh6w8BCI5q3qYYlu6zBomj2f6lib0uAi510bFlcIH5hWsAu6duzEDpcYO5hmRXuDrVSjiadAbmVTdCo5Ni68kbpPo/OCcXkAGcIAvDVZbtLd0VvMCK7VNxxesX0IADA3izWsRARXY6BhUY085CQmZ1pGf/vz4g9KP4dAgsAPLtgDGxVCpwva8CRnCrpuLmGJcjNHqM9HaTjt4z1hK+zLb5+cga+enwGnlsQgdsniYvbbUsvgiAI+OJEkTQcZWYu/L1Y3gidwQgHG6W02m9GUS2X/yciugwDC41oXloNOtbPPjI7FACgN4i9JwGXBRYnjQq3TxKX+t98WFyCv75FjxrT+ij+LnaI8GrfqHFRlI903UQ/cWr1TyZ6QyGX4WRhLZ7570k8vSUdif89iUJTL01yZinG/N8O/H3/BWk4KMLLEZ5OGoz3cYIgAKu//FFaHO9yJwtq8OjmVOl+RETXAwYWGtFUCjncTYWyoe72uC/W3+L1AFfbTtcsixN7OnaeLkF5fSsKTPUro+xUcLBRYqy3GFgcNUrMGePe6Xo3BxvMMM0Q2nqiSDp+ILsCALDlWAEAcXuBA+fFY+Z7Pn5TGJRyGbafKsGidw50uQ3A29+fw7c/lmD9ns7FwUREIxUDC4145mGhBRO84OGoQYi7vfTa5T0sgLi/UbS/M/QGAf89XiDVr5jPnT/eE4GudnjspjDYKBVdvudi04aMchkwOcAZAPDD+Qro2ow4aAopTTqDVEsT4e0IALh1oje2/OoGeGs1uFjeiEXvHMCfvz8HXZu4z5GuzYhjueJQ1XenS2AwCiAiuh4wsNCI94sbAhETOEqqEYnrsLT/5TUsZuZzPzyUKy3172c612+UHfY9e5M0vNSVRVE++HX8aHz4QCye/8lYAMCh8xU4nluFRp0BaoXlXz1zDwsATAl0wTdPzsT88Z5oMwr4S3I2HvzwmLTwXZNOrH+pNK2QS0R0PWBgoRHvril++N+j06X1WeKCxV2cbVUKabjocj+N9EaQqx3K6luldVn8R3UdbrqiVMjxVHw4Zoa7I8rfGQ42SlQ36aV7/TTSW9oTSSYTa1g6crFXY+PPp+Cdn02CWiHHD9kVOH2pDgfPV1qc9+2P3c9EIiIaCRhY6Lpz0xgPhLrb4/ZJPp1WtDXTqBR452eToVbIpWEXf5fO9S49oVLIcUOIGE7MNSuzx7jjt/PHQC4DIv2cYafuvOi0TCbDTyN9MHe8JwBxG4BDF8Tr48eK68Hs+LEExmscFtp3rhwL//oD3tx59pruQ0Q0kBhY6LqjtVMh+Zk5SLoz8qrnTfDV4sXbxkk/96aH5XIzOizTL5MBM8PdERPkgh1Pz8K/VsRc9dp7pvgBAL5Iv4QT+TUAgN8uiICjjRJl9a1Iy6/uU5ta9AYkbknHiveO4vSlOnxwMPeaww8R0UBhYCG6ip/HBeDBGcGYGjQKMaZdnftiRnh7YInyc4aLvRoAMNrTUVru/0pmhrvDy0mD2mY9dAYjfLQahHs44BZTL8vLX5/BP3+4iPNlDZ2u1bUZ8ebOs/gyvahTGPngUC62niiCTAYo5DI06gzIucLmj0RE1sbAQnQVMpkML/50HD59ZHqXwzY9FeruAC8ncdXdrqZCX41CLsOdk32ln6eFukEmk+GOyWLPS0ZhLV79JhNz/7wPT358AhfL24PL3qwyrN9zAU99ko47NxySCogBINW0VcGz88cgyrSGTMfXiYiGEgYWokEgk8mw8qZQjPV2wj0x/t1fcJm7TcNCAHBjmFg0PHu0O7547EY8tyACs0a7QxCAbScv4bZ1B6SdqM+Vtu8EnV5Qg2X/PCItSGde42VywChM8BUDy6nC/g0sgiCgvkXfr/ckousTAwvRIFk+LQjfPjUTvs69L94NcXfAnZN8Eexmj5sjPKTj0f7OeHROKD56IBbfPDkDIe72aNQZpLVezMNE908PkoaVTuRXo7pRh6IacUG8cT5O7YGlFz0sX6YXYVpSMk4W1HR6zWgU8N3pEixefxATX/oOX3RYQI+IqC8YWIiGibeXRGPPb+bA2U7d5evjfbS4xRRmzpaIPSvnTcND00JdMS1U7Jk5llstbQkQ4GInbitgCiynL9VdsfD2Uk0zapvae0s27L2A4toWfHw0v9O5q7aewsObUpFh6rF5/1Bubx+3x8rrW7EpJVfan4mIRiYGFqIRxLzPUWaxGDwulIlFtGEeDlLR8PHcKpw2DQeN9xHPD/dwgI1SjobWNuRVdd6jaGtaIWa9sQd3bjgIg1FAQVWTFIpSLlquDVNa14JPU8XtBx64MRhymbj/UX5l/+99JAgCVv47FS9+eRrvHcjt9/sT0dDBwEI0gpiX+D9bUo+immY06w1QKWQIdLHD1CBxLZgT+TU4WVgDoD2wKBVyRJhW2718WOjv+y8g8b8n0WYUcKG8Efuzy7Ers1R6Pa+yCZdMw0sA8MWJIhgFICZwFFbfNk7q2fn61KWrtr2+RY97N6bg1a/P9Hh69e6zZTiWKxYPd2wTEY08DCxEI0iYhwMUchlqm/XSInXBbvZQKuQIc3eAk0aJZr0B358Rf7mP82nfEmCir/h9x5lC7+zOxh+2iwvKmRfO+/hIfqdwkHJB7GURBAGfpYn7I91lKhT+aaS4r9JXJ8VVedsMRjTrOg/f7D5bhqO5VfjngRz8/qvTEISrhxaDUcAbO7Kkn0/kV6OqUXfVa4ho+GJgIRpBbJQKhJo2d/w6Q+zRCPNwAADI5TLEmHpZWk2bKY730UrXTrxsptCmlFy89d05AMBvF4zB+/dPBQAkny3DkYviHkYLxnsBaB8W+rGoDudKG6BWyvGTid7SOUq5DJnFdfjH/ouIeW0XFv71B7ToLUNLx32RPkzJw9vfn7vqs35xoghZpfVw0igR7GYPowDsO1fWwz8pAsSAmZpXjdpmzuSioY+BhWiEMdexmHs9wtwdpNc6Ln7n5qCGh2P7onXmmUInC2vwyKZUrN52GgDw5M1hWDknDGEejogNcoHBKKDNKCDcwwH3xfpbvJe5d2XeOE9obVUAgFH2amnhvNe2Z6KmSY+LFY3YfdYyXJh3oZ43TtyKYN3u89L07MsJgoB1u7MBAI/OCcOtE8TgtPtseU//mK7KaBS63Qk7Na8aecN8ob3DF6tw14ZDeH7rKWs3hahbDCxEI4y5jsX8+zbUoz2wmOtYAGCcj9ZiL6VwD0fYqhRo0hmw43QJBAFYMS0Qv547WjpnaVz7GjLx4zwxNcgFSrkMRTXNSLlQiS/TxenLd3VYNwYAbjMNC4nvI7bHfC4A1DTpcK5UnNH0hzsnYqopWJmnZ18us7geuZVN0KjkWDE9UJrqvS+rDG0G41X/fLpTVteCya9+j8mvfI+V/07FztMlnc45X1aPe/+Wgl9+ePya3svazpaIxdcd1+shGqoYWIhGmLHeThY/h3UILBN9tVArxL/2430sz1Mr5Vi/bBIenhWC1T8dh80PxuGlReMtQs2tE7ylbQXmj/eCvY0SkaZVcpf98zCqm/QIdLXDzA57JwHA4mgf/O7WCHzy8A3469JJAIA9Z8uloYjjpsLZEHd7uDnYYFqoeP2hC5YzkMzMNTgzwtxhp1ZiUsAoONupUNfShjTTfkt9lXKxEjVNetQ267H9VAke2ZyKsvoWi3P2nauAwSggu6xBWojvavIrm3DoCuHLmkpqxecqq2+1ckuIusfAQjTCjPVqDyIymbgtgJlGpZCGhaYEdN4b6eYITzz/k7F4YEYwZoS7ddrNWqNSYNODsfj7L6Yg2t8ZADDdFC6MAjDB1wkfP3QDlArLf1qUCjkemR2KG0JcEeHliNGeDtAZjFLvhXk4KNbUAzTdNLMo5WJll8W332eK15mHjxRyGWaPFrc8SD7bXhD85+/P4d6NKajuRTFutqmn5+YID3g5aSAI6LRPU0qHIJVT3v2w0Mr/pOJn/zyCDNPsrKGipE4MLLXN+k41RURDDQML0Qjj6WQDZzuxfsR/lB00KoXF62/eE4X1P5ssbZ7YW+N9tJhnKrYFgEXRPnCxV+OuyX749FfT4dPNSr4ymQyLosQhom3pYmHwUVNgMQ9ZTQpwho1SjvL6VlwotwwLl2qa8WNRHWQy4OYOzzDXFF4+Sy1Ci96AvMpGrNudjaO5VfigFwvXZZeJwyMzwtykWVQ5Fe2hxGAUcCSnPbBc3r7LNesM0ro3B8+3X5eaV9VlsWtpXQvWfPmjtBLxQCqube85KmcvCw1xDCxEI4xMJkOEl1jH0nE4yMzX2RYLI7079Z701WhPR6T+Xzz+dG8UbNWK7i8AsChK3Mzx0IUKnMivlqZSxwaLgcVG2d4TlHLZsJB5SvWUgFEWO13PH+8FX2dbVDS04tPjBfj7/otSHc/mw3mdehCadYYup1ebe1PCPR0Q5CrOuOrYi5JZXIf6lvZhoK4CS0FVE3SmmVjZZfUwdxIdNwWzH7LLcdeGFNy+/mCnqdjv7jmPD1Py8Mdvz3a6b38r6RBYOCxEQx0DC9EIFOnnDAAYayrAHWi9DT8BrnaIDXKBUQDuePcQ9AYBnk428BvV3jszLUQcFrq8jsVcvzJvvKfFcZVCjkdmhwAA1u+5gE9TxRlL9moFKht1Um8OALS2GbDgL/tx61/2Wyzpr2szIte0Im+YhwOCTVPEO/awXB6gLl42JLQ3qwwz39iDP2zPBNC+TQIAHM+rhtEo4JuMYum+v/zwmEWYMtfg7Mkqk0LP5YxGAXuzynpUP3MlgiBIQ0IAUH5ZnQ7RUMPAQjQCPTI7FGtuG4eHZoZYuylX9Jel0YjvMKQzNcjFIviYC28PX6yUVr6ta9HjsGnNl7njvHC5e2L84eZgg5K6FujajIjyd8aTt4QDAN47mCPVwxzLqUZeZRNyK5ssAlFuZSMMRgEONkp4OWkQ4mYKLB2mL5vf31xnc3kPy6fHxaC048cSCIKArA6BpbZZj3Nl9dKUbqVchrT8Gjzz6UkAQIveIO3zVN/SZjH01NF7B3Nw//vH8OcOa9Ucy62SglBPVDfpLQIRe1hoqGNgIRqBXOzVSLgx+IobJQ4F3lpb/HPFVLyfMBW3Rflg5Zwwi9cj/bSwUytQ3aSXeikOZFdAbxAQ6m6PYFOY6EijUuCXM4Olnx+dHYL7YgNgp1bgbEm9VEPScYG57063F+mah4NCPRwgk8mk98ivbEKbwYg2g1Fa4G5ZXCAA4GJFo7RmS4vegD1Z4r1L6lpQVNNsEVgA4MNDuSirb4W9WoEPEmKhkMvwTUYx8iobcfpSLdo6rP9ibltZfYvF9gfmUGQeYjIaBTz80XE89p80ZPdwinJxrWWNTFkdAwsNbX0KLOvXr0dQUBA0Gg3i4uJw9OjRK567detWxMTEwNnZGfb29oiOjsamTZsszhEEAatXr4a3tzdsbW0RHx+P7OzsvjSNiIaZm8Z4YN3SSRbbBADiEE+cqaZlt2nmj7lnwrzuSleWxQUg3MMBccEumDvOC1pbFe4xrQvzUUouAGBvVvsCc9+fKZUCh3mGkHmtGC8nDTQqOdqMAgqrm3GmuA71rW1w1Cgxd5wn1Ao5dG1GKUz8kF2Bpg51Mal51cgyBYiZpsXz/msKGzPD3TEj3E16xl2ZZThhGg4yF01/f6YUeZWNmPfn/Zj79j4U1TQjs7hOumdWaT0MRgF5VU2oNu2knZZfbfHnUdeix+vfnu20q3bH+hUAnaZu91ZqXpVFqCLqb70OLFu2bEFiYiLWrFmDtLQ0REVFYf78+Sgr63pJbBcXF7zwwgtISUlBRkYGEhISkJCQgJ07d0rnvPHGG/jrX/+KjRs34siRI7C3t8f8+fPR0sIxVaLr2a0TxOX9vzpZbKrbEIPGTWOuHFgcNSp8nzgbW341DQq5OMT08xvE3pDks2VIzatGdlkD5DLAwUaJioZWpBeIv+TPl1sGFrlc1l54W9Eo7c8UG+QCtVIu9cCYr9vxozjdWml63+/OlEqzb34WGwAAUjgyz3Ayh6/dZ0uRXlADAFh+QyDs1AqU1LXgvr8fRk2THo06A9YlZ+PLDrU4LXqj1DNjll7Q/v2J/Gr85C8/YOO+C1i19ZTFPlHFnQKL2M4mXVuXM5TaDEb8/J9HcM/GQ9BftjjfxfIG3LVh+C+kR0NbrwPL22+/jYceeggJCQkYN24cNm7cCDs7O7z33ntdnj9nzhzccccdGDt2LEJDQ/HUU08hMjISBw4cACD2rqxduxb/93//h8WLFyMyMhIfffQRLl26hC+++OKaHo6Ihrf5E7ygVsiRVVqPrSeKUNEgDqXEdFixtyfCPdu3FfiNqV5kcsAoaWq3eejFPJzScXZViKnw9mJFo1Twe5MpZJhfu1DWAL3BKM1gMgek70zrzPi72GJGuBtMOQYyWXvoih8rFg8fuViFw6Y9mm4IcZXWlSmubYGjRgkA+DS1EJ8eLwDQHoqySuqladMAcNIUek7kV+OejSkorG6W3veNne2bRZp7WMyFzuYhoWf+exKz39hjEW4A4H+phThwvgLHcqtxpsP7AZB2/84qrb/mlYaJrqRXgUWn0yE1NRXx8fHtN5DLER8fj5SUlG6vFwQBycnJyMrKwqxZswAAOTk5KCkpsbinVqtFXFzcFe/Z2tqKuro6iy8iGnm0tirMHiP+4n71mzMAgBnhblArez+avewGsYfDPONn9mh3zDMV7u48XQKDUcBF02vhHu2zq8y9KIcvVkpDNuY1X8yL8l2saMSRi+K6Kq72ajwyOxQAoDeIvSljPJ3gqFFJqxBH+TnD3bSPU5CbPULc7dFmFFDR0AqZDJjop5VmQSnkMvxjeQzix3rAYBRQ2aiDo41S2lwy87LAklVajxa9AZ8cLUCbUcCNYa7Y9vgMKOUy7D9XjkMXxF4i8wyhKNOMsvKGVhiNAvadK0ebUcA3p9oLeFv0BvwluX2Y/vJhJ/NQmsEo4FINe8ZpYPTqb31FRQUMBgM8PS2nE3p6eqKkpPN+G2a1tbVwcHCAWq3GwoULsW7dOsydOxcApOt6c8+kpCRotVrpy9/fv8vziGj4My8yV2Oq0bha/crVLJjghVGm2hAAmDPGA7PHuEOtlCO3sgnvH8yBrs0IG6Ucvh2mVwe7iaHE3HsS7e8MTycNACDUQwwzWSX1+PsPFwGI0629tBoEuNhJ9zCvizPHFL5+Gult0TZzLwsgDkc5alRYONEH908Pwrqlk3BDiCuemTcG5klU8yd4SSsNZxbX4YxpSEgmE0NDRmGttOLvI7NDMcFXi5/FiYHtjzuyxCnNph4W89YKlQ2tyKlslGpw9nTYnHLz4TyLIaQTl21/kN1hJeD8K2xYSXStBmWWkKOjI9LT03Hs2DG89tprSExMxN69e/t8v1WrVqG2tlb6Kigo6L/GEtGQEj/WE3YdFqSbc5X6lauxUSpwT4z4PzduDmqM93GCg40SPzX1VLz6jbhuSqi7g1T7ArT3sJgXf5vfYZXfEFOYSc2rxv5z5VAr5dLsoY47Y482BZYnbg7Hfx6KwwM3ts9kAixDmDmIqJVyvLRovNSTMtbbCT+PC4RKIcPPbwiUQtCRi5WoaNBBLgNuNE0F//BQLioadHDUKBEX7Cq9t51agZMFNTh8sUqaJTTeRwu5TNxa4Ydz7cXIZ0vqUVzbjPoWPdbvOQ8AuM0UHi/vYem4dUFe1fDewZqGrl4FFjc3NygUCpSWllocLy0thZdX5zURpDeRyxEWFobo6Gg888wzuPvuu5GUlAQA0nW9uaeNjQ2cnJwsvohoZLJVK6QhmHHeTlLvRl88OCMYMYGj8OQt4ZCbQskf7pyIpbHtvbThnparA4dcNn2644J15hoWQKwpefdnkzHBV+yxiAlsr7MxhwuNSoHpoW7Se5tNCRwFJ1OdyqQu9ngye3nxeJx6aT6i/Z0xxnTPOtOqu6HuDtKMI/Nwzk1jPKThM3dHGyw0hZ8dPxZLPSw+zhppxeB9HQILIM6m+ucPOahu0iPEzR6vLB4PmQworG6WZhWZt0Ewy69kDwsNjF4FFrVajSlTpiA5OVk6ZjQakZycjGnTpvX4PkajEa2tYoFXcHAwvLy8LO5ZV1eHI0eO9OqeRDRyPTwrBCFu9vjV7GtbCM/TSYP/PTody6cFScc0KgWS7ozE63dOxGhPB9w12c/imlH2ammacai7vcVmko4aFcZ4OkImA/50bxTix7WHmdhgMXjYqhRdrhnTkUohR+Lc0YgNdsGC8Vf+nz+ZTCbtDeXqYAMPx/atCcb7OCHK1DtjdvlqwAsmiPfedvISGk1DP15ajVRPk2JaFM9bK4bCz9OK8E/TUFfivNFwtlNjjKcYlMzDQhfLG9Fh6RgOCdGAUfb2gsTERKxYsQIxMTGIjY3F2rVr0djYiISEBADA8uXL4evrK/WgJCUlISYmBqGhoWhtbcX27duxadMmbNiwAYD4F/Dpp5/Gq6++ivDwcAQHB+PFF1+Ej48Pbr/99v57UiIatsb7aLH7N3MG9D3uiw3Afaapx5cLdrPHifwai00fzf79UBxqm/UWQQYAwjwc8dY9UXCxV0Gl6P7/De+/MRj3XzZU1J0IbyeU1Yu9IuN9tFI9CgCoFO07WJvNCHeDg41SWrNFa6uCnVoJD0cbnIY4TRoAHpoZgpe/PiNtSjnB1wk/MU0xnxTgjLMl9UjLr8b88V7SZpHmYaU8Uw+LIAho0Rt7vL8UUXd6HViWLFmC8vJyrF69GiUlJYiOjsaOHTukotn8/HzI5e1/ORsbG7Fy5UoUFhbC1tYWERER2Lx5M5YsWSKd89vf/haNjY14+OGHUVNTgxkzZmDHjh3QaPre9UtE1F8euDEYmxR5+IVpunJHbg42FpswdnT3FL8uj/eXCC9H7D9nDixOcLZTI8jVDrmVTZge6gZHjcrifBulAjdHeGDbSXEtF3NPioej5b+1d03xw4Z9F6Q1ZJ6dHyENY00KGIWPjxbgRF4NgPb6lSmBo3Astxr5VU0QBAF/TT6PvySfw8cP3YA4075QRNei14EFAB5//HE8/vjjXb52eTHtq6++ildfffWq95PJZHj55Zfx8ssv96U5REQD6rYoH6ngdCgx18YAkFYKnjXaHbkpebh9UtftvXWClxRYvMyBxak9cAW42EFrq8Kc0e74NLUQccEumGVapRcQ168BgIyiGugNRmlK800RHjiWW42G1jZUNerw3+MFMAri+i0MLNQf+hRYiIjI+iYHjIJSLkO4p6O0b9Tvbo3AHZN8pdlGl5s9xh0alRwteiO8nMw9LO2BxbzD99NzR0OjUuDhWSEWm1KGuNnDSaNEXUsbzlyqwznTkNBEXy28nDQoqWvBgfMV0mq5+86VQxCEXu/oTXQ5bn5IRDRMBbnZ48vHb8T790+VjtmplZgUMOqKAcFOrcSc0eI0an/TWjHuHYaExnmLdTC+zrZ45fYJ0jlmcrkMN5h6TF7bninVrIR7OCLAVTz330fa9y0qq2+VNq8kuhYMLEREw9h4H600tNNTLy0ajydvDpO2EOg4JHT5JpRdWfWTsbBTK3A0pwoGowBHGyU8nWwQaAo35h2tzS6fLk3UFwwsRETXGS+tBonzxkBrKxbldjUkdDXBbvZ46bbx0s9hng6QyWQWq/sCwOJosY5mXxYDC107BhYiouucj9YWU4NGYdZod/g623Z/AYB7Yvzwk4niNO+JpsXyzENCAOBoo8QTN4cDAI7nVaGhta2fW03XGxbdEhFd5+RyGT59ZHqvrpHJZPjTPdG4OaJY2log0LV9gbwbQl0R5uGAQFc75FU2IeVCpbRiMVFfsIeFiIj6xFatwN1T/OBiL85Q6jgkNCNMnAptXrxub1ZZ5xsQ9QIDCxER9YtRdip4azWQy8T1YID2Har3nC2DIAhXu5zoqjgkRERE/UImk+G9+6eipkkv7Z80PdQNGpUcl2pbkFlc36NZSERdYQ8LERH1m7HeTpgW2r6yrUalkIaHkjNLrdUsGgEYWIiIaEDdMlYstk0+yzoW6jsGFiIiGlC3mGYRnSyskTZUJOotBhYiIhpQHk4aRPppIQhi8a1ZVaMOmw7noUVvsGLraLhgYCEiogFnXqtlV4c6ljd2nMWLX/yIzYfzrNUsGkYYWIiIaMDFm+pYDp6vQJvBCAA4livuOZTFzRGpBxhYiIhowI3zdoKjjRKNOgPOltSjvkWPixWNAIC8qiYrt46GAwYWIiIacHK5DJMCRwEA0vKrcaqoFuZ15PIqG63YMhouGFiIiGhQTAkQA0tqXjVOFdZKx0vrWtGsY+EtXR1XuiUiokExJbA9sLQZLJfpz69qwhgvR2s0i4YJBhYiIhoUUf5ayGVAYXUz6lvaAABKuQxtRgG5lY0MLHRVHBIiIqJB4ahRYYyXuJdQbbMeAHCjadn+/EoW3tLVMbAQEdGgmRLoLH0f4maPib5aAEAuC2+pGwwsREQ0aMx1LAAQ6adFoKsdALGGhehqGFiIiGjQTAlwkb6P9HNGoKs9APawUPcYWIiIaND4u9jCR6sBAMQEjZJ6WIqqm6FrM1qzaQPq0PkKPPafNNQ26a3dlGGLs4SIiGjQyGQy/O0XMcitbESknzMEQYBGJUeL3oiimmYEu9kP6PvrDUb87rNTCPWwx8o5YQP6Xh29tj0Tpy/VITbIBSumBw3a+44k7GEhIqJBNdFPi9uifACIASbQpX+GhVr0Bvxt3wUcPF9xxXMOnK/AZ2mFePu7c/22WJ0gCPixqBZ/2J6JpX8/jLT8aovXKxpacfpSHQDW6lwL9rAQEZFVBbraIau0/pqmNhfVNONXm47jx6I6qBVyfPHYjRjn49TpvL1nywAAbUYBGYU1iAtx7fN7mr3ydSbeO5gj/fz+wVxMDmgvLu4YoAoYWPqMPSxERGRV5jqWvvSwCIKAbzKKsWjdAfxYJPZi6AxGPPFxGpp0bfjudAne/v4cGlvbIAgC9mSVS9emXtYT0hcGo4D/pRYAACb4igHp3GW7T/+Q3SGwVDf36v5tBiMEQej+xOsAe1iIiMiqzDOFDl+sQmubATZKRY+uy61oxG8/y8DRnCoAwHgfJ/zxrkg88MExXChvxPTXd6PGVOTarGvD0tgAiyGZtLyaa257RmEN6lra4KRR4t2fTcGsN/fgQnkDdG1GqJVyCIKAH7LbQ1JhVRMEQYBMJuv23vUtevx03QG4Odjgf49M69E1A8FgFPDwR8cR5umAJ28Oh72NdaIDe1iIiMiq5oxxh6ONEpnFdXj20wwYjT3rUXjqkxM4mlMFjUqOp+PD8dmj0zHBV4u1S6IhkwE1TXqoleKvuc2H8/G/1EIAgJuDGoC4a/S19l6Ye09uDHODv4stHGyUaDMKyKkQe4uyyxpQWtcKG1M76lvbpFV+u/NNRjHyKpuQmleNS7Ut19TOa5FX2Yjks2X48FAuNKqehcmBwMBCRERW5TfKDht+PgVKuQzbTl7Ck5+cwObDecgsrrviNZnFdThZWAuVQoadT8/C0/GjpV+m08Pc8M7SyfjtgjE4vOoWRPpp0aw3YMO+CwCAB2YEQ62Qo6pRh7xu6maO5lThL7uyUVbfdWAw957MDHeHTCbDaE8HAEBWqTgstP+c+HpciCvcHGwAAAVVPRsW+iytUPo+o6CmR9d0pGsz4n+phVf9c+yKIAjQG9qnmJsLhiO8nKCQW6eXB2BgISKiIWBGuBtevysSAPB1RjH+74sf8ZO//oDvz5R2ef6nx8Vf5rdEeEpDSh0tjPTGyjlhcLFX44mbwwEA5s6U+eO9MNFP3BIgNe/KdSxtBiNW/jsNf951Dre8tQ8fHMyBoUPvT32LHmn5NQCAmeHinkjmDRzNdSwHTAW3s8LFHhgAKKgWQ9LHR/NxILvrGU25FY04ltvetpOFtRavG4wCNu67gFOXHTc7c6kOi9cfxG8+PYm7NhzCj0Vdn9eVv+2/iNH/9y1SLlSK9zIFnq6KmAcTAwsREQ0Jd0/xw/v3T8X904MwKcAZggD83xenUNdiOYSiazPii/QiAMA9MX7d3jd+rAfGeou/bANc7BDiZo/JAc4A0GkKckeHLlSioqEVgDiU89JXZ3D/+0dR3aiTXjcYBYS42cPfRSwcHuMpBpas0no06dpw+KL4S39GuBv8R4nnFFQ1IaOwBqu2nsJj/0nrcghsq6l3Ra0Qf02fKqqxeP2bU8V4/duzePzjtE7DWt+dLsGidw5IPStNOgPuf/9Yj2YoGY0CPjiYC0EAvjT9GZ8x9bCM82ZgISIiAgDcFOGBlxaNx8cP3YBgN3uU1rXita8z8WV6Ee77ewpe/foMvkwvQlWjDu6ONpg92r3be8pkMqy6NQIqhQxLYwMgk8mkPY3MPSRdMYein8UF4JXbJ8BWpcAP2RW47Z0DSM4sRXKm2Ptj7l0BgNGmHpasknrsPluGFr0Rga52GOPpaNHDYg4ytc36TrOjjEYBn6WJ750wIwgAkFFYaxFszL0feZVNONFhuKi8vhXPfZaBNqOA+LEe2PObORjr7YSKhlYkfHDMYqinK+mFNSipE4e/juaKxczmIaHxVu5h4SwhIiIacjQqBZLunIj7/n4YW44XYMtxcerw4YtV0jl3TvKFUtGz/++eNdodZ15eAJXpfPM6KVkldahv0cNRo7I4v0VvwM4fS6T3iQlywdSgUfjVplTkVTbhwQ+PS+fODG8PTeYelvyqJmnY6icTvSGTyTr0sDSjpEMR7amiWoS4O0g/H86pRFFNMxxtlHj8pjB8cDAX9S1tyKtqklYCPpJTKZ3/5YkiTA4YBUEQ8Pznp1DdpMc4bye8u2wK1Eo5PkiYinl/3o/zZQ04WVCDmKD2/Zwut8P0zABwsbwRmcV1qGhohVwm1rBYE3tYiIhoSLohxBXL4gIAAK72ajw6JxSh7u31Kj0ZDupI1SHceDhpEORqB6MAfHuq/Zd0i94AQRCwK7MUjToD/EbZSr0xEV5O2PbYDKyYFghPJ7GA1tlOhRtC2xefc3WwkWYh7TMV3C6c6A0A0rBRflWTNBUbEHtPOvomoxiAGHQcNSqpdiSjsAaA2Itysby9V+brjGK0GYz4LK0I358phUohw5/ujZJmSHk6aTDNtECeudekK4Ig4Nsfxfc219Z+lJILAAh2s4et2nozhAD2sBAR0RD28uIJuHOyL8b7aKFRKfDr+NH4/EQh7G2UCPNwvKZ7/ywuAH/Yfhb/PHAR98T44eD5Sjzw4TF4azVSuFkU5WOx/onWToXfL56AlxaNx9mSejjZquBw2bokY7wcUXFe7AEJcrWThlLMPSzmKc9mpzoUxBqMAnaeFgPUTyLFoBPpq8WJ/BpkFNZicbSvFHbCPRxQ1ahDZaMOb+7MwvsHcwEAT90SLtXsmE0NdsGO0yU4nnvlmp3Tl+pQUNUMG6UcCyd6Y+uJInx+QhyaGu+j7e6Pc8Cxh4WIiIYshVyGKYEu0pRltVKOJVMD8NNIn2u+95KpAbBXK3CutAHbT5Xg2f+dhK7NiLzKJpwvawAA3D7Jt8trZTIZxno7wdfZttNroz3bg9TCSG8p8Hg7a9BxVrD52tNFtdLso+O5Vaho0MFJo5R6RSL9nAG097AcNQ0H3RjmhoWmUPO3/RehMxixYLwXHpkd2qlNU4NGSfe/0jo35uGgOWPcMXuMOMzVohdrXqw9QwhgYCEiouuU1laFJVPFIaent5xAcW0LAlzs8MZdkZgZ7oYHbgy2CB89NabDNT8xDQcB4pCUt7Y94Nwb4w9blQKNOgNyKsSA9K0pNMwd5yUN6USapmD/WFSHNoMRR0w9LLHBLlgc3R6oZo92x1+WRndZ1zPO2wn2agXqWtqkNWIAscD3jR1nsWDtfry79zwA4NYJ3ph6WZ2LtWcIARwSIiKi61jCjUH44FAO9Aax1+GPd0ViWqgr7p3q3+d7TgkcBblM7Gm5/Be9v4stimrEheOmhbpif3Y5UvOqxcJbNwephuQnE72ka0LcHWCvFoPN+wdzcda0xktssAtc7dW4N8YPLXoj/nhX5BW3NVAq5JgcOAo/ZFfgWG6VNGR0JKcK7+69IJ031tsJc8d5wt5GCV/n9rayh4WIiMiK/F3ssNA0vPSLGwIxLfTad28O93TEl4/NwEcPxHba/8dcx6JWyhHpp8VEX7H3JKOwFicKalBa1woHGyVmdJgqrZDLsNg0NPXa9kwAQJiHA9wcbCCTyfDG3VH469JJ3RbFmntNOhb8mguDb4nwwOFVt+Dbp2ZKewXFBovnezrZSKv0WhN7WIiI6Lr22h0TsHCiF+LHevbbPc0r6V4uwDRTKNrfGRqVQhruOZpTJdXN3DLWo1NPySuLJ8DVXo11u8VhG3OY6A1zYDmWWyVtwGgOLIuifeCl1VicPzPcDZ+fKJJmSVkbAwsREV3XnDQqLJjg3f2J/eCuKX44nFMpFcaae1jMi7OpFXIsnxbU6TqFXIZn5o3BlMBR+O/xAjw4I7jX7x3t7wyVQobSulYUVDVDo5Ijs7gOMhkwI8yt0/m3R/tCpZAjrg/haCAwsBAREQ0SH2db/PuXN0g/d6xP8XW2xYafT5ZmBXVlzhgPzBnj0af3tlUrMNFXi7T8GnyVcQmeTmKPykRfLVy7GPKRy2W4LeraZ2P1FwYWIiIiK1HIZVizaDwyCmvwzNwxGGWvHtD3WxYXiLT8Gry757wUjHqyvcFQwKJbIiIiK7o3xh+v3j5xwMMKANwxyReTApzRqDMgxbSfEQMLERERDSlyuQy/XzQe5slLjholov2drdqmnmJgISIiuo5E+jljSYy4zsys0e493kDS2ljDQkREdJ1Zc9t4jPdxwrzxXt2fPEQwsBAREV1nbNUK/KKL6dND2fDoByIiIqLrGgMLERERDXkMLERERDTk9SmwrF+/HkFBQdBoNIiLi8PRo0eveO4//vEPzJw5E6NGjcKoUaMQHx/f6fyGhgY8/vjj8PPzg62tLcaNG4eNGzf2pWlEREQ0AvU6sGzZsgWJiYlYs2YN0tLSEBUVhfnz56OsrKzL8/fu3YulS5diz549SElJgb+/P+bNm4eioiLpnMTEROzYsQObN29GZmYmnn76aTz++OPYtm1b35+MiIiIRgyZIAhCby6Ii4vD1KlT8c477wAAjEYj/P398cQTT+B3v/tdt9cbDAaMGjUK77zzDpYvXw4AmDBhApYsWYIXX3xROm/KlCm49dZb8eqrr3Z7z7q6Omi1WtTW1sLJyak3j0NERERW0pvf373qYdHpdEhNTUV8fHz7DeRyxMfHIyUlpUf3aGpqgl6vh4tL++6P06dPx7Zt21BUVARBELBnzx6cO3cO8+bN6/Iera2tqKurs/giIiKikatXgaWiogIGgwGenp4Wxz09PVFSUtKjezz33HPw8fGxCD3r1q3DuHHj4OfnB7VajQULFmD9+vWYNWtWl/dISkqCVquVvvz9/XvzGERERDTMDOosoddffx2ffPIJPv/8c2g0Gun4unXrcPjwYWzbtg2pqan405/+hMceewy7du3q8j6rVq1CbW2t9FVQUDBYj0BERERW0KuVbt3c3KBQKFBaWmpxvLS0FF5eV1/e96233sLrr7+OXbt2ITIyUjre3NyM559/Hp9//jkWLlwIAIiMjER6ejreeusti54YMxsbG9jY2PSm6URERDSM9aqHRa1WY8qUKUhOTpaOGY1GJCcnY9q0aVe87o033sArr7yCHTt2ICYmxuI1vV4PvV4PudyyKQqFAkajsTfNIyIiohGq13sJJSYmYsWKFYiJiUFsbCzWrl2LxsZGJCQkAACWL18OX19fJCUlAQD++Mc/YvXq1fjPf/6DoKAgqdbFwcEBDg4OcHJywuzZs/Hss8/C1tYWgYGB2LdvHz766CO8/fbb/fioRERENFz1OrAsWbIE5eXlWL16NUpKShAdHY0dO3ZIhbj5+fkWvSUbNmyATqfD3XffbXGfNWvW4KWXXgIAfPLJJ1i1ahWWLVuGqqoqBAYG4rXXXsMjjzxyDY9GREREI0Wv12EZimpra+Hs7IyCggKuw0JERDRM1NXVwd/fHzU1NdBqtVc9t9c9LENRfX09AHB6MxER0TBUX1/fbWAZET0sRqMRly5dgqOjI2QyWb/e25z+RnLvzUh/xpH+fACfcSQY6c8H8BlHgv5+PkEQUF9fDx8fn06Tby43InpY5HI5/Pz8BvQ9nJycRuR/fB2N9Gcc6c8H8BlHgpH+fACfcSToz+frrmfFbFAXjiMiIiLqCwYWIiIiGvIYWLphY2ODNWvWjOiVdUf6M4705wP4jCPBSH8+gM84Eljz+UZE0S0RERGNbOxhISIioiGPgYWIiIiGPAYWIiIiGvIYWIiIiGjIY2Dpxvr16xEUFASNRoO4uDgcPXrU2k3qk6SkJEydOhWOjo7w8PDA7bffjqysLItz5syZA5lMZvE1nDagfOmllzq1PyIiQnq9paUFjz32GFxdXeHg4IC77roLpaWlVmxx7wQFBXV6PplMhsceewzA8Pz89u/fj9tuuw0+Pj6QyWT44osvLF4XBAGrV6+Gt7c3bG1tER8fj+zsbItzqqqqsGzZMjg5OcHZ2RkPPvggGhoaBvEpru5qz6jX6/Hcc89h4sSJsLe3h4+PD5YvX45Lly5Z3KOrz/71118f5CfpWnef4f3339+p7QsWLLA4Zzh/hgC6/Hspk8nw5ptvSucM5c+wJ78fevLvZ35+PhYuXAg7Ozt4eHjg2WefRVtbW7+1k4HlKrZs2YLExESsWbMGaWlpiIqKwvz581FWVmbtpvXavn378Nhjj+Hw4cP4/vvvodfrMW/ePDQ2Nlqc99BDD6G4uFj6euONN6zU4r4ZP368RfsPHDggvfbrX/8aX331FT799FPs27cPly5dwp133mnF1vbOsWPHLJ7t+++/BwDcc8890jnD7fNrbGxEVFQU1q9f3+Xrb7zxBv76179i48aNOHLkCOzt7TF//ny0tLRI5yxbtgynT5/G999/j6+//hr79+/Hww8/PFiP0K2rPWNTUxPS0tLw4osvIi0tDVu3bkVWVhYWLVrU6dyXX37Z4rN94oknBqP53eruMwSABQsWWLT9448/tnh9OH+GACyerbi4GO+99x5kMhnuuusui/OG6mfYk98P3f37aTAYsHDhQuh0Ohw6dAgffvghPvjgA6xevbr/GirQFcXGxgqPPfaY9LPBYBB8fHyEpKQkK7aqf5SVlQkAhH379knHZs+eLTz11FPWa9Q1WrNmjRAVFdXlazU1NYJKpRI+/fRT6VhmZqYAQEhJSRmkFvavp556SggNDRWMRqMgCMP/8wMgfP7559LPRqNR8PLyEt58803pWE1NjWBjYyN8/PHHgiAIwpkzZwQAwrFjx6Rzvv32W0EmkwlFRUWD1vaeuvwZu3L06FEBgJCXlycdCwwMFP785z8PbOP6QVfPt2LFCmHx4sVXvGYkfoaLFy8Wbr75Zotjw+UzFITOvx968u/n9u3bBblcLpSUlEjnbNiwQXBychJaW1v7pV3sYbkCnU6H1NRUxMfHS8fkcjni4+ORkpJixZb1j9raWgCAi4uLxfF///vfcHNzw4QJE7Bq1So0NTVZo3l9lp2dDR8fH4SEhGDZsmXIz88HAKSmpkKv11t8nhEREQgICBiWn6dOp8PmzZvxwAMPWGz4Odw/v45ycnJQUlJi8ZlptVrExcVJn1lKSgqcnZ0RExMjnRMfHw+5XI4jR44Mepv7Q21tLWQyGZydnS2Ov/7663B1dcWkSZPw5ptv9mtX+0Dbu3cvPDw8MGbMGDz66KOorKyUXhtpn2FpaSm++eYbPPjgg51eGy6f4eW/H3ry72dKSgomTpwIT09P6Zz58+ejrq4Op0+f7pd2jYjNDwdCRUUFDAaDxR8+AHh6euLs2bNWalX/MBqNePrpp3HjjTdiwoQJ0vGf/exnCAwMhI+PDzIyMvDcc88hKysLW7dutWJrey4uLg4ffPABxowZg+LiYvz+97/HzJkz8eOPP6KkpARqtbrTLwFPT0+UlJRYp8HX4IsvvkBNTQ3uv/9+6dhw//wuZ/5cuvo7aH6tpKQEHh4eFq8rlUq4uLgMy8+1paUFzz33HJYuXWqxsdyTTz6JyZMnw8XFBYcOHcKqVatQXFyMt99+24qt7ZkFCxbgzjvvRHBwMC5cuIDnn38et956K1JSUqBQKEbcZ/jhhx/C0dGx03DzcPkMu/r90JN/P0tKSrr8u2p+rT8wsFyHHnvsMfz4448W9R0ALMaMJ06cCG9vb9xyyy24cOECQkNDB7uZvXbrrbdK30dGRiIuLg6BgYH473//C1tbWyu2rP/961//wq233gofHx/p2HD//K53er0e9957LwRBwIYNGyxeS0xMlL6PjIyEWq3Gr371KyQlJQ35JeDvu+8+6fuJEyciMjISoaGh2Lt3L2655RYrtmxgvPfee1i2bBk0Go3F8eHyGV7p98NQwCGhK3Bzc4NCoehUBV1aWgovLy8rteraPf744/j666+xZ88e+Pn5XfXcuLg4AMD58+cHo2n9ztnZGaNHj8b58+fh5eUFnU6Hmpoai3OG4+eZl5eHXbt24Ze//OVVzxvun5/5c7na30EvL69ORfBtbW2oqqoaVp+rOazk5eXh+++/t+hd6UpcXBza2tqQm5s7OA3sRyEhIXBzc5P+uxwpnyEA/PDDD8jKyur27yYwND/DK/1+6Mm/n15eXl3+XTW/1h8YWK5ArVZjypQpSE5Olo4ZjUYkJydj2rRpVmxZ3wiCgMcffxyff/45du/ejeDg4G6vSU9PBwB4e3sPcOsGRkNDAy5cuABvb29MmTIFKpXK4vPMyspCfn7+sPs833//fXh4eGDhwoVXPW+4f37BwcHw8vKy+Mzq6upw5MgR6TObNm0aampqkJqaKp2ze/duGI1GKbANdeawkp2djV27dsHV1bXba9LT0yGXyzsNpQwHhYWFqKyslP67HAmfodm//vUvTJkyBVFRUd2eO5Q+w+5+P/Tk389p06bh1KlTFuHTHL7HjRvXbw2lK/jkk08EGxsb4YMPPhDOnDkjPPzww4Kzs7NFFfRw8eijjwparVbYu3evUFxcLH01NTUJgiAI58+fF15++WXh+PHjQk5OjvDll18KISEhwqxZs6zc8p575plnhL179wo5OTnCwYMHhfj4eMHNzU0oKysTBEEQHnnkESEgIEDYvXu3cPz4cWHatGnCtGnTrNzq3jEYDEJAQIDw3HPPWRwfrp9ffX29cOLECeHEiRMCAOHtt98WTpw4Ic2Qef311wVnZ2fhyy+/FDIyMoTFixcLwcHBQnNzs3SPBQsWCJMmTRKOHDkiHDhwQAgPDxeWLl1qrUfq5GrPqNPphEWLFgl+fn5Cenq6xd9N88yKQ4cOCX/+85+F9PR04cKFC8LmzZsFd3d3Yfny5VZ+MtHVnq++vl74zW9+I6SkpAg5OTnCrl27hMmTJwvh4eFCS0uLdI/h/Bma1dbWCnZ2dsKGDRs6XT/UP8Pufj8IQvf/fra1tQkTJkwQ5s2bJ6Snpws7duwQ3N3dhVWrVvVbOxlYurFu3TohICBAUKvVQmxsrHD48GFrN6lPAHT59f777wuCIAj5+fnCrFmzBBcXF8HGxkYICwsTnn32WaG2tta6De+FJUuWCN7e3oJarRZ8fX2FJUuWCOfPn5deb25uFlauXCmMGjVKsLOzE+644w6huLjYii3uvZ07dwoAhKysLIvjw/Xz27NnT5f/Xa5YsUIQBHFq84svvih4enoKNjY2wi233NLp2SsrK4WlS5cKDg4OgpOTk5CQkCDU19db4Wm6drVnzMnJueLfzT179giCIAipqalCXFycoNVqBY1GI4wdO1b4wx/+YPEL35qu9nxNTU3CvHnzBHd3d0GlUgmBgYHCQw891Ol/+obzZ2j2t7/9TbC1tRVqamo6XT/UP8Pufj8IQs/+/czNzRVuvfVWwdbWVnBzcxOeeeYZQa/X91s7ZabGEhEREQ1ZrGEhIiKiIY+BhYiIiIY8BhYiIiIa8hhYiIiIaMhjYCEiIqIhj4GFiIiIhjwGFiIiIhryGFiIiIhoyGNgISIioiGPgYWIiIiGPAYWIiIiGvIYWIiIiGjI+39KGt2VCBKV+QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "id": "50dad873",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into evaluation mode (needed esp. for batchnorm)\n",
    "for layer in model.layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "id": "9f2951b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 1.9114720821380615\n",
      "val 2.019570827484131\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the loss\n",
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]   # That's a switch!\n",
    "    \n",
    "    logits = model(x)\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(split, loss.item())\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56c9c8da",
   "metadata": {},
   "source": [
    "Performance log:\n",
    "\n",
    "|                                      |train|val  |\n",
    "| ------------------------------------ | --- | --- |\n",
    "| original (3 characters, 12K params)  |2.058|2.105|\n",
    "| context: 3 -> 8 (22K params)         |1.918|2.033|\n",
    "| flat -> hierarchical (22K params)    |1.942|2.029|\n",
    "| fix bug in batchnorm (22K params)    |1.911|2.019|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "id": "68e034f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jaspinger.\n",
      "kelly.\n",
      "anna.\n",
      "milonna.\n",
      "bankin.\n",
      "darielle.\n",
      "denesse.\n",
      "julian.\n",
      "braya.\n",
      "zieha.\n",
      "ahmarice.\n",
      "aniy.\n",
      "aren.\n",
      "ieroseb.\n",
      "shalees.\n",
      "aryabah.\n",
      "jolana.\n",
      "olenity.\n",
      "nechlea.\n",
      "thivae.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the model\n",
    "for _ in range(20):\n",
    "    \n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all ...\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      logits = model(torch.tensor([context]))\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bb530a2",
   "metadata": {},
   "source": [
    "## Going Wider\n",
    "\n",
    "We can now increase the network's hidden layer sizes, as the components are checked for bugs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "id": "3500b1ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42); # For reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "13f82b7d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76579\n"
     ]
    }
   ],
   "source": [
    "n_embd = 24    # Dimensionality of the character embedding vectors\n",
    "n_hidden = 128 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd),   # Previously C = torch.randn((vocab_size, n_embd))\n",
    "    # The new layer at work now (block_size was 8 before)\n",
    "    FlattenConsecutive(2), Linear(n_embd * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "5f1eb983",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.6596\n",
      "  10000/ 200000: 2.0551\n",
      "  20000/ 200000: 2.0342\n",
      "  30000/ 200000: 2.6254\n",
      "  40000/ 200000: 2.1558\n",
      "  50000/ 200000: 1.7977\n",
      "  60000/ 200000: 2.2949\n",
      "  70000/ 200000: 1.8409\n",
      "  80000/ 200000: 1.6752\n",
      "  90000/ 200000: 2.1980\n",
      " 100000/ 200000: 1.9085\n",
      " 110000/ 200000: 2.0375\n",
      " 120000/ 200000: 1.7248\n",
      " 130000/ 200000: 1.8603\n",
      " 140000/ 200000: 1.8157\n",
      " 150000/ 200000: 1.9445\n",
      " 160000/ 200000: 1.7933\n",
      " 170000/ 200000: 1.6338\n",
      " 180000/ 200000: 1.5115\n",
      " 190000/ 200000: 1.8932\n"
     ]
    }
   ],
   "source": [
    "# Full Training\n",
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "id": "b8f11460",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABi7klEQVR4nO3deVzUdf4H8NcczAzncN8IggdegIKSpWaFV3ZZbua66bJtbZkdi9uW209ttwNrW7etTHfbrtW2rM3KzLzwyAOPQMQTL+S+kQEGmIGZ7++PmfnCBAiDwAC+no8Hj5XvfL9fPl8nmdd+Pu/P5yMRBEEAERERUR8mtXcDiIiIiDrCwEJERER9HgMLERER9XkMLERERNTnMbAQERFRn8fAQkRERH0eAwsRERH1eQwsRERE1OfJ7d2A7mA0GlFYWAhXV1dIJBJ7N4eIiIg6QRAE1NTUIDAwEFLptftQBkRgKSwsREhIiL2bQURERF2Ql5eH4ODga54zIAKLq6srANMDu7m52bk1RERE1BnV1dUICQkRP8evZUAEFsswkJubGwMLERFRP9OZcg4W3RIREVGf16XAsmbNGoSFhUGlUiE+Ph5Hjx5t99xNmzYhLi4O7u7ucHZ2RkxMDNavX291Tm1tLZYsWYLg4GA4Ojpi5MiRWLduXVeaRkRERAOQzUNCGzduRFJSEtatW4f4+Hi89dZbmDFjBrKysuDr69vqfE9PT7z44ouIjIyEQqHAli1bkJiYCF9fX8yYMQMAkJSUhN27d2PDhg0ICwvDjh07sHjxYgQGBuKee+65/qckIiKifk0iCIJgywXx8fEYP3483n33XQCmKcUhISF46qmn8MILL3TqHuPGjcPs2bPx8ssvAwBGjx6NefPmYfny5eI5sbGxmDVrFl555ZUO71ddXQ21Wg2NRsMaFiIion7Cls9vm4aE9Ho90tLSkJCQ0HwDqRQJCQlITU3t8HpBEJCSkoKsrCxMmTJFPH7zzTdj8+bNKCgogCAI2LNnD86fP4/p06e3eR+dTofq6mqrLyIiIhq4bBoSKi8vh8FggJ+fn9VxPz8/nDt3rt3rNBoNgoKCoNPpIJPJ8N5772HatGni6++88w4ee+wxBAcHQy6XQyqV4v3337cKNS0lJyfjz3/+sy1NJyIion6sV6Y1u7q6IiMjA7W1tUhJSUFSUhLCw8MxdepUAKbAcvjwYWzevBmhoaH48ccf8eSTTyIwMNCqN8di2bJlSEpKEr+3zOMmIiKigcmmwOLt7Q2ZTIaSkhKr4yUlJfD392/3OqlUiiFDhgAAYmJicPbsWSQnJ2Pq1Kmor6/Hn/70J3z99deYPXs2ACAqKgoZGRl488032wwsSqUSSqXSlqYTERFRP2ZTDYtCoUBsbCxSUlLEY0ajESkpKZg4cWKn72M0GqHT6QAAjY2NaGxsbLWHgEwmg9FotKV5RERENEDZPCSUlJSERYsWIS4uDhMmTMBbb70FrVaLxMREAMDChQsRFBSE5ORkAKZ6k7i4OERERECn02Hr1q1Yv3491q5dC8C0Ou2tt96K5557Do6OjggNDcW+ffvwn//8B6tXr+7GRyUiIqL+yubAMm/ePJSVlWHFihUoLi5GTEwMtm3bJhbi5ubmWvWWaLVaLF68GPn5+XB0dERkZCQ2bNiAefPmied8/vnnWLZsGRYsWIDKykqEhobi1VdfxeOPP94Nj0hERET9nc3rsPRFXIeFiIio/7Hl83tAbH7YUxoaDfjbjizUNxqw8u5RcJBx6yUiIiJ74CfwNUgkwPv7s7HhcC7qGw32bg4REdENi4HlGhQyKaTmHa8b9AwsRERE9sLAcg0SiQSODjIAYA8LERGRHTGwdMBRwcBCRERkbwwsHVBZelg4JERERGQ3DCwd4JAQERGR/TGwdMAyJNTAwEJERGQ3DCwdUMktgYX7GhEREdkLA0sHVArWsBAREdkbA0sHHB1Mf0WsYSEiIrIfBpYOWIpuWcNCRERkPwwsHXDkkBAREZHdMbB0QMVpzURERHbHwNIBBhYiIiL7Y2DpQHMNC6c1ExER2QsDSwdYdEtERGR/DCwd4DosRERE9sfA0gHuJURERGR/DCwdYGAhIiKyPwaWDqjMK92yhoWIiMh+GFg6IPawsIaFiIjIbhhYOmApum1oYmAhIiKyFwaWDjT3sHAdFiIiInthYOkA12EhIiKyPwaWDoibHzYaIAiCnVtDRER0Y2Jg6YBlLyGDUUCjgYGFiIjIHhhYOmCZ1gxwLRYiIiJ7YWDpgEImhVRi+rOOgYWIiMguGFg6IJFIuNotERGRnTGwdELLwlsiIiLqfQwsnaDiardERER2xcDSCRwSIiIisi8Glk5QcfE4IiIiu2Jg6QQuz09ERGRfDCydIG6AyB4WIiIiu2Bg6QRH8+JxrGEhIiKyDwaWTuAGiERERPbFwNIJ4josnNZMRERkFwwsnaCUc1ozERGRPTGwdAJXuiUiIrIvBpZOYA0LERGRfXUpsKxZswZhYWFQqVSIj4/H0aNH2z1306ZNiIuLg7u7O5ydnRETE4P169e3Ou/s2bO45557oFar4ezsjPHjxyM3N7crzet2zYGF67AQERHZg82BZePGjUhKSsLKlSuRnp6O6OhozJgxA6WlpW2e7+npiRdffBGpqanIzMxEYmIiEhMTsX37dvGcS5cuYdKkSYiMjMTevXuRmZmJ5cuXQ6VSdf3JupGKRbdERER2JREEQbDlgvj4eIwfPx7vvvsuAMBoNCIkJARPPfUUXnjhhU7dY9y4cZg9ezZefvllAMBDDz0EBweHNnteOqO6uhpqtRoajQZubm5duse1/C8tH3/48gRuHeaDT34zodvvT0REdCOy5fPbph4WvV6PtLQ0JCQkNN9AKkVCQgJSU1M7vF4QBKSkpCArKwtTpkwBYAo833//PYYNG4YZM2bA19cX8fHx+Oabb9q9j06nQ3V1tdVXT+Lmh0RERPZlU2ApLy+HwWCAn5+f1XE/Pz8UFxe3e51Go4GLiwsUCgVmz56Nd955B9OmTQMAlJaWora2FqtWrcLMmTOxY8cOzJkzB/fffz/27dvX5v2Sk5OhVqvFr5CQEFsew2Yq80q3LLolIiKyD3lv/BBXV1dkZGSgtrYWKSkpSEpKQnh4OKZOnQqj0VTIeu+99+L3v/89ACAmJgaHDh3CunXrcOutt7a637Jly5CUlCR+X11d3aOhpXnzQwYWIiIie7ApsHh7e0Mmk6GkpMTqeElJCfz9/du9TiqVYsiQIQBMYeTs2bNITk7G1KlT4e3tDblcjpEjR1pdM2LECBw4cKDN+ymVSiiVSluafl1UXIeFiIjIrmwaElIoFIiNjUVKSop4zGg0IiUlBRMnTuz0fYxGI3Q6nXjP8ePHIysry+qc8+fPIzQ01Jbm9RhOayYiIrIvm4eEkpKSsGjRIsTFxWHChAl46623oNVqkZiYCABYuHAhgoKCkJycDMBUbxIXF4eIiAjodDps3boV69evx9q1a8V7Pvfcc5g3bx6mTJmC2267Ddu2bcN3332HvXv3ds9TXicuHEdERGRfNgeWefPmoaysDCtWrEBxcTFiYmKwbds2sRA3NzcXUmlzx41Wq8XixYuRn58PR0dHREZGYsOGDZg3b554zpw5c7Bu3TokJyfj6aefxvDhw/HVV19h0qRJ3fCI16/l0vyCIEAikdi5RURERDcWm9dh6Yt6eh0WTV0jov+yAwBw/pVZUMi5owEREdH16rF1WG5UKkXzXxMLb4mIiHofA0snKGRSSM2jQKxjISIi6n0MLJ0gkUi4FgsREZEdMbB0kqXwtqGJgYWIiKi3MbB0koo9LERERHbDwNJJKm6ASEREZDcMLJ3ExeOIiIjsh4Glk5yVpsBSXd9k55YQERHdeBhYOsnfTQUAKK5usHNLiIiIbjwMLJ0U4O4IACiqqrdzS4iIiG48DCydFKg29bAUatjDQkRE1NsYWDopQG3uYdGwh4WIiKi3MbB0UoC7qYelqIo9LERERL2NgaWTLD0sFVo9pzYTERH1MgaWTvJwcoBSbvrrKuFMISIiol7FwNJJEokEgeaZQoUcFiIiIupVDCw2CDDPFGLhLRERUe9iYLFB80wh9rAQERH1JgYWGwSaZwoVcvE4IiKiXsXAYgP2sBAREdkHA4sNmmtYGFiIiIh6EwOLDcTF41h0S0RE1KsYWGxgGRKqqmtEvZ6LxxEREfUWBhYbuKnkcFbIAACF7GUhIiLqNQwsNpBIJAgwLx7HPYWIiIh6DwOLjSyFt+xhISIi6j0MLDYKNNexFHOmEBERUa9hYLGRZT+h3Mo6O7eEiIjoxsHAYqPh/q4AgLNF1XZuCRER0Y2DgcVGowLdAAAXSmqhbzLauTVEREQ3BgYWGwV7OMJVKYfeYMSlslp7N4eIiOiGwMBiI4lEghHmXpYzhRwWIiIi6g0MLF0wMsAcWFjHQkRE1CsYWLpgJHtYiIiIehUDSxe07GERBMHOrSEiIhr4GFi6YKifC+RSCTT1jSjkAnJEREQ9joGlC5RyGYb6mdZj4bAQERFRz2Ng6SLLsNDpQo2dW0JERDTwMbB0EQtviYiIeg8DSxdZVrw9kl2JmoZGO7eGiIhoYGNg6aK4UA+E+zhDU9+IDw5k27s5REREA1qXAsuaNWsQFhYGlUqF+Ph4HD16tN1zN23ahLi4OLi7u8PZ2RkxMTFYv359u+c//vjjkEgkeOutt7rStF4jl0nx+4RhAIAP9mfjqlZv5xYRERENXDYHlo0bNyIpKQkrV65Eeno6oqOjMWPGDJSWlrZ5vqenJ1588UWkpqYiMzMTiYmJSExMxPbt21ud+/XXX+Pw4cMIDAy0/UnsYPaYAIwIcEONrgn//PGyvZtDREQ0YNkcWFavXo1HH30UiYmJGDlyJNatWwcnJyd8+OGHbZ4/depUzJkzByNGjEBERASeeeYZREVF4cCBA1bnFRQU4KmnnsKnn34KBweHrj1NL5NKJVg6zdTL8vGhbFSzloWIiKhH2BRY9Ho90tLSkJCQ0HwDqRQJCQlITU3t8HpBEJCSkoKsrCxMmTJFPG40GvHwww/jueeew6hRo2xpkt3dMcIXIZ6OaGg04kRelb2bQ0RENCDJbTm5vLwcBoMBfn5+Vsf9/Pxw7ty5dq/TaDQICgqCTqeDTCbDe++9h2nTpomvv/7665DL5Xj66ac71Q6dTgedTid+X11tv6nFEokEMSEeyKusx4m8Kkwe6mO3thAREQ1UNgWWrnJ1dUVGRgZqa2uRkpKCpKQkhIeHY+rUqUhLS8M//vEPpKenQyKRdOp+ycnJ+POf/9zDre686GA1vjtRiBP5XESOiIioJ9g0JOTt7Q2ZTIaSkhKr4yUlJfD392//h0ilGDJkCGJiYrB06VLMnTsXycnJAID9+/ejtLQUgwYNglwuh1wuR05ODpYuXYqwsLA277ds2TJoNBrxKy8vz5bH6HZRwe4AgMz8Kru2g4iIaKCyKbAoFArExsYiJSVFPGY0GpGSkoKJEyd2+j5Go1Ec0nn44YeRmZmJjIwM8SswMBDPPfdcmzOJAECpVMLNzc3qy55GB7lBKgFKqnUoqeZmiERERN3N5iGhpKQkLFq0CHFxcZgwYQLeeustaLVaJCYmAgAWLlyIoKAgsQclOTkZcXFxiIiIgE6nw9atW7F+/XqsXbsWAODl5QUvLy+rn+Hg4AB/f38MHz78ep+vVzgp5Bjq64qskhqcyKvC9FHt9zYRERGR7WwOLPPmzUNZWRlWrFiB4uJixMTEYNu2bWIhbm5uLqTS5o4brVaLxYsXIz8/H46OjoiMjMSGDRswb9687nuKPiAqWI2skhpk5msYWIiIiLqZRBAEwd6NuF7V1dVQq9XQaDR2Gx5afzgHy785hclDvbH+kXi7tIGIiKg/seXzm3sJdZPoYDUA4GSBBgMgAxIREfUpDCzdJNLfDQqZFFV1jcitrLN3c4iIiAYUBpZuopBLMSLAFQCQyfVYiIiIuhUDSzca7m8KLJfKau3cEiIiooGFgaUbhXk7AwCyy7V2bgkREdHAwsDSjcLNgeUKAwsREVG3YmDpRi17WDhTiIiIqPswsHSjUE9TYKluaMLVukY7t4aIiGjgYGDpRo4KGQLVKgCsYyEiIupODCzdjIW3RERE3Y+BpZuFsfCWiIio2zGwdLPBXuYelgoGFiIiou7CwNLNBrOHhYiIqNsxsHSzlkNCnNpMRETUPRhYutkgTydIJYBWb0BZjc7ezSEiIhoQGFi6mUIuRZCHIwDOFCIiIuouDCw9YLC3CwDgCgtviYiIugUDSw8Y7OUEADhfwl2biYiIugMDSw8YFagGAHx4MBuvbzuHRoPRzi0iIiLq3xhYesCccUF4aHwIBAFYu/cSnvn8uL2bRERE1K8xsPQAB5kUqx6IwtvzxwIAtp8ugb6JvSxERERdxcDSg+6OCoCLUg6DUUBuJQtwiYiIuoqBpQdJJBJE+JgWkrtYysBCRETUVQwsPSzCxzTF+VIZZwwRERF1FQNLD4vwNQeWUgYWIiKirmJg6WGWISH2sBAREXUdA0sPG2LpYSnjZohERERdxcDSwwZ5OkMmlaBW14RSboZIRETUJQwsPUwhlyLU07RU/0XWsRAREXUJA0svCOdMISIiouvCwNILInzNhbfsYSEiIuoSBpZeMMSnufCWiIiIbMfA0gvEtVg4JERERNQlDCy9IMLbFFiKNA2oqOVMISIiIlsxsPQCtZMDQjwdAQAJq/fh44PZXJOFiIjIBgwsveSteWMR4eOMq3WNeOm7M/j8WJ69m0RERNRvMLD0kthQD2x/dgoSbwkDAGzOKLRvg4iIiPoRBpZeJJdJkXjzYADA0SuVqKrT27lFRERE/QMDSy8b5OWE4X6uMBgF7D5Xau/mEBER9QsMLHYwbaQfAGDnmRI7t4SIiKh/YGCxA0tg2Xe+DA2NBju3hoiIqO9jYLGDMUFq+LkpUac3IPVShb2bQ0RE1Od1KbCsWbMGYWFhUKlUiI+Px9GjR9s9d9OmTYiLi4O7uzucnZ0RExOD9evXi683Njbi+eefx5gxY+Ds7IzAwEAsXLgQhYUDdxaNVCpBwghTL8sr35/BhweyUallAS4REVF7bA4sGzduRFJSElauXIn09HRER0djxowZKC1tu4DU09MTL774IlJTU5GZmYnExEQkJiZi+/btAIC6ujqkp6dj+fLlSE9Px6ZNm5CVlYV77rnn+p6sj7t/XDAcZBJcKtPiL1vO4P73DkLfZLR3s4iIiPokiWDjkqvx8fEYP3483n33XQCA0WhESEgInnrqKbzwwguduse4ceMwe/ZsvPzyy22+fuzYMUyYMAE5OTkYNGhQh/errq6GWq2GRqOBm5tb5x/GzkqqG7D1ZBHeTrmAq3WNWLtgHGaNCbB3s4iIiHqFLZ/fNvWw6PV6pKWlISEhofkGUikSEhKQmpra4fWCICAlJQVZWVmYMmVKu+dpNBpIJBK4u7vb0rx+x89NhcRbBmP+BFMo4+q3REREbbMpsJSXl8NgMMDPz8/quJ+fH4qLi9u9TqPRwMXFBQqFArNnz8Y777yDadOmtXluQ0MDnn/+ecyfP7/dtKXT6VBdXW311Z89GBcCAPjxQhkKq+rt3BoiIqK+p1dmCbm6uiIjIwPHjh3Dq6++iqSkJOzdu7fVeY2NjXjwwQchCALWrl3b7v2Sk5OhVqvFr5CQkB5sfc8L83bGTeGeEATgy5/yrV6r1OrRZGBtCxER3dhsCize3t6QyWQoKbFe8KykpAT+/v7t/xCpFEOGDEFMTAyWLl2KuXPnIjk52eocS1jJycnBzp07rzmWtWzZMmg0GvErL6//D6U8NN40LPTFT3kwGk1lRQculOOm11Lw3P8y7dk0IiIiu7MpsCgUCsTGxiIlJUU8ZjQakZKSgokTJ3b6PkajETqdTvzeElYuXLiAXbt2wcvL65rXK5VKuLm5WX31dzNH+8NNJUdBVT0+PJiNOn0TXtiUCb3BiB2ni9nLQkRENzS5rRckJSVh0aJFiIuLw4QJE/DWW29Bq9UiMTERALBw4UIEBQWJPSjJycmIi4tDREQEdDodtm7divXr14tDPo2NjZg7dy7S09OxZcsWGAwGsR7G09MTCoWiu561T1M5yJA0bRhe+u4MXtt6FvvOlyH/qqmeRas34ExRNaKC3e3bSCIiIjuxObDMmzcPZWVlWLFiBYqLixETE4Nt27aJhbi5ubmQSps7brRaLRYvXoz8/Hw4OjoiMjISGzZswLx58wAABQUF2Lx5MwAgJibG6mft2bMHU6dO7eKj9T+Lbg7D6cJqfJmWj/0XygEAAWoVijQNOJpdycBCREQ3LJvXYemL+us6LG3RNRkw/1+HkZ5bhXuiAzEiwA2vbzuHGaP88M+H41CvN6Cgqg5DfF3t3VQiIqLrYsvnt809LNSzlHIZPv7NBOw8XYI7xwTgTJEGAPDTlasQBAG//c8xHLxYgc1LbmGPCxER3TC4+WEf5KZywAOxwXBUyDAmyB1KuRQVWj02HM7BwYumzRLTcq7auZVERES9h4Glj1PIpYgJcQcAvPz9WfH4lXKtnVpERETU+xhY+oEJgz0BwGpzxCsVdfZqDhERUa9jYOkH4sI8xT8P9zMV2+ZUsIeFiIhuHAws/UBsqAecFTIoZFK8Omc0ACDvaj0auZgcERHdIDhLqB9wUcrx5eM3Q4CAEf5uUDlI0dBoRMHVeoR5O9u7eURERD2OPSz9xMhAN4wKVEMqlSDMyxRSsjksRERENwgGln4o1MsJAJDDmUJERHSDYGDphyzDQJwpRERENwoGln7IMiR0hUNCRER0g2Bg6YcsgSWHPSxERHSDYGDph8K8TTUseZV1aOLUZiIiugEwsPRDfq4qqBykaDIKuFyuxT/3XcKRyxX2bhYREVGPYWDph6RSCUI9TcNCv/n4GJJ/OIdFHx1F9s9mDdXqmvDe3ovIq+TQERER9W8MLP2UZVgo/2o9AKCh0YjnvjwBg1EQz0neehZvbMvCqh/O2aWNRERE3YWBpZ8a7O0CAHBWyPDWvBi4KOX4KecqPjqYDcBU3/LFT3kAgKNXKiEIQrv3IiIi6uu4NH8/tSB+EK5q9fhl/CBEh7ijTm/An74+iTe2Z2GwtzN2nilBo8EUUspqdMi/Wo8QTyc7t5qIiKhrGFj6qRBPJ7w+N0r8fv6EEOy/UIYfThXjsfVp4nEvZwUqtHqk5VxlYCEion6LQ0IDhEQiwTvzx+L+sUEwGAUYjAJuHeaD+8YGAQDScq4CAJJ/OIuJySmI+csOTHh1F07kVdmx1URERJ3DwDKAyGVSvPmLaCyeGoFIf1e8OHsEYkM9AJgCy6WyWvxz32UUaRpQVdeI0hodPjiQbedWExERdYyBZYCRSiX448xIbHt2Cob5uYqB5VxxNdbuvQQAmDzUG+t+FQsA2HmmBFpdk93aS0RE1BkMLAOcn5sKQe6OMArA/9LyAQCPTQnHjFF+CPNyQn2jATvPlNi5lURERNfGwHIDsPSyAMAQXxdMGuINiUSCe2NM9S3fZBTYq2lERESdwsByA2gZWBbdHAaJRAIAuDcmEACw/0I5ymt1dmkbERFRZzCw3ADiwz0BAG4qOe43zxoCgHAfF0QFq2EwCth6sshezSMiIuoQA8sNINLfDf96OBaf/vYmOCutl965J9rUy7LjNOtYiIio7+LCcTeI6aP82zx+c4Q3ACAjrwoGowCZVNKbzSIiIuoU9rDc4Ib5ucDRQYZaXRMuldXauzlERERtYg/LDU4uk2JMsBpHsyuRkVuFYX6uSDlbgm8zCiEAcFHK8cwdQ+GvVtm7qUREdANjYCGMHeSOo9mVOJ5XhfvGBmHplydQVdcovu6mkmPZnSPs2EIiIrrRcUiIMDbEHYCpjuXgxXJU1TXC20WBubHBAIDj3G+IiIjsjIGFEBNiWqclq7gaG4/lAQDuigrE76aEAwBO5mvQZDACAEprGlBWwzVbiIiodzGwEPzVKgSoVTAKwLbTxQCAu6MDEO7jAhelHPWNBlwsq4VW14Q7/7Efd769H3V67j9ERES9h4GFAAAx5mEhAAhyd8TYEA/IpBKMCVIDAE7kVWH/hTKU1+pRVqPD3qwyO7WUiIhuRAwsBMA6sNwVHQCpeT2WqBBzYMnXYOeZUvGcH04V92r7iIjoxsbAQgCsA8vdUYHNx4NNx9NzrmL3uebVcHefLUFDowFnCqvxyMfHcIKFuURE1IM4rZkAADGD3DF2kDu8nJUYFegmHo82B5lzxTUAALWjA5wUMhRpGrDjTAne2nkel8u1KKiqx9anJ4s9M0RERN2JgYUAAEq5DF8vvqXV8QC1Cj6uSnFm0O2RvvBwUuDDg9lY9lUmtHoDAFOg2XqqCHdFBSIt5yoaGg24ZYh3rz4DERENXBwSomuSSCSIDlaL3yeM8MOdY0z7ElnCyi1DvAAAq3eex39Sr2DuukNY+OFRFFTV936DiYhoQGJgoQ5Fm+tYFDIpbh3ug3GDPODrqgQAzBkbhLW/ioW7kwMul2mx4tvTEATAYBSQcpY7QBMRUfdgYKEO3RbpC5lUgllj/OGilEMqleCV+0ZjXlwIXrp7FNxUDvjdlAjx/JEBphqYnWcYWIiIqHt0KbCsWbMGYWFhUKlUiI+Px9GjR9s9d9OmTYiLi4O7uzucnZ0RExOD9evXW50jCAJWrFiBgIAAODo6IiEhARcuXOhK06gHjA5S49ALt+P1B6LEY9NH+eP1uVFQOzkAABJvCcPiqRF4e/5YvD0/BgBw5HIlanWmBeaMRqHX201ERAOHzYFl48aNSEpKwsqVK5Geno7o6GjMmDEDpaWlbZ7v6emJF198EampqcjMzERiYiISExOxfft28Zw33ngDb7/9NtatW4cjR47A2dkZM2bMQENDQ9efjLqVn5sKKgdZu6+rHGT448xI3BMdiAgfF4R6OUFvMGL/+TJsPVmEyBXb8O/9lzv1s1LOliAtp7K7mk5ERAOARBAEm/6vb3x8PMaPH493330XAGA0GhESEoKnnnoKL7zwQqfuMW7cOMyePRsvv/wyBEFAYGAgli5dij/84Q8AAI1GAz8/P3z88cd46KGHOrxfdXU11Go1NBoN3NzcOjyfet5fvjuDDw9m45YhXjiRp0GtrgmuSjkOvHA71I4O4nmZ+VX4/Fgelk4bBi8XJc4WVWPWP/bDSSHDsRcT4KzkRDYiooHKls9vm3pY9Ho90tLSkJCQ0HwDqRQJCQlITU3t8HpBEJCSkoKsrCxMmTIFAJCdnY3i4mKre6rVasTHx3fqntQ3JYzwBQAcvFghDgvV6JrwyaEr4jlaXRMeX5+G/x7JxT9STEOAm9LzAQB1egN2n2u7146IiG48NgWW8vJyGAwG+Pn5WR338/NDcXH7S7VrNBq4uLhAoVBg9uzZeOeddzBt2jQAEK+z5Z46nQ7V1dVWX9S3jB/sCVeVqXfEVSXH8zMjAQAfHswWA8w/Ui6gUGMa9tuUXgBNfSO+ySgU77H1ZFEvt5qIiPqqXpkl5OrqioyMDBw7dgyvvvoqkpKSsHfv3i7fLzk5GWq1WvwKCQnpvsZSt3CQSfHAuGDIpRK88UAUHpsSjnBvZ1TVNeLvO89j3/kyfHAgG4Bp9dxaXRP+8OUJlNXooJCb/rPcfa4UWh13hSYiIhsDi7e3N2QyGUpKrKerlpSUwN/fv/0fIpViyJAhiImJwdKlSzF37lwkJycDgHidLfdctmwZNBqN+JWXl2fLY1AvWXHXSKT93zTMGhMAmVSCxbcNAQB8cCAbiz48CoNRwMxR/vh9wlAAzdOg58WFINTLCbomY7vDQttPF3OdFyKiG4hNgUWhUCA2NhYpKSniMaPRiJSUFEycOLHT9zEajdDpTEu9Dx48GP7+/lb3rK6uxpEjR9q9p1KphJubm9UX9T1SqUSc9gwA98UE4reTBmNUoBsUMim8XZRYcfdIPBAbDGdF8wyk+8cFYfaYAADA95mth4XKa3VY/Gk6ntiQLg4vERHRwGbzFIykpCQsWrQIcXFxmDBhAt566y1otVokJiYCABYuXIigoCCxByU5ORlxcXGIiIiATqfD1q1bsX79eqxduxaAaen3Z599Fq+88gqGDh2KwYMHY/ny5QgMDMR9993XfU9KdieXSfF/d40EADQZjOIxAJgzLggbDudisLczYkLcoZBL8d7eS9iTVYqahka4qpqDz6kCDQxGAQYIuFRaK27QSEREA5fNgWXevHkoKyvDihUrUFxcjJiYGGzbtk0sms3NzYVU2txxo9VqsXjxYuTn58PR0RGRkZHYsGED5s2bJ57zxz/+EVqtFo899hiqqqowadIkbNu2DSqVqhsekfoiS1CxeOr2oSir0WH+hEGQSCQYGeCGCB9nXCrT4rWt55B8/xjx3NOFzUXWFxhYiIhuCDavw9IXcR2WgenQxXIs+OAIBAFY96tYzBxtqml68tN0fG+eQfS7W8OxbNYIezaTiIi6qMfWYSHqTTcP8Rb3KHphUyaKzVOgTxdqxHMulNTapW1ERNS7GFioT0uaNgxRwWpU1TXi/f2XUatrwpWKOvH1C6U1VucLgoAvfsrDmUKuzUNENJAwsFCfppBL8aR5OvQPJ4vEIOJqXrI//2o96vTNM4W+ySjAH/+XiaQvMtq9p6a+EVP/ugdLvzjRcw0nIqJuxcBCfd6tw3zgrJChUNOAz47mAgAmDPaEp7MCggBcLtMCMPWu/Hu/aTG68yU1qNcb2rxfeu5VXKmowzcZBe2eQ0REfQsDC/V5KgcZ7hhhmoX2TUYBAGBUoBuG+LoAaB4WOpJdKc4gMgpAVklNG3cDrpSbAo7BKOBkgabNc4iIqG9hYKF+4c4xphlCljltIwPVGGoJLObCW8tS/xZni9quY7EEFgA4nnu1u5tKREQ9gIGF+oWpw33h1GI13FGBbs2BpbQWV8q12GVeqt+yU3R7gSW7RdHu8dyqHmoxERF1JwYW6hdUDjLcHmkKImpHBwR7OGKonysA4FxxNZZtOglBAG6P9MVdUYEA0O5MoZY9LOm5VzEAliIiIhrwGFio37h/XBAAIH6wJyQSidjDkldZj9TLFXBWyPDHmcMxIsC0+NC54hoYjdZhRN9kRP7V5h6W0hodiszruxARUd/FwEL9xu2RfvjqiYl4/YEoAICPqxJuKtP0ZplUgjULxiHS3w3hPs5QyKWo1TUhr0U4AYD8q3UwCoCTQoZRgaZgk95BHcverFKs2XNR3P+IiIh6HwML9SuxoZ7wcFYAMG2cOWGwFwDg5XtHY+pw05CRg0yKYX6m3pef17FcqTANB4V6OWPcIA8A165jEQQBz/0vE3/dnoVvMwq79VmIiKjzGFioX/vHQzFIWXorfhk/yOr4SPOw0M/rWLLLTT0uYV5OGDvIHcC1ZwqV1uhQVqMDALy//zIEQUB2uRb3rTmIjw9mt3sdERF1LwYW6teclXJE+Li0Om6pYzlTZL0Wi6XgNszbGWPNPSynCqrx+rZz+DajoNWwT8vAc664BnuySvH0Z8eRkVeFv2w5g7QcU9g5XajBF8fyWMBLRNRD5PZuAFFPsASWUwUanCmsRoinI1xVDuKQ0GAvZ4R5OcHfTYXi6gas3XsJAKBrNOLB8SHifSwbLUokpjVglvz3OOrMq+MaBSDpiww8Ojkcf/nuDPQGI9wcHcRdpYmIqPuwh4UGJEtgKa5uwJ1v78fNybtxNLtSDCxh3s6QSCT49NF4/N/sEbg5wlQLs+98mdV9zphrYB6+KRQyqUQMK68/MAaBahVyKurwf9+cgt7cM/PdCdvqXHRNBhbzEhF1AgMLDUhqRwcsnTYMIwPcoHZ0QI2uCS9sykTB1XoAphoWAIjwccFvJ4cjadowAEDq5QqrqdCWIaHpI/1xV1QAAGBubDDmjR+ENx+MFs+bM9Y05TrlXAm0uubNGK+lSFOPCa+m4LH1adf5tEREAx+HhGjAeuqOoXjqjqHQ1DXi9r/tFTdJdFbI4OOqtDo3KtgdTgoZKrV6ZJXUYESAG2oaGnHFvCruyEA3vBIyGtNH+mPaSNO+RjdHeGPjYzdBJpUgNtQD6blXkVNRh5RzpbgnOrDD9n2bUQhNfSN2nyvF6UINRgWqu/lvgIho4GAPCw14aicHPD8zUvw+1Ms0HNSSQi5FXJgnACD1UgUAU5EtAASoVfB0VsBV5YDZUQFQyJv/2cSHeyEuzLSQ3d3mFXY7Oyz0w6li8c+fH83rwpMREd04GFjohjA3NhgxIe4AgMHezm2eY6ljOWQOLJbhIMsU6Y7cFW0aMtqXVYbqhsZWr3+fWYTffnIMRZp6FFTV40RelfjaN8cLUG+ujyEiotYYWOiGIJVKsPrBaMwc5Y9HJg9u8xxLYDmSXQGDURBnCFlWxO3IcD9XDPF1gd5gxKy39mPG33/EZ0dzxdffTrmAXWdL8eLXp7DN3LsyPswDgzydUKNrwpZMLkxHRNQeBha6YYT7uGDdw7HiCrc/NypQDVeVHDUNTThdqBFnCI3sZGCRSCSYF2eaEl1QVY+skhqs3nkegiCgTt+EC6WmIabd50rx7u4LAIA7xwRgnnka9YbDOaiq07d7/8Kf9coQEd1IGFiIzGRSCeLNS/2/suUsssw1LLYUwz4yaTC+f3oSPnv0JsilEpTV6JB/tR6nCqrRch/Gq3WmIaOZo/3xi7hgyKUSnMjXYNzLOzHvn6lIy6m0uq8gCPjVv49gznsH292FmohoIGNgIWph8lBvAMDRK5VoNAjwcVUi2MOx09dLpRKMClRjYoQXRgWZgk5azlVk5lcBAG4d5iNOqR47yB0Bakf4uqrw93kxGObnAqMAHMmuxNx1qXh5yxmxriUjrwqXy7UwCsD3Jzl0REQ3Hk5rJmph3vgQ1OkN0DcZ4aKSY9IQ71YzijorLtQDJ/KqkJZzFVX1ph6V8WEeePqOoVi5+RSevn2oeO7d0YG4OzoQeZV1+Puu89iUXoAPDmTjap0eqx+MwdaTReK5204V47kZka1+HhHRQMbAQtSCykGGJ6ZGdMu9YkM98MGBbKTlXIVWb1pMLirYHbGhHtjy1OQ2rwnxdMLqB2MwfaQ/Ht+Qhm+OF+Cp24di68nmKdCXyrS4WFqDIb6u3dJOIqL+gENCRD0kNtRU3HuuuBo55gXoooI7Vw8zc7Q/bhvuA6MAPLsxAwVV9XB0kOGmcNNaMdtPl+BiaS0W/Pswvs0o6JkHICLqQxhYiHqIn5sKwR6OYrFtqJcT3J0Unb7+yduGAIA4M+j2Eb64L8a0BcCm9Hws+vAoDl6swDu7L3apfamXKjD77f2t9k8iIuqLGFiIepCllwUwDQfZIi7MExMGe4rfzx4TgISRfpBKTMNCBVWmfZEultaiUtv+dOirWj30TdYbLJZUN2DJf9NxurAa61Ov2NQuIiJ7YGAh6kEtA0t0J4eDWlpi7mVxUsgwdbgPvF2UGG/eQsDbRYEgd9MMpmNXWk+D3nOuFAv+fRhjX96JF77KFF8zGAU8+3kGKswhJyOvCoIggIioL2NgIepB19PDApimWa9+MBr/XhgHJ4WpRn7p9OG4PdIXHydOwK3DfQAAx7KtA8uXP+Uj8eNjOHjRtM3A9tPFMJjHpv754yWkXq6Ak0IGuVSC8lo98s27WBMR9VUMLEQ9aLifK8K9nRGgVmFMkO09LBKJBPePC8bNQ7zFYxMGe+LDX4/H6CA1Jph7W37ew/LtCVMh7t3RgXBWyKDVG8SVdjceM220uPyukeIqvifM68QQEfVVDCxEPUguk2LzU5Ow/fdT4KiQdfv9x5trXE4VVkOrM02dbjQYcTy3CoBpSMnSs5ORW4UiTT1yKuoglQB3RQUgusVrRER9GQMLUQ9zUcrhpnLokXsHuTsiyN0RBqOADPNsorNF1ajTG+CmkmOorwtiBrkDMNWqHLls6okZHaSGq8pB3ME6o5v3KMq/WoefftbrQ0R0PRhYiPq5uDBTncxRcx2L5X/jwjwhlUqsQsmRbFNNS7y5Z8YSZk4WaNBosJ5J1J4tmYX4244sGI2tC3UFQcAXx/KQsHof5q5LxbZTRW3cgYjIdgwsRP3c+J/Vsfx05arV8bHmwHK+pAb7skxrrlg2eRzs5Qw3lRy6JqO42eO11Omb8IcvT+Cd3Rdx2Bx+LARBwLJNJ/HHrzLR0GgKP69uPYuGRsN1PiEREQMLUb9nWaslLecqSqob8JN5p+fx5p4XXzcVAtUqGAWgUNMAiaS59kUqlSDaHGg2nyjE/H8dxk2vpWDpFyfwxU95WJ96Bev2XUKRxjSL6MfzZWIYSc+5atWOrJIafH4sD1IJkDRtGPzclMirrMdHB6/09F8BEd0AuJcQUT831NcFMSHuyMirwpOfpqO8Vg+FXIoxLdZ9iRnkjkLzfkQjA9ygdmyuqYkJccf+C+X414+XxWNfpefjq/R88ft9WWX47LGbsP10iXjsp58FFkt9zC1DvPH0HUMR5O6IpV+ewJo9F/FAbBB8XVXd++BEdENhDwtRPyeRSPDKfaMhkTSHiJhgdyjlzbOSLHUsQPNwUFuvTQjzxAeL4vDIpMG4OcILM0b5QS6VIPVyBdJyKrHrbHNgSc+5alXHcviyaYjopnDT/eeMDUJ0sBq1uib86t9HkFdZ123PTEQ3HgYWogFgdJAav4oPFb+3FOJaxIQ0fx8f7mn12uShPnhgXDCemzEc/300HneM8MPyu0biv4/ehH8+HId7YgIBAM98noGahiZ4uyjgpJChuqEJF8tqAZjqV46Yi33jWww3vTE3Gr6uSpwvqcW9aw7ieK51rwwA5FbUYcl/0/Gf1Cuo17PehYjaxsBCNED8YfpweDmbNlecGGHdizImSA1XpRyODjJxsTkLhVyKvz0YjSdvGwK5rPWvhN9NiQAAcTXc6aP8xV4ZS4HvBfN+RioHqdWKvsP9XfHtklswOsgNlVo9kr440Wp20eqdWdiSWYQV357GpNd3Y/vp4q7/JRDRgMXAQjRAqJ0c8N9Hb8Kbv4jGpBYr4wKAo0KGz393Ezb+7iZ4OHd+x2jAFDpuj/QVv585yl/cciDNPAR1xDwcFBvqAYXc+tdKgNoRnz16E1yVcmSXa3HoUvPsIq2uSayL8XVVokKrx7JNJ7m3ERG1wqJbogFkuL8rhvu7tvnaqEDbtwaweGJqBHafK4WHkwNuCveCwRwo0swzkg6bC25v+ll9jIWrygH3jwvCJ6k5+PRIDiYNNQWqHWeKUd9owGBvZ/zwzGTE/GUHKrV6XCitxTC/1s9RVafHvWsOorHJiHtighDu44zDlypQUFWPVQ9EYbC3c5efkYj6ti71sKxZswZhYWFQqVSIj4/H0aNH2z33/fffx+TJk+Hh4QEPDw8kJCS0Or+2thZLlixBcHAwHB0dMXLkSKxbt64rTSOiHjA+zBMf/Xo81j8SD4VcinGDTD0sVyrqUFaja16QLrztwAIAC24y1djsOFOCkuoGAMDXxwsBAPfGBELlIENcqGm4ylLA+3M/nCpGTkUdCjUNWLfvEv74v0xsOl6AI9mVeCflQrc868XSGjz35Ql8d6Kw1WuNBiO++CkPFbW6bvlZRNR5NgeWjRs3IikpCStXrkR6ejqio6MxY8YMlJaWtnn+3r17MX/+fOzZswepqakICQnB9OnTUVBQIJ6TlJSEbdu2YcOGDTh79iyeffZZLFmyBJs3b+76kxFRt7ot0hejzRs4qh0dMMzPBQDw20+OobxWD6VciuiQ9ntxhvm5YkKYJwxGARuP5aG0pgEHLpgWsrsvJghAc8GuZYr0z209aVo5966oAMwY5Ye4UA/MnxACAPj+ZBE09Y0wGgWs2XMRX5g3eeyshkYDXtp8GjPe2o8v0/Lx+rZzrc755NAV/PF/mXhzx3mb7k1E18/mwLJ69Wo8+uijSExMFHtCnJyc8OGHH7Z5/qefforFixcjJiYGkZGR+Pe//w2j0YiUlBTxnEOHDmHRokWYOnUqwsLC8NhjjyE6OvqaPTdEZF+3DvMBAJzI1wAwLWDXcip1WxbcNAgA8K8fL+PRT36CUQDGDnJHmHko5yZzsfCR7IpWdSyaukakmutfkqYNwz8fjsP/nrgZr80Zg+F+rtA1GbE5owD/S8vHX7dn4flNmSitabC6hyAI2HmmBAVV9a3a9v6Pl/HxoSswmIuCC6rqW63Se/BiOQDgTKGmg78dIupuNgUWvV6PtLQ0JCQkNN9AKkVCQgJSU1M7dY+6ujo0NjbC07N5psLNN9+MzZs3o6CgAIIgYM+ePTh//jymT59uS/OIqBc9NyMSn/xmAl5/YAxevHMEXpszpsNrZo72xyBPJ9TqmsSgY+ldAYCoYDWUcinKa/W4ZJ4ybbHzbAmajAIi/V0R7uMiHpdIJHjI3Mvy8aEreO2HswAAQQB+OGk942hvVhke/c9PSNqY0aptKedMvcT/N3sEXJRyCAKs1o4xGgVxnZvL5dpOFQYfvlyB0ww3RN3CpqLb8vJyGAwG+Pn5WR338/PDuXOtu0/b8vzzzyMwMNAq9Lzzzjt47LHHEBwcDLlcDqlUivfffx9Tpkxp8x46nQ46XfMYcnV1tS2PQUTdQCGXir0snaWUy7Dj91OQnnMVx65cRUOTQQwbltdjQz1w6FIFUi9XYohvc+HtD+bhoJmj/Vvdd87YICT/cA6XyrQAAAeZBI0GAVsyC7Ho5jDxvH3nTUNQaTlXUadvgpPC9CtQU9+IzPwqAMCdYwLwTUYBThVUI7tci6Hm4t+skhrUNDQBAGoamlBeq4ePq7LdZy3S1ONX/z4CV5Ucx15MaHPKOBF1Xq/+C1q1ahU+//xzfP3111Cpmpfpfuedd3D48GFs3rwZaWlp+Nvf/oYnn3wSu3btavM+ycnJUKvV4ldISEib5xFR36NykOHmId54JmEonp8Z2WoYybIS75EWhbc1DY3Yf8E0HHPnmIBW93R3UmDmKFOQkUiAtx8aCwA4duWquA8SAHFxuyajgIzcKvH44csVMApAuI8zAt0dEeZlGqK6UqEVz7FsLmmRXa7FtWTkVqHJKOBqXSMulNZe81wi6phNgcXb2xsymQwlJSVWx0tKSuDv3/r/9bT05ptvYtWqVdixYweioqLE4/X19fjTn/6E1atX4+6770ZUVBSWLFmCefPm4c0332zzXsuWLYNGoxG/8vJsK64jor7rpnDLTKFKcdhl97lS6A1GhPs4Y6ivS5vXPTYlHK5KOZ6cOgSzxgSImz9+n2nqmdHUNeJccXNv7NEWAcRSm2JZv8YyPTq7vHlI6NgV61V6s8utQ8hPVyqx+NM0cQbUqRZDQZbeG4vqhkYs25SJvVnWkxV+vqgeETWzKbAoFArExsZaFcxaCmgnTpzY7nVvvPEGXn75ZWzbtg1xcXFWrzU2NqKxsRFSqXVTZDIZjEZjm/dTKpVwc3Oz+iKigSFmkLu5jkWHTHOdy7cZpinGd40JgEQiafO60UFqZL40HX+YMdx0bpRpS4Et5sBy9EolWpadtOwxOWAOLLeYA4vYw2LuRREEAcfMvTPDzUNEl3/Ww/LX7VnYerIYHx+6AgA4WdAcjjLyrOtY/rXvMj47modXvz8rHlv+zSlE/XkHLpexN4aoLTYPCSUlJeH999/HJ598grNnz+KJJ56AVqtFYmIiAGDhwoVYtmyZeP7rr7+O5cuX48MPP0RYWBiKi4tRXFyM2lrTP0o3NzfceuuteO6557B3715kZ2fj448/xn/+8x/MmTOnmx6TiPoLpVwm1qlsNK95Yqk9uXds0LUutQozs8b4QyoBMvKqcLmsFkezm1fjBYD0nCo0GoworKrH5TItpJLmjRsts5YsQ0L5V+tRXN0AB5kEc8aZ2nC5rDmw1OsNOG4eYjp0yTTD6XRBc0g5kVcl/rlO34T1h3MAmLY0qKrTo8lgxKb0fNTqmvC/tOZdsn+u0WC0GuIiupHYHFgsQzUrVqxATEwMMjIysG3bNrEQNzc3F0VFReL5a9euhV6vx9y5cxEQECB+tRzu+fzzzzF+/HgsWLAAI0eOxKpVq/Dqq6/i8ccf74ZHJKL+Zt54U13a5oxCfJmWD4NRQFSwGhE+bQ8HtcXXVYWpw01bCry29ZxYv/KrmwZB7eiA+kYDThdWi8NBUcHuUDs6AGgeEirSNKBebxB7Y0YHqTEywNSj27KGJS3nKvQGU4/wyfwqnC+pRYVWL76eVVIjTpH+X1o+NPWN4mvpuVdxurAaWvPGj9faS+mFr05iYvJucYXhrkq9VIH71hzEhZKa67oPUW/q0tL8S5YswZIlS9p8be/evVbfX7lypcP7+fv746OPPupKU4hoALppsBdCvZyQU1GH1TtNi7TdG3Pt3pW2/OnOSPx4vgy7zjbX3d0U7oXxYR7YdbYURy5XiKvqttx/ycPJAW4qOaobmpBTqRUDy/gwT4T7mMJMToUWBqMAmVSCQ5fKxWuNAvDhgWwAQKS/Kyq0epTV6HC6UIOYEA/8e7/pNcv903Kuwt2xeX+nS2VaXCytxZCf1eoUaerx9XFT78verDLEhlpvYmmLdfsuISOvCl+m5eNPd47o8n2IehPn2RFRnyOVSvBgnKmXRd9khFQC3B3denZQR4b4uuI3kwaL3w/ydEKA2hHjzTtW/23HeezJMg03TR/VvFyDRCIRe1nOl9SKGzTeHOGFQLUjFHIpGg0C8q+ainItGzp6mjeW/Pq4aSXvMUFqRAebVv/NyNNg++li5FbWwd3JAb+fNgyAacdry9YGlhGtHWdMvSynCzVib8znR/Ngqck9WdD1tV2aDEZx08qOZjoR9SUMLETUJ82NDYbU/AF+yxBv+Lqqrn1BO56+Yyh8zeulTDAv/T/e/L96gxEKuRRvzx+LqGB3q+ssgeWD/ZdRqdXD11WJSUO8IZVKMNhclHu5XIvqhuY1XB6/NVy8L2AaQoo233dvVile2nwaAPDwTaGYbN4A8kR+FY6ah6vmmUPa9tMl+Oe+S5j99gHMeutHFGnq8fmxXLFtJ/M14gyqjLyqNlfuNRoFHLtSiVpdk9Xxs0U14rGcCgYW6j8YWIioT/JzU2GWec2V+RMGdfk+Lko5/vZgNGJC3PFr8yJyowPVCPF0hLeLEp89Go97ogNbXWcpvLWsyDs3Nlhc/E2c9lymxbHsShgFIMzLCfePC7a6x+ggNaJD3AEA+y+Uo7RGhyG+LnhiagTCvV3g7uSAhkYjqhua4KKU45mEoZBITEW6yT+YFuMs1DTg7ncOoqRaB09nBWRSCSq0ehRpGpBVXIP73zuIB9elotHQPKtS32TEMxsz8It1qVj+zSmrNll6cwAgp6KOU6mp32BgIaI+669zo7B5yS1tLhZni8lDffDNk7eImzcq5FLs/P2tOPD8be3WglhCiYVliAqAWMdyubxWHA6aGOENbxelOO1ZKgFGBLgiKrh5Q0ilXIp3fzkWTgo5pFKJuOs1YJq9FKB2tDo2f8IguDs5oNy8O/S88SHiOjQnCzT44VQRjIJp36Ntp0zDSLW6JjzyyTFxt+kfThWhTt/cy2LpzQEAXZMRxdXW+y0R9VUMLETUZzkp5K2GarqLykEGlUP7mzVa1mIBTLtIh7UIMJYws/10Cb78ybRw5c3mjRsnmv83wscFTgo53J0UiPQ3hZgVd49EpH/zulGWKdYAEG9eMO8h8wyp39wyGK/NGY1/PRwHhUwKuVSC+eMHYYw5dJ0q0CDlbPPCcx8fugJBEPDUf9Ox/0I5nBQyeDkr0NBoxF5znY5lmAgA5ObxtivXWcdSUt0gbhhJ1JMYWIiI2tAyoFimWVtEmHs5ymp0qG5ogreLQqxJuX9cEBxkEtzdYphp3a9i8XHiePzyZ0NbcS0Di7mu5hdxIch8aTpW3D0SEokEEwZ74usnb8aXj0/EIC8njDH32Ow6W4qTBRpIJKa9k9JyruKFr05iT1YZFHIpPv1tPObGmoaotpr3YbpYVourdY1wdJCJwSq7gzqWgqp6PPLxMXx6JKfVa1tPFiH+tRRxVhRRT+rStGYiooFO7eiAmaP8UVzdgFmjrYekYoLd8btbw6HVNWHqMF/cPMRL3EgxKtgdWS/PglTavIhdmLezVQCyiA5xR6BaBZlMgjFB7uJxN5WD1XmjApuHlSzDWmeLTCvpxoS4I9TTCd9kFGKjubfnxTtHYOwgD0glEvzzx8vYfa4UDY0GcS2acaHuGOLrgv0XypFTUYf2FFTV46F/pSKvsh5HsyvxwLhgq14py7DTjxfK8OiU8HbvQ9QdGFiIiNqx7uHYNo9LpRIsm9X++iUtw8q1qBxk+OGZKYDEVFfTGSMD3CCTSsRhmIQRfrhliDe+MW9fcHukLxZODAUARAWrEeTuiIKqeuw5V4r95hWDJ4R5wcPZFIqyW2w/cKWiDsdzr+JKRR0aDUZsySxEXqVpBlKNrgk/ni/DdPMmk4IgiAGo5aq/RD2FgYWIyI7UTg4dn9SCykGGob4uOFdsWqU2YYQfhvu74t6YQFwoqcUbc6PELQokEglmjfbHvw9k4+nPj6PRYAo5EyO8UG9eeTenQgt9kxH3rz2IUy32P7II9XJCdLA7Np8oxHeZRWJguVBai0rzar4FVfWo1xvgqJBBEAQYBUBmDm1NBiO+TMvH1OE+CFA7duFviMiEgYWIqJ8ZHaTGueIaBHs4YpifqZ7mHw+NbfPcO6MC8O8D2Wg0CHBVyvHEbRGYMNgTueahoJyKOqScLcGpgmo4yCSICnbHcH9XODrIoHZ0wEMTQlBY1YDNJwqx60wJ6vRNcFLIxRWCLbLLtRgZ6IaHPziK7HIttj49GWonB3x86Ape+f4s7h8XhNUPxvTo3wsNbAwsRET9zLSRfvhfWj4ejAtpd/dqi7Eh7vjzPaPQaDDiwfEhYn1MoLsKcqkEuiYj3tl9EQDwm0mD2xzq8nFRIsTTEXmV9Ug5W4q7owNx5LL1fkaXy2vh66YUd77edroI88YPwg/m6dbXOxuJiLOEiIj6mRmj/JG67HYsuW1Ih+dKJBIsujkMv50cblXMK5dJMcjTCQBwxlzA+4vYkHbvcXeUadbTlsxCc/2KqYclzMt0j8tlWqtdqbdkFqG0pgHpuaZtAEprdDY+JZE1BhYion4oQO3Y6eLe9oSawwYAjB3k3mrDxZbuMgeWPefKsOFILspr9VDKpbhvrGlTystltVaB5dClCnz5Uz7MOwigtEYnbidA1BUMLEREN6iWU63b612xGBHgitsjfaE3GMXl/scN8hAXwrtcrkVGfvOmjAajgLdTLojf65uMqK633teIyBYMLERENyjLar4qBynu6mA3bIlEgvcWjMO0kc27Wt8U7oUIyzYFZVpxE8iZ5plEuibT/kaWGUNltdwGgLqOgYWI6AY1dbgPPJ0VePRn9S3tUTnI8N6CcZg/IQTeLkrcFR2AQV5OkEpMexhV1TVCIZdi6fRh4jURPs7iVgal1axjoa7jLCEiohtUqJcz0pdPs+kaB5kUyfdH4bU5gjhDKcTTSVwxd1SgG4b6uWJkgBvOFFVjxih/ZORV4WJpLQtv6bqwh4WIiGzWcjp1eItamGjzZpUv3TMK98YE4jeTBsPXVQkAKK3hkBB1HXtYiIjouoT7uGCPeUfomBB3AMCEwZ6YYN7Q0ccSWDgkRNeBPSxERHRdwn1a9LCYA0tLvq4qAFyLha4PAwsREV2Xob6uAEw7XIe1WNvFwteNQ0J0/TgkRERE12V8mAeeun0IRgWq29wqQBwSYg8LXQcGFiIiui4SiQRLpw9v93XLkFAZa1joOnBIiIiIepRlSKhG14R6vcHOraH+ioGFiIh6lKtSDpWD6eOmjMNC1EUMLERE1KMkEkmLmUIsvKWuYWAhIqIe58vCW7pODCxERNTjxKnN1exhoa5hYCEioh7n48IeFro+DCxERNTjfN242i1dHwYWIiLqcS0XjztdqMG+82V2bhH1N1w4joiIepyl6PbwpQrMfvsAAGBX0q0Y4utiz2ZRP8IeFiIi6nGWac16g1E8dqms1l7NoX6IPSxERNTjhvm54LbhPlA5yFBao0NazlUUVtXbu1nUjzCwEBFRj5PLpPgocQIA4JUtZxhYyGYcEiIiol4V6O4IACis4pos1HkMLERE1KssgaWAPSxkAwYWIiLqVUFiDwsDC3UeAwsREfWqQHfTjKGyWh30TcYOziYyYWAhIqJe5emsgFIuhSAAJdxbiDqJgYWIiHqVRCJhHQvZjIGFiIh6nWVYiHUs1FldCixr1qxBWFgYVCoV4uPjcfTo0XbPff/99zF58mR4eHjAw8MDCQkJbZ5/9uxZ3HPPPVCr1XB2dsb48eORm5vbleYREVEfF6hm4S3ZxubAsnHjRiQlJWHlypVIT09HdHQ0ZsyYgdLS0jbP37t3L+bPn489e/YgNTUVISEhmD59OgoKCsRzLl26hEmTJiEyMhJ79+5FZmYmli9fDpVK1fUnIyKiPqt5SIg1LNQ5EkEQBFsuiI+Px/jx4/Huu+8CAIxGI0JCQvDUU0/hhRde6PB6g8EADw8PvPvuu1i4cCEA4KGHHoKDgwPWr1/fhUcAqquroVarodFo4Obm1qV7EBFR7/niWB7++FUmbh3mg09+M8HezSE7seXz26YeFr1ej7S0NCQkJDTfQCpFQkICUlNTO3WPuro6NDY2wtPTE4Ap8Hz//fcYNmwYZsyYAV9fX8THx+Obb75p9x46nQ7V1dVWX0RE1H8EsIaFbGRTYCkvL4fBYICfn5/VcT8/PxQXF3fqHs8//zwCAwPF0FNaWora2lqsWrUKM2fOxI4dOzBnzhzcf//92LdvX5v3SE5OhlqtFr9CQkJseQwiIrKzwBaLx9nY0U83qF6dJbRq1Sp8/vnn+Prrr8X6FKPRtGjQvffei9///veIiYnBCy+8gLvuugvr1q1r8z7Lli2DRqMRv/Ly8nrtGYiI6PpZim61egOqG5oAAJfKavG79T/hpc2n7dk06qNs2q3Z29sbMpkMJSUlVsdLSkrg7+9/zWvffPNNrFq1Crt27UJUVJTVPeVyOUaOHGl1/ogRI3DgwIE276VUKqFUKm1pOhER9SGOChk8nRWo1OpxpVyLvVllWLPnIvQG0/+Jfer2IfBy4e95amZTD4tCoUBsbCxSUlLEY0ajESkpKZg4cWK7173xxht4+eWXsW3bNsTFxbW65/jx45GVlWV1/Pz58wgNDbWleURE1I9Y1mJ5+IMj+Puu89AbjJBITK+dKbKtNjGvsg5G47WHli6W1nIrgH7M5iGhpKQkvP/++/jkk09w9uxZPPHEE9BqtUhMTAQALFy4EMuWLRPPf/3117F8+XJ8+OGHCAsLQ3FxMYqLi1FbWyue89xzz2Hjxo14//33cfHiRbz77rv47rvvsHjx4m54RCIi6osCzMNC1Q1N8HZR4O35YzFrtKm3/nRh+4Fl47FcfHAgG03m3pjkrWcx+Y09+Puu8+1es+N0MRJW78NrW8924xNQb7JpSAgA5s2bh7KyMqxYsQLFxcWIiYnBtm3bxELc3NxcSKXNOWjt2rXQ6/WYO3eu1X1WrlyJl156CQAwZ84crFu3DsnJyXj66acxfPhwfPXVV5g0adJ1PBoREfVlU4b5YPe5UjwYF4wXZo6A2skBeZV12HqyGGfaCSw7Thfj+a9OAgAOXCjD2EEe+OePlwEAHx7IxiOTBsPdSYGLpTWobmjCuEEepuvOmEoZvj5egBdnj4CDjAu99zc2r8PSF3EdFiKi/knfZIRC3hwe9mSVIvGjY4jwcUbK0qlW517V6jHt7z+ivFbX6j6uKjlqGpqwdNowzBoTgHvePQB9kxF7/jAVIZ5OuPWve5BTUQcA+PS38bhliHePPpeFrskAAFDKZb3y8/qbHluHhYiIqDu1DCsAMCrQ9KF1uVyLOn2TeFwQBCz/9hTKa3UY5ueCL343Ed7motxHJw/GK/eNBgB8dOgKlvw3HXV6A5qMAnafK0VpdYMYVgBg++nOLcPRkXdSLuDvO8+3Oy1b32TEfWsOYfLre1DT0NgtP/NGZvOQEBERUU/xdVXB20WJ8lodzhXXYGyIO7afLsG7ey7gVEE1ZFIJ3vxFNKKC3bH92ck4X1KLm8I9YTAKeHNHFvIq61Gp1Yv3232uFD6upmDjIJOg0SBg++livHT3KEilki6380JJDf6201QzM22kH0YHqVuds/GnPJw1Fw8fvlyJaSP9Wp1DncceFiIi6lMsvSynC6uxdt8lPL4hDacKquHoIMOf7xmFqGB3AICXixITI7wgkUggl0nx2ORwAIBEAqy4y7RURurlCuy/UAYAuH9sMFyUcpRU63Aiv+q62rj1ZHMvzZbMolavNzQa8O7uC+L3hy6VX9fP64zLZbX4Ki2/w9lS/RUDCxER9SkjzYFl//kyrNl9EQDwm1sG4+ALt+NXN7W/3MWD40Pw65vD8PoDUUi8JQxB7o7QNxnxVZpps92bh3hh6nAfAMD20yXt3qcztp5sDilbMgtbDQttOJyDkmodLJ04qZcqruvnWWh1Tdh8orDN6dkvfHUSS788gU9Sr3TLz2rp8OUKfHeisNvvawsGFiIi6lMsPSw7zpRAqzcgKliN5XeNgKez4prXKeUyvHTPKDwYFwKJRILbIk3hxLIY3fgwT8wYZZo2/cOpolYhQ6trwru7L+D3GzOQ+NFRLNuUiYMXy2H4WY/FxdJaZJXUQC6VwNFBhvyr9TiRr7G6z3t7LwEAlk4fDgA4V1yDijaKhW31tx3n8fRnx/GPFOsp3AajgJMFpja8u/sianVNbV1us0tltXj0Pz/hoX8dxv99cwpVdfqOL+ohDCxERNSnjAq0rgf544xISCS215vcHukr/jnI3RGB7o64LdIXjg4y5FTUIS3nqvi6wSjgqc+O480d5/H18QLsySrDZ0fzsODfRzD1zT3Iq2wu2v3B3Lsyaag3Esx1KVta9D5sO1WMSq0egzyd8NiUcET6uwIw1bEIgoBzxdXi7CFbCIKAHWdMQ1FfpxdYDf3kVtahvtF0zwqtHh/sz7b5/j93Iq8KM9/6ETvPlEAmleCuqADYc14xAwsREfUpoZ5OcFaYpgHfHOGFSUO7NgV5Yrg3lOZZSHFhpvVYXJRyzI4KAABsPNa8D90r35/B7nOlUMql+MP0YVh1/xj8Mn4Q3J0ckFdZj6QvMsSelu/NgeXO0QG4y3yv708WiQHiu0xTeLl/XBAcZFJMjPACAKReLsc/Ui5g5lv78eZ269XdW7pYWoPVO88jq7jG6nh2uRb5V027WxdqGpCW2xy4zpmLey3P+68fLyG7XHtdG0t+cugKGg0CYkM9sP3ZyXh1zhh4dNDL1ZMYWIiIqE+RSiW4fYQfHB1keGFWZJfv46iQYcow07DQLRHNoWfe+BAAppBRq2vCf1Kv4KODVwAAqx+MwZLbh+KhCYPw2pwx+G7JJLgo5Th25SrW7r2IDYdzcK7YNBw0fZQfbh3mA1elHEWaBqRerkClVo8DF0wFtndFBQIAJoabAsvmjEL8I8VUiHv4cmWr9pbX6vDM58cx7e8/4u2UC0j6IsPq9R/Pl1l937Km5Jw53NwdHYjRQW7Q6g247c29iPrzDqw1D091RBAEsTamXm8Qp3//6c5IDPF17dQ9ehIDCxER9TlvzYvB4T/dIc4I6qrX5ozBPx6KwdzYYPFYXKgHwr2dUac34MWvT4q7Qz83Y7jY+2IR4umElXebZhy9ueM8/u+bUwCA2VEBcHdSQOUgw71jTcFk1Q/nsPVkEZqMAkYGuGGIrwsAID7cC1KJaQsCS4fH+ZKaVrUxf9p0Et9mFEIQTDOdThdW43xJcy/LPnNguWWIKQBtPVkkbk9wrtjUwzIywA2vPxCFUYFukEklqGlowvv7L3fY0yIIAn75/hHcvGo3civqkHLOVD8U7OEorhZsbwwsRETU58ikEqgdHa77Pj6uStwbE2S15opEIsEv4ky9LN9mFMIoAL+IDcbiqRFt3mNubDBmmot1PZwcsPyukXhjbpT4+jN3DIOLUo6TBRq8vu0cAFNPh4Xa0UFcp2VUoBuUcil0TUbkVGjFcwqr6rHrrGnm0he/m4iEEabamK+Pm2Y4NTQakHrZNNNo2awR8HByQHmtHofMs48sPSyR/q4YFajG909PxomV0yGVAJVaPUprrl3we+zKVaRerkB5rQ4vbMrEN+afe29MYJfqh3oCAwsREd1wHogNgswcYiaGe+HVOWPa/WCWSCT4x/wYfLAoDvv+eBsemTTYaql9H1clnrp9CACgpsE0O+eun/XULJ0+HHeO8ce6X8ViqJ+p56Vl78nnx/JgFExtmTDYE3PGBgEAvj1uKq49dqUSDY1G+LkpMSrQDXeOMd3/24xCaHVN4kq+w/2bh25clHKE+5h+Vnt7M1n8p8VU6EOXKrDrbCkA4N6YoGte15sYWIiI6Ibj66rCH6YPx8xRphDx8y0Cfk4pl+GOEX5wU7Xd6/PrW8IQ6uUEABg7yB0hnk5Wr986zAfvLYhFiKcThvuZpm1bekUaDUZ8fjQXALDgpkEATDOcXFVyFGoacCS7EnvOlYn3kUgkYqDZklmIw+aeF19XJbzM2xVYjAgw/awzRe0HltLqBmw7ZapXeTCueehsRIAbhvnZv3bFgkvzExHRDemJdoaAukIpl+H1B6Lwf9+cEntb2mOZ5myZBZRythSlNTp4uygwfaRp6EnlIMNdUQH47Ggenv78OMrMQzqWIuLYUA+MCVLjZIEGL285Y7pvQOvNA0cGuOG7E4XiFgEWhVX1+Mt3ZxAd4o6qOj2ajALiQj2QfH8ULpbWIj23Cg+M6zu9KwADCxERUbe4KdwLu5Ju7fC84T8LLJ8eyQEAPBgXYtXTc19MED47moeyGh0kEmBOTJC48J1EIsFvJw/GM59n4Ip5OGiEf+vekBEBpmM/Dyzv7b2IbaeLsa3FRpAPTwyFTCrBR7+egB8vlGHWaP9OP3tvYGAhIiLqRZYelisVWmQV12D/hXJIJMD8CYOszpsw2BNP3zEUtQ1NeHhiKAZ7O1u9fueYAKz64RyKNA0ArOtXLEaae12yy7VoaDRA5SBDk8EoDgH5uipRWqNDgFqFWaNNdTFqJwerouG+goGFiIioF/m4KuHu5ICqukb8ZYtpSvWtw3xa1b1IJBIkTRvW7n0cZFIk3hKG17aaZiZF+rceEvJxVcLLWYEKrR5ZxTWIDnHH0exKlNfq4e7kgP3P34a0nKsI8XDqsI7H3vp264iIiAYYiUSC4eZi1oMXTQWzC+Lb39TxWh6aMAi+rkoEqFWI8HVu9bpEImlVeGtZqXfmKH8o5TLcHOHdKiz1RexhISIi6mWR/q44km1a7TZArcJt5l2kbeWmcsD2Z6dAIoHVVOuWRga64cDFcpwtqrYaDrJMje4vGFiIiIh62fAWwzcPjR8EuazrAx4d7e/TsvD2SHYlKrR6eDg5iHsc9RccEiIiIuplkeYQIZNKxL2NeoplSOhEngZPf3YcADBztD8criMk2QN7WIiIiHpZTLA7Hp08GGHezvBXq3r0Z0X4uMBFKUetrgkVWj1UDlI8NH5Qxxf2MRLhevae7iOqq6uhVquh0Wjg5ta6SpqIiOhGduRyBU4XVmNUoBtGB6nhrOwb/RW2fH73jRYTERFRj4kP90J8eP+qWfm5/jWARURERDckBhYiIiLq8xhYiIiIqM9jYCEiIqI+j4GFiIiI+jwGFiIiIurzGFiIiIioz2NgISIioj6PgYWIiIj6PAYWIiIi6vMYWIiIiKjPY2AhIiKiPo+BhYiIiPq8AbFbsyAIAEzbVBMREVH/YPnctnyOX8uACCw1NTUAgJCQEDu3hIiIiGxVU1MDtVp9zXMkQmdiTR9nNBpRWFgIV1dXSCSSbr13dXU1QkJCkJeXBzc3t269d18x0J9xoD8fwGccCAb68wF8xoGgu59PEATU1NQgMDAQUum1q1QGRA+LVCpFcHBwj/4MNze3AfkfX0sD/RkH+vMBfMaBYKA/H8BnHAi68/k66lmxYNEtERER9XkMLERERNTnMbB0QKlUYuXKlVAqlfZuSo8Z6M840J8P4DMOBAP9+QA+40Bgz+cbEEW3RERENLCxh4WIiIj6PAYWIiIi6vMYWIiIiKjPY2AhIiKiPo+BpQNr1qxBWFgYVCoV4uPjcfToUXs3qUuSk5Mxfvx4uLq6wtfXF/fddx+ysrKszpk6dSokEonV1+OPP26nFtvupZdeatX+yMhI8fWGhgY8+eST8PLygouLCx544AGUlJTYscW2CQsLa/V8EokETz75JID++f79+OOPuPvuuxEYGAiJRIJvvvnG6nVBELBixQoEBATA0dERCQkJuHDhgtU5lZWVWLBgAdzc3ODu7o5HHnkEtbW1vfgU13atZ2xsbMTzzz+PMWPGwNnZGYGBgVi4cCEKCwut7tHWe79q1apefpK2dfQe/vrXv27V9pkzZ1qd05/fQwBt/ruUSCT461//Kp7Tl9/Dznw+dOb3Z25uLmbPng0nJyf4+vriueeeQ1NTU7e1k4HlGjZu3IikpCSsXLkS6enpiI6OxowZM1BaWmrvptls3759ePLJJ3H48GHs3LkTjY2NmD59OrRardV5jz76KIqKisSvN954w04t7ppRo0ZZtf/AgQPia7///e/x3Xff4csvv8S+fftQWFiI+++/346ttc2xY8esnm3nzp0AgF/84hfiOf3t/dNqtYiOjsaaNWvafP2NN97A22+/jXXr1uHIkSNwdnbGjBkz0NDQIJ6zYMECnD59Gjt37sSWLVvw448/4rHHHuutR+jQtZ6xrq4O6enpWL58OdLT07Fp0yZkZWXhnnvuaXXuX/7yF6v39qmnnuqN5neoo/cQAGbOnGnV9s8++8zq9f78HgKweraioiJ8+OGHkEgkeOCBB6zO66vvYWc+Hzr6/WkwGDB79mzo9XocOnQIn3zyCT7++GOsWLGi+xoqULsmTJggPPnkk+L3BoNBCAwMFJKTk+3Yqu5RWloqABD27dsnHrv11luFZ555xn6Nuk4rV64UoqOj23ytqqpKcHBwEL788kvx2NmzZwUAQmpqai+1sHs988wzQkREhGA0GgVB6P/vHwDh66+/Fr83Go2Cv7+/8Ne//lU8VlVVJSiVSuGzzz4TBEEQzpw5IwAQjh07Jp7zww8/CBKJRCgoKOi1tnfWz5+xLUePHhUACDk5OeKx0NBQ4e9//3vPNq4btPV8ixYtEu699952rxmI7+G9994r3H777VbH+st7KAitPx868/tz69atglQqFYqLi8Vz1q5dK7i5uQk6na5b2sUelnbo9XqkpaUhISFBPCaVSpGQkIDU1FQ7tqx7aDQaAICnp6fV8U8//RTe3t4YPXo0li1bhrq6Ons0r8suXLiAwMBAhIeHY8GCBcjNzQUApKWlobGx0er9jIyMxKBBg/rl+6nX67Fhwwb85je/sdrws7+/fy1lZ2ejuLjY6j1Tq9WIj48X37PU1FS4u7sjLi5OPCchIQFSqRRHjhzp9TZ3B41GA4lEAnd3d6vjq1atgpeXF8aOHYu//vWv3drV3tP27t0LX19fDB8+HE888QQqKirE1wbae1hSUoLvv/8ejzzySKvX+st7+PPPh878/kxNTcWYMWPg5+cnnjNjxgxUV1fj9OnT3dKuAbH5YU8oLy+HwWCw+ssHAD8/P5w7d85OreoeRqMRzz77LG655RaMHj1aPP7LX/4SoaGhCAwMRGZmJp5//nlkZWVh06ZNdmxt58XHx+Pjjz/G8OHDUVRUhD//+c+YPHkyTp06heLiYigUilYfAn5+figuLrZPg6/DN998g6qqKvz6178Wj/X39+/nLO9LW/8GLa8VFxfD19fX6nW5XA5PT89++b42NDTg+eefx/z58602lnv66acxbtw4eHp64tChQ1i2bBmKioqwevVqO7a2c2bOnIn7778fgwcPxqVLl/CnP/0Js2bNQmpqKmQy2YB7Dz/55BO4urq2Gm7uL+9hW58Pnfn9WVxc3Oa/Vctr3YGB5Qb05JNP4tSpU1b1HQCsxozHjBmDgIAA3HHHHbh06RIiIiJ6u5k2mzVrlvjnqKgoxMfHIzQ0FF988QUcHR3t2LLu98EHH2DWrFkIDAwUj/X39+9G19jYiAcffBCCIGDt2rVWryUlJYl/joqKgkKhwO9+9zskJyf3+SXgH3roIfHPY8aMQVRUFCIiIrB3717ccccddmxZz/jwww+xYMECqFQqq+P95T1s7/OhL+CQUDu8vb0hk8laVUGXlJTA39/fTq26fkuWLMGWLVuwZ88eBAcHX/Pc+Ph4AMDFixd7o2ndzt3dHcOGDcPFixfh7+8PvV6Pqqoqq3P64/uZk5ODXbt24be//e01z+vv75/lfbnWv0F/f/9WRfBNTU2orKzsV++rJazk5ORg586dVr0rbYmPj0dTUxOuXLnSOw3sRuHh4fD29hb/uxwo7yEA7N+/H1lZWR3+2wT65nvY3udDZ35/+vv7t/lv1fJad2BgaYdCoUBsbCxSUlLEY0ajESkpKZg4caIdW9Y1giBgyZIl+Prrr7F7924MHjy4w2syMjIAAAEBAT3cup5RW1uLS5cuISAgALGxsXBwcLB6P7OyspCbm9vv3s+PPvoIvr6+mD179jXP6+/v3+DBg+Hv72/1nlVXV+PIkSPiezZx4kRUVVUhLS1NPGf37t0wGo1iYOvrLGHlwoUL2LVrF7y8vDq8JiMjA1KptNVQSn+Qn5+PiooK8b/LgfAeWnzwwQeIjY1FdHR0h+f2pfewo8+Hzvz+nDhxIk6ePGkVPi3he+TIkd3WUGrH559/LiiVSuHjjz8Wzpw5Izz22GOCu7u7VRV0f/HEE08IarVa2Lt3r1BUVCR+1dXVCYIgCBcvXhT+8pe/CD/99JOQnZ0tfPvtt0J4eLgwZcoUO7e885YuXSrs3btXyM7OFg4ePCgkJCQI3t7eQmlpqSAIgvD4448LgwYNEnbv3i389NNPwsSJE4WJEyfaudW2MRgMwqBBg4Tnn3/e6nh/ff9qamqE48ePC8ePHxcACKtXrxaOHz8uzpBZtWqV4O7uLnz77bdCZmamcO+99wqDBw8W6uvrxXvMnDlTGDt2rHDkyBHhwIEDwtChQ4X58+fb65FaudYz6vV64Z577hGCg4OFjIwMq3+blpkVhw4dEv7+978LGRkZwqVLl4QNGzYIPj4+wsKFC+38ZCbXer6amhrhD3/4g5CamipkZ2cLu3btEsaNGycMHTpUaGhoEO/Rn99DC41GIzg5OQlr165tdX1ffw87+nwQhI5/fzY1NQmjR48Wpk+fLmRkZAjbtm0TfHx8hGXLlnVbOxlYOvDOO+8IgwYNEhQKhTBhwgTh8OHD9m5SlwBo8+ujjz4SBEEQcnNzhSlTpgienp6CUqkUhgwZIjz33HOCRqOxb8NtMG/ePCEgIEBQKBRCUFCQMG/ePOHixYvi6/X19cLixYsFDw8PwcnJSZgzZ45QVFRkxxbbbvv27QIAISsry+p4f33/9uzZ0+Z/l4sWLRIEwTS1efny5YKfn5+gVCqFO+64o9WzV1RUCPPnzxdcXFwENzc3ITExUaipqbHD07TtWs+YnZ3d7r/NPXv2CIIgCGlpaUJ8fLygVqsFlUoljBgxQnjttdesPvDt6VrPV1dXJ0yfPl3w8fERHBwchNDQUOHRRx9t9X/6+vN7aPHPf/5TcHR0FKqqqlpd39ffw44+HwShc78/r1y5IsyaNUtwdHQUvL29haVLlwqNjY3d1k6JubFEREREfRZrWIiIiKjPY2AhIiKiPo+BhYiIiPo8BhYiIiLq8xhYiIiIqM9jYCEiIqI+j4GFiIiI+jwGFiIiIurzGFiIiIioz2NgISIioj6PgYWIiIj6PAYWIiIi6vP+HxDE1VzFDLCiAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "id": "0844da6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Put layers into evaluation mode (needed esp. for batchnorm)\n",
    "for layer in model.layers:\n",
    "    layer.training = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "28bf7e11",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 1.7686810493469238\n",
      "val 1.9930694103240967\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the loss\n",
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]   # That's a switch!\n",
    "    \n",
    "    logits = model(x)\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(split, loss.item())\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "961d265e",
   "metadata": {},
   "source": [
    "Performance log:\n",
    "\n",
    "|                                      |train|val  |\n",
    "| ------------------------------------ | --- | --- |\n",
    "| original (3 characters, 12K params)  |2.058|2.105|\n",
    "| context: 3 -> 8 (22K params)         |1.918|2.033|\n",
    "| flat -> hierarchical (22K params)    |1.942|2.029|\n",
    "| fix bug in batchnorm (22K params)    |1.911|2.019|\n",
    "| scale up the network (76K params)    |1.769|1.993|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "id": "52211f08",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apoljwah.\n",
      "tayone.\n",
      "jossadette.\n",
      "helejwa.\n",
      "jamyah.\n",
      "abrsei.\n",
      "loge.\n",
      "brennet.\n",
      "blayah.\n",
      "aragras.\n",
      "anny.\n",
      "annayvi.\n",
      "aurah.\n",
      "jayce.\n",
      "nodine.\n",
      "remit.\n",
      "dniya.\n",
      "ganiyla.\n",
      "demary.\n",
      "anaya.\n"
     ]
    }
   ],
   "source": [
    "# Sample from the model\n",
    "for _ in range(20):\n",
    "    \n",
    "    out = []\n",
    "    context = [0] * block_size # initialize with all ...\n",
    "    while True:\n",
    "      # Forward pass though the neural net\n",
    "      logits = model(torch.tensor([context]))\n",
    "      probs = F.softmax(logits, dim=1)\n",
    "      # Sample from the distribution\n",
    "      ix = torch.multinomial(probs, num_samples=1).item()\n",
    "      # Shift the context window and track the samples\n",
    "      context = context[1:] + [ix]\n",
    "      out.append(ix)\n",
    "      # As soon as we sample the special '.' token, break\n",
    "      if ix == 0:\n",
    "        break\n",
    "    \n",
    "    # Decode and print the generated words (Inferences)\n",
    "    print(''.join(itos[i] for i in out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a7aae9c",
   "metadata": {},
   "source": [
    "Normally, you would look deeply at both training loss and validation loss during model optimization.<br>\n",
    "We didn't do that, just taking the graph with the training loss as some sort of guide was sufficient for our purposes.<br>\n",
    "Technically, we now implemented the method layed out in the [WaveNet](https://arxiv.org/abs/1609.03499) paper.\n",
    "\n",
    "Now, you can name what we just did by its more familiar name: Simple 1D convolutions. \n",
    "\n",
    "![](https://media.tenor.com/Vr9P0tRw8QIAAAAM/look-holy.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b0e086b",
   "metadata": {},
   "source": [
    "## Development Process of Deep Neural Networks\n",
    "\n",
    "1. We spent a lot of time in the documentation page of [PyTorch](https://pytorch.org/docs/stable/nn.html) (actually, this documentation is not that good)\n",
    "2. There's a lot of gymnastics around these multidimensional arrays we trickle through a model (3D, 2D, what input shapes, what output shapes, what hidden traps)\n",
    "3. Prototyping shapes and layers in Jupyter notebooks, put VS Code and Jupyter Notebook on the side\n",
    "\n",
    "### Rough Roadmap\n",
    "\n",
    "We'll consider using the 'real' `torch.nn` to be an unlocked achievement from now on.<br>\n",
    "We know what it's doing in complex enough terms. **RNNs and LSTMs will follow.**\n",
    "\n",
    "### Challenge for now\n",
    "\n",
    "Beat the $1.993$ validation loss of the last model through changing the model and/or its parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "id": "1efeaf1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "77145\n"
     ]
    }
   ],
   "source": [
    "n_embd = 26    # RAISED Dimensionality of the character embedding vectors\n",
    "n_hidden = 128 # Number of neurons in the hidden layer of the MLP\n",
    "\n",
    "# This sets up the actual network's design\n",
    "model = Sequential([\n",
    "    Embedding(vocab_size, n_embd),   # Previously C = torch.randn((vocab_size, n_embd))\n",
    "    # The new layer at work now (block_size was 8 before)\n",
    "    FlattenConsecutive(2), Linear(n_embd * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    FlattenConsecutive(2), Linear(n_hidden * 2, n_hidden, bias=False), BatchNorm1d(n_hidden), Tanh(),\n",
    "    Linear(n_hidden, vocab_size),\n",
    "])\n",
    "\n",
    "# Parameter init\n",
    "with torch.no_grad():\n",
    "  layers[-1].weight *= 0.1 # Making last layer less confident\n",
    "\n",
    "# before: [p for layer in layers for p in layer.parameters()]\n",
    "parameters = model.parameters()\n",
    "print(sum(p.nelement() for p in parameters)) # Number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "id": "b4f29a7d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.3710\n",
      "  10000/ 200000: 2.2575\n",
      "  20000/ 200000: 2.4482\n",
      "  30000/ 200000: 1.9136\n",
      "  40000/ 200000: 2.3482\n",
      "  50000/ 200000: 1.9565\n",
      "  60000/ 200000: 1.8290\n",
      "  70000/ 200000: 1.7270\n",
      "  80000/ 200000: 1.8911\n",
      "  90000/ 200000: 1.6795\n",
      " 100000/ 200000: 2.0539\n",
      " 110000/ 200000: 2.2396\n",
      " 120000/ 200000: 2.2207\n",
      " 130000/ 200000: 1.3878\n",
      " 140000/ 200000: 1.5571\n",
      " 150000/ 200000: 1.8661\n",
      " 160000/ 200000: 1.4331\n",
      " 170000/ 200000: 2.0536\n",
      " 180000/ 200000: 1.9294\n",
      " 190000/ 200000: 1.8688\n"
     ]
    }
   ],
   "source": [
    "# Full Training\n",
    "# Same optimization as last time (written a bit differently)\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "for i in range(max_steps):\n",
    "    \n",
    "    # Mini-batch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,))\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # X,Y's of this batch\n",
    "    \n",
    "    # Forward-Pass\n",
    "    logits = model(Xb)\n",
    "    loss = F.cross_entropy(logits, Yb)  # Loss function\n",
    "    \n",
    "    # Backward-Pass\n",
    "    for p in parameters:\n",
    "        # Making sure that prior iteration's gradients are gone\n",
    "        p.grad = None\n",
    "    loss.backward() # This calculates the current (new) gradients\n",
    "    \n",
    "    # Update (SGD with no fluff)\n",
    "    lr = 0.1 if i < 150000 else 0.01 if i < 180000 else 0.001 # Learning rate decay\n",
    "    for p in parameters:\n",
    "        p.data += -lr * p.grad\n",
    "\n",
    "    # Printing track stats every once in a while\n",
    "    if i % 10000 == 0:\n",
    "        print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    \n",
    "    # Adding current loss value to historic losses set (for later display)\n",
    "    lossi.append(loss.log10().item())\n",
    "    \n",
    "    #break # We see if this works first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "id": "052bc248",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 1.7690072059631348\n",
      "val 1.984166145324707\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABhqElEQVR4nO3dd3xV9eH/8de9N3svskhC2GGFMCMKiBIBi9VqtWitWNpq3a2xVtGq39b6C7WuqlRat2CVDnAXhcgQCUMgzBBWICGQBWTv3PP74yYXUhLIhSQ3Ce/n43Efj+Tczzn5HA7kvvlMk2EYBiIiIiJdmNnZFRARERE5FwUWERER6fIUWERERKTLU2ARERGRLk+BRURERLo8BRYRERHp8hRYREREpMtTYBEREZEuz8XZFWgPVquVo0eP4uvri8lkcnZ1REREpA0Mw6CsrIzIyEjM5rO3ofSIwHL06FGio6OdXQ0RERE5Dzk5OURFRZ21TI8ILL6+voDthv38/JxcGxEREWmL0tJSoqOj7Z/jZ9MjAktTN5Cfn58Ci4iISDfTluEcGnQrIiIiXZ4Ci4iIiHR5CiwiIiLS5SmwiIiISJenwCIiIiJdngKLiIiIdHkKLCIiItLlnVdgmT9/PrGxsXh4eJCYmMjGjRtbLbtkyRLGjh1LQEAA3t7eJCQksHDhwmZlysvLue+++4iKisLT05OhQ4eyYMGC86maiIiI9EAOLxy3ePFikpOTWbBgAYmJibz00ktMnz6dzMxMQkNDzygfFBTE448/TlxcHG5ubnz22WfMmTOH0NBQpk+fDkBycjJff/01ixYtIjY2lq+++op77rmHyMhIrr322gu/SxEREenWTIZhGI6ckJiYyLhx43j11VcB28aD0dHR3H///Tz66KNtusbo0aOZOXMmTz/9NADDhw9n1qxZPPHEE/YyY8aM4eqrr+aPf/zjOa9XWlqKv78/JSUlWulWRESkm3Dk89uhLqHa2lo2b95MUlLSqQuYzSQlJZGWlnbO8w3DIDU1lczMTCZPnmw/fumll/LJJ5+Qm5uLYRisXLmSvXv3Mm3atBavU1NTQ2lpabOXiIiI9FwOdQkVFRXR0NBAWFhYs+NhYWHs2bOn1fNKSkro3bs3NTU1WCwW/vrXv3LVVVfZ33/llVe48847iYqKwsXFBbPZzOuvv94s1JwuJSWF3//+945UXURERLqxTtn80NfXl/T0dMrLy0lNTSU5OZl+/foxZcoUwBZY1q9fzyeffEKfPn1Ys2YN9957L5GRkc1ac5rMnTuX5ORk+/dNuz22t+q6Bp77MpPq+gae+v4wXC2aVCUiIuIMDgWWkJAQLBYL+fn5zY7n5+cTHh7e6nlms5kBAwYAkJCQQEZGBikpKUyZMoWqqioee+wxli5dysyZMwGIj48nPT2d5557rsXA4u7ujru7uyNVPy8mE7yxNguA386IU2ARERFxEoc+gd3c3BgzZgypqan2Y1arldTUVCZMmNDm61itVmpqagCoq6ujrq4Os7l5VSwWC1ar1ZHqtTs3i5mmHa+r6xqcWhcREZGLmcNdQsnJydx+++2MHTuW8ePH89JLL1FRUcGcOXMAmD17Nr179yYlJQWwjTcZO3Ys/fv3p6amhi+++IKFCxfy2muvAeDn58fll1/Oww8/jKenJ3369GH16tW89957vPDCC+14q44zmUx4uFioqmuguta54UlERORi5nBgmTVrFoWFhTz55JPk5eWRkJDAsmXL7ANxs7Ozm7WWVFRUcM8993DkyBE8PT2Ji4tj0aJFzJo1y17mww8/ZO7cudx6662cOHGCPn368Mwzz3DXXXe1wy1eGA9Xsy2w1KuFRURExFkcXoelK+rIdVguTUnlaEk1n9x3GfFRAe16bRERkYtZh63DcjHycLUAUF2nLiERERFnUWA5B/fGwFKlQbciIiJOo8ByDp6utj8izRISERFxHgWWczjVJaTAIiIi4iwKLOegwCIiIuJ8Cizn4GHvEtKgWxEREWdRYDkHtbCIiIg4nwLLOWhas4iIiPMpsJyDh4umNYuIiDibAss5eGhas4iIiNMpsJyDZ2OXUI32EhIREXEaBZZzaBrDUlWrwCIiIuIsCiznoGnNIiIizqfAcg72WULqEhIREXEaBZZz0DosIiIizqfAcg72MSzqEhIREXEaBZZzaBrDUqMWFhEREadRYDkHT3UJiYiIOJ0CyzloaX4RERHnU2A5h6YuIS3NLyIi4jwKLOfg7qIuIREREWdTYDkHT7empfmtWK2Gk2sjIiJycVJgOYemMSxgCy0iIiLS+RRYzsHD5dQfkbqFREREnEOB5RxcLGZczCZAy/OLiIg4iwJLG3hqarOIiIhTKbC0gXvT8vy1amERERFxBgWWNmhai0VdQiIiIs6hwNIGWp5fRETEuRRY2qBpanONxrCIiIg4hQJLG2h5fhEREedSYGkDD3UJiYiIOJUCSxtox2YRERHnUmBpg6bAoi4hERER51BgaYOm5fnVJSQiIuIcCixtcGqWkAKLiIiIMyiwtIGnW+MYFu3WLCIi4hQKLG3Q1CWkpflFREScQ4GlDdw1rVlERMSpFFjawL40v7qEREREnEKBpQ08tFuziIiIU51XYJk/fz6xsbF4eHiQmJjIxo0bWy27ZMkSxo4dS0BAAN7e3iQkJLBw4cIzymVkZHDttdfi7++Pt7c348aNIzs7+3yq1+6aluav0W7NIiIiTuFwYFm8eDHJyck89dRTbNmyhZEjRzJ9+nQKCgpaLB8UFMTjjz9OWloa27dvZ86cOcyZM4cvv/zSXubAgQNMnDiRuLg4Vq1axfbt23niiSfw8PA4/ztrR1qaX0RExLlMhmEYjpyQmJjIuHHjePXVVwGwWq1ER0dz//338+ijj7bpGqNHj2bmzJk8/fTTANx88824urq22PLSFqWlpfj7+1NSUoKfn995XeNsVu4pYM47mxjR259P75/Y7tcXERG5GDny+e1QC0ttbS2bN28mKSnp1AXMZpKSkkhLSzvn+YZhkJqaSmZmJpMnTwZsgefzzz9n0KBBTJ8+ndDQUBITE/noo49avU5NTQ2lpaXNXh3JXbs1i4iIOJVDgaWoqIiGhgbCwsKaHQ8LCyMvL6/V80pKSvDx8cHNzY2ZM2fyyiuvcNVVVwFQUFBAeXk58+bNY8aMGXz11Vdcf/313HDDDaxevbrF66WkpODv729/RUdHO3IbDlOXkIiIiHO5dMYP8fX1JT09nfLyclJTU0lOTqZfv35MmTIFq9U2Vfi6667jwQcfBCAhIYF169axYMECLr/88jOuN3fuXJKTk+3fl5aWdmho8XDRbs0iIiLO5FBgCQkJwWKxkJ+f3+x4fn4+4eHhrZ5nNpsZMGAAYAsjGRkZpKSkMGXKFEJCQnBxcWHo0KHNzhkyZAhr165t8Xru7u64u7s7UvUL0rQ0v/YSEhERcQ6HuoTc3NwYM2YMqamp9mNWq5XU1FQmTJjQ5utYrVZqamrs1xw3bhyZmZnNyuzdu5c+ffo4Ur0O46ExLCIiIk7lcJdQcnIyt99+O2PHjmX8+PG89NJLVFRUMGfOHABmz55N7969SUlJAWzjTcaOHUv//v2pqanhiy++YOHChbz22mv2az788MPMmjWLyZMnc8UVV7Bs2TI+/fRTVq1a1T53eYGauoTqrQb1DVZcLFpvT0REpDM5HFhmzZpFYWEhTz75JHl5eSQkJLBs2TL7QNzs7GzM5lMf6BUVFdxzzz0cOXIET09P4uLiWLRoEbNmzbKXuf7661mwYAEpKSk88MADDB48mP/85z9MnNg1phA3DboF2/L8PgosIiIincrhdVi6oo5eh8VqNej32BcAbHo8iV6+nTd+RkREpKfqsHVYLlZmswl3F9sflaY2i4iIdD4FljZq6hbSfkIiIiKdT4GljTxdtRaLiIiIsyiwtJGmNouIiDiPAksbaXl+ERER51FgaSN3dQmJiIg4jQJLG/m625asKauuc3JNRERELj4KLG0U5O0GwImKWifXRERE5OKjwNJGTYHluAKLiIhIp1NgaaPgphaWcgUWERGRzqbA0kZBPmphERERcRYFljYK8moaw1Lj5JqIiIhcfBRY2kiDbkVERJxHgaWNgtUlJCIi4jQKLG0U5O0OQFl1PXUNWjxORESkMymwtFGApytmk+3rk2plERER6VQKLG1kNpsI9FK3kIiIiDMosDhAA29FREScQ4HFAVrtVkRExDkUWBzQNFPoRLnWYhEREelMCiwOUJeQiIiIcyiwOCBIg25FREScQoHFAWphERERcQ4FFgcE+dgWj1NgERER6VwKLA4IVguLiIiIUyiwOEBdQiIiIs6hwOKAphaWk5W1WK2Gk2sjIiJy8VBgcUBgY2CxGlBcVefk2oiIiFw8FFgc4Gox4+fhAsCJCi0eJyIi0lkUWBwU3DhT6Hi5xrGIiIh0FgUWB2ngrYiISOdTYHFQoFa7FRER6XQKLA6yzxRSYBEREek0CiwOCmrcsblQOzaLiIh0GgUWBw3o5QPA9iMlTq6JiIjIxUOBxUHj+wYBsDO3hMraeifXRkRE5OKgwOKgqEBPIvw9qLcapGcXO7s6IiIiFwUFFgeZTCbGxdpaWTZknXBybURERC4OCiznoalbaNMhBRYREZHOoMByHpoCy5bsk9TWW51cGxERkZ7vvALL/PnziY2NxcPDg8TERDZu3Nhq2SVLljB27FgCAgLw9vYmISGBhQsXtlr+rrvuwmQy8dJLL51P1TrFgF4+BHq5Ul1nZedRzRYSERHpaA4HlsWLF5OcnMxTTz3Fli1bGDlyJNOnT6egoKDF8kFBQTz++OOkpaWxfft25syZw5w5c/jyyy/PKLt06VLWr19PZGSk43fSicxmE2Mbx7Fs0jgWERGRDudwYHnhhRe44447mDNnDkOHDmXBggV4eXnx1ltvtVh+ypQpXH/99QwZMoT+/fvzq1/9ivj4eNauXdusXG5uLvfffz/vv/8+rq6u53c3nSixsVtoowKLiIhIh3MosNTW1rJ582aSkpJOXcBsJikpibS0tHOebxgGqampZGZmMnnyZPtxq9XKbbfdxsMPP8ywYcPOeZ2amhpKS0ubvTpbUwvL5uyTGIbR6T9fRETkYuJQYCkqKqKhoYGwsLBmx8PCwsjLy2v1vJKSEnx8fHBzc2PmzJm88sorXHXVVfb3//SnP+Hi4sIDDzzQpnqkpKTg7+9vf0VHRztyG+1iaIQfbi5miivrOHS8stN/voiIyMWkU2YJ+fr6kp6ezqZNm3jmmWdITk5m1apVAGzevJm//OUvvPPOO5hMpjZdb+7cuZSUlNhfOTk5HVj7lrm5mBnR2x+ArdknO/3ni4iIXEwcCiwhISFYLBby8/ObHc/Pzyc8PLz1H2I2M2DAABISEnjooYe48cYbSUlJAeCbb76hoKCAmJgYXFxccHFx4fDhwzz00EPExsa2eD13d3f8/PyavZwhIToAgK1a8VZERKRDORRY3NzcGDNmDKmpqfZjVquV1NRUJkyY0ObrWK1Wampsux3fdtttbN++nfT0dPsrMjKShx9+uMWZRF3JqJgAALbmqIVFRESkI7k4ekJycjK33347Y8eOZfz48bz00ktUVFQwZ84cAGbPnk3v3r3tLSgpKSmMHTuW/v37U1NTwxdffMHChQt57bXXAAgODiY4OLjZz3B1dSU8PJzBgwdf6P11qFExgQDsOVZGVW0Dnm4WJ9dIRESkZ3I4sMyaNYvCwkKefPJJ8vLySEhIYNmyZfaBuNnZ2ZjNpxpuKioquOeeezhy5Aienp7ExcWxaNEiZs2a1X534SSR/h6E+rpTUFbDzqMl9j2GREREpH2ZjB4wJ7e0tBR/f39KSko6fTzLne99x1e783nse3HcObl/p/5sERGR7syRz2/tJXSBmrqFNPBWRESk4yiwXKCmgbfpOcVOrYeIiEhPpsBygeKj/LGYTRwrqeZocZWzqyMiItIjKbBcIC83F4Y3LiC37sBxJ9dGRESkZ1JgaQeX9bdNy/52f5GTayIiItIzKbC0g4kDQgBbYOkBk65ERES6HAWWdjC6TyDuLmYKymrYX1Du7OqIiIj0OAos7cDD1WJfNE7dQiIiIu1PgaWdXDrANo5l7X4NvBUREWlvCiztpGkcy4aDx6lvsDq5NiIiIj2LAks7GRbpj5+HC2U19WzPLXF2dURERHoUBZZ2YjGbmDjQ1sry1a58J9dGRESkZ1FgaUffj48E4OP0XKxWTW8WERFpLwos7ejKIaH4ebhwrKSa9Qc1+FZERKS9KLC0I3cXCzMbW1mWbM11cm1ERER6DgWWdnbD6N4A/HfHMapqG5xcGxERkZ5BgaWdje0TSFSgJxW1DXy1O8/Z1REREekRFFjamclk4oZRtlaWv685aF+TpcFqUF2nFhcREZHzocDSAWZfGou/pyu7jpaycP1hispr+P4ra0n8f6mcqKh1dvVERES6HQWWDhDi485vZwwG4Pmv9nLr6xvYfayUkqo6Nh064eTaiYiIdD8KLB3klnExJEQHUF5TT2Z+mf14xrFSJ9ZKRESke1Jg6SBms4lnrh+Oh6uZEB93fpwYAyiwiIiInA8XZ1egJxsW6c+q31yBn6cL6dnF/GNDNhnHys59ooiIiDSjFpYOFu7vgZebC0Mi/ADIPlFJWXWdk2slIiLSvSiwdJJAbzci/D0A2JOnVhYRERFHKLB0oqZWFo1jERERcYwCSycaEuELKLCIiIg4SoGlEw2N8AdgtwbeioiIOESBpRM1tbBk5pXSYDWcXBsREZHuQ4GlE/UJ9sbT1UJ1nZWsogpnV0dERKTbUGDpRBazicHhtlaWDVnHnVwbERGR7kOBpZNdNiAYgKc+3sWHG7OdXBsREZHuQYGlkz0wdSA/SIik3mrw6JIdvLvukLOrJCIi0uUpsHQydxcLL85K4O4p/QF469ssJ9dIRESk61NgcQKTycS9VwzAxWzi8PFKDh/XAFwREZGzUWBxEh93F0b3CQRgzd5CJ9dGRESka1NgcaLLB/UCYM2+IifXREREpGtTYHGiyQNtgSXtwHHqGqxOro2IiEjXpcDiRMMi/Qj2dqO8pp4th086uzoiIiJdlgKLE5nNJiYODAFgzT6NYxEREWmNAouTTWrsFlqtgbciIiKtOq/AMn/+fGJjY/Hw8CAxMZGNGze2WnbJkiWMHTuWgIAAvL29SUhIYOHChfb36+rqeOSRRxgxYgTe3t5ERkYye/Zsjh49ej5V63YmDwrBYjaxM7eUj9NznV0dERGRLsnhwLJ48WKSk5N56qmn2LJlCyNHjmT69OkUFBS0WD4oKIjHH3+ctLQ0tm/fzpw5c5gzZw5ffvklAJWVlWzZsoUnnniCLVu2sGTJEjIzM7n22msv7M66iVBfD+67YgAAv1u6k5wTlU6ukYiISNdjMgzDcOSExMRExo0bx6uvvgqA1WolOjqa+++/n0cffbRN1xg9ejQzZ87k6aefbvH9TZs2MX78eA4fPkxMTMw5r1daWoq/vz8lJSX4+fm1/Wa6iPoGKzf9LY2t2cWMiw3kwzsnYDGbnF0tERGRDuXI57dDLSy1tbVs3ryZpKSkUxcwm0lKSiItLe2c5xuGQWpqKpmZmUyePLnVciUlJZhMJgICAlp8v6amhtLS0mav7szFYuYvs0bh7WZh06GTLN+dD8Dh4xVMevZr/rJin5NrKCIi4lwOBZaioiIaGhoICwtrdjwsLIy8vLxWzyspKcHHxwc3NzdmzpzJK6+8wlVXXdVi2erqah555BFuueWWVtNWSkoK/v7+9ld0dLQjt9ElxQR78ZNL+gCwZMsRAN5cm0XOiSqWbD3izKqJiIg4XafMEvL19SU9PZ1NmzbxzDPPkJyczKpVq84oV1dXx49+9CMMw+C1115r9Xpz586lpKTE/srJyenA2neeG0ZHAbAys4Dc4iqWbrENws05UUltvRaWExGRi5eLI4VDQkKwWCzk5+c3O56fn094eHir55nNZgYMsA0sTUhIICMjg5SUFKZMmWIv0xRWDh8+zNdff33Wvix3d3fc3d0dqXq3MDjcl2GRfuw6Wsp9/9hCWU09AFYDck5W0r+Xj5NrKCIi4hwOtbC4ubkxZswYUlNT7cesViupqalMmDChzdexWq3U1NTYv28KK/v27WPFihUEBwc7Uq0epamVZWt2cbPjWYXa0VlERC5eDncJJScn8/rrr/Puu++SkZHB3XffTUVFBXPmzAFg9uzZzJ07114+JSWF5cuXc/DgQTIyMnj++edZuHAhP/nJTwBbWLnxxhv57rvveP/992loaCAvL4+8vDxqa2vb6Ta7j2tHRtpnCLmYTVzSLwiArCIFFhERuXg51CUEMGvWLAoLC3nyySfJy8sjISGBZcuW2QfiZmdnYzafykEVFRXcc889HDlyBE9PT+Li4li0aBGzZs0CIDc3l08++QSwdRedbuXKlc26jS4GvXzdmTKoF6l7Cpg+LJx+vbxZf/AEWccVWERE5OLl8DosXVF3X4flfx0oLGfBqgM8MHUgmw6dIPmf25jQL5gP7rzE2VUTERFpN458fjvcwiIdr38vH/5800gACsttY30OqYVFREQuYtr8sIvrF+INwLGSaqpqG5xcGxEREedQYOniArzcCPByBdTKIiIiFy8Flm6gb2Mri2YKiYjIxUqBpRtQYBERkYudAks30DdYgUVERC5uCizdQN9e5w4slbX1vL/hMB9tze2saomIiHQaTWvuBpq6hPYXlHOgsLzZnkKHiipYsjWXhWmHOFlZB8ComAD6NLbKiIiI9AQKLN1A3xBvXC0mSqrqmPr8aqKDPAnwdKO23kpmftkZ5bdkn1RgERGRHkVdQt2Al5sLC34yhimDe+FqMZFzoooduSVk5pdhMZuYNDCEl28ZxU8vjQUg/X82ThQREenu1MLSTUwdEsbUIWGUVNaRkVdKZW09dQ0GY/sEEuzjbi/3zrpDbM0pdl5FRUREOoACSzfj7+XKJf2CW3xvVHQAABnHSqmua8DD1dKJNRMREek46hLqQaICPQnxcaOuwWDX0VJnV0dERKTdKLD0ICaTiYTGVpb0nGIMw2DdgSLSDhwn41gptfVW51ZQRETkPKlLqIdJiA5gRUYB6TnFvLhiHy+n7rO/NyomgCV3X4rJZHJiDUVERBynFpYeJiE6EIBVmQW8+rUtrMQGe2Exm9iaXcz2IyXOrJ6IiMh5UWDpYeKj/TGZoKy6HqsB14/qzaqHr2DmiAgA/rPliJNrKCIi4jgFlh7Gz8PVvhJu7wBPfn/dMAB+OCYKgE+2HaWmvsFp9RMRETkfCiw90M3jogn38+AvNyfg5+EKwMQBIYT5uVNcWcfKPQVOrqGIiIhjFFh6oF9M6sf6x6YyNjbIfsxiNvGDUb0B+PdmbZAoIiLdiwLLReTG0bZuoVWZBRSUVju5NiIiIm2nwHIRGRjmy9g+gdRbDV7/5qCzqyMiItJmCiwXmfuuHADAovXZFJXXOLk2IiIibaPAcpG5fFAvRkb5U1XXoFYWERHpNhRYLjImk4kHpg4EYGHaYftYlqyiCm57cwO//fc2rFbDmVUUERE5g5bmvwhdGRfK8N5+7Mwt5aoX13DTmCg+3JRDeU09AHHhfvxsYl8n11JEROQUtbBchEwmE8/flMDgMF9Kqup4Y20W5TX1RAd5AvCnZXs4UFju5FqKiIicosBykRoc7ssXv5pEyg0jiAv35f4rB/D1Q1OYNDCEmnorD/1zGw3qGhIRkS7CZBhGt/9UKi0txd/fn5KSEvz8/JxdnW7taHEV019cQ1lNPW//dBxXxIU6u0oiItJDOfL5rRYWaSYywJPrRkUCsCIj38m1ERERsVFgkTNMjQsD4Os9BfSABjgREekBFFjkDBP6B+PpauFYSTW7j5UCkJlXRkXjLCIREZHOpsAiZ/BwtXDZgBAAvs4o4OP0XKa/tIZZf0+juq7BybUTEZGLkQKLtChpiG2w7dL0XH730U4AduaW8vtPd59RVgvNiYhIR1NgkRZd2Tg76GBhBWXV9cQGe2EywQcbs1my5Yi93PLd+fR//AuWbj3S2qVEREQumAKLtCjUz4P4KH8APFzNvPXTcTxwpW1J/6c+3kVNva1r6MtdeRgGvP3tIWdVVURELgIKLNKqWxNjcDGb+MO1w+nXy4cHpg7E18OFspp6sooqAOwr4m4/UsLR4qpWr3W0uEpdRyIict4UWKRVs8bFkPH0DH40LhoAi9nEwFAfAPbml2MYBvsLTi3hv3y3bd2Wd77N4vmvMqmttwLw11X7uXTe17zy9f5OvgMREekptPmhnJWrpXmmHRTmy5bsYvbll1FYHkRZ9ampzl/uyqNfL2/+r3Fg7sHCCm4Y3Zs/f5kJwHeHT7T6c46X1/DY0h385JI+TBrYqwPuREREujMFFnHIwDBfAPbml3GgwNYt5Otu6ybakHWiWbfQ5zuO8cXOYzStPZd7li6jDzfl8OWufPJKaxRYRETkDOoSEocMCrN1Ce0rKGd/4/iVcX2DiAv3pcFqcOh4JcHebvz5xngsZhOGAbHBXgDknqxqdeXcLYdPArArt4TKWi1QJyIizZ1XYJk/fz6xsbF4eHiQmJjIxo0bWy27ZMkSxo4dS0BAAN7e3iQkJLBw4cJmZQzD4MknnyQiIgJPT0+SkpLYt2/f+VRNOtjAUFsLy+HjlWQ0roI7INSHacPC7WUeuTqOm8ZG8/fbxnDzuGgW/SIRkwlq6q0UldeecU3DMNiaUwxAvdUgvfFrERGRJg4HlsWLF5OcnMxTTz3Fli1bGDlyJNOnT6egoKDF8kFBQTz++OOkpaWxfft25syZw5w5c/jyyy/tZZ599llefvllFixYwIYNG/D29mb69OlUV1ef/51Jhwjzc8fXw4UGq8GKxkG2/Xt5c11CJO4uZi7pF8SNo6MAmDokjHk/jCcq0IswXw8AjpysPOOah45XcqLiVJD57tDJTrgTERHpThwOLC+88AJ33HEHc+bMYejQoSxYsAAvLy/eeuutFstPmTKF66+/niFDhtC/f39+9atfER8fz9q1awHb/65feuklfve733HdddcRHx/Pe++9x9GjR/noo48u6Oak/ZlMJgY1jmMpKKsBbC0s/Xv5sOGxqbz3s0TMZtMZ50UFegItj2Np6g5qsulQ64NzRUTk4uRQYKmtrWXz5s0kJSWduoDZTFJSEmlpaec83zAMUlNTyczMZPLkyQBkZWWRl5fX7Jr+/v4kJia2es2amhpKS0ubvaTzNI1jadIvxPZ9gJcbbi4t/5Xq3RhYjpxsIbBk2wLLpIG2/Yu2HD5JfYO13eorIiLdn0OBpaioiIaGBsLCwpodDwsLIy8vr9XzSkpK8PHxwc3NjZkzZ/LKK69w1VVXAdjPc+SaKSkp+Pv721/R0dGO3IZcoAGN41gAgr3dCPR2O+c59haWxsDyxjcHeeif26iqbWBLdjEAN4+LwdfDhYraBvbklbV/xUVEpNvqlGnNvr6+pKenU15eTmpqKsnJyfTr148pU6ac1/Xmzp1LcnKy/fvS0lKFlk50egtL/1Cfs5Q8pXeAbabQkZOVVNc18Kdle6hrMDAMg8w8WwvZ2NhAxvQJZFVmIZsOnWB4b//2r7yIiHRLDrWwhISEYLFYyM/Pb3Y8Pz+f8PDwVs6ydRsNGDCAhIQEHnroIW688UZSUlIA7Oc5ck13d3f8/PyavaTzNI1hAejfq22B5fQxLDtyS6hrsE1vXrI1F6sBvQM8CfPzYFxsEKCBtyIi0pxDgcXNzY0xY8aQmppqP2a1WklNTWXChAltvo7VaqWmxjZgs2/fvoSHhze7ZmlpKRs2bHDomtJ5Qn3d8fOwNc717+XdpnNOH8PSFEZOH+8yKiYAwB5YNmSd0N5DIiJi5/AsoeTkZF5//XXeffddMjIyuPvuu6moqGDOnDkAzJ49m7lz59rLp6SksHz5cg4ePEhGRgbPP/88Cxcu5Cc/+Qlgm3Xy61//mj/+8Y988skn7Nixg9mzZxMZGckPfvCD9rlLaVcmk4nRfQKBU0HjXHoH2AJLZW0DK/fYpsD/aupAhkbYWscm9A8GYGS0P77uLhSV17A5W60sIiJi4/AYllmzZlFYWMiTTz5JXl4eCQkJLFu2zD5oNjs7G7P5VA6qqKjgnnvu4ciRI3h6ehIXF8eiRYuYNWuWvcxvf/tbKioquPPOOykuLmbixIksW7YMDw+PdrhF6Qgv/iiBg0XljOkT1KbyHq4WQnzcKSqvYWPjtOVL+gVzy/gY1uwtZGZ8BADuLhauGhbGki25fLbtqL3FRURELm4mo7W10ruR0tJS/P39KSkp0XiWLuwH87+1r2LrZjGz4/fTcHexnFFu5Z4C5ryziRAfdzY8NhVLC+u6iIhI9+fI57f2EpJO0zSOBWBElH+LYQXgsgEhBHi5UlRew4aDxzureiIi0oUpsEiniTotsIxpHAPTEjcXMzMa9yb6dPsxDhSWs2TLEWrqGzq8jiIi0jUpsEiniQo4FVhGx7QeWACuiY8E4F/f5TD1+dUk/3Mbz3ye0Wr5ovIaXlqxl7wS7T8lItITKbBIp4kK9LJ/PbpPwFnLXtIviF6+7tRbDUyNQ1j+sSGb7ONnbp4I8NeVB3hpxT5+/u6mNrXEVNc10KBp0yIi3YYCi3SaoZF+eLiaGRnlT6jv2WeAuVjM/P22MTxxzVDWPnIlkwaGUG81eGF5ZovlN2TZxrrsOlrK81/tBWyhJOdEJfvyyzh62qaLx0qqmJCSyh3vfddOdyYiIh2tU5bmFwEI8/Ng+YOX4+Petr92o2ICGdXYdfTb6XF8s28tH287SqC3G2v3FdGvlzcLfjKG8pp6Mo6d2gDz72sOknGslE2HTlBdd2oTxX/8IpFLB4SwZEsuJyvr+HpPAYVlNfTydW/fGxURkXanFhbpVNFBXm3aLPF/jYjyZ2Z8BIYBb397iH0F5Xy5K59tR0rYml2M1YDoIE9+ckkMAN/sK6K6zoqbixlPV9tspDfXZgHw6baj9ut+u7+oHe5KREQ6mlpYpNt4dEYcOScq6eXjzonKWrZmF7NsZx5uFtsgl3F9gvjdzKH4uLvi5WZh6pBQhkb4cbCogqnPr+brzAJWZhY02wl6zb5CfjCqt7NuSURE2kiBRbqN6CAvPrlvIgCfbT/Kff/YyrKdxwj3t42HGRsbhIerhUevjmt2Xv9ePlzaP5h1B46TvDgdsO2HVFBWw9p9RRiGgcl0anG6mvoG3CzmZsdERMS51CUk3dKUwaG4uZg5dLySjVm2pf7HxbY+VfrWxD4AnKysA+Dh6YPxcDVTUFbD3vxye7nNh08y5IllvLhiXwfWXkREHKXAIt2Sj7sLkwf2AsBqQICXK/17+bRaftqwMPvgWk9XCzPjIxjf17bh4jf7Cu3lPtt+FKsBb35zkIqa+g68AxERcYQCi3RbM4aH278e2ycQ81n2HHK1mLllfIz9PC83FyYPDAFsA3SbbD5s2yG6orah2eBcERFxLo1hkW4raUgoLmYT9VaDsW3Y1fn+KwfQN8SLKwfbdhaf2BhYNmQdp7quAathsOvoqenRH2zM5ubGkCMiIs6lFhbptgK83Lh2ZCTuLmauGhp2zvKuFjPXj4rC38sVgMFhvoT5uVNdZ+Xb/UVsyymhwWoQ5O2Gq8XEtiMl7DpaQn2DlX35Zfx3xzE+3JhNdd2plXSziirIOdHy6rsiItJ+1MIi3dqzN8bz9A+G493GxehOZzKZ+N6ICN7+9hCfbjvKwDBfACb0t41t+Xz7Me7/x1aKymsorT41nuVYSTUPXjWIovIarnn5G1wsZlY/PIUAL8fXlxERkbZRC4t0ay4W83mFlSbfH2nbZHH57nzWNo5lGRMTyI8bu4IOFlVQWl2Pt5uFfiHeACzdmothGHy27SgVtQ2UVNWxaP3hC7wTERE5G7WwyEVtVHQAvQM8yS2uIu2gbT+isbGBjOjtz9yr4yitruOKwaGMigmkpr6BsX9cQfaJStJzivko/dSg3HfWHeIXk/rh0biqboPV4N+bc4iPCmBIhJ9T7k1EpCdRC4tc1Ewmk72VBWxTnodE+GEymfjl5f15eHocY2ODsJhNeLm5MK1xrMzLqftIzynGYjYR5udOUXkt/958xH6dN9ce5JH/7OAX735HfYP1jJ/7vz7amstrqw5gGNpBWkSkJQosctH7/sgI+9cjo/1xtbT+z+K6BNsy/iszbWu3TBwQwl2X9wfg9W8OUt9gJedEJS8uty08l1tcxYqMgrP+/PKaeh7+9zb+tGwPu0/bxFFERE5RYJGL3tAIP/r1so1PGdOn9dVywTYVOrBxlhHA9aN6M2tcNAFerhw+XsmNC9L4zb+2UVVnW94f4N11h856zfUHjlPXYGtZ2ZZTcgF3IiLScymwyEXPZDLxyIw4xvQJ5OZxZ193xdViZma8rUXGy83CtGFheLm58OwP4/F1dyE9p5gNWSdws5h5e844LGYTaQePsyev9ZaT01fa3ZFb3Gq5ugZrsynVIiIXEwUWEWD6sHD+c/elRAd5nbPsbZfE4uvhwu2XxuLlZhu3Pm1YOF8lT+aKwbbtAn4zfRCXDQhh+jDbmJd317U+i+j0lXbP1sJy53vfcUlKKvml1W26JxGRnkSBRcRBg8N92fF/03lkRvNdoSP8PXl7zni2PTWNOyfbxrXcPiEWgKVbj1DeuDfRugNFjPi/L1m0/jA5Jyo5WFRB08bQmfllLbai5BZXsTKzkOLKOpbvzu+4mxMR6aIUWETamb/nqTEu4/sGERvsRXWdlW/22rp+3lt3mLLqep7+bLd9/ZYxMYGE+LjRYDVaHHi74rSQcnoXkojIxUKBRaQDmUwmpg6xdQutyCiguq6BNY2Bo6beyt/WHARg8qBexEcFALA9p/iM66zIOBVY1u0/Tl0bpkqLiPQkCiwiHWzqkFAAVmYWsHZfEZW1DQR7u+Hucuqf36SBIcRH+QOw/UjzcSyl1XWsb1zUzt3FTFlNPdtyitlfUM70F9eccxZSa+oarKzeW0hNvQbyikjXp8Ai0sHGxQbh6+HCiYpanl++F4DvjYjggakDAVsXUnxUwKnAkmsLLDknKqlvsLJmbyF1DQb9ennbN3lcs7eQpz7ZSWZ+GfNX7sdqbb7g3NHiKp7+bPdZB+i+l3aY29/ayL3vb9GCdSLS5WlpfpEO5moxM2VwKJ9uO0pG4/iUacPCuKRfMFarwfDe/ljMJkb0DgDgQGE5dy3czLJdecSF++LXOCbmqiFh9A/14bPtx3g37TAlVXUAFJTVsPNoib1LCeBPy/bwcfpRCstqePmWUdQ3WLn57+uptxr8664JuFrMrG4cU7Mio4B/bz7CTWOjO+8PRUTEQWphEekESY3dQgC+7i4k9g3G1WLm/qkDuSLO9l4vX3ci/T0wDFi2Kw+APXllbMw6AcBVQ8OYNDAEwB5WmhanO3013eq6Bvsg3S935VFSZZtZ9N3hk6TnFLPl8EkarAZbDp+0n/OHT3eTW1zVUbcvInLBFFhEOsGUQaFYzLa5y1PiQnFzafmf3ri+QQAMDPVh4c/Hc23jPkdRgZ6Migkkwt+TQWE+AAR5u/Ho1bap1amnDcpdvbeQilrbuJSaeiufbjvKO6eNc/lmXxEZx0opr6nH192FUTEBlNXU83+f7GrfmxYRaUcKLCKdwN/LlYkDbK0j34+PaLXck9cM5a+3jubT+ycyaWAvXr5lFP/91ST++csJ9sBz/agoAB773hCuTYjEZIJdR0vJK7GNV/l8+zEAQnzcAfjryv1saGylAVizr5BNh2zfj+4TyNPXDQdg7b6iM8bCiIh0FQosIp3khR+N5P1fJDJtWHirZYJ93PneiAg8XC32Y0Mi/IgM8LR/f9fl/Uh/8ipuHBNFiI87o6IDAEjdk2/rDmpsbfnTD0fgYjZxtDHIjI+1td7syC3hq122MuP7BjEkwg93FzNVdQ1kn6hs13sWEWkvCiwinSTYx53LGltZLoTJZCLAy83+fdM6L59tO8bH6blU1jbQO8CTK+NC7VOqAR6aNoi4cF8MA9Iap0mPiw3CYjYxKMwX4Kx7HjnirbVZ/OLd7+yr+4qIXCgFFpFuLqkxsKQdPM4j/9kBwMz4CEwmE7cm9gFgRG9/xvcNYvKgXvbz3Cxm+1TqweG2wJJxrAywjYmZ8/ZGsooqHK7PjiMlPP35blZk5PPfHcfO/8ZERE6jwCLSzQ0O9yX5qkEMDLUNxnW1mLhhdG/AtoLuv++awFs/HYfJZGLywFOBZWS0v73rKa4xsGTm2QLLiyv2sjKzkLsXbXZoh2ir1eB3H++kaVmX0zd2PB85JypZmHZIK/uKiNZhEekJHpg6kAemDuRkRS11DVZC/Tzs741tHLti+zoQD1cz1XXWZsfjwv0AW5fQiYpadh0tbfy+jKc/280z149oUz0+3JTDtpxiLGYTDVaDtfttA3nNjQOGHVFZW89P3tzA4eOV+Hi42Acbn4/K2nqKK+uI8PfAZHK8LiLifGphEelBAr3dmoWV/+XhauHq4RGYTdhXzQWIi7C1sBw+UUlqRj6GAYFetgXr3t+QzbKdZ+/aSTtwnOTF6fz+U9vU6EdnxOHjblvdd+fRkrOe25o/f5nJ4eO2QcCZeeXndY1/bMjmiudWMeypL7l03tf8Z0vueV1HRJxPgUXkIpNywwjW/PYKRscE2o+F+LgT4uOGYcDb3x4C4IbRUfxycj8A3lybZS+bmpHPy6n7qGpc6+WV1H3c8vp6lmzNpabeysQBIcy5LJZL+wcDtm0EHLUx60SztWMOncdYmgarwbz/ZpBVVGHvovo4XYFFpLtSYBG5yHi4WogK9DrjeFO30O7G7QMmDgixD9rdkl1MaXUd1XUN/OrDdF5YvpcbXlvHC19l2vdHumlMFEvuuZSFPx+Pi8VsH+C75hzjWAzDYG9+GfWN41QMw+DxpTswDOjXyxuAQ8cdDyzbjxRTWl2Pn4cLH955CWALQo6MyRGRrkOBRUSAUzOFAFzMJsb3DSIm2Iu+Id40WA3W7T/O2n1F9qnKGcdKefnr/QD8Omkgf75pJKNjAu1jRJoG+G45fJLS6jqOnKyktv7MwbN/W3OQaS+u4W9rDgK2vZH2FZRjNsELP0oAbIHF0Q0amwb8Xto/hMS+QUT4e1BTb7VvdSAi3ct5BZb58+cTGxuLh4cHiYmJbNy4sdWyr7/+OpMmTSIwMJDAwECSkpLOKF9eXs59991HVFQUnp6eDB06lAULFpxP1UTkPMWdFlhGxwTi7W4bk3+5vaWkkP/utO1xdE18BAmNC9b9fGJfftW48/TpYoK9iA32ot5qkPhMKhP/tJIrnlvFysxT+x6VVNYxf6Ut9KzOtHUd7Woc8zIg1IdhkX5YzCaq66zkl9Y0u35BWTX78svs35+oqOWpj3faV/Fd2xhYJg4MwWQy2fdhOp8uKhFxPocDy+LFi0lOTuapp55iy5YtjBw5kunTp1NQUNBi+VWrVnHLLbewcuVK0tLSiI6OZtq0aeTmnupLTk5OZtmyZSxatIiMjAx+/etfc9999/HJJ5+c/52JiEOauoQALh0QbP968iDbB/3qzEL7Kro/uaQP/75rAqt+M4Unrhna6sybpkXtqhq7YXKLq5jz9iYeXJxOVW0Db649SFm1rcVmR24J9Q1WdubauqSGR/rjajETFWhb5fd/14T5yRsbmPbSGpbtPIbVavCrD7fybtphHvhgK8fLa9iSbdvcsamlZ/JpwetClVXXceVzq7jn/c0OtfycqKjlw43ZWlBP5Dw4HFheeOEF7rjjDubMmWNvCfHy8uKtt95qsfz777/PPffcQ0JCAnFxcbzxxhtYrVZSU1PtZdatW8ftt9/OlClTiI2N5c4772TkyJFnbbkRkfY1MMyHptnHE09bkfeSfsG4WczkFldRUlVHiI8b42KDcLGYiQ3xPus1H7xqEC/8aCRL7rmUbU9O4+cT+2I2wdKtudz8+nreahzgC7ZQs7+wnJ25thaWYb1ti9rFBtt+xuHTxrEUlFazN78cw4AHF29j7pId9i6gYyXVPPDhVuqtBjFBXsQE28brXNY/BJMJ9uaX2/ddAtuYmcpaxwLEpkMnOFhUwRc78vh6T8v/WWvJq1/v59ElO1iYdtihnyciDgaW2tpaNm/eTFJS0qkLmM0kJSWRlpbWpmtUVlZSV1dHUNCpNSAuvfRSPvnkE3JzczEMg5UrV7J3716mTZvW4jVqamooLS1t9hKRC+PhauHxmUP5+cS+zWYQebm5MK7vqe+vGhpu34jxXHzcXbhhdBSjYwLx93LliWuG8sEdlxDg5cq2nGLKa+oZGuFHYuMu1dtyiu1rwAyPtLX49G0MRVmnBZb0nGL711V1DSz+LgeAK+NsWxF8u9+29cDEgaeCV6C3G/FRAcCpVpasogqum/8tY55ewbr9bV/krmlFYIBnl2XS0MZNI5u2Pji9K0tE2sahwFJUVERDQwNhYWHNjoeFhZGXl9emazzyyCNERkY2Cz2vvPIKQ4cOJSoqCjc3N2bMmMH8+fOZPHlyi9dISUnB39/f/oqOjnbkNkSkFT+f2Jcnrhl6xkJvp6+Qe/Xw1jdvbIvEfsEsuftSYoO9MJvgtzMGkxATAMCqzEJyi6sAGNoYWPo0tpCcPrW5KbBcPTyc/o0zia4dGcnfbxvDgMYVf231br530+WN3z//VSbJi9O55uVv2H6khKq6Bn69OJ3j5c3HybSmaSYVQGZ+WZunSzd1ax3WJpMiDuvUWULz5s3jww8/ZOnSpXh4nFrc6pVXXmH9+vV88sknbN68meeff557772XFStWtHiduXPnUlJSYn/l5OR01i2IXJSmDgnFZIIQHzcm9A8+9wnn0K+XD18+OJnVD1/BlMGhJDS2fCzfbRsjExvsha+HbeG6pm6npkXkALYdKQZg0sBe/OuuS/nLzQk8e2M8LhYzv5k2GACzCSb0ax5YvhcfgZuLmfzSGpZszaWitoHxfYMYEOpDQVkNv/339jaNScloDCxNa828sHxvizOgTldZW8+xxq6o0+/FGQ4fr+D/PtnV5oAm0hU4tDR/SEgIFouF/Pz8Zsfz8/MJDz/7/7qee+455s2bx4oVK4iPj7cfr6qq4rHHHmPp0qXMnDkTgPj4eNLT03nuueeatcQ0cXd3x93d3ZGqi8gFGBDqy/u/SCTExx1XS/v8P8fdxUJ0kK31ZGTjjKP6xq6VpvErAH2DT63FYm18f3uObZxLQnQAQd5uXJfQ215++rAw5l4dR4iPO/6Nq/U2iQv3Y/3cqXx36ARbc4rpHeDJzeOi2Ztfzg/++i2pewr485eZPDx9cKsDiatqG+ytPfNuiOf6v37LkZNVrDtQxJTBoS2eA3Co6FRIKSqvoaKm3j4Tq7P94dPdpO4pwNPNwiMz4pxSBxFHOfSbx83NjTFjxjQbMNs0gHbChAmtnvfss8/y9NNPs2zZMsaOHdvsvbq6Ourq6jCbm1fFYrFgtWrDM5Gu4tL+IQwK8z13wfMQ4e9BiM+p/4QMjzwVWKICPXFpmtpcVs3BonLKaurxdLUwKMznjGuZTCZ+eXl/fjim5b2HgrzdmDYsnEdmxPGTS/rgYjEzNNKPJ64ZCsBfVx0g+Z/bqKlveYG5zPwyrIattSkm2Itpw2z/WUvNOPvg2/+d5ZTd2C10vLyGExW1Zz23PZVU1dnH8GxvbKkS6Q4c/q9ScnIyr7/+Ou+++y4ZGRncfffdVFRUMGfOHABmz57N3Llz7eX/9Kc/8cQTT/DWW28RGxtLXl4eeXl5lJfb9gbx8/Pj8ssv5+GHH2bVqlVkZWXxzjvv8N5773H99de3022KSFdmMplIiD4VUob3PjXF2uW0qc2HiipJb2xdGdHbH5d2au0BuO2SPqTcMAKL2cTSrbn86G/r2V9w5h5GTd1BQyJsdbxqqK1VxbYHU+vdSVlFza+VfaKS6roGvvfyN1zz8jedtgLvV7vyqGtobKk6UuLwgnwizuLwv/ZZs2bx3HPP8eSTT5KQkEB6ejrLli2zD8TNzs7m2LFTG6W99tpr1NbWcuONNxIREWF/Pffcc/YyH374IePGjePWW29l6NChzJs3j2eeeYa77rqrHW5RRLqDkY3jWACGndbCAqfGsRw6XkF6jm19lZHRzcu0h1vGx/D2T8fh6+HCtpxivvfyN/xt9YFms4D+N7Bc2j8ED1czR0uq2X2slIxjpcx4aQ3//K752LqD/9vCcrySHbkl5JfWcLSkmi2HT7b7/bTk8x2nfj+XVdc7fTyNSFudVwfqfffdx3333dfie6tWrWr2/aFDh855vfDwcN5+++3zqYqI9BCjGqdSRwV6EuTt1uw921oshXy7v8je6tE07qW9TR7Uiy9/PZlHl+xgzd5CUv67h//uzOO5m0YyINTntMBi6x7zcLUwaWAvlu/O58udeXydWcCevDL+tvoAPxp7agZj07iXfiHeHCyq4PCJCgxOBaE1+4q4dEDzQcLtrbiy1r4CcKivOwVlNWzPLTnnejoiXYH2EhKRLuGyAcH8buYQnr9p5BnvDW1szfhs+zH25NnWMEnooMACEBngybtzxvGnH47A192F9MbWluW789nTuAZLUwsLwFWNK/ouWH3QvlLvgcIK+xRtODWGpWnF3cPHK9maXWx/f+3+M1fgraptsA80bg9f7cqn3moQF+7L9MaxN00L9Yl0dQosItIlmEwmfjGpH4n9zpw2/cMxUTz9g+H2sSwxQV70DvDs8PrMGhfDlw9OZtLAEGrrrdy1aDNlNfW4Wcz073VqwO8VcbZp37WNO057uNp+ta5tHNx6sqKWk5V1wKm9mbJPVDZbAG9nbmmzacYrducz+unl3LVoc7vd06fbjwK2vaBGRNm61DTwVroLBRYR6fIsZhO3XdKHVb+Zwns/G897Pxvf6rTj9hYZ4MnbPx3H1cPD7WNZBoT6NJve3cvX3d7iMzomgDsn9QNgzV5b90vTKr0R/h72XbGzT1RyrKQas+nUar7fHrCt0Lts5zHuWrSZqroGvtqd79AO01/sOMb1f/32jAHD+/LL+GZfESYTXBMfSXxjYNmZW4rVamjwrXR5Ciwi0m24WMxMHtSr08dcuFjM/OXmUSQNsc0IGhsbeEaZ30wbzNS4UJ7/UQKXD7a1oqzdX0SD1SCr0BZY+oZ4E+7ngZuLmaZ8EBfuZ7/uN3sLWbr1CPf+w7YXUtNYnheX7wUgv7Sar3bltRouSqvreGzpDrZmF/P8V5nN3vv7moMATBsaRmyINwN6+eDhaqa8pp71WceZ9uIarnhuFZ9vP4ZhGJRW15GZV9auXVIiF8I5qxaJiHQzbi5mXvvJGL7ZV8jY2KAz3r9sQAiXNQ6ajQ70xNfDhZKqOnbkltjHr/QN8cZsNhEd6MmBxhAzKiaASQN78fo3WXyy7Sj/2nwEgBtG9ebBqwZx5fOrSDt4nFdS9/Hmt1kUV9bx11tH870REWfU4W+rD1Dc2PW0bFceh4oqiA3x5lhJFR81bh/wy8v7A7YQNizSn82HT/KLd7+jstY2rfref2yhl687hWW27qlHZsRx95T+7fbnKHK+1MIiItJGrhYzV8aF4efhetZyLhYzl/W3hZdv9hY2CywAfYJPtRAlRAcwvm8Qbi5mahqX979zcj+eu2kk0UFezBpnm2n0/PK99jDS0iJ1+aXVvLk2C7B1PRkG9u/f/vYQdQ0G4/sGNdvYckTjisKVtQ2E+Lhz1+X98XS12MOK7dws6hravojnG98c5O5Fm3njm4Pa5FHalQKLiEgHmDTIFljmr9rPFztta5/0a9yoMaZxSwKwTef2cLUwbWgYFrOJ3187jMe+N8S+AeW9VwzAzcX2q3p8Y8vOt/uLMAyD+gYrf/h0N3e89x1z3t5EdZ2VMX0C7TOt/rU5h/kr97Mw7TAAd1/evKVkVOOmk95uFt6ZM45Hr45jzW+v4B93JLLhsamE+NimPjft8dQkq6iixYXyDh+v4JkvMvjvzjz++HkGV724hpWZZ18BWKSt1CUkItIBLh/Uy76lANimZo9rDBxNgcXXw4V+ja0uf7l5FH+oqjtjDZoIf0/+fdcESqvqGRsbSPzvvyKvtJoDheXsySvjrW+zmpV/ZEYc42IDGd7bj525pfz5S9tYlkv6BTFlcK9mZb83IoIjJ6uYNDCE4Y2tLb183enla9sm4eZx0by6cj+L1h/meyMiqKip5+Wv9/HmN1nUW40zuqYWph3GMCAu3BfDsG1jsGZvIVecZY8lkbZSYBER6QBRgV58cOclHC+vYXRMIKF+p3aoH983CLPJtn5LU0uKxWw6I6w0iT9tFeDxsUGs3V/EN/uKWJFha/mYGR/B+Ngg+gR7Mb6vLRT9euog7lj4Hb0DPHlg6kBuGNX7jJlVrhYz914xoNV7uCUxhr+u2s+6A8d5fc1B3lybRV5ptf39ZTvz7IGlsrbevrrvb2cMpqSqjgcXb2P7Ea3zIu1DgUVEpIOMa2FwLsDw3v5888iVBLcSUM7msgEhrN1fxD+/O0LGsVJMJnh0Rpx95+smSUPD2PDYVAK93M57h+3eAZ5cGRfKiowCnvkiA4DoIE9+ODqKl1bsY/XeQhqsBhaziY/Tj1JaXU9MkBdTBoXap3LvOlpCfYO1Xfd9kouT/gaJiDhB7wBPPFwtDp83aaBtbEzTFgETB4ScEVaahPp6nHdYaTLnsr6AbZbUr5MGsvzBy7nvigH4e7pSUlVHes5JDMPg3XWHAJg9oQ9ms4m+wd74urtQXWdlb/6Zm0iKOEotLCIi3cjQCD8CvVztK+feMj6mQ3/eZQNC+OjeywjzcyfC/9TqwpMH9eLTbUdZuaeQExV17Mkrw9PVwk1jbLOazGYTI6L8WXfgONuPFDM00q+1HyHSJmphERHpRsxmk32TxGBvN5Ia9zHqSAnRAc3CCsAVjQN4v9qdxx8+2wXA7ZfG4u91asp309ibbRrHIu1AgUVEpJu5cUwUAL+8vJ99ynNnmzyoFyYT7M0vJ+dEFRH+Htx/ZfMBvCO1X5G0I3UJiYh0M1cMDmXP0zNwd1JYAQjxcSc+KoBtjRs4Pj5zCN7uzT9SmjZYzMwro7qu4bzG7Ig0UQuLiEg35OFq6bQNIFszbaitO+qyAcHMbGGrgN4BngR7u1FvNdjdOEhY5HyphUVERM7LLyb1JczPg6uGhrUYnkwmE/FR/qzMLGR7TnGzbQFEHKUWFhEROS/uLhZuHBOFv2freys1DbzdnquBt3JhFFhERKTDDImwTWfeX6C1WOTCKLCIiEiHGRBq2yvpQEH5GZslijhCgUVERDpMTJA3FrOJitoGCspqnF0d6cYUWEREpMO4uZjtu1MfULeQXAAFFhER6VD9ezV2CxUqsMj5U2AREZEO1a+XDwAHCiucXBPpzhRYRESkQ6mFRdqDAouIiHSo/o0tLAfVwiIXQIFFREQ6VFOXUG5xFVW1DU6ujXRXCiwiItKhgrzdCPSyrYZ7sEjdQnJ+FFhERKTDqVtILpQCi4iIdLh+GngrF0iBRUREOlx/TW2WC6TAIiIiHe5Ul5BaWOT8KLCIiEiHGxhmCyz78sspra5zcm2kO1JgERGRDhcT5MWAUB9qG6x8tSvf2dWRbkiBRUREOpzJZOLakZEAfJye6+TaSHekwCIiIp2iKbCsO3CcwrIaJ9dGuhsFFhER6RSxId6MjA6gwWrwxY5jzq6OdDMKLCIi0mnULSTnS4FFREQ6zffjIzCZYEt2MYePa00WaTsFFhER6TShfh5cPqgXAO+sOwTA3vwyrnx+Fa+vOejEmklXp8AiIiKd6meX9QXgn5tyKKmq44mPdnKwsIK/pO6jsrbeybWTruq8Asv8+fOJjY3Fw8ODxMRENm7c2GrZ119/nUmTJhEYGEhgYCBJSUktls/IyODaa6/F398fb29vxo0bR3Z29vlUT0REurBJA0MYFOZDRW0D97y/mQ1ZJwAor6nns20ajCstcziwLF68mOTkZJ566im2bNnCyJEjmT59OgUFBS2WX7VqFbfccgsrV64kLS2N6Ohopk2bRm7uqQFXBw4cYOLEicTFxbFq1Sq2b9/OE088gYeHx/nfmYiIdEkmk4mfT7S1sny7/zhgW1gO4B8b9R9VaZnJMAzDkRMSExMZN24cr776KgBWq5Xo6Gjuv/9+Hn300XOe39DQQGBgIK+++iqzZ88G4Oabb8bV1ZWFCxeexy1AaWkp/v7+lJSU4Ofnd17XEBGRzlNd18Bl877meEUt4X4e/OuuCVz5/CrqGgy+eGASQyP1u/xi4Mjnt0MtLLW1tWzevJmkpKRTFzCbSUpKIi0trU3XqKyspK6ujqCgIMAWeD7//HMGDRrE9OnTCQ0NJTExkY8++qjVa9TU1FBaWtrsJSIi3YeHq4VfJw3EzWLmD9cNIzrIi2lDwwH4QK0s0gKHAktRURENDQ2EhYU1Ox4WFkZeXl6brvHII48QGRlpDz0FBQWUl5czb948ZsyYwVdffcX111/PDTfcwOrVq1u8RkpKCv7+/vZXdHS0I7chIiJdwG0TYsn84wymDbMFlR8nxgDw0dZcauobznl+2oHjvPBVZpvKtlVlbT1Wq0MdD9JJOnWW0Lx58/jwww9ZunSpfXyK1WoF4LrrruPBBx8kISGBRx99lGuuuYYFCxa0eJ25c+dSUlJif+Xk5HTaPYiISPsxmUz2ryf0CybMz52ymnrWHTje6jmGYfDW2ixufWM9L3+9n8Wb2uczYGduCcOf+pI/fp7RLteT9uVQYAkJCcFisZCf33ynzfz8fMLDw8967nPPPce8efP46quviI+Pb3ZNFxcXhg4d2qz8kCFDWp0l5O7ujp+fX7OXiIh0b2aziaQhthb85btb39H595/u5g+f7aapIeSjre2zau6afYVYDVuXlKZXdz0OBRY3NzfGjBlDamqq/ZjVaiU1NZUJEya0et6zzz7L008/zbJlyxg7duwZ1xw3bhyZmZnNju/du5c+ffo4Uj0REenmmrqHlu/Ox2o1OHy8gic/3knOiUoAtmaf5J11hzCZ4L4rBmBuXDU3+3jlBf/sAwW2lXer6hrOGpjEORzuEkpOTub111/n3XffJSMjg7vvvpuKigrmzJkDwOzZs5k7d669/J/+9CeeeOIJ3nrrLWJjY8nLyyMvL4/y8nJ7mYcffpjFixfz+uuvs3//fl599VU+/fRT7rnnnna4RRER6S4m9AvG192FwrIatmSf5P4PtvJe2mF+vTgdq9VgweoDANwwKorfTB/Mpf1DAPhk24W3shwoPPW59Om2o83eW7mngHHPrGDdgaIL/jlyfhwOLLNmzeK5557jySefJCEhgfT0dJYtW2YfiJudnc2xY6cW/nnttdeora3lxhtvJCIiwv567rnn7GWuv/56FixYwLPPPsuIESN44403+M9//sPEiRPb4RZFRKS7cHMxc/lg29L9D/1rG9uPlACw+fBJnvsqk68aWz7uurwfANcm2DZT/Cj9KA6u0tGMYRjNAsvqvYUUV9bav39n3SEKy2r4bLsWtnMWh9dh6Yq0DouISM/xybajPPDBVvv3o2IC2JpdbP8+aUgYb9xuG15QWl3H2D+uoLbeyucPTGRYpH+za7226gDvrMvivZ8lMjjct9WfWVBWzfhnUjGboF8vH/YXlJNywwhuGR9DXYOVkb//israBiYNDGHhzxPb94YvYh22DouIiEhHmzK4F64W2+yhhOgAPrjjEvr38ra/f/eU/vav/TxcmRoXCsAfPt3NyYpTrSJr9hbyp2V7yC+tYek5BuY2jV+JDvLihtG9Afgk3dYttP1ICZW1tqnT2ScufKxMRzEMo0dPyVZgERGRLsXPw5Ufjo4iwMuVlBtG4OFq4enrhuNiNjF5UC/G9AlsVv6Xl/fHy83ChqwTXDt/LasyCzhUVEHyP7fZy2w+fOKsP7OpO6h/Lx++H2/rZlqfdZycE5WknTZuJfdkFfUN1va61XZjGAZzl+wg4Q9fkVVU4ezqdAgFFhER6XLm/TCezb+7iiERtm6CSweEsPaRK/n7bWPOKJsQHcCSey4lOsiTnBNV/PTtTUx5bhVF5TX0DvAEYFtOCdV1zReYe3/DYeYu2U5NfcNpgcWb6CAvJg0MwWic4px28NSaMPVWg2Ml1R112+ft4/SjfLgph9LqenvLUE+jwCIiIl2SxWxq9n24vwcerpYWy8aF+/HJvRP50dgoe0jxcXfhnTnjCPZ2o7bBys7cEnv5mvoGnv5sNx9szOG/O/I4UGhrlejfyweAWxNty2r887scNh8+CYBn489ua7eQYRg88/lubn1jPS98lcn6g8cvaGBwa46VVPHExzvt36/e2/JmxN2di7MrICIi0h4Cvd149saRABSW1WAxmwjydmNsbCBf7spn06GTjI217WNna3Gxde38Z8sRDjYFllBbYEkaEkqYnzv5pTUAhPi4MyzSj9V7C8k+UcllbajPu+sO8fo3WYBtV+qXv97PLeNj+OMPhp8Rxs6XYRj89t/bKauuZ1CYD3vzy0nPKaa4spYAL7d2+RldhVpYRESkx+nl606Qt+0De1xjSPnu0KlxLGmnLf2/dn8RucVVwKkWFheLmZvHxdjLXNIviD7BXkDbWlh25pbw/77YA8At42O4LiESk8nWxXTP+5ubdU81WA0qas5vZd31B0/wzb4i3FzMvPaTMQwO88VqwDf7et56MQosIiLSozW1qmzOPmmfRbO+cVyKxWyiqZcm0MvVHnIAbh4fbW8JuaRfMDFBZw8su4+WcvPf07jhr9/y07c3Uttg5aqhYfy/64fzl5tHMf/Ho3GzmPlyVz43LlhHVlEFaQeOM/nZlVw672v2F5S3eF2AZTvzeObz3WeMw2laSG/W2Gj69/Kxr2Gzem+hQ39G3YECi4iI9GjDIv3wcDVTXFnHgcJyqusa2JxtG5fy84l97eWaWleaRPh78ouJfRkU5sOM4eFENwaWnBYCS3VdA/d/sIX1B0+wJbuYovJaIv09+PON8fYNHr83IoJ3fzaeQC9XduaWcvVf1vDjN9aTW1xFSVUdjy3d0eIYl/Kaen7zr228/k0W76Udsh/POFbK6r2FmE1wxyTbQnqXDzoVWM41XubIycouOeOpNQosIiLSo7lazIyKtk2F3nToJOk5xdTWW+nl6859Vw7A3cX2Ufi/gQVg7veG8NWDlxPi437WFpYXV+zlQGEFvXzdee3W0bx262iW3nvZGeNIJvQP5r+/mkxi3yCq66wYBlw/qjeerhY2Zp3gX98dOePaS7ccobyxy+i1VQfsX/+tsXXleyMiiGnsrhobG4inq4XCshoyjpW1+meyfHc+E/+0kle+3n/2P7wuRIFFRER6vLGxtsCyeFM2KzNts2gm9AvGz8OVaxrXXRkZHXDWazS1sBRX1lFSVWc/viX7JK+vOQjA/7t+BFePiODqERGE+Xm0eJ1wfw/+cccl/PnGeBb9PJEXZyXw4FUDAXjmiwyKymvsZQ3D4L20w4Ct++pkZR1vr81i3f4iPm3cJuCuy08tpOfuYuHS/sGAbYYT2Fp/Hl+6g4f+uc3eJbaq8c9g2c68Vu/3REUtDy5O55/f5XTI7CZHKbCIiEiPd9OYaPw8XNh2pIS/rbaFi0v62T7Yn/7BMP5+2xh+NDbqrNfwcXchuHGMy+ndQilfZGBtbCm5amhYm+pjMZu4aWw0EwfaNm/82WV9GRrhR0lVnb3lBCDt4HH2FZTj5WbhD9cNA+CVlfu59c0NNFgNpsaFMrx38+0Irm9cqfeddYcap1Vv4P0N2fxnyxEy8koByMyztb7sLShrFr5O95/NR1i6NZff/ns7t76xgcPHnbsgnQKLiIj0eDHBXrz503H27h+wdc8AeLm5MG1YOC6Wc38kNnW9NAWWIycr2XToJCYTPDIj7rzr52Ix85vpgwD4cFOOfdbQwsbWletH9eaWcTEMDvOltt7WlXTL+GhevmXUGde6Jj6Sx75nq8vr32TZ15EB2+wlwzDY0xhYDAO2Zp884xpgazlqsu7Acaa/tIaDha0PDO5oCiwiInJRGBcbxPwfj8ZiNjEg1IfYxvDhiP8dx/J5Y7fM+Nggwv1b7gJqqymDQukb4k1ZdT1Lthxha/ZJ++7UsyfEYjabeO6mkUwbGsYbs8eSckM83u4tL6d25+T+PN3YIhPp78H0YbaWn+1HSjhysso+DgZoFmiaGIZhDyzP3hjPpf2Duax/CH1DvM8o21m0cJyIiFw0koaGseo3U/B2d7HP3nHE/waWT7fblsH//sjIC66b2Wzip5fG8tQnu3hjbRZ19VYarAbXxEfYd5oeEeXP32ePbdP1bpsQy5TBoYT4uJO6J58vd+WzM7fE3h3U5LtDZwaWYyXV5JfaFt+7Jj6Cm8ZEUVHbcF5/Zu1FLSwiInJRiQ7yarbeiqPngq2lYl9+GTtzS7GYTVw9PLxd6vbDMVH4urtw+HglR0uq6RviTcoNI877etFBXni6WRjROM4l41gZ2xu3KBjauE9Tek7xGdObm1pXhkT44uVmC3c+rbTmdBYFFhERkTYaHxuEq8XEjtwSbnl9PQCXDQgh2Me9Xa7v4+7Cj8ZFA+Dhaua1n4zG18P1gq8bE+SFn4cLtQ1WPt12qlXI18OFqrqGM6ZAbzlcDMDomMD/vZTTKLCIiIi0UWyIN3+7bQxuLmaKymsB+H58RLv+jHuvGMB1CZG8dusY4sL92uWaJpOJEVG2VpasIttsn6GRfvZAsvnwiWblm1pYFFhERES6qSvjbINe3V3M+HrYZhi1pyBvN/5y8yiuiAtt1+v+7/TnIeG+jO1jCySfbT/GO99mkZqRT3VdA7uP2qY/j4oJaNc6XAgNuhUREXHQ5EG9SH3ocgwD/D0vvMumM8T3DrB/HeTtRi9fd8Y0Lqj33eGTfNc4W+jS/sHUNlgJ9nazDzLuChRYREREzkNUYNf5MG+LEae1sAwO88VkMpHYN5jbLulDzslKXC1mUjPyWde4k/WomECnzgr6XwosIiIiF4HoIE/8PV0pqaojLsI2TdpiNvH0D4bby6zYnc/9H2ylqq7Bvp1BV6ExLCIiIhcBk8nEmMYxKwmt7JuUNDSMJfdcyq+TBvLjxJhOrN25qYVFRETkIvH0D4YzY18R349vfaG7IRF+DIlon9lJ7UmBRURE5CLRO8DTvs5Ld6MuIREREenyFFhERESky1NgERERkS5PgUVERES6PAUWERER6fIUWERERKTLU2ARERGRLk+BRURERLo8BRYRERHp8hRYREREpMtTYBEREZEuT4FFREREujwFFhEREenyesRuzYZhAFBaWurkmoiIiEhbNX1uN32On02PCCxlZWUAREd3zy2zRURELmZlZWX4+/uftYzJaEus6eKsVitHjx7F19cXk8nUrtcuLS0lOjqanJwc/Pz82vXaXUVPv8eefn+ge+wJevr9ge6xJ2jv+zMMg7KyMiIjIzGbzz5KpUe0sJjNZqKiojr0Z/j5+fXIv3yn6+n32NPvD3SPPUFPvz/QPfYE7Xl/52pZaaJBtyIiItLlKbCIiIhIl6fAcg7u7u489dRTuLu7O7sqHaan32NPvz/QPfYEPf3+QPfYEzjz/nrEoFsRERHp2dTCIiIiIl2eAouIiIh0eQosIiIi0uUpsIiIiEiXp8ByDvPnzyc2NhYPDw8SExPZuHGjs6t0XlJSUhg3bhy+vr6Ehobygx/8gMzMzGZlpkyZgslkava66667nFRjx/3f//3fGfWPi4uzv19dXc29995LcHAwPj4+/PCHPyQ/P9+JNXZMbGzsGfdnMpm49957ge75/NasWcP3v/99IiMjMZlMfPTRR83eNwyDJ598koiICDw9PUlKSmLfvn3Nypw4cYJbb70VPz8/AgIC+PnPf055eXkn3sXZne0e6+rqeOSRRxgxYgTe3t5ERkYye/Zsjh492uwaLT37efPmdfKdtOxcz/CnP/3pGXWfMWNGszLd+RkCLf67NJlM/PnPf7aX6crPsC2fD235/Zmdnc3MmTPx8vIiNDSUhx9+mPr6+narpwLLWSxevJjk5GSeeuoptmzZwsiRI5k+fToFBQXOrprDVq9ezb333sv69etZvnw5dXV1TJs2jYqKimbl7rjjDo4dO2Z/Pfvss06q8fkZNmxYs/qvXbvW/t6DDz7Ip59+yr/+9S9Wr17N0aNHueGGG5xYW8ds2rSp2b0tX74cgJtuusleprs9v4qKCkaOHMn8+fNbfP/ZZ5/l5ZdfZsGCBWzYsAFvb2+mT59OdXW1vcytt97Krl27WL58OZ999hlr1qzhzjvv7KxbOKez3WNlZSVbtmzhiSeeYMuWLSxZsoTMzEyuvfbaM8r+4Q9/aPZs77///s6o/jmd6xkCzJgxo1ndP/jgg2bvd+dnCDS7t2PHjvHWW29hMpn44Q9/2KxcV32Gbfl8ONfvz4aGBmbOnEltbS3r1q3j3Xff5Z133uHJJ59sv4oa0qrx48cb9957r/37hoYGIzIy0khJSXFirdpHQUGBARirV6+2H7v88suNX/3qV86r1AV66qmnjJEjR7b4XnFxseHq6mr861//sh/LyMgwACMtLa2Tati+fvWrXxn9+/c3rFarYRjd//kBxtKlS+3fW61WIzw83Pjzn/9sP1ZcXGy4u7sbH3zwgWEYhrF7924DMDZt2mQv89///tcwmUxGbm5up9W9rf73HluyceNGAzAOHz5sP9anTx/jxRdf7NjKtYOW7u/22283rrvuulbP6YnP8LrrrjOuvPLKZse6yzM0jDM/H9ry+/OLL74wzGazkZeXZy/z2muvGX5+fkZNTU271EstLK2ora1l8+bNJCUl2Y+ZzWaSkpJIS0tzYs3aR0lJCQBBQUHNjr///vuEhIQwfPhw5s6dS2VlpTOqd9727dtHZGQk/fr149ZbbyU7OxuAzZs3U1dX1+x5xsXFERMT0y2fZ21tLYsWLeJnP/tZsw0/u/vzO11WVhZ5eXnNnpm/vz+JiYn2Z5aWlkZAQABjx461l0lKSsJsNrNhw4ZOr3N7KCkpwWQyERAQ0Oz4vHnzCA4OZtSoUfz5z39u16b2jrZq1SpCQ0MZPHgwd999N8ePH7e/19OeYX5+Pp9//jk///nPz3ivuzzD//18aMvvz7S0NEaMGEFYWJi9zPTp0yktLWXXrl3tUq8esflhRygqKqKhoaHZHz5AWFgYe/bscVKt2ofVauXXv/41l112GcOHD7cf//GPf0yfPn2IjIxk+/btPPLII2RmZrJkyRIn1rbtEhMTeeeddxg8eDDHjh3j97//PZMmTWLnzp3k5eXh5uZ2xodAWFgYeXl5zqnwBfjoo48oLi7mpz/9qf1Yd39+/6vpubT0b7Dpvby8PEJDQ5u97+LiQlBQULd8rtXV1TzyyCPccsstzTaWe+CBBxg9ejRBQUGsW7eOuXPncuzYMV544QUn1rZtZsyYwQ033EDfvn05cOAAjz32GFdffTVpaWlYLJYe9wzfffddfH19z+hu7i7PsKXPh7b8/szLy2vx32rTe+1BgeUidO+997Jz585m4zuAZn3GI0aMICIigqlTp3LgwAH69+/f2dV02NVXX23/Oj4+nsTERPr06cM///lPPD09nViz9vfmm29y9dVXExkZaT/W3Z/fxa6uro4f/ehHGIbBa6+91uy95ORk+9fx8fG4ubnxy1/+kpSUlC6/BPzNN99s/3rEiBHEx8fTv39/Vq1axdSpU51Ys47x1ltvceutt+Lh4dHseHd5hq19PnQF6hJqRUhICBaL5YxR0Pn5+YSHhzupVhfuvvvu47PPPmPlypVERUWdtWxiYiIA+/fv74yqtbuAgAAGDRrE/v37CQ8Pp7a2luLi4mZluuPzPHz4MCtWrOAXv/jFWct19+fX9FzO9m8wPDz8jEHw9fX1nDhxols916awcvjwYZYvX96sdaUliYmJ1NfXc+jQoc6pYDvq168fISEh9r+XPeUZAnzzzTdkZmae898mdM1n2NrnQ1t+f4aHh7f4b7XpvfagwNIKNzc3xowZQ2pqqv2Y1WolNTWVCRMmOLFm58cwDO677z6WLl3K119/Td++fc95Tnp6OgAREREdXLuOUV5ezoEDB4iIiGDMmDG4uro2e56ZmZlkZ2d3u+f59ttvExoaysyZM89arrs/v759+xIeHt7smZWWlrJhwwb7M5swYQLFxcVs3rzZXubrr7/GarXaA1tX1xRW9u3bx4oVKwgODj7nOenp6ZjN5jO6UrqDI0eOcPz4cfvfy57wDJu8+eabjBkzhpEjR56zbFd6huf6fGjL788JEyawY8eOZuGzKXwPHTq03Soqrfjwww8Nd3d345133jF2795t3HnnnUZAQECzUdDdxd133234+/sbq1atMo4dO2Z/VVZWGoZhGPv37zf+8Ic/GN99952RlZVlfPzxx0a/fv2MyZMnO7nmbffQQw8Zq1atMrKysoxvv/3WSEpKMkJCQoyCggLDMAzjrrvuMmJiYoyvv/7a+O6774wJEyYYEyZMcHKtHdPQ0GDExMQYjzzySLPj3fX5lZWVGVu3bjW2bt1qAMYLL7xgbN261T5DZt68eUZAQIDx8ccfG9u3bzeuu+46o2/fvkZVVZX9GjNmzDBGjRplbNiwwVi7dq0xcOBA45ZbbnHWLZ3hbPdYW1trXHvttUZUVJSRnp7e7N9m08yKdevWGS+++KKRnp5uHDhwwFi0aJHRq1cvY/bs2U6+M5uz3V9ZWZnxm9/8xkhLSzOysrKMFStWGKNHjzYGDhxoVFdX26/RnZ9hk5KSEsPLy8t47bXXzji/qz/Dc30+GMa5f3/W19cbw4cPN6ZNm2akp6cby5YtM3r16mXMnTu33eqpwHIOr7zyihETE2O4ubkZ48ePN9avX+/sKp0XoMXX22+/bRiGYWRnZxuTJ082goKCDHd3d2PAgAHGww8/bJSUlDi34g6YNWuWERERYbi5uRm9e/c2Zs2aZezfv9/+flVVlXHPPfcYgYGBhpeXl3H99dcbx44dc2KNHffll18agJGZmdnseHd9fitXrmzx7+Xtt99uGIZtavMTTzxhhIWFGe7u7sbUqVPPuPfjx48bt9xyi+Hj42P4+fkZc+bMMcrKypxwNy072z1mZWW1+m9z5cqVhmEYxubNm43ExETD39/f8PDwMIYMGWL8v//3/5p94DvT2e6vsrLSmDZtmtGrVy/D1dXV6NOnj3HHHXec8Z++7vwMm/ztb38zPD09jeLi4jPO7+rP8FyfD4bRtt+fhw4dMq6++mrD09PTCAkJMR566CGjrq6u3eppaqysiIiISJelMSwiIiLS5SmwiIiISJenwCIiIiJdngKLiIiIdHkKLCIiItLlKbCIiIhIl6fAIiIiIl2eAouIiIh0eQosIiIi0uUpsIiIiEiXp8AiIiIiXZ4Ci4iIiHR5/x9VCgvUJU33rwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(torch.tensor(lossi).view(-1, 1000).mean(1));\n",
    "\n",
    "# Put layers into evaluation mode (needed esp. for batchnorm)\n",
    "for layer in model.layers:\n",
    "    layer.training = False\n",
    "\n",
    "# Evaluate the loss\n",
    "# See Makemore #2. We did this before but it's refactored here for convenience:\n",
    "@torch.no_grad() # Decorator, disables gradient tracking (no \"book-keeping\" on torch's side for this function)\n",
    "def split_loss(split):\n",
    "    x, y = {\n",
    "        'train': (Xtr, Ytr),\n",
    "        'val': (Xdev, Ydev),\n",
    "        'test': (Xte, Yte),\n",
    "    }[split]   # That's a switch!\n",
    "    \n",
    "    logits = model(x)\n",
    "    \n",
    "    loss = F.cross_entropy(logits, y)\n",
    "    print(split, loss.item())\n",
    "\n",
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb0acc2a",
   "metadata": {},
   "source": [
    "<center>Notebook by <a href=\"https://github.com/mk2112\" target=\"_blank\">mk2112</a>.</center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "e2277cd2c28b4b86c92dac38c2e6de867ac05bb102342d6ba35d1a43ffc78954"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}