Проект_по_AB_тестам_ipynb_Комиссарова_Т_.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "96ac1922",
   "metadata": {
    "id": "96ac1922"
   },
   "source": [
    "# Шаблон для проекта по A/B - тестам."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e01e5ec9",
   "metadata": {
    "id": "e01e5ec9"
   },
   "source": [
    "### `chat_id ` из Телеграма:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "Of7L1OK6jEKT",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Of7L1OK6jEKT",
    "outputId": "546b1e89-c22c-4666-9eb5-058761a0a1ac"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "446110916"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "446110916"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fb036f5",
   "metadata": {
    "id": "1fb036f5"
   },
   "source": [
    "------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8c6f14f",
   "metadata": {
    "id": "d8c6f14f"
   },
   "source": [
    "##  <ins>Задание 1.</ins>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cMi1ZWvej6zB",
   "metadata": {
    "id": "cMi1ZWvej6zB"
   },
   "source": [
    "### 0. Запосить у бота в Телеграме условие командой `/get_project1_desc`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "YOqEUgrJfBaM",
   "metadata": {
    "id": "YOqEUgrJfBaM"
   },
   "source": [
    "**Контекст**\n",
    "\n",
    "В связи с постановлением Центрального Банка\n",
    "наше текущее содержание диалога с клиентом (скрипт)\n",
    "операторов считается слишком жёстким,\n",
    "и нам нужно придумать более мягкий скрипт,\n",
    "который будет всё ещё иметь хорошие показатели.\n",
    "У нас есть два варианта более мягкого скрипта,\n",
    "и мы хотим выбрать лучший из них.\n",
    "При этом у нас нет времени \n",
    "чтобы ждать вызревания долгих метрик вроде PV.\n",
    "        \n",
    "**Тест**\n",
    "\n",
    "Мы проводим тест,\n",
    "в рамках которого мы разбиваем весь поток\n",
    "на две части рановероятно,\n",
    "где одной части мы продаём одним вариантом скрипта,\n",
    "другой - другим.\n",
    "Целью этого теста\n",
    "является поиск наилучшего скрипта.\n",
    "\n",
    "**Статистические вводные**\n",
    "\n",
    "Уровень значимости 5%.\n",
    "Мы хотели бы различать 8%-ное\n",
    "изменение целевой метрики\n",
    "с вероятностью 80%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "L6bIrzR6fKxd",
   "metadata": {
    "id": "L6bIrzR6fKxd"
   },
   "source": [
    "Значит, **alpha** = 0.05\n",
    "**Lift** = 0.08\n",
    "**beta** = 0.20"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b-XTZ2r7fvAL",
   "metadata": {
    "id": "b-XTZ2r7fvAL"
   },
   "source": [
    "Посмотрим датасет"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "z8efTf5BgKXt",
   "metadata": {
    "id": "z8efTf5BgKXt"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from scipy.stats import norm\n",
    "from scipy.stats import mannwhitneyu\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import math\n",
    "import io"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "Btsu2jR4gzLW",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 73
    },
    "id": "Btsu2jR4gzLW",
    "outputId": "02602fd5-84a2-4e55-ec5c-da13fd3e6693"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-5b9f36c9-af7a-4842-ab3e-78444a166102\" name=\"files[]\" multiple disabled\n",
       "        style=\"border:none\" />\n",
       "     <output id=\"result-5b9f36c9-af7a-4842-ab3e-78444a166102\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script>// Copyright 2017 Google LLC\n",
       "//\n",
       "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
       "// you may not use this file except in compliance with the License.\n",
       "// You may obtain a copy of the License at\n",
       "//\n",
       "//      http://www.apache.org/licenses/LICENSE-2.0\n",
       "//\n",
       "// Unless required by applicable law or agreed to in writing, software\n",
       "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
       "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
       "// See the License for the specific language governing permissions and\n",
       "// limitations under the License.\n",
       "\n",
       "/**\n",
       " * @fileoverview Helpers for google.colab Python module.\n",
       " */\n",
       "(function(scope) {\n",
       "function span(text, styleAttributes = {}) {\n",
       "  const element = document.createElement('span');\n",
       "  element.textContent = text;\n",
       "  for (const key of Object.keys(styleAttributes)) {\n",
       "    element.style[key] = styleAttributes[key];\n",
       "  }\n",
       "  return element;\n",
       "}\n",
       "\n",
       "// Max number of bytes which will be uploaded at a time.\n",
       "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
       "\n",
       "function _uploadFiles(inputId, outputId) {\n",
       "  const steps = uploadFilesStep(inputId, outputId);\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  // Cache steps on the outputElement to make it available for the next call\n",
       "  // to uploadFilesContinue from Python.\n",
       "  outputElement.steps = steps;\n",
       "\n",
       "  return _uploadFilesContinue(outputId);\n",
       "}\n",
       "\n",
       "// This is roughly an async generator (not supported in the browser yet),\n",
       "// where there are multiple asynchronous steps and the Python side is going\n",
       "// to poll for completion of each step.\n",
       "// This uses a Promise to block the python side on completion of each step,\n",
       "// then passes the result of the previous step as the input to the next step.\n",
       "function _uploadFilesContinue(outputId) {\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  const steps = outputElement.steps;\n",
       "\n",
       "  const next = steps.next(outputElement.lastPromiseValue);\n",
       "  return Promise.resolve(next.value.promise).then((value) => {\n",
       "    // Cache the last promise value to make it available to the next\n",
       "    // step of the generator.\n",
       "    outputElement.lastPromiseValue = value;\n",
       "    return next.value.response;\n",
       "  });\n",
       "}\n",
       "\n",
       "/**\n",
       " * Generator function which is called between each async step of the upload\n",
       " * process.\n",
       " * @param {string} inputId Element ID of the input file picker element.\n",
       " * @param {string} outputId Element ID of the output display.\n",
       " * @return {!Iterable<!Object>} Iterable of next steps.\n",
       " */\n",
       "function* uploadFilesStep(inputId, outputId) {\n",
       "  const inputElement = document.getElementById(inputId);\n",
       "  inputElement.disabled = false;\n",
       "\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  outputElement.innerHTML = '';\n",
       "\n",
       "  const pickedPromise = new Promise((resolve) => {\n",
       "    inputElement.addEventListener('change', (e) => {\n",
       "      resolve(e.target.files);\n",
       "    });\n",
       "  });\n",
       "\n",
       "  const cancel = document.createElement('button');\n",
       "  inputElement.parentElement.appendChild(cancel);\n",
       "  cancel.textContent = 'Cancel upload';\n",
       "  const cancelPromise = new Promise((resolve) => {\n",
       "    cancel.onclick = () => {\n",
       "      resolve(null);\n",
       "    };\n",
       "  });\n",
       "\n",
       "  // Wait for the user to pick the files.\n",
       "  const files = yield {\n",
       "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
       "    response: {\n",
       "      action: 'starting',\n",
       "    }\n",
       "  };\n",
       "\n",
       "  cancel.remove();\n",
       "\n",
       "  // Disable the input element since further picks are not allowed.\n",
       "  inputElement.disabled = true;\n",
       "\n",
       "  if (!files) {\n",
       "    return {\n",
       "      response: {\n",
       "        action: 'complete',\n",
       "      }\n",
       "    };\n",
       "  }\n",
       "\n",
       "  for (const file of files) {\n",
       "    const li = document.createElement('li');\n",
       "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
       "    li.append(span(\n",
       "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
       "        `last modified: ${\n",
       "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
       "                                    'n/a'} - `));\n",
       "    const percent = span('0% done');\n",
       "    li.appendChild(percent);\n",
       "\n",
       "    outputElement.appendChild(li);\n",
       "\n",
       "    const fileDataPromise = new Promise((resolve) => {\n",
       "      const reader = new FileReader();\n",
       "      reader.onload = (e) => {\n",
       "        resolve(e.target.result);\n",
       "      };\n",
       "      reader.readAsArrayBuffer(file);\n",
       "    });\n",
       "    // Wait for the data to be ready.\n",
       "    let fileData = yield {\n",
       "      promise: fileDataPromise,\n",
       "      response: {\n",
       "        action: 'continue',\n",
       "      }\n",
       "    };\n",
       "\n",
       "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
       "    let position = 0;\n",
       "    do {\n",
       "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
       "      const chunk = new Uint8Array(fileData, position, length);\n",
       "      position += length;\n",
       "\n",
       "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
       "      yield {\n",
       "        response: {\n",
       "          action: 'append',\n",
       "          file: file.name,\n",
       "          data: base64,\n",
       "        },\n",
       "      };\n",
       "\n",
       "      let percentDone = fileData.byteLength === 0 ?\n",
       "          100 :\n",
       "          Math.round((position / fileData.byteLength) * 100);\n",
       "      percent.textContent = `${percentDone}% done`;\n",
       "\n",
       "    } while (position < fileData.byteLength);\n",
       "  }\n",
       "\n",
       "  // All done.\n",
       "  yield {\n",
       "    response: {\n",
       "      action: 'complete',\n",
       "    }\n",
       "  };\n",
       "}\n",
       "\n",
       "scope.google = scope.google || {};\n",
       "scope.google.colab = scope.google.colab || {};\n",
       "scope.google.colab._files = {\n",
       "  _uploadFiles,\n",
       "  _uploadFilesContinue,\n",
       "};\n",
       "})(self);\n",
       "</script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving hist_telesales.csv to hist_telesales.csv\n"
     ]
    }
   ],
   "source": [
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "xn-shPF9hA3r",
   "metadata": {
    "id": "xn-shPF9hA3r"
   },
   "outputs": [],
   "source": [
    "import io\n",
    "hist_data = pd.read_csv(io.BytesIO(uploaded['hist_telesales.csv']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "_4yjksmPftwr",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "_4yjksmPftwr",
    "outputId": "6e982e1e-31c3-4f18-98b4-4c4ae15c1957"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-0f5dcce0-c4e3-415e-aacf-6a48d16a21aa\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Флаг дозвона</th>\n",
       "      <th>Флаг продажи</th>\n",
       "      <th>Расходы</th>\n",
       "      <th>PV</th>\n",
       "      <th>NPV</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>90</td>\n",
       "      <td>0</td>\n",
       "      <td>-90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>-5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>68</td>\n",
       "      <td>0</td>\n",
       "      <td>-68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>-22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0f5dcce0-c4e3-415e-aacf-6a48d16a21aa')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-0f5dcce0-c4e3-415e-aacf-6a48d16a21aa button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-0f5dcce0-c4e3-415e-aacf-6a48d16a21aa');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   ID  Флаг дозвона  Флаг продажи  Расходы  PV  NPV\n",
       "0   0             1             0       90   0  -90\n",
       "1   1             0             0        5   0   -5\n",
       "2   2             0             0       68   0  -68\n",
       "3   3             1             0       22   0  -22\n",
       "4   4             1             0       22   0  -22"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "uz8X5pZUmWDw",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "id": "uz8X5pZUmWDw",
    "outputId": "ea25c028-e6ec-4368-ada8-a10c20c02c57"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-4ccc2410-9c60-4d0c-8c74-92a494b65e92\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Флаг дозвона</th>\n",
       "      <th>Флаг продажи</th>\n",
       "      <th>Расходы</th>\n",
       "      <th>PV</th>\n",
       "      <th>NPV</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>367</td>\n",
       "      <td>2397</td>\n",
       "      <td>2030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>49</td>\n",
       "      <td>1213</td>\n",
       "      <td>1164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>493</td>\n",
       "      <td>1987</td>\n",
       "      <td>1494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>183</td>\n",
       "      <td>1536</td>\n",
       "      <td>1353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>67</td>\n",
       "      <td>1311</td>\n",
       "      <td>1244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72147</th>\n",
       "      <td>72147</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>858</td>\n",
       "      <td>1274</td>\n",
       "      <td>416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72149</th>\n",
       "      <td>72149</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>111</td>\n",
       "      <td>1301</td>\n",
       "      <td>1190</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72153</th>\n",
       "      <td>72153</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>448</td>\n",
       "      <td>1385</td>\n",
       "      <td>937</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72156</th>\n",
       "      <td>72156</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>577</td>\n",
       "      <td>1346</td>\n",
       "      <td>769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72160</th>\n",
       "      <td>72160</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>132</td>\n",
       "      <td>1385</td>\n",
       "      <td>1253</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>21805 rows × 6 columns</p>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4ccc2410-9c60-4d0c-8c74-92a494b65e92')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-4ccc2410-9c60-4d0c-8c74-92a494b65e92 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-4ccc2410-9c60-4d0c-8c74-92a494b65e92');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "          ID  Флаг дозвона  Флаг продажи  Расходы    PV   NPV\n",
       "5          5             1             1      367  2397  2030\n",
       "6          6             1             1       49  1213  1164\n",
       "14        14             1             1      493  1987  1494\n",
       "19        19             1             1      183  1536  1353\n",
       "21        21             1             1       67  1311  1244\n",
       "...      ...           ...           ...      ...   ...   ...\n",
       "72147  72147             1             1      858  1274   416\n",
       "72149  72149             1             1      111  1301  1190\n",
       "72153  72153             1             1      448  1385   937\n",
       "72156  72156             1             1      577  1346   769\n",
       "72160  72160             1             1      132  1385  1253\n",
       "\n",
       "[21805 rows x 6 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist_data.loc[hist_data['Флаг продажи'] == 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dd4c840",
   "metadata": {
    "id": "0dd4c840"
   },
   "source": [
    "### 1. Необходимо выбрать метрику и привести аргументацию."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05uerrkykO4J",
   "metadata": {
    "id": "05uerrkykO4J"
   },
   "source": [
    "В качестве целевой метрики я бы выбрала **флаг продажи**. Данная метрика отражает, насколько эффективен скрипт. При этом она не отличается продолжительностью вызревания, как, например, PV и NPV.\n",
    "\n",
    "**Нулевая гипотеза (Н0):**\n",
    "\n",
    "Введение более мягкого скрипта диалога не повлияло на процент продаж. Различия между тестовой и контрольной группой случайны и статистически не значимы.\n",
    "\n",
    "У нас будет две выборки Х1 с распределением Бернулли р1( контроль) и Х2 с распределением Бернулли р2(тест). \n",
    "\n",
    "Тогда Н0: р1 = р2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "NC1uCKKeY7oq",
   "metadata": {
    "id": "NC1uCKKeY7oq"
   },
   "source": [
    "Из исторических данных найдём вероятность р0 того, что флаг продажи == 1, которую будем использовать в дальнейших расчетах"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "vG7kmtMEjHOU",
   "metadata": {
    "id": "vG7kmtMEjHOU"
   },
   "outputs": [],
   "source": [
    "p0 = hist_data['Флаг продажи'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b644cd0a",
   "metadata": {
    "id": "b644cd0a"
   },
   "source": [
    "### 2. Альтернатива в критерии."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HQXeD9Cct6lV",
   "metadata": {
    "id": "HQXeD9Cct6lV"
   },
   "source": [
    "\n",
    "**Альтрнативная гипотеза (Н1):** \n",
    "\n",
    "Введение более мягкого скрипта диалога подняло процент продаж на 8% и более. Процент продаж в тестовой группе выше, чем в контрольной и это различие - результат изменений. Мы тестируем отклонение вверх, значит, проверяем правостороннюю гипотезу.\n",
    "\n",
    "Н1: р1 < p2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "KFMpcB0mjJXN",
   "metadata": {
    "id": "KFMpcB0mjJXN"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "727aefc4",
   "metadata": {
    "id": "727aefc4"
   },
   "source": [
    "### 3.1. Каков размер выборки? Привести аргументацию и написать как получилось то или иное число."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jY1qvVadvtP1",
   "metadata": {
    "id": "jY1qvVadvtP1"
   },
   "source": [
    "Сейчас будем считать.. У нас односторонняя альтернатива и бинарное распределение. Используем соответствующую формулу"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "T2kOG31ZsfMx",
   "metadata": {
    "id": "T2kOG31ZsfMx"
   },
   "source": [
    "MDE = lift * mu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2IRuHjvVs17t",
   "metadata": {
    "id": "2IRuHjvVs17t"
   },
   "outputs": [],
   "source": [
    "Lift = 0.08\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "roqQS75850Lc",
   "metadata": {
    "id": "roqQS75850Lc"
   },
   "outputs": [],
   "source": [
    "MDE = Lift * p0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "fYijOurfjLNp",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fYijOurfjLNp",
    "outputId": "d45970ea-0d06-4880-b54c-c192cd70adb1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1.6448536269514722, -0.8416212335729142)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "norm.ppf(0.95), norm.ppf(0.20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "P8P2MsIv5Hof",
   "metadata": {
    "id": "P8P2MsIv5Hof"
   },
   "outputs": [],
   "source": [
    "n = (2 * p0 * (1 - p0) * (norm.ppf(0.95) - norm.ppf(0.20)) ** 2) / MDE ** 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "073NwWKg6Sh8",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "073NwWKg6Sh8",
    "outputId": "31d2f700-4a55-41fb-9cfe-c325779c0167"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4462"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "math.ceil(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df8ee73",
   "metadata": {
    "id": "8df8ee73"
   },
   "source": [
    "### 3.2. Необходимо запросить у бота в Телеграме выборку полученного размера командой `/get_project1_sample {размер выборки}` и вписать <ins>код выборки</ins>:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "9vCrM3wcjM0H",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9vCrM3wcjM0H",
    "outputId": "b628ea5b-9e72-4abc-86c9-10a06cdd329e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "260"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "260"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e2d9b0",
   "metadata": {
    "id": "22e2d9b0"
   },
   "source": [
    "### 4. Принятие решения. Расписать подробно с аргументами."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tVMtwlLO8Uzd",
   "metadata": {
    "id": "tVMtwlLO8Uzd"
   },
   "source": [
    "Импортируем сгенерированные выборки test_group, control_group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "C9NO-_o3jON1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 108
    },
    "id": "C9NO-_o3jON1",
    "outputId": "3b9c1558-cf66-4b81-baa1-354ddc1e8f5d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-8e408c41-ebea-4804-9b0e-6bf0d4bcea0a\" name=\"files[]\" multiple disabled\n",
       "        style=\"border:none\" />\n",
       "     <output id=\"result-8e408c41-ebea-4804-9b0e-6bf0d4bcea0a\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script>// Copyright 2017 Google LLC\n",
       "//\n",
       "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
       "// you may not use this file except in compliance with the License.\n",
       "// You may obtain a copy of the License at\n",
       "//\n",
       "//      http://www.apache.org/licenses/LICENSE-2.0\n",
       "//\n",
       "// Unless required by applicable law or agreed to in writing, software\n",
       "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
       "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
       "// See the License for the specific language governing permissions and\n",
       "// limitations under the License.\n",
       "\n",
       "/**\n",
       " * @fileoverview Helpers for google.colab Python module.\n",
       " */\n",
       "(function(scope) {\n",
       "function span(text, styleAttributes = {}) {\n",
       "  const element = document.createElement('span');\n",
       "  element.textContent = text;\n",
       "  for (const key of Object.keys(styleAttributes)) {\n",
       "    element.style[key] = styleAttributes[key];\n",
       "  }\n",
       "  return element;\n",
       "}\n",
       "\n",
       "// Max number of bytes which will be uploaded at a time.\n",
       "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
       "\n",
       "function _uploadFiles(inputId, outputId) {\n",
       "  const steps = uploadFilesStep(inputId, outputId);\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  // Cache steps on the outputElement to make it available for the next call\n",
       "  // to uploadFilesContinue from Python.\n",
       "  outputElement.steps = steps;\n",
       "\n",
       "  return _uploadFilesContinue(outputId);\n",
       "}\n",
       "\n",
       "// This is roughly an async generator (not supported in the browser yet),\n",
       "// where there are multiple asynchronous steps and the Python side is going\n",
       "// to poll for completion of each step.\n",
       "// This uses a Promise to block the python side on completion of each step,\n",
       "// then passes the result of the previous step as the input to the next step.\n",
       "function _uploadFilesContinue(outputId) {\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  const steps = outputElement.steps;\n",
       "\n",
       "  const next = steps.next(outputElement.lastPromiseValue);\n",
       "  return Promise.resolve(next.value.promise).then((value) => {\n",
       "    // Cache the last promise value to make it available to the next\n",
       "    // step of the generator.\n",
       "    outputElement.lastPromiseValue = value;\n",
       "    return next.value.response;\n",
       "  });\n",
       "}\n",
       "\n",
       "/**\n",
       " * Generator function which is called between each async step of the upload\n",
       " * process.\n",
       " * @param {string} inputId Element ID of the input file picker element.\n",
       " * @param {string} outputId Element ID of the output display.\n",
       " * @return {!Iterable<!Object>} Iterable of next steps.\n",
       " */\n",
       "function* uploadFilesStep(inputId, outputId) {\n",
       "  const inputElement = document.getElementById(inputId);\n",
       "  inputElement.disabled = false;\n",
       "\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  outputElement.innerHTML = '';\n",
       "\n",
       "  const pickedPromise = new Promise((resolve) => {\n",
       "    inputElement.addEventListener('change', (e) => {\n",
       "      resolve(e.target.files);\n",
       "    });\n",
       "  });\n",
       "\n",
       "  const cancel = document.createElement('button');\n",
       "  inputElement.parentElement.appendChild(cancel);\n",
       "  cancel.textContent = 'Cancel upload';\n",
       "  const cancelPromise = new Promise((resolve) => {\n",
       "    cancel.onclick = () => {\n",
       "      resolve(null);\n",
       "    };\n",
       "  });\n",
       "\n",
       "  // Wait for the user to pick the files.\n",
       "  const files = yield {\n",
       "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
       "    response: {\n",
       "      action: 'starting',\n",
       "    }\n",
       "  };\n",
       "\n",
       "  cancel.remove();\n",
       "\n",
       "  // Disable the input element since further picks are not allowed.\n",
       "  inputElement.disabled = true;\n",
       "\n",
       "  if (!files) {\n",
       "    return {\n",
       "      response: {\n",
       "        action: 'complete',\n",
       "      }\n",
       "    };\n",
       "  }\n",
       "\n",
       "  for (const file of files) {\n",
       "    const li = document.createElement('li');\n",
       "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
       "    li.append(span(\n",
       "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
       "        `last modified: ${\n",
       "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
       "                                    'n/a'} - `));\n",
       "    const percent = span('0% done');\n",
       "    li.appendChild(percent);\n",
       "\n",
       "    outputElement.appendChild(li);\n",
       "\n",
       "    const fileDataPromise = new Promise((resolve) => {\n",
       "      const reader = new FileReader();\n",
       "      reader.onload = (e) => {\n",
       "        resolve(e.target.result);\n",
       "      };\n",
       "      reader.readAsArrayBuffer(file);\n",
       "    });\n",
       "    // Wait for the data to be ready.\n",
       "    let fileData = yield {\n",
       "      promise: fileDataPromise,\n",
       "      response: {\n",
       "        action: 'continue',\n",
       "      }\n",
       "    };\n",
       "\n",
       "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
       "    let position = 0;\n",
       "    do {\n",
       "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
       "      const chunk = new Uint8Array(fileData, position, length);\n",
       "      position += length;\n",
       "\n",
       "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
       "      yield {\n",
       "        response: {\n",
       "          action: 'append',\n",
       "          file: file.name,\n",
       "          data: base64,\n",
       "        },\n",
       "      };\n",
       "\n",
       "      let percentDone = fileData.byteLength === 0 ?\n",
       "          100 :\n",
       "          Math.round((position / fileData.byteLength) * 100);\n",
       "      percent.textContent = `${percentDone}% done`;\n",
       "\n",
       "    } while (position < fileData.byteLength);\n",
       "  }\n",
       "\n",
       "  // All done.\n",
       "  yield {\n",
       "    response: {\n",
       "      action: 'complete',\n",
       "    }\n",
       "  };\n",
       "}\n",
       "\n",
       "scope.google = scope.google || {};\n",
       "scope.google.colab = scope.google.colab || {};\n",
       "scope.google.colab._files = {\n",
       "  _uploadFiles,\n",
       "  _uploadFilesContinue,\n",
       "};\n",
       "})(self);\n",
       "</script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving Контроль.csv to Контроль (1).csv\n",
      "Saving Тест.csv to Тест (1).csv\n"
     ]
    }
   ],
   "source": [
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "bnnCglSd8wDH",
   "metadata": {
    "id": "bnnCglSd8wDH"
   },
   "outputs": [],
   "source": [
    "test_group = pd.read_csv(io.BytesIO(uploaded['Тест.csv']))\n",
    "control_group = pd.read_csv(io.BytesIO(uploaded['Контроль.csv']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "nM_FB42I9MR0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nM_FB42I9MR0",
    "outputId": "3fc92045-f032-451e-a7e4-a365b25ce063"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Флаг продажи    0.292022\n",
       "dtype: float64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_group[['Флаг продажи']].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "NJwyl8hn9E6r",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NJwyl8hn9E6r",
    "outputId": "6ac12144-71a8-452d-a371-f542ea2a0a29"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Флаг продажи    0.304348\n",
       "dtype: float64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "control_group[['Флаг продажи']].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "304Mm94ouodB",
   "metadata": {
    "id": "304Mm94ouodB"
   },
   "source": [
    "Целевая метрика: флаг продажи.\n",
    "\n",
    "Как вариант, в качестве критерия проверки однородности я бы попробовала Z-тест. Так как он позволяет работать с бинарным распределением"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "IVnKL3eF9E9j",
   "metadata": {
    "id": "IVnKL3eF9E9j"
   },
   "outputs": [],
   "source": [
    "# proportions_ztest ([s1, s2], [n1, n2], alternative = )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "pWcvOhzt9FAH",
   "metadata": {
    "id": "pWcvOhzt9FAH"
   },
   "outputs": [],
   "source": [
    "s1 = control_group['Флаг продажи'].sum()\n",
    "s2 = test_group['Флаг продажи'].sum()\n",
    "n1 = control_group['Флаг продажи'].count()\n",
    "n2 = test_group['Флаг продажи'].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "C5Ze68JkB1rz",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "C5Ze68JkB1rz",
    "outputId": "abbe1bdd-4dee-4bae-d0fc-a4abb311e08f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1358, 4462, 1303, 4462)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1, n1, s2, n2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mSE_VfnUvkjS",
   "metadata": {
    "id": "mSE_VfnUvkjS"
   },
   "source": [
    "## Z-тест "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "ZhOrnDIhB7Gu",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZhOrnDIhB7Gu",
    "outputId": "72d84ef1-4c52-4b4d-a43f-232b0412ba75"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1.2727081557947966, 0.8984391887732965)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from statsmodels.stats.proportion import proportions_ztest\n",
    "proportions_ztest ([s1, s2], [n1, n2], alternative = 'smaller')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91MWH5EdvtY5",
   "metadata": {
    "id": "91MWH5EdvtY5"
   },
   "source": [
    "## U-критерий Манна-Уитни"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "nMNskXOHt16f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nMNskXOHt16f",
    "outputId": "997bd9b1-952f-4a9c-aa15-d27161c24f07"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8984274517476196"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u1, p = mannwhitneyu(control_group['Флаг продажи'], test_group['Флаг продажи'], alternative=\"less\")\n",
    "p"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9jZ1KRazCyeF",
   "metadata": {
    "id": "9jZ1KRazCyeF"
   },
   "source": [
    "p-value = 0.89, это означает, что мы должны принять нулевую гипотезу о том, что различия в выборках случайны. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea196b1",
   "metadata": {
    "id": "7ea196b1"
   },
   "source": [
    "##  <ins>Задание 2.</ins>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "M0mI4jfTkCk6",
   "metadata": {
    "id": "M0mI4jfTkCk6"
   },
   "source": [
    "### 0. Запосить у бота в Телеграме условие командой `/get_project2_desc`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bl4XUp3ozNL8",
   "metadata": {
    "id": "bl4XUp3ozNL8"
   },
   "source": [
    "**Описание данных**\n",
    "\n",
    "Процесс оформления кредитной карты (КК) устроен следующим образом:\n",
    "\n",
    "1. человек заходит на сайт и заполняет информацию о себе (в том числе возраст и доход клиента) для получения кредитной карты;\n",
    "\n",
    "2. человек проходит скоринг (в том числе в рамках которого оценивается вероятность банкротства клиента);\n",
    "\n",
    "3. создаётся заявка на оформление кредитной карты;\n",
    "\n",
    "4. человеку выпускается виртуальная карта, а также предлагается наша услуга к этому счёту;\n",
    "\n",
    "5. с некоторой вероятностью человек утилизирует счёт (начинает им пользоваться);\n",
    "\n",
    "6. если человек не утилизирует счёт, мы несём некоторые затраты на время оператора и поддержку инфраструктуры;\n",
    "\n",
    "7. если человек утилизировал карту, мы имеем некоторый заработок (PV) и повышенные расходы.\n",
    "\n",
    "\n",
    "**Описание колонок в исторических данных.**\n",
    "\n",
    "* ID - номер заявки, прошедшей скоринг\n",
    "\n",
    "* Возраст - возраст клиента, указанный при оформлении КК\n",
    "\n",
    "* Доход клиента - доход клиента, указанный при оформлении КК\n",
    "\n",
    "* Вероятность банкротства - оценка вероятности банкротства клиента, полученная с помощью скоринга по параметрам клиента\n",
    "\n",
    "* Флаг утилизация счёта - совершение первой расходной операции на счёте КК\n",
    "\n",
    "* Расходы - сумма затрат на заявку\n",
    "\n",
    "* PV КК - 0 если счёт не утилизирован, иначе = PV кредитной карты\n",
    "\n",
    "* PV услуги - 0 если счёт не утилизирован, иначе = PV этой услуги\n",
    "\n",
    "* NPV = PV КК + PV услуги - Расходы"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "RF79Z977vKkJ",
   "metadata": {
    "id": "RF79Z977vKkJ"
   },
   "source": [
    "**Контекст**\n",
    "\n",
    "\n",
    "ЦБ смягчило правило,\n",
    "ограничивающее варианты продажи услуги.\n",
    "Мы хотим попробовать\n",
    "более агрессивную продажу услуги,\n",
    "но боимся,\n",
    "что это повлияет на экономику основного продукта (КК)\n",
    "из-за негатива клиентов.\n",
    "\n",
    "**Тест**\n",
    "\n",
    "\n",
    "Мы проводим тест,\n",
    "в рамках которого мы разбиваем весь поток\n",
    "на две части рановероятно,\n",
    "где одной части (контролю) \n",
    "мы будем предлагать услугу текущим способом,\n",
    "а второй части (тесту) - новым более агрессивным способом.\n",
    "Целью этого теста \n",
    "является увеличение доходности КК + услуги.\n",
    "\n",
    "**Статистические вводные**\n",
    "\n",
    "Проверка однородности выбранного параметра\n",
    "осуществяется с уровнем значимости 2%.\n",
    "\n",
    "Уровень значимости 5%.\n",
    "Мы хотели бы различать 3%-ное\n",
    "изменение целевой метрики\n",
    "с вероятностью 85%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jSlx0h6Axakf",
   "metadata": {
    "id": "jSlx0h6Axakf"
   },
   "source": [
    "**alpha** = 0.05 **lift** = 0.03 **beta** = 0.85"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4691844",
   "metadata": {
    "id": "e4691844"
   },
   "source": [
    "### 1. Необходимо выбрать метрику и привести аргументацию."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "Q3Zae2tSjQFW",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 73
    },
    "id": "Q3Zae2tSjQFW",
    "outputId": "0654aa37-9522-4f3f-bf49-3954db67697e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-0ed800e1-0009-48a1-b4f8-ffd680ea4942\" name=\"files[]\" multiple disabled\n",
       "        style=\"border:none\" />\n",
       "     <output id=\"result-0ed800e1-0009-48a1-b4f8-ffd680ea4942\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script>// Copyright 2017 Google LLC\n",
       "//\n",
       "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
       "// you may not use this file except in compliance with the License.\n",
       "// You may obtain a copy of the License at\n",
       "//\n",
       "//      http://www.apache.org/licenses/LICENSE-2.0\n",
       "//\n",
       "// Unless required by applicable law or agreed to in writing, software\n",
       "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
       "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
       "// See the License for the specific language governing permissions and\n",
       "// limitations under the License.\n",
       "\n",
       "/**\n",
       " * @fileoverview Helpers for google.colab Python module.\n",
       " */\n",
       "(function(scope) {\n",
       "function span(text, styleAttributes = {}) {\n",
       "  const element = document.createElement('span');\n",
       "  element.textContent = text;\n",
       "  for (const key of Object.keys(styleAttributes)) {\n",
       "    element.style[key] = styleAttributes[key];\n",
       "  }\n",
       "  return element;\n",
       "}\n",
       "\n",
       "// Max number of bytes which will be uploaded at a time.\n",
       "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
       "\n",
       "function _uploadFiles(inputId, outputId) {\n",
       "  const steps = uploadFilesStep(inputId, outputId);\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  // Cache steps on the outputElement to make it available for the next call\n",
       "  // to uploadFilesContinue from Python.\n",
       "  outputElement.steps = steps;\n",
       "\n",
       "  return _uploadFilesContinue(outputId);\n",
       "}\n",
       "\n",
       "// This is roughly an async generator (not supported in the browser yet),\n",
       "// where there are multiple asynchronous steps and the Python side is going\n",
       "// to poll for completion of each step.\n",
       "// This uses a Promise to block the python side on completion of each step,\n",
       "// then passes the result of the previous step as the input to the next step.\n",
       "function _uploadFilesContinue(outputId) {\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  const steps = outputElement.steps;\n",
       "\n",
       "  const next = steps.next(outputElement.lastPromiseValue);\n",
       "  return Promise.resolve(next.value.promise).then((value) => {\n",
       "    // Cache the last promise value to make it available to the next\n",
       "    // step of the generator.\n",
       "    outputElement.lastPromiseValue = value;\n",
       "    return next.value.response;\n",
       "  });\n",
       "}\n",
       "\n",
       "/**\n",
       " * Generator function which is called between each async step of the upload\n",
       " * process.\n",
       " * @param {string} inputId Element ID of the input file picker element.\n",
       " * @param {string} outputId Element ID of the output display.\n",
       " * @return {!Iterable<!Object>} Iterable of next steps.\n",
       " */\n",
       "function* uploadFilesStep(inputId, outputId) {\n",
       "  const inputElement = document.getElementById(inputId);\n",
       "  inputElement.disabled = false;\n",
       "\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  outputElement.innerHTML = '';\n",
       "\n",
       "  const pickedPromise = new Promise((resolve) => {\n",
       "    inputElement.addEventListener('change', (e) => {\n",
       "      resolve(e.target.files);\n",
       "    });\n",
       "  });\n",
       "\n",
       "  const cancel = document.createElement('button');\n",
       "  inputElement.parentElement.appendChild(cancel);\n",
       "  cancel.textContent = 'Cancel upload';\n",
       "  const cancelPromise = new Promise((resolve) => {\n",
       "    cancel.onclick = () => {\n",
       "      resolve(null);\n",
       "    };\n",
       "  });\n",
       "\n",
       "  // Wait for the user to pick the files.\n",
       "  const files = yield {\n",
       "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
       "    response: {\n",
       "      action: 'starting',\n",
       "    }\n",
       "  };\n",
       "\n",
       "  cancel.remove();\n",
       "\n",
       "  // Disable the input element since further picks are not allowed.\n",
       "  inputElement.disabled = true;\n",
       "\n",
       "  if (!files) {\n",
       "    return {\n",
       "      response: {\n",
       "        action: 'complete',\n",
       "      }\n",
       "    };\n",
       "  }\n",
       "\n",
       "  for (const file of files) {\n",
       "    const li = document.createElement('li');\n",
       "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
       "    li.append(span(\n",
       "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
       "        `last modified: ${\n",
       "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
       "                                    'n/a'} - `));\n",
       "    const percent = span('0% done');\n",
       "    li.appendChild(percent);\n",
       "\n",
       "    outputElement.appendChild(li);\n",
       "\n",
       "    const fileDataPromise = new Promise((resolve) => {\n",
       "      const reader = new FileReader();\n",
       "      reader.onload = (e) => {\n",
       "        resolve(e.target.result);\n",
       "      };\n",
       "      reader.readAsArrayBuffer(file);\n",
       "    });\n",
       "    // Wait for the data to be ready.\n",
       "    let fileData = yield {\n",
       "      promise: fileDataPromise,\n",
       "      response: {\n",
       "        action: 'continue',\n",
       "      }\n",
       "    };\n",
       "\n",
       "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
       "    let position = 0;\n",
       "    do {\n",
       "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
       "      const chunk = new Uint8Array(fileData, position, length);\n",
       "      position += length;\n",
       "\n",
       "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
       "      yield {\n",
       "        response: {\n",
       "          action: 'append',\n",
       "          file: file.name,\n",
       "          data: base64,\n",
       "        },\n",
       "      };\n",
       "\n",
       "      let percentDone = fileData.byteLength === 0 ?\n",
       "          100 :\n",
       "          Math.round((position / fileData.byteLength) * 100);\n",
       "      percent.textContent = `${percentDone}% done`;\n",
       "\n",
       "    } while (position < fileData.byteLength);\n",
       "  }\n",
       "\n",
       "  // All done.\n",
       "  yield {\n",
       "    response: {\n",
       "      action: 'complete',\n",
       "    }\n",
       "  };\n",
       "}\n",
       "\n",
       "scope.google = scope.google || {};\n",
       "scope.google.colab = scope.google.colab || {};\n",
       "scope.google.colab._files = {\n",
       "  _uploadFiles,\n",
       "  _uploadFilesContinue,\n",
       "};\n",
       "})(self);\n",
       "</script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving hist_credit_card.csv to hist_credit_card.csv\n"
     ]
    }
   ],
   "source": [
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "I-Y1zi35vyM3",
   "metadata": {
    "id": "I-Y1zi35vyM3"
   },
   "outputs": [],
   "source": [
    "hist_credit_card = pd.read_csv(io.BytesIO(uploaded['hist_credit_card.csv']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "OPRh45_dvyPC",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "OPRh45_dvyPC",
    "outputId": "12649ce4-d2ff-41f6-a669-d4d19cebc812"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-a35340ff-33d8-4f30-afce-391e8ea7b855\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Возраст</th>\n",
       "      <th>Доход клиента</th>\n",
       "      <th>Вероятность банкротства</th>\n",
       "      <th>Флаг утилизации счёта</th>\n",
       "      <th>Расходы</th>\n",
       "      <th>PV КК</th>\n",
       "      <th>PV услуги</th>\n",
       "      <th>NPV</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>19</td>\n",
       "      <td>21620.835463</td>\n",
       "      <td>0.138061</td>\n",
       "      <td>0</td>\n",
       "      <td>102</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>24897.990863</td>\n",
       "      <td>0.035508</td>\n",
       "      <td>1</td>\n",
       "      <td>409</td>\n",
       "      <td>11686</td>\n",
       "      <td>1754</td>\n",
       "      <td>13031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>50</td>\n",
       "      <td>23989.526947</td>\n",
       "      <td>0.098793</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>18</td>\n",
       "      <td>38442.409756</td>\n",
       "      <td>0.365661</td>\n",
       "      <td>1</td>\n",
       "      <td>788</td>\n",
       "      <td>13738</td>\n",
       "      <td>1578</td>\n",
       "      <td>14528</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "      <td>21291.521612</td>\n",
       "      <td>0.036909</td>\n",
       "      <td>1</td>\n",
       "      <td>1048</td>\n",
       "      <td>6594</td>\n",
       "      <td>2213</td>\n",
       "      <td>7759</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a35340ff-33d8-4f30-afce-391e8ea7b855')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-a35340ff-33d8-4f30-afce-391e8ea7b855 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-a35340ff-33d8-4f30-afce-391e8ea7b855');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   ID  Возраст  Доход клиента  Вероятность банкротства  Флаг утилизации счёта  \\\n",
       "0   0       19   21620.835463                 0.138061                      0   \n",
       "1   1       27   24897.990863                 0.035508                      1   \n",
       "2   2       50   23989.526947                 0.098793                      0   \n",
       "3   3       18   38442.409756                 0.365661                      1   \n",
       "4   4       24   21291.521612                 0.036909                      1   \n",
       "\n",
       "   Расходы  PV КК  PV услуги    NPV  \n",
       "0      102      0          0   -102  \n",
       "1      409  11686       1754  13031  \n",
       "2       16      0          0    -16  \n",
       "3      788  13738       1578  14528  \n",
       "4     1048   6594       2213   7759  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist_credit_card.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jbtep1FCGcvY",
   "metadata": {
    "id": "jbtep1FCGcvY"
   },
   "source": [
    "Мы хотим увеличивать доходность КК + услуги. Её увеличение происходит при утилизации счёт, иначе она равна нулю. В качестве *целевой метрики* возьмём **Флаг утилизации счёта**, так как она учитывает изменения этих показателей. Мы будем проверять, вызывает ли более агрессивное продвижение услуги негатив **на уровне клиента**. Поэтому нас интересует не конкретная сумма, которую мы получим с клиента (NPV/PV), а сам факт утилизации.\n",
    "\n",
    "Таким образом, мы ожидаем возрастания конверсии в утилизацию счёта на тесте.\n",
    "\n",
    "**Нулевая гипотеза (Н0):**\n",
    "\n",
    "Введение более агрессивной продажи услуги не повлияло на конверсию в утилизацию счёта. Различия между тестовой и контрольной группой случайны и статистически не значимы."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ttHcxx-gvyVB",
   "metadata": {
    "id": "ttHcxx-gvyVB"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "e36b594c",
   "metadata": {
    "id": "e36b594c"
   },
   "source": [
    "### 2. Альтернатива в критерии."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "E65axJwMIpNi",
   "metadata": {
    "id": "E65axJwMIpNi"
   },
   "source": [
    "**Альтернативная гипотеза (Н1):**\n",
    "\n",
    "Введение более агрессивной продажи услуги подняло долю утилизации счёта на 3%. Конверсия в утилизацию у представителей тестовой группы больше, чем у контрольной, эти различия - результат изменений. \n",
    "\n",
    "Тестируем изменение в большую сторону, значит, проверяем правостороннюю альтернативу."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "pBmniztEjRQP",
   "metadata": {
    "id": "pBmniztEjRQP"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "692b97f3",
   "metadata": {
    "id": "692b97f3"
   },
   "source": [
    "### 3. Нужно выбрать параметр(-ы), влияющий(-ие) на целевую метрику. Привести аргументацию."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fp6vqXwHJ-m4",
   "metadata": {
    "id": "fp6vqXwHJ-m4"
   },
   "source": [
    "Параметры, влияющие на целевую метрику необходимо выбрать из соц-дем характеристик и присвоенного скоринга. Корреляцию с целевой метрикой проверим при помощи матрицы корреляции."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "UTolzZEIQr4p",
   "metadata": {
    "id": "UTolzZEIQr4p"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4bYv6W9ljUEg",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 612
    },
    "id": "4bYv6W9ljUEg",
    "outputId": "db637626-0c75-41cb-b694-184e02176869"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr8AAAJTCAYAAAABn34XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACbbklEQVR4nOzddVxUWRsH8N/Q3WliYGBh19quGIsYa78CNrYCitiNidiu3V1rYqAYqKuiWICFCgaIAlLS8/6BXh1hTGBw5vfdz/2sc+fcM8+cYeCZM889VyQWi8UgIiIiIlIASrIOgIiIiIiooDD5JSIiIiKFweSXiIiIiBQGk18iIiIiUhhMfomIiIhIYTD5JSIiIiKFweSXiIiIiBQGk18iIiIiUhhMfomIiIhIYTD5JSIiIiKFweSXiIiIiH7Z+fPnYW9vjyJFikAkEuHgwYPfPMbf3x81atSAuro6ypYti40bN+Z7nEx+iYiIiOiXJSUloVq1ali+fPl3tX/y5AnatWuHZs2aISgoCKNGjUL//v1x4sSJfI1TJBaLxfn6CERERESkUEQiEQ4cOIAOHTpIbePh4YGjR4/i7t27wr7u3bsjLi4Ovr6++RYbZ36JiIiIKFepqamIj4+X2FJTU/Ok78uXL6Nly5YS++zs7HD58uU86V8alXztnUhBpL8Jk3UI9EGb6oNlHQJ9wC8WCxdlEee7CouTEfk3q/lRXv1d8lq2GdOmTZPYN2XKFEydOvWX+46MjIS5ubnEPnNzc8THx+P9+/fQ1NT85cfIDZNfIiIiIsqVp6cnXF1dJfapq6vLKJq8weSXiIiISN5kZeZJN+rq6vmW7FpYWCAqKkpiX1RUFPT09PJt1hdg8ktEREQkf8RZso7gm+rXr49jx45J7Dt16hTq16+fr4/LAiAiIiIi+mWJiYkICgpCUFAQgOylzIKCghAeHg4gu4TC0dFRaO/i4oKwsDCMHTsWoaGhWLFiBXbv3o3Ro0fna5yc+SUiIiKSN1kFP/N7/fp1NGvWTLj9sVbYyckJGzduxKtXr4REGABKlSqFo0ePYvTo0Vi8eDGKFSuGtWvXws7OLl/j5Dq/RHmAqz0UHlztofDgn5fChas9FB4FsdpD2st7edKPWpFKedJPYcJ3AhEREREpDJY9EBEREckbGZQ9/C6Y/BIRERHJm99gtQdZYdkDERERESkMzvwSERERyZs8usiFPGLyS0RERCRvWPYgFZNfIiIiInnDE96kYs0vERERESkMzvwSERERyRkxyx6kYvJLREREJG9Y9iAVyx6IiIiISGFw5peIiIhI3rDsQSomv0RERETyhuv8SsWyByIiIiJSGJz5JSIiIpI3LHuQiskvERERkbzhag9SseyBiIiIiBQGZ36JiIiI5A3LHqRi8ktEREQkb1j2IBWTXyIiIiI5IxZzqTNpWPNLRERERAqDM79ERERE8oY1v1Ix+SUiIiKSN6z5lYplD0RERESkMDjzS0RERCRvWPYgFZNfIiIiInmTxdUepGHZAxEREREpDM78EhEREckblj1IxeSXiIiISN5wtQepWPZARERERAqDM79ERERE8oZlD1Ix+SUiIiKSNyx7kIrJLxEREZG8YfIrFWt+iYiIiEhhcOaXfpqzszM2bdok3DYyMkLt2rUxb948VK1aVYaR0Y+4HnQHG7bvRXDoI0S/jcFir0lo0biBrMP6rbR3skfXQX/DyNQIj0PCsGzyCtwPui+1feN2jeDs7gSLYuZ48fQF1sxeh6tnr0m0cXJzRNseraGjr4N714KxePwSvHj6Uri/5/AeqNu8DspUKo2MtAx0qNw5x+OcjjiRY9/MobPhf+jcLzzbws/Z3RFte7SBjr4O7l67lz12T15+9RgHJ3t0dekivIZLJy2XeA1V1VUxeNIgNHNoClU1VVw7dx1Lxi9F7Ju4HH3pGehi9alVMLU0RXubjkiKTwIAGJkZwWXyQJSvWg5FrIrgwPqDWDF1VZ4+98LE3skeXQb9DSNTQ4SFhGH55BW4H/RAavtG7RrB2d0R5h/eF2tnr8e1L94Xjm690aZHG+joa+PetWAsGb8ULz+8L6rWq4oFe+bl2vewv0bgwa0HKFa6GEZ4DUdJ6xLQ1tXG26i3OPuvP7Ys2orMDPm6KIRYLF/PJy9x5pd+SevWrfHq1Su8evUKfn5+UFFRwV9//SXrsOgHvH+fgvJlS2OC2xBZh/JbamrfBC6TBmKLzza4tB2KsOAwzNkyCwbG+rm2t6lpgwnLPOG70xcubYYg4MQlTFs7BVblSwptug3uio59HLB4/FIMsx+JlPcpmLN1NlTVVYU2KqoqOH/0PA5vOfrV+Oa5LkCXGt2FLeDEpbx54oVU9yFd0bFPB/h4LsEw+xFISU7BnK1eEmP3pab2TeAyeRA2L9oKlzZD8Dg4DHO3zoaBsYHQZsgUF9T7sx6mDZqJ0X+7w8TcGFPXTMm1P/cFbggLeZJjv6qaKt69fYeti7fjcXDYLz/XwqyJfWMMmjQAW322YkjbYQgLDsPsr74vKmL8snHw3XkCg9sMxaUTlzF17WSJ90XXwV3QoY8DloxfghH2o5DyPgVeW2cJr21wYDC61eghsR3bfhyvnr3Cg1vZSXdGRgZO7zsNz17j0bdpf6yc9g/a9GgNR7fe+T8oBS0rK282OcTkl36Juro6LCwsYGFhAVtbW4wbNw4RERGIjo4GANy5cwfNmzeHpqYmjI2NMXDgQCQmJgrHjxs3DkWKFIGamhqKFi0KDw8PZH14s/n7+0MkEuHo0aOoWrUqNDQ0UK9ePdy9e1c4/u3bt+jRoweKFi0KLS0tVKlSBTt27JCIMSsrC/PmzUPZsmWhrq6OEiVKYNasWQAAkUgkdfP398/n0SscGtWvjREDndCySUNZh/Jb6jygE47t8MWJ3ScR/jAcPp5LkJqSitbd7HJt36lfB1zzv47d/+xF+KMIbFywGY/uPoKDk4NEm21Ld+DSyct4EvoEc0fNg7G5MRrafZqR3+y9BfvWHsCT0JxJ1ucS4xMRGx0rbOmp6XnzxAupTv06YuuS7bh08jLCQrLHzsTcGH/YSf/5/ntgZxzbcRwndp/Es4fh8Bm3OPs17J79GmrraqFN99ZYNf0fBF0KwsM7DzHPdSEq166EijUqSPRl3/svaOtrY/eqvTkeJ+p5FJZPWYlT+04jKSEpb594IdN5QCcc3+GLk7tPIfxhOBZ7LkVqSirspLwvOnx4X+z5Zy8iHkVg04f3RXun9kKbjv06YvvSHbh88gqehD7BvFHzJd4XGekZEj/r8bHxaNCqPk7uOSX0ERkeiZO7TyEs5Alev3iNK6eu4MzBs6hcp3L+DggVKkx+Kc8kJiZi69atKFu2LIyNjZGUlAQ7OzsYGhri2rVr2LNnD06fPo1hw4YJx7Rq1QpHjhzBo0ePsHbtWqxevRpbt26V6HfMmDFYuHAhrl27BlNTU9jb2yM9PfsPeEpKCmrWrImjR4/i7t27GDhwIHr37o2rV68Kx3t6emLOnDmYNGkSgoODsX37dpibmwOAMGv96tUrAMC+ffuE2w0a8Kt/+joVVRWUq2KNGxdvCPvEYjFuXLgJm5o2uR5jU6Mibly8KbHv2rlA2NSsCACwLGEBY3Nj3Ljwqc+khGSEBIXCpkbFH45xxMxh2HdrN5YdXoLW3Vr98PG/k6+OXc3cx054DS98ek2E1/DDeFtXKQdVNVUEftZvxOMIRD2Pgk2NT69zSesS6D2qF+aOnAexAi8zpaKqAusq1rh5UXJMb164iYpSXgebGhUl2gPA9XOBQnuLEhYwNjeSeJ2SE5IRGhSKilLeF/X/rAddQ12c2HVSaqxFrCxRq0lN3Lly57uf329DnJU3mxxizS/9kiNHjkBHRwcAkJSUBEtLSxw5cgRKSkrYvn07UlJSsHnzZmhrawMAli1bBnt7e8ydOxfm5uZo3ry50FdmZiY0NTWRmSlZpzRlyhT8+eefAIBNmzahWLFiOHDgALp27YqiRYvC3d1daDt8+HCcOHECu3fvRp06dZCQkIDFixdj2bJlcHJyAgCUKVMGf/zxBwDAwsJC4rGMjIxy7COSRt9ID8oqyoiNjpPYH/smFsXLFs/1GENTQ8S+iZXYF/cmFkamhh/uN/rQh2SfcdFxMDIz+qH4NizYhKCAIKS+T0XNxjUxYuZwaGhp4uCGf3+on9+FtLGLjY6F4Yfx/dKn11DyNfn8NTQyM0RaappQu/t5GyOz7H5V1VQxYbknVs9ai9cvo2FZ0jIvntJvSU/q+yLuG+8LyfZxb+KE98XH/8fleG3jYGiW+2vbursdAs8F4k3kmxz3LTrgDevKZaGmoYajW49h04LN3/HMfjNyWrKQF5j80i9p1qwZVq5cCQCIjY3FihUr0KZNG1y9ehUhISGoVq2akPgCQMOGDZGVlYX79+8Ls6+zZ8/GzJkz8f79ewwbNgyOjo4Sj1G/fn3h30ZGRihfvjxCQkIAZCfMs2fPxu7du/HixQukpaUhNTUVWlpaAICQkBCkpqaiRYsWefacU1NTkZqaKrFPKTUV6urqefYYRHlh2+Ltwr8f3XsMDS0NdHXpIjfJb4uOzTF6zkjh9niniTKLpf+4vgh/GIHT+/1kFgN9YmJhgppNamLW4Nm53j97yGxo6mihtE1pDJjQD38P6ow9uZSqkHxi8ku/RFtbG2XLlhVur127Fvr6+lizZs139+Hi4oJOnTohMDAQo0aNQqdOndCsWbPvOnb+/PlYvHgxfHx8UKVKFWhra2PUqFFIS0sDAGhqav7YE/oOXl5emDZtmsS+iWNGYPLYkVKOIHn1LiYemRmZMDQ1kNhvaGKYYybxo9joWBiaSM5UGZgYIuZD+9jomA99GCDmdcynNqYGeHzv8S/FG3ozFL1H9YKqmirS037/2t9LJy8j5GaocFtVLfvEpy/HztDUUOrYfXoNJV8TQxNDoY+Y17FQU1eDtp62xOxvdpvs1822oS1KVbDCyXbHs+8UZf/vwO292LZ0OzYt3PJrT/Y3Ei/1fWEg/Jx/Kft9Idne4LP2H/9vkOO1NcDjezlPHrTr1goJsQm4fOpKro8X/Sp7Njj8YTiUlZQwcu4I7Fu9XzjnRC7IaclCXmDNL+UpkUgEJSUlvH//HhUrVsStW7eQlPTpj0VAQACUlJRQvnx5YZ+RkREqVKiAXr164Y8//sC+ffsk+rxy5dMvr9jYWDx48AAVK1YU+nNwcMD//vc/VKtWDaVLl8aDB5+W0rG2toampib8/PJuNsbT0xPv3r2T2DxGuuRZ//T7yEjPwIM7D1GjYXVhn0gkQvU/bBEcGJzrMcE3QlC9oa3EvpqNaiA4MPvbjFfhkXgb9RbV//jUp5aOFiraVkDwjZBfirdMpTKIj0uQi8QXAN4nvcfLpy+F7dmDZ3gb9RY1chu7wNzH7uNrWP0PW2Gf8Bp+GO+Hdx4gPS1dot9ipYvBvJg5gm9kv85TB07HwFaDMdAue1s4ZhEAYFQnV/y78XBeP/VCLSM9Aw/vPITtZz/nIpEItn/YIkTK65Db+6JGoxpC+8jwSLyNipF4nbR0tFDBtgJCcnlftOryJ07tO/1dy5eJlERQUVGBSEn07Sf3O+FqD1Jx5pd+SWpqKiIjIwFkJ6bLli1DYmIi7O3tUadOHUyZMgVOTk6YOnUqoqOjMXz4cPTu3VsoeVixYgWaNGkCbW1tXLx4EadOncLixYslHmP69OkwNjaGubk5JkyYABMTE3To0AFAdnK7d+9eXLp0CYaGhvD29kZUVBRsbLJPQtHQ0ICHhwfGjh0LNTU1NGzYENHR0bh37x769ev3U89ZXV09R4lDelrOmrLfRXLye4Q//7QG6ouXUQh98Bj6erqwtDCTYWS/h31r9mOstzvu336A+0H30alfR2hoasB3d/ZJNh6LxuBN5Busm7sBALB/3UF475mPvwd2xn9+V9GsfROUq2qNReN8hD73rzuIXsN74MWTF4iMiISzuxPeRr2VWKbMrIgpdA10YVbEDErKSihjUxoA8OLpS6Qkp6Bey7owNDFEyM0QpKWmo2ajGugxrDv2/CPfX+3uX3cAvUb0xPMPY9fH3Rlvot7i4okAoc38nXNx0TcA/248BADYu3ofPBaNwYNbDxEaFIrO/TtBQ1MDJ3Zlr5OclJCM4zt9MXjyICTEJSApIRnDZwzBvev3EHIje+b51bNXEnHoG+kBAJ49CpeYLf74OmlqaULf2ABlbEojIz0Dzx6G59+gyMC+NfsxxtsdD28/ROhn74sTH94XYxa5423kW6z/8L44uO4gFuyZj84DO+Gq31U0bd8U5apaY/G4T38PDqw7gJ7De+DFk5cf3heOOd4XQPYsvGVJS/ju8M0RV/MOzZCRkYEnoU+RnpaOclWt0XdcH5w7fF7u1vkl6Zj80i/x9fWFpWX2iR26urqoUKEC9uzZg6ZNmwIATpw4gZEjR6J27drQ0tJC586d4e3tLRx/9OhRTJkyBQkJCShevDjGjx+Pvn37SjzGnDlzMHLkSDx8+BC2trY4fPgw1NTUAAATJ05EWFgY7OzsoKWlhYEDB6JDhw549+6dcPykSZOgoqKCyZMn4+XLl7C0tISLC2dqP7ob+hB9h3sIt+ctXQ0AcGjTErMmuskqrN+G/+Fz0DfSh7ObY/bX68Fh8Ow9QTgxx6yoKbI++/oxODAYs4fPQZ8xTug71hkvnr7ElP7T8PT+M6HNrpW7oaGlgdFzRkJHL/tCDeN6T5BYpszJ3RF2XT6t3vDPiezae7cuY3Drym1kZmTCwckeg6cMgkgkwounL7Fq+j84tv14Po+IbO1ckT12rnNHQUdPB3eu3YXn/8ZLjF2RkpbQN/q03qz/4XPQN9aHs/un13Bc7wkSJ2CtmLYK4iwxpqyeBFU1NVw/dx2Lxy/94fhWn/x0UYvy1cqhZcfmiIyIRK/6jl856vdz7vB56Bvpw9GtNwxNDREWHIYJvSd+9r4wg1gsFtoHB4bAa/hcOI9xQp+xznj59CWm9p8u8b7YvXIPNLQ0MGrOCOF9Mb73xBzL97Xubod71+4h4vHzHHFlZmai2+CuKFq6KEQiEaKev8ahjYexb+3+/BkIWWLZg1Qi8ec/fUSFiL+/P5o1a4bY2FgYGBjIOpyvSn8j3wvW/07aVB8s6xDoA/55KVyURax0LCxORuSclc5r748vyZN+NNuMyJN+ChPO/BIRERHJGzmt180L/BhIRERERAqDM79UaDVt2pRfmxIREf0M1vxKxeSXiIiISN6w7EEqlj0QERERkcJg8ktEREQkb8RZebP9hOXLl8PKygoaGhqoW7curl69+tX2Pj4+KF++PDQ1NVG8eHGMHj0aKSkpP/XY34NlD0RERETyRkZlD7t27YKrqytWrVqFunXrwsfHB3Z2drh//z7MzHJeOGn79u0YN24c1q9fjwYNGuDBgwdwdnaGSCSSuC5AXuLMLxERERHlCW9vbwwYMAB9+vSBjY0NVq1aBS0tLaxfvz7X9pcuXULDhg3Rs2dPWFlZoVWrVujRo8c3Z4t/BZNfIiIiInmTR2UPqampiI+Pl9hSU1Nzfci0tDQEBgaiZcuWwj4lJSW0bNkSly9fzvWYBg0aIDAwUEh2w8LCcOzYMbRt2zbvx+RjTPnWMxERERHJRlZWnmxeXl7Q19eX2Ly8vHJ9yDdv3iAzMxPm5uYS+83NzREZGZnrMT179sT06dPxxx9/QFVVFWXKlEHTpk0xfvz4PB+Sj5j8EhEREVGuPD098e7dO4nN09Mzz/r39/fH7NmzsWLFCty4cQP79+/H0aNHMWPGjDx7jC/xhDciIiIieZNHJ7ypq6tDXV39u9qamJhAWVkZUVFREvujoqJgYWGR6zGTJk1C79690b9/fwBAlSpVkJSUhIEDB2LChAlQUsr7eVrO/BIRERHJG7E4b7YfoKamhpo1a8LPz0/Yl5WVBT8/P9SvXz/XY5KTk3MkuMrKyh+eQv5c5ZUzv0RERETyRkZLnbm6usLJyQm1atVCnTp14OPjg6SkJPTp0wcA4OjoiKJFiwp1w/b29vD29kb16tVRt25dPHr0CJMmTYK9vb2QBOc1Jr9ERERElCe6deuG6OhoTJ48GZGRkbC1tYWvr69wElx4eLjETO/EiRMhEokwceJEvHjxAqamprC3t8esWbPyLUaROL/mlIkUSPqbMFmHQB+0qT5Y1iHQB/zzUrgoi1jpWFicjPDN98d4v21SnvSj2Sv/TjyTFc78EhEREcmbn7w0sSLgx0AiIiIiUhic+SUiIiKSNzI64e13wOSXiIiISN6w5l4qlj0QERERkcLgzC8RERGRvGHZg1RMfomIiIjkDZNfqVj2QEREREQKgzO/RERERPKG6/xKxeSXiIiISM6Is7jagzRMfomIiIjkDWt+pWLNLxEREREpDM78EhEREckb1vxKxeSXiIiISN6w5lcqlj0QERERkcLgzC8RERGRvOEJb1Ix+SUiIiKSN0x+pWLZAxEREREpDM78EhEREckbMU94k4bJLxEREZG8YdmDVEx+iYiIiOQNlzqTijW/RERERKQwOPNLREREJG94hTepmPwSERERyRuWPUjFsgciIiIiUhic+SXKA22qD5Z1CPTB8ZsrZR0CfcD3ReGSBc4EKhIxV3uQiskvERERkbxh2YNULHsgIiIiIoXBmV8iIiIiecPVHqRi8ktEREQkb1j2IBXLHoiIiIhIYXDml4iIiEjecLUHqZj8EhEREckblj1IxeSXiIiISN7whDepWPNLRERERAqDM79ERERE8oZlD1Ix+SUiIiKSM7y8sXQseyAiIiIihcGZXyIiIiJ5w7IHqZj8EhEREckbJr9SseyBiIiIiBQGZ36JiIiI5A3X+ZWKyS8RERGRvGHZg1QseyAiIiIihcGZXyIiIiI5I+bMr1RMfomIiIjkDZNfqZj8EhEREckbXuFNKtb8EhEREZHC4MwvERERkbxh2YNUTH6JiIiI5A2TX6lY9kBEREREeWb58uWwsrKChoYG6tati6tXr361fVxcHIYOHQpLS0uoq6ujXLlyOHbsWL7Fx5lfIiIiIjkjFstm5nfXrl1wdXXFqlWrULduXfj4+MDOzg7379+HmZlZjvZpaWn4888/YWZmhr1796Jo0aJ49uwZDAwM8i1GJr9ERERE8kZGZQ/e3t4YMGAA+vTpAwBYtWoVjh49ivXr12PcuHE52q9fvx4xMTG4dOkSVFVVAQBWVlb5GiPLHoiIiIgoV6mpqYiPj5fYUlNTc22blpaGwMBAtGzZUtinpKSEli1b4vLly7kec+jQIdSvXx9Dhw6Fubk5KleujNmzZyMzMzNfng/A5JeIiIhI/mSJ82Tz8vKCvr6+xObl5ZXrQ7558waZmZkwNzeX2G9ubo7IyMhcjwkLC8PevXuRmZmJY8eOYdKkSVi4cCFmzpyZ50PyEcseiIiIiORMXl3e2NPTE66urhL71NXV86RvAMjKyoKZmRlWr14NZWVl1KxZEy9evMD8+fMxZcqUPHuczzH5JSIiIqJcqaurf3eya2JiAmVlZURFRUnsj4qKgoWFRa7HWFpaQlVVFcrKysK+ihUrIjIyEmlpaVBTU/v54KVg2QMRERGRvMmjsocfoaamhpo1a8LPz+9TGFlZ8PPzQ/369XM9pmHDhnj06BGyPrsc84MHD2BpaZkviS/A5JeIiIhI/mTl0faDXF1dsWbNGmzatAkhISEYPHgwkpKShNUfHB0d4enpKbQfPHgwYmJiMHLkSDx48ABHjx7F7NmzMXTo0J984t/GsgciIiIiOZNXNb8/qlu3boiOjsbkyZMRGRkJW1tb+Pr6CifBhYeHQ0np09xr8eLFceLECYwePRpVq1ZF0aJFMXLkSHh4eORbjCKxrFZBJpIjLYvbyToE+uD4zZWyDoE+aFN9sKxDICqUTkecyPfHiOvVPE/6Mdh2Jk/6KUw480tEREQkb2Q08/s7YPJLREREJG9+ol5XUfCENyIiIiJSGJz5JSIiIpIzsjrh7XfA5JfoN9TeyR5dB/0NI1MjPA4Jw7LJK3A/6L7U9o3bNYKzuxMsipnjxdMXWDN7Ha6evSbRxsnNEW17tIaOvg7uXQvG4vFL8OLpS+H+nsN7oG7zOihTqTQy0jLQoXLnHI+T20kcM4fOhv+hc7/wbBXH9aA72LB9L4JDHyH6bQwWe01Ci8YNZB1Wofetn93cfOs9pKquCpdJA9GsfVOoqqni+rlALJ6wFHFv4oQ2ZkVMMXL2cFRrUA3vk1Jwau8prJ2zHlmZn75vbt6hGboN7oqipYogKT4J1/yvY/XMNYiPSxDadOrXEfa928GsqBnexcTjwrELWDtnPdJT0/NukAqALH4v6RroYtj0IajXsi7EWWJcOH4Ry6esREpyCgDAcfT/4OjaO8djv09OgX15BwBAyXIl4ezmCOsqZWFR3AIrpq7C/nUH8mJIZItlD1Kx7KEAxcXFQSQS5dgMDAxkHRr9RpraN4HLpIHY4rMNLm2HIiw4DHO2zIKBsX6u7W1q2mDCMk/47vSFS5shCDhxCdPWToFV+ZJCm26Du6JjHwcsHr8Uw+xHIuV9CuZsnQ1VdVWhjYqqCs4fPY/DW45+Nb55rgvQpUZ3YQs4cSlvnrgCeP8+BeXLlsYEtyGyDuW38T0/u1/6nvfQkCkuqN+yHqa7zIRrF3cYmxth6urJwv1KSkqYtWkGVFRVMbLDaMwbPR+tuvwJZ3cnoU2lWjbw8BmD4zt90b/FQMwYPAvlbctj9LxRQpvmHZqh/7i+2OKzDX2bDcDCMd5oYt8E/Tz65O1A5TNZ/V7yXOKBkuVKwqOnJyb2mYwqdavAde4o4f7d/+yV+H3UpUZ3PL3/DOePnhfaaGiq41X4K6ydsx5vo97m/eBQocPkVwb27duHV69e4dWrV/Dx8ZF1OPSb6TygE47t8MWJ3ScR/jAcPp5LkJqSitbdcl9urVO/Drjmfx27/9mL8EcR2LhgMx7dfQQHJweJNtuW7sClk5fxJPQJ5o6aB2NzYzS0+zTruNl7C/atPYAnoU++Gl9ifCJio2OF7XebvZKlRvVrY8RAJ7Rs0lDWofw2vudn90vfeg9p62qhdTc7rJz+D4Iu3cLDO48w380blWtXQsXqFQAANRvXQAnrEvAaORePg8Nwzf86Ni7YDAdHe6ioZn+palPTBlHPo3Bww7+IjIjC3Wv3cHTrUVSwLS/EYlPTBnev38OZg2cR9TwKgedv4Oy//qhQrXzOwAsxWfxeKlG2OOo0qw3vsYsQGnQfd6/dw/LJK9C0fRMYmxsBAFKSUyR+HxmaGMKqfEkc3/npW6r7tx5g9ay18D90Dulp8vP7SpwlzpNNHjH5LUAZGRkAAGNjY1hYWMDCwgL6+jk/FYeHh8PBwQE6OjrQ09ND165dhetki8VitGzZEnZ2dvi4RHNMTAyKFSuGyZM/zUqcO3cOderUgbq6OiwtLTFu3Djh8T/y9/f/4Vnopk2bYtSoUcLttWvXwsDAADdu3JBoZ2VllaPvgwcPSjxuXFyc0L53794SbQAgIiICXbt2hYGBAYyMjODg4ICnT58K9zs7O6NDhw4Sj7tx40bhOWzcuDHXmXaRSAQrKysAwOPHj+Hg4ABzc3Po6Oigdu3aOH369FfHQJZUVFVQroo1blz8NN5isRg3LtyETU2bXI+xqVERNy7elNh37VwgbGpWBABYlrCAsbkxblz41GdSQjJCgkJhU6PiD8c4YuYw7Lu1G8sOL0Hrbq1++Hii7/UzP7vf8x6yrmINVTVVifdNxOMIRD2PEt43NjVt8CT0qUQZxPVz16Gtpw2rctmzl8GBwTC1NEWdZrUBAAYmBmjUrhGunvn01X5wYDDKVbFG+Q8JsWUJC9RpVhv/ffH1f2Emq99LNjUrIiEuAQ9uPxTaBF64AXGWGBU+fEj5UtserRHxOAJ3r979uSf7O5HRFd5+B0x+C1BqaioAQF1dXWqbrKwsODg4ICYmBufOncOpU6cQFhaGbt26AQBEIhE2bdqEa9euYcmSJQAAFxcXFC1aVEh+X7x4gbZt26J27dq4desWVq5ciXXr1mHmzJm5Pub9+/d/ahZ69+7dGD16NA4dOoQaNWpI3CcWizF9+nRhhvtrAgMDcejQIYl96enpsLOzg66uLi5cuICAgADo6OigdevWSEtL+674unXrJjHDXqxYMeH2tWvZf1gSExPRtm1b+Pn54ebNm2jdujXs7e0RHh7+AyNRcPSN9KCsoozY6DiJ/bFvYmFoapjrMYamhoh9EyuxL+5NLIw+tDc0NfrQh2SfcdFxMDIz+qH4NizYhBmDZ8GjpycuHLuIETOHo0Mfh28fSPQTfuZn93veQ0ZmRkhLTUNSfNIXbeKExzQyNUTcF++rj31+7Ofe9WB4jZiLiSvGwzfsKPbe3IWkhCQsmbhMOObMwbPYuHAzfPYthG/YUWwJ2ITbV25jx7Kd3z8QMiar30uGpkaIeyt5f1ZmFuLjEmBkmvP1V1VXRfOOzSVmfUkx8YS3AhQTEwMA0NXVldrGz88Pd+7cwZMnT1C8eHEAwObNm1GpUiVcu3YNtWvXRtGiRfHPP//A0dERkZGROHbsGG7evAkVleyXc8WKFShevDiWLVsGkUiEChUq4OXLl/Dw8MDkyZOFywp+TMaLFi0KbW3tXGehpTl+/Dj69OmDPXv2oHHjxjnuT09Ph5GRESwsLL7Zl6urK8aMGYNJkyYJ+3bt2oWsrCysXbsWIpEIALBhwwYYGBjA398frVp9e0ZRU1MTmpqaAAB9fX0oKyvniKdatWqoVq2acHvGjBk4cOAADh06hGHDhuXab2pqqjB2H2WJs6Ak4mfJbYu3C/9+dO8xNLQ00NWlCw5u+FeGUZG8aN6hGUbPGSncnuA86SutZa+EdQkMmTYYW3224dq5QBibG2HghP4Y5TUCC8csAgBUq1cVPYd1x5IJyxAaFIoiVkUwdOpg9BrZU+L9RL/uj9YNoaWtiZN7T8k6lAIhltNZ27zAv9YF6MWLFwAAS0tLqW1CQkJQvHhxIfEFABsbGxgYGCAkJETY16VLF3Ts2BFz5szBggULYG1tLdFH/fr1haQRABo2bIjExEQ8f/5c2Pf27VuoqKhAS0vrh57H1atX0blzZ2hra6Nu3bq5tomPj4e2tvY3+zp48CDCwsLg5uYmsf/WrVt49OgRdHV1oaOjAx0dHRgZGSElJQWPHz8W2h05ckS4X0dHBy4uLj/0XBITE+Hu7o6KFSvCwMAAOjo6CAkJ+erMr5eXF/T19SW2p/FhP/S4P+tdTDwyMzJhaGogsd/QxBCx0bG5HvOxzu1zBiaGiPnQPjY65kMfkn0amBog5nXML8UbejMUZkVMoaom/eQjou91+dQVDGo9RNjexcQD+LGf3e95D8W8joGauhq09bS/aGMgvF9iomNh8MX76mOfH/vpMbQb7l27h93/7MWT0Ce4fi4QSyYsQ5vurYXZS+cxTji93w/Hd/riSehTBPhewrq5G9BjaDeJ3+GFmax+L8VGx8DAWPJ+JWUl6BnoIiY65+vfpntrXPH7T6JURa6x7EEqJr8FKDg4GKampjAy+rGvknOTnJyMwMBAKCsr4+HDh98+IBdhYWEoWbLkD/+CvXz5Mry9vVG1atVcZ0fj4+ORlJSEIkWKfLWf9PR0jB07FrNmzRJmaD9KTExEzZo1ERQUJLE9ePAAPXv2FNo1a9ZM4v7p06f/0HNxd3fHgQMHMHv2bFy4cAFBQUGoUqXKV0srPD098e7dO4nNSq/0Dz3uz8pIz8CDOw9Ro2F1YZ9IJEL1P2wRHBic6zHBN0JQvaGtxL6ajWogODD7w9Sr8Ei8jXqL6n986lNLRwsVbSsg+EYIfkWZSmUQH5cgVyeRkOy8T3qPl09fCtuzB89++Gf3e95DD+88RHpaukSbYqWLwbyYufC+CQ4MRqkKVhKrGdRsVANJ8Ul49jD7w7OGpoZwbsZHmR+WQfv4a1ddQx1ZX5xUlJX1sc3vkfzK6vdScGAIdA10YV2lrNCmekNbiJRECL0ZKtG3RXFz2DaoplAlD+KsvNnkEcseCpCfnx8aNPj6mp0VK1ZEREQEIiIihNnf4OBgxMXFwcbm04kDbm5uUFJSwvHjx9G2bVu0a9cOzZs3F/rYt28fxGKx8MszICAAurq6KFasmNDHuXPn0KhRox9+Hr1794aLiwvatGmDypUr48CBA+jYsaNw/7Vr1yASiWBra/vVflauXAkdHR307p1zDcYaNWpg165dMDMzg56entQ+tLW1Ubbsp198ZmZmP/RcAgIC4OzsLMSfmJgocVJdbtTV1XPUbRdkycO+Nfsx1tsd928/wP2g++jUryM0NDXgu/skAMBj0Ri8iXyDdXM3AAD2rzsI7z3z8ffAzvjP7yqatW+CclWtsWicj9Dn/nUH0Wt4D7x48gKREZFwdnfC26i3EsuUmRUxha6BLsyKmEFJWQllbLIT/hdPXyIlOQX1WtaFoYkhQm6GIC01HTUb1UCPYd2x55+9BTY2v7vk5PcIf/5pDdMXL6MQ+uAx9PV0YWnxYz/biuJ7fnbn7ZiDAN9L+HdT9rkF33oPJSUkw3fXCbhMHoj4uAQkJyZh2PShuHc9GCEfkqrA8zcQ/jAc4xaPxepZ62BkZgjnMc74d/Nh4cPe5dNX4Dp3FOx7/4Xr567DyMwIQ6a6IORmKN5GZc9MXjl9BZ0HdMKje48QejMURayKwtndCVdO/yckwb8DWfxeCn8Ugatnr8F17ij4jF8KFRVlDJ8xFP6Hzgnj+1HrbnaIeR2Da7mcSKiiqoKS1iWy/62mChMLY5SxKY33ySl4+Y31oun3xOS3ALx//x7bt2/H8ePHsXz5ckRGRgr3vXv3DmKxGJGRkTA1NUXLli1RpUoV9OrVCz4+PsjIyMCQIUPQpEkT1KpVCwBw9OhRrF+/HpcvX0aNGjUwZswYODk54fbt2zA0NMSQIUPg4+OD4cOHY9iwYbh//z6mTJkCV1dXKCkpIS0tDYcPH8aZM2ewb98+IZ6PsURHR8PU1FTq8/k4c12yZEnMnz8fgwcPRuPGjWFsbIyzZ89i6NChaNu27TcT0Xnz5uHw4cO5zm706tUL8+fPh4ODA6ZPn45ixYrh2bNn2L9/P8aOHSuRxP8Ka2tr7N+/H/b29hCJRJg0aVKh/4Pjf/gc9I304ezmCENTQzwODoNn7wnCV3lmRU2R9dnH9eDAYMwePgd9xjih71hnvHj6ElP6T8PT+8+ENrtW7oaGlgZGzxkJHT0d3L12D+N6T5BYpszJ3RF2XT7VWv9zYiUAwK3LGNy6chuZGZlwcLLH4CmDIBKJ8OLpS6ya/g+ObT+ezyMiP+6GPkTf4R7C7XlLVwMAHNq0xKyJbtIOU2jf87NbpKQl9I0+fYj+1nsIAFZMW4WsrCxMWT3pw0UurmPJhE8nqmVlZWGC82SMnD0cS/5dhJTkFJzcexobF2wS2pzccwpa2ppwcGqPQZMGIDE+CUEBQVjjtU5os3XJdojFYvQZ4wwTC2O8e/sOl09fwfp5G/NnwPKJrH4veY2Yi+EzhmL+jjnCRS6WTV4hEZtIJEKrLq1wYs+pXH+/G5sbC7/PAKCrSxd0demCW5dvwa3r2LwaooJXuP+UyZRI/OV3MpTnNm7ciD59vr1g+ZMnT2BlZYXw8HAMHz4cfn5+UFJSQuvWrbF06VKYm5sjOjoaVapUwciRI+Hp6Qkgu3ygfv36KFOmDHbt2gUge1Z3zJgxuHXrFoyMjODk5ISZM2dCRUUF/v7+aNasmdQ4SpYsKXX2s2nTprC1tRVWhhCLxfjzzz9hZmaG7du3o1SpUmjZsiXmz58vsWyaSCTCgQMH0KFDB+Hx//rrLxw+fDjXNgAQGRkJDw8PHDt2DAkJCShatChatGiBBQsWQE9PD87OzoiLi5NYHm3jxo0YNWqUxDJqH/dPnTo1x/N6+vQp+vbtiytXrsDExAQeHh7Ys2ePxHP8Hi2L576WJRW84zdXfrsRFYg21QfLOgSiQim3q2Hmteg/m+RJP6an5O8KnUx+C8DGjRuxceNG+Pv7S20jEomE5De/+fv7Y+rUqbnGExcXB1tb229+9U+SmPwWHkx+Cw8mv0S5Y/IrWyx7KACamprfPMnN3NwcysrKBRKPmpqa1HiUlJS+WvJAREREhZ+8nqyWFzjzS5QHOPNbeHDmt/DgzC9R7gpi5jeqWd7M/Jqflb+ZXy51RkREREQKg2UPRERERPJG/HusEy0LTH6JiIiI5AxrfqVj2QMRERERKQzO/BIRERHJGXEWyx6kYfJLREREJGdY9iAdk18iIiIiOSPmCW9SseaXiIiIiBQGZ36JiIiI5AzLHqRj8ktEREQkZ3jCm3QseyAiIiIihcGZXyIiIiI5IxbLOoLCi8kvERERkZxh2YN0LHsgIiIiIoXBmV8iIiIiOcOZX+mY/BIRERHJGdb8SseyByIiIiJSGJz5JSIiIpIzLHuQjskvERERkZwRi5n8SsPkl4iIiEjO8PLG0rHml4iIiIgUBmd+iYiIiORMFssepGLyS0RERCRnWPMrHcseiIiIiEhhcOaXiIiISM5wqTPpmPwSERERyRle4U06lj0QERERkcLgzC8RERGRnGHZg3RMfomIiIjkDJc6k45lD0RERESkMDjzS0RERCRnuM6vdJz5JSIiIpIzYnHebD9j+fLlsLKygoaGBurWrYurV69+13E7d+6ESCRChw4dfu6BvxOTXyIiIiI5kyUW5cn2o3bt2gVXV1dMmTIFN27cQLVq1WBnZ4fXr19/9binT5/C3d0djRo1+tmn/N2Y/BIRERFRnvD29saAAQPQp08f2NjYYNWqVdDS0sL69eulHpOZmYlevXph2rRpKF26dL7HyOSXiIiISM6IxaI82X5EWloaAgMD0bJlS2GfkpISWrZsicuXL0s9bvr06TAzM0O/fv1++vn+CJ7wRkRERCRn8uoKb6mpqUhNTZXYp66uDnV19Rxt37x5g8zMTJibm0vsNzc3R2hoaK79X7x4EevWrUNQUFDeBPwdOPNLRERERLny8vKCvr6+xObl5ZUnfSckJKB3795Ys2YNTExM8qTP78GZXyIiIiI5k1cXufD09ISrq6vEvtxmfQHAxMQEysrKiIqKktgfFRUFCwuLHO0fP36Mp0+fwt7eXtiXlZUFAFBRUcH9+/dRpkyZX30KOTD5JcoD4rz6fol+WZvqg2UdAn1w/OZKWYdAn9Eskv9n0VPhkVfr/EorcciNmpoaatasCT8/P2G5sqysLPj5+WHYsGE52leoUAF37tyR2Ddx4kQkJCRg8eLFKF68+C/Hnxsmv0RERESUJ1xdXeHk5IRatWqhTp068PHxQVJSEvr06QMAcHR0RNGiReHl5QUNDQ1UrlxZ4ngDAwMAyLE/LzH5JSIiIpIzeVX28KO6deuG6OhoTJ48GZGRkbC1tYWvr69wElx4eDiUlGR7yplIzO9riX5Zi2KtZB0CfSAS8ZKehQXLHgoXlj0UHhlpL/L9Ma4U6ZQn/dR7uT9P+ilMuNoDERERESkMlj0QERERyRlZlT38Dpj8EhEREcmZvFrtQR4x+SUiIiKSM1myDqAQY80vERERESkMzvwSERERyRkxWPYgDZNfIiIiIjmTxYVspWLZAxEREREpDM78EhEREcmZLJY9SMXkl4iIiEjOsOZXOpY9EBEREZHC4MwvERERkZzhOr/SMfklIiIikjMse5COZQ9EREREpDA480tEREQkZ1j2IB2TXyIiIiI5w+RXOia/RERERHKGNb/SseaXiIiIiBQGZ36JiIiI5EwWJ36lYvJLREREJGd4eWPpWPZARERERAqDM79EREREckYs6wAKMSa/RERERHKGS51Jx7IHIiIiIlIYnPklIiIikjNZIp7wJg2TXyIiIiI5w5pf6Vj2QEREREQKgzO/RERERHKGJ7xJx+SXiIiISM7wCm/SMfklIiIikjO8wpt0rPklIiIiIoXBmV8iIiIiOcPVHqRj8ktEREQkZ1jzKx3LHoiIiIhIYXDml4iIiEjOcKkz6Zj8yon09HSoqqrKOgwiIiIqBFjzKx3LHn5TBw4cQLt27WBlZQUdHR00atRI1iGRDDi7O2J34A4ce3QY83bMQdFSRb55jIOTPbZd3ozjj45g2eElKG9bXuJ+VXVVjJg5DAfu7MWR+/9iyupJMDQxyLUvPQNd7Ly2DX7PT0JbT1vYb2RmhPHLxmHT+fU4Fe6LIVNdful5FjZObo7YdX07jj48hHnb56Co1bfHvb2TPbZe2oRjDw9j6aHFuY778JlDsf/2HhwOPYgp/0yCwRfjblbEFLM2TseRB/9iz81dGDihP5SUJX+NN+/QDP+cWIkjD/7Fruvb4b7AFXoGuhJtOvXriA3+a3H04SFs/28rBk8ZBFV1fnj+3PWgOxg6dgqate+Fyg3bwO/8JVmHJLemTnFHxLMbSHj3CCeO70TZsqW+2t5j7DBcvnQUsW/v4+XzW9i3dx3KlSsj3G9oaACfRTNw7+55JLx7hLBHV7HIezr09HS/0ispkh9Kfp2dnSESiYTN2NgYrVu3xu3bt/MrPsqFl5cXBgwYgL/++gtHjx5FUFAQjh07JuuwqIB1H9IVHft0gI/nEgyzH4GU5BTM2er11SSmqX0TuEwehM2LtsKlzRA8Dg7D3K2zYWBsILQZMsUF9f6sh2mDZmL03+4wMTfG1DVTcu3PfYEbwkKe5NivqqaKd2/fYevi7XgcHPbLz7Uw6Ta4Kzr2ccDi8UsxzH4kUt6nYM7W2d8e90kDscVnG1zaDkVYcBjmbJkFA2N9oc2QKS6o37IeprvMhGsXdxibG2Hq6snC/UpKSpi1aQZUVFUxssNozBs9H626/AlndyehTaVaNvDwGYPjO33Rv8VAzBg8C+Vty2P0vFFCm+YdmqH/uL7Y4rMNfZsNwMIx3mhi3wT9PPrk7UD95t6/T0H5sqUxwW2IrEORa2Pch2DY0L4YMmwcGvxhj6TkZBw7sg3q6upSj2ncqB5WrtyEho3s0bptD6iqqOL40e3Q0tIEABQpYo4iRczh4TED1aq3QL/+o2Fn1wxrVi8sqKdVKGSJ8maTRz8889u6dWu8evUKr169gp+fH1RUVPDXX3/lR2yUi7CwMMyePRvnzp3D4MGDUalSJZQtWxZGRkayDo0KWKd+HbF1yXZcOnkZYSFPMHfUPJiYG+MPu4ZSj/l7YGcc23EcJ3afxLOH4fAZtxipKalo3d0OAKCtq4U23Vtj1fR/EHQpCA/vPMQ814WoXLsSKtaoINGXfe+/oK2vjd2r9uZ4nKjnUVg+ZSVO7TuNpISkvH3iMtapXwdsW7oDl05expPQ7HE3NjdGQ7sGUo/pPKATju3wxYndJxH+MBw+nkuyx73bp3Fv3c0OK6f/g6BLt/DwziPMd/POHvfq2eNes3ENlLAuAa+Rc/E4OAzX/K9j44LNcHC0h4pqdgWbTU0bRD2PwsEN/yIyIgp3r93D0a1HUeGzWWabmja4e/0ezhw8i6jnUQg8fwNn//VHhWrlcwauwBrVr40RA53Qson09xP9uhHD+2O212IcPnwSd+6EwLnPSBQpYg4HBzupx7Sz/x82b9mN4OAHuH07GH37j0LJksVQs0ZVAMC9e/fRtdtAHDl6CmFhz3DWPwCTJs/FX+1aQllZuaCemsxl5dEmj344+VVXV4eFhQUsLCxga2uLcePGISIiAtHR0UKbiIgIdO3aFQYGBjAyMoKDgwOePn0q3O/s7IwOHTpg2rRpMDU1hZ6eHlxcXJCWlia0SU1NxYgRI2BmZgYNDQ388ccfuHbtmkQsR44cQbVq1aCpqSnMRnfo0AEA0LRpU4lZ6s+3qVOnAgCsrKzg4+Mj0efH2H4kjnv37uGvv/6Cnp4edHV10ahRIzx+/BhTp06VGkPTpk1zfbxvOXHiBMqUKYNZs2bB1NQUurq66NSpE54/fy60efz4MRwcHGBubg4dHR3Url0bp0+flujne55706ZNMWrUKOH22rVrYWBggBs3bgAA/P39IRKJcPToUVStWhUaGhqoV68e7t69K9Hvvn37UKlSJairq8PKygoLFy6UeIxvvU6pqanw8PBA8eLFoa6ujrJly2LdunV4+vSp1GNFIhGePn0qxCgSiaCkpAQzMzP069cPKSkpQgweHh4oV64ctLS0ULp0aUyaNAnp6enf/ZrIgmUJCxibG+PGhRvCvqSEZIQEhcKmZsVcj1FRVUG5Kta4ceGmsE8sFuPGhZuwqZF9jHWVclBVU0XgZ/1GPI5A1PMo2NSwEfaVtC6B3qN6Ye7IeRCL5fXXY05fHfca3xj3i5+OEca9ZvaYWlexhqqaKm5c/PTaCOP+4fW0qWmDJ6FPEfcmTmhz/dx1aOtpw6pcSQBAcGAwTC1NUadZbQCAgYkBGrVrhKtnPv3OCg4MRrkq1kLZhWUJC9RpVhv/nZX8vUaU30qVKgFLS3P4nbko7IuPT8DVqzdRr27N7+5HX18PABATGye9jZ4u4uMTkZmZ+dPxkvz4pZrfxMREbN26FWXLloWxsTGA7BOv7OzsoKuriwsXLiAgIAA6Ojpo3bq1RHLr5+eHkJAQ+Pv7Y8eOHdi/fz+mTZsm3D927Fjs27cPmzZtwo0bN1C2bFnY2dkhJiYGABAXF4du3bqhadOmCA4OxqtXr9C1a1fh+P379wsz1PXr14ebm5tw293d/buf47fiePHiBRo3bgx1dXWcOXMGgYGB6Nu3LzIyMuDu7i48ppubG+rXry/c3r9//0+NeXR0NG7duoWIiAgcP34cZ8+eRVRUFDp06ACxWCy8Lm3btoWfnx9u3ryJ1q1bw97eHuHh4T/1mACwe/dujB49GocOHUKNGjUk7hszZgwWLlyIa9euwdTUFPb29kLyGBgYiK5du6J79+64c+cOpk6dikmTJmHjxo0Avu91cnR0xI4dO7BkyRKEhITgn3/+gY6ODooXLy60vXr1KgDg6tWrwr7ixYsLMd6/fx8vXrzA1q1bsWvXLmzYsEG4T1dXFxs3bkRwcDAWL16MNWvWYNGiRT89VgXB0DR7pj/2s0QIAGKjY2FoapjrMfpGelBWUUZsdKzkMW9iYWSW3Z+RmSHSUtOQFJ+US5vsflXVVDFhuSdWz1qL1y+joUikjXtcdJwwhl/6NO6Sx8S++fRaGZkZSRn3OOExjUwNEffmi9fuQ58f+7l3PRheI+Zi4orx8A07ir03dyEpIQlLJi4Tjjlz8Cw2LtwMn30L4Rt2FFsCNuH2ldvYsWzn9w8EUR6wMDcDAERFSf4eiXr9BhYWZt/Vh0gkgveCaQgIuIp79+7n2sbY2BATxo/C2nXbfi3g3wxnfqX74dUejhw5Ah0dHQBAUlISLC0tceTIESgpZefRu3btQlZWFtauXQuRKLtYZMOGDTAwMIC/vz9atWoFAFBTU8P69euhpaWFSpUqYfr06RgzZgxmzJiB9+/fY+XKldi4cSPatGkDAFizZg1OnTqFdevWYcyYMXjw4AGSk5Ph4eGBIkWyTzbR1NREamoqAEiUAaipqUFHRwcWFhY/9FyTkpK+Gcfy5cuhr6+PnTt3CqstlCtXTujj41jp6OhATU3th2P4UlZWFpSVlbF9+3Yhudu+fTvKlCkDPz8/tGzZEtWqVUO1atWEY2bMmIEDBw7g0KFDGDZs2A8/5vHjx9GnTx/s2bMHjRs3znH/lClT8OeffwIANm3ahGLFiuHAgQPo2rUrvL290aJFC0yaNAlA9tgEBwdj/vz5cHZ2/ubr9ODBA+zevRunTp1Cy5YtAQClS5cW7v/Y9uNMrqmpaa5jbGZmBgMDAyQlJUFNTQ36+p9qLSdOnCj828rKCu7u7ti5cyfGjh2b63ikpqYKP2cfZYmzoCTKv/NHW3RsjtFzRgq3xztN/Err/NV/XF+EP4zA6f1+MouhoDTv0Exi3Cc4T5JhNN9WwroEhkwbjK0+23DtXCCMzY0wcEJ/jPIagYVjsj/QVatXFT2HdceSCcsQGhSKIlZFMHTqYPQa2RPbFm+X8TMgedajR0esXD5XuN3ewfGX+1y6ZDYqVSqPJs065nq/rq4ODv+7GSEhDzBtumLV/IrltF43L/xw8tusWTOsXLkSABAbG4sVK1agTZs2uHr1KkqWLIlbt27h0aNH0NWVPKsyJSUFjx8/Fm5Xq1YNWlpawu369esjMTERERERePfuHdLT09Gw4adaK1VVVdSpUwchISEAgOLFi0NFRQU7duzA6NGjheT7R3l4eEgkP6mpqWjXrh2A7PKBb8URFBSERo0a/dIyYx8/UKiqqqJEiRIYOXIk+vbtK7V98eLFJWY1S5YsiWLFiiE4OBgtW7ZEYmIipk6diqNHj+LVq1fIyMjA+/fvc8z8fu25f3T16lWsXr0aOjo6qFu3bq7x1K9fX/i3kZERypcvL4xPSEgIHBwcJNo3bNgQPj4+yMzM/Gb9VVBQEJSVldGkSZOvtvuWYsWKQSwWIzk5Gb169UKPHj2E+3bt2oUlS5bg8ePHSExMREZGBvT09KT25eXlJfEtBQBY6ZZGab0yUo74dZdOXkbIzVDhtqpa9s+boYkBYl7HCPsNTQ3x+N7jHMcDwLuYeGRmZOaYGTY0MRT6iHkdCzV1NWjraUvMQma3yZ51tG1oi1IVrHCy3fHsOz/8gj1wey+2Ld2OTQu3/NqTLUQun7qC0KBPs0nSxt3A1OA7xt1AYr+hiaEwCx/zOkbKuBsgNvrDaxMdm2OFiI99fuynx9BuuHftHnb/k12H/ST0CVKSU+Cz3xsb5m9CzOsYOI9xwun9fji+0/dDm6fQ0NTA6LkjsX3JDuEbJKK8dvjwSVy9+qm0R11dDQBgbm6KyMjXwn5zMxME3br3zf4W+8xEu7Yt0axFJ7x48SrH/To62jh2ZBsSEpLQuUt/ZGRk5MGz+H3I66xtXvjhjFFbWxtly5ZF2bJlUbt2baxduxZJSUlYs2YNgOyv3GvWrImgoCCJ7cGDB+jZs2eeBW5paYmVK1di9uzZ0NDQgI6ODrZt+/GvNMaMGSMRZ/v27X/oeE1NzR9+zC81a9YMQUFBuHTpEhwdHdG/f/8cdcUfGRrm/pU2AGGm3d3dHQcOHMDs2bNx4cIFBAUFoUqVKhJlJ8D3PffLly/D29sbVatW/alZ41+VF+MLABcuXMCtW7dw8uRJBAQEwNvbG0D28+vVqxfatm2LI0eO4ObNm5gwYUKOsfqcp6cn3r17J7FZ6X59aZ5f9T7pPV4+fSlszx48w9uot6jxR3WhjZaOFiraVkBwYEiufWSkZ+DBnYeo/oetsE8kEqH6H7YIvpF9zMM7D5Celi7Rb7HSxWBezBzBN4IBAFMHTsfAVoMx0C57+zijOKqTK/7deDivn7pMSRv36rmN+42vj3uNhp+OEcY9MHtMH955mD3uDXMZ9w+vZ3BgMEpVsJJYIaJmoxpIik/Cs4fZH2w1NDVyJK+ZmVkfHjP7trqGOrKyJNtkZX1sw6kiyj+JiUl4/PipsAUHP8CrV1Fo3uwPoY2urg7q1KmOK/8FfrWvxT4z0cGhNf6064qnTyNy3K+rqwPfYzuQlpaGDp2cc3xbR4rtly9y8fFEovfv3wMAatSogV27dsHMzOyrs2e3bt3C+/fvheTmypUrQh2niYkJ1NTUEBAQgJIls0/kSE9Px7Vr1yROwHJycsKGDRtQvXp1jBo1Ch4eHj9czG5iYoKyZcsKt3V1dREXFwcAKFOmzDfjqFq1KjZt2vRLF5n4+IECACpWrIg5c+bg1q1bqF27do62FSpUQEREBCIiIoTZ32fPnuH58+ewsck+eSYgIADOzs7o2DH7a6DExESJEw6/57l/1Lt3b7i4uKBNmzaoXLkyDhw4IPT70ZUrV1CiRAkA2d8GPHjwABUrVhSeT0BAgET7gIAAlCtX7rvOuq1SpQqysrJw7tw5oezhZ5QqVQoGBgYoW7YsOnfujAMHDsDNzQ2XLl1CyZIlMWHCBKHts2fPvtqXurp6jmV48rPkQZr96w6g14ieeP7kBSIjItHH3Rlvot7i4olP4z1/51xc9A3AvxsPAQD2rt4Hj0Vj8ODWQ4QGhaJz/07Q0NTAiV0nAGSfvHV8py8GTx6EhLgEJCUkY/iMIbh3/R5CbmTPPL96JjnDom+U/T5/9ihcYtayjE12eYqmlib0jQ1QxqY0MtIzhETtd7V/3UH0Gt4DLz6Mu7O7E95GvUXAiU/rwM7bMQcBvpfw76bscd+3Zj/Gervj/u0HuB90H536dYSGpgZ8d58EkD3uvrtOwGXyQMTHJSA5MQnDpg/FvevBwox/4PkbCH8YjnGLx2L1rHUwMjOE8xhn/Lv5MNLTsmvsL5++Ate5o2Df+y9cP3cdRmZGGDLVBSE3Q/E2KnsG+crpK+g8oBMe3XuE0JuhKGJVFM7uTrhy+j8hCSYgOfk9wp+/FG6/eBmF0AePoa+nC8vvrEelb1uydC3Ge47Aw0dhePo0AtOmjsHLl1H4998TQpuTvrtw8N/jWLFyI4DsUoce3TugU+e+SEhIhLm5KQDg3bsEpKSkCImvppYGHJ2HQ09PV1jjNzr6rcL8nCvGs/w5P5z8pqamIjIyEkB2orNs2TIkJibC3t4eANCrVy/Mnz8fDg4OmD59OooVK4Znz55h//79GDt2LIoVKwYASEtLQ79+/TBx4kQ8ffoUU6ZMwbBhw6CkpARtbW0MHjwYY8aMgZGREUqUKIF58+YhOTkZ/fr1E2Jxc3ODSCTCokWLoKqqmmvy9iu+J45hw4Zh6dKl6N69Ozw9PaGvr48rV66gTp06KF/++5YOysrKQkpKCtLT03Hs2DG8ffsWlStXzrXtn3/+iYoVK6Jnz57CSVkjR46Era0tmjdvDgCwtrbG/v37YW9vD5FIhEmTJv30m/1jTW7JkiUxf/58DB48GI0bNxZOcASA6dOnw9jYGObm5pgwYQJMTEyEVSPc3NxQu3ZtzJgxA926dcPly5exbNkyrFix4rse38rKCk5OTujbty+WLFmCatWq4dmzZ3j9+rXECY7f8vr1a6SkpODZs2c4fPiwcFEQa2trhIeHY+fOnahduzaOHj2KAwcOfHe/srRzxW5oaGnAde4o6Ojp4M61u/D833ikp35aqaJISUvoG32aKfQ/fA76xvpwdnfMLpEIDsO43hMkTuBaMW0VxFliTFk9Capqarh+7joWj1/6w/GtPrlK+Hf5auXQsmNzREZEolf9X6/zk6VdK7PHffSckdDR08Hda/cwrveEXMb904d//8PnoG+kD2e3T+Pu2XuCxMoNK6atQlZW1odxV8X1c9exZMKnE9WysrIwwXkyRs4ejiX/LkJKcgpO7j2NjQs2CW1O7jkFLW1NODi1x6BJA5AYn4SggCCs8VontNm6ZDvEYjH6jHGGiYUx3r19h8unr2D9vI35M2C/qbuhD9F3uIdwe97S1QAAhzYtMWuim6zCkjvzF6yAtrYWVq2YBwMDPQQEXEM7+/9JzNSWLl0SJiafzg8Z7JK9tvUZv30SffXtNxqbt+xGjepVULdu9onZD0IlL05Sxrounj17DkXAAibpfjj59fX1haWlJYDsmcIKFSpgz549wtJdWlpaOH/+PDw8PNCpUyckJCSgaNGiaNGihcRMcIsWLWBtbY3GjRsjNTUVPXr0EJa2AoA5c+YgKysLvXv3RkJCAmrVqoUTJ04IX/vv2LEDu3fvxo0bN/L1sr7fisPY2BhnzpzBmDFj0KRJEygrK8PW1laiTvhbDh8+DE1NTaioqMDKygpLly5FvXr1cm2rpKSEf//9FyNGjECTJk2gpKSEP//8E0uXLhW+svT29kbfvn3RoEEDmJiYwMPDA/Hx8b88FoMGDcLevXsxfPhwbN/+6cSYOXPmYOTIkXj48CFsbW1x+PBhqKll13LVqFEDu3fvxuTJkzFjxgxYWlpi+vTpcHZ2/u7HXblyJcaPH48hQ4bg7du3KFGiBMaPH/9DsX/8IGJiYoJWrVph3rx5AID27dtj9OjRGDZsmFDzPGnSJImfxcJs44LN2Lhgs9T7c0s0/914SJgJzk16ajqWTFwmsULA19y6fBstirXKsT+3ffJi08LN2LRQ+rj/r4FTjn3/bjokzATnJj01HUsnLsfSicultnn94jUmOH39pLuDGw/h4Fde36zMLGzx2YYtPop15vuPqlOjKu4GHJd1GAph6rQFmDptgdT7y5aT/Huoolb0q/2dO3/5m21IsYnEMji7wdnZGXFxcTh48GBBPzTlIX9/fzRr1gyxsbEwMDCQdTgyJc+J3u+GdauFx/GbK2UdAn1Gs0gjWYdAH2Skvcj3x1hc4n950s/I8K150k9h8ss1v0RERERUuLDmV7qCP0uHiIiIiEhGZJL8bty4kSUPcqBp06YQi8UKX/JARERU2MjyCm/Lly+HlZUVNDQ0ULduXeEqrLlZs2YNGjVqBENDQxgaGqJly5ZfbZ8XOPNLREREJGfEebT9qF27dsHV1RVTpkzBjRs3UK1aNdjZ2eH169e5tvf390ePHj1w9uxZXL58GcWLF0erVq3w4kX+1UUz+SUiIiKiPOHt7Y0BAwagT58+sLGxwapVq6ClpYX169fn2n7btm0YMmQIbG1tUaFCBaxduxZZWVnw8/PLtxiZ/BIRERHJmSxR3mw/Ii0tDYGBgRIXpVJSUkLLli1x+fLl7+ojOTkZ6enpwnUG8gNXeyAiIiKSM3m12kNqamqOy0PndqVTAHjz5g0yMzNhbm4usd/c3ByhoaHf9XgeHh4oUqTIL13V9Vs480tEREQkZ/Kq5tfLywv6+voSm5eXV77EPGfOHOzcuRMHDhyAhoZGvjwGwJlfIiIiIpLC09MTrq6uEvtym/UFsq+iqqysjKioKIn9UVFRsLCw+OrjLFiwAHPmzMHp06dRtWrVXwv6GzjzS0RERCRnsiDOk01dXR16enoSm7TkV01NDTVr1pQ4We3jyWv169eXGuu8efMwY8YM+Pr6olatWnk+Fl/izC8RERGRnJHVFd5cXV3h5OSEWrVqoU6dOvDx8UFSUhL69OkDAHB0dETRokWF0om5c+di8uTJ2L59O6ysrBAZGQkA0NHRgY6OTr7EyOSXiIiIiPJEt27dEB0djcmTJyMyMhK2trbw9fUVToILDw+HktKnwoOVK1ciLS0Nf//9t0Q/U6ZMwdSpU/MlRia/RERERHLmZy5QkVeGDRuGYcOG5Xqfv7+/xO2nT5/mf0BfYPJLREREJGdkVfbwO+AJb0RERESkMDjzS0RERCRnfvTqbIqEyS8RERGRnMmSadVv4cayByIiIiJSGJz5JSIiIpIznPeVjskvERERkZzhag/SMfklIiIikjOs+ZWONb9EREREpDA480tEREQkZzjvKx2TXyIiIiI5w5pf6Vj2QEREREQKgzO/RERERHKGJ7xJx+SXiIiISM4w9ZWOZQ9EREREpDA480tEREQkZ3jCm3RMfomIiIjkjJiFD1Kx7IGIiIiIFAZnfomIiIjkDMsepGPyS0RERCRnuNSZdEx+iYiIiOQMU1/pWPNLRERERAqDM79EREREcoZlD9Ix+SUiIiKSMzzhTTqWPRARERGRwuDMLxEREZGc4UUupGPyS0RERCRnWPYgHcseiIiIiEhhcOaXKA8oi/g5srDgGc6Fh2aRRrIOgT7z/uUFWYdABYhlD9Ix+SUiIiKSMyx7kI7TVURERESkMDjzS0RERCRnssQse5CGyS8RERGRnGHqKx2TXyIiIiI5w5N/pWPNLxEREREpDM78EhEREckZLnUmHZNfIiIiIjnDpc6kY9kDERERESkMzvwSERERyRme8CYdk18iIiIiOcOaX+lY9kBERERECoMzv0RERERyhie8Scfkl4iIiEjOiHl5Y6lY9kBERERECoMzv0RERERyhqs9SMfkl4iIiEjOsOZXOia/RERERHKGS51Jx5pfIiIiIlIYnPklIiIikjOs+ZWOyS8RERGRnOFSZ9Kx7IGIiIiI8szy5cthZWUFDQ0N1K1bF1evXv1q+z179qBChQrQ0NBAlSpVcOzYsXyNj8kvERERkZzJyqPtR+3atQuurq6YMmUKbty4gWrVqsHOzg6vX7/Otf2lS5fQo0cP9OvXDzdv3kSHDh3QoUMH3L179yce/fuIxJwXJ/plrYq3lnUI9AHr3AoP/6j8++NFP+79ywuyDoE+UDUpne+PkVd/l05G+P5Q+7p166J27dpYtmwZACArKwvFixfH8OHDMW7cuBztu3XrhqSkJBw5ckTYV69ePdja2mLVqlW/FrwUnPklIiIiol+WlpaGwMBAtGzZUtinpKSEli1b4vLly7kec/nyZYn2AGBnZye1fV7gCW9EREREciavvgVLTU1FamqqxD51dXWoq6vnaPvmzRtkZmbC3NxcYr+5uTlCQ0Nz7T8yMjLX9pGRkb8YuXSc+SUiIiKSM2KxOE82Ly8v6OvrS2xeXl6yfnq/hDO/RERERJQrT09PuLq6SuzLbdYXAExMTKCsrIyoqCiJ/VFRUbCwsMj1GAsLix9qnxc480tEREQkZ7IgzpNNXV0denp6Epu05FdNTQ01a9aEn5/fpziysuDn54f69evnekz9+vUl2gPAqVOnpLbPC5z5JSIiIpIzYhmtfOPq6gonJyfUqlULderUgY+PD5KSktCnTx8AgKOjI4oWLSqUTowcORJNmjTBwoUL0a5dO+zcuRPXr1/H6tWr8y1GJr9EREREciZLRivZduvWDdHR0Zg8eTIiIyNha2sLX19f4aS28PBwKCl9Kjxo0KABtm/fjokTJ2L8+PGwtrbGwYMHUbly5XyLkev8EuUBrvNbeHCd38KD6/wWLlznt/AoiHV+GxdtkSf9nH/h9+1GvxnO/BIRERHJGU4DSMfkl4iIiEjO8Fsw6bjaAxEREREpDM78EhEREckZzvxKx+SXiIiISM5wPQPpWPZARERERAqDyS/99lxcXNCjRw8AgI6ODo4cOSLjiIiIiGQrr67wJo9kWvbg6OiI2NhYHD58WJZh0G9u+vTpyMrKAgAEBQXB0tJSxhHlP3sne3QZ9DeMTA0RFhKG5ZNX4H7QA6ntG7VrBGd3R5gXM8eLpy+wdvZ6XDt7TaKNo1tvtOnRBjr62rh3LRhLxi/Fy6cvAQBV61XFgj3zcu172F8j8ODWAxQrXQwjvIajpHUJaOtq423UW5z91x9bFm1FZkZm3j15GWvvZI+ug/6GkakRHoeEYdnkFbgfdF9q+8btGsHZ3QkWH8Z+zex1uPrF2Du5OaJtj9bQ0dfBvWvBWDx+CV58GHsA0DXQxbDpQ1CvZV2Is8S4cPwilk9ZiZTkFACA4+j/wdG1d47Hfp+cAvvyDgCAkuVKwtnNEdZVysKiuAVWTF2F/esO5MWQ/JamTnFHv749YWCgh0uXrmPocE88evREanuPscPQoUMbVChfFu/fp+DylevwHD8bDx48BgAYGhpgymQ3/PlnE5QoXgTR0TH495Avpkydj/j4hIJ6WnLpetAdbNi+F8GhjxD9NgaLvSahReMGsg6r0JPVFd5+BwU+83vv3j1069YNxYoVw5YtW3DkyBHo6uqiTZs2OHXqVEGHQ3LAzMwMFhYWAICyZctCW1tbxhHlryb2jTFo0gBs9dmKIW2HISw4DLO3zIKBsX6u7W1qVsT4ZePgu/MEBrcZiksnLmPq2smwKl9SaNN1cBd06OOAJeOXYIT9KKS8T4HX1llQVVcFAAQHBqNbjR4S27Htx/Hq2Ss8uJWddGdkZOD0vtPw7DUefZv2x8pp/6BNj9ZwdMuZlP2umto3gcukgdjisw0ubYciLDgMc7469jaYsMwTvjt94dJmCAJOXMK0tVMkxr7b4K7o2McBi8cvxTD7kUh5n4I5W2cLYw8Anks8ULJcSXj09MTEPpNRpW4VuM4dJdy/+5+96FKju8T29P4znD96XmijoamOV+GvsHbOeryNepv3g/MbGeM+BMOG9sWQYePQ4A97JCUn49iRbVBXV5d6TONG9bBy5SY0bGSP1m17QFVFFcePboeWliYAoEgRcxQpYg4PjxmoVr0F+vUfDTu7ZlizemFBPS259f59CsqXLY0JbkNkHcpvRSwW58kmjwo0+T1w4ACqVauG1NRUbN26FV27dkXr1q1x/PhxWFhYoFWrVli+fLnEMf7+/hCJRBKbgYGBcP+1a9fw559/wsTEBPr6+mjSpAlu3Lgh0ceXx4tEIixbtixHfGKxGGXLlsWCBQsk9gcFBUEkEuHRo0ewsrLKtT+RSISNGzcKj3fw4EHh+HXr1kEkEmHUqFES/U6dOjVHHx06dBDut7Kygo+Pj3Dbz88vR5vevXvDzMwM6urqKF26tETsb9++RY8ePVC0aFFoaWmhSpUq2LFjh0QMTZs2zTUuW1tb4fbGjRslxhwAGjduDJFIhKCgIACfXqe4uDiJ2L4ciy9lZWVh3rx5KFu2LNTV1VGiRAnMmjVL6uN+GW9qairc3d1RtGhRaGtro27duvD395eISdr2vWNU2HQe0AnHd/ji5O5TCH8YjsWeS5Gakgq7bna5tu/QrwOu+V/Hnn/2IuJRBDYt2IxHdx+hvVN7oU3Hfh2xfekOXD55BU9Cn2DeqPkwNjdGQ7vs2ZWM9AzERscKW3xsPBq0qo+Tez59YI0Mj8TJ3acQFvIEr1+8xpVTV3Dm4FlUrpN/l6gsaJ0HdMKxHb44sfskwh+Gw8dzCVJTUtFayth3+jD2u//Zi/BHEdj4YewdnBwk2mxbugOXTl7Gk9AnmDtqnsTYlyhbHHWa1Yb32EUIDbqPu9fuYfnkFWjavgmMzY0AACnJKRKvj6GJIazKl8TxnSeEx7l/6wFWz1oL/0PnkJ6Wno+jVPiNGN4fs70W4/Dhk7hzJwTOfUaiSBFzODjk/joCQDv7/2Hzlt0IDn6A27eD0bf/KJQsWQw1a1QFANy7dx9duw3EkaOnEBb2DGf9AzBp8lz81a4llJWVC+qpyaVG9WtjxEAntGzSUNahkJwo0OR31KhRaNq0KQ4ePIimTZtCU1MT6urq+OOPP7BhwwY4Oztj7NixSEpKEo75+Knj/v37ePXqlUQyCAAJCQlwcnLCxYsXceXKFVhbW6Nt27ZISJD8mmnDhg149eqVsPXt2zdHfCKRCH379sWGDRtyHNu4cWOULVsW165dE/ooVqwYfHx8hNvdunXL0WdSUhImTZoEHR2dHPeJxWJUqlRJOL5r165Sxy4rKwtubm45+unevTtOnz6Nhw8fYtasWfD09MT589mzPSkpKahZsyaOHj2Ku3fvYuDAgejduzeuXr0q9XG+x/79+3Hz5s2vtgkMDMShQ4e+2ZenpyfmzJmDSZMmITg4GNu3bxeu//09hg0bhsuXL2Pnzp24ffs2unTpgtatW+Phw4do0KCBMLb79u0DAImfASD/xii/qKiqwLqKNW5e/DT+YrEYNy/cRMWaFXM9xqZGRYn2AHD9XKDQ3qKEBYzNjXDjwqc2yQnJCA0KRcUaufdZ/8960DXUxYldJ6XGWsTKErWa1MSdK3e++/kVZiqqKihXxRo3Ln76cC0Wi3Hjwk3Y1LTJ9RibGhVx44uxv3YuEDYfxt6yhAWMzY1x48KnPpMSkhESFAqbD2NvU7MiEuIS8OD2Q6FN4IUbEGeJUaF6hVwft22P1oh4HIG7V3l54S+VKlUClpbm8DtzUdgXH5+Aq1dvol7dmt/dj76+HgAgJjZOehs9XcTHJyIzU37Kfuj3wZpf6Qqs5jcqKgrh4eEYPXq01Dbt27fHxo0bcffuXdStWxcAkJ6ePUPxcWZPX1/y68XmzZtL3F69ejUMDAxw7tw5/PXXX8J+AwMD4avxr3F2dsbkyZNx9epV1KlTB+np6di+fbswo2pqaiq0VVZWhr6+/lf7nTdvHmxsbJCRkZHjvvT0dGhqagrHa2pqIjU1Ndd+Nm3ahNTUVDg4OCAxMVHY365dO+HfMTExUFFREX7RFi1aFO7u7sL9w4cPx4kTJ7B7927UqVPnm2ORm/T0dHh4eMDDwwOTJk2S2s7V1RVjxoz5apuEhAQsXrwYy5Ytg5OTEwCgTJky+OOPPwBkj0dKSorU48PDw7FhwwaEh4ejSJEiAAB3d3f4+vpiw4YNmD17tjC2RkbZM2Rfvlb5MUb5Sc9ID8oqyoiNjpPYH/smDsXLFs/1GENTQ8S+kWwf9yYORqaGACD8P+6LNrHRcTA0M8y1z9bd7RB4LhBvIt/kuG/RAW9YVy4LNQ01HN16DJsWbP6OZ1b46Usd+9hvjH2sxL64N7HCmBuaGn3oQ7LPuOg4GJkZCW3i3kren5WZhfi4BBh9OP5zquqqaN6xOXYu3/W9T02hWJibAQCioqIl9ke9fgMLC7Pv6kMkEsF7wTQEBFzFvXu513sbGxtiwvhRWLtu268FTPST5LVkIS8U2MyvmpoaACA5OVlqm4/3aWhoCPvi4+OhpKQETU3NXI+JiorCgAEDYG1tDX19fejp6SExMRHh4eE/FWeRIkXQrl07rF+/HgBw+PBhpKamokuXLj/c18uXL+Ht7Y2FC3Ov+YqPj/+u+tTk5GRMnDgR8+bNg4pKzs8rLi4u0NTURK1atTBp0iQ0a9YMAJCZmYkZM2agSpUqMDIygo6ODk6cOPHTYwMAy5cvh76+Pnr16iW1zcGDBxEWFgY3N7ev9hUSEoLU1FS0aNEi1/srVaqE1NRUYdb2S3fu3EFmZibKlSsHHR0dYTt37hweP378Xc/nZ8YoNTUV8fHxEluWOOu7Hk8emFiYoGaTmvD97Cv1z80eMhtD2g7D7GFzUKdFbfw9qHMBR6jY/mjdEFramji5l+dQAECPHh0RF/NA2FRVf33OZ+mS2ahUqTx6/i/3GlRdXR0c/nczQkIeYNp01vwSFTYFNvNraGiIunXrYvPmzRg5cmSOpC8jIwP//PMPihUrhsqVP9UIvnz5Eubm5lBSyj1Pd3Jywtu3b7F48WKULFkS6urqqF+/PtLS0n461v79+6N3795YtGgRNmzYgG7dukFLS+uH+5kwYQK6dOmCatWq5Xr/y5cvhRnLr5k/fz7Kly8Pe3v7XBPB6dOnY8SIEThz5gymTp2Kjh07omLFipg/fz4WL14MHx8fVKlSBdra2hg1atRPj01sbCxmzJiBAwcOCDWzX0pPT8fYsWMxa9YsqR9YPvrW/ZUrV4aHhwe6dOkCDQ0NKCkp4f3790I9cmJiIpSVlREYGJijpi63MpPc/MwYeXl5Ydq0aRL7SuuWQRn9st/1mL8iPiYemRmZMDQ1kNhvaGKAmOjYXI/JrgGVbG/wWfuP/zcwMUDM65hPfZoa4PG9sBz92XVrhYTYBFw+dSXXx4t+lT0bHP4wHMpKShg5dwT2rd4vrMjxu3ondewNEfvVsZecPTcwMRTGPDY65kMfkmNvYGqAx/ceC20MjCUfU0lZCXoGuoiJjsGX2nRvjSt+/+WYyVdUhw+fxNWrn0pP1NWzJ2LMzU0RGfla2G9uZoKgW/e+2d9in5lo17YlmrXohBcvXuW4X0dHG8eObENCQhI6d+mf67d+RAVBXksW8kKB1vyuXbsWKSkpqFixIqZNm4YnT57gxYsXmD17NipXroy7d+9i27ZtEonMtWvXUL16dal9BgQEYMSIEWjbti0qVaoEdXV1vHmT86vYH9G2bVtoa2tj5cqV8PX1zbU++FuCgoKwd+9ezJw5M9f7s7KycOPGja8+NyC7RnXhwoVSZ4+B7NUObGxsMGzYMBQvXhxHjx4FkD02Dg4O+N///odq1aqhdOnSePBA+nJY3zJjxgw0atQIjRs3ltpm5cqV0NHRQe/e3z7D39raGpqamvDz85PaZs6cOYiLi8Pt27cRFBSEWrVqCfdVr14dmZmZeP36NcqWLSuxfU+JC/BzY+Tp6Yl3795JbKX0Sn/X4/2qjPQMPLzzELYNbYV9IpEItn/YIiQwJNdjgm+EoPpn7QGgRqMaQvvI8Ei8jYpB9T8+tdHS0UIF2woIuZGzz1Zd/sSpfae/a/kykZIIKioqECnl/mHpd5KRnoEHdx6iRsNP71mRSITqf9giODA412NyG/uajWog+MPYvwqPxNuot6j+x6c+tXS0UNG2AoI/jH1wYAh0DXRhXeXTh6vqDW0hUhIh9GaoRN8Wxc1h26CaxIluii4xMQmPHz8VtuDgB3j1KgrNm/0htNHV1UGdOtVx5b/Ar/a12GcmOji0xp92XfH0aUSO+3V1deB7bAfS0tLQoZOz1DI2ooIgzqP/5FGBrvNbuXJl3L9/Hxs2bMDFixcREhKC9PR0XL58GX379kWfPn2EmtrExESsXbsW27dvx65d0mvXrK2tsWXLFtSqVQvx8fEYM2bMN2cUv0VZWRnOzs7w9PSEtbU16tev/8N9LFiwAG5ubrnO7EZERGDq1Kl4/fp1rifJfW758uXo3LlzrklyXFwcDh48iHr16kFNTQ1HjhzBnTt3hLbW1tbYu3cvLl26BENDQ3h7eyMqKgo2NpIn52RmZkrU1mZkZEAsFiMtLU2iXGX16tU5VtL40rx583D48GGpM8Of09DQgIeHB8aOHQs1NTU0bNgQ0dHRuHfvHvr16ye009PTg55e9skln7+25cqVQ69eveDo6IiFCxeievXqiI6Ohp+fH6pWrSpRDy3N947R59TV1XMsiaQkKrjPkfvW7McYb3c8vP0QoUH30alfR2hoauDE7uyTz8YscsfbyLdYPzf7xM2D6w5iwZ756DywE676XUXT9k1Rrqo1Fo9bLPR5YN0B9BzeAy+evERkRCSc3R3xNuotAk5cknhs24a2sCxpCd8dvjniat6hGTIyMvAk9CnS09JRrqo1+o7rg3OHz8vNOr/71uzHWG933L/9APc/G3vfD2PvsWgM3kS+wboPY79/3UF475mPvwd2xn9+V9GsfROUq2qNReN8hD73rzuIXsN74MWTFx/G3kli7MMfReDq2WtwnTsKPuOXQkVFGcNnDIX/oXN4GyU589u6mx1iXsfkWMMZyD5hr6R1iex/q6nCxMIYZWxK431yirCes6JYsnQtxnuOwMNHYXj6NALTpo7By5dR+PffTx8aTvruwsF/j2PFyo0AsksdenTvgE6d+yIhIRHm5tl/q969S0BKSoqQ+GpqacDReTj09HShp6cLAIiOfvvbf/MhS8nJ7xH+/NPP6IuXUQh98Bj6erqw/M46baLPFfhFLtTV1eHi4gIXFxc4OzsLCdyXTp06hTVr1uCff/7B33//LbW/devWYeDAgahRowaKFy+O2bNnS5zA9LP69euH2bNno0+fPj91vK6uLsaOHZvrfYsXL8ajR49w8uRJlChR4qv9ZGVlCUt/fUksFmPjxo0YPXo0UlNTUbp0aaxcuVKooZ04cSLCwsJgZ2cHLS0tDBw4EB06dMC7d+8k+lm2bFmuS7+1atVKWDYsPT0dgwYNQrly5b4ab7NmzYSa4+8xadIkqKioYPLkyXj58iUsLS3h4uLy3cdv2LABM2fOhJubG168eAETExPUq1dP4mTHr/neMSpMzh0+D30jfTi69YahqSHCgsMwofdE4Wtus6JmEic6BAeGwGv4XDiPcUKfsc54+fQlpvafjqf3nwltdq/cAw0tDYyaMwI6ejq4e+0exveeiPRUySWxWne3w71r9xDx+HmOuDIzM9FtcFcULV0UIpEIUc9f49DGw9i3dn/+DIQM+B8+B30jfTi7OcLQ1BCPg8Pg2XvCZ2NvKlH/HRwYjNnD56DPGCf0HeuMF09fYkr/aRJjv2vlbmhoaWD0nJHC2I/rPUFi7L1GzMXwGUMxf8cc4SIXyyavkIhNJBKhVZdWOLHnVK6JlrG5Mf45sVK43dWlC7q6dMGty7fg1jX331Xyav6CFdDW1sKqFfNgYKCHgIBraGf/P4mZ2tKlS8LE5NMJhYNdsk/KPeMnWXrWt99obN6yGzWqV0HdujUAAA9CJT80lrGui2fPcr5n6PvcDX2IvsM9hNvzlq4GADi0aYlZE79+bokiy+IJb1KJxDwdMFcXLlxAixYtEBER8UNLb8mLoKAgjBo1Skh+6etaFW8t6xDoA9a5FR7+UVxqrTB5//KCrEOgD1RN8r9UrpJ53Tzp517Uf3nST2Ei08sbF0apqamIjo7G1KlT0aVLF4VMfAFASUlJKHkgIiIikhcFfnnjwm7Hjh0oWbIk4uLiMG/ePFmHIzNVq1bFyZPSL2BAREREhVeWWJwnmzxi8vsFZ2dnZGZmIjAwEEWLFpV1OEREREQ/jKs9SMeyByIiIiI5I6+ztnmBM79EREREpDA480tEREQkZ+S1ZCEvMPklIiIikjMse5COZQ9EREREpDA480tEREQkZ1j2IB2TXyIiIiI5IxbnvMw5ZWPZAxEREREpDM78EhEREcmZLJY9SMXkl4iIiEjOiLnag1QseyAiIiIihcGZXyIiIiI5w7IH6Zj8EhEREckZlj1Ix+SXiIiISM7wCm/SseaXiIiIiBQGZ36JiIiI5Ayv8CYdk18iIiIiOcOaX+lY9kBERERECoMzv0RERERyhkudScfkl4iIiEjOsOxBOpY9EBEREZHC4MwvERERkZzhOr/SMfklIiIikjMse5COZQ9EREREpDA480tEREQkZ7jag3RMfomIiIjkDMsepGPyS0RERCRneMKbdKz5JSIiIiKFwZlfIiIiIjkjZs2vVJz5JSIiIpIzWWJxnmz5JSYmBr169YKenh4MDAzQr18/JCYmfrX98OHDUb58eWhqaqJEiRIYMWIE3r1798OPzeSXiIiIiApUr169cO/ePZw6dQpHjhzB+fPnMXDgQKntX758iZcvX2LBggW4e/cuNm7cCF9fX/Tr1++HH1sk5umARL+sVfHWsg6BPuDyPoWHf9RdWYdAn3n/8oKsQ6APVE1K5/tjaGiUyJN+UlLC86Sfz4WEhMDGxgbXrl1DrVq1AAC+vr5o27Ytnj9/jiJFinxXP3v27MH//vc/JCUlQUXl+yt5OfNLREREJGfEefRfamoq4uPjJbbU1NRfiu3y5cswMDAQEl8AaNmyJZSUlPDff/99dz/v3r2Dnp7eDyW+AJNfIiIiIpLCy8sL+vr6EpuXl9cv9RkZGQkzMzOJfSoqKjAyMkJkZOR39fHmzRvMmDHjq6US0jD5JSIiIpIzYrE4TzZPT0+8e/dOYvP09Mz1MceNGweRSPTVLTQ09JefW3x8PNq1awcbGxtMnTr1h4/nUmdEREREciavTulSV1eHurr6d7V1c3ODs7PzV9uULl0aFhYWeP36tcT+jIwMxMTEwMLC4qvHJyQkoHXr1tDV1cWBAwegqqr6XbF9jskvEREREf0yU1NTmJqafrNd/fr1ERcXh8DAQNSsWRMAcObMGWRlZaFu3bpSj4uPj4ednR3U1dVx6NAhaGho/FScLHsgIiIikjPiPNryQ8WKFdG6dWsMGDAAV69eRUBAAIYNG4bu3bsLKz28ePECFSpUwNWrVwFkJ76tWrVCUlIS1q1bh/j4eERGRiIyMhKZmZk/9Pic+SXKAycjfGUdwi9LTU2Fl5cXPD09v/srLsoffC0KD74WhQdfix+TkfZC1iF81bZt2zBs2DC0aNECSkpK6Ny5M5YsWSLcn56ejvv37yM5ORkAcOPGDWEliLJly0r09eTJE1hZWX33Y3OdXyICkP2pWl9fX1g6hmSHr0Xhwdei8OBrQXmFZQ9EREREpDCY/BIRERGRwmDyS0REREQKg8kvEQHIXstxypQpPJGkEOBrUXjwtSg8+FpQXuEJb0RERESkMDjzS0REREQKg8kvERERESkMJr9EREREpDCY/BIRERGRwmDyS0REREQKQ0XWARCR7Jw/fx4NGjSAigp/FRQGz58/x6FDhxAeHo60tDSJ+7y9vWUUlWLJzMzEokWLsHv37lxfh5iYGBlFppiuX78u9bXYv3+/jKKi3x1nfokUWLNmzfjHvJDw8/ND+fLlsXLlSixcuBBnz57Fhg0bsH79egQFBck6PIUxbdo0eHt7o1u3bnj37h1cXV3RqVMnKCkpYerUqbIOT6Hs3LkTDRo0QEhICA4cOID09HTcu3cPZ86cgb6+vqzDo98Yk18iBcZlvgsPT09PuLu7486dO9DQ0MC+ffsQERGBJk2aoEuXLrIOT2Fs27YNa9asgZubG1RUVNCjRw+sXbsWkydPxpUrV2QdnkKZPXs2Fi1ahMOHD0NNTQ2LFy9GaGgounbtihIlSsg6PPqN8SIXRApMSUkJBw4cgKGhYa73N27cuIAjUly6uroICgpCmTJlYGhoiIsXL6JSpUq4desWHBwc8PTpU1mHqBC0tbUREhKCEiVKwNLSEkePHkWNGjUQFhaG6tWr4927d7IOUWFoa2vj3r17sLKygrGxMfz9/VGlShWEhISgefPmePXqlaxDpN8UC/2IFFzHjh1z3S8SiZCZmVnA0SgubW1toabR0tISjx8/RqVKlQAAb968kWVoCqVYsWJ49eoVSpQogTJlyuDkyZOoUaMGrl27xsvqFjBDQ0MkJCQAAIoWLYq7d++iSpUqiIuLQ3Jysoyjo98Zk18iBRcZGQkzMzNZh6Hw6tWrh4sXL6JixYpo27Yt3NzccOfOHezfvx/16tWTdXgKo2PHjvDz80PdunUxfPhw/O9//8O6desQHh6O0aNHyzo8hdK4cWOcOnUKVapUQZcuXTBy5EicOXMGp06dQosWLWQdHv3GWPZApMCUlZXx6tUrJr+FQFhYGBITE1G1alUkJSXBzc0Nly5dgrW1Nby9vVGyZElZh6iQLl++jMuXL8Pa2hr29vayDkehxMTEICUlBUWKFEFWVhbmzZsnvCcmTpwotVyL6FuY/BIpMCUlJc78EhGRQuFqD0QKLCsri4lvIVG6dGm8fftW1mEovLS0NPzzzz/YtWsXAGDRokVo3LgxBg0ahLi4ONkGp2CUlZXx+vVrWYdBcogzv0QKzMvLC+bm5ujbt6/E/vXr1yM6OhoeHh4yikzxcBa+cOjfvz927doFdXV1tG3bFhcvXkTnzp1x+PBh1KlTB5s3b5Z1iAqD7wnKL0x+iRSYlZUVtm/fjgYNGkjs/++//9C9e3c8efJERpEpHiUlJURFRcHU1FTWoSg0c3NzbNu2DVZWVihXrpxwctV///2H9u3bIyoqStYhKgy+Jyi/cLUHIgUWGRkJS0vLHPtNTU25hqYM1KpVC8rKyrneFxYWVsDRKKY3b96gSpUqMDc3h5aWFkqVKgUg+4Mil5wreBYWFlLv41KM9LOY/BIpsOLFiyMgIED4A/9RQEAAihQpIqOoFJebmxsv2ypjYrEYiYmJ0NDQgJKSEhITExEfH4/4+HhZh6aQ9u7dCyMjI1mHQXKGyS+RAhswYABGjRqF9PR0NG/eHADg5+eHsWPHws3NTcbRKRaRSITu3buzvrEQKFeuHIDsRLh69erCv0UikSzDUjgikQgNGzbke4LyHJNfIgU2ZswYvH37FkOGDBGuLqahoQEPDw94enrKODrFwtMvCoezZ8/KOgT6gO8Jyi884Y2IkJiYiJCQEGhqasLa2pqXcZWBTZs2oXv37hx7GQsPD0fx4sU5y1sInDt3Dg0bNoSKCufpKG8x+SUiIvqAVz0sXB4/fowNGzbg8ePHWLx4MczMzHD8+HGUKFEClSpVknV49JvixykiBXf9+nXs3r0b4eHhQunDR/v375dRVIrH0NDwq7ONMTExBRiN4uJ8kGwdO3YMzZo1g6amJs6dO4c2bdqgYcOGOH/+PGbNmgUzMzPcunUL69atw969e2UdLv2mmPwSKbCdO3fC0dERdnZ2OHnyJFq1aoUHDx4gKioKHTt2lHV4CmXRokUQiUQQi8UYPHgwpk+fztlHGXn+/DlSUlJyva9EiRIFHI1i6datG27duoXSpUtj3LhxmDlzJlxdXaGrqyu0ad68OZYtWybDKOl3x7IHIgVWtWpVDBo0CEOHDoWuri5u3bqFUqVKYdCgQbC0tMS0adNkHaJC+vhalC5dWtahKBwlJaVcZ+A/rvbAtWULjo6ODu7cuYNSpUpJvCeePn2KChUqSP2AQvQtSrIOgIhk5/Hjx2jXrh0AQE1NDUlJSRCJRBg9ejRWr14t4+iIZOO///5DWFiYxPbkyRNeaKQA9OrVC69fvwYAGBgY5HqxnZs3b6Jo0aIFHRrJEZY9ECkwQ0NDJCQkAACKFi2Ku3fvokqVKoiLi0NycrKMo1NsXG1ANkQiEUqUKMGSExnR09ODklL2vFz37t3h4eGBPXv2QCQSISsrCwEBAXB3d4ejo6OMI6XfGZNfIgXWuHFjnDp1ClWqVEGXLl0wcuRInDlzBqdOnUKLFi1kHZ5C6dSpk/DvlJQUuLi4QFtbW9jHkw8LBisBZWvlypXCv2fPno2hQ4eiePHiyMzMhI2NDTIzM9GzZ09MnDhRhlHS7441v0QKLCYmBikpKShSpAiysrIwb948XLp0CdbW1pg4cSIMDQ1lHaLC6NOnz1fv37BhQwFFotiePXuGEiVKcOa9EAkPD8fdu3eRmJiI6tWrw9raWtYh0W+OyS8REdEHGzZsgI6ODrp06SKxf8+ePUhOToaTk5OMIiOivMLkl0jBZWZm4sCBAwgJCQEA2NjYwMHBgVdVkoGMjAz4+/vj8ePH6NmzJ3R1dfHy5Uvo6elBR0dH1uEphHLlyuGff/5Bs2bNJPafO3cOAwcOxP3792UUmeJxdXX96v3e3t4FFAnJG/51I1Jg9+7dQ/v27REZGYny5csDAObOnQtTU1McPnwYlStXlnGEiuPZs2do3bo1wsPDkZqaij///BO6urqYO3cuUlNTsWrVKlmHqBDCw8NRqlSpHPtLliyJ8PBwGUSkuG7evClx++LFi6hZsyY0NTVZlkK/hMkvkQLr378/KlWqhOvXrwv1vbGxsXB2dsbAgQNx6dIlGUeoOEaOHIlatWrh1q1bMDY2FvZ37NgRAwYMkGFkisXMzAy3b9+GlZWVxP4vXxfKf2fPnpW4rauri+3bt3P9a/plTH6JFFhQUJBE4gtkL382a9Ys1K5dW4aRKZ4LFy7g0qVLUFNTk9hvZWWFFy9eyCgqxdOjRw+MGDECurq6aNy4MYDskoeRI0eie/fuMo5OsbFKk/IKk18iBVauXDlERUWhUqVKEvtfv36NsmXLyigqxZSVlZXr1cOeP38ucWlXyl8zZszA06dP0aJFC6HuPSsrC46Ojpg9e7aMo1Nc+/fvR0pKCtdfpjzBE96IFNixY8cwduxYTJ06FfXq1QMAXLlyBdOnT8ecOXPwxx9/CG319PRkFaZC6NatG/T19bF69Wro6uri9u3bMDU1hYODA0qUKMGlzgrYw4cPERQUBE1NTVSpUgUlS5aUdUgKx9DQECKRCCkpKUhNTYWHhwc/gFCeYPJLpMA+XkkJ+HRFsY+/Ej6/LRKJcp2VpLzz/Plz2NnZQSwW4+HDh6hVqxYePnwIExMTnD9/njNeBWT69Olwd3eHlpaWrENReBs3boRIJIKmpiYqVaqU4xsqop/F5JdIgZ07d+672zZp0iQfIyEge6mznTt34vbt20hMTESNGjXQq1cvaGpqyjo0haGsrIxXr17xwwaRHGPyS0RE9IGSkhIiIyOZ/BYCXl5eMDc3R9++fSX2r1+/HtHR0fDw8JBRZPS7Y/JLpOC2bt2KzMxMODo64vz589i3bx9q1KgBZ2dnWYemUA4dOvTV+9u3b19AkSg2JSUluLu7S72oyOTJkws4IsVlZWWF7du3o0GDBhL7//vvP3Tv3h1PnjyRUWT0u+NqD0QKbNy4cVi9ejW0tLRw48YN7Ny5Ew0bNsSGDRsQFRXFmZUC1KFDhxx11x+x5rpgBQQE5FhyDsh+HZj8FpzIyEhYWlrm2G9qaopXr17JICKSF0x+iRTY1q1bsWPHDpQrVw5ly5bF1q1b0aNHD2zduhUzZsxg8luAevXqhcOHD2Ps2LFwc3ODurq6rENSWAcOHGDZQyFQvHhxBAQE5LjiXkBAAIoUKSKjqEgeKH27CRHJq+joaFSqVAmlSpWChoYGatasCQBo1KgRIiIiZBydYtmyZQv8/Pxw8uRJlCtXDtu2bZN1SEQyNWDAAIwaNQobNmzAs2fP8OzZM6xfvx6jR4/mVQ/pl3Dml0iBmZiY4M2bNyhWrBgmTpwozHYlJSXxwgoyULNmTfj7++PgwYPw8PDAokWL4O3tLVxpjPJfkyZNci15oII3ZswYvH37FkOGDEFaWhoAQENDAx4eHvD09JRxdPQ74wlvRArM09MTdnZ2aNq0qcT+lStX4ujRozhy5IhsAlNA8fHxErfT0tKwcuVKzJ8/H82bN8fBgwdlE5gC+3LNa5KNxMREhISEQFNTE9bW1iwJol/G5JeIqBBQUlLKNcniRUYK3ubNmzF//nw8fPgQQPZlwMeMGYPevXvLODIiygsseyAiREdH4/79+wCA8uXLw9TUVMYRKZ6zZ8/KOgQC4O3tjUmTJmHYsGFo2LAhAODixYtwcXHBmzdvMHr0aBlHSES/ijO/RAosKSkJw4cPx5YtW4SZRWVlZTg6OmLp0qW8xCspnFKlSmHatGlwdHSU2L9p0yZMnTqVa8sSyQEmv0QKbNCgQTh9+jSWLVsmMcs1YsQI/Pnnn1i5cqWMI1QcvMhF4aChoYG7d++ibNmyEvsfPnyIKlWqICUlRUaREVFeYfJLpMBMTEywd+/eHCe8nT17Fl27dkV0dLRsAlNASkrSV55kzW/BqVy5Mnr27Inx48dL7J85cyZ27dqFO3fuyCgyIsorrPklUmDJyckwNzfPsd/MzAzJyckyiEixvXr1KtfXgwrOtGnT0K1bN5w/f174NiQgIAB+fn7YvXu3jKNTLPw2hPILZ36JFFiLFi1gbGyMzZs3Q0NDAwDw/v17ODk5ISYmBqdPn5ZxhIpDSUkJUVFRPNmwEAgMDMSiRYsQEhICAKhYsSLc3NxQvXp1GUemWD5fAYWX/Ka8xOSXSIHdvXsXdnZ2SE1NRbVq1QAAt27dgrq6Ok6ePIlKlSrJOELFoaSkhJkzZ8LQ0BDa2tooUqQIbG1tYWJiIuvQiGSid+/evOQ35Qsmv0QKLjk5Gdu2bUNoaCiA7FmuXr16QVNTU8aRKRYrKysAQHp6OuLj45GUlAQlJSW0a9cOW7ZsgZ6enmwDVCCZmZk4cOCAMPNrY2MDBwcHqKiwUrCgBQYGws3NDU+ePMHs2bPRq1cvWYdEcoDJL5ECe/v2LYyNjQEA4eHhWLt2Ld6/f4/27dujUaNGMo5OsSUkJODq1asYNmwYmjRpglWrVsk6JIVw7949tG/fHpGRkShfvjwA4MGDBzA1NcXhw4dRuXJlGUeomD5e8ltXV5eX/KZfxuSXSAHduXMH9vb2iIiIgLW1NXbu3InWrVsLs41JSUnYu3cvOnToIOtQFd7Zs2fRr18/hIWFyToUhVC/fn2Ymppi06ZNMDQ0BADExsbC2dkZ0dHRuHTpkowjVBy85DflFya/RAqoTZs2UFFRwbhx47BlyxYcOXIEdnZ2WLNmDQBg+PDhCAwMxJUrV2QcKVHB0tTUxPXr13PUu9+9exe1a9fG+/fvZRSZ4uElvym/sICJSAFdu3YNZ86cQdWqVVGtWjWsXr0aQ4YMEdaaHT58OOrVqyfjKBVLUFAQbG1tc+yPjY3F8OHDsXXr1oIPSgGVK1cOUVFROZLf169f57jwBeUvXvKb8guTXyIFFBMTAwsLCwCAjo4OtLW1ha94AcDQ0BAJCQmyCk8hNWvWDEeOHBHWlgWAf//9Fy4uLqhSpYoMI1MsXl5eGDFiBKZOnSp8ALxy5QqmT5+OuXPnSnwVz5MQ81eTJk1kHQLJKZY9ECmgL9eU1dXVxe3bt1GqVCkAQFRUFIoUKcKvFQvQ2rVr4ebmht27d6NOnToYOnQojh49innz5mHQoEGyDk9hfH6lvS/XmP38Nr92z3+3b9/+6v1Vq1YtoEhI3nDml0hBOTs7C+tmpqSkwMXFBdra2gCA1NRUWYamkPr37w89PT107twZOjo6qFq1Ku7cuYMSJUrIOjSFwq/aCw9bW1te5ILyBWd+iRRQnz59vqvdhg0b8jkS+tLx48fx999/Y9GiRRg4cKCswyGSmcaNGyMoKAjjxo1Dz549c5z8VrJkSRlFRr87Jr9ERIWAq6ur8O///vsPV65cQe/evWFkZAQA8Pb2llVoCoUnHhYu+/fvx7hx42BgYIBFixZJ1MQT/SylbzchIqL8dvPmTWFTU1ND48aN8ezZM9y8eRNBQUGyDk9hNGvWDAEBARL7/v33X9jY2OD169cyikpxderUCcHBwejZsyccHBzQqVMnPHr0SNZh0W+OM79EREQf8MTDwisuLg4zZszAmjVr0LdvX/j4+Mg6JPpNMfklIiL6zO7du9G3b1/hxMO1a9fyxEMZMDQ0zPUiF0lJScjIyOAJb/TTuNoDERHRZ7p27QpdXV38/fff+Pvvv5n4yghndim/cOaXiIjoA554SCT/mPwSERF90KxZM6n3iUQinDlzpgCjodxkZGSgVatWAAA1NTX4+vrKOCL63TD5JSIiokKnevXqudb8isVi3L59Gzdu3ICSkhIv/00/jDW/RESF2Nu3b1G7dm0AgKmpKf777z8ZR0RUMDp06JDr/vT0dNy+fRvVqlUr2IBIbnDml4ioEPhYU/olsViM+Ph4xMTEQElJCbq6ugUcmWLp1KnTV+/fv39/AUVC0qSkpEBbW5urPdBP48wvEVEhEBcXBx8fH+jr6+fY7+rqmmM/5Y/Px3n79u2wt7fnB45CJrdSCKIfwZlfIqJCQElJCZGRkTAzM5PYHxUVhSJFinCWSwZ0dXVx69YtlC5dWtah0GdSU1OhpaXF9wT9NM78EhEVAiKRCAkJCdDV1YWmpqaswyGSuc+Xnfsck176VUx+iYgKAbFYjHLlygEAVFRUULJkSTRu3Bjt27eXcWREsnHz5k2p9zVu3LgAIyF5w+SXiKgQOHv2LIDsr3Tfvn2LsLAwnDt37psnYFHeWrJkifDvjIwMbNy4ESYmJsK+ESNGyCIshfTxPUGU11jzS0RUiO3btw9dunRB06ZNYWRkhL1798o6JLlWqlQpqfeJRCKEhYUVYDRElB+Y/BIRFWLp6em4dOkSgOyrWdWvX1/GEREVnOvXr2P37t0IDw9HWlqaxH1cdo5+lpKsAyAiIknPnz/H8+fPAQCqqqpo0qQJmjRpwsSXFMrOnTvRoEEDhISE4MCBA0hPT8e9e/dw5swZLv1Hv4Qzv0REhUBWVhZmzpyJhQsXIjExEUD2Ultubm6YMGEClJQ4V1FQnj9/jkOHDuU62+jt7S2jqBRP1apVMWjQIAwdOlRYdq5UqVIYNGgQLC0tMW3aNFmHSL8pnvBGRFQITJgwAevWrcOcOXPQsGFDAMDFixcxdepUpKSkYNasWTKOUDH4+fmhffv2KF26NEJDQ1G5cmU8ffoUYrEYNWrUkHV4CuXx48do164dgOySn6SkJIhEIowePRrNmzdn8ks/jVMJRESFwKZNm7B27VoMHjwYVatWRdWqVTFkyBCsWbMGGzdulHV4CsPT0xPu7u64c+cONDQ0sG/fPkRERKBJkybo0qWLrMNTKIaGhkhISAAAFC1aFHfv3gWQfdXD5ORkWYZGvzkmv0REhUBMTAwqVKiQY3+FChUQExMjg4gUU0hICBwdHQFkr7f8/v176OjoYPr06Zg7d66Mo1MsjRs3xqlTpwAAXbp0wciRIzFgwAD06NEDLVq0kHF09Dtj8ktEVAhUq1YNy5Yty7F/2bJlqFatmgwiUkza2tpCna+lpSUeP34s3PfmzRtZhaWQli1bhu7duwPILgtydXVFVFQUOnfujHXr1sk4OvqdseaXiKgQmDdvHtq1a4fTp08LqzpcvnwZEREROHbsmIyjUxz16tXDxYsXUbFiRbRt2xZubm64c+cO9u/fj3r16sk6PIViZPT/9u48qOrr/v/4637YvChcQCVoJcG1WNEqsVbtiJrWJaaumbrUxgXJqKnRRGuncaIGW6tx1KDV4MS4xGqDHUNSp2ZVQ7Vqxo3SuC84BcEtIYACXrzK7w9v+EnU9NuIHLif52PGGTiHP14jM87bc9/nfSIqv7YsS7/73e8MpoEvYdoDANQS+fn5WrlypU6cOCFJatu2rZ577jk1bdrUcDL7yM7O1rVr19ShQweVlJRoxowZ2rt3r1q3bq2lS5fqscceMx3RVv75z3/q5s2b6tmzp06ePKlt27YpPj5evXr1Mh0NdRjFLwAAqHWWLFmil156SQEBAZo9e7aWLFmi6OhoHTlyRKtWrVJiYqLpiKij6PkFgFrglVde0a1bt+5aLyoq0qhRowwksredO3dqxYoVWrlypTIyMkzHsaXU1FT9+c9/1p49ezRnzhwlJyfr8OHDWrx4sVJSUkzHQx3GyS8A1ALR0dGKjo7Wxo0b1aJFC0lSRkaGxowZo6ioKO3fv99wQt/k8XiUkJCg999/X2FhYcrLy9PQoUOVmZmpZs2aSbr96EV8fLzee+89NWnSxHBi+6hXr57OnDmjZs2aKTg4WIcPH1ZsbKzOnTun9u3bVz4GA/yvOPkFgFrg3//+t5o1a6aOHTtq9erVmjlzpvr27atnnnlGe/fuNR3PZ/n7++vzzz+vHCc3depUBQYG6uzZszp37pzOnTunM2fOKCAgQFOnTjWc1l5cLpcKCwslSWPHjlWjRo0kSeXl5QoKCjKYDHUd0x4AoBYIDw/XX//6V82aNUsTJ06Uv7+/PvjgA+aZ1oBGjRrJ7XZLkj755BNlZGTo0Ucfrdx/7LHHlJKSwu+ihvXr10+5ubmKi4tTampq5fpnn32mDh06GEyGuo6TXwCoJf70pz9p2bJlGjVqlFq0aKGpU6cqKyvLdCyf165dO+3evVvS7Y/aHQ7HXT/jcDju2ZONh2fDhg168skn71ofMmSI3n33XQOJ4CsofgGgFujfv7+Sk5P11ltvadOmTcrMzFRCQoK6du2qRYsWmY7n0yZMmKCXX35Zx48fV58+ffTCCy8oPz+/cj8vL0/Tp09X7969DabE11wul8LCwkzHQB3GhTcAqAX69Omjt956666Zvtu2bVNSUpIuXLhgKJk9LFiwQAsWLFDLli2VlZUlf39/RUdHS5Jyc3MVGxurrVu3KiYmxmxQGxk2bNi37qenp9dQEvgail8AqOW++OKLyss+eHjOnTunjz76SFeuXKlscQgPD1dsbKz69Olzz3YIPDyWZSkkJESDBw+Wn5/fXfvr1q0zkAq+gOIXAGqRQ4cO6fjx45KkH/zgB4qPjzecCDBjx44dmjFjhm7cuFH5/DdQHSh+AaAWuHz5skaOHKmMjIzKfsbCwkL17t1baWlpaty4sdmAgAEVFRVav369Zs+erdjYWC1dupRJD3hgXHgDgFrg+eef19WrV3X06FEVFBSooKBAR44cUXFxMfNlYVsOh0Pjx4/X6dOnlZCQoISEBCUmJla5kAj8rzj5BYBawOVyafv27frRj35UZX3//v3q27dv5bB/wC6WL19+11p+fr5WrlwpSbp69WpNR4KP4JELAKgFbt26pYCAgLvWAwICmC8LW3rttdfuuc7lTzwoTn4BoBYYPHiwCgsL9fbbb1eOO8vLy9Po0aMVHh7OUH9DDh06pKysLMXFxalLly6m4wCoBhS/AFAL5ObmatCgQTp69GiV+bJxcXHaunWrmjVrZjih/axbt05JSUlq2LChCgoKtGLFCk2aNMl0LAAPiOIXAGqJiooKbd++XSdOnJAktW3bVj/72c8Mp7Kvjh07auzYsXrxxRe1efNmzZkzRydPnjQdyzZ45AIPC8UvANRiFRUVys3NlST5+fnpe9/7nuFE9hEREaF9+/bp+9//vq5du6aGDRvK7XabjmUblmVp+PDhcjqd99znkQt8V1x4A4Ba7PLly2revLkqKioUFRXFiKcadOPGDQUGBkq6ffHQ4/EYTmQ/y5cvV2RkpOkY8DEUvwBgUERExLfuf/3hHBMfasadH7Vfv35dkyZNUv369fn7N8DhcPCkNB4K2h4AwCDLspSSkiKXy3XP/cLCQk2fPl03b96s4WT2NG7cuG8tuPioveZYlqW4uDg5nU7Vr19fTZs2VadOnTRixAgugOKBUPwCgEGWZenixYv3/Wj30qVLatq0KcUvbCc5OVkVFRW6ceOGiouLlZ+fr8OHD+vKlSv65JNP1K1bN9MRUUdR/AKAQRS/tcsTTzyh9PR0hYWFmY6Ce6ioqFBSUpKys7P16aefmo6DOoqeXwAwbN++fYqIiFBQUJBCQkLUpEkTii9DMjIyVF5ebjoG7sPhcCg5OVnLli0zHQV1GCe/AGCQZVlVvv+637Rx48bq3r27+vXrp+eee46T3xpiWZYuXbqkxo0bm44C4CGh+AUAg4qKiiRJHo9HbrdbBQUFysvL07Fjx7Rjxw598MEHkkTxW0Msy1L37t0rR5x9086dO2s4kX1Nnz79vnsOh0NLliypwTTwJRS/AFCLrVmzRs8++6x69eqliIgIbdmyxXQkn2ZZlmbMmKEGDRrcc3/u3Lk1nMi+LMtSSEiIHn/8cX2zVHE4HPxHBN8ZPb8AUIuNHj1a/v63/6m+30tXqD4Oh0MzZ87kYYVaYPXq1ZozZ478/f21ZMkStW/f3nQk+AhOfgEA8Ppv0zdQs0pLS/Xqq6/qtdde04gRI/SHP/xBjzzyiOlYqOOs//4jAADYw9y5c+/b8oCaFxwcrOTkZJ08eVI3b95UmzZtNG/ePJWVlZmOhjqMk18AALzOnTsnj8ej1q1bV1k/ffq0AgICFBMTYyaYDW3duvWutczMTC1evFgul0vnz583kAq+gOIXAACvnj17KjExUWPHjq2yvnHjRr355pvKyMgwE8yGvjkG8Jtu3bpVQ0ngayh+AQDwCg0N1eHDh9WqVasq62fOnFHnzp1VWFhoJhiAakPPLwAAXg6HQ1evXr1rvaioiFnLNWzDhg1yu92mY8AHcfILAIDXwIED5XQ69fbbb8vPz0/S7QdGRowYoZKSkspHR/Dw+fn56cKFC0zeQLWj+AUAwOvYsWNKSEhQWFiYevToIUnavXu3iouLtXPnTsXFxRlOaB+MncPDQvELAMAd8vPztWLFCmVlZcnpdKpDhw6aMmWKIiIiTEezFcuytHz5coWGht5zf8yYMTWcCL6C4hcAANQ6lmWpWbNmle0nd3I4HMrOzjaQCr6A4hcAgDsUFhZqzZo1On78uCSpXbt2SkxMlMvlMpzMXmh7wMNC8QsAgNfBgwfVr18/OZ1OdenSRZJ04MABlZWV6eOPP1Z8fLzhhPZB8YuHheIXAACvHj16qFWrVlq9erX8/f0lSR6PR0lJScrOztauXbsMJ7SP5s2b6+DBg2rYsKHpKPAxFL8AAHg5nU5lZmYqNja2yvqxY8fUuXNnlZaWGkpmP9nZ2WrRooXpGPBBPHIBAIBXaGiocnJy7lrPzc1VSEiIgUT21apVK/Xu3VsbN27U9evXTceBD6H4BQDAa8SIEZowYYI2b96s3Nxc5ebmKi0tTUlJSRo1apTpeLZy+PBhdejQQdOnT1dUVJQmTpyo/fv3m44FH0DbAwAAXuXl5Zo5c6ZWrVolj8cjSQoICNDkyZO1cOFCBQUFGU5oPx6PR1u3btX69ev14Ycfqk2bNkpMTNQzzzyjxo0bm46HOojiFwCAbygtLdXZs2clSS1btlRwcLDhRHC73Xr99df10ksvqby8XIGBgRo+fLheffVVNWnSxHQ81CEUvwAAoNY6ePCg1q5dq7S0NNWvX19jx47VhAkTdP78eSUnJ6u4uJh2CPxPKH4BAPAaNmzYt+6np6fXUBIsXbpU69at08mTJzVgwAAlJSVpwIABsqz/f13p/PnziomJqWxRAf4v/E0HAACgtrjzFbe//OUvGjhwIFMeDElNTVViYqLGjRt337aGyMhIrVmzpoaToa7j5BcAgHsICQlRVlYWs2YBH8OoMwAAANgGxS8AAABsg55fAAC8li9fXvm1x+PR+vXr1ahRo8q1qVOnmogFoBrR8wsAgFfz5s3vu+dwOJSdnV2DaQA8DBS/AACg1nn99deVmJioevXqmY4CH0PPLwAAXvPmzVNpaanpGJD0/PPPq7i42HQM+CBOfgEA8PLz89OFCxcUGRlpOortWZalixcv8rtAtePkFwAAL86DAN/HtAcAAO6wePFiNWjQ4J57c+bMqeE09vbRRx9VeXXvToMGDarhNPAVtD0AAOBlWZa6deumwMDAu/YcDod27txpIJU9Wdb9P5x2OBy6efNmDaaBL6H4BQDAiz5TwPfR8wsAAADboOcXAACvnj173rPlAWaUlJToH//4h3JyclReXl5lj9f28F3R9gAAAGqdzMxMDRgwQKWlpSopKVFERIS++OILBQcHKzIyktf28J1R/AIA4DVs2LBv3U9PT6+hJOjVq5fatGmjVatWyeVyKSsrSwEBAfrVr36ladOm/dffFXA/9PwCAOD13nvvKTAwUC6XSy6XS9u2bZNlWZXfo+b861//0owZM2RZlvz8/OR2uxUdHa1FixZp1qxZpuOhDqPnFwCAOyxfvrxy2sOWLVu0aNEitWjRwnAq+wkICKgcdxYZGamcnBy1bdtWLpdLubm5htOhLuPkFwAAr3r16un69euSbr/2Vl5ermXLljFT1oBOnTrpwIEDkm5fRJwzZ442bdqkF154QXFxcYbToS6j+AUAwKtNmzZKSUnRxYsXlZKSopCQEB06dEi9e/fWpUuXTMezlT/+8Y9q0qSJJGn+/PkKDw/X5MmTdeXKFb3xxhuG06Eu48IbAABef//73zVy5EiVlZXJ399fqampGjNmjKZPn64tW7YoPz/fdEQAD4jiFwCAOxQVFenUqVOKjo5WVFRU5XpaWppGjhxpMBmA6kDxCwAAap2IiIhv3S8oKKihJPA1THsAAOAOhYWFWrNmjY4fPy5JateunRITExl1VsMKCwuVkpLC3zuqHSe/AAB4HTx4UP369ZPT6VSXLl0kSQcOHFBZWZk+/vhjxcfHG05oH5Zl6eLFi5Vj54DqQvELAIBXjx491KpVK61evVr+/rc/HPV4PEpKSlJ2drZ27dplOKF9+Pn56dSpU2ratKmcTqfpOPAhFL8AAHg5nU5lZmYqNja2yvqxY8fUuXNnlZaWGkpmP5ZlyeFwVH79yCOPqFOnTkpMTNTQoUMNp0NdRs8vAABeoaGhysnJuav4zc3NVUhIiKFU9vTpp5+qoqJCN27cUHFxsfLz83XgwAENHz5cb7zxhsaPH286Iuooil8AALxGjBihCRMmaPHixerevbskac+ePZo5c6ZGjRplOJ299OzZ857r8fHxWrp0KcUvvjOKXwAAvBYvXiyHw6ExY8bI4/FIkgICAjR58mQtXLjQcDpI0rPPPquYmBjTMVCH0fMLAMA3lJaW6uzZs5Kkli1bKjg42HAieyorK9ORI0dUr149xcXFVfYAAw/CMh0AAIDaJjg4WGFhYQoLC6PwNSQ1NVWRkZHq2rWrOnbsqObNm2vPnj2mY8EHUPwCAODl8Xg0e/ZsuVwuxcTEKCYmRi6XSy+//LJu3LhhOp5tbN++Xb/+9a/1i1/8Qu+88442bNigRx99VIMHD9aXX35pOh7qONoeAADwmjx5stLT0zVv3jx169ZNkrRv3z698sorGjJkiFJTUw0ntIcBAwYoKipKa9eurVzzeDxq3769JkyYoN/85jcG06Guo/gFAMDL5XIpLS1NTz75ZJX1999/X6NGjVJRUZGhZPYSGhqqDz/8sHLixtcWLFigzz77TH/7298MJYMvYNoDAABeQUFB95wk0Lx5cwUGBtZ8IBsaNGiQSkpKNH/+fAUFBVXZy8vL05kzZ/T000/rnXfeMZQQdR09vwAAeE2ZMkW///3v5Xa7K9fcbrfmz5+vKVOmGExmH2FhYZKkkJAQuVyuKn+cTqf8/f3lcrnMhkSdRtsDAABeQ4cO1Y4dOxQUFKQf/vCHkqSsrCyVl5frpz/9aZWfTU9PNxHRFqKjo7Vp0yYlJCRUWZ83b57OnDmjDRs2GEoGX0DbAwAAXmFhYXr66aerrEVHRxtKY18/+clPtGzZsirFb2lpqdauXatZs2YZTAZfwMkvAACoVY4ePaqOHTvqiSee0FNPPaXS0lJt2LBB169f18mTJxUQEGA6Iuowil8AAO7g8XiUkZGhs2fP6pe//KVCQkKUn5+v0NBQNWjQwHQ823j33Xf14osvKicnR5L04x//WG+++abatWtnOBnqOopfAAC8/vOf/6h///7KycmR2+3WqVOn1KJFC02bNk1ut1urVq0yHdF2Ll++rMDAwMqLcMCDYtoDAABe06ZNU+fOnfXVV1/J6XRWrn99EQ41LzIyksIX1YoLbwAAeO3evVt79+69a6ZvTEyM8vLyDKUCUJ04+QUAwOvWrVu6efPmXevnz59XSEiIgUQAqhvFLwAAXn379lVKSkrl9w6HQ9euXdPcuXM1YMAAc8EAVBsuvAEA4HX+/Hn169dPFRUVOn36tDp37qzTp0+rUaNG2rVrlyIjI01HBPCAKH4BALiDx+PR5s2blZWVpWvXrik+Pl6jR4+ucgEOQN1F8QsAAADbYNoDAABeX375pRo2bChJys3N1erVq1VWVqaBAwdWeWoXQN3FyS8AwPY+//xzDRw4ULm5uWrdurXS0tLUv39/lZSUyLIslZSUaMuWLRoyZIjpqAAeENMeAAC299vf/lbt27fXrl271KtXL/385z/XU089paKiIn311VeaOHGiFi5caDomgGrAyS8AwPYaNWqknTt3qkOHDrp27ZpCQ0N14MABPf7445KkEydOqGvXriosLDQbFMAD4+QXAGB7BQUFioqKkiQ1aNBA9evXV3h4eOV+eHi4rl69aioegGpE8QsAgG4/aPFt3wPwDUx7AABA0rhx4xQUFCRJun79uiZNmqT69etLktxut8loAKoRPb8AANsbP378/+nn1q1b95CTAHjYKH4BAABgG/T8AgAAwDYofgEAAGAbFL8AAACwDYpfAAAA2AbFLwAAAGyD4hcAAAC2QfELAAAA26D4BQAAgG1Q/AIAAMA2/h/2yaYncD3jJQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "corr_matrix = hist_credit_card[['Возраст', 'Доход клиента', 'Вероятность банкротства', 'Флаг утилизации счёта']].corr()\n",
    "sns.heatmap(corr_matrix, annot = True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ppKTOyF5Roeh",
   "metadata": {
    "id": "ppKTOyF5Roeh"
   },
   "source": [
    "Мы ищем прокси-метрику для нашего теста, соответственно она должна быть однонаправленна с целевой метрикой. Наивысшая положительная корреляция с **Возрастом**. Выберем этот параметр для проверки на гомогенность."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b40ecbb",
   "metadata": {
    "id": "9b40ecbb"
   },
   "source": [
    "### 4.1. Каков размер выборки? Привести аргументацию и написать как получилось то или иное число."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "nqR2gk72K8Zh",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nqR2gk72K8Zh",
    "outputId": "ad7c8614-4183-42d9-bee2-257014e78b53"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    123255.000000\n",
       "mean          0.741544\n",
       "std           0.437788\n",
       "min           0.000000\n",
       "25%           0.000000\n",
       "50%           1.000000\n",
       "75%           1.000000\n",
       "max           1.000000\n",
       "Name: Флаг утилизации счёта, dtype: float64"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist_credit_card['Флаг утилизации счёта'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ZEHLGMU-L8H6",
   "metadata": {
    "id": "ZEHLGMU-L8H6"
   },
   "source": [
    "##### Расчёт MDE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "UGik1pqYlnOQ",
   "metadata": {
    "id": "UGik1pqYlnOQ"
   },
   "outputs": [],
   "source": [
    "lift = 0.03\n",
    "mu = hist_credit_card['Флаг утилизации счёта'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "5XmjZKmAlnQx",
   "metadata": {
    "id": "5XmjZKmAlnQx"
   },
   "outputs": [],
   "source": [
    "MDE = lift * mu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "amKRo6g4mXR3",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "amKRo6g4mXR3",
    "outputId": "b05125bc-3acc-4b23-a369-45411906d566"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.02224631860776439"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MDE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "TM6tFOE7mfVm",
   "metadata": {
    "id": "TM6tFOE7mfVm"
   },
   "source": [
    "#### Размер выборки"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "T3IHQxTgmsfA",
   "metadata": {
    "id": "T3IHQxTgmsfA"
   },
   "source": [
    "У нас бинарное распределение, односторонняя альтернатива. На исторических данных вычислим вероятность р0 того, что флаг примет значение 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8LBZB6PmmXUx",
   "metadata": {
    "id": "8LBZB6PmmXUx"
   },
   "outputs": [],
   "source": [
    "p0 = hist_credit_card['Флаг утилизации счёта'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9-FMjEuDmXYy",
   "metadata": {
    "id": "9-FMjEuDmXYy"
   },
   "outputs": [],
   "source": [
    "n = (2 * p0 * (1 - p0) * (norm.ppf(0.95) - norm.ppf(0.15)) ** 2) / MDE ** 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "LmKssnwJmXcw",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LmKssnwJmXcw",
    "outputId": "813306d9-b058-466b-8812-7c9299b2ca28"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5569"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "math.ceil(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13ea8ee",
   "metadata": {
    "id": "b13ea8ee"
   },
   "source": [
    "### 4.2. Необходимо запросить у бота в Телеграме выборку полученного размера командой `/get_project2_sample {размер выборки}` и вписать <ins>код выборки</ins>:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "F5c-oqIBjVJS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "F5c-oqIBjVJS",
    "outputId": "6bce7770-e9ea-4298-ff6c-44930f3f68f0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19511"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "19511"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6945c8f",
   "metadata": {
    "id": "f6945c8f"
   },
   "source": [
    "### 5. Проверка на однородность, применение критерия. Принятие решения. Расписать подробно с аргументами."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "LVlEnA4DjWp5",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 108
    },
    "id": "LVlEnA4DjWp5",
    "outputId": "c9a4809f-1e78-4185-918e-20f32f3cca94"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-685b6457-6b16-43fd-a18a-7ba5cd356ae7\" name=\"files[]\" multiple disabled\n",
       "        style=\"border:none\" />\n",
       "     <output id=\"result-685b6457-6b16-43fd-a18a-7ba5cd356ae7\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script>// Copyright 2017 Google LLC\n",
       "//\n",
       "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
       "// you may not use this file except in compliance with the License.\n",
       "// You may obtain a copy of the License at\n",
       "//\n",
       "//      http://www.apache.org/licenses/LICENSE-2.0\n",
       "//\n",
       "// Unless required by applicable law or agreed to in writing, software\n",
       "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
       "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
       "// See the License for the specific language governing permissions and\n",
       "// limitations under the License.\n",
       "\n",
       "/**\n",
       " * @fileoverview Helpers for google.colab Python module.\n",
       " */\n",
       "(function(scope) {\n",
       "function span(text, styleAttributes = {}) {\n",
       "  const element = document.createElement('span');\n",
       "  element.textContent = text;\n",
       "  for (const key of Object.keys(styleAttributes)) {\n",
       "    element.style[key] = styleAttributes[key];\n",
       "  }\n",
       "  return element;\n",
       "}\n",
       "\n",
       "// Max number of bytes which will be uploaded at a time.\n",
       "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
       "\n",
       "function _uploadFiles(inputId, outputId) {\n",
       "  const steps = uploadFilesStep(inputId, outputId);\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  // Cache steps on the outputElement to make it available for the next call\n",
       "  // to uploadFilesContinue from Python.\n",
       "  outputElement.steps = steps;\n",
       "\n",
       "  return _uploadFilesContinue(outputId);\n",
       "}\n",
       "\n",
       "// This is roughly an async generator (not supported in the browser yet),\n",
       "// where there are multiple asynchronous steps and the Python side is going\n",
       "// to poll for completion of each step.\n",
       "// This uses a Promise to block the python side on completion of each step,\n",
       "// then passes the result of the previous step as the input to the next step.\n",
       "function _uploadFilesContinue(outputId) {\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  const steps = outputElement.steps;\n",
       "\n",
       "  const next = steps.next(outputElement.lastPromiseValue);\n",
       "  return Promise.resolve(next.value.promise).then((value) => {\n",
       "    // Cache the last promise value to make it available to the next\n",
       "    // step of the generator.\n",
       "    outputElement.lastPromiseValue = value;\n",
       "    return next.value.response;\n",
       "  });\n",
       "}\n",
       "\n",
       "/**\n",
       " * Generator function which is called between each async step of the upload\n",
       " * process.\n",
       " * @param {string} inputId Element ID of the input file picker element.\n",
       " * @param {string} outputId Element ID of the output display.\n",
       " * @return {!Iterable<!Object>} Iterable of next steps.\n",
       " */\n",
       "function* uploadFilesStep(inputId, outputId) {\n",
       "  const inputElement = document.getElementById(inputId);\n",
       "  inputElement.disabled = false;\n",
       "\n",
       "  const outputElement = document.getElementById(outputId);\n",
       "  outputElement.innerHTML = '';\n",
       "\n",
       "  const pickedPromise = new Promise((resolve) => {\n",
       "    inputElement.addEventListener('change', (e) => {\n",
       "      resolve(e.target.files);\n",
       "    });\n",
       "  });\n",
       "\n",
       "  const cancel = document.createElement('button');\n",
       "  inputElement.parentElement.appendChild(cancel);\n",
       "  cancel.textContent = 'Cancel upload';\n",
       "  const cancelPromise = new Promise((resolve) => {\n",
       "    cancel.onclick = () => {\n",
       "      resolve(null);\n",
       "    };\n",
       "  });\n",
       "\n",
       "  // Wait for the user to pick the files.\n",
       "  const files = yield {\n",
       "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
       "    response: {\n",
       "      action: 'starting',\n",
       "    }\n",
       "  };\n",
       "\n",
       "  cancel.remove();\n",
       "\n",
       "  // Disable the input element since further picks are not allowed.\n",
       "  inputElement.disabled = true;\n",
       "\n",
       "  if (!files) {\n",
       "    return {\n",
       "      response: {\n",
       "        action: 'complete',\n",
       "      }\n",
       "    };\n",
       "  }\n",
       "\n",
       "  for (const file of files) {\n",
       "    const li = document.createElement('li');\n",
       "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
       "    li.append(span(\n",
       "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
       "        `last modified: ${\n",
       "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
       "                                    'n/a'} - `));\n",
       "    const percent = span('0% done');\n",
       "    li.appendChild(percent);\n",
       "\n",
       "    outputElement.appendChild(li);\n",
       "\n",
       "    const fileDataPromise = new Promise((resolve) => {\n",
       "      const reader = new FileReader();\n",
       "      reader.onload = (e) => {\n",
       "        resolve(e.target.result);\n",
       "      };\n",
       "      reader.readAsArrayBuffer(file);\n",
       "    });\n",
       "    // Wait for the data to be ready.\n",
       "    let fileData = yield {\n",
       "      promise: fileDataPromise,\n",
       "      response: {\n",
       "        action: 'continue',\n",
       "      }\n",
       "    };\n",
       "\n",
       "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
       "    let position = 0;\n",
       "    do {\n",
       "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
       "      const chunk = new Uint8Array(fileData, position, length);\n",
       "      position += length;\n",
       "\n",
       "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
       "      yield {\n",
       "        response: {\n",
       "          action: 'append',\n",
       "          file: file.name,\n",
       "          data: base64,\n",
       "        },\n",
       "      };\n",
       "\n",
       "      let percentDone = fileData.byteLength === 0 ?\n",
       "          100 :\n",
       "          Math.round((position / fileData.byteLength) * 100);\n",
       "      percent.textContent = `${percentDone}% done`;\n",
       "\n",
       "    } while (position < fileData.byteLength);\n",
       "  }\n",
       "\n",
       "  // All done.\n",
       "  yield {\n",
       "    response: {\n",
       "      action: 'complete',\n",
       "    }\n",
       "  };\n",
       "}\n",
       "\n",
       "scope.google = scope.google || {};\n",
       "scope.google.colab = scope.google.colab || {};\n",
       "scope.google.colab._files = {\n",
       "  _uploadFiles,\n",
       "  _uploadFilesContinue,\n",
       "};\n",
       "})(self);\n",
       "</script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving Контроль.csv to Контроль (2).csv\n",
      "Saving Тест.csv to Тест (2).csv\n"
     ]
    }
   ],
   "source": [
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "-90fLBdEoOmF",
   "metadata": {
    "id": "-90fLBdEoOmF"
   },
   "outputs": [],
   "source": [
    "test_group = pd.read_csv(io.BytesIO(uploaded['Тест.csv']))\n",
    "control_group = pd.read_csv(io.BytesIO(uploaded['Контроль.csv']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "L9rMfAJwoWeP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 143
    },
    "id": "L9rMfAJwoWeP",
    "outputId": "54d66bce-caba-45bf-dd9b-f5641f0623a5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-d60c30eb-56c5-4f27-a47d-f757f122e493\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Возраст</th>\n",
       "      <th>Доход клиента</th>\n",
       "      <th>Вероятность банкротства</th>\n",
       "      <th>Флаг утилизации счёта</th>\n",
       "      <th>Расходы</th>\n",
       "      <th>PV КК</th>\n",
       "      <th>PV услуги</th>\n",
       "      <th>NPV</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "      <td>20855.797437</td>\n",
       "      <td>0.017804</td>\n",
       "      <td>1</td>\n",
       "      <td>193</td>\n",
       "      <td>4969</td>\n",
       "      <td>2818</td>\n",
       "      <td>7594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>69</td>\n",
       "      <td>32627.896361</td>\n",
       "      <td>0.035978</td>\n",
       "      <td>1</td>\n",
       "      <td>2484</td>\n",
       "      <td>5336</td>\n",
       "      <td>2716</td>\n",
       "      <td>5568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>60</td>\n",
       "      <td>27385.898097</td>\n",
       "      <td>0.385621</td>\n",
       "      <td>0</td>\n",
       "      <td>54</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-54</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d60c30eb-56c5-4f27-a47d-f757f122e493')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-d60c30eb-56c5-4f27-a47d-f757f122e493 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-d60c30eb-56c5-4f27-a47d-f757f122e493');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   ID  Возраст  Доход клиента  Вероятность банкротства  Флаг утилизации счёта  \\\n",
       "0   0       18   20855.797437                 0.017804                      1   \n",
       "1   1       69   32627.896361                 0.035978                      1   \n",
       "2   2       60   27385.898097                 0.385621                      0   \n",
       "\n",
       "   Расходы  PV КК  PV услуги   NPV  \n",
       "0      193   4969       2818  7594  \n",
       "1     2484   5336       2716  5568  \n",
       "2       54      0          0   -54  "
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_group.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98rSMs9LX18R",
   "metadata": {
    "id": "98rSMs9LX18R"
   },
   "source": [
    "#### Проверка на однородность"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2V56nRlhtGYI",
   "metadata": {
    "id": "2V56nRlhtGYI"
   },
   "source": [
    "Проверим гомогенность наших групп по признаку **возраст**\n",
    "\n",
    "**H0**: выборки пришли из одного распределения\n",
    "\n",
    "**H1**: выборки из разных распределений\n",
    "\n",
    "Будем использовать *критерий Anderson*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "CBpJJDUnrdAq",
   "metadata": {
    "id": "CBpJJDUnrdAq"
   },
   "outputs": [],
   "source": [
    "from scipy.stats import laplace, norm, ks_2samp, anderson_ksamp, cramervonmises_2samp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "QnSPEJt1rdb1",
   "metadata": {
    "id": "QnSPEJt1rdb1"
   },
   "outputs": [],
   "source": [
    "x = control_group['Возраст'].copy()\n",
    "y = test_group['Возраст'].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "L_TRAqFwrnh2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "L_TRAqFwrnh2",
    "outputId": "309c73a0-fac1-4818-c276-9e6cff0ffafa"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-65-5bacac867c02>:1: UserWarning: p-value capped: true value larger than 0.25\n",
      "  anderson_ksamp([x, y])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Anderson_ksampResult(statistic=-1.0183622926072189, critical_values=array([0.325, 1.226, 1.961, 2.718, 3.752, 4.592, 6.546]), pvalue=0.25)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anderson_ksamp([x, y])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vRwxoUrltf6S",
   "metadata": {
    "id": "vRwxoUrltf6S"
   },
   "source": [
    "Р-value = более 0.25, что не позволяет отвергнуть нулевую гипотезу при заданном уровне значимости. Соответственно, выборки пришли из одного распределения."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "FV8PsiStX7TD",
   "metadata": {
    "id": "FV8PsiStX7TD"
   },
   "source": [
    "#### Применение критерия\n",
    "\n",
    "имеем бинарное распределение, попробуем использовать Z-test Фишера в качестве критерия "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "LDKfq41rroF5",
   "metadata": {
    "id": "LDKfq41rroF5"
   },
   "outputs": [],
   "source": [
    "# proportions_ztest ([s1, s2], [n1, n2], alternative = )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "WrqMvcQ4Y0N8",
   "metadata": {
    "id": "WrqMvcQ4Y0N8"
   },
   "outputs": [],
   "source": [
    "s1 = control_group['Флаг утилизации счёта'].sum()\n",
    "n1 = control_group['Флаг утилизации счёта'].count()\n",
    "s2 = test_group['Флаг утилизации счёта'].sum()\n",
    "n2 = test_group['Флаг утилизации счёта'].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "2zEYSTvNZHrS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2zEYSTvNZHrS",
    "outputId": "e58b708e-43c6-43ef-865a-147f2be15ae5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-0.9961457864657371, 0.15958965800129604)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from statsmodels.stats.proportion import proportions_ztest\n",
    "proportions_ztest ([s1, s2], [n1, n2], alternative = 'smaller')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ws9mhziad4e",
   "metadata": {
    "id": "4ws9mhziad4e"
   },
   "source": [
    "U-критерий Манна-Уитни"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "SBn9gXy4ZRi1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SBn9gXy4ZRi1",
    "outputId": "01f51df3-f249-4e22-94d3-db92f8960485"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1596014652646937"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u1, p = mannwhitneyu(control_group['Флаг утилизации счёта'], test_group['Флаг утилизации счёта'], alternative=\"less\")\n",
    "p"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33As-4H-aivP",
   "metadata": {
    "id": "33As-4H-aivP"
   },
   "source": [
    "p-value более 15%, это не позволяет отвергнуть нулевую гипотезу о том, что тест и контроль не имеют статзначимых различий.."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}