06Sep2022_RemoteDeployment_Lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W_RuDIS7w9SF"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/datarobot-community/DRU-MLOps/blob/master/11May2022_RemoteDeployment_Lab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pcHrfkRkwUDC"
   },
   "source": [
    "# **MLOps Remote Deployment - Laboratory**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cLTGmStO-gGe"
   },
   "source": [
    "Welcome to the MLOps Remote Deployment Monitoring Hands-On Lab!\n",
    "\n",
    "**Pre-requisites:**\n",
    "1. You will need a DataRobot account and an API key.  \n",
    "2. Add your API Key to the first cell in the notebook. The API Key is found in the Developer Tools which is located on the profile icon in the DataRobot GUI App.\n",
    "3. Once you create a model package and deploy it, you will need the model ID and deployment ID\n",
    "\n",
    "\n",
    "**Documentation:**\n",
    "\n",
    "The MLOps Agent tarball includes documentation in the /docs folder.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TiQhSVSe8d7R"
   },
   "source": [
    "### ***You will complete certain lines of code in this notebook to provide the necessary functionality!***\n",
    "\n",
    "HINTS: \n",
    "* Shell commands that take no parameters are shown as ___\n",
    "* API calls that take no parameters are shown as \\_\\_\\_()\n",
    "* API calls that take 1 parameter are shown as \\_\\_\\_(\\_\\_\\_)\n",
    "* API calls that take 2 parameters are shown as \\_\\_\\_(\\___ , \\_\\__)\n",
    "* You get the idea.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 0.- Some necessary Python modules will be installed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install folium==0.2.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### You will get an error message when installing the next module - please disregard it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U 'boto3<2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install 'urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KmXI8EwnKs2R"
   },
   "source": [
    "# 1.- Create and deploy a remote model package via the GUI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5p2RSJQekkJS"
   },
   "source": [
    "This is something that you have done already via the MLOps GUI."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n73VMdJBI8-t"
   },
   "source": [
    "# 2.- Specify Model ID and Deployment ID\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ooOxNPA0pWhR"
   },
   "source": [
    "We need to supply the Deployment ID and Model ID found in the code sample provided in MLOps under \"Predictions\" -> \"Monitoring\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "La924wIpzE0q"
   },
   "outputs": [],
   "source": [
    "DEPLOYMENT_ID = \"\"\n",
    "MODEL_ID = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AP50QKUSlf3N"
   },
   "source": [
    "# 3.- Add your API_KEY and the location of the DataRobot instance you are using.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xO-YkhlNMJ0d"
   },
   "outputs": [],
   "source": [
    "import yaml\n",
    "import requests\n",
    "import re\n",
    "API_KEY = \"\"\n",
    "DR_URL = \"https://app.datarobot.com\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hH8kNamy0fQe"
   },
   "source": [
    "The following two shell commands will show you \\\n",
    "a) where we are within the Colab runtime and \\\n",
    "b) what is contained within it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SKt1hmm_CXlF"
   },
   "outputs": [],
   "source": [
    "%pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CQYHjeTOCZrj"
   },
   "outputs": [],
   "source": [
    "%ls -al"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qSB9WHitSpg8"
   },
   "source": [
    "# 4.- Download the MLOps Agent tarball to the local Colab directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "INs7K0_Fpjax"
   },
   "outputs": [],
   "source": [
    "# This cell downloads the MLOps Agent tarball\n",
    "url = DR_URL + \"/api/v2/mlopsInstaller\"\n",
    "\n",
    "headers = {'Authorization': 'Bearer {}'.format(API_KEY)}\n",
    "response = requests.request(\"GET\", url, headers=headers)\n",
    "if 'UNAUTHORIZED' in response.reason:\n",
    "    print('Put your real API key in')\n",
    "with open(\"mlops-agents.tar.gz\", \"wb\") as f:\n",
    "    f.write(response.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "bCy3QeyFZ7rW"
   },
   "outputs": [],
   "source": [
    "# We grab the tarball's filename - we need the version number of the tarball\n",
    "d = response.headers['content-disposition']\n",
    "fname = re.findall(\"filename=(.+).tar.gz\", d)[0]\n",
    "n = fname.rfind(\"-\")\n",
    "filename = fname[:n]\n",
    "filename"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_DYJydZ-Cdey"
   },
   "outputs": [],
   "source": [
    "%ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7UFVSkdw1qpJ"
   },
   "source": [
    "As shown by the output of the previous shell command, we now have the MLOps Agent tarball within the runtime."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gk7-p5MR8dTR"
   },
   "source": [
    "# 5.- Untar the MLOPs Agent tarball, and then create a tmp directory to spool the predictions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gVWLKh-CMQBv"
   },
   "outputs": [],
   "source": [
    "# Untar the tarball\n",
    "!tar -xvf /content/mlops-agents.tar.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "A4ZF00o-8dTW"
   },
   "outputs": [],
   "source": [
    "# Here we create the directory where the spool file will be located\n",
    "# This is where the MLOps Agent will look for prediction data\n",
    "%cd $filename\n",
    "!mkdir -p /tmp/ta\n",
    "%ls -al"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iFDxyVAx8dTZ"
   },
   "source": [
    "# 6.-  Install the MLOps libraries.\n",
    "\n",
    "### We need to install the correct versions of the MLOps Agent Libraries, i.e., those found within the latest version of the MLOps Agentt tarball.\n",
    "\n",
    "### First, we are going to use a bit of regex to extract the latest version number from the name of the tarball"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "version_number = re.search(\"\\d+(\\.\\d+)+\", filename).group(0)\n",
    "version_number"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We will now assemble the names of the Python Wheel files that will be used for the install (please remember these are found within the /lib directory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlops_library_wheel = \"lib/datarobot_mlops-\" + version_number + \"-py2.py3-none-any.whl\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlops_client_wheel = \"lib/datarobot_mlops_connected_client-\" + version_number + \"-py3-none-any.whl\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### In order to invoke these names from the command line, we will expose them as environment variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!export mlops_library_wheel\n",
    "!export mlops_client_wheel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We now install the MLOps Agent library and the MLOps Connected Client library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vZmRJck88dTa"
   },
   "outputs": [],
   "source": [
    "!pip install $mlops_library_wheel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install $mlops_client_wheel "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vHnHZ-bh8dTd"
   },
   "source": [
    "# 7.- Edit mlops.agent.conf.yaml\n",
    "\n",
    "This file contains the properties used in the configuration of the MLOps service.  For this notebook, you will only need to set the DR host and your API token.\n",
    "\n",
    "For this purpose, we will edit the Configuration YAML file by reading it into a dictionary, modifying the corresponding fields in it, and then writing this dictionary back to the YAML file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QQr9bvXW8dTe"
   },
   "outputs": [],
   "source": [
    "with open(r'conf/mlops.agent.conf.yaml') as file:      # read the yaml file as a dictionary\n",
    "    documents = yaml.safe_load(file)\n",
    "\n",
    "# Set your DR host:\n",
    "documents['mlopsUrl'] = DR_URL                         # set the required values in this dictionary\n",
    "\n",
    "# Set your API token\n",
    "documents['apiToken'] = API_KEY\n",
    "\n",
    "with open('conf/mlops.agent.conf.yaml', \"w\") as f:     # write back the dictionary to the yaml file\n",
    "    yaml.dump(documents, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4tnrBpIXoP0n"
   },
   "source": [
    "In this notebook we will use FS_SPOOL as the messaging channel. More sophisticated monitoring will likely use other channels.\n",
    "\n",
    "channelConfigs:\n",
    "   - type: “FS_SPOOL”\n",
    "     details: {name: “bench”, spoolDirectoryPath: “/tmp/ta”}\n",
    "   - type: “SQS_SPOOL”\n",
    "     details: {name: “sqsSpool”, queueUrl: “https://SQS_URL”}\n",
    "   - type: “PUBSUB_SPOOL”\n",
    "     details: {name: “pubsubSpool”, projectId: “yourprojectId”, topicName: “yourtopicName”}\n",
    "   - type: “RABBITMQ_SPOOL”\n",
    "     details: {name: “rabbit”, queueName: “rabbitmq”, queueUrl: “https://SQS_URL”}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7HJ4TMHkIl9a"
   },
   "source": [
    "Verify the changes in the mlops.agent.conf.yaml.  You should see the correct MLOps URL and API token.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "oaII-yRZ8dTg"
   },
   "outputs": [],
   "source": [
    "print(open('conf/mlops.agent.conf.yaml').read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E-z9kqii8dTp"
   },
   "source": [
    "# 8.- Start the agent and get its status\n",
    "\n",
    "The following shell commands are required to \\\n",
    "a) start the MLOps Agent service. \\\n",
    "b) get the status of the MLOps Agent service. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pFfZ0Wsg8dTq"
   },
   "outputs": [],
   "source": [
    "# Start the agent\n",
    "!___"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3m8RqUBV8dTt",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Get agent status\n",
    "!___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J2hQkAYG8dT0"
   },
   "source": [
    "# 9.- Load scoring data \n",
    "\n",
    "* We now load the scoring data that will be used to obtain predictions. Navigate to the folder where the class material is and select the file named \"**surgical-complication-scoring.csv**\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data for surgical complications is loaded (\"surgical-complication-scoring.csv\"). \n",
    "# The target is \"complication\"\n",
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yqI3KAE38dT1"
   },
   "outputs": [],
   "source": [
    "# Some required modules\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import time\n",
    "import csv\n",
    "import datetime\n",
    "import joblib\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# The scoring data is read into a dataframe\n",
    "SCORING_DF = pd.read_csv(\"surgical-complication-scoring.csv\")\n",
    "\n",
    "columns = list(SCORING_DF.columns)\n",
    "\n",
    "# We make a copy of the scoring data as a Numpy array\n",
    "scoring_data = SCORING_DF.to_numpy()\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We grab the target column name, as well as the labels for the positive & negstive class\n",
    "target_column_name = columns[len(columns) - 1]\n",
    "orig_labels = SCORING_DF[target_column_name].tolist()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AmeZP5emw_SF"
   },
   "source": [
    "# 10.- Upload a pickle file with a pre-trained ML model pipeline to Google Colab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "phQ1SHC3x23o"
   },
   "source": [
    "We will load a pickle file named \"pipeline.pkl\" (found in the zip file that contains the class material); this file contains a pre-trained ML model pipeline. Navigate to the folder where the class material is and select the file named \"**pipeline.pkl**\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EMwqJ21pSivy"
   },
   "outputs": [],
   "source": [
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ICed3RBQ8dT_"
   },
   "source": [
    "# 11.- Make predictions\n",
    "\n",
    "## 11.1.- We call the remote model's predict function and send prediction data to MLOps. Note that the model is supplied using the pickle file uploaded in the previous step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "L-VqByyy_LCG"
   },
   "outputs": [],
   "source": [
    "# MLOps Agent Library imports\n",
    "from datarobot.mlops.mlops import MLOps\n",
    "from datarobot.mlops.connected.client import MLOpsClient\n",
    "from datarobot.mlops.constants import Constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "n5RFjpepz5o7"
   },
   "outputs": [],
   "source": [
    "# Some necessary variables will be defined first\n",
    "\n",
    "CLASS_NAMES = ['0', \"1\"]\n",
    "\n",
    "# Here we define the parameters of the spool file that is used as messaging channel\n",
    "# Spool directory path must match the Monitoring Agent path configured by admin in the YAML configuration file.\n",
    "SPOOL_DIR = \"/tmp/ta\"\n",
    "MLOPS_FILESYSTEM_MAX_FILE_SIZE = 104857600\n",
    "MLOPS_FILESYSTEM_MAX_NUM_FILES = 5\n",
    "\n",
    "# name of the file that contains actuals\n",
    "ACTUALS_OUTPUT_FILE = \"actuals.csv\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vPscl_CBdemM"
   },
   "outputs": [],
   "source": [
    "# Spool file parameters are defined as environment variables\n",
    "!export MLOPS_FILESYSTEM_MAX_FILE_SIZE\n",
    "!export MLOPS_FILESYSTEM_MAX_NUM_FILES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## We are now ready to make predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "l-PeFHso0ew_"
   },
   "outputs": [],
   "source": [
    "# load pickle file with model pipeline\n",
    "model = joblib.load(filename=\"pipeline.pkl\")\n",
    "\n",
    "# Get predictions\n",
    "start_time = time.time()\n",
    "predictions = model.predict_proba(scoring_data).tolist()\n",
    "end_time = time.time()\n",
    "\n",
    "# number of predictions\n",
    "num_predictions = len(predictions)\n",
    "\n",
    "# time required to generate the predictions\n",
    "prediction_time = end_time - start_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xgD8o-MR0li3"
   },
   "outputs": [],
   "source": [
    "# Generate assocation ids for the predictions so we can match them with actuals\n",
    "# this is necessary for accuracy monitoring\n",
    "# The association ids are generated by taking the current time and appending a row counter to it\n",
    "def generate_unique_association_ids(num_samples):\n",
    "    ts = time.time()\n",
    "    return [\"x_{}_{}\".format(ts, i) for i in range(num_samples)]\n",
    "\n",
    "patient_id = generate_unique_association_ids(len(scoring_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now that we have predictions, it's time to send them to MLOps!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3CNlodKzcXjP"
   },
   "outputs": [],
   "source": [
    "# Initialize a MLOps instance\n",
    "mlops = MLOps().___(___) \\\n",
    "               .___(___) \\\n",
    "               .___(___) \\\n",
    "               .___()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eTmpSIOqncL-"
   },
   "outputs": [],
   "source": [
    "# MLOPS: report the number of predictions in the request and the execution time.\n",
    "print(\"Send MLOps deployment stats\")\n",
    "mlops.___(___, ___)\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1rkoiz8YnjaH"
   },
   "outputs": [],
   "source": [
    "# MLOPS: report the predictions data: features, predictions, class_names\n",
    "print(\"Send MLOps prediction data\")\n",
    "mlops.___(features_df=___,  predictions=___, class_names=___, association_ids=___)\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D6HObkyAJm7t"
   },
   "source": [
    "## 11.2.- In the next steps we are simulating a situation in which we receive a file with actual outcomes observed by the business. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LxGAVcyO8dT_"
   },
   "outputs": [],
   "source": [
    "# We are now going to define a function to write a simulated actuals file to the Colab runtime\n",
    "import pytz\n",
    "def write_actuals_file(out_filename, test_data_labels, association_ids):\n",
    "    \"\"\"\n",
    "      Generate a CSV file with the association ids and labels, this example\n",
    "      uses a dataset that has labels already.\n",
    "      In a real use case actuals (labels) will show after prediction is done.\n",
    "\n",
    "    :param out_filename:      name of csv file\n",
    "    :param test_data_labels:  actual values (labels)\n",
    "    :param association_ids:   association id list used for predictions\n",
    "    \"\"\"\n",
    "    with open(out_filename, mode=\"w\") as actuals_csv_file:\n",
    "        writer = csv.writer(actuals_csv_file, delimiter=\",\")\n",
    "        writer.writerow(\n",
    "            [\n",
    "                Constants.ACTUALS_ASSOCIATION_ID_KEY,\n",
    "                Constants.ACTUALS_VALUE_KEY,\n",
    "                Constants.ACTUALS_TIMESTAMP_KEY\n",
    "            ]\n",
    "        )\n",
    "        tz = pytz.timezone(\"America/Los_Angeles\")\n",
    "        for (association_id, label) in zip(association_ids, test_data_labels):\n",
    "            actual_timestamp = datetime.datetime.now().replace(tzinfo=tz).isoformat()\n",
    "            writer.writerow([association_id, \"1\" if label else \"0\", actual_timestamp])   \n",
    "            \n",
    "    print(\"Wrote actuals file: %s\" % out_filename)\n",
    "       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "JcvczFeF04hC"
   },
   "outputs": [],
   "source": [
    "# Write csv file with labels and association IDs\n",
    "write_actuals_file(ACTUALS_OUTPUT_FILE, orig_labels, patient_id)\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "rDF_DvT3oXSb"
   },
   "outputs": [],
   "source": [
    "# MLOPS: release MLOps resources when finished.\n",
    "mlops.___()\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ak-Oh7EN8dUC"
   },
   "source": [
    "# 12.- Upload actuals back to MLOps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cs0p2stg09a0"
   },
   "outputs": [],
   "source": [
    "# A couple of utility functions\n",
    "\n",
    "# If we deal with regression we return a number, otherwise a string\n",
    "def _get_correct_actual_value(deployment_type, value):\n",
    "    if deployment_type == \"Regression\":\n",
    "        return float(value)\n",
    "    return str(value)\n",
    "\n",
    "# convert True/False strigns to boolean values\n",
    "def _get_correct_flag_value(value_str):\n",
    "    if value_str == \"True\":\n",
    "        return True\n",
    "    return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "WjnMjSnZ8dUC"
   },
   "outputs": [],
   "source": [
    "# We now define another function to \n",
    "# 1) Read data from the \"actuals.csv\" file\n",
    "# 2) Place the actual values in an array named \"actuals\"\n",
    "# 3) Place actuals in the messaging channel\n",
    "\n",
    "def upload_actuals():\n",
    "    \n",
    "    # create connected client object\n",
    "    print(\"Connect MLOps client\")           \n",
    "    mlops_connected_client = ___(___, ___)\n",
    "\n",
    "    # get deployment type\n",
    "    deployment_type = mlops_connected_client.___(___)\n",
    "\n",
    "    # read actuals file\n",
    "    actuals = []           # THIS IS THE ARRAY THAT WILL CONTAIN ACTUALS (AS REPORTED BY THE HOSPITAL)\n",
    "    with open(ACTUALS_OUTPUT_FILE, mode=\"r\") as actuals_csv_file:\n",
    "        reader = csv.DictReader(actuals_csv_file)\n",
    "        for row in reader:\n",
    "            actual = {}\n",
    "            for key, value in row.items():\n",
    "                if key == Constants.ACTUALS_WAS_ACTED_ON_KEY:\n",
    "                    value = _get_correct_flag_value(value)\n",
    "                if key == Constants.ACTUALS_VALUE_KEY:\n",
    "                    value = _get_correct_actual_value(deployment_type, value)\n",
    "                actual[key] = value\n",
    "            actuals.append(actual)\n",
    "         \n",
    "    # Upload actuals to MLOps\n",
    "    print(\"Submit actuals\")\n",
    "    mlops_connected_client.___(___, ___)\n",
    "    \n",
    "    print(\"Done!\")    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XUXOwmPV1CLd"
   },
   "outputs": [],
   "source": [
    "upload_actuals()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d3GLIykY8dUF"
   },
   "source": [
    "# 13.- Stop the mlops service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Ye0gxF7xmwoX"
   },
   "outputs": [],
   "source": [
    "%ls bin/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xRTyi3Tr8dUF"
   },
   "outputs": [],
   "source": [
    "!___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cAAScaS0FbtQ"
   },
   "source": [
    "# 14.- Inspect the MLOps agent logs\n",
    "\n",
    "### The log file will be found under the /logs directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EyA854mr8dUI"
   },
   "outputs": [],
   "source": [
    "cat logs/mlops.agent.log"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "29Apr2021 - MLOps II Laboratory",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}