Classification_ECG_template.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Classification_ECG_template.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/KristynaPijackova/Tutorials_NNs_and_signals/blob/main/Classification_ECG_template.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Classification of ECG signals\n",
        "\n",
        "The aim of this notebook is to introduce you how you can use recurrent layers and 1d convolutional layer for classification tasks with signals. It will also show you how you can prepare a dataset for training. \n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "We are going to use ECG5000 dataset, which has 5000 samples of ECG signals with 5 classes: \n",
        "\n",
        "1 - Normal\n",
        "\n",
        "2 - R-on-T Premature Ventricular Contraction\n",
        "\n",
        "3 - Premature Ventricular Contraction \n",
        "\n",
        "4 - Supraventricular Premature beat\n",
        "\n",
        "5 - Unclassified Beat\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "Lu7FunAaX2N_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Few jupyter notebook shortcuts you can use to make your life easier**\n",
        "\n",
        "Run cell: `shift + enter`\n",
        "\n",
        "Undo last action (inside a cell): `ctrl + m + z`\n",
        "\n",
        "Find and replace: `ctrl + h`\n",
        "\n",
        "Insert code cell above: `ctrl + a`\n",
        "\n",
        "Insert code cell below: `ctrl + b`\n",
        "\n",
        "Delete cell: `ctrl + m + d`"
      ],
      "metadata": {
        "id": "sDlt45qytFoM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Download data\n",
        "\n",
        "For this we use data from following page: \n",
        "\n",
        "http://www.timeseriesclassification.com/description.php?Dataset=ECG5000\n",
        "\n",
        "Options you have to download the data:\n",
        "\n",
        "\n",
        "1.   Go to the website, download it localy or if you have  and upload back into your Colab workspace. However, uploading data in here isn't the quickest and you lose the data and have to re-do the process everytime you switch workspace (**not recommended**).\n",
        "\n",
        "2.   Use command !wget along with the link which lets you download the data. Command !unzip along with the name with the zip file then unzips the content of the file.\n",
        "```\n",
        "! wget http://www.timeseriesclassification.com/Downloads/ECG5000.zip\n",
        "! unzip ECG5000.zip\n",
        "```\n",
        "\n",
        "3. Another option is to store your data in gdrive and use another command to download the data into Colab.\n",
        "```\n",
        "!gdown --id 1jo255jnoJniagZZd3IKbixb1i0kUIQCr\n",
        "!unzip /content/data.zip\n",
        "```\n",
        "To get the file id go the the sharable link for your data and copy the higlihted part (between d/ and /view?...)\n",
        "\n",
        "    https:// drive.google.com/file/d/__1jo255jnoJniagZZd3IKbixb1i0kUIQCr__/view?usp=sharing\n",
        "\n",
        "Mind that you lose the data with the later two options as well after you switch workspace, but the process of uploading the data back this way is much faster.\n"
      ],
      "metadata": {
        "id": "VbYkDGBAX_4x"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Download the ECG dataset \n",
        "\n",
        "Here we use the direct link to the website along with !wget and unzip the file - if you don't know the name of the file you can see the name of the downloaded file in the left panel if you click on the folder icon 📁"
      ],
      "metadata": {
        "id": "EeUv5JthcxuJ"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RWqSe_Ge6DJQ"
      },
      "outputs": [],
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Or you can use another unix command !ls /content/ (this is how the folder where you work with in here is named). \n",
        "\n",
        "This shows you the file stored in here. "
      ],
      "metadata": {
        "id": "j4yHVe0pdUU6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "uTa-aoQZdXXM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we know how our file is named we can unzip it"
      ],
      "metadata": {
        "id": "UQAIJMiSdsjX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "BLM0hRB2dylJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We see there are quite few files - we are going to use the text files `ECG5000_TEST.txt` and `ECG5000_TRAIN.txt`\n",
        "\n",
        "And we can once again use another unix command !head to get a peak at how are data looks like."
      ],
      "metadata": {
        "id": "kUN3U4M2dpf_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "qMiaPz-aXMZy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Import libraries\n",
        "\n",
        "Now that we have a basic idea what we are going to work with, we can import libraries we are about to use."
      ],
      "metadata": {
        "id": "RLaBEwMUfHiO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "4nF9o-trWOXB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Let's prepare our data\n",
        "\n",
        "We start by creating a dataframe - basicly we just get content of the textfiles into pandas dataframe so it is easier for us to work with it."
      ],
      "metadata": {
        "id": "yZuoZ51Ef1rz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "UYt3gvQbWTxe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "CRe357VuXYfW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "See? It's basicaly the same thing as wee saw just few moments ago 😉\n",
        "\n",
        "However, since pandas df doesn't like just numerical columns, we add a prefix to it, so we can work with them."
      ],
      "metadata": {
        "id": "CU02ZJu2gTNe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "v12I1mXZgs-b"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Analyze the dataset\n",
        "\n",
        "In machine and deep learning it is important to know what data you are working with. Not just what is the data meant to be, but what is the structure, what values can I expect, what is the classes distribution, how does the data even look like, etc...\n",
        "\n",
        "So the following part is supposed to get us eve more familiar with the data we are working with."
      ],
      "metadata": {
        "id": "_DnlnzMgiFNv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here we can see basic info about the data - we have 141 columns c0 to c140. \n",
        "The data is stored as float64 and the memory usage is 4.8 MB. "
      ],
      "metadata": {
        "id": "AfKvPsVOiw2p"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "Xl8cBLtQhNpd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next we can use `.describe()` which gets us a bit of a statistical view on the data. \n",
        "\n",
        "What we can see is that the max.min value are +/-7, the mean value is more or less in range between (-1,1) so we might be okay without normalization. "
      ],
      "metadata": {
        "id": "xBMMFcpejBo0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "cseuV3FrfwQQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We could also have a peak at how the classes are distributed in the dataset. \n",
        "\n",
        "This dataset stores the info about the classes in the first column, thus we index the df and use `.value_counts()` to see byt how many samples is each class represented.\n",
        "\n"
      ],
      "metadata": {
        "id": "5D8RO1ngX1J4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "nMZ0xAloXh4I"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "WSvriZrCkk7n"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "And we can even visualize it... "
      ],
      "metadata": {
        "id": "7OGtQQ0AlDuy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "yQyOP9ia_AoY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "BEhVeV3VkodN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Handling imbalanced datasets\n",
        "\n",
        "As we can see, out dataset quite imbalanced - **which is not good!**\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "So what can we actually do about it? \n",
        "\n",
        "*   Undersampling\n",
        "*   Oversampling\n",
        "*   Combine under- and oversampling\n",
        "*   Generate new data - not ideal, not always possible\n",
        "*   Generate new data with generative networks\n",
        "\n",
        "---\n",
        "\n",
        "Few blog posts about over- and undersampling\n",
        "\n",
        "*   [5 Techniques to Handle Imbalanced Data For a Classification Problem](https://www.analyticsvidhya.com/blog/2021/06/5-techniques-to-handle-imbalanced-data-for-a-classification-problem/)\n",
        "\n",
        "\n",
        "* [Stop using SMOTE to handle all your Imbalanced Data](https://towardsdatascience.com/stop-using-smote-to-handle-all-your-imbalanced-data-34403399d3be)\n",
        " \n",
        "\n",
        "---\n",
        "\n",
        "Library which helps us deal with imbalanced dataset\n",
        "\n",
        "[imbalanced-learn](https://imbalanced-learn.org/stable/auto_examples/index.html)\n",
        "\n",
        "SMOTE =  Synthetic Minority Oversampling TEchnique\n"
      ],
      "metadata": {
        "id": "1QsUJC3fHR90"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here is a simple function which will apply under/oversampling method to our data. It returns arrays of new samples and their labels."
      ],
      "metadata": {
        "id": "KCXPLIYtosem"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from imblearn.combine import SMOTETomek\n",
        "def over_under_sampling(dataframe):\n",
        "    \"\"\"\n",
        "    Use SMOTETomek technique to oversample our dataset. \n",
        "    \n",
        "    This function is written to be applied to our datasets, \n",
        "    where the first column holds the labels, and the rest is the \n",
        "    time sequence. \n",
        "\n",
        "    It passes the under-represented data - classes 2-5 along\n",
        "    with the dominant class 1 into the SMOTETomek over- & undersampler\n",
        "    to balance the dataset. \n",
        "    \"\"\"\n",
        "    # lists to store the created values in\n",
        "    x_res = []\n",
        "    y_res = []\n",
        "\n",
        "    for i in range(2,6):\n",
        "\n",
        "        # create copy of the dataframe\n",
        "        df_copy = dataframe.copy()\n",
        "        # choose samples of i-th class\n",
        "        df = df_copy[df_copy['c0'] == i]\n",
        "        # add samples from 1st class\n",
        "        df = df.append(df_copy[df_copy['c0'] == 1])\n",
        "        # split the dataframe into x - data and y - labels\n",
        "        x = df.values[:,1:]\n",
        "        y = df.values[:,0]\n",
        "\n",
        "        # define the imbalance function\n",
        "        smtomek = SMOTETomek(random_state=42)\n",
        "        # fit it on our data\n",
        "        x_r, y_r = smtomek.fit_resample(x, y)\n",
        "        \n",
        "        # we want to skip the data we fit it on - only want the new data\n",
        "        skip = y.shape[0]\n",
        "        # append the data into our above lists\n",
        "        x_res.append(x_r[skip:,:])\n",
        "        y_res.append(y_r[skip:])\n",
        "\n",
        "    # return the data as concatenated arrays -> only one array of all samples\n",
        "    # instead of a list of arrays\n",
        "    return np.concatenate(x_res), np.concatenate(y_res)"
      ],
      "metadata": {
        "id": "pE-V6nKTeBqH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here we call the above function"
      ],
      "metadata": {
        "id": "BwO2OaYOo8vs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "7w0BGDxjf_nQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Split the dataframe into data-points and labels\n",
        "\n",
        "The first column of our dataframe 'c0' holds the labels from 1 to 5 which represent the classes we want to classify. \n",
        "\n",
        "At this point we will separate the data from the labels and create `x_train, x_test, y_train, y_test` arrays."
      ],
      "metadata": {
        "id": "1crZzRXaPPJO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "RMmAoNs2P2yo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "And now we combine it with our original data."
      ],
      "metadata": {
        "id": "McjNz311pCFE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "Q7XfRotYiV2J"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's see how the before imbalanced data looks now..."
      ],
      "metadata": {
        "id": "7rzQXxWqpID7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "vj4n6QtLi9w0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "That's better, isn't it? But we could maybe check how the new synthetic data looks like and if it is somewhat similar to the original data?"
      ],
      "metadata": {
        "id": "3g-YqR6YpRxR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Let's visualize the signals"
      ],
      "metadata": {
        "id": "4HwdTnQnHe5b"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's write a for loop where we get few indexes of our target class with the help of `np.where()` and then plot two subplots one with original data and one with the synthetical for comparisson."
      ],
      "metadata": {
        "id": "82cyezzMsvlO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "nJf5T91zxDmf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## And now we can get to training\n",
        "\n",
        "Just few last things we need to do, so that our models can train on the data\n",
        "\n"
      ],
      "metadata": {
        "id": "Sa3WZB-EuBQ7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We normalize the data with the help of MinMaxScaler that was imported from sklearn.\n",
        "\n",
        "Here the scaler holds the MinMaxScaler and we fit it on our training data in the second row of the code, which will be applied on the data we want to scale just in a second with `data_scaler.transform(data_we_want_to_scale)`"
      ],
      "metadata": {
        "id": "-iUEr97UwROW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "JUQDz6O9yRym"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "fydKAe1EyUE_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Rigth now we have data with dimensions (n, 140), however we need (n,140,1)"
      ],
      "metadata": {
        "id": "_WNBZ2mWKUVC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "K8aavQSazklf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "6mOZwMaDOX14"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "ZFJYarB62YHk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Define functions to plot loss, accuracy and confusion matrix"
      ],
      "metadata": {
        "id": "6xEz-6Iq6QSi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "cpBraKaD6Xfh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Model with Conv1D"
      ],
      "metadata": {
        "id": "MIF3vd8EAUTU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Comment to the input shape: \n",
        "\n",
        "We have data with 140 timesteps and only one feature (array 140x1). However, if you happen to have data which have more features - I/Q representation with (tx2) shape where t are the timesteps you can simply change the 1 to match the number of your feature representations. "
      ],
      "metadata": {
        "id": "LxTdSX19Xn_B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "5-2rHd41x81D"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In `.summary()` you can see the output shape has None at the beginning - this is for the batchsize which can vary - you set the value in the `.fit()` function"
      ],
      "metadata": {
        "id": "6HRnggr7XyWm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "Gk4nsMD20ylr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "C_5Fb2opzz8o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "S-azHsBL80Vc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## LSTM"
      ],
      "metadata": {
        "id": "cJ6WXaKaP51-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Comment to the input shape: \n",
        "\n",
        "We have data with 140 timesteps and only one feature (array 140x1). However, if you happen to have data which have more features - I/Q representation with (tx2) shape where t are the timesteps you can simply change the 1 to match the number of your feature representations. \n",
        "\n",
        "However unlike when using conv1d only, you could set the timesteps value to None. This allows you to train and test your data on a variable sequence length. \n",
        "\n",
        "The only thing you have to do when training the model is to have batches of the same size. "
      ],
      "metadata": {
        "id": "pRWLOM1WYbsI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "cEOawcl3P9Is"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "BuR6e71kQ-cJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "t8hsc0HvRAlB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "3fOewe47SfoR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## GRU"
      ],
      "metadata": {
        "id": "0qNNwgX0Pqiq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "ObH9NarB8H7H"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "6i2MZHOGBOzM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "fbY5XLF7BiPL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "fBylcEtX1kMh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conv1d + LSTM"
      ],
      "metadata": {
        "id": "PhshmzcgPhc2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "g91pjjffBi8N"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "yZrW_hNqP3Ip"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "51gFnWnQP43V"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "w3PaJXp8-ygX"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}