Internship project at VERIMAG
The goal of this project is to train a machine learning model to predict IOH events before they occur.
Table of contents
- Selecting, downloading and cleaning up data
- Preprocessing the data
- Training the model
- Evaluating the model
Our training data come from the VitalDB open dataset. The python script used to process the data is in create_dataset.py.
This repository can be run in a Google Colab Notebook
Note that most functions are multithreaded.
usage: main.py [-h] [-d] [-c] [-f] [-r] [-p] [-l] [-t] [-e EPOCHS] [-T] [-n -1..6388]
options:
-h, --help show this help message and exit
-d, --download download [-n] cases from VitalDB that have all tracks listed in env.json.
-c, --clean clean [-n] downloaded cases and pickle them to {env.DATA_FOLDER}preprocessed/
-f, --force don't skip already preprocessed data, process it again
-r, --reshape reshape [-n] cases from the preprocessed data into the format required by Rocket and Sklearn
(windows of env.WINDOW_SIZE seconds)and pickle them to {env.DATA_FOLDER}ready/
-p, --preprocess preprocess (clean + reshape) [-n] cases from D:\come\datavital and pickle them to
D:\come\dataready/cases/
-l, --label create label dataframes shifted by env.PRED_WINDOW minutes to train the model and pickle them
to D:\come\dataready/labels/
-t, --train_sgd train the SGD model on [-n] cases from D:\come\dataready/test
-e EPOCHS, --epochs EPOCHS
number of epochs to train the model
-T, --test_sgd test the model on [-n] cases from {env.DATA_FOLDER}test/
-n -1..6388, --max-number -1..6388
max number of cases to apply the chosen functions to, -1 for all cases, defaults to 10
Example for a complete process, from download to train on 50 cases :
python3 main.py -dcrpltn 50
You can visualize the data by running the following program :
python plotting.py
The resulting matplotlib figure will look like this : This view combines both the raw and preprocessed data. Gray zones are before and after anesthesia, so they are cropped in the processing. Arterial pressure is drawn in blue and overlayed with mean arterial pressure in orange. IOH events are marked with a red vertical line 1 minute wide.
You can also find an interactive version of this figure in this Google Colab notebook.