Skip to content
/ PPG-N-Back-Clip Public

Photoplethysmogram-based Real-Time Cognitive Load Assessment Using Multi-Feature Fusion Model

License

MIT license
2 stars 2 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

honorforlee/PPG-N-Back-Clip

BranchesTags

Repository files navigation

Build Status Python License Watchers Stargazers Forks

PPG · N-Back · Clip

Photoplethysmogram-based Real-Time Cognitive Load Assessment Using Multi-Feature Fusion Model

Installation

Requirements

Installing with Virtualenv

On Unix, Linux, BSD, macOS, and Cygwin:

git clone https://github.com/iRB-Lab/PPG-N-Back-Clip.git
cd PPG-N-Back-Clip
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt

Quick Start

On Unix, Linux, BSD, macOS, and Cygwin:

./scripts/process_data.sh
./scripts/classify.sh

Usage

Data Processing

Raw data segmentation
python segment.py
Preprocessing
python preprocess.py
Feature extraction
python extract.py
Training set and test set spliting
python split.py

Classification

python classify.py

Data Definition

N-Back Task Meta Data

  • Location: data/raw/meta/
  • Filename format: <participant>-<session_id>.json
Sample Data
{
  "rest_start_timestamp": <timestamp>,
  "blocks": [
    {
      "level": <level>,
      "level_alias": <level_alias>,
      "header": <header>,
      "image_src": <image_src>,
      "image_alt": <image_alt>,
      "stimuli": [
        {
          "stimulus": <stimulus>,
          "load_time": <value>,
          "unload_time": <value>,
          "response_time": <value>,
          "is_target": <bool>,
          "answer": <bool>,
          "correct": <bool>,
          "timestamp": {
            "load": <timestamp>,
            "response": <timestamp>
          }
        },
        ...
      ],
      "total_time": <value>,
      "rsme": <rsme>
    },
    ...
  ]
}

PPG Signal Data

  • Location: data/raw/ppg/
  • Filename format: <participant>-<session_id>-<year>_<month>_<day>_<hour>_<minute>_<second>.txt
Sample Data
109
110
109
109
...

BIOPAC Signal Data

  • Location: data/raw/biopac/
  • Filename format: <participant>-<session_id>-<seconds_before_start>.txt
Sample Data
SampleData.acq
1 msec/sample
3 channels
ECG - ECG100C
mV
PPG - PPG100C
Volts
Skin conductance - GSR100C
microsiemens
min	CH1	CH2	CH9
	1161418	1161418	1161418
0	-2.26685	-0.194092	3.43475
1.66667E-05	-2.25769	-0.197449	3.44086
3.33333E-05	-2.24915	-0.198975	3.4378
...

Segmented Signal Data

  • Location (complete data): data/segmented/
  • Location (incomplete data): data/segmented/incomplete/
  • Filename format: <participant>.json
Sample Data
{
  "1": {
    "rest": {
      "ppg": {
        "sample_rate": <value>,
        "signal": [ ... ]
      },
      "ecg": {
        "sample_rate": <value>,
        "signal": [ ... ]
      },
      "skin_conductance": {
        "sample_rate": <value>,
        "signal": [ ... ]
      }
    },
    "blocks": [
      {
        "level": <level>,
        "stimuli": [
          {
            "stimulus": <stimulus>,
            "correct": <bool>,
            "is_target": <bool>,
            "answer": <bool>,
            "response_time": <value>
          },
          ...
        ],
        "rmse": <rmse>,
        "ppg": {
          "smaple_rate": <value>,
          "signal": [ ... ]
        },
        "ecg": {
          "smaple_rate": <value>,
          "signal": [ ... ]
        },
        "skin_conductance": {
          "smaple_rate": <value>,
          "signal": [ ... ]
        }
      },
      ...
    ]
  },
  "2": { ... }
}

Preprocessed Data

  • Location: data/preprocessed/
  • Filename format: <participant>.json
Sample Data
{
  "1": {
    "rest": {
      "ppg": {
        "sample_rate": <value>,
        "single_waveforms": [
          [ ... ],
          ...
        ]
      },
      "ecg": {
        "sample_rate": <value>,
        "rri": [ ... ],
        "rri_interpolated": [ ... ]
      },
      "skin_conductance": {
        "sample_rate": <value>,
        "signal": [ ... ]
      }
    },
    "blocks": [
      {
        "level": <level>,
        "stimuli": [
          {
            "stimulus": <stimulus>,
            "correct": <bool>,
            "is_target": <bool>,
            "answer": <bool>,
            "response_time": <value>
          },
          ...
        ],
        "rmse": <rmse>,
        "ppg": {
          "smaple_rate": <value>,
          "single_waveforms": [
            [ ... ],
            ...
          ]
        },
        "ecg": {
          "smaple_rate": <value>,
          "rri": [ ... ],
          "rri_interpolated": [ ... ]
        },
        "skin_conductance": {
          "smaple_rate": <value>,
          "signal": [ ... ]
        }
      },
      ...
    ]
  },
  "2": { ... }
}

Extracted Feature Data

  • Location: data/extracted/
  • Filename format: <participant>.json
Sample Data
{
  "1": {
    "rest": {
      "ppg": {
        "sample_rate": <value>,
        "ppg45": [
            [ ... ],
            ...
        ],
        "svri": [ ... ]
      },
      "skin_conductance": {
        "sample_rate": <value>,
        "average_level": <value>,
        "minimum_level": <value>
      },
      "ecg": {
        "sample_rate": <value>,
        "average_rri": <value>,
        "rmssd": <value>,
        "mf_hrv_power": <value>,
        "hf_hrv_power": <value>
      }
    },
    "blocks": [
      {
        "level": <level>,
        "stimuli": [
          {
            "stimulus": <stimulus>,
            "correct": <bool>,
            "is_target": <bool>,
            "answer": <bool>,
            "response_time": <value>
          },
          ...
        ],
        "rmse": <value>,
        "ppg": {
          "smaple_rate": <value>,
          "ppg45": [
            [ ... ],
            ...
          ],
          "svri": [ ... ]
        },
        "skin_conductance": {
          "sample_rate": <value>,
          "average_level": <value>,
          "minimum_level": <value>
        },
        "ecg": {
          "sample_rate": <value>,
          "average_rri": <value>,
          "rmssd": <value>,
          "mf_hrv_power": <value>,
          "hf_hrv_power": <value>
        }
      },
      ...
    ]
  },
  "2": { ... }
}

Splited Feature Data

  • Location: data/splited/
  • Filename format: <participant>.json
Sample Data
{
  "train": {
    "0": [
      {
        "ppg45": [
          [ ... ],
          ...
        ],
        "ppg45_cr": [
          [ ... ],
          ,,,
        ],
        "svri": [ ... ],
        "svri_cr": [ ... ],
        "average_skin_conductance_level": <value>,
        "average_skin_conductance_level_cr": <value>,
        "minimum_skin_conductance_level": <value>,
        "minimum_skin_conductance_level_cr": <value>,
        "average_rri": <value>,
        "average_rri_cr": <value>,
        "rmssd": <value>,
        "rmssd_cr": <value>,
        "mf_hrv_power": <value>,
        "mf_hrv_power_cr": <value>,
        "hf_hrv_power": <value>,
        "hf_hrv_power_cr": <value>
      },
      ...
    ],
    "1": [ ... ],
    "2": [ ... ]
  },
  "test": { ... }
}

Sensors and Features

Sensor Feature Dimension
PPG finger clip PPG-45 (39 time-domain, 9 frequency-domain) 45
Stress-induced vascular response index (sVRI) 1
Skin conductance electrodes Average skin conductance level 1
Minimum skin conductance level 1
ECG Electrodes Heart rate (R-R interval, RRI) 1
Root mean squared successive difference (RMSSD) 1
Mid-frequency heart rate variability (MF-HRV) 1
High-frequency heart rate variability (HF-HRV) 1

PPG-45 Feature Definition

# Feature Description
1 x Systolic peak
2 y Diastolic peak
3 z Dicrotic notch
4 tpi Pulse interval
5 y/x Augmentation index
6 (x-y)/x Relative augmentation index
7 z/x
8 (y-z)/x
9 t1 Systolic peak time
10 t2 Diastolic peak time
11 t3 Dicrotic notch time
12 ∆T Time between systolic and diastolic peaks
13 w Full width at half systolic peak
14 A2/A1 Inflection point area ratio
15 t1/x Systolic peak rising slope
16 y/(tpi-t3) Diastolic peak falling slope
17 t1/tpi
18 t2/tpi
19 t3/tpi
20 ∆T/tpi
21 ta1
22 tb1
23 te1
24 tf1
25 b2/a2
26 e2/a2
27 (b2+e2)/a2
28 ta2
29 tb2
30 ta1/tpi
31 tb1/tpi
32 te1/tpi
33 tf1/tpi
34 ta2/tpi
35 tb2/tpi
36 (ta1+ta2)/tpi
37 (tb1+tb2)/tpi
38 (te1+t2)/tpi
39 (tf1+t3)/tpi
40 fbase Fundamental component frequency
41 |sbase| Fundamental component magnitude
42 f2 2nd harmonic frequency
43 |s2| 2nd harmonic magnitude
44 f3 3rd harmonic frequency
45 |s3| 3rd harmonic magnitude

API Reference

Module: ppg

Excerpt from ppg/__init__.py:

BASE_DIR = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))

Module: ppg.params

Excerpt from ppg/params.py:

TOTAL_SESSION_NUM = 2
REST_DURATION = 5 * 60
BLOCK_DURATION = 2 * 60

MINIMUM_PULSE_CYCLE = 0.5
MAXIMUM_PULSE_CYCLE = 1.2

PPG_SAMPLE_RATE = 200
PPG_FIR_FILTER_TAP_NUM = 200
PPG_FILTER_CUTOFF = [0.5, 5.0]
PPG_SYSTOLIC_PEAK_DETECTION_THRESHOLD_COEFFICIENT = 0.5

BIOPAC_HEADER_LINES = 11
BIOPAC_MSEC_PER_SAMPLE_LINE_NUM = 2
BIOPAC_ECG_CHANNEL = 1
BIOPAC_SKIN_CONDUCTANCE_CHANNEL = 3

ECG_R_PEAK_DETECTION_THRESHOLD = 2.0
ECG_MF_HRV_CUTOFF = [0.07, 0.15]
ECG_HF_HRV_CUTOFF = [0.15, 0.5]

TRAINING_DATA_RATIO = 0.75

Module: ppg.signal

Peak Finding
extrema = find_extrema(signal)
PPG Signal Smoothing
smoothed_ppg_signal = smooth_ppg_signal(
    signal,
    sample_rate=PPG_SAMPLE_RATE,
    numtaps=PPG_FIR_FILTER_TAP_NUM,
    cutoff=PPG_FILTER_CUTOFF
)
PPG Single-Waveform Validation
result = validate_ppg_single_waveform(single_waveform, sample_rate=PPG_SAMPLE_RATE)
PPG Single-Waveform Extraction
single_waveforms = extract_ppg_single_waveform(signal, sample_rate=PPG_SAMPLE_RATE)
RRI Extraction
rri, rri_time = extract_rri(signal, sample_rate)
RRI Interpolation
rri_interpolated = interpolate_rri(rri, rri_time, sample_rate)

Module: ppg.feature

PPG Features

PPG-45
extract_ppg45(single_waveform, sample_rate=PPG_SAMPLE_RATE)
Stress-Induced Vascular Response Index (sVRI)
svri = extract_svri(single_waveform)

Skin Conductance Features

Average Skin Conductance Level
average_skin_conductance_level = extract_average_skin_conductance_level(signal)
Minimum Skin Conductance Level
minimum_skin_conductance_level = extract_minimum_skin_conductance_level(signal)

ECG Features

Heart Rate (R-R Interval, RRI)
avarage_rri = extract_average_rri(rri)
Root Mean Squared Successive Difference (RMSSD)
rmssd = extract_rmssd(rri)
Middle/High-Frequency Heart Rate Variability (MF/HF-HRV)
mf_hrv_power, hf_hrv_power = extract_hrv_power(rri, sample_rate)

Module: ppg.learn

Get Feature Set
train_features, train_labels, test_features, test_labels = get_feature_set(data, level_set, feature_type_set)

Classifiers

Logistic Regression Classifier
classifier = logistic_regression_classifier(features, labels)
Support Vector Classifier
classifier = support_vector_classifier(features, labels)
Gaussian Naïve Bayes Classifier
classifier = gaussian_naive_bayes_classifier(features, labels)
Decision Tree Classifier
classifier = decision_tree_classifier(features, labels)
Random Forest Classifier
classifier = random_forest_classifier(features, labels)
AdaBoost Classifier
classifier = adaboost_classifier(features, labels)
Gradient Boosting Classifier
classifier = gradient_boosting_classifier(features, labels)
Voting Classifier
classifier = voting_classifier(estimators, features, labels)

Module: ppg.utils

make_dirs_for_file(pathname)
boolean = exist_file(pathname, overwrite=False, display_info=True)
text_data = load_text(pathname, display_info=True)
json_data = load_json(pathname, display_info=True)
dump_json(data, pathname, overwrite=False, display_info=True)
classifier_object = load_model(pathname, display_info=True)
dump_model(model, pathname, overwrite=False, display_info=True)
export_csv(data, fieldnames, pathname, overwrite=False, display_info=True)
datetime = parse_iso_time_string(timestamp)
change_ratio = get_change_ratio(data, baseline)
set_matplotlib_backend(backend=None)
plot(args, backend=None)
semilogy(args, backend=None)

File Structure

├── data/
│   ├── raw/
│   │   ├── meta/
│   │   │   ├── <participant>-<session_id>.json
│   │   │   └── ...
│   │   ├── ppg/
│   │   │   ├── <participant>-<session_id>-<year>_<month>_<day>_<hour>_<minute>_<second>.json
│   │   │   └── ...
│   │   └── biopac/
│   │       ├── <participant>-<session_id>-<seconds_before_start>.json
│   │       └── ...
│   ├── segmented/
|   |   ├── incomplete/
|   |   |   ├── <participant>.json
│   |   |   └── ...
│   │   ├── <participant>.json
│   │   └── ...
│   ├── preprocessed/
│   │   ├── <participant>.json
│   │   └── ...
│   └── extracted/
│       ├── <participant>.json
│       └── ...
├── models/
│   └── ...
├── results/
│   └── ...
├── ppg/
│   ├── __init__.py
│   ├── params.py
│   ├── signal.py
│   ├── feature.py
│   ├── learn.py
│   └── utils.py
├── scripts/
│   ├── process_data.sh
│   └── classify.sh
├── segment.py
├── preprocess.py
├── extract.py
├── split.py
├── classify.py
├── requirements.txt
├── README.md
├── LICENSE
└── .gitignore

License

MIT License

About

Photoplethysmogram-based Real-Time Cognitive Load Assessment Using Multi-Feature Fusion Model

Resources

Readme

License

MIT license
Activity

Stars

2 stars

Watchers

2 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages