diff --git a/docs/about-us.md b/docs/about-us.md index a07456d..2f437c2 100644 --- a/docs/about-us.md +++ b/docs/about-us.md @@ -72,7 +72,7 @@ Francesco Negro: - Contribution:   :fontawesome-solid-brain: Knowledge sharing   :fontawesome-solid-file-code: Code sharing   :octicons-codescan-checkmark-24: Accuracy check -- Prof. Negro is a Full Professor at the Department of Clinical and Experimental Sciences at Universita’ degli Studi di Brescia (IT). His research interests include applied physiology of the human motor system, signal processing of intramuscular and surface electromyography, and modeling of spinal neural networks. +- Francesco Negro is a Full Professor at the Department of Clinical and Experimental Sciences at Universita’ degli Studi di Brescia (IT). His research interests include applied physiology of the human motor system, signal processing of intramuscular and surface electromyography, and modeling of spinal neural networks. Andrea Casolo: @@ -81,3 +81,12 @@ Andrea Casolo: - Contribution:   :fontawesome-solid-brain: Knowledge sharing   :octicons-codescan-checkmark-24: Accuracy check - Andrea Casolo is an Assistant Professor at the Department of Biomedical Sciences, University of Padova (IT). He obtained a MSc in Health and Physical Activity (2016) and a PhD in Human Movement and Sport Sciences (2020) from the University of Rome "Foro Italico". His research interests focus on the neural control of movement and the study of neuromuscular plasticity to physical exercise investigated with high-density surface electromyography. + +Giuseppe De Vito: + +- giuseppe.devito@unipd.it + +- Contribution:   :fontawesome-solid-brain: Knowledge sharing + +- Giuseppe De Vito is a full Professor of Human Physiology in the Department of Biomedical Sciences at University of Padova (IT). He was, from 2007 until 2019, Professor and Dean in the School of Public Health, Physiotherapy & Sports Science at University College Dublin (IE) (Head of School between 2014 and 2019). Giuseppe does research in Human and Exercise Physiology. + diff --git a/docs/api_openfiles.md b/docs/api_openfiles.md index 86e9e89..a7a6788 100644 --- a/docs/api_openfiles.md +++ b/docs/api_openfiles.md @@ -1,45 +1,32 @@ Description ----------- -This module contains all the functions that are necessary to open or save -MATLAB (.mat), JSON (.json) or custom (.csv) files.
-MATLAB files are used to store data from the DEMUSE and the OTBiolab+ -software while JSON files are used to save and load files from this -library.
-The choice of saving files in the open standard JSON file format was -preferred over the MATLAB file format since it has a better integration -with Python and has a very high cross-platform compatibility. +This module contains all the functions that are necessary to open or save MATLAB (.mat), JSON (.json) or custom (.csv) files. .mat files are currently used to store data from the DEMUSE and the OTBiolab+ software, while .csv files are used to store custom data. Instead, .json files are used to save and load files from this library.
+The choice of saving files in the open standard JSON file format was preferred over the MATLAB file format since it has a better integration with Python and has a very high cross-platform compatibility. Function's scope ---------------- - **emg_from_samplefile**:
Used to load the sample file provided with the library. - **emg_from_otb** and **emg_from_demuse**:
- Used to load .mat files coming from the DEMUSE or the OTBiolab+ - software. Demuse has a fixed file structure while the OTB file, in - order to be compatible with this library should be exported with a - strict structure as described in the function emg_from_otb. - In both cases, the input file is a .mat file. -- **refsig_from_otb**:
- Used to load files from the OTBiolab+ software that contain only - the REF_SIGNAL. + Used to load .mat files coming from the DEMUSE or the OTBiolab+ software. Demuse has a fixed file structure while the OTB file, in order to be compatible with this library should be exported with a strict structure as described in the function emg_from_otb. In both cases, the input file is a .mat file. - **emg_from_customcsv**:
Used to load custom file formats contained in .csv files. +- **refsig_from_otb** and **refsig_from_customcsv**:
+ Used to load files from the OTBiolab+ software or from a custom .csv file that contain only the REF_SIGNAL. - **save_json_emgfile**, **emg_from_json**:
- Used to save the working file to a .json file or to load the .json - file. + Used to save the working file to a .json file or to load the .json file. - **askopenfile**, **asksavefile**:
- A quick GUI implementation that allows users to select the file to - open or save. + A quick GUI implementation that allows users to select the file to open or save. Notes ----- Once opened, the file is returned as a dictionary with keys:
-"SOURCE" : source of the file (i.e., "CUSTOM", "DEMUSE", "OTB")
+"SOURCE" : source of the file (i.e., "CUSTOMCSV", "DEMUSE", "OTB")
+"FILENAME" : the name of the opened file
"RAW_SIGNAL" : the raw EMG signal
"REF_SIGNAL" : the reference signal
-"PNR" : pulse to noise ratio
-"SIL" : silouette score
+"ACCURACY" : accuracy score (depending on source file type)
"IPTS" : pulse train (decomposed source)
"MUPULSES" : instants of firing
"FSAMP" : sampling frequency
@@ -47,12 +34,15 @@ Once opened, the file is returned as a dictionary with keys:
"EMG_LENGTH" : length of the emg file (in samples)
"NUMBER_OF_MUS" : total number of MUs
"BINARY_MUS_FIRING" : binary representation of MUs firings
+"EXTRAS" : additional custom values
-The only exception is when OTB files are loaded with just the reference signal: +The only exception is when files are loaded with just the reference signal: -"SOURCE": source of the file (i.e., "OTB_REFSIG")
-"FSAMP": sampling frequency
-"REF_SIGNAL": the reference signal
+"SOURCE" : source of the file (i.e., "CUSTOMCSV_REFSIG", "OTB_REFSIG")
+"FILENAME" : the name of the opened file
+"FSAMP" : sampling frequency
+"REF_SIGNAL" : the reference signal
+"EXTRAS" : additional custom values
Additional informations can be found in the [info module](api_info.md#openhdemg.library.info.info.data) and in the @@ -97,6 +87,13 @@ Furthermore, all the users are encouraged to read the dedicated tutorial [Struct
+::: openhdemg.library.openfiles.refsig_from_customcsv + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.openfiles.save_json_emgfile options: show_root_full_path: False diff --git a/docs/api_plotemg.md b/docs/api_plotemg.md index 22dadfc..dc815d4 100644 --- a/docs/api_plotemg.md +++ b/docs/api_plotemg.md @@ -1,7 +1,6 @@ Description ----------- -This module contains all the functions used to visualise the emg file, -the MUs properties or to save figures. +This module contains all the functions used to visualise the content of the imported EMG file, the MUs properties or to save figures.
diff --git a/docs/api_tools.md b/docs/api_tools.md index d743908..f65b7bd 100644 --- a/docs/api_tools.md +++ b/docs/api_tools.md @@ -21,6 +21,13 @@ shortcuts necessary to operate with the HD-EMG recordings.
+::: openhdemg.library.tools.mupulses_from_binary + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.tools.resize_emgfile options: show_root_full_path: False @@ -42,6 +49,13 @@ shortcuts necessary to operate with the HD-EMG recordings.
+::: openhdemg.library.tools.delete_empty_mus + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.tools.sort_mus options: show_root_full_path: False diff --git a/docs/gui_advanced.md b/docs/gui_advanced.md index 3e74c6f..286d27d 100644 --- a/docs/gui_advanced.md +++ b/docs/gui_advanced.md @@ -8,9 +8,10 @@ So far, we have included three advanced analyses in the *openhdemg* GUI. - `Motor Unit Tracking` - `Duplicate Removal` -- `Conduction Velocity Calculation` +- `Conduction Velocity Estimation` For all of those, the specification of a `Matrix Orientation` and a `Matrix Code` is required. The `Matrix Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window` when using the `Plot EMG`function. The `Matrix Orientation` can be either **0** or **180** and must be chosen from the dropdown list. + The `Matrix Code` must be specified according to the one you used during acquisition. So far, the codes - `GR08MM1305` @@ -18,13 +19,13 @@ The `Matrix Code` must be specified according to the one you used during acquisi - `GR10MM0808` - `None` -are implemented. You must choose one from the respective dropdown list. -In case you selected `None`, the entrybox `Rows, Columns` will appear. Please specify the number of rows and columns of your used matrix since you now bypass included matrix codes. In example, specifying +are implemented. You must choose one from the respective dropdown list. In case you selected `None`, the entrybox `Rows, Columns` will appear. Please specify the number of rows and columns of your used matrix since you now bypass included matrix codes. `Orientation` is ignored when `Matrix Code` is `None`. In example, specifying ```Python Rows, Columns: 13, 5 ``` means that your File has 65 channels. + Once you specified these parameter, you can click the `Advaned Analysis` button to start your analysis. ----------------------------------------- @@ -39,9 +40,9 @@ When you want to track MUs across two different files, you need to select the `M - `OTB` (.mat file exportable by OTBiolab+) - `DEMUSE` (.mat file used in DEMUSE) - `OPENHDEMG` (emgfile or reference signal stored in .json format) - - `CUSTOM` (custom data from a .csv file) + - `CUSTOMCSV` (custom data from a .csv file) - Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the **Type of file** corresponding to the type of your file. In case you selected `OTB` specify the `extension factor` in the dropdown. + Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the **Type of file** corresponding to the type of your file. In case you selected `OTB`, specify the `extension factor` in the dropdown. 2. Load the files according to specified `Type of file`using the `Load File 1` and `Load File 2` buttons. @@ -65,8 +66,7 @@ When you want to remove MUs duplicates across different files, you need to selec 1. You should specify How to remove the duplicated MUs in the `Which` dropdown. You can choose between - munumber: Duplicated MUs are removed from the file with more MUs. - - PNR: The MU with the lowest PNR is removed. - - SIL: The MU with the lowest SIL is removed. + - accuracy: The MU with the lowest accuracy score is removed. 2. By clicking the `Remove Duplicates` button, you start the removal process. diff --git a/docs/gui_basics.md b/docs/gui_basics.md index a00749d..ac9625f 100644 --- a/docs/gui_basics.md +++ b/docs/gui_basics.md @@ -181,6 +181,7 @@ Subsequently to specifying the MVC, you can calculate a number of basic MUs prop - The absolute/relative recruitment/derecruitment thresholds - The discharge rate at recruitment, derecruitment, during the steady-state phase and during the entire contraction +- The individual and average accuracy - The coefficient of variation of interspike interval - The coefficient of variation of force signal @@ -237,7 +238,7 @@ These three setting options are universally used in all plots. There are two mor ``` means that your File has 65 channels. -2. You need to specify the `Orientation` in row two and column four in the left side of the `Plot Window`. The `Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window`. +2. You need to specify the `Orientation` in row two and column four in the left side of the `Plot Window`. The `Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window`. `Orientation` is ignored when `Matrix Code` is `None`. ### Plot Raw EMG Signal 1. Click the `Plot EMGsig` button in row four and column one in the left side of the `Plot Window`, to plot the raw emg signal of your analysis file. diff --git a/docs/gui_intro.md b/docs/gui_intro.md index 5a72c63..0542f12 100644 --- a/docs/gui_intro.md +++ b/docs/gui_intro.md @@ -1,6 +1,7 @@ # Graphical Interface Welcome, to the *openhdemg* Graphical User Interface (GUI) introduction! + The *openhdemg* GUI incorporates all relevant high-level functions of the *openhdemg* library. The GUI allows you to successfully perform High-Density Electromyography (HD-EMG) data anlysis **without any programming skills required**. Moreover, there is no downside to using the GUI even if you are an experienced programmer. The GUI can be simply accessed from the command line with: @@ -27,9 +28,10 @@ This is your starting point for every analysis. On the left hand side you can fi - `DEMUSE` (.mat file used in DEMUSE) - `OTB_REFSIG` (Reference signal in the .mat file exportable by OTBiolab+) - `OPENHDEMG` (emgfile or reference signal stored in .json format) - - `CUSTOM` (custom data from a .csv file) + - `CUSTOMCSV` (custom data from a .csv file) + - `CUSTOMCSV_REFSIG` (Reference signal in a custom .csv file) - Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the `Type of file` corresponding to the type of your file. + Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the `Type of file` corresponding to the type of your file. In case you selected `OTB`, specify the `extension factor` in the dropdown. 2. To actually load the file, click the **Load File** button and select the file you want to analyse. In case of occurence, follow the error messages and repeat this and the previos step. @@ -38,6 +40,7 @@ This is your starting point for every analysis. On the left hand side you can fi ## Viewing an analysis file It doesn't get any simpler than this! + Once a file is successfully loaded as described above, you can click the `View MUs` button to plot/view your file. In the middle section of the GUI, a plot containing your data should appear. ---------------------------------------- diff --git a/docs/quick-start.md b/docs/quick-start.md index f30f900..539fc0d 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -193,9 +193,9 @@ There might be cases in which we need to remove one or more MUs from our *emgfil From the visual inspection of our plots, we can see that the firings pattern of MU number 2 (remember, Python is in base 0!!!) is not really regular. We might therefore have doubts about its quality. -A way to assess the quality of the MUs is to look at the separation between the signal and the noise. This is efficiently measured by the silouette (SIL) score. +A way to assess the quality of the MUs is to look at the separation between the signal and the noise. This is efficiently measured by accuracy scores. -This score is automatically calculated while importing the *emgfile* and can be easily accessed as `emgfile["SIL"]`. +This score is automatically calculated while importing the *emgfile* and can be easily accessed as `emgfile["ACCURACY"]`. In our sample file, the accuracy is calculated by the Silhouette (SIL) score (Negro 2016). ```Python # Import the library with the short name 'emg' @@ -205,7 +205,7 @@ import openhdemg.library as emg emgfile = emg.emg_from_samplefile() # Print the SIL score -print(emgfile["SIL"]) +print(emgfile["ACCURACY"]) """Output 0 @@ -217,7 +217,7 @@ print(emgfile["SIL"]) """ ``` -Our suspicion was right, MU number 2 has the lowest SIL score. +Our suspicion was right, MU number 2 has the lowest accuracy score. In order to remove this MU, we can use the function [delete_mus](api_tools.md#openhdemg.library.tools.delete_mus). @@ -285,35 +285,29 @@ results = emg.basic_mus_properties( print(results) """ - MVC MU_number PNR avg_PNR SIL avg_SIL abs_RT \ -0 634.0 0 27.480307 29.877575 0.899082 0.922923 30.621759 -1 NaN 1 28.946493 NaN 0.919601 NaN 32.427026 -2 NaN 2 28.640680 NaN 0.917190 NaN 68.371911 -3 NaN 3 34.442821 NaN 0.955819 NaN 118.504004 - - abs_DERT rel_RT rel_DERT DR_rec DR_derec DR_start_steady \ -0 36.168135 4.829930 5.704753 7.548770 5.449581 11.788779 -1 31.167703 5.114673 4.916041 8.344515 5.333535 11.254445 -2 67.308703 10.784213 10.616515 5.699017 3.691367 9.007505 -3 102.761472 18.691483 16.208434 5.701081 4.662196 7.393645 - - DR_end_steady DR_all_steady DR_all COVisi_steady COVisi_all \ -0 10.401857 11.154952 10.693076 6.833642 19.104306 -1 9.999033 10.751960 10.543011 8.364553 15.408739 -2 7.053079 8.168471 7.949294 10.097045 23.324503 -3 6.430807 6.908502 6.814687 11.211862 16.319474 - - COV_steady -0 1.422424 -1 NaN -2 NaN -3 NaN + MVC MU_number ACCURACY avg_ACCURACY abs_RT abs_DERT \ +0 634.0 0 0.899082 0.922923 30.621759 36.168135 +1 NaN 1 0.919601 NaN 32.427026 31.167703 +2 NaN 2 0.917190 NaN 68.371911 67.308703 +3 NaN 3 0.955819 NaN 118.504004 102.761472 + + rel_RT rel_DERT DR_rec DR_derec DR_start_steady DR_end_steady \ +0 4.829930 5.704753 7.548770 5.449581 11.788779 10.401857 +1 5.114673 4.916041 8.344515 5.333535 11.254445 9.999033 +2 10.784213 10.616515 5.699017 3.691367 9.007505 7.053079 +3 18.691483 16.208434 5.701081 4.662196 7.393645 6.430807 + + DR_all_steady DR_all COVisi_steady COVisi_all COV_steady +0 11.154952 10.693076 6.833642 19.104306 1.422424 +1 10.751960 10.543011 8.364553 15.408739 NaN +2 8.168471 7.949294 10.097045 23.324503 NaN +3 6.908502 6.814687 11.211862 16.319474 NaN """ ``` ## 7. Save the results and the edited file -It looks like we got a lot of results, which makes it extremely inefficient to copy them manually. +It looks like we got a lot of results, which makes of it extremely inefficient to copy them manually. Obviously, this can be automated using one attribute of the *results* object and we can conveniently save all the results in a .csv file. diff --git a/docs/tutorials/emgfile_structure.md b/docs/tutorials/emgfile_structure.md index 974d4bc..c1389c8 100644 --- a/docs/tutorials/emgfile_structure.md +++ b/docs/tutorials/emgfile_structure.md @@ -36,17 +36,16 @@ print(type(emgfile)) print(emgfile.keys()) """Output -dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'PNR', 'SIL', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) """ ``` That means that the `emgfile` contains the following keys (or variables, in simpler terms): -- "SOURCE" : source of the file (i.e., "CUSTOM", "DEMUSE", "OTB") +- "SOURCE" : source of the file (i.e., "CUSTOMCSV", "DEMUSE", "OTB") - "RAW_SIGNAL" : the raw EMG signal - "REF_SIGNAL" : the reference signal -- "PNR" : pulse to noise ratio -- "SIL" : silouette score +- "ACCURACY" : accuracy score (depending on source file type) - "IPTS" : pulse train (decomposed source) - "MUPULSES" : instants of firing - "FSAMP" : sampling frequency @@ -54,6 +53,7 @@ That means that the `emgfile` contains the following keys (or variables, in simp - "EMG_LENGTH" : length of the emg file (in samples) - "NUMBER_OF_MUS" : total number of MUs - "BINARY_MUS_FIRING" : binary representation of MUs firings +- "EXTRAS" : additional custom values Each key has a specific content and structure that will be presented in the next code block. @@ -71,14 +71,14 @@ info = emg.info() info.data(emgfile) """Output -Data structure of the emgfile loaded with the function emg_from_otb. --------------------------------------------------------------------- +Data structure of the emgfile +----------------------------- emgfile type is: emgfile keys are: -dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'PNR', 'SIL', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING', 'EXTRAS']) Any key can be acced as emgfile[key]. @@ -90,18 +90,18 @@ otb_testfile.mat MUST NOTE: emgfile from OTB has 64 channels, from DEMUSE 65 (includes empty channel). emgfile['RAW_SIGNAL'] is a of value: - 0 1 2 3 4 5 6 7 8 ... 55 56 57 58 59 60 61 62 63 -0 10.172526 5.086263 12.715657 11.189778 9.155273 8.138021 9.155273 13.224284 2.034505 ... 7.120768 6.612142 10.172526 8.138021 10.681152 2.034505 14.750163 4.577637 11.698405 -1 14.750163 8.138021 12.715657 12.715657 10.681152 6.612142 13.732910 16.276041 3.051758 ... 4.577637 3.560384 11.698405 7.120768 10.681152 0.508626 10.681152 4.069010 11.698405 -2 6.103516 1.017253 6.103516 15.767415 6.103516 3.051758 6.103516 11.698405 2.034505 ... 1.525879 1.525879 3.560384 -1.017253 4.069010 -4.577637 8.138021 -1.525879 5.086263 -3 -3.051758 -7.120768 -3.051758 4.577637 -4.069010 -8.138021 -2.543132 2.543132 -7.120768 ... -8.646647 -9.155273 -3.560384 -9.155273 -6.103516 -13.732910 -1.017253 -11.698405 -2.543132 -4 -11.189778 -15.767415 -15.767415 -5.086263 -11.698405 -13.732910 -7.120768 -3.560384 -12.207031 ... -15.767415 -18.310547 -12.207031 -12.715657 -11.189778 -17.293295 -8.646647 -17.293295 -11.189778 -... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... -66555 11.189778 17.801920 16.276041 17.801920 2.034505 22.379557 8.646647 14.750163 14.750163 ... -2.034505 0.508626 2.034505 2.034505 13.224284 0.000000 10.172526 10.172526 17.801920 -66556 12.715657 22.888184 21.362305 20.853678 10.172526 26.448568 12.207031 19.836426 16.276041 ... 2.034505 7.120768 4.577637 8.646647 14.241536 5.086263 18.819174 16.276041 16.276041 -66557 6.103516 7.120768 12.207031 12.715657 0.508626 16.276041 3.051758 9.663899 5.594889 ... -5.594889 -1.525879 -6.103516 -1.525879 6.103516 -1.525879 7.629395 8.646647 8.646647 -66558 -9.663899 -9.663899 -7.120768 -7.629395 -14.241536 -1.017253 -14.750163 -7.629395 -10.681152 ... -23.905436 -17.801920 -22.888184 -20.853678 -10.681152 -17.801920 -13.224284 -8.646647 -9.155273 -66559 0.508626 1.017253 0.000000 4.577637 -2.543132 6.612142 -3.051758 1.525879 -2.034505 ... -12.715657 -6.612142 -14.750163 -10.172526 0.000000 -6.103516 1.017253 -3.051758 -2.543132 + 0 1 2 3 4 5 6 7 8 ... 55 56 57 58 59 60 61 62 63 +0 10.172526 5.086263 12.715657 11.189778 9.155273 8.138021 9.155273 13.224284 2.034505 ... 7.120768 6.612142 10.172526 8.138021 10.681152 2.034505 14.750163 4.577637 11.698405 +1 14.750163 8.138021 12.715657 12.715657 10.681152 6.612142 13.732910 16.276041 3.051758 ... 4.577637 3.560384 11.698405 7.120768 10.681152 0.508626 10.681152 4.069010 11.698405 +2 6.103516 1.017253 6.103516 15.767415 6.103516 3.051758 6.103516 11.698405 2.034505 ... 1.525879 1.525879 3.560384 -1.017253 4.069010 -4.577637 8.138021 -1.525879 5.086263 +3 -3.051758 -7.120768 -3.051758 4.577637 -4.069010 -8.138021 -2.543132 2.543132 -7.120768 ... -8.646647 -9.155273 -3.560384 -9.155273 -6.103516 -13.732910 -1.017253 -11.698405 -2.543132 +4 -11.189778 -15.767415 -15.767415 -5.086263 -11.698405 -13.732910 -7.120768 -3.560384 -12.207031 ... -15.767415 -18.310547 -12.207031 -12.715657 -11.189778 -17.293295 -8.646647 -17.293295 -11.189778 +... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... +66555 11.189778 17.801920 16.276041 17.801920 2.034505 22.379557 8.646647 14.750163 14.750163 ... -2.034505 0.508626 2.034505 2.034505 13.224284 0.000000 10.172526 10.172526 17.801920 +66556 12.715657 22.888184 21.362305 20.853678 10.172526 26.448568 12.207031 19.836426 16.276041 ... 2.034505 7.120768 4.577637 8.646647 14.241536 5.086263 18.819174 16.276041 16.276041 +66557 6.103516 7.120768 12.207031 12.715657 0.508626 16.276041 3.051758 9.663899 5.594889 ... -5.594889 -1.525879 -6.103516 -1.525879 6.103516 -1.525879 7.629395 8.646647 8.646647 +66558 -9.663899 -9.663899 -7.120768 -7.629395 -14.241536 -1.017253 -14.750163 -7.629395 -10.681152 ... -23.905436 -17.801920 -22.888184 -20.853678 -10.681152 -17.801920 -13.224284 -8.646647 -9.155273 +66559 0.508626 1.017253 0.000000 4.577637 -2.543132 6.612142 -3.051758 1.525879 -2.034505 ... -12.715657 -6.612142 -14.750163 -10.172526 0.000000 -6.103516 1.017253 -3.051758 -2.543132 [66560 rows x 64 columns] @@ -121,15 +121,7 @@ emgfile['REF_SIGNAL'] is a of value: [66560 rows x 1 columns] -emgfile['PNR'] is a of value: - 0 -0 33.609118 -1 34.442821 -2 28.640680 -3 27.480307 -4 28.946493 - -emgfile['SIL'] is a of value: +emgfile['ACCURACY'] is a of value: 0 0 0.879079 1 0.955819 @@ -170,11 +162,11 @@ emgfile['MUPULSES'][0] is a of value: 53161 53390 53795 54154 54386 54823 55032 55283 55653 56026 56282 56538 56931 57578 57871 58429 59077] -emgfile['FSAMP'] is a of value: -2048 +emgfile['FSAMP'] is a of value: +2048.0 -emgfile['IED'] is a of value: -8 +emgfile['IED'] is a of value: +8.0 emgfile['EMG_LENGTH'] is a of value: 66560 @@ -197,6 +189,11 @@ emgfile['BINARY_MUS_FIRING'] is a of value 66559 0.0 0.0 0.0 0.0 0.0 [66560 rows x 5 columns] + +emgfile['EXTRAS'] is a of value: +Empty DataFrame +Columns: [0] +Index: [] """ ``` @@ -210,16 +207,17 @@ At the moment, the only alternative to the basic `emgfile` structure is reserved In this case, the `emg_refsig` is a Python dictionary with the following keys: -- "SOURCE": source of the file (i.e., "OTB_REFSIG") +- "SOURCE": source of the file (i.e., "CUSTOMCSV_REFSIG", "OTB_REFSIG") - "FSAMP": sampling frequency - "REF_SIGNAL": the reference signal +- "EXTRAS" : additional custom values ## Modify the emgfile to fit your needs This is a fundamental part of this tutorial. You can modify the `emgfile` as you wish, but there are 2 simple rules that must be followed to allow a seamless integration with the *openhdemg* functions. 1. Do not alter the `emgfile` keys. You should not add or remove keys and you should not alter the data type under each key. If you need to do so, please remember that some of the built-in functions might not work anymore and you might encounter unexpected errors. -2. Preserve data structures. If there is missing data you should fill the `emgfile` keys with the original data structure and np.nan values. The original data structure is the one presented in section [Structure of the emgfile](#structure-of-the-emgfile). +2. Preserve data structures. If there is missing data you should fill the `emgfile` keys with the original data structure. The original data structure is the one presented in section [Structure of the emgfile](#structure-of-the-emgfile). For example, the reference signal is by default contained in a pd.DataFrame. Therefore, if the reference signal is absent, the dict key "REF_SIGNAL" should contain an empty pd.DataFrame. To modify the `emgfile` you can simply act as for modifying any Python dictionary: @@ -237,7 +235,7 @@ emgfile = emg.emg_from_samplefile() print(emgfile.keys()) """Output -dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'PNR', 'SIL', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) """ # Visualise the original data structure contained in the 'REF_SIGNAL' key @@ -249,22 +247,24 @@ print(type(emgfile['REF_SIGNAL'])) # Replace the current 'REF_SIGNAL' with a random reference signal. random_data = np.random.randint(0 ,20, size=(emgfile['EMG_LENGTH'], 1)) -rand_ref = pd.DataFrame(random_data, columns=['REF_SIGNAL']) +rand_ref = pd.DataFrame(random_data, columns=[0]) emgfile['REF_SIGNAL'] = rand_ref print(emgfile['REF_SIGNAL']) """Output - REF_SIGNAL -0 37 -2 25 -3 14 -4 91 -... ... -66555 62 -66556 57 -66558 1 -66559 77 + 0 +0 2 +1 15 +2 13 +3 18 +4 3 +... .. +66555 13 +66556 11 +66557 13 +66558 1 +66559 7 """ ``` @@ -272,7 +272,7 @@ print(emgfile['REF_SIGNAL']) *openhdemg* offers a number of built-in functions to load the data from different sources. However, there might be special circumnstances that require more flexibility. In this case, the user can create custom functions to load any type of decomposed HD-EMG files. However, in order to interact with the *openhdemg* functions, the `emgfile` loaded with any custom function must respect the original [structure](#structure-of-the-emgfile). -In case your decomposed HD-EMG file does not contain a specific variable, you should mantain the original `emgfile` keys and the original data structure for each key, but filled with np.nan values. +In case your decomposed HD-EMG file does not contain a specific variable, you should mantain the original `emgfile` keys and the original data structure, although empty. ## More questions? diff --git a/docs/what's-new.md b/docs/what's-new.md index 839df6f..0bfaced 100644 --- a/docs/what's-new.md +++ b/docs/what's-new.md @@ -1,4 +1,47 @@ -:octicons-tag-24: 0.1.0-beta.1   :octicons-clock-24: June 2023 +## :octicons-tag-24: 0.1.0-beta.2 +:octicons-clock-24: September 2023 + +This release introduces important changes. It is mainly addressing the necessity of maximum flexibility and easy integration with any custom or propietary file source. This relase is not bakward compatible. + +MAJOR CHANGES: + +- **Accuracy Measurement:** Replaced the double accuracy measures in the `emgfile` (i.e., “SIL” and “PNR”) with a single accuracy measure named “ACCURACY.” For files containing the decomposed source (also named “IPTS”), the “ACCURACY” variable will contain the silhouette score (Negro et al. 2016). For files that do not contain the decomposed source, the accuracy will be the original (often proprietary) accuracy estimate. This allows for maximum flexibility and is fundamental to interface the *openhdemg* library with any proprietary and custom implementation of the different decomposition algorithms currently available. + + To accommodate this change, all the functions in the `openfile` module have been updated. Consequently, the functions using the “SIL” or “PNR” variables have also been modified. Specifially: + + - The `basic_mus_properties` function has a new input parameter (i.e., “accuracy”) to customize the returned accuracy estimate. + - In the function `remove_duplicates_between`, the input parameter “which” now only accepts “munumber” and “accuracy” instead of “munumber,” “SIL,” and “PNR.” + +- **EXTRAS Variable:** Introduced a new “EXTRAS” variable to store any custom information in the opened file. This will be accessible in the `emgfile` dictionary with the “EXTRAS” key. This variable must contain a pd.DataFrame structure and will be preserved when saving the file. This change extends the customisability of the `emgfile`. + +- **Handling Missing Variables:** Replaced “np.nan” with empty "pd.DataFrame” for missing variables upon import of files. This change ensures consistency and avoids compatibility issues with other functions. + +- **File Import Restriction:** Restricted flexibility in the import of files. To import decomposed HD-EMG files, these must contain at least the raw EMG signal and one of the times of discharge of each MU ("MUPULSES") or their binary representation. This change ensures consistency and avoids compatibility issues with other functions. + +**OTHER CHANGES:** + +- **Sampling Frequency** and **Interelectrode Distance:** Sampling frequency and interelectrode distance are now represented by float point values to accommodate different source files. + +- **emg_from_customcsv** and **emg_from_otb:** Improved robustness and flexibility, with the possibility to load custom information in “EXTRAS.” + +- **emg_from_demuse:** Improved robustness and flexibility. + +- **New Functions:** + - `refsig_from_customcsv` to load the reference signal from a custom .csv file. + - `delete_empty_mus` to delete all the MUs without firings. + +- **Exposed Function:** Exposed `mupulses_from_binary` to extract the times of firing from the binary representation of MUs firings. + +- **Dependency Management:** Addressed reported functioning issues related to external dependencies invoked by *openhdemg*. Stricter rules have been adopted in the setup.py file for automatically installing the correct version of these dependencies. + +- **Bug Fixes:** + - Fixed a BUG in the GUI when saving results in Excel files. The bug was due to changes in newer pandas versions. + - Fixed a BUG in the function “sort_mus” when empty MUs were present. + +
+ +## :octicons-tag-24: 0.1.0-beta.1 +:octicons-clock-24: June 2023 What's new? Well, everything. This is our first release, if you are using it, congratulations, you are a pioneer!