Leaderboard and result reports

Report for the evaluation of pipelines on MOABB datasets

Motor imagery

Left/Right hand classes

Data filtered between 7-35 Hz. Epoch length are dataset-dependent and use maximum window length. Average score is computed across subjects.

TODO:

• add sessions above chance level, using combrisson algorithm to compute chance level
• check results on collab (C) and local machine (L) and complete confirmed column

Dataset	Evaluation type	Pipeline	Avg score	Confirmed
001-2014	WithinSession	CSP + LDA	0.823360	L
		MDM	0.816943	L
		CSP + SVM	0.828946	L
		Log Variance LDA	0.779562	L
		Tangent Space LR	0.874123	L
		Log Variance SVM	0.711478	L
		Tangent Space SVM	0.864214	L
		CSP + optSVM	0.820544	L
004-2014	WithinSession	CSP + LDA	0.800957	L
		MDM	0.776555	L
		CSP + SVM	0.796910	L
		Log Variance LDA	0.785073	L
		Tangent Space LR	0.800938	L
		Log Variance SVM	0.779423	L
		Tangent Space SVM	0.798726	L
		CSP + optSVM	0.791219	L
Cho2017	WithinSession	CSP + LDA	0.711742
		MDM	0.635015
		CSP + SVM	0.712101
		Log Variance LDA	0.645397
		Tangent Space LR	0.745154
		Log Variance SVM	0.661121
		Tangent Space SVM	0.743365
		CSP + optSVM	0.706311
Grosse-Wentrup 2009	WithinSession	CSP + LDA	0.764378
		MDM	0.642911
		CSP + SVM	0.771578
		Log Variance LDA	0.787111
		Tangent Space LR	0.876000
		Log Variance SVM	0.809867
		Tangent Space SVM	0.884600
		CSP + optSVM	0.775644
Lee2019_MI	WithinSession	CSP + LDA	0.768815
		MDM	0.702296
		CSP + SVM	0.769074
		Log Variance LDA	0.662278
		Tangent Space LR	0.830926
		Log Variance SVM	0.740407
		Tangent Space SVM	0.841815
		CSP + optSVM	0.765315
Physionet Motor Imagery	WithinSession	CSP + LDA	0.657480
		MDM	0.547599
		CSP + SVM	0.651111
		Log Variance LDA	0.619386
		Tangent Space LR	0.672834
		Log Variance SVM	0.624919
		Tangent Space SVM	0.694299
		CSP + optSVM	0.648555
Schirrmeister2017	WithinSession	CSP + LDA	0.772348
		MDM	0.615261
		CSP + SVM	0.771553
		Log Variance LDA	0.784386
		Tangent Space LR	0.872200
		Log Variance SVM	0.787537
		Tangent Space SVM	0.876428
		CSP + optSVM	0.769153
Shin2017A	WithinSession	CSP + LDA	0.722989
		MDM	0.629885
		CSP + SVM	0.713218
		Log Variance LDA	0.617816
		Tangent Space LR	0.693103
		Log Variance SVM	0.615517
		Tangent Space SVM	0.721839
		CSP + optSVM	0.729310
Weibo 2014	WithinSession	CSP + LDA	0.807191
		MDM	0.588015
		CSP + SVM	0.801518
		Log Variance LDA	0.741323
		Tangent Space LR	0.836191
		Log Variance SVM	0.728597
		Tangent Space SVM	0.837777
		CSP + optSVM	0.798406
Zhou 2016	WithinSession	CSP + LDA	0.931489
		MDM	0.907018
		CSP + SVM	0.932315
		Log Variance LDA	0.883861
		Tangent Space LR	0.941601
		Log Variance SVM	0.854676
		Tangent Space SVM	0.938613
		CSP + optSVM	0.928818

Right hand/feet classes

Dataset	Pipeline	Evaluation type	Average score across subjects	Sessions above chance level

All classes of dataset

Dataset	Pipeline	Evaluation type	Avg score	#Classes	Confirmed
Alexandre Motor Imagery	WithinSession	DLCSPauto + shLDA	0.606250	3
		CSP + LDA	0.610417	3
		TS + optSVM	0.675000	3
		MDM	0.606250	3
		CSP + SVM	0.622917	3
		AM + optSVM	0.497917	3
		Tangent Space LR	0.691667	3
		CSP + optSVM	0.622917	3
Zhou 2016	WithinSession	DLCSPauto + shLDA	0.820614	3
		CSP + LDA	0.829631	3
		TS + optSVM	0.834106	3
		MDM	0.760476	3
		CSP + SVM	0.831694	3
		AM + optSVM	0.742366	3
		Tangent Space LR	0.848793	3
		CSP + optSVM	0.818829	3