Awesome-Weakly Supervised Facial Behavior Analysis

A curated list of weakly supervised learning methods for facial behavior analysis in both classification and regression tasks based on our survey paper "Weakly Supervised Learning for Facial Behavior Analysis: A Review". In addition, this repository includes basic datasets widely used in the framework of weakly supervised learning for facial behavior analysis.

[arXiv]

If you find this work useful in your research, please consider citing our work 📝 and giving a star 🌟 :

@article{granger2021weakly,
  title={Weakly supervised learning for facial behavior analysis: A review},
  author={Praveen, R. Gnana and Cardinal, Patrick and Granger, Eric},
  journal={arXiv preprint arXiv:2101.09858},
  year={2021}
}

👀 Update frequently!

📋 What's New

• [May. 2024] Create awesome-Weakly Supervised Facial Behavior Analysis repository.
• [Oct. 2024] Updated the list of papers.

👥 Contributing

Please feel free to refer this repository for your FER research/development and send me pull requests or email Praveen to add links.

Table of Contents

• Classification

  • Inexact Supervised Learning
  • Incomplete Supervised Learning
  • Inaccurate Supervised Learning

• Regression

  • Inexact Supervised Learning
  • Incomplete Supervised Learning

Classification

Inexact Supervised Learning

Paper	Venue	Year	Code	Task
Weakly supervised pain localization using multiple instance learning	FG	2013	N/A	Pain
Multi-instance Hidden Markov Model for facial expression recognition	FG	2015	N/A	Expression
Learning pain from au combinations: A weakly supervised approach via multiple instance learning	IEEE TAFFC	2022	N/A	Pain
Regularized Multi-Concept MIL for weakly-supervised facial behavior categorization	BMVC	2014	N/A	Expression
Lomo: Latent ordinal model for facial analysis in videos	CVPR	2016	N/A	Expression
Identifying user-specific facial affects from spontaneous expressions with minimal annotation	IEEE TAFFC	2016	N/A	Expression
From Emotions to Action Units with Hidden and Semi-Hidden-Task Learning	ICCV	2015	N/A	AU
Exploring Domain Knowledge for Facial Expression-Assisted Action Unit Activation Recognition	IEEE TAFFC	2020	N/A	AU
Weakly supervised facial action unit recognition through adversarial training	CVPR	2018	N/A	AU
Weakly Supervised Facial Action Unit Recognition With Domain Knowledge	IEEE TCYB	2018	N/A	AU
Classifier Learning With Prior Probabilities for Facial Action Unit Recognition	CVPR	2018	N/A	AU

Incomplete Supervised Learning

Paper	Venue	Year	Code	Task
A Weakly Supervised learning technique for classifying facial expressions	PRL	2019	N/A	Expressions
Towards Semi-Supervised Deep Facial Expression Recognition With an Adaptive Confidence Margin	CVPR	2022	Github	Expressions
Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher	IEEE TAFFC	2023	N/A	Expressions
Margin-Mix: Semi–Supervised Learning for Face Expression Recognition	ECCV	2020	N/A	Expressions
Rethinking Pseudo-Labeling for Semi-Supervised Facial Expression Recognition With Contrastive Self-Supervised Learning	IEEE Access	2023	N/A	Expressions
Semi-supervised facial expression recognition using reduced spatial features and Deep Belief Networks	Neurocomputing	2019	N/A	Expressions
Exploiting Sparsity and Co-occurrence Structure for Action Unit Recognition	FG	2015	N/A	AU
Multi-label learning with missing labels for image annotation and facial action unit recognition	PR	2015	N/A	AU
Facial action unit recognition under incomplete data based on multi-label learning with missing labels	PR	2016	N/A	AU
Handling missing labels and class imbalance challenges simultaneously for facial action unit recognition	MTA	2019	N/A	AU
Expression-assisted facial action unit recognition under incomplete AU annotation	PR	2017	N/A	AU
Dual Semi-Supervised Learning for Facial Action Unit Recognition	AAAI	2019	N/A	AU
Deep Facial Action Unit Recognition From Partially Labeled Data	ICCV	2017	N/A	AU
Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition	NeurIPS	2019	Github	AU

Inaccurate Supervised Learning

Paper	Venue	Year	Code	Task
Suppressing Uncertainties for Large-Scale Facial Expression Recognition	CVPR	2020	Github	Expressions
Relative Uncertainty Learning for Facial Expression Recognition	NeurIPS	2021	Github	Expressions
Dive Into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition	CVPR	2021	N/A	Expressions
Teaching with Soft Label Smoothing for Mitigating Noisy Labels in Facial Expressions	ECCV	2022	Github	Expressions
Self-Paced Label Distribution Learning for In-The-Wild Facial Expression Recognition	ACM MM	2022	N/A	Expressions
Training deep networks for facial expression recognition with crowd-sourced label distribution	ACM ICMI	2016	N/A	Expressions
Facial Expression Recognition with Inconsistently Annotated Datasets	ECCV	2018	N/A	Expressions
MAN: Mining Ambiguity and Noise for Facial Expression Recognition in the Wild	PRL	2022	N/A	Expressions
Learn from All: Erasing Attention Consistency for Noisy Label Facial Expression Recognition	ECCV	2022	Github	Expressions
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild	CVPR	2017	N/A	Expressions
Learning Facial Action Units From Web Images With Scalable Weakly Supervised Clustering	CVPR	2018	Github	AU
Recognition of Action Units in the Wild with Deep Nets and a New Global-Local Loss	ICCV	2017	N/A	AU

Regression

Inexact Supervised Learning

Paper	Venue	Year	Code	Task
Multi-Instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation	ACCV	2016	N/A	Pain
Deep Weakly Supervised Domain Adaptation for Pain Localization in Videos	FG	2020	N/A	Pain
Deep domain adaptation with ordinal regression for pain assessment using weakly-labeled videos	IVC	2021	Github	Pain
Deep Recurrent Multi-instance Learning with Spatio-temporal Features for Engagement Intensity Prediction	ACM ICMI	2018	N/A	Expression
Multi-Instance Dynamic Ordinal Random Fields for Weakly Supervised Facial Behavior Analysis	TIP	2018	N/A	AU
Bilateral Ordinal Relevance Multi-Instance Regression for Facial Action Unit Intensity Estimation	CVPR	2018	N/A	AU

Incomplete Supervised Learning

Paper	Venue	Year	Code	Task
Facial Expression Intensity Estimation Using Ordinal Information	CVPR	2016	N/A	Expressions
Weakly-Supervised Deep Convolutional Neural Network Learning for Facial Action Unit Intensity Estimation	CVPR	2018	N/A	AU
Joint Representation and Estimator Learning for Facial Action Unit Intensity Estimation	CVPR	2019	N/A	AU
Context-Aware Feature and Label Fusion for Facial Action Unit Intensity Estimation With Partially Labeled Data	ICCV	2019	N/A	AU
Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units	IEEE Access	2019	N/A	AU
Deep Facial Action Unit Recognition and Intensity Estimation from Partially Labelled Data	IEEE TAFFC	2021	N/A	AU
Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning	ACCV	2020	N/A	AU