Machine-Learning-Research-Papers

A list of research papers in the domain of machine learning, deep learning and related fields.

I have curated a list of research papers that I come across and read. I'll keep on updating the list of papers and their summary as I read them every week.

How to read a Research Paper

Professor Andrew Ng gave some awesome tips on how to read a research paper. I have summarised the tips in this PDF.

Table of Contents

The list of papers can be viewed based on differentiating criteria's such as (Conference venue, Year Published, Topic Covered, Authors, etc.).

The following filtered formats are available to view paper's list:

• All Papers
• Read and Summarised Papers
• Conference-wise Filtered Papers
• Year-wise Filtered Papers
• Topic-wise Filtered Papers
• Category-wise Filtered Papers
• Author-wise Filtered Papers

All Papers

	Paper Name	Status	Topic	Category	Year	Conference	Author	Summary	Link
0	ZF Net (Visualizing and Understanding Convolutional Networks)	Read	CNNs, CV , Image	Visualization	2014	ECCV	Matthew D. Zeiler, Rob Fergus	Visualize CNN Filters / Kernels using De-Convolutions on CNN filter activations.	link
1	Inception-v1 (Going Deeper With Convolutions)	Read	CNNs, CV , Image	Architecture	2015	CVPR	Christian Szegedy, Wei Liu	Propose the use of 1x1 conv operations to reduce the number of parameters in a deep and wide CNN	link
2	ResNet (Deep Residual Learning for Image Recognition)	Read	CNNs, CV , Image	Architecture	2016	CVPR	Kaiming He, Xiangyu Zhang	Introduces Residual or Skip Connections to allow increase in the depth of a DNN	link
3	MobileNet (Efficient Convolutional Neural Networks for Mobile Vision Applications)	Pending	CNNs, CV , Image	Architecture, Optimization-No. of params	2017	arXiv	Andrew G. Howard, Menglong Zhu		link
4	Evaluation of neural network architectures for embedded systems	Read	CNNs, CV , Image	Comparison	2017	IEEE ISCAS	Adam Paszke, Alfredo Canziani, Eugenio Culurciello	Compare CNN classification architectures on accuracy, memory footprint, parameters, operations count, inference time and power consumption.	link
5	SqueezeNet	Read	CNNs, CV , Image	Architecture, Optimization-No. of params	2016	arXiv	Forrest N. Iandola, Song Han	Explores model compression by using 1x1 convolutions called fire modules.	link
6	Pruning Filters for Efficient ConvNets	Pending	CNNs, CV , Image	Optimization-No. of params	2017	arXiv	Asim Kadav, Hao Li		link
7	Attention is All you Need	Read	Attention, Text , Transformers	Architecture	2017	NIPS	Ashish Vaswani, Illia Polosukhin, Noam Shazeer, Łukasz Kaiser	Talks about Transformer architecture which brings SOTA performance for different tasks in NLP	link
8	GPT-2 (Language Models are Unsupervised Multitask Learners)	Pending	Attention, Text , Transformers		2019		Alec Radford, Dario Amodei, Ilya Sutskever, Jeffrey Wu		link
9	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	Read	Attention, Text , Transformers	Embeddings	2018	NAACL	Jacob Devlin, Kenton Lee, Kristina Toutanova, Ming-Wei Chang	BERT is an extension to Transformer based architecture which introduces a masked word pretraining and next sentence prediction task to pretrain the model for a wide variety of tasks.	link
10	SAGAN: Self-Attention Generative Adversarial Networks	Pending	Attention, GANs, Image	Architecture	2018	arXiv	Augustus Odena, Dimitris Metaxas, Han Zhang, Ian Goodfellow		link
11	Single Headed Attention RNN: Stop Thinking With Your Head	Pending	Attention, LSTMs, Text	Optimization-No. of params	2019	arXiv	Stephen Merity		link
12	Reformer: The Efficient Transformer	Read	Attention, Text , Transformers	Architecture, Optimization-Memory, Optimization-No. of params	2020	arXiv	Anselm Levskaya, Lukasz Kaiser, Nikita Kitaev	Overcome time and memory complexity of Transformers by bucketing Query, Keys and using Reversible residual connections.	link
13	A 2019 guide to Human Pose Estimation with Deep Learning	Pending	CV , Pose Estimation	Comparison	2019	Blog	Sudharshan Chandra Babu		link
14	A Simple yet Effective Baseline for 3D Human Pose Estimation	Pending	CV , Pose Estimation		2017	ICCV	James J. Little, Javier Romero, Julieta Martinez, Rayat Hossain		link
15	Bag of Tricks for Image Classification with Convolutional Neural Networks	Read	CV , Image	Optimizations, Tips & Tricks	2018	arXiv	Tong He, Zhi Zhang	Shows a dozen tricks (mixup, label smoothing, etc.) to improve CNN accuracy and training time.	link
16	Class-Balanced Loss Based on Effective Number of Samples	Pending	Loss Function	Tips & Tricks	2019	CVPR	Menglin Jia, Yin Cui		link
17	Self-Normalizing Neural Networks	Pending	Activation Function, Tabular	Optimizations, Tips & Tricks	2017	NIPS	Andreas Mayr, Günter Klambauer, Thomas Unterthiner		link
18	A Comprehensive Guide on Activation Functions	This week	Activation Function		2020	Blog	Ygor Rebouças Serpa		link
19	Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet	Reading	CNNs, CV , Image		2019	arXiv	Matthias Bethge, Wieland Brendel		link
20	Breaking neural networks with adversarial attacks	Pending	CNNs, Image	Adversarial	2019	Blog	Anant Jain		link
21	The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	Read	NN Initialization, NNs	Optimization-No. of params, Tips & Tricks	2019	ICLR	Jonathan Frankle, Michael Carbin	Lottery ticket hypothesis: dense, randomly-initialized, feed-forward networks contain subnetworks (winning tickets) that—when trained in isolation— reach test accuracy comparable to the original network in a similar number of iterations.	link
22	All you need is a good init	Pending	NN Initialization	Tips & Tricks	2015	arXiv	Dmytro Mishkin, Jiri Matas		link
23	Pix2Pix: Image-to-Image Translation with Conditional Adversarial Nets	Read	GANs, Image		2017	CVPR	Alexei A. Efros, Jun-Yan Zhu, Phillip Isola, Tinghui Zhou	Image to image translation using Conditional GANs and dataset of image pairs from one domain to another.	link
24	CycleGAN: Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks	Pending	GANs, Image	Architecture	2017	ICCV	Alexei A. Efros, Jun-Yan Zhu, Phillip Isola, Taesung Park		link
25	Language-Agnostic BERT Sentence Embedding	Read	Attention, Siamese Network, Text , Transformers	Embeddings	2020	arXiv	Fangxiaoyu Feng, Yinfei Yang	A BERT model with multilingual sentence embeddings learned over 112 languages and Zero-shot learning over unseen languages.	link
26	Phrase-Based & Neural Unsupervised Machine Translation	Pending	NMT, Text , Transformers	Unsupervised	2018	arXiv	Alexis Conneau, Guillaume Lample, Ludovic Denoyer, Marc'Aurelio Ranzato, Myle Ott		link
27	Unsupervised Machine Translation Using Monolingual Corpora Only	Pending	GANs, NMT, Text , Transformers	Unsupervised	2017	arXiv	Alexis Conneau, Guillaume Lample, Ludovic Denoyer, Marc'Aurelio Ranzato, Myle Ott		link
28	Cross-lingual Language Model Pretraining	Pending	NMT, Text , Transformers	Unsupervised	2019	arXiv	Alexis Conneau, Guillaume Lample		link
29	Word2Vec: Efficient Estimation of Word Representations in Vector Space	Pending	Text	Embeddings, Tips & Tricks	2013	arXiv	Greg Corrado, Jeffrey Dean, Kai Chen, Tomas Mikolov		link
30	Capsule Networks: Dynamic Routing Between Capsules	Pending	CV , Image	Architecture	2017	arXiv	Geoffrey E Hinton, Nicholas Frosst, Sara Sabour		link
31	Graph Neural Network: Relational inductive biases, deep learning, and graph networks	Pending	GraphNN	Architecture	2018	arXiv	Jessica B. Hamrick, Oriol Vinyals, Peter W. Battaglia		link
32	Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs	Pending	CNNs, Image		2020	arXiv	Ari S. Morcos, David J. Schwab, Jonathan Frankle		link
33	Arbitrary Style Transfer in Real-Time With Adaptive Instance Normalization	Pending	CNNs, Image		2017	ICCV	Serge Belongie, Xun Huang		link
34	How Does Batch Normalization Help Optimization?	Pending	NNs, Normalization	Optimizations	2018	arXiv	Aleksander Madry, Andrew Ilyas, Dimitris Tsipras, Shibani Santurkar		link
35	WGAN: Wasserstein GAN	Pending	GANs, Loss Function		2017	arXiv	Léon Bottou, Martin Arjovsky, Soumith Chintala		link
36	Group Normalization	Pending	NNs, Normalization	Optimizations	2018	arXiv	Kaiming He, Yuxin Wu		link
37	Spectral Normalization for GANs	Pending	GANs, Normalization	Optimizations	2018	arXiv	Masanori Koyama, Takeru Miyato, Toshiki Kataoka, Yuichi Yoshida		link
38	One-shot Text Field Labeling using Attention and Belief Propagation for Structure Information Extraction	Pending	Image , Text		2020	arXiv	Jun Huang, Mengli Cheng, Minghui Qiu, Wei Lin, Xing Shi		link
39	Perceptual Losses for Real-Time Style Transfer and Super-Resolution	Pending	Loss Function, NNs		2016	ECCV	Alexandre Alahi, Justin Johnson, Li Fei-Fei		link
40	Topological Loss: Beyond the Pixel-Wise Loss for Topology-Aware Delineation	Pending	Image , Loss Function, Segmentation		2018	CVPR	Agata Mosinska, Mateusz Koziński, Pablo Márquez-Neila, Pascal Fua		link
41	Understanding Loss Functions in Computer Vision	Pending	CV , GANs, Image , Loss Function	Comparison, Tips & Tricks	2020	Blog	Sowmya Yellapragada		link
42	NADAM: Incorporating Nesterov Momentum into Adam	Pending	NNs, Optimizers	Comparison	2016		Timothy Dozat		link
43	Deep Double Descent: Where Bigger Models and More Data Hurt	Pending	NNs		2019	arXiv	Boaz Barak, Gal Kaplun, Ilya Sutskever, Preetum Nakkiran, Tristan Yang, Yamini Bansal		link
44	StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks	Pending	GANs, Image		2019	CVPR	Samuli Laine, Tero Karras, Timo Aila		link
45	Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?	Pending	GANs, Image		2019	ICCV	Peter Wonka, Rameen Abdal, Yipeng Qin		link
46	Improved Techniques for Training GANs	Pending	GANs, Image	Semi-Supervised	2016	NIPS	Alec Radford, Ian Goodfellow, Tim Salimans, Vicki Cheung, Wojciech Zaremba, Xi Chen		link
47	AnimeGAN: Towards the Automatic Anime Characters Creation with Generative Adversarial Networks	Pending	GANs, Image		2017	NIPS	Jiakai Zhang, Minjun Li, Yanghua Jin		link
48	Progressive Growing of GANs for Improved Quality, Stability, and Variation	Pending	GANs, Image	Tips & Tricks	2018	ICLR	Jaakko Lehtinen, Samuli Laine, Tero Karras, Timo Aila		link
49	BEGAN: Boundary Equilibrium Generative Adversarial Networks	Pending	GANs, Image		2017	arXiv	David Berthelot, Luke Metz, Thomas Schumm		link
50	Adam: A Method for Stochastic Optimization	Pending	NNs, Optimizers		2015	ICLR	Diederik P. Kingma, Jimmy Ba		link
51	StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation	Pending	GANs, Image		2018	CVPR	Jaegul Choo, Jung-Woo Ha, Minje Choi, Munyoung Kim, Sunghun Kim, Yunjey Choi		link
52	IMLE-GAN: Inclusive GAN: Improving Data and Minority Coverage in Generative Models	Pending	GANs		2020	arXiv	Jitendra Malik, Ke Li, Larry Davis, Mario Fritz, Ning Yu, Peng Zhou		link
53	Few-Shot Learning with Localization in Realistic Settings	Pending	CNNs, Image	Few-shot-learning	2019	CVPR	Bharath Hariharan, Davis Wertheimer		link
54	Revisiting Pose-Normalization for Fine-Grained Few-Shot Recognition	Pending	CNNs, Image	Few-shot-learning	2020	CVPR	Bharath Hariharan, Davis Wertheimer, Luming Tang		link
55	ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning	Pending	AGI, Dataset, Text		2019	AAAI	Maarten Sap, Noah A. Smith, Ronan Le Bras, Yejin Choi		link
56	COMET: Commonsense Transformers for Automatic Knowledge Graph Construction	Pending	AGI, Text , Transformers		2019	ACL	Antoine Bosselut, Hannah Rashkin, Yejin Choi		link
57	VisualCOMET: Reasoning about the Dynamic Context of a Still Image	Pending	AGI, Dataset, Image , Text , Transformers		2020	ECCV	Ali Farhadi, Chandra Bhagavatula, Jae Sung Park, Yejin Choi		link
58	Occupancy Anticipation for Efficient Exploration and Navigation	Pending	CNNs, Image	Reinforcement-Learning	2020	ECCV	Kristen Grauman, Santhosh K. Ramakrishnan, Ziad Al-Halah		link
59	T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	Read	Attention, Text , Transformers		2020	JMLR	Colin Raffel, Noam Shazeer, Peter J. Liu, Wei Liu, Yanqi Zhou	Presents a Text-to-Text transformer model with multi-task learning capabilities, simultaneously solving problems such as machine translation, document summarization, question answering, and classification tasks.	link
60	GPT-f: Generative Language Modeling for Automated Theorem Proving	Pending	Attention, Transformers		2020	arXiv	Ilya Sutskever, Stanislas Polu		link
61	Vision Transformer: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	Pending	Attention, Image , Transformers		2021	ICLR	Alexey Dosovitskiy, Jakob Uszkoreit, Lucas Beyer, Neil Houlsby		link
62	MuZero: Mastering Go, chess, shogi and Atari without rules	Pending		Reinforcement-Learning	2020	Nature	David Silver, Demis Hassabis, Ioannis Antonoglou, Julian Schrittwiese		link
63	Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask	Read	NN Initialization, NNs	Comparison, Optimization-No. of params, Tips & Tricks	2019	NeurIPS	Hattie Zhou, Janice Lan, Jason Yosinski, Rosanne Liu	Follow up on Lottery Ticket Hypothesis exploring the effects of different Masking criteria as well as Mask-1 and Mask-0 actions.	link
64	DALL·E: Creating Images from Text	Pending	Image , Text , Transformers		2021	Blog	Aditya Ramesh, Gabriel Goh, Ilya Sutskever, Mikhail Pavlov, Scott Gray		link
65	CLIP: Connecting Text and Images	Pending	Image , Text , Transformers	Multimodal, Pre-Training	2021	arXiv	Alec Radford, Ilya Sutskever, Jong Wook Kim		link
66	Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision	This week	Image , Text , Transformers	Multimodal	2020	EMNLP	Hao Tan, Mohit Bansal		link
67	SpanBERT: Improving Pre-training by Representing and Predicting Spans	Read	Question-Answering, Text , Transformers	Pre-Training	2020	TACL	Danqi Chen, Mandar Joshi	A different pre-training strategy for BERT model to improve performance for Question Answering task.	link
68	Learning to Extract Attribute Value from Product via Question Answering: A Multi-task Approach	Read	Question-Answering, Text , Transformers	Zero-shot-learning	2020	KDD	Li Yang, Qifan Wang	Question Answering BERT model used to extract attributes from products. Introduce further No Answer loss and distillation to promote zero shot learning.	link
69	TransGAN: Two Transformers Can Make One Strong GAN	Pending	GANs, Image , Transformers	Architecture	2021	arXiv	Shiyu Chang, Yifan Jiang, Zhangyang Wang		link
70	Interpreting Deep Learning Models in Natural Language Processing: A Review	Pending	Text	Comparison, Visualization	2021	arXiv	Diyi Yang, Xiaofei Sun		link
71	Symbolic Knowledge Distillation: from General Language Models to Commonsense Models	Pending	Dataset, Text , Transformers	Optimizations, Tips & Tricks	2021	arXiv	Chandra Bhagavatula, Jack Hessel, Peter West, Yejin Choi		link
72	Chain of Thought Prompting Elicits Reasoning in Large Language Models	Pending	Question-Answering, Text , Transformers		2022	arXiv	Denny Zhou, Jason Wei, Xuezhi Wang		link
73	Transforming Sequence Tagging Into A Seq2Seq Task	Pending	Generative, Text	Comparison, Tips & Tricks	2022	arXiv	Iftekhar Naim, Karthik Raman, Krishna Srinivasan		link
74	Large Language Models are Zero-Shot Reasoners	Pending	Generative, Question-Answering, Text	Tips & Tricks, Zero-shot-learning	2022	arXiv	Takeshi Kojima, Yusuke Iwasawa		link
75	Flan-T5: Scaling Instruction-Finetuned Language Models	Pending	Generative, Text , Transformers	Architecture, Pre-Training	2022	arXiv	Hyung Won Chung, Le Hou		link
76	Decoding a Neural Retriever’s Latent Space for Query Suggestion	Pending	Text	Embeddings, Latent space	2022	arXiv	Christian Buck, Leonard Adolphs, Michelle Chen Huebscher		link
77	Training Compute-Optimal Large Language Models	Pending	Large-Language-Models, Transformers	Architecture, Optimization-No. of params, Pre-Training, Tips & Tricks	2022	arXiv	Jordan Hoffmann, Laurent Sifre, Oriol Vinyals, Sebastian Borgeaud		link
78	VL-T5: Unifying Vision-and-Language Tasks via Text Generation	Read	CNNs, CV , Generative, Image , Large-Language-Models, Question-Answering, Text , Transformers	Architecture, Embeddings, Multimodal, Pre-Training	2021	arXiv	Hao Tan, Jaemin Cho, Jie Le, Mohit Bansal	Unifying two modalities (image and text) together in a single transformer model to solve multiple tasks in a single architecture using text prefixes similar to T5.	link
79	Scaling Instruction-Finetuned Language Models (FLAN)	Pending	Generative, Large-Language-Models, Question-Answering, Text , Transformers	Instruction-Finetuning	2022	arXiv	Hyung Won Chung, Jason Wei, Jeffrey Dean, Le Hou, Quoc V. Le, Shayne Longpre	https://arxiv.org/abs/2210.11416 introduces FLAN (Fine-tuned LAnguage Net), an instruction finetuning method, and presents the results of its application. The study demonstrates that by fine-tuning the 540B PaLM model on 1836 tasks while incorporating Chain-of-Thought Reasoning data, FLAN achieves improvements in generalization, human usability, and zero-shot reasoning over the base model. The paper also provides detailed information on how each these aspects was evaluated.	link
80	ReAct: Synergizing Reasoning and Acting in Language Models	Pending	Generative, Large-Language-Models, Text	Optimizations, Tips & Tricks	2023	ICLR	Dian Yu, Izhak Shafran, Jeffrey Zhao, Karthik Narasimhan, Nan Du, Shunyu Yao, Yuan Cao	This paper introduces ReAct, a novel approach that leverages Large Language Models (LLMs) to interleave reasoning traces and task-specific actions. ReAct outperforms existing methods on various language and decision-making tasks, addressing issues like hallucination, error propagation, and improving human interpretability and trustworthiness.	link
81	Training language models to follow instructions with human feedback	Pending	Generative, Large-Language-Models, Training Method	Instruction-Finetuning, Reinforcement-Learning, Semi-Supervised	2022	arXiv	Carroll L. Wainwright, Diogo Almeida, Jan Leike, Jeff Wu, Long Ouyang, Pamela Mishkin, Paul Christiano, Ryan Lowe, Xu Jiang	This paper presents InstructGPT, a model fine-tuned with human feedback to better align with user intent across various tasks. Despite having significantly fewer parameters than larger models, InstructGPT outperforms them in human evaluations, demonstrating improved truthfulness, reduced toxicity, and minimal performance regressions on public NLP datasets, highlighting the potential of fine-tuning with human feedback for enhancing language model alignment with human intent.	link
82	Constitutional AI: Harmlessness from AI Feedback	Pending	Generative, Large-Language-Models, Training Method	Instruction-Finetuning, Reinforcement-Learning, Unsupervised	2022	arXiv	Jared Kaplan, Yuntao Ba	The paper introduces Constitutional AI, a method for training a safe AI assistant without human-labeled data on harmful outputs. It combines supervised learning and reinforcement learning phases, enabling the AI to engage with harmful queries by explaining its objections, thus improving control, transparency, and human-judged performance with minimal human oversight.	link
83	Self-Alignment with Instruction Backtranslation	Pending	Generative, Large-Language-Models, Training Method	Instruction-Finetuning	2023	arXiv	Jason Weston, Mike Lewis, Ping Yu, Xian Li	The paper introduces a scalable method called "instruction backtranslation" to create a high-quality instruction-following language model. This method involves self-augmentation and self-curation of training examples generated from web documents, resulting in a model that outperforms others in its category without relying on distillation data, showcasing its effective self-alignment capability.	link
84	Table-GPT: Table-tuned GPT for Diverse Table Tasks	Pending	Generative, Large-Language-Models, Training Method	Instruction-Finetuning	2023	arXiv			link
85	Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering	Pending	Large-Language-Models	Prompting, Tips & Tricks	2024	arXiv	Dedy Kredo, Itamar Friedman, Tal Ridnik	This paper introduces AlphaCodium, a novel test-based, multi-stage, code-oriented iterative approach for improving the performance of Language Model Models (LLMs) on code generation tasks.	link
86	Large Language Models for Data Annotation: A Survey	This week	Dataset, Generative, Large-Language-Models	Prompting, Tips & Tricks	2024	arXiv	Alimohammad Beigi, Zhen Tan		link