This repository contains a list of NLP paper summaries intended to make NLP techniques and topics more approachable and accessible. Work in progress. There is a lot more to be added/improved in the coming days.
If you have blogged about an NLP paper or technique or find an interesting read out there, I encourage you to share with our community. To add your blog posts or summaries to this list just hit on the edit button on this README.md. You can then add your entry by modiying the readme file and submitting a PR which will be reviewed before going live. Alternatively, we can work on transferring the summaries to this repo itself so as to make them more accessible, which is what I am currently doing with some of my own paper summaries below.
If you would like to contribute by blogging about an NLP paper/technique, you can check out our suggestion/guidance at this issue.
And if you need any ideas on how else to contribute to this repo, take a look in the issues section. We are in need of maintainers.
For now, I have adopted a few tracks from ACL for the grouping but this can change based on the granularity of grouping that is needed. Open to ideas here.
Note that we currently provide the source of the where the summary originated from. We are working with a few authors to migrate the content directly to this repo so that summaries are centralized and easily accessible. This also simplifies the way others can contribute to this project. When a summary has been fully migrated to this repo, we will tag the summary as "GitHub" under the "Summary" tab of the tables below to identify them easily.
We are including an extra TL;DR section wherever applicable. This is not meant as a full-fledged summary but rather covers the key points of each paper and serves as a refresher for those who have previously encountered the paper.
If you are facing any issues submitting your PR, just send me an email at ellfae@gmail.com or DM me on Twitter.
- Cognitive Modeling and Psycholinguistics
- Computational Social Science and Social Media
- Dialogue and Interactive Systems
- Ethics and NLP
- Generation
- Interpretability and Analysis of Models for NLP
- Language Modeling
- Machine Translation
- Multi-Task Learning
- Resources and Evaluation
- Sentiment Analysis, Stylistic Analysis, and Argument Mining
- Summarization
- Overviews / Surveys / Highlights
- Miscellaneous (To be categorized)
| Title | Summary | Paper Source |
|---|---|---|
| Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Multimodal Classification for Analysing Social Media | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Towards a Human-like Open-Domain Chatbot | Google AI Blog | Paper |
| Title | Summary | Paper Source | TL;DR |
|---|---|---|---|
| What makes a good conversation? How controllable attributes affect human judgements | Abigail See | Paper | Richard Csaky |
| Learning from Dialogue after Deployment: Feed Yourself, Chatbot! | - | Paper | Richard Csaky |
| TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents | - | Paper | Richard Csaky |
| Learning to Select Knowledge for Response Generation in Dialog Systems | - | Paper | Richard Csaky |
| Consistent Dialogue Generation with Self-supervised Feature Learning | - | Paper | Richard Csaky |
| Evaluating Coherence in Dialogue Systems using Entailment | - | Paper | Richard Csaky |
| Pretraining Methods for Dialog Context Representation Learning | - | Paper | Richard Csaky |
| Self-Supervised Dialogue Learning | - | Paper | Richard Csaky |
| Structured Fusion Networks for Dialog | - | Paper | Richard Csaky |
| Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References | - | Paper | Richard Csaky |
| Multi-Granularity Representations of Dialog | - | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Personalizing Dialogue Agents: I have a dog, do you have pets too? | Paper | Richard Csaky |
| Building a Conversational Agent Overnight with Dialogue Self-Play | Paper | Richard Csaky |
| Topic-based Evaluation for Conversational Bots | Paper | Richard Csaky |
| Improving Variational Encoder-Decoders in Dialogue Generation | Paper | Richard Csaky |
| A Hierarchical Latent Structure for Variational Conversation Modeling | Paper | Richard Csaky |
| Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation | Paper | Richard Csaky |
| Sounding Board: A User-Centric and Content-Driven Social Chatbot | Paper | Richard Csaky |
| DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder | Paper | Richard Csaky |
| Reinforcing Coherence for Sequence to Sequence Model in Dialogue Generation | Paper | Richard Csaky |
| Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders | Paper | Richard Csaky |
| Multi-turn Dialogue Response Generation in an Adversarial Learning Framework | Paper | Richard Csaky |
| Zero-Shot Dialog Generation with Cross-Domain Latent Actions | Paper | Richard Csaky |
| Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts | Paper | Richard Csaky |
| Why Do Neural Response Generation Models Prefer Universal Replies? | Paper | Richard Csaky |
| Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner | Paper | Richard Csaky |
| Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints | Paper | Richard Csaky |
| Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models | Paper | Richard Csaky |
| Training Millions of Personalized Dialogue Agents | Paper | Richard Csaky |
| Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization | Paper | Richard Csaky |
| Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity | Paper | Richard Csaky |
| A Dataset for Document Grounded Conversations | Paper | Richard Csaky |
| Talking to myself: self-dialogues as data for conversational agents | Paper | Richard Csaky |
| Neural Approaches to Conversational AI | Paper | Richard Csaky |
| Contextual Topic Modeling for Dialog Systems | Paper | Richard Csaky |
| Automatic Evaluation of Neural Personality-based Chatbots | Paper | Richard Csaky |
| NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation | Paper | Richard Csaky |
| MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling | Paper | Richard Csaky |
| The RLLChatbot: a solution to the ConvAI challenge | Paper | Richard Csaky |
| Neural Response Ranking for Social Conversation: A Data-Efficient Approach | Paper | Richard Csaky |
| Generating Multiple Diverse Responses for Short-Text Conversation | Paper | Richard Csaky |
| Wizard of Wikipedia: Knowledge-Powered Conversational agents | Paper | Richard Csaky |
| Importance of a Search Strategy in Neural Dialogue Modelling | Paper | Richard Csaky |
| A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems | Paper | Richard Csaky |
| Adversarial Learning for Neural Dialogue Generation | Paper | Richard Csaky |
| Hierarchical Recurrent Attention Network for Response Generation | Paper | Richard Csaky |
| A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue | Paper | Richard Csaky |
| Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models | Paper | Richard Csaky |
| A Knowledge-Grounded Neural Conversation Model | Paper | Richard Csaky |
| Batch Policy Gradient Methods for Improving Neural Conversation Models | Paper | Richard Csaky |
| Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning | Paper | Richard Csaky |
| Learning Conversational Systems that Interleave Task and Non-Task Content | Paper | Richard Csaky |
| Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders | Paper | Richard Csaky |
| Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory | Paper | Richard Csaky |
| Not All Dialogues are Created Equal: Instance Weighting for Neural Conversational Models | Paper | Richard Csaky |
| Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems | Paper | Richard Csaky |
| A Conditional Variational Framework for Dialog Generation | Paper | Richard Csaky |
| ParlAI: A Dialog Research Software Platform | Paper | Richard Csaky |
| Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols | Paper | Richard Csaky |
| Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability | Paper | Richard Csaky |
| Personalization in Goal-Oriented Dialog | Paper | Richard Csaky |
| Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog | Paper | Richard Csaky |
| Deal or No Deal? End-to-End Learning for Negotiation Dialogues | Paper | Richard Csaky |
| DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks | Paper | Richard Csaky |
| Enterprise to Computer: Star Trek chatbot | Paper | Richard Csaky |
| Domain Aware Neural Dialog System | Paper | Richard Csaky |
| Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses | Paper | Richard Csaky |
| A Deep Reinforcement Learning Chatbot | Paper | Richard Csaky |
| Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena | Paper | Richard Csaky |
| Flexible End-to-End Dialogue System for Knowledge Grounded Conversation | Paper | Richard Csaky |
| Edina: Building an Open Domain Socialbot with Self-dialogues | Paper | Richard Csaky |
| Interactive Policy Learning In End-to-End Trainable Task-Oriented Neural Dialog Models | Paper | Richard Csaky |
| Augmenting End-to-End Dialog Systems with Commonsense Knowledge | Paper | Richard Csaky |
| Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models | Paper | Richard Csaky |
| DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset | Paper | Richard Csaky |
| A Dual Encoder Sequence to Sequence Model for Open-Domain Dialogue Modeling | Paper | Richard Csaky |
| Adversarial Advantage Actor-Critic Model For Task-Completion Dialogue Policy Learning | Paper | Richard Csaky |
| Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models | Paper | Richard Csaky |
| End-to-end Adversarial Learning for Generative Conversational Agents | Paper | Richard Csaky |
| Fine Grained Knowledge Transfer for Personalized Task-oriented Dialogue Systems | Paper | Richard Csaky |
| BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems | Paper | Richard Csaky |
| End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning | Paper | Richard Csaky |
| Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning | Paper | Richard Csaky |
| RubyStar: A Non-Task-Oriented Mixture Model Dialog System | Paper | Richard Csaky |
| Examining Cooperation in Visual Dialog Models | Paper | Richard Csaky |
| Why Do Neural Dialog Systems Generate Short and Meaningless Replies? A Comparison between Dialog and Translation | Paper | Richard Csaky |
| End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient | Paper | Richard Csaky |
| Toward Continual Learning for Conversational Agents | Paper | Richard Csaky |
| An Ensemble Model with Ranking for Social Dialogue | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Incorporating Copying Mechanism in Sequence-to-Sequence Learning | Paper | Richard Csaky |
| A Persona-Based Neural Conversation Model | Paper | Richard Csaky |
| How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation | Paper | Richard Csaky |
| StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation | Paper | Richard Csaky |
| A Network-based End-to-End Trainable Task-oriented Dialogue System | Paper | Richard Csaky |
| A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues | Paper | Richard Csaky |
| Deep Reinforcement Learning for Dialogue Generation | Paper | Richard Csaky |
| An Attentional Neural Conversation Model with Improved Specificity | Paper | Richard Csaky |
| Topic Aware Neural Response Generation | Paper | Richard Csaky |
| Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation | Paper | Richard Csaky |
| Neural Discourse Modeling of Conversations | Paper | Richard Csaky |
| Neural Contextual Conversation Learning with Labeled Question-Answering Pairs | Paper | Richard Csaky |
| Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation | Paper | Richard Csaky |
| Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems | Paper | Richard Csaky |
| Dialogue Learning With Human-In-The-Loop | Paper | Richard Csaky |
| Deep Active Learning for Dialogue Generation | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Neural Responding Machine for Short-Text Conversation | Paper | Richard Csaky |
| A Neural Conversational Model | Paper | Richard Csaky |
| A Neural Network Approach to Context-Sensitive Generation of Conversational Responses | Paper | Richard Csaky |
| Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models | Paper | Richard Csaky |
| A Diversity-Promoting Objective Function for Neural Conversation Models | Paper | Richard Csaky |
| Attention with Intention for a Neural Network Conversation Model | Paper | Richard Csaky |
| A Survey of Available Corpora for Building Data-Driven Dialogue Systems | Paper | Richard Csaky |
| Title | Summary | Paper Source |
|---|---|---|
| Defending Against Neural Fake News | Ai2 Blog | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Troubling Trends in Machine Learning Scholarship | dair.ai | Paper |
| Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Encode, Tag, Realize: High-Precision Text Editing | Google AI Blog | Paper |
| Title | Paper Source | TL;DR |
|---|---|---|
| Latent Predictor Networks for Code Generation | Paper | Richard Csaky |
| Title | Summary | Paper Source |
|---|---|---|
| Revealing the Dark Secrets of BERT | Text Machine Blog | Paper |
| Probing Neural Network Comprehension of Natural Language Arguments | The Gradient | Paper |
| What Does BERT Look At? An Analysis of BERT's Attention | dair.ai | Paper |
| Are Sixteen Heads Really Better than One? | CMU ML Blog | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Olive Oil is Made of Olives, Baby Oil is Made for Babies: Interpreting Noun Compounds using Paraphrases in a Neural Model | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Reformer: The Efficient Transformer | Pragmatic ML | Paper |
| ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | Google AI Blog | Paper |
| Title | Summary | Paper Source | TL;DR |
|---|---|---|---|
| Language Models are Unsupervised Multitask Learners | - | Paper | Richard Csaky |
| Plug and Play Language Models: A Simple Approach to Controlled Text Generation | Uber Engineering | Paper | - |
| ALBERT: A Lite BERT for Self-Supervised Learning Of Language Representations | Amit Chaudhary | Paper | - |
| Fine-Tuning GPT-2 from Human Preferences | Open AI | Paper | - |
| XLNet: Generalized Autoregressive Pretraining for Language Understanding | dair.ai | Paper | - |
| Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context | dair.ai | Paper | - |
| Title | Paper Source | TL;DR |
|---|---|---|
| BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Adversarial Generation of Natural Language | Paper | Richard Csaky |
| Training RNNs as Fast as CNNs | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Beyond BLEU: Training Neural Machine Translation with Semantic Similarity | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Convolutional Sequence to Sequence Learning | Paper | Richard Csaky |
| Depthwise Separable Convolutions for Neural Machine Translation | Paper | Richard Csaky |
| Attention Is All You Need | Paper | Richard Csaky |
| A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models | Paper | Richard Csaky |
| Cold Fusion: Training Seq2Seq Models Together with Language Models | Paper | Richard Csaky |
| Dynamic Evaluation of Neural Sequence Models | Paper | Richard Csaky |
| Emergent Translation in Multi-Agent Communication | Paper | Richard Csaky |
| Plan, Attend, Generate: Planning for Sequence-to-Sequence Models | Paper | Richard Csaky |
| Unsupervised Machine Translation Using Monolingual Corpora Only | Paper | Richard Csaky |
| Classical Structured Prediction Losses for Sequence to Sequence Learning | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Sequence-to-Sequence Learning as Beam-Search Optimization | Paper | Richard Csaky |
| Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation | Paper | Richard Csaky |
| Temporal Attention Model for Neural Machine Translation | Paper | Richard Csaky |
| Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation | Paper | Richard Csaky |
| Can Active Memory Replace Attention? | Paper | Richard Csaky |
| Neural Machine Translation in Linear Time | Paper | Richard Csaky |
| Unsupervised Pretraining for sequence to sequence learning | Paper | Richard Csaky |
| Title | Paper Source | TL;DR |
|---|---|---|
| Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation | Paper | Richard Csaky |
| Sequence to Sequence Learning with Neural Networks | Paper | Richard Csaky |
| Neural Machine Translation by Jointly Learning to Align and Translate | Paper | Richard Csaky |
| Title | Summary | Paper Source |
|---|---|---|
| A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks | dair.ai | Paper |
| Title | Paper Source | TL;DR |
|---|---|---|
| One Model To Learn Them All | Paper | Richard Csaky |
| Title | Summary | Paper Source |
|---|---|---|
| nocaps: novel object captioning at scale | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| DialogueRNN: An Attentive RNN for Emotion Detection in Conversations | dair.ai | Paper |
| Exploring Emoji Usage and Prediction Through a Temporal Variation Lens | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Context-Dependent Sentiment Analysis in User-Generated Videos | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Get To The Point: Summarization with Pointer-Generator Networks | Abigail See | Paper |
| Title | Paper Source | TL;DR |
|---|---|---|
| Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond | Paper | Richard Csaky |
| Title | Summary | Paper Source |
|---|---|---|
| Generalized Language Models | Lilian Weng | - |
| 10 ML & NLP Research Highlights of 2019 | Sebastian Ruder | - |
| Google Research: Looking Back at 2019, and Forward to 2020 and Beyond | Google AI Blog | - |
| NLP Year in Review — 2019 | dair.ai | - |
| Title | Summary | Paper Source |
|---|---|---|
| A Light Introduction to Transfer Learning for NLP | dair.ai | - |
| Attention? Attention! | Lilian Weng | - |
| Title | Summary | Paper Source |
|---|---|---|
| Recent Trends in Deep Learning Based Natural Language Processing | dair.ai | Paper |
| Title | Summary | Paper Source |
|---|---|---|
| Parameter-Efficient Transfer Learning for NLP | dair.ai | Paper |
- Categorize papers; I am aiming to categorize paper summaries by using the categories in the NLP Progress or prominent tracks used in NLP conferences; if you have any suggestions/questions please open an issue or send me a message directly on Twitter or via email.
- Include more info about each paper summaries, excerpt, authors, source, year, etc.
- More paper summaries are coming!
- Move all paper summaries to GitHub so others can also edit, contribute, and review