README.md

Awesome RAG

A curated list of retrieval-augmented generation (RAG) in large language models.
Suggest and discuss possible enhancements on the Potential Additions page.

Table of Content:

• Awesome RAG
  • Papers
    • Survey
    • General
  • Resources
    • Lectures, Talks, Tutorials
    • Workshops
  • Tools
  • Other Collections

Papers

Survey

2024

• Retrieval-Augmented Generation for Large Language Models: A Survey
  Paper Code arXiv

• Retrieval-Augmented Generation for Natural Language Processing: A Survey
  Paper arXiv

2023

• Benchmarking Large Language Models in Retrieval-Augmented Generation
  Paper arXiv

2022

• A Survey on Retrieval-Augmented Text Generation
  Paper arXiv

General

2024

• Learning to Retrieve In-Context Examples for Large Language Models
  Paper Code EACL

• Reliable, Adaptable, and Attributable Language Models with Retrieval
  Paper arXiv

2023

• Active Retrieval Augmented Generation
  Paper Code EMNLP Architecture

• ⭐ REPLUG: Retrieval-Augmented Black-Box Language Models
  Paper arXiv Architecture

• Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
  Paper Code EMNLP

• InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
  Paper Code arXiv

• Retrieve Anything To Augment Large Language Models
  Paper Code arXiv

• Reimagining Retrieval Augmented Language Models for Answering Queries
  Paper ACL

• In-Context Retrieval-Augmented Language Models
  Paper Code TACL Architecture

• Query Rewriting for Retrieval-Augmented Large Language Models
  Paper Code EMNLP

• Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
  Paper PMLR

• Universal Information Extraction with Meta-Pretrained Self-Retrieval
  Paper Code ACL

• RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
  Paper arXiv

• Unlimiformer: Long-Range Transformers with Unlimited Length Input
  Paper Code NeurIPS Architecture

• Nonparametric Masked Language Modeling
  Paper Code ACL Training

2022

• Improving language models by retrieving from trillions of tokens
  Paper Blog PMLR Architecture

• Recitation-Augmented Language Models
  Paper Code ICLR

• ⭐ Atlas: Few-shot Learning with Retrieval Augmented Language Models
  Paper Code Blog Training JMLR

• You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LM
  Paper ACL Architecture

• Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval
  Paper Code ICML Architecture

• Memorizing Transformers
  Paper Code ICLR Architecture

• Training Language Models with Memory Augmentation
  Paper EMNLP Training

• Unsupervised Dense Information Retrieval with Contrastive Learning
  Paper Code arXiv Training

• Teaching language models to support answers with verified quotes
  Paper arXiv Application

• kNN-Prompt: Nearest Neighbor Zero-Shot Inference
  Paper Code EMNLP Application

2021

• Efficient Nearest Neighbor Language Models
  Paper Code EMNLP Architecture

• Mention Memory: incorporating textual knowledge into Transformers through entity mention attention
  Paper Code arXiv Architecture

2020

• ⭐ REALM: Retrieval-Augmented Language Model Pre-Training
  Paper Code HuggingFace PMLR Architecture

• Generalization through Memorization: Nearest Neighbor Language Models
  Paper Code ICLR Architecture

• Entities as Experts: Sparse Memory Access with Entity Supervision
  Paper EMNLP Architecture

• Dense Passage Retrieval for Open-Domain Question Answering
  Paper Code EMNLP Training

• ⭐ Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  Paper HuggingFace NeurIPS

Resources

Lectures, Talks, Tutorials

2024

• Modular RAG and RAG Flow
  Yunfan Gao (2024) Tutorial
Blog I Blog II

2023

• Stanford CS25: V3 I Retrieval Augmented Language Models
  Douwe Kiela (2023) Lecture
Video

• Building RAG-based LLM Applications for Production
  Anyscale (2023) Tutorial
Blog

• Multi-Vector Retriever for RAG on tables, text, and images
  LangChain (2023) Tutorial
Blog

• Retrieval-based Language Models and Applications
  Asai et al. (2023) Tutorial ACL
Website Video

• Advanced RAG Techniques: an Illustrated Overview
  Ivan Ilin (2023) Tutorial
Blog

• Retrieval Augmented Language Modeling
  Melissa Dell (2023) Lecture
Video

Workshops

2024

• Towards Knowledgeable Language Models
  Zoey Sha Li, Manling Li, Michael JQ Zhang, Eunsol Choi, Mor Geva, Peter Hase
  @ACL 2024, August 12-17
  Website

Tools

• LangChain
  LangChain is a framework for developing applications powered by language models.
  Website

• LlamaIndex
  LlamaIndex is a simple, flexible data framework for connecting custom data sources to large language models.
  Website

• Verba
  Verba is an open-source application designed to offer an end-to-end, streamlined, and user-friendly interface for Retrieval-Augmented Generation (RAG) out of the box.
  Website

• NEUM
  Open-source RAG framework optimized for large-scale and real-time data.
  Website

• Unstructured
  Unstructured.io offers a powerful toolkit that handles the ingestion and data preprocessing step, allowing you to focus on the more exciting downstream steps in your machine learning pipeline. Unstructured has over a dozen data connectors that easily integrate with various data sources, including AWS S3, Discord, Slack, Wikipedia, and more.
  Website

Other Collections

• Awesome LLM RAG
• Awesome RAG
• Awesome LLM with RAG
• RAG-Survey
• Awesome LLM Reader: A Repository of Retrieval-augmented LLMs