README.md

IR From Bag-of-words to BERT and Beyond through Practical Experiments

This is the official repository of "IR From Bag-of-words to BERT and Beyond through Practical Experiments", a Search Solutions 2022 full-day tutorial with PyTerrier search toolkit.

About the tutorial

Advances from the natural language processing community have recently sparked a renaissance in the task of adhoc search. Particularly, large contextualized language modeling techniques, such as BERT, have equipped ranking models with a far deeper understanding of language than the capabilities of previous bag-of-words (BoW) models. Applying these techniques to a new task is tricky, requiring knowledge of deep learning frameworks, and significant scripting and data munging. In this full-day tutorial, we build up from foundational retrieval principles to the latest neural ranking techniques. We provide background on classical (e.g., BoW), modern (e.g., Learning to Rank) and contemporary (e.g., BERT search ranking and re-ranking techniques. Going further, we detail and demonstrate how these can be easily experimentally applied to new search tasks in a new declarative style of conducting experiments exemplified by the PyTerrier search toolkit.

This tutorial is interactive in nature for participants; it is broken into sessions, each of which mixes explanatory presentation with hands-on activities using prepared Jupyter notebooks running on the Google Colab platform.

At the end of the tutorial, participants will be comfortable accessing classical inverted index data structures, building declarative retrieval pipelines, and conducting experiments using state-of-the-art neural ranking models.

Authors

• Sean MacAvaney, University of Glasgow, UK
• Craig Macdonald, University of Glasgow, UK
• Nicola Tonellotto, University of Pisa, IT

Contents

• Part 1: Classical IR: indexing, retrieval and evaluation; Modern Retrieval Architectures: PyTerrier data model and operators, towards re-rankers and learning-to-rank
  • Slides
  • Notebook 1
  • Notebook 2
• Part 2: Contemporary Retrieval Architectures: Neural re-rankers such as BERT, EPIC, ColBERT
  • Slides
  • Notebook 3
• Part 3: Recent Advances beyond the classical inverted index: neural inverted index augmentation, nearest neighbor search, dense retrieval
  • Slides
  • Notebook 4
  • Notebook 5

Useful Links

• PyTerrier: [Github] [Documentation]
• OpenNIR: [Github] [Documentation]
• PyTerrier_ColBERT: [Github]
• PyTerrier_T5: [Github]
• PyTerrier_doc2query: [Github]
• PyTerrier_DeepCT: [Github]
• PyTerrier_ANCE: [Github]
• PyTerrier_SPLADE: [Github]

Citation Policy

If you make using of any of these slides, notebooks, or additional PyTerrier plugins, please cite our tutorial abstract:

@inproceedings{cikm2021-tut-bow2b,
  author = {MacAvaney, Sean and Macdonald, Craig and Tonellotto, Nicola},
  title = {IR From Bag-of-words to BERT and Beyond through Practical Experiments: A CIKM 2021 tutorial with PyTerrier and OpenNIR},
  booktitle = {Proceedings of CIKM 2021},
  year = {2021}
}