Attention Is All You Need
In a nutshellThe Transformer replaces recurrence and convolution with attention. It connects words directly across a sequence, enabling more parallel training and strong machine translation performance.
Language models
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
THE PAPER, IN A NUTSHELL
RAG combines a language model with a searchable collection of documents. Retrieved passages give the generator access to external knowledge when producing an answer.
Read the paper
The original PDF. Select a passage to explore it with Jenni.
Preparing your reading space…
The key takeaways
01
Retrieval and generation can be combined in a trainable system.
02
An external document index complements knowledge stored in model parameters.
03
The approach is evaluated on knowledge-intensive language tasks.
How the research works
A dense retriever selects passages from Wikipedia and a sequence-to-sequence model generates responses conditioned on those passages.
A thoughtful reading
The quality of an answer depends on both retrieval and generation. Access to relevant passages does not guarantee a correct or fully supported response.
Editorial overview · Always refer to the original study.