A world of research.
A little more clarity.

Ashish Vaswani, Noam Shazeer, Niki Parmar +5 authors

In a nutshellThe Transformer replaces recurrence and convolution with attention. It connects words directly across a sequence, enabling more parallel training and strong machine translation performance.

Jacob Devlin, Ming-Wei Chang, Kenton Lee +1 author

In a nutshellBERT learns language representations using context from both sides of a word. A pretrained model can then be adapted to different language understanding tasks with small architectural changes.

Patrick Lewis, Ethan Perez, Aleksandra Piktus +9 authors

In a nutshellRAG combines a language model with a searchable collection of documents. Retrieved passages give the generator access to external knowledge when producing an answer.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov +9 authors

In a nutshellVision Transformer treats an image as a sequence of patches. With large-scale pretraining, this straightforward adaptation of a Transformer performs strongly on image recognition.

Kaiming He, Xiangyu Zhang, Shaoqing Ren +1 author

In a nutshellResidual connections make very deep neural networks easier to optimize. Instead of learning an entire transformation, layers learn a change relative to their input.