ResearchInteractive previewWrite with Jenni The Transformer replaces recurrence and convolution with attention. It connects words directly across a sequence, enabling more parallel training and strong machine translation performance.
The original PDF. Select a passage to explore it with Jenni.
Attention alone can support an encoder–decoder translation architecture.
The model reports 28.4 BLEU on English-to-German translation.
Parallel computation reduces the sequential work required during training.
The authors evaluate an attention-based encoder and decoder on WMT 2014 translation tasks, comparing translation quality and training cost with earlier architectures.
The experiments focus primarily on translation. Results do not establish that attention is the best choice for every task or resource budget.
Editorial overview · Always refer to the original study.