Attention Is All You Need
In a nutshellThe Transformer replaces recurrence and convolution with attention. It connects words directly across a sequence, enabling more parallel training and strong machine translation performance.
Language models
RESEARCH AUTHOR
Niki Parmar
Niki Parmar is a credited coauthor of “Attention Is All You Need”. Explore the papers, research findings, and collaborators represented in this collection.
Papers & publications
1 in this collectionRead the original papers, explore an overview, and collect ideas for your own research.
EXPLORE THE WORK
Research in focus
Attention Is All You Need
The authors evaluate an attention-based encoder and decoder on WMT 2014 translation tasks, comparing translation quality and training cost with earlier architectures.
- Attention alone can support an encoder–decoder translation architecture.
- The model reports 28.4 BLEU on English-to-German translation.
- Parallel computation reduces the sequential work required during training.
Reading context: The experiments focus primarily on translation. Results do not establish that attention is the best choice for every task or resource budget.
Read the original on arXiv ↗