Deep Residual Learning for Image Recognition
In a nutshellResidual connections make very deep neural networks easier to optimize. Instead of learning an entire transformation, layers learn a change relative to their input.
Computer vision
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
THE PAPER, IN A NUTSHELL
Vision Transformer treats an image as a sequence of patches. With large-scale pretraining, this straightforward adaptation of a Transformer performs strongly on image recognition.
Read the paper
The original PDF. Select a passage to explore it with Jenni.
Preparing your reading space…
The key takeaways
01
Image patches can serve as the input tokens of a Transformer.
02
Large-scale pretraining is central to the reported performance.
03
The learned representations transfer to several image recognition benchmarks.
How the research works
Images are divided into fixed-size patches and encoded as sequences. The model is pretrained on large datasets and fine-tuned on image classification tasks.
A thoughtful reading
The strong results depend on the training-data regime. They should not be read as a universal advantage over convolutional models on small datasets.
Editorial overview · Always refer to the original study.