RESEARCH AUTHOR

Jacob Devlin

Jacob Devlin is a credited coauthor of “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”. Explore the papers, research findings, and collaborators represented in this collection.

Language models

Papers & publications

1 in this collection

Read the original papers, explore an overview, and collect ideas for your own research.

Open access2018arXiv

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee +1 author

In a nutshellBERT learns language representations using context from both sides of a word. A pretrained model can then be adapted to different language understanding tasks with small architectural changes.

Language models

Read & chat

EXPLORE THE WORK

Research in focus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Pretraining on unlabeled text is followed by supervised fine-tuning and evaluation on language understanding and question-answering benchmarks.

Bidirectional context improves language representations.
One pretrained model transfers to multiple downstream tasks.
The paper reports improvements on eleven language understanding tasks.

Reading context: Downstream fine-tuning still needs task-specific training data. These results concern language understanding, rather than open-ended text generation.

Read the original on arXiv ↗