Skip to content
Ephizen logo ephizen / wiki
2017 · NeurIPS

Attention Is All You Need

Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin

2017 NeurIPS
TL;DR
Replaces recurrence and convolutions with self-attention. Introduces the Transformer architecture that powers every modern LLM.
Read paper (opens in new tab)
BACKLOG · WORK IN PROGRESS

This paper is being written.

The metadata and shape of this page are stable, but the body content isn't ready yet. We'll publish it once it meets the bar of teaching something new with worked examples and real tools.

Back to papers Track progress on GitHub