Self-Attention

The mechanism where every token in a sequence computes attention scores against every other token in the same sequence to figure out which parts of the input matter to which other parts.

Deep Learning

BACKLOG · WORK IN PROGRESS

This term is being written.

The metadata and shape of this page are stable, but the body content isn't ready yet. We'll publish it once it meets the bar of teaching something new with worked examples and real tools.

Back to wiki Track progress on GitHub