Skip to content
Ephizen logo ephizen / wiki
aka Key-Value Cache

KV Cache

Storing the key and value tensors from previous tokens during autoregressive generation so you don't recompute attention over the entire sequence at every step.

LLMs
BACKLOG · WORK IN PROGRESS

This term is being written.

The metadata and shape of this page are stable, but the body content isn't ready yet. We'll publish it once it meets the bar of teaching something new with worked examples and real tools.

Back to wiki Track progress on GitHub