How an LLM Works, as a City

Residual stream idle

12 numbers standing in for the 4,096+ a real model carries. Blue is negative, warm is positive.

Attention softmax over the KV cache

Runs at the Attention Plaza.

Next token logits → softmax

Computed at the Vocabulary Stadium.

Context

Output

nothing yet

Districts click to fly there