Watch a real language model turn a sentence into a reply.
This runs an actual TinyStories-8M model client-side, in your browser — no server, no API. Type something below and step through what the model actually computes: tokenize, embed, attend, predict.
Tokenize
Models don't read words — they read numbers. Before anything else can happen, your sentence gets chopped into pieces from a fixed vocabulary: whole common words when possible, familiar fragments when a word is rare or long.
TinyStories-8M's real byte-pair tokenizer splits your sentence into its actual vocabulary pieces. Hover a token to see what it is.
Dashed = a subword continuation piece. Orange border = a token the model generated itself.
Embed
Each token becomes a vector — a list of numbers standing in for its meaning in context. This is the model's actual internal language: coordinates in a space where related meanings sit close together.
Each token's real internal vector, pulled straight from the model's first attention layer (its value projections, reassembled across all 16 heads) — first 10 of 256 dimensions shown. Orange = positive, grey = negative. Hover a bar for its exact value.
Attend
Understanding a word often depends on the rest of the sentence — attention lets every token weigh how much every other token matters to it. It's how the model tracks relationships across a whole sentence, not just one word in isolation.
Click a token to see how strongly it relates to every other token — arc thickness and glow show the strength, computed from genuine vectors (see notes below).
Predict
The model doesn't output a sentence — it outputs one probability per possible next token. It picks the likeliest, appends it, and runs the whole process again. Every reply you've ever seen from an LLM was built this way, one token at a time.
Real softmax probabilities over TinyStories-8M's full 50257-token vocabulary, top 8 shown.
Press "Generate next token" to append the top pick and run the model again on the new sequence.