Inference & Decoding

Top-K Sampling

Top-K sampling keeps only the K most likely next tokens, drops the long tail, renormalizes the survivors, and samples from that smaller set.

Card 141 of LLMs Visual Card

Top-K sampling starts from the probability distribution over the next token and cuts it down to a fixed count. Keep the K most likely tokens, drop everything else, renormalize the survivors so they sum to one, then sample.

The histogram makes the operation visible. The model gives some probability to a long list of tokens, including many that are technically possible but poor in context. With K set to three, only “cat”, “dog”, and “bird” survive. The rest of the bars are cut before sampling, so an unlikely token from the tail cannot appear just because random sampling happened to hit it.

Renormalization is the step people often skip when describing the method. Once the tail is removed, the surviving probabilities no longer sum to one. The lower chart shows the remaining three tokens inflated into a valid distribution, and the actual sample is drawn from that smaller set. Temperature can still be applied before this cutoff, but the cutoff itself is a separate operation.

The limitation is that K is fixed even when the model’s confidence changes. If the model is already sure, keeping fifty options may keep too much noise. If the model is uncertain, keeping only three may remove reasonable alternatives. Top-P sampling changes that by truncating to probability mass rather than to a fixed number of tokens.

Keep exploring

Each card is part of a larger map of LLM concepts. Move to the next card, follow a related concept, or return to the full curriculum view.

About the visual cards Browse the map