VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 09: Heaps and top-k

Lesson 9.12 of 12 in this module Unrolled Pro

Beam Search with a Heap

Find a high-probability output sequence from a language model without trying them all. From exhaustive search over V^L sequences, to greedy decoding (one token at a time, which can walk into a dead end), to beam search: keep the B best partial sequences at every step and select the next B with a heap, O(L · B · V log B). Then sequences that end at different lengths and length normalisation, pruning each beam's expansions to its top tokens, constraints such as no repeated n-grams, and best-first search, which is exact but can need unbounded memory.

about 10 min 4 Python examples 2 quick checks

Problem. A language model gives, for any prefix, a probability for each next token. The probability of a sequence is the product of its token probabilities, so its log-probability is the sum. Find a sequence of length L with the highest log-probability, or as close as possible within a compute budget.

text
P(first)       a 0.5   b 0.4   c 0.1
P(next | a)    a 0.35  b 0.35  c 0.3
P(next | b)    a 0.05  b 0.9   c 0.05

greedy:  a, then a     → 0.5 × 0.35 = 0.175
best:    b, then b     → 0.4 × 0.9  = 0.36

Greedy takes the best first token and gets stuck behind it. Exhaustive search finds the best, but there are V^L sequences: 50,000 tokens and 20 steps is 10⁹⁴. Beam search sits in between, and its core step is the top-B selection from lessons 9.3 and 9.9.

Why it matters. Beam search is the standard decoder for translation, speech recognition, summarisation with encoder-decoder models, and structured generation where the single most likely output matters more than diversity. Chat models usually sample (lesson 9.9) instead, and knowing why is part of the interview answer.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Beam Search with a Heap"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee