VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 09: Heaps and top-k

Lesson 9.1 of 12 in this module Unrolled Pro

Heaps from First Principles

A heap keeps the smallest item ready in O(1) and accepts inserts and removals in O(log n), without sorting everything. From scanning a list for the minimum (O(n) per pop) and keeping a sorted list (O(n) per insert), to a complete binary tree stored in an array where parent and children are found by arithmetic, with sift-up and sift-down written from scratch and checked against heapq. Then why building a heap from n items is O(n), max-heaps by negation, tie-breaking tuples, and when a heap is the wrong tool.

about 10 min 4 Python examples 1 quick checks

The question this module answers: how do you keep the best few items ready while items keep arriving? Sorting answers "what's the smallest?" once, at O(n log n). But when items arrive and leave all the time (requests by priority, events by time, candidates by score), you want the current minimum now, and each insert and removal to be cheap. That structure is a priority queue, and the standard way to build one is a binary heap.

ML context.

  • Top-k: the k highest logits for sampling (lesson 9.9), the k nearest embeddings (lesson 9.10), the k most frequent tokens (lesson 9.5).
  • Merging sorted streams: k sorted shards or posting lists (lessons 9.7 and 9.11).
  • Search: beam search keeps the best partial sequences (lesson 9.12); Dijkstra and A* expand the cheapest frontier node (module 11).
  • Scheduling: the next job to run, the next event in a simulation, the request with the earliest deadline.

1. The problem: repeatedly take the minimum

Task: support push(x) and pop_min() in any interleaving.

Either push or pop is O(n). With n = 10⁶ items in a scheduler or a k-NN search, that's 10⁶ operations per call. A heap makes both O(log n): about 20 for a million items.

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Heaps from First Principles"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee