VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 05: Binary search

Lesson 5.10 of 10 in this module Unrolled Pro

Largest Batch Size That Fits in Memory

An ML twin: find the largest batch size whose training step doesn't run out of GPU memory, when every trial costs seconds and an out-of-memory error is expensive. From trying 1, 2, 3, … (O(answer) trials), to doubling until failure then binary searching the gap (O(log answer) trials), to using a fitted memory model to search only a narrow window. Then safety margins, multiples of 8, batch size × sequence length frontiers, and checks that aren't quite monotone.

about 10 min 4 Python examples 2 quick checks

Task. You have a function fits(b) that runs one training step at batch size b and reports whether it ran out of memory (OOM). Bigger batches never use less memory, so fits is T…T F…F. Find the largest b that fits, with as few calls as possible, given an upper limit B_max (say 65,536).

text
fits(b) = True for b ≤ 384, False for b > 384     →   384

ML context. This is the batch-size finder that training frameworks run before a job, and the same search sets the largest sequence length, the most concurrent requests a serving replica can hold in its KV cache, or the most images per GPU for inference. Each call allocates the model, runs a forward and backward pass, and frees everything: seconds per trial, and an OOM can leave the allocator fragmented. The number of trials is the cost.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Largest Batch Size That Fits in Memory"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee