VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 15: The ML coding round

Lesson 15.3 of 12 in this module Unrolled Pro

Mini-batches with shuffling and padding

Turn a list of variable-length token sequences into shuffled mini-batches of padded arrays with attention masks. From padding every sequence to the longest one in the whole dataset (most of the compute spent on padding), to shuffling indices with a seeded generator and padding each batch only to its own longest sequence, built with one vectorised mask assignment, to bucketing similar lengths together so padding drops several-fold. Then token budgets instead of fixed batch sizes, sharding across workers, streams that don't fit in memory, and class-balanced sampling.

about 10 min 3 Python examples 2 quick checks

Problem. You have N tokenised examples (lists of token IDs of different lengths) with labels. Each epoch, produce mini-batches of B examples in a fresh random order (reproducible from a seed). Each batch is a tokens array (B × L) padded with pad_id, a boolean mask (B × L) marking real tokens, and the labels. Every example appears exactly once per epoch.

text
seqs = [[5, 3, 9], [7], [2, 8], [4, 4, 4, 4]],  B = 2,  pad_id = 0
one batch:  tokens = [[2, 8, 0, 0],      mask = [[1, 1, 0, 0],
                      [4, 4, 4, 4]]              [1, 1, 1, 1]]

The model's cost is proportional to B × L including padding. So the real optimisation target isn't Python speed; it's how many padding tokens you feed the GPU. Shuffling (for unbiased gradient estimates) fights against grouping similar lengths (for less padding); bucketing (Step 3) gets most of both.

ML context. This is the data loader of every sequence model, and a frequent ML-round question because the details matter: a missing mask lets attention read padding, a shuffle without a seed breaks reproducibility, and a fixed batch size with long-tailed lengths wastes most of the GPU. Token-budget batching (Change A) is how large-model training keeps memory constant per step.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Mini-batches with shuffling and padding"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee