Problem. A sweep config lists candidate values for each hyperparameter:
layers 12, 24, 36
width 768, 1024, 1536, 2048
precision bf16, fp32
seq_len 512, 1024, 2048, 4096
batch 16, 32, 64, 128, 256
lr 1e-4, 3e-4, 1e-3, 3e-3
That's 3 × 4 × 2 × 4 × 5 × 4 = 1,920 combinations. Most can't run: they exceed the 80 GB of GPU memory, or put too many or too few tokens in a step (batch × seq_len must be between 32,768 and 262,144), or break a stability rule (wide models need lr ≤ 1e-3). Return every valid configuration, and spend as little work as possible on invalid ones.
Why it matters. Real sweep spaces have ten or more axes: 10⁷ to 10⁹ raw combinations, and the expensive part of a sweep is launching runs. A config generator that understands the constraints produces only runs that can start, and the pruning ideas here (bounds on partial choices, breaking out of sorted values) are the same ones that make backtracking fast everywhere in this module.