VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
Easy10 XPoptimization

Why does my loss explode after a few steps?

Loss drops for 20 steps, then jumps up and becomes NaN. Gradients are not clipped. What is the first thing to try?

Stuck? Show the solution

Answer: B. Lower the learning rate (add warmup and gradient clipping)

Steps that are too large overshoot the minimum and diverge. A smaller learning rate, a warmup period and gradient clipping are the standard fixes.

Learn the idea behind itOptimization — Turning Gradients Into Learning