VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
All projects

LLMs · Intermediate · by Andrej Karpathy

nanoGPT: Train & fine-tune medium GPTs

The simplest, fastest repository for training and fine-tuning medium-sized GPTs in PyTorch with clean, hackable, zero-abstraction code.

1–2 weekends 41500 on GitHubPythonPyTorchCUDATransformers
Open the code on GitHub

What you'll learn

  • Minimal causal self-attention implementation from scratch
  • Pre-training from character-level Shakespeare to OpenWebText
  • Fine-tuning with LoRA and prompt adaptation
  • Benchmarking MFU (Model FLOPs Utilization)

Read first

The lessons that explain what this project uses.

Part of the LLMs in Depth path.

Milestones

0/4

Built it? Show it.

Add your own repo to earn 150 XP and the Shipper badge. It appears on your profile, so it's ready for your resume.

More llms projects