VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
All projects

Inference · Advanced · by SGLang Team

SGLang: Fast serving framework with RadixAttention

High-performance serving engine for complex LLM workflows with RadixAttention automatic KV cache reuse across multiple calls.

2–3 weeks 11500 on GitHubPythonCUDAFlashInferTriton
Open the code on GitHub

What you'll learn

  • Radix tree LRU eviction for KV caches
  • Fast structured output constrained decoding
  • Multi-turn chat prefix reuse and prompt caching
  • Compiler optimization with FlashInfer and custom kernels

Read first

The lessons that explain what this project uses.

Part of the Inference Engineering path.

Milestones

0/4

Built it? Show it.

Add your own repo to earn 150 XP and the Shipper badge. It appears on your profile, so it's ready for your resume.

More inference projects