VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
All projects

Inference · Advanced · by Georgi Gerganov

llama.cpp: Fast LLM inference in C/C++

Port of LLaMA model in pure C/C++ with 1.5-bit to 8-bit integer quantization, running on Apple Silicon Metal, AVX2 x86, and NVIDIA GPUs.

2–3 weeks 71000 on GitHubC/C++MetalCUDAGGML
Open the code on GitHub

What you'll learn

  • GGML tensor library memory layouts and compute graphs
  • GGUF model file structure and quantization methods (k-quants)
  • Apple Metal, ARM NEON, and AVX2 vector SIMD acceleration
  • Memory-mapped weights (mmap) for zero-copy model loading

Read first

The lessons that explain what this project uses.

Part of the Inference Engineering path.

Milestones

0/4

Built it? Show it.

Add your own repo to earn 150 XP and the Shipper badge. It appears on your profile, so it's ready for your resume.

More inference projects