VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D14: Distributed Systems · Distributed Storage — Replication, Partitioning and Object Stores

Distributed Storage — Replication, Partitioning and Object Stores

15 min read·Advanced· Unrolled Pro
Distributed Storage — Replication, Partitioning and Object Stores

You already know: indexes, transactions, replicas and partitions on one database (D1 Databases); consistency models, quorums and consensus (Distributed systems fundamentals); logs and change data capture (Messaging & event processing); vector indexes (D8 Vector databases).

Why this matters for Atlas. Data outgrows one machine in two ways: too many reads and writes (load) and too much data (size). The two tools are replication (copies of the same data on several machines: availability, read scaling, locality) and partitioning or sharding (different data on different machines: write scaling, capacity). Each creates its own problems: replication creates lag and conflicts; partitioning creates hotspots and expensive rebalancing. For bulk data (datasets, checkpoints, model weights), object storage trades features for nearly unlimited, cheap, durable capacity. This lesson builds each piece with numbers.


1. Replication and its lag

The question. Students answer a quiz, then see their old progress. Why?

Start from nothing. In leader-follower replication, writes go to the leader, which streams its log to followers; reads can go to any replica. Replication is usually asynchronous (the leader confirms the write before followers apply it), so followers lag by milliseconds normally, and by seconds or minutes under load or after network trouble. A read served by a lagging follower shows stale data, including the student's own write. Remedies, cheapest first:

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Distributed Storage — Replication, Partitioning and Object Stores"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee