NOTE
1.5 How Distributed Storage Works
The core building blocks of distributed storage: partitioning, replication, consistency, metadata, routing, and failure recovery.
This is a historical learning note and may contain outdated or incomplete understanding.
1. What Is Distributed Storage?
Distributed storage keeps a logical dataset across multiple machines while presenting a usable storage abstraction to clients.
2. Why Distribute Storage?
A single machine has finite capacity, throughput, and availability. A distributed design can scale storage and I/O horizontally and tolerate individual machine failures, but it must coordinate where data lives and which copy is authoritative.
3. Core Building Blocks
- Replication: keep redundant copies for availability and read scale.
- Partitioning: split a dataset across machines.
- Consistency: define what clients may observe while replicas diverge.
- Cluster metadata: track partition placement, membership, and ownership.
A complete system also needs request routing, rebalancing, failure detection, repair, checksums or other corruption detection, and operational tooling.
4. Examples
Databases, search engines, caches, object/file systems, and event logs all use these building blocks in different combinations. Elasticsearch, Redis, MySQL deployments, and FastDFS illustrate different points in this design space rather than one universal architecture.