NOTE
Designing a Cache System
Cache layers, cache-aside/read-through/write-through/write-back, consistency races, invalidation, stampedes, penetration, hot keys, TTL, and source-of-truth recovery.
This is a historical learning note and may contain outdated or incomplete understanding.
1. Why Cache?
Caches trade additional memory/state complexity for lower latency and reduced downstream load.
A cache is valuable only when reuse is high enough to justify invalidation and operational complexity.
2. Cache Layers
Possible layers include browser/HTTP cache, CDN, reverse proxy, in-process memory, distributed cache, and database/page cache.
Each layer has a different freshness and invalidation boundary.
3. Cache-Aside
Typical read:
- get cache;
- on miss read source of truth;
- populate cache;
- return.
Typical write:
- commit source-of-truth write;
- invalidate cache.
Updating the database before deleting the cache is a common practical choice because the most dangerous race window is smaller than delete-first, but it still cannot provide perfect atomic consistency across two independent systems.
4. Cache Consistency
Options depend on requirements:
- TTL for bounded staleness;
- retry/queue failed invalidations;
- CDC/binlog-driven invalidation;
- versioned cache values;
- lock/serialization for rare strong-consistency paths;
- bypass cache for reads that must observe the latest committed write.
Do not invoke CAP as a blanket claim that cache consistency is impossible; define the actual failure and staleness model.
5. Stampede / Breakdown
Many concurrent misses for one hot key can overload the origin.
Use request coalescing, stale-while-revalidate, logical expiry, TTL jitter, and bounded origin concurrency.
6. Penetration
Requests for nonexistent keys can bypass cache repeatedly. Cache negative results with short TTL or use a Bloom filter when appropriate.
7. Hot and Big Keys
One key can overload one shard; one huge value can create network/serialization/blocking spikes. Design key cardinality/size budgets up front.
8. Write-Through / Write-Back
Write-through synchronously updates backing state through the cache abstraction.
Write-back acknowledges after cache/log state and flushes backing storage later, increasing write throughput at the cost of durability/consistency complexity.
9. Recovery
A cache should normally be rebuildable from a source of truth. If losing the cache loses irreplaceable business state, it is no longer merely a cache.