2.1 CPU Caches, Coherence, and Memory Ordering

1. Why CPU Caches Exist

CPU cores can execute instructions much faster than DRAM can satisfy arbitrary memory accesses. Cache hierarchies keep recently or nearby used data closer to the core.

A typical hierarchy contains private L1 caches, larger L2 caches, and a shared or partially shared last-level cache, although exact topology varies by CPU.

2. Cache Lines

Caches transfer data in fixed-size blocks called cache lines rather than one source-language variable at a time.

A cache lookup conceptually uses parts of the address to identify a set/index, compare tags, and select an offset within a line.

This has practical consequences:

• spatial locality matters;
• unrelated variables can share one line;
• false sharing can cause heavy coherence traffic even when threads modify different variables.

3. Cache Coherence

When multiple cores cache the same physical line, the system must keep their views coherent.

MESI is a common teaching model with states such as:

• Modified — this cache has a dirty exclusive copy;
• Exclusive — this cache has the only clean copy;
• Shared — multiple caches may have clean copies;
• Invalid — the line cannot be used.

Real processors may use MESI variants such as MOESI/MESIF and directory/snoop-based protocols. Application code should depend on the architecture’s memory model, not one specific coherence state machine.

4. Coherence Is Not Consistency

Cache coherence answers a per-location question: how writes to one cache line become visible across cores.

Memory consistency / memory ordering answers a broader question: in what orders may loads and stores to different locations become observable?

A system can maintain coherence while still allowing observations that differ from source-code order.

5. Why Reordering Appears

Modern CPUs use mechanisms such as:

• store buffers;
• speculative/out-of-order execution;
• load queues;
• invalidation handling;
• compiler instruction reordering.

These improve performance while obeying the architecture’s ordering rules.

x86-64 provides a relatively strong memory model; ARM and other architectures allow more ordering freedom. Therefore, a concurrent algorithm that happens to work on x86 may still be incorrect if it lacks language-level synchronization.

6. Memory Barriers / Fences

A memory barrier constrains ordering and visibility of memory operations according to the CPU architecture.

It should not be understood as a universal command that simply “flushes the store buffer” or “empties an invalidation queue.” Implementations differ, and the architectural contract is about permitted observations/orderings.

Common conceptual fence classes include load-load, load-store, store-store, and store-load constraints. Architectures expose different instructions and may combine several constraints in one fence.

7. Language-Level Synchronization

Application code should normally rely on language/library primitives:

• mutexes;
• atomics;
• channels/queues;
• volatile/acquire-release constructs where the language defines them.

The compiler and runtime then emit the fences or atomic instructions required for x86, ARM, and other targets.

This is the bridge between hardware memory ordering and the concurrency-series concepts such as Java volatile, Go atomics, and lock implementations.