Distributed Systems

An overview of why distributed systems are needed and how replication, partitioning, consistency, transactions, communication, and common distributed components fit together.

Requirements and failure cases for locks across processes or machines, with Redis- and ZooKeeper-style approaches and the importance of ownership and fencing.

Design goals and trade-offs for globally unique IDs using database sequences, UUIDs, Redis counters, and Snowflake-style timestamp/worker/sequence layouts.

Why process-local sessions break under horizontal scaling and how replication, shared session stores, and centralized authentication address the problem.

The core building blocks of distributed storage: partitioning, replication, consistency, metadata, routing, and failure recovery.

BASE as an availability-oriented distributed-systems design idea: basic availability, soft state, eventual consistency, and its relationship to flexible transactions.

The CAP theorem: consistency, availability, partition tolerance, and the C/A trade-off a distributed system faces when a network partition occurs.

What cluster metadata represents, why routing and ownership depend on it, and centralized versus peer-to-peer metadata management.

Why replicated distributed systems face consistency problems, how consistency models define observable guarantees, and how consensus algorithms help nodes agree on state.

A compact introduction to distributing computation across machines, with batch and stream processing as two common execution models.

Synchronous and asynchronous service communication, RPC versus resource-oriented HTTP APIs, API compatibility, and message encoding choices.

How state placement changes routing, scalability, recovery, and storage dependencies in distributed services.

Deployment strategies for multi-instance services: downtime, blue-green, rolling, canary rollout, A/B testing, and practical rollback mechanisms.

Failure detection and common resilience behaviors including failover, fail-fast, fail-safe, fail-silent, failback, hedged/forked calls, and broadcast calls.

How distributed nodes exchange cluster metadata and the trade-offs between centralized coordination and peer-to-peer dissemination.

Why transactions become harder across services and data stores, and how 2PC, TCC, Saga, reliable messaging, reconciliation, and best-effort notification address different consistency requirements.

How two-phase commit coordinates atomic commit across transactional resources, and why blocking, coordinator failure, and long-held resources are its main costs.

Application-level distributed transactions using Try, Confirm, and Cancel operations, including reservations, compensation, retries, idempotency, and business-code cost.

How a durable business change and a durable outgoing message are coupled so downstream services can converge through retries and idempotent consumption.

A notification pattern for external or loosely coupled systems using bounded retries, durable result lookup, and reconciliation instead of atomic cross-system commit.

Long-running distributed transactions as a sequence of local transactions plus compensating actions, with orchestration/choreography and isolation trade-offs.

The CanCommit, PreCommit, and DoCommit structure of 3PC, why it was proposed to reduce blocking, and why partitions still make it uncommon in practice.

A conceptual comparison between resource-level two-phase commit and application-level TCC, both of which separate preparation from the final decision.

How to commit business state and an outgoing event in one local database transaction, then asynchronously publish with retries and idempotent consumption.

A compact implementation-oriented view of two-phase commit and XA resource coordination.

An implementation-oriented summary of application-level Try/Confirm/Cancel transactions and their idempotency and compensation requirements.

Using an MQ for retryable notifications while keeping an authoritative query path for receivers that miss or cannot process a callback.

RocketMQ's half-message, local-transaction, commit/rollback, and transaction-check protocol for coupling producer-side local state with message visibility.

Why distributed nodes need consensus, how Paxos, Raft, ZAB, and Gossip differ, and how consensus relates to strong or eventual consistency.

Basic Paxos roles and two-phase decision flow, why contention is expensive, and how Multi-Paxos uses a stable leader to make repeated consensus practical.

An entry point to ZooKeeper Atomic Broadcast (ZAB), ZooKeeper's leader-based protocol for ordered, reliable state updates.

Raft's leader, follower, and candidate roles; leader election; replicated-log operation; quorum commitment; and recovery after leader failure.

How gossip protocols spread information through randomized peer-to-peer exchanges, why they scale well, and the redundancy and convergence trade-offs they introduce.

A practical map of linearizability, eventual and causal consistency, read-your-writes, session consistency, monotonic reads and writes, and consistent-prefix reads.

Why distributed systems keep multiple replicas of the same data, how replication improves availability and read capacity, and the major design dimensions behind replication.

Why multiple leaders are useful across regions or offline-capable systems, and how concurrent writes create conflicts that require prevention, detection, or resolution.

Leaderless replication with multi-replica reads/writes, quorum-style coordination, version reconciliation, read repair, and anti-entropy.

How leader/follower replication handles writes, reads, initial synchronization, incremental catch-up, leader election, and failover.

Physical and logical change logs, statement- versus row-oriented replication, and why deterministic replay and schema/storage coupling matter.

Synchronous, asynchronous, and semi-synchronous replication and how acknowledgement rules change latency, durability, availability, and replica lag.

The three common replication topologies: single-leader, multi-leader, and leaderless replication, and the trade-offs each introduces.

How partitioning splits a dataset across nodes, why it scales storage and read/write throughput, and how partitioning differs from replication.

How to choose a partition key and compare explicit, random, range, modulo-hash, consistent-hash, virtual-node, and hash-slot partitioning.

How shard keys determine targeted routing, why missing shard keys cause scatter/gather, and the difference between local and global secondary indexes.

Three ways to locate a partition owner: client-side routing, a dedicated proxy, or server-side forwarding/redirects.

How logical partitions are assigned to machines, static versus dynamic placement, and what changes when a cluster rebalances.