NOTE
Designing Load Balancing
Endpoint discovery, selection algorithms, health/outlier detection, connection reuse, locality, weighting, and overload-aware routing.
This is a historical learning note and may contain outdated or incomplete understanding.
A load balancer maintains/selects from candidate endpoints and routes work according to policy. Round-robin is simple; least-loaded/latency-aware/weighted/rendezvous or consistent-hash-style policies solve different requirements.
Selection depends on trustworthy endpoint health, locality, connection state, and capacity. Remove/outlier-deprioritize failing instances carefully to avoid synchronized flapping.
Load balancing cannot create capacity. When all endpoints are saturated, use admission control/load shedding rather than endlessly redistributing overload.