NOTE
Scaling Services
Vertical vs. horizontal scaling, statelessness, load balancing, autoscaling signals, warm-up, downstream capacity, and stateful partitioning.
This is a historical learning note and may contain outdated or incomplete understanding.
1. Vertical Scaling
Give one instance more CPU, memory, storage, or network capacity.
It is operationally simple but eventually hits hardware/cost/failure-domain limits.
2. Horizontal Scaling
Add more instances and distribute traffic.
This works best for stateless or partitionable services. Shared mutable state, local sessions, singleton jobs, and hot keys can limit scale-out.
3. Load Balancing
Choose balancing based on traffic and connection model:
- round robin / least requests;
- consistent hashing for locality;
- endpoint health and zone awareness.
Sticky routing improves locality but can create imbalance and complicate failover.
4. Autoscaling
CPU alone is often a lagging or misleading signal.
Useful scaling signals can include:
- concurrency/in-flight requests;
- queue lag;
- latency;
- CPU/memory;
- custom work-rate metrics.
Scale early enough to cover startup/warm-up delay.
5. Downstream Capacity
Before doubling application replicas, verify database, cache, broker, and dependency capacity. Otherwise horizontal scaling amplifies pressure on the actual bottleneck.
6. Stateful Services
Stateful systems typically scale through partitioning/sharding plus replication. Rebalancing data becomes part of the scaling operation.