Scalability

Why it matters

Traffic rarely grows smoothly. A product gets featured, a campaign lands, a holiday arrives, and load is ten times yesterday's by lunchtime. Scalability is the property that lets you answer that with money and machines rather than a rewrite. Systems that lack it do not fail gracefully; they fall over at the exact moment the business is going well.

The idea

A system is scalable when adding resources increases its capacity roughly in proportion. Double the servers, handle double the traffic. The two ways to add resources are different in kind.

Vertical scaling (scale up) means a bigger machine: more cores, more memory, faster disks. It is simple, needs no code changes, and preserves the single-machine guarantees a database relies on. It has a ceiling, the price curve bends sharply near the top, and the one machine is still one machine when it fails.

Horizontal scaling (scale out) means more machines behind a load balancer. The ceiling is far higher and failures are survivable, but only if the machines are interchangeable. That is the catch, and it leads to the single most useful idea in this module.

Stateless services scale; state does not

A stateless server keeps nothing between requests that another server would need. Sessions live in a shared store, uploads go to object storage, caches are a separate tier. Any server can take any request, so the load balancer can spread traffic freely and a dead server costs nothing but capacity. Push all state to the edges (clients and shared stores) and the middle tier becomes trivially horizontal.

The stores that hold the state are where scaling gets hard. A relational database scales vertically well and horizontally badly, because the whole point of it is to make guarantees across all its data at once. Caches, queues and object stores were designed to be partitioned and scale out naturally. The Scaling stage of this track is largely about moving state into the stores that can take it.

The bottleneck always moves

Scaling is iterative. Fix the slowest component and the next slowest becomes the limit: web tier, then database reads, then database writes, then the network between regions. The craft is knowing the order in which bottlenecks appear for a given shape of traffic, so you can see the next one coming.

	Vertical (scale up)	Horizontal (scale out)
How	Bigger machine	More machines
Code changes	None	Must be stateless, or partition state
Ceiling	Largest instance available	Very high
Failure	Still a single point	One of many; survivable
Cost curve	Steep at the top	Roughly linear
Best for	Databases, early stages, simplicity	Web and API tiers, workers, caches

Trade-offs

Scaling out buys headroom with complexity: load balancing, service discovery, deploys across a fleet, and the distributed-systems problems of the Distribution stage. Scaling up buys simplicity with a ceiling and a single point of failure. Most real systems do both: scale the stateless tiers out, scale the database up until forced to partition, and move as much work as possible into stores that partition by design.

In an interview

Make the stateless tier explicit and say why it matters: "The API servers are stateless, sessions are in Redis, so I can add servers behind the balancer as traffic grows. The database is the component that will not scale that way, so that is where I will spend the design effort." Then predict the next bottleneck before being asked.