Getting content close, and spreading the load

Two problems remain once the protocols work. The speed of light, and one server not being enough.

Latency has a floor

Chennai to a server in Virginia is about 13,000 km. Light in fibre covers that in roughly 65ms, so a round trip is 130ms before any processing. No optimisation removes it.

A content delivery network puts copies near the user instead.

without: user (Chennai) -> origin (Virginia)     130 ms round trip
with:    user (Chennai) -> edge (Chennai)          5 ms round trip

The user is directed to a nearby edge by DNS returning a different address by region, or by anycast where the same address is announced from many places and routing chooses the nearest.

What belongs on a CDN

Static content obviously: images, scripts, stylesheets, video. Modern CDNs also cache HTML with short TTLs, terminate TLS at the edge so the handshake is local even for uncacheable content, and run code at the edge.

That last point matters: TLS setup is round trips, so terminating nearby improves even a request that must reach the origin.

Cache invalidation

The recurring problem. Two approaches:

Purge, telling the CDN to drop an object. Works, takes time to propagate.

Versioned filenames, app.a3f9c2.js, cached forever because the name changes when the content does. This is why build tools put hashes in filenames, and it is the better answer because it never needs invalidating at all.

Load balancing

Least connections handles uneven request cost better than round robin, which assumes every request is equal.

Health checks and sticky sessions

A balancer must remove a failed server, which means asking it whether it is well. A good check exercises the dependencies, since a server whose database connection is gone answers a trivial check happily while failing every request. A check that is too thorough is its own risk: one slow dependency can mark the entire fleet unhealthy at once.

Sticky sessions pin a user to one server so session state in memory keeps working. It also means an uneven load and losing sessions when a server dies. The better answer is to keep session state in a shared store so any server can handle any request, which is what makes the tier genuinely stateless and horizontally scalable.