Hide outline
Feedback
Guide
Was this lesson helpful?
Dive Deeper
Generate a follow-up sub-lesson on any aspect of this topic
Guide complete
How CDNs Reduce Latency With Edge Caches
Generate a follow-up sub-lesson on any aspect of this topic
How CDNs Reduce Latency With Edge Caches
TLDR;
- A nearby edge cache turns most image and video requests into short round trips instead of cross ocean ones.
- A simple TTL choice can trade a small staleness window for a large drop in origin request volume.
- Personalization and headers can silently break caching by changing the cache key in unsafe ways.
A small news site often starts with a single origin server that serves everything, including HTML, images, and video, and distant readers feel the network more than the code. If a user in another region has a round trip time to the origin, each required request pays that geography tax before any rendering starts. The same user might have a round trip to a nearby metro edge, which is why latency complaints show up first on large assets like article.jpg and video.mp4. Let’s anchor that gap visually before changing the architecture.
The map makes the core constraint visible. Three user regions sit far from one origin, so the long path dominates, while the path to an edge node is the cheap hop we want most requests to take. The only way the edge helps is if it can answer from stored bytes, which is exactly what a CDN edge cache is built to do.
A CDN helps when the edge answers without contacting the origin, because distance dominates latency for large static bytes. A cache hit means the edge already has the response for a request and returns it immediately. A cache miss means the edge does not have it and must fetch it from the origin, then optionally store it for later. Even with the same origin hardware, pushing the common case onto a path can pull median page load down, while misses still pay the trip. Let’s tune the hit rate and watch both latency and origin load move together.
The knob illustrates the tradeoff. As the hit rate rises, more requests end at the edge and fewer traverse the long RTT to the origin, so median latency drops and origin requests per minute fall. What does not change is the miss penalty, since the edge still has to cross the path at least once per uncached object.
A CDN turns a normal HTTP fetch into a two tier lookup where routing decides which cache checks first. The edge selection usually starts with DNS, where the CDN’s DNS answers with an IP that is close to the client, not the origin. The client then sends GET /images/article.jpg to that edge, the edge looks up a cache entry keyed by some combination of URL, headers, and query parameters, and it either returns bytes immediately or it goes to the origin. Here is the miss then hit sequence for one image.
In the sequence, the first request misses at the edge, triggers an origin fetch, and the edge stores the response so the second request becomes a hit. The important components are the DNS based routing step, the edge cache lookup, and the origin fetch that only happens on a miss. The behavior change is that repeated demand across many nearby users collapses into one origin fetch plus many edge hits.
Design rule
A CDN only reduces latency when the cache key is stable across users who share content.
Most CDNs use origin pull, meaning the edge fetches objects from the origin on demand when it sees a miss. The alternative is origin push, meaning the publisher uploads or pre positions objects at edges ahead of traffic, often called pre warming. Pull keeps operations simple and ensures unused content never gets shipped, but the first requester in each region pays the miss latency and the origin can see spikes when a new object goes viral. Push reduces first request latency for planned launches, but it increases operational effort and raises freshness risk if old versions are pushed and not invalidated. The choice is easiest to see as a set of tradeoffs.
The matrix makes the dividing line clear. Pull is usually better when traffic is unpredictable and content changes often, because the edge learns demand naturally and the origin remains the source of truth. Push is usually better when a small set of big assets must be fast immediately in many regions, because it shifts work earlier and smooths the launch spike that would otherwise hammer the origin.
A time to live (TTL) is the duration an edge is allowed to treat a cached response as fresh before it must revalidate or refetch. TTL is typically driven by HTTP caching headers like Cache-Control: max-age=..., and CDNs can also support explicit purge or invalidation when an editor updates an image that is already cached globally. Longer TTLs increase hit rate and cut origin traffic, but they widen the staleness window, which is how long users might see an old version after a change. We can estimate both origin load and that staleness window from a few inputs.
From the estimate, origin requests per minute roughly scale with misses, not total demand. A common heuristic is \text{origin_rpm}\approx\text{total_rpm}\cdot(1-\text{hit_rate}) assuming misses always go to origin and there is no shielding tier. The staleness window is bounded by TTL when you do not purge, because an edge can legally serve the cached version until it expires, so shortening TTL reduces worst case staleness but pushes more validation traffic back to the origin.
A CDN works best for responses that many users share and that do not change on every request. For the news site, the clean fits are static assets and large media, because they are expensive to transfer and identical for many readers. Things that vary per user or per session usually require careful cache key design or should bypass the CDN cache entirely. Let’s sort a few concrete assets from the site.
The cards highlight the real boundary. logo.png, article.jpg, and video.mp4 are typically safe and high impact because they are shared and heavy. The homepage HTML can be cacheable if it is mostly the same for all readers and varies only by language or device. A personalized feed like feed.json is usually a poor fit because cookies or Authorization headers make each user’s response different, which collapses the shared cache benefit and increases correctness risk.
Edge caching fails in ways that look like performance wins until users see the wrong bytes. If the cache key ignores a header that changes the response, the edge can serve one user another user’s personalized data. If TTLs are long and purge is slow, the CDN stays available during origin trouble but serves stale content, which is a consistency choice. If many edges expire the same object at once, they can all stampede the origin at the same moment, which is the thundering herd problem. Let’s reveal three concrete failure cases and the mitigations that usually fix them.
The reveal should make the failure mechanics concrete. The wrong cache key problem is usually mitigated by varying on the right inputs using Vary and by disabling caching when Set-Cookie or Authorization is present. Stale but available is mitigated with shorter TTLs for fast changing content and with controlled use of stale-while-revalidate behavior when acceptable. Thundering herd is mitigated with request coalescing at the edge, jittering expirations so not everything expires at once, and adding an origin shield layer so many edges do not all hit the origin directly.
For the small news site, the defensible design is to use the CDN as a geographically distributed edge cache for shared assets, let origin pull handle the long tail, and pick TTLs that match editorial update frequency plus a purge path for breaking changes. Those decisions work because the site’s heaviest bytes are identical across readers, so the cache key stays stable and hits accumulate quickly in each region.
The constraint that forces a redesign is real time personalization across most responses, because it turns one shared object into many per user objects and makes availability and correctness pull in opposite directions. At that point the edge stops being a simple byte cache and becomes a place where you either compute per user variants safely or you route more traffic back to the origin and accept the geography cost.