1. Why a CDN Exists

Suppose Atlas publishes a news site from one US region. A reader in London requests the story’s 2.4 MiB hero image:

London reader
    -> public Internet
    -> Atlas load balancer in the US
    -> image service
    -> object storage
    -> the same long path back to London

The Atlas deployment is the origin: the authoritative system that stores or generates the response. The origin might contain load balancers, application servers, databases, and object storage rather than one literal machine. What matters is that every reader request travels to infrastructure responsible for the canonical content.

An origin-side cache can avoid regenerating the image or reading it repeatedly from storage. That helps the origin’s CPU and storage path. It does not solve the whole delivery problem. Even if the origin already has the image ready in memory, every London reader still crosses the Atlantic, reaches the origin network, and receives another 2.4 MiB copy across the same long path.

The problem is not that the origin is a bad server. The problem is that the same reusable bytes are produced in one place and consumed repeatedly around the world.

A content delivery network, or CDN, moves reusable responses closer to readers. It operates geographically distributed servers in points of presence, or PoPs. A server handling requests in a PoP is commonly called an edge server.

origin-only
    London reader --------------------------> US origin

with a warm CDN cache
    London reader ----> London edge cache

The origin remains authoritative. The edge stores a disposable copy and reuses it only when HTTP rules and CDN policy say that doing so is safe.

For 100,000 London readers requesting the same 2.4 MiB image, the origin-only path transmits approximately:

100,000 * 2.4 MiB = 240,000 MiB ~= 234 GiB

With a warm edge cache, the CDN can fetch a small number of upstream copies and deliver the rest locally. Real systems have several cache nodes, fills, evictions, ranges, and revalidations, so the origin will not necessarily send exactly one copy. The important change is that demand is served by reuse near the readers rather than repeated work at the origin.

This improves several different dimensions:

• latency: a warm response takes the client-to-edge path instead of the client-to-origin path;
• origin load: a hit avoids an origin request, application work, storage access, and response bytes;
• burst capacity: many PoPs can serve an already cached popular object;
• failure tolerance: policy may permit a bounded stale response while the origin is unavailable.

A CDN shifts work rather than eliminating it. Edge storage and bandwidth cost money, cold misses still reach the origin, and a wrong cache decision can distribute stale or private data extremely quickly.

What to remember: An origin cache avoids repeated computation. A CDN changes the delivery path: it serves reusable responses near readers, reduces long-distance transfer, and protects the origin from repeated traffic.

2. A CDN Is a Delivery Fleet

A CDN is not just “a cache in front of the origin.” It is a globally distributed delivery system that answers several questions for every request:

where should this user connect?
which customer, certificate, and policy apply?
can this edge answer safely from local state?
if not, how should the CDN fetch without hurting the origin?
how does the fleet stay correct during deploys, purges, and failures?

Those questions divide the CDN into a few cooperating parts:

Caching is one mechanism inside that design. A CDN also routes traffic, absorbs bursts, terminates TLS close to users, applies security controls, reduces long-distance bandwidth, coordinates global invalidation, and fails traffic between locations.

The rest of the article follows one Atlas image through those responsibilities: route the reader to an edge, establish request context, decide whether the edge can answer, fill through the hierarchy when it cannot, protect the origin, and keep the fleet correct when content or infrastructure changes.

What to remember: Caching is important, but the CDN design problem is global delivery: placement, policy, origin protection, consistency, failure handling, and observability.

3. Where Edge Reuse Fits

A CDN is a geographically distributed reverse proxy. A reverse proxy accepts a request on behalf of an upstream server, applies policy, and either answers locally or forwards the request upstream.

Edge reuse is only one branch of the serving path, but it is the branch that creates most of the latency and bandwidth benefit for static assets, public documents, images, video segments, and downloads.

The edge does not merely save “a file.” It stores a reusable HTTP response and the metadata needed to decide whether this response can answer a later request:

cache entry
    identity        which requests this entry can answer
    status          200, 301, 404, ...
    headers         Content-Type, Cache-Control, ETag, Vary, ...
    body            image bytes, HTML, JSON, video segment, ...
    timing          when it was generated, received, and validated
    storage state   memory, disk, or another cache layer

A stored response is useful only when three questions all have safe answers:

1. Eligibility: may a shared cache store and reuse this response?
2. Identity: does this later request ask for the same representation?
3. Freshness: may the stored response be reused now without contacting the origin?

Those questions distinguish a CDN edge from a directory of downloaded files.

Browser Cache, Origin Cache, and CDN Cache

The same HTTP caching model appears in several locations, but the trust and cost boundaries differ:

A private cache, such as a browser cache, can store a response for one user. A shared cache, such as a CDN, might reuse one entry for unrelated users. Shared reuse therefore requires much stricter identity and authorization boundaries.

What Is a Good Candidate?

Responses with wide, safe reuse produce the strongest benefit:

An uncacheable request can still benefit from edge TLS termination, attack filtering, compression, or an optimized backbone. Passing through a CDN is not the same as being answered from its cache.

When a request reaches an edge server, it can therefore take four broad serving paths:

fresh reusable entry     -> return it
stale reusable entry     -> validate it or apply stale policy
no matching entry        -> fetch upstream and perhaps store the response
unsafe or ineligible     -> bypass shared caching

What to remember: Edge reuse is a serving decision. The CDN must prove that a stored response is safe for this request before it can turn local bytes into a user-visible answer.

4. The Atlas Image Request

Atlas publishes a story about a solar eclipse. The page references this image:

https://media.atlas.example/eclipse/hero?w=1200

The browser can display AVIF, so it sends:

GET /eclipse/hero?w=1200 HTTP/1.1
Host: media.atlas.example
Accept: image/avif,image/webp,image/*;q=0.8

The origin currently has representation hero-v7-avif, 2.4 MiB in size:

HTTP/1.1 200 OK
Content-Type: image/avif
Content-Length: 2516582
ETag: "hero-v7-avif"
Vary: Accept
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=30, stale-if-error=86400

This response carries its reuse contract. For now, read the headers this way:

• Cache-Control says whether the response may be stored and how long it stays fresh.
• Vary: Accept says that the image format depends on the browser’s Accept header.
• ETag gives the response a version-like validator for later checks.
• the stale directives define bounded exceptions after normal freshness ends.

Each of those ideas gets its own section later. The important point here is that the origin does not merely return image bytes. It also tells shared caches how those bytes may be reused.

The first London reader will cause this path:

browser
    -> DNS and Internet routing
    -> London PoP ingress
    -> tenant and security policy
    -> cache identity and variant
    -> edge cache node
    -> regional shield
    -> Atlas origin

The first reader creates the stored copy. No reusable Atlas AVIF entry exists near London, so the edge forwards the canonical request upstream. The origin returns the image and cache policy. The CDN streams the response to the reader while storing an eligible copy.

The next matching reader gets the benefit. The edge finds the stored representation, confirms that it is eligible and still fresh, and returns it without contacting the origin.

The stored response remains reusable until time, eviction, invalidation, or a changed request makes reuse unsafe.

Most CDNs use pull caching. Atlas publishes the image at its origin and routes the hostname through the CDN. It does not upload the image to every PoP before anyone requests it. Some operators deliberately prewarm or preposition important objects before a launch. That reduces first-reader latency but consumes storage and fill bandwidth in locations that might never request the object. Pull caching pays the fill cost only where demand appears.

What to remember: A miss is an upstream fetch, not a failure. If the response is eligible, that fetch creates the stored entry that later requests can reuse.

The next question is how the browser reaches an Atlas-capable edge location.

5. How the Browser Reaches an Edge

The hostname media.atlas.example points traffic at the CDN instead of directly at the Atlas origin. CDNs commonly combine DNS steering with Anycast routing.

DNS Chooses a Service Address

The authoritative DNS system can select an answer using resolver location, measured latency, PoP health, available capacity, and customer policy. A CNAME can lead from the Atlas hostname to a CDN-controlled hostname whose answers change over time.

DNS does not run for every HTTP request. Recursive resolvers and clients cache answers according to their TTL. A shorter TTL allows faster redirection but increases DNS traffic; a longer TTL reduces lookup work but leaves old answers in use longer. The resolver’s location can also differ from the reader’s location, so DNS steering is an informed approximation rather than exact per-user routing.

Anycast Chooses a Reachable PoP

With Anycast, several PoPs advertise reachability to the same service IP address. Internet routing selects one advertised path. “Nearest” means preferred by current routing topology and policy—not necessarily the closest building or even the path with the lowest measured latency.

For the Atlas request, DNS returns 203.0.113.80, advertised from London, Amsterdam, and Paris. Current routing sends the new connection to London:

If London withdraws that route, later connections can reach another PoP. Existing TCP or QUIC connections might fail and reconnect; Anycast changes reachability, not application-session ownership or transport state.

DNS and Anycast solve different parts of steering:

DNS       hostname -> service address
Anycast   service address -> one currently reachable PoP

They find a delivery location. They do not decide whether the request is cacheable or which stored object it represents.

Ingress Establishes Request Context

Inside the London PoP, the connection passes through several logical stages:

1. a local load balancer selects an ingress proxy;
2. TLS Server Name Indication selects candidate certificate and tenant state;
3. HTTP Host or :authority confirms the requested tenant;
4. rate limits, bot policy, and a web application firewall run where enabled;
5. request normalization produces one canonical internal interpretation;
6. cache policy decides whether shared lookup is allowed;
7. the cache key identifies the requested representation.

The order is security-sensitive. The cache and origin must not interpret the same path, authority, percent encoding, duplicate header, or message length in different ways. Ambiguous input is rejected or canonicalized once, and that canonical interpretation is used for lookup, forwarding, logging, and purge.

Multi-tenant isolation begins here as well. An internal tenant identifier or validated hostname must participate in routing and key construction; otherwise two customers that both publish /logo.svg could collide.

What to remember: Steering chooses an edge location. Ingress then proves which tenant and request the edge is handling before any cache lookup occurs.

6. The Edge Serving Decision: Identity and Eligibility

After routing and ingress, the edge has to decide whether local state can answer the request. That requires an exact identity. “The same URL” is often too vague: query parameters can select image size, request headers can select format or language, and identical paths can belong to different tenants.

A representative internal identity for the Atlas request is:

tenant       = atlas
scheme       = https
authority    = media.atlas.example
path         = /eclipse/hero
query        = w=1200
method       = GET
variant      = accept-format:avif

HTTP caching uses the request method and target URI as the primary cache key. Stored Vary metadata then distinguishes representations selected by request headers. A CDN can add controlled tenant partitioning and normalization, but it must preserve the application’s semantics.

Every key choice trades correctness against reuse:

• omit w=1200, and different image sizes can collide;
• preserve every tracking parameter, and equivalent URLs fragment into many cold entries;
• omit the Atlas tenant, and two customers can share data accidentally;
• include a unique request ID, and every request becomes a miss;
• ignore a representation-selecting header, and a client can receive a format it cannot use.

Query normalization should therefore use an allowlist of understood rules. A signature, authorization token, experiment selector, or origin-routing parameter can affect the response even if it resembles disposable tracking data.

Vary Adds Request-Header Dimensions

Atlas returns:

Vary: Accept

The primary URI can now have several stored variants:

Accept selects AVIF -> hero-v7-avif
Accept selects WebP -> hero-v7-webp
Accept selects JPEG -> hero-v7-jpeg

For a later request, the cache compares the fields named by the stored response’s Vary value. Only a matching variant can be reused. Vary: * never matches a later request.

Variation must be bounded. Vary: User-Agent can produce thousands of nearly unique entries. A CDN might convert raw device or format headers into a small, audited set of classes, provided the edge and origin use the same classification.

Eligibility Comes Before Reuse

Even a perfectly matching key is not enough. Shared caches must apply response policy:

no-cache is commonly misunderstood: it does not mean “do not store.” It means “do not reuse without successful validation.” no-store is the directive that forbids storage.

Cookies should not automatically become an unbounded cache-key dimension. Personalized routes normally bypass shared caching. If a bounded cookie-derived variant such as a country or plan tier is truly safe, it is extracted through explicit, reviewed policy.

Most CDN caching policies concentrate on GET and HEAD. Other methods are not made reusable merely because they return a body; both HTTP semantics and explicit CDN policy must permit storage and later reuse. Likewise, some status codes are cacheable by default while others require explicit freshness. The safe operational rule is to configure understood methods, routes, and statuses rather than treating every response as a candidate.

The most important safety rule is simple:

Never let a cache hit skip authorization that an equivalent miss would have performed.

What to remember: The key proves identity; cache directives and security policy prove eligibility. Safe shared reuse needs both.

7. Edge Storage Is a Locality System

The Atlas story so far says “the London edge stores a response.” In a real PoP, that does not mean one server owns one giant cache. Edge storage is a locality system: keep hot bytes close to the network path that needs them, refill cold bytes through controlled upstream paths, and avoid random duplication across the fleet.

One PoP Contains Several Cache Layers

The London PoP is not one enormous map. A typical request can pass through:

1. process memory for the hottest metadata and small bodies;
2. PoP-local memory or disk for a larger working set;
3. a regional or origin shield shared by several PoPs;
4. the origin when no reusable upstream entry exists.

Inside the PoP, a dispatcher can use consistent or rendezvous hashing to map a key onto a cache node. This keeps repeated requests for one object near the same storage instead of duplicating it randomly across every node.

If node E3 disappears, only the keys assigned to its portion of the mapping move to other nodes. The mapping is an optimization, not durable authority: lost cache bytes can be fetched again.

Hot objects can exceed one node’s network capacity even if storage is balanced. A CDN can replicate a hot key, use two-choice routing, or serve it from several worker memories. Placement must account for request rate as well as object size.

The Response Fills Outward

For the first request, E3 misses in memory and disk. Its shield also misses and fetches hero-v7-avif from Atlas:

origin 200 response
    -> shield stores an eligible response
    -> E3 stores it on disk
    -> E3 may promote it to memory
    -> reader receives bytes as they arrive

The edge does not need to receive all 2.4 MiB before sending the first bytes to the reader. It can stream and store concurrently. The entry must not become reusable, however, until headers, length, storage completion, and any integrity checks are valid. A disconnected or truncated fill is not a complete cached response.

What to remember: A CDN cache is usually a hierarchy. Memory, local disk, peer nodes, shields, and the origin are different places a request can look before it gives up and fetches from the authoritative source.

8. Miss Coordination Protects Upstream Systems

A single cold miss is harmless. Ten thousand readers requesting a newly published object before its first fill finishes can overwhelm the shield or origin. CDN design therefore treats misses as load that must be coordinated, not just as cache failures.

Without coordination:

10,000 edge misses -> 10,000 upstream fetches

With request collapsing, the first miss becomes the leader for that cache identity. Equivalent later requests become followers:

1 leader             -> shield or origin fetch
9,999 followers      -> wait for the same result

Equivalence matters. Requests with different authorization, range, variant, or cache policy cannot safely join merely because their paths look similar. When the response headers arrive, the cache must also confirm that the result is reusable for the waiting requests.

Collapsing needs bounded failure behavior:

• a maximum waiter count and wait deadline;
• leader cancellation that cannot strand followers;
• retry limits and randomized delay;
• correct propagation of non-cacheable and error responses;
• a generation check so an older fill cannot publish after a purge.

The same pattern can operate at multiple layers. Edge collapsing protects a shield, and shield collapsing protects the origin across several PoPs.

The standardized Cache-Status header can describe a hit, forwarded miss, stored result, or collapsed request. For example:

Cache-Status: AtlasEdge; fwd=uri-miss; collapsed; stored

That is diagnostic metadata, not a caching instruction. Public responses should avoid exposing sensitive cache keys or internal topology.

What to remember: A viral object should create many downstream responses, not many simultaneous origin computations. Request collapsing changes concurrency, while caching changes reuse over time.

9. Freshness Is a Serving Policy

An eligible, matching response can still be too old for ordinary reuse. Freshness is how the origin and CDN agree on when the edge may answer independently and when it must involve an upstream system. HTTP calls a stored response fresh while its freshness lifetime exceeds its calculated current age:

fresh = current_age < freshness_lifetime

For the Atlas response, s-maxage=600 takes precedence in the shared CDN cache over max-age=60:

Cache-Control: public, max-age=60, s-maxage=600

The CDN can normally reuse the response for ten minutes. A browser receiving the same response has a one-minute lifetime, but it does not necessarily receive a new minute. If the CDN sends Age: 420, the browser knows the response has already aged well beyond its browser-cache lifetime.

Explicit freshness is normally chosen in this order: a shared cache uses s-maxage when present, otherwise max-age, otherwise an Expires time relative to Date. When no explicit lifetime exists, HTTP can permit heuristic freshness, where the cache estimates a conservative lifetime from metadata such as Last-Modified. Important CDN routes generally use an explicit policy so freshness is intentional rather than inferred.

Age is the cache’s estimate of how long ago the response was generated or last validated at the origin. Correct calculation includes inherited Age, apparent origin age, transfer time, and time resident in each cache. A downstream cache cannot reset the age merely because it received the response from an upstream cache.

At age 420 seconds, the London edge can return a fresh hit:

HTTP/1.1 200 OK
Age: 420
ETag: "hero-v7-avif"
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=30, stale-if-error=86400
Cache-Status: AtlasEdge; hit; ttl=180

No origin request occurs. At age 600, the response becomes stale. Stale does not mean corrupt, deleted, or unusable under every circumstance. It means the ordinary freshness permission has ended; the cache now needs validation or a specific rule allowing stale reuse.

Freshness and storage residency are independent:

A TTL controls reuse semantics. Eviction manages finite storage. Recency, frequency, body size, fetch cost, tenant quota, and admission policy can all influence which entries remain. Removing London’s local copy does not purge Amsterdam or tell the origin that the representation has changed.

What to remember: Presence answers “do we have bytes?” Freshness answers “may we reuse them without contacting the origin?” They are separate decisions.

10. Revalidation and Stale Service Reduce Origin Coupling

When hero-v7-avif becomes stale, the edge could send every reader back to the origin or download the entire 2.4 MiB body again. Revalidation gives the CDN a cheaper control path: ask whether the stored representation changed before refilling the body.

The ETag validator allows this conditional request:

GET /eclipse/hero?w=1200 HTTP/1.1
Host: media.atlas.example
Accept: image/avif,image/webp,image/*;q=0.8
If-None-Match: "hero-v7-avif"

If the selected representation is unchanged, the origin returns:

HTTP/1.1 304 Not Modified
ETag: "hero-v7-avif"
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=30, stale-if-error=86400

The 304 carries no image body. It tells the cache to reuse the stored body while updating the response metadata used for future decisions.

If Atlas has published hero-v8-avif, the origin instead returns 200 OK with the new body and ETag: "hero-v8-avif". The cache completes and validates that replacement before publishing it as the reusable entry.

Entity tags can be strong or weak. A strong validator identifies a byte-equivalent representation and is needed for operations such as safely combining partial ranges. A weak validator can express semantic equivalence but does not prove that every byte is identical.

Last-Modified is another validator. A cache can send If-Modified-Since: <stored timestamp> when no suitable entity tag exists. Timestamps have coarser semantics and can miss multiple changes within their resolution, so an origin-generated ETag is usually the clearer validator for versioned or rapidly changing representations.

Revalidation saves transfer, but a synchronous check still adds origin latency to the request that first discovers staleness.

stale-while-revalidate Hides That Latency

Atlas permits 30 seconds of stale-while-revalidate. At age 601, the edge may return the stale v7 body immediately while launching one background revalidation. Equivalent requests can continue using that stored body during the window while collapsing prevents duplicate refreshes.

If the origin returns 304, the entry becomes fresh again. If it returns v8, later readers receive the replacement. The earlier readers deliberately traded up to 30 seconds of additional staleness for lower latency.

This is an application decision, not a universal default. A versioned asset can tolerate generous caching. A safety alert, price, authorization decision, or account balance may require synchronous validation or no shared caching.

An emergency invalidation overrides ordinary stale permission. If Atlas marks v7 forbidden, the edge cannot keep serving it merely because the HTTP stale window has time remaining.

stale-if-error Trades Recency for Availability

Suppose the origin later returns an eligible 503 Service Unavailable. stale-if-error=86400 allows a previously valid stored response to be used within a bounded stale window instead of returning that error.

This works only when:

• a usable response was cached before the failure;
• the error is eligible under the configured policy;
• stale age remains within the allowed boundary;
• no security or editorial invalidation forbids the object;
• observability records that stale fallback occurred.

Serving stale indefinitely would hide a prolonged outage and violate the content’s freshness contract. A CDN improves availability only for state it already has and is allowed to reuse.

What to remember: Revalidation asks whether old bytes are still current. Stale directives define bounded cases where latency or availability is more valuable than immediate recency.

11. Special Delivery Cases Need Explicit Policy

The successful hero image is the simplest case. Missing objects, large files, redirects, and errors can also be cached, but only with carefully bounded semantics.

Negative Caching

Suppose the page references /eclipse/caption.vtt before that file exists. Without negative caching, every reader repeats the same origin 404.

HTTP/1.1 404 Not Found
Cache-Control: public, s-maxage=15

A 15-second cached 404 can protect the origin while allowing recovery soon after the object appears.

Negative caching should use an allowlist of understood status codes and short, explicit TTLs. Publication can purge an old negative entry immediately. A mistaken negative key is particularly damaging because it turns one routing, variant, or authorization error into a distributed outage.

Range Requests

Video players and download clients often request part of a large object:

GET /eclipse/film.mp4 HTTP/1.1
Range: bytes=1048576-2097151

The response identifies both the selected interval and complete object:

HTTP/1.1 206 Partial Content
Content-Range: bytes 1048576-2097151/503316480
ETag: "film-v3"

The CDN can store a complete body, independent segments, or a larger internal slice than the client requested. It must never assemble a response from ranges belonging to different versions. Safe combination requires a shared strong validator and consistent total length.

Range traffic also changes capacity: many viewers begin at offset zero, while seeking creates a wide working set. Segment size and prefetching trade hit ratio against memory, disk I/O, latency, and wasted upstream bytes.

Admission Is Not Automatic

An eligible response need not be worth storing at every layer. Caching a stream of one-hit objects can evict a small hot working set. A CDN might:

• keep metadata and small hot bodies in memory;
• admit larger objects to disk first;
• promote only after repeated access;
• stream an oversized object without storing the complete body;
• reserve tenant quotas so one workload cannot evict every other tenant.

Admission and eviction are resource policies. They cannot make an ineligible response safe to reuse.

12. Shields Protect the Origin from Global Demand

Edge collapsing limits one PoP to one upstream miss for a key. Without another layer, 200 PoPs can still produce 200 near-simultaneous origin requests for a globally popular object.

An origin shield is a cache near the origin or at a well-connected hub:

many readers -> many edge PoPs -> a few shields -> origin

The shield creates reuse and collapse across PoPs. It can also bound origin concurrency, queue briefly, shed low-priority work, enforce retry budgets, and open a circuit breaker when the origin is failing.

Every layer must construct compatible identities. If the edge removes a tracking query parameter but the shield preserves it, the shield’s hit ratio fragments. If the shield omits an authorization or tenant boundary enforced at the edge, data can leak.

Retries also require one owner. If browser, edge, shield, and origin proxy each retry independently, one upstream failure multiplies traffic. A retry budget defines which layer retries, how often, and with what randomized delay.

A shield adds a hop on a true miss. Its value is global aggregation and origin protection, not proximity to the reader.

Approximate request offload can be reasoned about in stages. If cacheable request rate is Q, edge request hit ratio is H, collapsed miss fan-in is C, and shield hit ratio is S:

requests reaching shield ~= Q * (1 - H) / C
requests reaching origin ~= Q * (1 - H) * (1 - S) / C

These are planning approximations. Variants, revalidation, ranges, errors, retry, regional skew, and bypass traffic all change the real result.

Request hit ratio and byte hit ratio answer different questions:

request_hit_ratio = cache hits / cacheable requests
byte_hit_ratio    = bytes from cache / cacheable response bytes

A million cached 2 KiB icons can produce a high request hit ratio while one uncached 5 GiB download dominates origin bandwidth. Both metrics are needed.

What to remember: Edge caches protect readers from distance. A shield protects the origin from the combined miss stream of the edge fleet.

13. Content Change Is a Global Coordination Problem

Normal freshness says how long an old representation may remain reusable. It does not instantly remove a response already copied into many PoPs.

Prefer Versioned URLs for Immutable Assets

Atlas can publish a corrected image under a new identity:

/eclipse/hero.v8.avif

and update the page to reference it. hero.v7.avif can retain a long TTL because its URL never changes meaning. Versioning converts “remove every old copy now” into “make new documents reference a new object.”

This is the simplest model for compiled assets, application bundles, images, and other immutable content. It is not sufficient when the old URL itself must stop working—for example after a legal removal, leaked secret, unsafe file, or urgent correction on a stable URL.

A Purge Is a Distributed State Change

Suppose hero-v7 contains an incorrect photo credit and cannot wait for the ten-minute freshness lifetime. Atlas issues a purge for the canonical identity and its Accept variants.

A purge system must:

1. authenticate and durably record the request;
2. assign an ordered generation or operation ID;
3. distribute it to relevant regions and PoPs;
4. mark matching memory and disk entries unusable;
5. track application and retry disconnected sites;
6. prevent an older in-flight fill from publishing after invalidation.

The fill race is easy to miss:

generation 9820: miss for v7 starts
generation 9821: purge commits
generation 9820: old fetch completes

If the fill publishes without checking its starting generation, it resurrects the purged object. The completed fetch must be discarded or evaluated under the new generation.

Purge propagation is not physically instantaneous. An API must define whether “complete” means accepted centrally, delivered to all currently healthy PoPs, or verified no longer servable. Disconnected PoPs need an explicit rule for how long they may serve without current invalidation state.

What to remember: TTL handles ordinary change over time. Versioned URLs avoid changing an existing identity. Purge is the emergency distributed path for making an existing identity unusable.

14. The Control Plane Turns Edges into One CDN

One cached web server in London can reduce latency for London. A CDN is the coordination that makes servers across many networks and locations behave as one delivery service.

The fleet separates a high-volume data plane from a lower-volume control plane.

The data plane handles live requests:

connection -> security -> normalized request -> cache decision -> response

The control plane distributes the state that tells those request handlers how to behave:

tenant configuration
certificates
purge generations
routing and health policy
content-placement plans

The edge must not perform a global control-plane query for every request. Each PoP keeps a validated local snapshot so the data plane can remain fast and can continue safely through a bounded control-plane interruption.

Control-Plane State Has Different Consistency Needs

A purge, certificate update, and routing-health sample are all “control data,” but they do not need the same contract:

• purge and security configuration need durable ordering;
• certificate state must respect validity and tenant binding;
• routing policy needs a coherent version;
• short-lived CPU samples can be eventually consistent and disposable.

Safety-critical updates are committed to replicated authority before they are acknowledged. Regional fanout services distribute immutable versions:

tenant config       version 418
certificate         generation 73
purge               generation 9821
routing policy      version 204

An edge validates a complete snapshot and installs it atomically. It does not combine half of configuration 418 with half of 417.

If a control-plane leader fails, a replica can take over from committed state. Terms or epochs fence the old leader so it cannot later publish an older certificate, resurrect a purge generation, or create a competing routing decision.

A PoP Can Be Reachable but Not Safe to Serve

During control-plane isolation, the London PoP has cached bytes and can still receive traffic. Whether it may serve depends on the state involved:

• immutable public objects can often continue under the last-known-good policy;
• ordinary configuration can remain valid for a bounded lease;
• a new tenant cannot activate without locally proven configuration;
• an expired certificate cannot be stretched by a cache lease;
• security-sensitive purge state can require withdrawal or fail-closed behavior after a short deadline.

Routing readiness must include more than “the machines answer health checks.” A PoP should advertise a tenant only while it has sufficiently current, internally consistent certificate, configuration, purge, and serving state.

What to remember: Edge servers answer requests from local state. The control plane makes that local behavior coherent across the fleet without becoming a synchronous dependency of every cache lookup.

15. Failover Moves Traffic and Cold State

Cached bodies are replaceable, but failures still change load and correctness conditions. Each layer has a different recovery action.

One Cache Node Fails

When E3 fails, consistent hashing remaps its keys to remaining nodes. The lost bodies can refill from disk peers, shield, or origin. The dangerous part is the cold wave: many newly assigned keys can miss at once. Admission limits, collapsing, and shield hits must prevent recovery from becoming an origin incident.

One PoP Fails

The CDN withdraws or de-preferences London’s route and changes DNS steering. New connections reach Amsterdam or another healthy site. Existing transports may fail and reconnect. The destination PoP needs reserved network, CPU, storage, and TLS capacity for spillover; routing cannot create capacity that was never provisioned.

A Shield Fails

Edges can select another shield or use direct origin access under a strict global budget. Allowing every PoP to bypass blindly turns a shield outage into an origin outage.

The Origin Fails

Fresh hits continue without origin access. Eligible stale entries can use stale-if-error. Requests with no stored response, or content forbidden from stale reuse, fail. The CDN cannot manufacture authoritative content it never received.

The Invalidation Stream Is Interrupted

A disconnected PoP can serve only within its control-policy lease. On reconnection it retrieves missed purge generations or a fresh snapshot before advertising full readiness. Otherwise it could reintroduce content that the rest of the fleet has already forbidden.

Failure often moves several metrics at once:

route shift
    -> TLS connection surge at failover PoP
    -> cache remapping and lower hit ratio
    -> shield queue growth
    -> higher origin load

Designs must be tested in that combined recovery state, not only at steady state.

What to remember: A cache node stores replaceable data, but losing it is not free. Recovery consumes cold-fill bandwidth and shifts pressure to every upstream layer.

16. CDN Security Starts with Request Identity

Many cache vulnerabilities are disagreements about what a request means:

• cache poisoning: attacker-controlled input changes an origin response but is absent from the key, so a victim receives the stored result;
• cache deception: a personalized route is made to resemble a public static resource;
• tenant collision: host or tenant identity is missing from the key;
• variant confusion: Vary is absent or interpreted differently;
• cache-layer disagreement: edge and shield normalize into different identities;
• purge mismatch: invalidation names a different canonical identity from lookup;
• signed-URL bypass: authorization input is removed before the decision it protects.

The core defenses are consistency and explicit policy:

• normalize once, then use the same canonical form for keying, forwarding, logging, and purge;
• reject ambiguous authority, path, length, and transfer semantics;
• partition storage by tenant and security policy;
• cache only routes deliberately classified as safe;
• bound every variant dimension;
• include or validate every input that changes the response;
• apply equivalent authorization on hits and misses.

Increasing hit ratio is never worth crossing a privacy or authorization boundary.

Bound Work Before It Becomes an Incident

An internet-facing CDN also limits:

• request and connection establishment rate;
• header and body sizes;
• normalization and security-processing work;
• variants and key cardinality per route or tenant;
• collapsed followers and wait duration;
• memory and disk occupancy;
• concurrent shield and origin fetches;
• retries and stale-serving duration;
• purge fanout and tag expansion.

These bounds protect against attacks, client bugs, configuration mistakes, and ordinary traffic spikes.

17. Capacity Planning Starts with Bytes and Failover

Connection count matters, but large reusable bodies make bandwidth and storage locality dominant.

For event or object request rate R, encoded body size B, and downstream fan-out F:

downstream body bandwidth ~= R * B * F

Video makes the scale visible. One million viewers consuming 5 Mbit/s each require approximately:

1,000,000 * 5 Mbit/s = 5 Tbit/s

An origin RAM cache can avoid repeated encoding and disk reads. It cannot avoid sending roughly 5 Tbit/s out of the origin network. A CDN distributes those bytes across many PoPs and access-network paths. One upstream copy of a popular segment at a location can support many local plays.

The small application-control path and large media path can remain separate:

player -> application API -> authentication, entitlement, manifest
player -> nearby CDN      -> high-volume media segments

Capacity planning includes failure states:

• one node’s hot keys moving to neighbors;
• one PoP’s traffic moving to regional peers;
• cold-cache refill after restart or deployment;
• a shield-bypass budget;
• broad purge bursts;
• TLS and QUIC handshakes after route changes;
• log delivery and control-plane catch-up during recovery.

Steady-state unused capacity is what makes failover possible.

18. Observe the Whole Delivery Path

A useful request record explains the decisions rather than reporting only a final “hit” or “miss”:

request_id          r-8f3a
pop / node          lon-2 / E3
tenant              atlas
key_hash            7c91...
variant             avif
cache_status        hit | miss | stale | revalidated | bypass
age / ttl           420 / 180
collapse_role       leader | follower | none
upstream            S-LON | origin | none
purge_generation    9821
bytes               2516582
latency              edge=4ms upstream=0ms total=6ms

Monitor distributions by tenant, route, PoP, and status rather than relying on fleet-wide averages:

• request and byte hit ratio;
• fresh, stale, revalidated, bypass, and negative-hit rates;
• key cardinality and variant explosion;
• object admission, eviction, and size distributions;
• collapsed followers, leader duration, and waiter timeouts;
• shield hit ratio, queue depth, and origin concurrency;
• revalidation 304 versus replacement 200 rate;
• purge commit, delivery, application, and verification latency;
• active configuration, certificate, routing, and purge versions per PoP;
• stale-if-error age and volume;
• route withdrawals, node remaps, and regional spillover.

The most useful incident view connects ownership stages. A route shift followed by a TLS surge, cache cooling, shield congestion, and origin saturation is one causal chain, not five unrelated alerts.

Test failure paths directly: stop a cache node, isolate a PoP from purge distribution, make the shield fail, slow the origin, race a purge with an in-flight fill, and redirect a region’s traffic. Verify that queues remain bounded and forbidden entries do not reappear.

19. When a CDN Is the Wrong Tool

CDN delivery is most valuable when many users request the same safely reusable responses. It is not a reason to mark every route public.

A CDN can still proxy, filter, terminate TLS, and carry these requests. The question is specifically whether shared response reuse is correct and valuable.

20. The CDN Design Checklist

A CDN design is complete only when the live request path and the fleet-control path are both specified. The Atlas image now has one connected request history:

1. Atlas publishes hero-v7-avif and an HTTP policy describing shared reuse.
2. DNS returns a CDN service address and routing sends the reader to London.
3. TLS and HTTP authority select the Atlas tenant.
4. ingress normalizes the request and applies security and cache eligibility.
5. the URI, tenant, method, query, and Vary: Accept identify the AVIF variant.
6. consistent hashing assigns that identity to edge node E3.
7. memory and disk miss, then the shield misses.
8. one collapsed leader fetches from Atlas while equivalent followers wait.
9. the response streams to readers and fills shield, disk, and memory.
10. later requests hit while current_age < 600.
11. after staleness, one conditional request sends If-None-Match: "hero-v7-avif".
12. 304 Not Modified refreshes metadata without resending 2.4 MiB.
13. an eligible origin failure uses the bounded stale-if-error copy.
14. an urgent correction commits purge generation 9821 and blocks older fills.
15. the next miss stores corrected hero-v8-avif.
16. if E3 fails, its identities remap and refill through the shield.
17. if London withdraws, a new connection reaches another prepared PoP.

The request can be compressed into five decisions:

where?       DNS and routing choose a PoP
who/what?    ingress establishes tenant and canonical request
which entry? cache key and Vary identify a representation
reuse now?   eligibility, freshness, validation, and stale policy decide
if not?      hierarchy, collapsing, shield, and origin produce a response

Around those decisions, the fleet distributes configuration, invalidation, capacity, and failure recovery.

When designing a CDN, make each of these choices explicit:

What the CDN Can and Cannot Provide

With correct origin and edge policy, a CDN can provide:

• lower latency for warm, reusable responses;
• reduced origin requests and response bytes;
• representation-safe reuse through keys and Vary;
• bounded freshness and conditional revalidation;
• controlled stale service during refresh or eligible failure;
• distributed invalidation with observable propagation;
• remapping and rerouting around nodes and PoPs.

It cannot automatically provide:

• the correct application-specific cache key;
• safety for a private response mistakenly declared public;
• simultaneous purge at every isolated site;
• unlimited availability when no reusable response exists;
• correct range assembly without validators;
• capacity that was not reserved for failover;
• agreement when the origin and cache parse a request differently.

The origin, publishing system, and CDN share one HTTP caching contract. A fast hit is useful only when every layer agrees that it is the right response for this reader, at this time.

Compact Glossary

References

1. IETF, RFC 9111: HTTP Caching
2. IETF, RFC 9110: HTTP Semantics
3. IETF, RFC 5861: stale-while-revalidate and stale-if-error
4. IETF, RFC 9211: The Cache-Status HTTP Response Header Field
5. IETF, RFC 4786: Operation of Anycast Services
6. Karger et al., Consistent Hashing and Random Trees