CDNs and edge delivery

Distance is latency. A user in Mumbai fetching an image from a server in Virginia pays for a long round trip, on every request. A CDN keeps copies close to the user.

Crux: a CDN caches responses at edge servers worldwide. You win latency, origin load and bandwidth cost. You pay with staleness, so control time to live (TTL) and invalidation deliberately.

What it is and why it exists

A content delivery network (CDN) is a network of edge servers (points of presence) spread around the world. Users are directed to a nearby one, usually by DNS or anycast routing. If it has the content, it answers at once. If not, it fetches from your origin (your servers or blob storage), keeps a copy, and answers.

Benefits:

• Lower latency, since the data is nearby and connections are reused.
• Less load on the origin, since most requests never reach it.
• Absorbing spikes and some denial-of-service traffic.
• Lower bandwidth cost from the origin in many setups. Check this with your provider's pricing, which I am not quoting.

Examples: CloudFront, Cloudflare, Akamai, Fastly.

How it works, at the level you must explain

Hits, misses and the cache key

• A cache hit is served from the edge. A miss goes to the origin.
• The cache key says what counts as the same object: usually the URL, and optionally chosen headers, query strings or cookies. Be careful here. Including a cookie or a rarely-repeated query string in the key can make every request unique and kill the hit rate.
• The cache hit ratio is the number to watch. A high ratio means the CDN is doing its job.

Freshness: TTL, headers and invalidation

• The origin controls caching with HTTP headers: Cache-Control: max-age=... (how long to keep), s-maxage (for shared caches), private (do not cache at the edge), no-store. ETag and Last-Modified allow cheap revalidation: the edge asks "has it changed?" and gets a short "not modified" reply.
• A TTL is the time an edge keeps a copy. Long TTL means high hit ratio and more staleness.
• Invalidation (purge) removes an object before its TTL ends. It exists, but it is slow, sometimes costly, and propagates across the network over seconds to minutes.
• A better habit is versioned URLs: put a hash or version in the file name (app.3f9a1.js). A change means a new URL, so there is nothing to purge, and you can use a very long TTL.
• Stale-while-revalidate serves a stale copy while it fetches a fresh one in the background, keeping latency low.

What to cache

• Static assets: images, video segments, scripts, styles, downloads. The classic fit.
• Video streaming: video is cut into small segments, which cache well.
• API responses that are the same for everyone and change slowly (a public product list). Use short TTLs.
• Not personalised or authenticated responses, unless the cache key is designed with care. A mistake here can show one user's data to another.

Protecting the origin

• Edge servers on a miss can send many requests for the same object at once. Request collapsing merges them into one origin fetch.
• An origin shield (a middle caching tier) means only one place talks to the origin, rather than every edge.
• Keep the origin able to handle a cold cache after a purge or a new release. Sudden misses everywhere can overload it.

More than caching

Many CDNs also terminate TLS near users, compress, resize images, run small pieces of code at the edge (for redirects, auth checks, A/B tests), and filter attacks with a web application firewall.

Pull and push

Most CDNs are pull: the edge fetches from the origin on the first miss. Easy, and the origin stays the only source. Push means you upload content to the CDN ahead of demand. It suits large, known releases (a game patch, a big launch video) where you want to avoid a cold start. Pull plus a pre-warm request to important URLs gives much of the same effect.

Private content

A CDN is public by default. To protect content, use signed URLs or signed cookies: a signature with an expiry, checked at the edge. Pair with a private origin that only accepts requests from the CDN, so people cannot bypass it. For a subscription video service, signed cookies let one grant cover a whole stream of segments.

Headers that decide the hit ratio

• Vary: tells caches that a response differs by a header (language, encoding). Each variant is cached separately, so vary only on what really changes the response.
• Query strings: ignore the ones that do not change content (tracking parameters), or the same file is cached many times.
• Negative caching: a short cache of "not found" or error responses protects the origin from repeated misses on bad URLs.
• Range requests: large files can be cached and served in pieces.
• Compression and image formats: serve the right format per client, with a bounded set of variants.

Estimating the hit ratio and origin load

Our own numbers: 20,000 requests per second at peak and a hit ratio of 95% leaves 1,000 requests per second reaching the origin. At 80%, the origin sees 4,000, four times as much. Small changes in hit ratio are large changes for the origin, so the cache key and TTL matter. The long tail matters too: if most files are requested once a month, each goes to the origin and the cache gives little. Layered caching (edge, then regional, then shield) raises the effective ratio by turning many edge misses into one.

Dynamic content

A CDN can speed up uncacheable requests too: users connect to a nearby edge over a short hop, and the edge forwards over a warm, optimised connection to the origin. The gain is lower connection and handshake cost, not caching.

Failures

If the origin is down, the CDN can serve stale content for a while (stale-if-error), which is a good default for content sites. If the CDN itself has an outage, DNS-level failover to a second provider is possible but adds cost and complexity. Decide whether that availability is worth it.

When to use it, trade-offs, numbers

• What: a global cache of responses at the network edge.
• Use when: you serve static files, media or any read-heavy content to users in many places.
• Know: cache key, TTL and headers, invalidation versus versioned URLs, hit ratio, and what must not be cached.
• Careful: staleness, caching of private data, a low hit ratio for unique or rarely-requested content (a long tail of files adds little), and cost per request and per gigabyte.

Which tool?

• CloudFront is the default to name on AWS with S3 as origin. Cloudflare and Fastly are strong alternatives, and also offer edge compute.
• An in-memory cache (Redis) sits behind your servers and is not near the user. It helps your database, not user distance.
• Browser cache: the first cache of all, controlled by the same headers.
• Regional replicas of a database or service: for dynamic data that a CDN cannot cache.

Numbers: a CDN round trip commonly saves tens to a few hundred milliseconds compared with a far origin, but this depends on geography, so I would say "less than the round trip to the origin" and avoid a precise figure. I am leaving out prices and edge-location counts, which change.

Common mistakes

• Caching personalised responses, and leaking data between users.
• Relying on purges, instead of versioned file names.
• A cache key that includes needless variation, so the hit ratio collapses.
• No plan for the origin when the cache is cold.
• Putting a CDN in front of content that is rarely re-requested.

• Leaving a public origin open so users bypass the CDN, and its protections.
• Tracking query parameters in the cache key.
• Sending the full cookie set to the edge for static files.
• Treating a high TTL as free, when a bad file is then stuck at the edge until it expires or is purged.

Check yourself

• What happens on a cache miss at the edge?
• How do max-age and an ETag each help?
• Why prefer versioned URLs to purging?
• What would make the hit ratio low, and how would you notice?
• Which responses must not be cached at the edge?
• How does an origin shield protect the origin?
• What are signed URLs for?
• If the hit ratio drops from 95% to 80%, how does origin load change?
• When would you push content to a CDN ahead of time?