DNS & CDN

At 10:00 the BoxOffice stadium sale opens and 2 million phones ask the same two questions almost at once: "where is boxoffice.com?" and "give me the event page". If both questions travel all the way to the BoxOffice servers in São Paulo, the servers drown before a single seat is sold. The Domain Name System (DNS) and a Content Delivery Network (CDN) exist so that most of those questions are answered close to the fan, by machines that BoxOffice does not even operate.

This page covers the first two hops of the request's journey: DNS decides where a client goes; the CDN decides whether the request needs to go any further.

DNS: the Internet's phone book

Computers talk to IP addresses (203.0.113.10, 2001:db8::10); people type names (boxoffice.com). DNS translates the name into addresses. It is one of the oldest and largest distributed systems in existence: a hierarchical, delegated, heavily cached database¹.

• Hierarchical: names are read from right to left. The root (.) knows who is responsible for com.; com. knows who is responsible for boxoffice.com.; boxoffice.com. knows the address of api.boxoffice.com.
• Delegated: each level is run by a different organisation. BoxOffice controls only its own zone, boxoffice.com, through its authoritative name servers (usually a managed service such as Route 53 or Cloudflare DNS).
• Cached: every answer carries a TTL (time to live, in seconds). Resolvers keep the answer for that long, so the vast majority of lookups never reach the authoritative servers.

Resolution, step by step

When the fan's phone needs api.boxoffice.com, it does not walk the hierarchy itself. A tiny stub resolver in the operating system asks a recursive resolver (the one from the mobile carrier, or a public one such as 1.1.1.1 or 8.8.8.8), which does the work and caches the results:

sequenceDiagram
    participant P as Phone (stub resolver)
    participant R as Recursive resolver
    participant Root as Root server
    participant TLD as .com TLD server
    participant A as boxoffice.com authoritative
    P->>R: A? api.boxoffice.com
    Note over R: cache miss
    R->>Root: A? api.boxoffice.com
    Root-->>R: ask .com (NS a.gtld-servers.net)
    R->>TLD: A? api.boxoffice.com
    TLD-->>R: ask boxoffice.com (NS ns1.boxoffice-dns.net)
    R->>A: A? api.boxoffice.com
    A-->>R: 203.0.113.10, TTL 60
    Note over R: cache for 60 s
    R-->>P: 203.0.113.10
    P->>R: A? api.boxoffice.com (another fan, 5 s later)
    R-->>P: 203.0.113.10 (from cache)

A cold lookup costs several round trips (tens to hundreds of milliseconds); a cached one costs a single round trip to the resolver, or nothing at all when the answer is in the operating system or browser cache. In practice the root and .com answers are almost always cached too (their TTLs are one to two days), so a typical miss costs just one trip from the resolver to the authoritative server.

Record types

TTL: freshness versus load

The TTL is a classic trade-off between how fast a change propagates and how much load and latency the lookups cost:

Routing with DNS: GeoDNS, latency, weights and failover

Because the authoritative server chooses the answer, it can give different answers to different clients. Managed DNS services offer several routing policies⁴:

DNS failover is the simplest form of multi-region disaster recovery: the DNS provider probes https://sa.boxoffice.com/health every 10–30 s from several locations; after a few failures it stops returning the primary's address and returns the secondary's.

sequenceDiagram
    participant HC as DNS health checker
    participant P as Region SP (primary)
    participant S as Region US (secondary)
    participant F as Fan's resolver
    HC->>P: GET /health
    P-->>HC: 200 OK
    F->>HC: A? api.boxoffice.com
    HC-->>F: SP address (TTL 60)
    Note over P: region outage
    HC->>P: GET /health
    Note over HC: 3 failures in a row: mark SP unhealthy
    F->>HC: A? api.boxoffice.com (after its TTL expired)
    HC-->>F: US address (TTL 60)
    F->>S: traffic now goes to the secondary

Try it. Open the multi-region scenario and run the load profile. Then trigger the outage of the primary region (chaos high, or disable the region in the panel). Watch how long errors last before DNS sends traffic to the secondary, and change the DNS TTL to see how it stretches or shortens that window.

Open the full playground

Anycast

With ordinary (unicast) addressing, one IP address belongs to one place. With anycast, the same IP address is announced over BGP (the Border Gateway Protocol that glues the Internet's networks together) from many locations at once, and each packet is delivered by the network to the nearest one, in routing terms.

flowchart LR
    f1([Fan in São Paulo]) --> sp["Edge São Paulo<br/>announces 198.51.100.1"]
    f2([Fan in Lisbon]) --> li["Edge Lisbon<br/>announces 198.51.100.1"]
    f3([Fan in Tokyo]) --> tk["Edge Tokyo<br/>announces 198.51.100.1"]

Anycast gives three things at once, without any DNS trick:

• Low latency: every client reaches a nearby location.
• Automatic failover: if a location stops announcing the address, routing converges to the next-nearest one, typically within seconds.
• DDoS absorption: an attack from a global botnet is split among all the locations instead of hitting one (see edge security).

The root DNS servers, public resolvers like 1.1.1.1 and 8.8.8.8, and every large CDN use anycast. Anycast works best with short, mostly stateless exchanges (DNS over UDP, HTTP requests at an edge cache); a route change in the middle of a long TCP connection can break it, which large providers mitigate with careful routing.

CDN: bring the content to the fan

A CDN is a network of caching reverse proxies — points of presence (PoPs), hundreds of them around the world — that store copies of your content close to users³. A fan in Recife who asks for the BoxOffice logo receives it from a PoP in Recife or Fortaleza, not from the origin in São Paulo; a fan in Lisbon receives it from Lisbon, not from across the Atlantic.

A CDN does three jobs at once⁵:

1. Latency: content travels a few kilometres instead of thousands. A round trip across the Atlantic costs around 100–150 ms; to a PoP in the same city, a few milliseconds.
2. Offload: every cache hit is a request the origin (your servers) never sees. This is often the main reason for a CDN in a system design.
3. Protection: the CDN's capacity and anycast network absorb traffic spikes and attacks before they reach you.

sequenceDiagram
    participant F1 as Fan 1 (Recife)
    participant E as CDN PoP Recife
    participant O as Origin (São Paulo)
    participant F2 as Fan 2 (Recife)
    F1->>E: GET /img/stadium.webp
    Note over E: MISS: not in cache
    E->>O: GET /img/stadium.webp
    O-->>E: 200, Cache-Control: max-age=86400
    E-->>F1: 200 (stored for 1 day)
    F2->>E: GET /img/stadium.webp
    Note over E: HIT
    E-->>F2: 200 from the PoP (origin untouched)

Pull versus push

Most designs use pull in front of an origin, and that origin is frequently an object store (S3, Cloud Storage, Azure Blob — the objstore component) for static files and media.

The cache key

The CDN stores each response under a cache key — by default, the host plus the URL path plus the query string. Two requests with the same key share the cached copy. Getting the key right is most of the work:

• Too specific a key fragments the cache: /events/42?utm_source=instagram and /events/42?utm_source=email are the same page, but with the default key they are cached twice and both miss. Remove tracking parameters from the key, or sort query parameters.
• Too generic a key leaks data: if the response depends on a cookie (session=…) or a header (Accept-Language) that is not in the key, one fan may receive a page personalised for another.
• The Vary response header tells caches which request headers change the response (Vary: Accept-Encoding is normal; Vary: Cookie effectively disables shared caching).

Cache-Control: the origin decides

The origin tells every cache — the browser, the CDN, any proxy in between — what it may do with a response through the Cache-Control header²:

Revalidation uses validators: the origin sends an ETag (a version identifier) or Last-Modified; the cache asks If-None-Match: "v42", and the origin answers 304 Not Modified with no body when nothing changed — cheap for both sides.

# fingerprinted JavaScript bundle: cache "forever"
Cache-Control: public, max-age=31536000, immutable

# event page HTML: short in browsers, longer at the CDN, survive origin hiccups
Cache-Control: public, max-age=60, s-maxage=300, stale-while-revalidate=60, stale-if-error=3600

# seat availability JSON during the sale: micro-cache at the CDN for one second
Cache-Control: public, max-age=0, s-maxage=1, stale-while-revalidate=1

# the fan's order and payment pages: never cache
Cache-Control: private, no-store

Invalidation: purge or version?

The event's poster image changed. How do the 300 PoPs learn about it? There are two strategies:

• Purge (invalidation): call the CDN API to remove /img/stadium.webp (or a tag, or a prefix) from every PoP. The next request misses and fetches the new version. Purges are not instantaneous — they propagate in seconds to minutes, depending on the provider — and browsers that already cached the old version with max-age will not be purged at all.
• Versioned (fingerprinted) URLs: never change a file in place. The build produces app.3f9a1c.js; a new build produces app.b71e02.js and the HTML references the new name. Old and new versions coexist, caches never serve the wrong one, and the files can be cached with immutable for a year. Only the small HTML that references them needs a short TTL.

Prefer versioned URLs for every static asset and keep purges for the exceptional cases (an error on a published page, legal take-downs). This is the edge version of the oldest joke in computing — there are only two hard things in computer science: cache invalidation and naming things — and versioned URLs solve the first by doing the second. Invalidation in general is studied in the caching class.

Origin shield and tiered caching

With 300 PoPs, a new file can cause 300 misses — one per PoP — that all hit the origin. A tiered cache (or origin shield) adds a middle layer: PoPs ask a regional cache, and only the regional cache asks the origin. The origin sees one request per object instead of one per PoP, and request collapsing at the shield merges simultaneous misses for the same object into a single origin fetch.

flowchart LR
    p1[PoP Recife] --> sh[Origin shield<br/>São Paulo]
    p2[PoP Porto Alegre] --> sh
    p3[PoP Manaus] --> sh
    sh --> o[(Origin)]

How much does the hit ratio matter?

The fraction of requests the CDN serves from cache is the hit ratio. The load on the origin is the miss fraction times the traffic, so every extra "nine" of hit ratio divides the origin load by ten. Suppose (illustrative assumption) 200 000 requests per second reach the edge during the sale:

Going from 90% to 99% hit ratio looks like a small improvement ("only nine points"), but it cuts the origin load from 20 000 to 2 000 requests per second — ten times fewer servers. When you discuss a CDN in an interview, talk about the miss rate, not the hit rate: it is the number that sizes the origin.

Try it. Load the CDN scenario and run the load profile. Lower the CDN hit ratio (or the TTL) step by step and watch the request rate and p99 latency at the app servers grow; then raise it and see how few app replicas you need. Finally, purge the cache during the run: the sudden wave of misses is a small-scale stampede on the origin.

Open the full playground

Caching dynamic content: micro-caching

"Dynamic" does not mean "uncacheable". The seat-availability JSON of the stadium show changes all the time, but nobody is harmed if it is one second old. If every PoP caches it for 1 s (s-maxage=1), the origin receives at most one request per second per PoP (or per shield, with tiered caching), no matter how many fans are polling. With 2 million fans polling every 5 seconds — 400 000 requests per second — and 50 PoPs serving Brazil, the origin sees about 50 requests per second for that endpoint. The seat map is at most a second stale; the hold itself (POST /api/holds) is never cached and is checked against the database, so a stale map can never cause a double booking.

Edge compute

Modern CDNs also run your code in the PoPs: Cloudflare Workers, AWS CloudFront Functions and Lambda@Edge, Fastly Compute, Akamai EdgeWorkers. These are small functions (milliseconds of CPU, limited memory, usually JavaScript or WebAssembly) executed on each request, before the cache or the origin. Good uses:

• Validate a waiting-room token or a JWT signature at the edge and reject invalid requests before they cross the ocean.
• Rewrite and redirect: A/B tests, language redirects, normalising the cache key (removing utm_* parameters).
• Assemble responses from cached fragments, or add security headers.
• Serve a waiting-room page entirely from the edge while the origin is protected.

Bad uses: anything that needs your database on every request (the function is far from the data, so each call crosses the network anyway), long computations, and business logic that would be better tested and versioned in a service.

Trade-offs at a glance

How it connects

flowchart LR
    TTL["DNS TTL"] --> FO["failover speed<br/>→ multi-region (class 3)"]
    ANY["anycast"] --> DDOS["DDoS absorption<br/>→ edge security"]
    GEO["GeoDNS / latency routing"] --> HOME["home region per event<br/>→ CAP & PACELC"]
    HIT["CDN hit ratio"] --> ORIGIN["origin load<br/>→ Estimation"]
    CC["Cache-Control, TTL"] --> STALE["staleness = eventual consistency<br/>→ Replication & Consistency"]
    INV["invalidation, versioned URLs"] --> CACHE["cache invalidation<br/>→ Caching"]
    EC["edge compute"] --> WR["waiting-room token check<br/>→ API gateway"]

Palette components: dns · cdn · objstore (the usual CDN origin for images and video) · cache (the next cache layer behind the CDN). Scenarios: cdn-static, multi-region, video-streaming. Continues in: Caching — patterns, Distributed Systems — CAP & PACELC, Case study: video streaming.

Key takeaways