Overview

It is 09:59:58 on a Tuesday. BoxOffice has announced that tickets for a stadium show go on sale at 10:00: 60 thousand seats, about 2 million fans with the page open, many of them pressing "refresh" every second. At 10:00:00 every phone sends a request at once. Most of those requests should never reach the inventory service that holds the seats — and the ones that do must arrive at a healthy replica, in a fair order, from real fans and not from scalper bots.

Everything that happens between the fan's phone and the first BoxOffice service is the edge: name resolution, content delivery, traffic filtering, load balancing, and the API gateway. This class is about that path. It decides how fast the first byte arrives, how much load the core services see, and whether a flash sale is a success or a 10-minute outage on the evening news.

In this course, the edge means the set of components that receive, filter, route and distribute traffic before it reaches the application services. In the course simulator they form the TRAFFIC & EDGE category of the palette: dns, cdn, waf, lb, apigw and ingress¹.

Roadmap

1. This overview — the request's journey through the edge, reverse proxies, TLS termination, a latency budget, and edge security (WAF, bots, DDoS).
2. DNS & CDN — name resolution, TTLs, GeoDNS and anycast; content delivery networks, cache keys, invalidation and edge compute.
3. Load Balancing — L4 vs L7, algorithms, consistent hashing, service discovery, health checks, and Kubernetes Ingress / Gateway API.
4. API Gateway & Rate Limiting — what a gateway centralises, the BFF pattern, and the rate-limiting algorithms.
5. Identity: OAuth 2.1, OIDC & JWT — who the caller is: tokens issued by auth and validated once, at the gateway.
6. Protocols — HTTP/1.1, HTTP/2, HTTP/3 and QUIC, TLS 1.3, WebSockets, SSE and gRPC.
7. Quiz — check your understanding.

Slides

Click the slides and use ← → to move through them; F goes full screen and P opens the presenter view, with the notes. Open the slides in a new tab

The request's journey

A fan opens https://boxoffice.com/events/stadium-show and taps "Choose seats". Before any Java code runs, the request crosses several components, each one with a precise job. Press play and follow it:

sequenceDiagram
    participant F as Fan's phone
    participant D as DNS
    participant C as CDN edge
    participant W as WAF
    participant L as Load balancer
    participant G as API gateway
    participant I as Ingress
    participant S as inventory service
    F->>D: boxoffice.com? (A/AAAA)
    D-->>F: 203.0.113.10 (nearest edge, TTL 60 s)
    F->>C: GET /events/stadium-show (TLS)
    C-->>F: HTML, JS, images from the edge cache
    F->>C: GET /api/events/42/seats
    C->>W: cache miss: forward to origin
    W->>L: request passed the rules (not a bot)
    L->>G: pick a healthy gateway replica
    G->>I: token valid, under the rate limit
    I->>S: route /api/events/** to inventory
    S-->>F: seat map (JSON)

Read the diagram as a series of decisions, made as early (and as cheaply) as possible:

1. DNS decides where the client goes: which IP address, which region, which edge location.
2. The CDN (Content Delivery Network) decides whether the request needs to go further at all: if the response is cached at the edge, the journey ends here, a few milliseconds from the fan.
3. The WAF (Web Application Firewall) decides whether the request is allowed: malformed, malicious or abusive traffic is dropped here.
4. The load balancer decides which replica serves the request, and only sends it to healthy ones.
5. The API gateway decides whether the caller is allowed and within its quota: authentication, rate limits, routing to the right service.
6. The ingress (inside Kubernetes) decides which Service in the cluster receives the request, by host and path.

Each hop removes work from the next one. In a well-designed flash sale, the vast majority of requests end at the CDN (static assets, the event page, even a seat map cached for one second), many bots end at the WAF, and only a controlled, fair stream reaches inventory.

Real deployments merge some of these boxes. Cloudflare or CloudFront combine DNS, CDN, WAF and DDoS protection in one product; an AWS Application Load Balancer can terminate TLS and route by path, doing part of the gateway's job; in Kubernetes the ingress controller often is the L7 load balancer. In an interview, draw the responsibilities as separate boxes first; then say which ones a single product covers.

Reverse proxies

Almost every box on the edge is a reverse proxy: a server that receives requests on behalf of other servers, forwards them, and returns the answer as if it were its own. The client only knows the proxy's address; the backends stay hidden in a private network.

flowchart LR
    c1([Fan 1]) --> rp["Reverse proxy<br/>boxoffice.com:443"]
    c2([Fan 2]) --> rp
    c3([Bot]) -.->|blocked| rp
    rp --> a1[gateway replica 1]
    rp --> a2[gateway replica 2]
    rp --> a3[gateway replica 3]

Because every request passes through it, a reverse proxy is the natural place to put cross-cutting concerns once instead of in every service:

The opposite, a forward proxy, sits in front of clients (a corporate proxy, for example). On the edge we only care about reverse proxies.

Where TLS terminates

TLS (Transport Layer Security) encrypts the traffic between two endpoints. On the edge, there are three common choices of where the encrypted connection ends:

Modern architectures follow a zero-trust posture: terminate public TLS at the edge, and encrypt again inside the cluster — often automatically, with mutual TLS (mTLS) from a service mesh (see Network & Security).

A latency budget for the edge

Every hop adds time. Suppose BoxOffice wants the seat map API to answer in under 200 ms at p99 for fans in Brazil. Where does the time go? The figures below are illustrative assumptions for a São Paulo user and a São Paulo region; the method matters more than the numbers.

Three lessons come out of this kind of table:

• Connection setup is expensive and avoidable. DNS caching, keep-alive and TLS session resumption remove about a third of the cold cost. HTTP/3 removes even more (Protocols).
• Distance dominates. If the origin were in Virginia instead of São Paulo, each round trip would cost about 120 ms instead of 10 ms, and a cold request would pay several of them. This is why CDNs and edge locations exist.
• The cheapest request is the one that never reaches the origin. A cache hit at the edge answers in the time of one RTT to the nearest point of presence.

Edge security: WAF, bots and DDoS

On sale day, BoxOffice does not only receive fans. It receives scalper bots that try to grab the best seats in the first second, credential-stuffing scripts that test leaked passwords against /login, and possibly a DDoS (Distributed Denial of Service) attack from someone who wants the sale to fail. The edge is the first and cheapest place to stop all of them, because it sees all the traffic before it costs anything to the core.

Web Application Firewall (WAF)

A WAF inspects HTTP requests (L7) and blocks the ones that match rules. Rules come in three families:

• Managed rule sets that detect common attacks — SQL injection, cross-site scripting (XSS), path traversal, known vulnerable endpoints. The reference is the OWASP Core Rule Set (CRS), used by ModSecurity and Coraza and mirrored by the cloud WAFs².
• Reputation rules: block or challenge IP addresses from known botnets, anonymising proxies, or data-centre ranges that real fans do not use.
• Custom rules for your application: "at most 5 POST /login per IP per minute", "only allow PUT /holds with a valid waiting-room token", "block requests without a User-Agent".

WAF rules start in count (log-only) mode, are tuned against real traffic to remove false positives, and only then switched to block. A WAF that blocks real fans during the sale is worse than no WAF.

Bot mitigation

Bots are not all bad (search engine crawlers, uptime monitors), and the bad ones increasingly look like humans. Mitigation combines several signals, cheapest first:

The waiting room deserves a special mention for BoxOffice: when 2 million fans arrive for 60 thousand seats, letting all of them into the seat map at once guarantees an outage and rewards whoever is fastest — usually a bot. A waiting room at the edge hands each visitor a signed token with a position, and admits, say, 1 000 users per second to the purchase flow — the rate inventory sustains in the estimation class. The core sees a steady, sized load; the rest of the fans see a progress bar served from the CDN. You will model it with a queue in the messaging class and the ticketing case study.

DDoS

A denial-of-service attack tries to exhaust a resource — bandwidth, connections, CPU — so that legitimate users cannot be served. "Distributed" means it comes from many sources at once (a botnet), so blocking a single IP does not help³.

The key idea is asymmetry: you cannot out-provision an attacker with your own servers, but a large CDN/DDoS provider has more edge capacity than any single attack. With anycast, the same IP address is announced from every edge location, so traffic from each bot goes to the nearest location and the flood is divided among all of them (DNS & CDN). Volumetric attacks are therefore a problem you buy a solution for (AWS Shield, Cloud Armor, Azure DDoS Protection, Cloudflare)⁴; L7 attacks are a problem you also have to design for — expensive endpoints must be cacheable, rate-limited or protected by a challenge.

Try it. Load the flash-sale scenario and press Run. The design as drawn has no CDN, gateway or waiting room — only users → lb → app → inventory-db, with buyers also calling the payment provider (500 req/s quota). When the sale opens (t = 60 s) the traffic jumps 20×: app saturates and errors and p99 explode; raise its replicas and the bottleneck only moves to inventory-db and the payment provider. Then open it in the full playground (↗) and follow the hints — a CDN in front, a cache for the catalog reads, and an API gateway that sends buyers to a waiting-room queue drained by order workers — or press Show a possible solution. Compare p99 latency, error rate and served req/s, and read the consumer lag: it is how long a buyer waits in the line.

Open the full playground

The edge concept map

Each edge component answers one question and hands the request to the next. The map links each palette component (green) to the concepts it relies on (blue) and to the subpage or class that teaches them:

flowchart LR
    client["client · mobile"]:::comp --> dns["dns"]:::comp
    dns --> cdn["cdn"]:::comp
    cdn --> waf["waf"]:::comp
    waf --> lb["lb"]:::comp
    lb --> apigw["apigw"]:::comp
    apigw --> ingress["ingress"]:::comp
    ingress --> app["app services"]:::comp
    dns -.- c1["TTL, GeoDNS, anycast,<br/>DNS failover"]:::tech
    cdn -.- c2["cache keys, Cache-Control,<br/>hit ratio, micro-caching"]:::tech
    waf -.- c3["OWASP rules, bots,<br/>waiting room, DDoS"]:::tech
    lb -.- c4["L4/L7, algorithms,<br/>consistent hashing, health checks"]:::tech
    apigw -.- c5["auth offload, rate limiting,<br/>BFF"]:::tech
    ingress -.- c6["Ingress, Gateway API,<br/>service discovery"]:::tech
    c1 -.- d1["eventual consistency<br/>(class 3)"]
    c2 -.- d2["caching patterns<br/>(class 7)"]
    c4 -.- d3["stateless services,<br/>autoscaling (class 5)"]
    c5 -.- d4["identity, JWT<br/>(Identity, this class)"]
    classDef comp fill:#1e9e5a22,stroke:#1e9e5a
    classDef tech fill:#2f6fed22,stroke:#2f6fed

Key takeaways

What breaks next

The edge decides which replica serves each request, and the gateway checks the caller's token once for every service — but both assume something they cannot enforce: that any replica can serve any request. A replica that keeps the fan's session or a seat hold in its own memory loses it when it dies, or when the next request lands on another replica. Lab 3 shows the other limit: three catalog replicas behind Nginx multiply the cheap requests, but not the mixed workload, where PostgreSQL is the bottleneck (in a test run, db at ~670% CPU and each replica at ~20%). What a replica must be (stateless), how services are cut, and the gateway that lab 4 builds with auth behind this edge are class 5 — Compute & Services.

Related

• Hands-on 3 — Edge: Nginx reverse proxy and load balancer over 3 replicas, health checks and rate limiting, measured with k6.
• Hands-on 4 — Gateway & identity: Spring Cloud Gateway with JWT validation.
• Exercise — Playground design: solve playground challenges with SLO and cost goals.
• Case studies: Rate limiter and Ticketing (BoxOffice flash sale).
• System Design Playground and the component catalog — Traffic & Edge.
• Interview preparation.
• Previous class: Distributed Systems. Next class: Compute & Services. The edge comes back in Caching, Reliability and Network & Security.