Traffic & Edge

The edge is everything between the client and your application code: it finds your servers (DNS), serves what can be served from close to the user (CDN), spreads the traffic over many replicas (load balancer), blocks what should never reach you (WAF), and applies the cross-cutting rules of your API — authentication, rate limits, routing (API gateway, Ingress). A request that is answered or rejected at the edge is the cheapest request you will ever serve.

In BoxOffice, the edge is what keeps the 10:00 flash sale alive: the event page and images come from the CDN, bots are filtered by the WAF, the API gateway limits each user to a few purchase attempts per minute, and the load balancer spreads the survivors over the inventory and order replicas.

flowchart LR
    C[Client] -->|1 resolve name| DNS
    C -->|2 static & media| CDN
    CDN -->|3 dynamic API calls| WAF
    WAF --> GW[API Gateway]
    GW --> LB[Load Balancer / Ingress]
    LB --> A1[App Server]
    LB --> A2[App Server]

DNS

What it is. The Domain Name System is the internet's phone book: it turns api.boxoffice.example into an IP address. Resolution goes from the client's stub resolver to a recursive resolver (your ISP, 8.8.8.8, 1.1.1.1), which asks the root, the top-level domain (.example) and finally your authoritative name servers. Answers are cached for the record's TTL (time to live). Managed DNS services add traffic steering: different answers by the user's location (GeoDNS), by weight (canary) or by health (failover).

When to use.

• Always — every public system has DNS. The design question is which routing policy to use.
• Failover between regions: health-checked records answer with the secondary region when the primary is down (scenario multi-region).
• Latency/geo routing: send Brazilian users to sa-east-1 and European users to eu-west-1.
• Weighted routing for migrations and canaries at the coarsest level (10% of the users to the new stack).

When not to (or watch out).

• DNS is not a fast load balancer: clients and resolvers cache answers and some ignore the TTL, so changes take seconds to many minutes to propagate.
• Do not rely on DNS round robin for health: a dead IP may keep receiving traffic until caches expire. Put a load balancer behind DNS.
• Very low TTLs (a few seconds) increase query volume and latency for little gain.

Key numbers.

Failure modes.

• DNS provider outage — nobody can resolve your name even though the servers are fine → a second DNS provider, long TTLs on stable records.
• Slow failover — cached answers point to the dead region → shorter TTL on failover records, a load balancer with health checks behind each name.
• Misconfiguration — a wrong record or an expired domain takes the whole site down → infrastructure as code, review, monitoring of the resolution from outside.
• DDoS on the name servers → anycast DNS provider with DDoS protection.

In the playground. dns is a split node: it distributes requests among its targets. Its routing mode can be failover (primary/secondary: all traffic goes to the primary while it is healthy) or weighted (proportional split). Failure detection (≈ 30 s, ≈ 3 s with a health check) plus the answers cached by clients for ttlS (default 60 s) make failover take time — watch the error spike during a region outage. Scenarios: multi-region, cdn-static.

Real products.

Learn more: Edge › DNS & CDN · Reliability › Disaster recovery · Network & security › Cloud network

CDN

What it is. A Content Delivery Network is a fleet of caching servers (edges, points of presence) spread around the world. A user fetches event-hero.jpg from an edge a few milliseconds away; only on a miss does the edge go to your origin (object store or app). Modern CDNs also terminate TLS, absorb DDoS attacks, run small functions at the edge (edge compute) and cache API responses when told to.

When to use.

• Static assets (JavaScript, CSS, fonts) and media (images, video segments): almost always.
• Read-heavy public content that is the same for everybody: BoxOffice event pages during the flash sale can be cached for a few seconds — 2 M fans reading the same page become a handful of origin requests.
• Video streaming: segments of popular titles are served almost entirely from the edges (scenario video-streaming).
• As a shield: the CDN absorbs traffic spikes and attacks before they reach the origin.

When not to (or watch out).

• Personalized or sensitive responses (cart, order history) must not be cached in a shared cache: use Cache-Control: private or no-store.
• Strongly consistent data (seat availability at the moment of purchase) cannot come from a cache that may be seconds old.
• The cache key matters: query strings, cookies or headers in the key can explode the number of variants and kill the hit ratio.
• Invalidation is slow and sometimes paid: prefer versioned file names (app.3f9a1c.js) over purges.

Key numbers.

Failure modes.

• Cold cache / purge storm — after a deploy or a full purge, every request goes to the origin → stagger purges, use versioned URLs, origin shield (a mid-tier cache).
• Cache stampede on a popular object expiring → request coalescing at the edge, stale-while-revalidate.
• Caching private data — one user sees another user's page → correct Cache-Control, never cache responses that set cookies.
• Stale content — a cancelled event still shows "on sale" → short TTL for dynamic pages, purge on change.
• CDN provider outage → multi-CDN with DNS steering for the most critical properties.

In the playground. cdn is a cache node: reads hit with probability hitRatio and return from the edge with a short latency (≈ 15 ms median); misses and writes are forwarded to the origin. Events such as a cache flush drop the hit ratio to 0 and it warms up over cacheWarmS (default 60 s) — watch the origin load jump. Cost includes egress per GB (respKB, default 50 KB per response, at ≈ 0.03 USD/GB). Scenarios: cdn-static, video-streaming, flash-sale.

Real products.

Learn more: Edge › DNS & CDN · Caching › Patterns · Case study: video streaming · Case study: ticketing

Load Balancer

What it is. A load balancer receives connections on one address and spreads them across a pool of healthy backends. A layer-4 (L4) balancer works with TCP/UDP connections and does not look inside them — very fast, protocol agnostic. A layer-7 (L7) balancer understands HTTP: it can route by path or header, terminate TLS, retry, and spread individual requests of a long-lived connection. Health checks remove broken replicas from the pool.

When to use.

• As soon as there is more than one replica of anything: it is what makes horizontal scaling and zero-downtime deploys possible.
• To hide failures: a crashed replica is removed after a few failed health checks.
• L7 when you need path-based routing (/api/catalog → catalog, /api/orders → order), TLS termination, HTTP/2 or gRPC balancing.
• L4 for raw TCP (databases, MQTT), extreme throughput or when TLS must reach the backend untouched.

When not to (or watch out).

• The load balancer itself must be redundant (managed LBs are; a single Nginx VM is a single point of failure).
• Sticky sessions pin users to replicas and break even distribution and failover; prefer stateless app servers with session data in a cache.
• Round robin ignores load: for requests with very different costs, use least-connections or least-outstanding-requests.
• L4 balancing of long-lived connections (gRPC, WebSockets) can leave new replicas idle — balance per request (L7) or reconnect periodically.

Key numbers.

Failure modes.

• Routing to a dead replica until health checks notice → short intervals, passive health checks (outlier detection), retries of idempotent requests on another replica.
• Uneven load — one replica gets the slow requests → least-outstanding-requests, consistent hashing only when needed.
• Health check that lies — /health returns 200 while the database is unreachable (or returns 500 because a non-critical dependency is down, emptying the pool) → separate liveness and readiness, check only what the instance needs to serve.
• Connection draining missing — deploys cut in-flight requests → deregistration delay, graceful shutdown.

In the playground. lb is a split node: requests are distributed among healthy targets, proportionally to capacity (round robin weighted by capacity). Without a healthcheck node, a crashed replica is detected after ≈ 30 s; with one, after ≈ 3 s — run a chaos profile and compare the error spikes. Scenarios: scale-out, monolith, db-replicas.

Real products.

Learn more: Edge › Load balancing · Compute › Services · Intro › Scaling journey · Hands-on 3

WAF

What it is. A Web Application Firewall inspects HTTP requests and blocks the ones that match attack patterns: SQL injection, cross-site scripting (XSS), path traversal, known exploits, and abusive behavior (too many requests per IP, suspicious user agents, bots). It usually runs at the edge — attached to the CDN, the load balancer or the API gateway — with managed rule sets maintained by the vendor plus your own rules.

When to use.

• In front of every public HTTP endpoint, especially login, search and checkout.
• Against bots and scalpers in BoxOffice: rate rules per IP/fingerprint, CAPTCHA or JavaScript challenges on the purchase flow, geo rules.
• For a quick "virtual patch" when a vulnerability is disclosed and the code fix takes days.
• As part of DDoS protection at layer 7 (HTTP floods).

When not to (or watch out).

• It is a safety net, not a substitute for secure code: parameterized queries, output encoding and authorization checks still belong in the application.
• False positives block real customers (a band name with an apostrophe looks like SQL injection) → start rules in "count/log" mode, then enforce.
• Rate limits by IP punish users behind carrier-grade NAT (a whole stadium on a few IPs) → combine with user/session identity at the API gateway.

Key numbers.

Failure modes.

• False positives — legitimate purchases blocked → monitoring of blocked requests, exceptions per path, staged rollout of rules.
• Fail-open vs fail-closed — if the WAF fails, do you let everything through or block everything? Decide per endpoint.
• Bypass — attackers hit the origin directly → allow traffic to the origin only from the CDN/WAF IP ranges or through a private connection.
• Rule update breaks traffic → version rules as code, test before enabling.

In the playground. waf is a call node in the request path with a small latency (≈ 1 ms) and a capacity like any other node; it stops a share blockRate (default 0.95) of the clients' bot traffic (botShare) and costs money per request. Place it between the client/CDN and the gateway to see its effect on latency and cost. Chaos events such as viral spikes and flash crowds show why bot filtering at the edge protects the backend. Scenarios: flash-sale, rate-limiter.

Real products.

Learn more: Edge (overview) · Network & security › Zero trust · Case study: ticketing

API Gateway

What it is. The single front door of your API. Clients call one host; the gateway routes each request to the right service and applies cross-cutting concerns in one place: authentication (validating JWTs), rate limiting and quotas, request/response transformation, caching, CORS, API keys, observability. A BFF (Backend for Frontend) is a gateway tailored to one client type, aggregating several service calls into one response for a mobile screen.

When to use.

• With several services behind one public API (BoxOffice: catalog, inventory, order, search, assistant).
• To protect the backend: per-user and per-endpoint rate limits (token bucket, sliding window) answering 429 Too Many Requests with Retry-After.
• To centralize token validation so each service does not reimplement it.
• To publish APIs to partners (API keys, usage plans, developer portal).

When not to (or watch out).

• Do not put business logic in the gateway; it becomes a distributed monolith that every team must change.
• It is in the path of every request: it must be redundant, autoscaled and fast.
• Rate limiting across many gateway replicas needs shared state (Redis) or approximate local limits.
• For a single service, a load balancer may be enough.

Key numbers.

Failure modes.

• Gateway overload — it becomes the bottleneck → autoscaling, keep plugins light, shed load early.
• Rate-limiter store down → choose fail-open (availability) or fail-closed (protection) per route; fall back to local in-memory limits.
• Timeouts not set — a slow service holds gateway connections and the gateway dies with it → per-route timeouts, circuit breakers.
• Retries at every layer (client, gateway, service) multiply the load → retry in one layer only, with backoff and budget.

In the playground. apigw is a call node (and a split node when it has two or more targets of the same type). Its rateLimit (default 0 = off) caps the accepted rate: the excess is rejected with errors (429) instead of overloading the backend. Edge parameters on its outgoing calls (timeoutMs, retries, backoff, circuitBreaker) model the resilience policies. Scenarios: rate-limiter, retry-storm, flash-sale.

Real products.

Learn more: Edge › API gateway · Case study: rate limiter · Network & security › Identity · Hands-on 4

Ingress

What it is. In Kubernetes, an Ingress is a resource that declares how external HTTP(S) traffic reaches the cluster's Services (boxoffice.example/api/catalog → Service catalog). The rules are executed by an Ingress controller (Nginx, Traefik, a cloud load balancer). The newer Gateway API (Gateway, HTTPRoute, …) is the successor: more expressive (header matching, traffic splitting for canaries) and with roles split between platform and application teams.

When to use.

• Whenever services run on Kubernetes and must be reachable from outside over HTTP(S).
• For path/host routing, TLS termination with automatically renewed certificates (cert-manager), and canary traffic splits (Gateway API).
• To keep routing declarative and versioned with the application manifests (GitOps).

When not to (or watch out).

• Ingress handles HTTP(S); raw TCP/UDP needs a Service of type LoadBalancer or Gateway API routes for TCP/UDP.
• The original Ingress API is limited; advanced behavior lives in controller-specific annotations that do not port between controllers. Prefer the Gateway API for new designs.
• The controller pods are in the path of every request: run several replicas and give them resources.

Key numbers.

Failure modes.

• Controller not redundant or under-provisioned → several replicas, autoscaling, pod disruption budgets.
• Bad rule shadows another route → review, tests of the routing table, Gateway API's clearer precedence.
• Expired certificate → cert-manager with alerts on expiry.
• Readiness not configured — traffic reaches pods that are still starting → readiness probes on the backends.

In the playground. ingress is a split node like the load balancer: it spreads requests among healthy targets in proportion to their capacity. Use it to model a Kubernetes cluster's entry point in front of app replicas with autoscale enabled. Scenarios: scale-out, observability-stack.

Real products.

Learn more: Edge › API gateway · Cloud-native › Kubernetes · Cloud-native › Delivery · Hands-on 10