System Architecture ยท Insper

Case Studies

Twelve interview problems, a handful of ideas

"Design Ticketmaster." Go.

Two million fans, sixty thousand seats, on sale at 10:00, 45 minutes on the clock.
What do you do in the first five minutes?

  • Ask: reserved seats or general admission? Biggest on-sale? Hold time? Own payments?
  • State targets with numbers: zero double booking, hold p99 < 300 ms, survive 2 M arrivals in 2 min.
  • Say what is out of scope: resale, refunds, dynamic pricing.
  • Only then estimate โ€” and only then draw.

The framework, with a clock

# Step Min You produce
1 Clarify requirements 5 functional, non-functional with numbers, out of scope
2 Estimate 5 queries per second (QPS), avg/peak, storage, bandwidth, memory โ†’ a decision
3 API and data model 5โ€“7 few endpoints, entities, store per entity and why
4 High-level design 10 one diagram: read path and write path
5 Deep dives 10โ€“15 two or three hard parts, alternatives, a choice
6 Bottlenecks and trade-offs 5 what breaks first, what you gave up
7 Wrap-up 2โ€“3 summary, next steps, how to measure

Every case page uses this numbering, plus: try it in the playground, follow-up questions, rubric.

Study with the playground

Predictwhat breaks first?
โ†’
Runload / spike profile
โ†’
Observeutilization, p99, errors, backlog, cost
โ†’
Change one thingreplicas, cache, queue, timeout
โ†’
Explainwith the numbers of section 2
  • Every case embeds its scenario: ticketing, flash-sale, news-feed, distributed-cache, โ€ฆ
  • Profiles: load (steady), spike (10:00), stress / breakpoint (the knee), chaos (failures).
  • Remove the cache, bypass the queue, kill a node: see why each component is there.

The simulator's numbers are illustrative; the shapes (knee, backlog, cascade) are real.

Numbers that make decisions

URL shortener: 100 M new links/month, 100 reads per link.

  • Writes: 100 M / 2.6 M s โ‰ˆ 40/s โ†’ a single database could take them.
  • Reads: ร— 100 โ‰ˆ 4 000/s, peak ~12 k/s โ†’ design for reads.
  • 6 B links in 5 years ร— 500 B โ‰ˆ 3 TB โ†’ partition by code.
  • Hot set โ‰ˆ 20% of a day's reads ร— 500 B โ‰ˆ 33 GB โ†’ fits in RAM โ†’ cache.
  • 7 base62 characters = 3.5 trillion codes โ†’ length is not the problem; uniqueness is.

Every estimate ends with "soโ€ฆ". A number that changes nothing was not worth computing.

The cases (1/2): warm-ups and medium

Case Level The core idea
URL shortener โ˜… collision-free IDs (counter ranges + base62), cache hot links, 302 for analytics
Rate limiter โ˜…โ˜… token bucket / sliding window, atomic Lua in Redis, fail-open vs. closed
Distributed cache โ˜…โ˜… consistent hashing + virtual nodes, eviction, replication, hot keys
Search autocomplete โ˜…โ˜… top-k per prefix precomputed offline, served from memory, debounced
Notification system โ˜…โ˜… queue per channel, retries + dead-letter queue (DLQ), dedupe, preferences, provider failover

Scenarios: url-shortener, rate-limiter, distributed-cache, read-heavy-cache / sharding, search-autocomplete, notification-system.

The cases (2/2): hard

Case Level The core idea
News feed โ˜…โ˜…โ˜… fan-out on write vs. read; hybrid for celebrities; timeline of IDs
Chat โ˜…โ˜…โ˜… stateful WebSocket gateways, per-conversation sequence, dedupe
Ride sharing โ˜…โ˜…โ˜… latest positions in a geo index; one driver, one ride
Video streaming โ˜…โ˜…โ˜… bytes never touch your servers; transcoding directed acyclic graph (DAG); adaptive bitrate (ABR) + content delivery network (CDN)
Payment system โ˜…โ˜…โ˜… idempotency keys, double-entry ledger, reconciliation
Ticketing (BoxOffice) โ˜…โ˜…โ˜… waiting room + Little's Law; seat holds without double booking
AI assistant โ˜…โ˜…โ˜… retrieval-augmented generation (RAG), large language model (LLM) gateway, tools with permissions, guardrails, evals

Scenarios: news-feed, chat, ride-sharing, video-streaming, payment-system, ticketing / flash-sale, ai-assistant.

Patterns that repeat

Pattern Where it shows up
Cache the hot reads shortener, feed, autocomplete, video, catalog
Slow work off the critical path (queue + workers) clicks, notifications, fan-out, transcoding, outbox
Make retries safe (idempotency, dedupe, conditional writes) payments, seats, chat, notifications, ride requests
Choose the partition key conversation, city, event, user, key ring
Protect what cannot scale (admission, rate limits, bulkheads) waiting room, rate limiter, payment provider, LLM quotas
Precompute vs. compute on read feed fan-out, top-k, renditions
Strong consistency only where it matters seats, money, driver assignment โ€” not feeds or counters

Rate limiter: atomic or wrong

  • Naive GET โ†’ check โ†’ SET from 20 gateways: both read 9, both allow โ†’ 11 > 10.
  • Fix: one Lua script per request, atomic on the shard that owns the key; server clock (TIME); time to live (TTL) on every key.
  • Fixed window lets 2ร— through at the boundary; token bucket or sliding-window counter in production.
  • Counters for 1 M clients โ‰ˆ 200 MB; a sliding log would need ~30 GB.

Per-client limits do not stop a million legitimate fans at 10:00. That needs load shedding, concurrency limits โ€” and a waiting room.

Distributed cache: memory or throughput?

1.1 TB of dataรท 64 GB per node
โ†’
17 nodesby memory
vs.
2 M ops/sรท 80 k per node
โ†’
25 nodesby throughput
โ†’
50 nodes25 primaries + 1 replica each
  • Size by both and take the larger. hash % N moves almost every key when N changes; consistent hashing moves ~1/N.
  • Each point of hit ratio is 20 k database req/s: 95% โ†’ 100 k req/s, 90% โ†’ 200 k req/s.
  • Asynchronous replication can lose an acknowledged write at failover: a cache, not the source of truth.

Scenario: distributed-cache (17 nodes saturate before memory runs out).

Search autocomplete: the cheapest request is never sent

one request per keystroke~14 k req/s
โ†’
debounce 100โ€“150 ms~4.6 k req/s
โ†’
tour announcement ร—5~23 k req/s
โ†’
browser + CDN answer half~12 k req/s at the origin
  • A trie with the top-k cached in every node: typing col walks three nodes and returns a list โ€” no subtree walk.
  • The whole index (~30 M nodes, ~6 GB) fits in one server's memory โ†’ replicate it, do not shard it.
  • Built offline from the logs into immutable snapshots, loaded with an atomic swap; trending is a small overlay.

Notifications: absorb the burst, respect the provider

presale campaign7.5 M messages
โ†’
queue per channeltransactional โ‰  bulk
โ†’
workers at a controlled ratespread over 30 min
โ†’
e-mail provider1,000/s โ†’ 2.5 M take ~42 min
  • Fired in one minute: 125 k/s sends and ~8 k app requests/s from fans who tap; spread over 30 min: ~280 req/s.
  • The provider's limit decides the duration, not the worker count; transactional e-mails need reserved capacity.
  • Idempotency key per notification, retries with backoff + jitter, a dead-letter queue (DLQ) after the last attempt.

News feed: who pays for the fan-out?

  • Push (fan-out on write): reads are one cache lookup; a post costs followers inserts.
  • Pull (fan-out on read): writes are free; every feed open does hundreds of queries.
  • Followers are heavy-tailed: ~0.3% of authors (โ‰ฅ 100 k followers) cause ~84% of push writes (simulated).
  • Hybrid: push for ordinary authors, pull celebrities at read time and merge.
  • Timelines hold IDs, trimmed, for active users; hydrate posts on read; paginate with cursors.

News feed: the queue behind a celebrity

230 posts/s ร— 20046 k inserts/s
+
one post by a 50 M account50 M inserts, push
โ†’
FIFO queue, 20 workers100 k inserts/s
โ†’
ordinary posts 500 s latebacklog clears after 926 s
push
hybrid (โ‰ฅ 100 k pulled)
pull
celebrity post
50 M inserts
0 inserts
0 inserts
steady inserts/s
46,000
7,406
0
reads per feed open
1
10 (1 + 9 celebrities)
200 โ†’ 7 M reads/s

Simulator feed-fanout on the news-feed page (section 6.1): pick the strategy, the threshold and the workers.

Chat: stateful edge, stateless core

Alice's phone
โ†’
L4 load balancerlong-lived TCP
โ†’
gateway gw-031holds the socket
โ†’
chat serviceseq 88, store, ack
โ†’
pub/subchannel gw-117
โ†’
Bob on gw-117
  • 15 M concurrent sockets โ‰ˆ 300 gateways that only hold connections and route frames.
  • Session registry (user/device โ†’ gateway) + pub/sub channel per gateway.
  • Order: sequence number per conversation from a single owner โ€” never phone timestamps.
  • Exactly-once display = at-least-once delivery + clientMsgId dedupe.
  • Offline: store first, push notification, sync from the device's last seq.
  • Presence from the connection; publish changes only, to subscribers only.

Ride sharing: two consistency levels

Eventual: positions

  • 1 M drivers every 4 s = 250 k writes/s.
  • Only the latest matters: ~100 MB in an in-memory geo index (geohash / H3).
  • Search the cell and its neighbors; expand the ring.

Strong: assignment

  • Lock the driver before the offer: SET โ€ฆ NX PX 15000.
  • Accept = conditional update WHERE status = 'MATCHING'.
  • Partial unique index: one active ride per driver.

BoxOffice tie-in: 60 000 fans leave the stadium at once โ€” a predictable hot cell.

Video: follow the bytes

  • Viewers pull ~34 PB/day, ~9 Tbit/s at peak (illustrative) โ†’ the CDN serves the bytes.
  • Uploads go straight to object storage (pre-signed, multipart, resumable).
  • Transcoding = DAG of idempotent tasks on a queue; publish low resolutions first.
  • Adaptive bitrate: ladder of renditions, ~4 s segments, the player picks each next segment.
  • Origin shield + request collapsing for viral videos; pre-position the head, pull the long tail.

"My servers handle metadata, orchestration and signatures โ€” never video bytes."

Payments: exactly-once is a design, not a feature

  • Networks only give at-least-once; the exactly-once effect comes from idempotency keys.
  • Store key + request hash + response; replay the response on retry; send the key to the payment service provider (PSP) too.
  • A timeout means UNKNOWN, not failed: ask the provider, never charge again blindly.
  • Double-entry ledger: append-only entries that sum to zero; balances are derived.
  • Webhooks are signed, late, duplicated โ†’ idempotent handlers.
  • Reconciliation against the provider's settlement files catches everything else.

The unknown outcome

CREATED
call PSP โ†’
PROCESSING
โ†’
the provider answers
SUCCEEDEDok or webhook captured
FAILEDdeclined
UNKNOWNtimeout / 5xx
UNKNOWN โ†’
ask the providerstatus query or webhook, then decide

A timeout is not a failure: the charge may have happened. Charging again blindly is how fans pay twice.

Flagship: the BoxOffice flash sale

Quantity Value Soโ€ฆ
Requests without a waiting room ~167 k/s hot seat rows cannot take it โ†’ admission control
Waiting-room joins ~17 k/s static page on the CDN, cheap status
Admission rate (10 k shoppers, 5 min) ~33 fans/s Little's Law: ฮป = L / W
Orders ~20/s one well-indexed SQL primary is enough
Payment calls in flight (2 s) ~40 size the bulkhead for 40, not 2 M
Whole inventory ~6 MB storage is not the problem; contention is

The flash sale as a funnel

2 M fansin 120 s
โ†’
CDNwaiting page, polls cached 1 s
โ†’
waiting room~17 k joins/s
โ†’ 33/s โ†’
shoppersโ‰ค 10 k at once
โ†’
holds + orders~20 orders/s
โ†’
payment provider~40 calls in flight
  • Each stage passes less than it receives: the burst is turned into a measured flow before it reaches the hot seat rows.
  • The admission rate comes from Little's Law (ฮป = L / W = 10,000 / 300 s); the provider's bulkhead from the same law (20/s ร— 2 s).
  • Scenario ticketing: the same numbers in the playground; compare the requests that reach the inventory database before and after.

Seat holds without double booking

UPDATE seat
   SET status = 'HELD', hold_id = $1, held_until = now() + interval '10 minutes'
 WHERE event_id = $2 AND seat_id = ANY($3)
   AND (status = 'AVAILABLE' OR (status = 'HELD' AND held_until < now()));
-- rows updated == seats requested, otherwise ROLLBACK โ†’ 409
  • Losers fail fast (0 rows) โ€” no lock waits, expired holds reclaimed lazily.
  • Order = state machine: HELD โ†’ PENDING_PAYMENT โ†’ CONFIRMED, idempotent transitions, outbox โ†’ Kafka.
  • General admission = one hot counter โ†’ shard it, or move it to Redis with a durable guard.

AI assistant: an LLM is one component

  • RAG: chunk โ†’ embed โ†’ vector DB; hybrid search + rerank; answers with citations.
  • LLM gateway: route small vs. large model, fallbacks, quotas, cache, cost per conversation.
  • Agent loop with a tool registry (Model Context Protocol, MCP): look up an order, start a refund.
  • Authorization outside the model; side effects need the fan's confirmation.
  • Safety mesh: direct and indirect prompt injection, PII, output validation.
  • Evals offline (test sets) and online (feedback, escalation rate); stream tokens (time to first token, TTFT).

What separates the levels

Level Typical answer
Junior Reasonable boxes and tables; caching mentioned; misses the hard part or solves it with "more servers"
Mid Requirements and estimates drive the design; queues, caches, idempotency in the right places; names the hard part
Senior Quantifies; compares alternatives with their failure behavior; designs for failure (timeouts, reconciliation, failover); states consistency choices explicitly; knows when a single PostgreSQL is still the right answer

Each case ends with its own rubric โ€” grade yourself, then repeat a week later without looking.

Pitfalls

  • Jumping to technologies before requirements and numbers.
  • Estimates that never turn into a decision.
  • Only the happy path: no "what happens when X is down?".
  • Over-engineering: five databases and three regions for 100 req/s.
  • A 40-minute monologue โ€” check in after every step.

The concept map

Pattern Cases Classes Components
Cache the hot reads URL shortener, news feed, autocomplete, video Caching, Edge cache cdn
Slow work off the critical path notifications, video, news feed, payments Messaging, Compute queue kafka worker
Safe retries, strong consistency where it matters payments, ticketing, chat, ride sharing Distributed, Databases sql payment
Choose the partition key chat, ride sharing, ticketing, distributed cache Databases nosql stream cache
Protect what cannot scale ticketing, rate limiter, AI assistant Edge, Reliability waf apigw queue llmgw

Each case page ends with a "How it connects" diagram: the classes it tests and the palette components it uses.

Takeaways

  1. Use the framework and its clock: clarify, estimate, API/data, design, deep dives, trade-offs, wrap-up.
  2. Every number must end in a decision.
  3. Twelve problems, a handful of patterns: cache, async, idempotency, partition key, admission control.
  4. Strong consistency only where it matters: seats, money, assignments.
  5. Design the failure path as carefully as the happy path.
  6. Practice: attempt first, compare, run the scenario, answer the follow-ups, grade yourself, repeat.

Outline: (1) the interview framework and timing; (2) the twelve cases and the patterns that repeat; (3) one idea from each case; (4) the BoxOffice flash sale in depth; (5) how to study with the playground and the rubric.

Let the class answer first. Most students start drawing boxes. The point: requirements change the design completely (a 300-seat theater vs. the Eras Tour).

Moved next to the framework, in the page's order: the loop is how every case is studied. Change one thing at a time, or you will not know which change moved the metric.

The 84% comes from the simulation on the news-feed page (synthetic heavy-tailed distribution). The point is the shape, not the exact number.

The numbers come from the simulator's model, which uses the page's estimates. With 10 workers the same post makes ordinary posts 1,000 s late and the backlog needs 12,500 s (3.5 h).

Ask students which pitfall they recognize in their own design docs. Most will say the first two.