System Design Interviews

The 10 a.m. question

A scene that is going to happen to you.

Tuesday, 10 a.m. The last round of the internship process you wanted most, at a company that sells tickets online. No code this time: a shared whiteboard, an engineer from the platform team and 45 minutes. After the introductions, she writes a single line on the board:

"Design a ticketing site for a stadium show. Tickets go on sale Saturday at 10:00."
That is all she says. She wants to see how you think.

There is a weak answer: grab the marker and draw, in silence, the usual boxes (a load balancer, three servers, a cache, a database) and say "it scales". And there is a strong one, which starts with questions back: "How many seats, and how many fans do we expect at 10:00? Can we ever sell the same seat twice? How long can a fan hold a seat while paying?" With the answers (say 60,000 seats and 2 million fans) the problem changes shape. Serving 2 million copies of the event page is the easy part: that is what a CDN (content delivery network) is for. The hard part is that 2 million people compete for 60,000 rows in a database in the same minute, and that not one of those rows may be sold twice.

The difference between the two answers is not having memorized more boxes. It is knowing how to connect what you learned: requirements and estimates, where the contention is, what must be strongly consistent and what can be eventually consistent, how a queue turns a spike into a line, and what breaks first. This page lays out that path. At the end, we come back to the 10 a.m. question with the full answer.

What a system-design interview looks like

Four kinds of question, one skill: reasoning about a system out loud, with numbers and trade-offs.

Open design

"Design a URL shortener"

The classic 45-minute question. There is no single answer: there are requirements you clarify, numbers you estimate, a design you justify and trade-offs you name. The interviewer steers you into one or two deep dives.

Component deep dive

"Make this rate limiter distributed"

One box of a design, in depth: the algorithm (token bucket, sliding window), where the state lives, what happens when that state is unavailable. Tests whether you know the component beyond its name.

Troubleshooting

"p99 tripled after the deploy"

A symptom and a request for a diagnosis. Tests whether you read metrics, traces and logs, and whether you know the usual suspects: saturation, a slow dependency, a cold cache, retries, a lock.

Estimation

"How many servers do we need?"

Back-of-the-envelope math: users to requests per second, peak factor, storage per year, bandwidth, Little's Law. The order of magnitude is what matters, and saying your assumptions out loud matters more.

What separates candidates

For an internship or a first job, nobody expects you to have run a system with millions of users. What sinks candidates are the basic traps: drawing before asking, designing without a single number, naming technologies without saying why ("I'd use Kafka"), ignoring what happens when something fails, and going silent. Avoid those five and you are already ahead of most.

What the interviewer is assessing

Scoping the problem Realistic estimates A sound high-level design Technical depth Trade-offs and bottlenecks Communication and driving

Dimension	Strong signal	Red flag
Requirements	Separates functional from non-functional requirements; asks about scale, consistency and latency before drawing	Starts drawing in the first minute
Estimation	Turns users into requests per second and gigabytes, with the peak, and uses the numbers to decide	Numbers that never influence a decision, or no numbers at all
High-level design	A simple design that covers every requirement, with clear data flow	Twenty boxes with no explanation of what flows between them
Depth	Goes deep where the problem is hard (for BoxOffice: seat contention)	Goes deep where it is easy (choosing a frontend framework)
Trade-offs	"X over Y because… the cost is… I'd revisit if…"	"X is the best", "it scales"
Failure	Asks what happens when each box fails; timeouts, retries, idempotency	A design that only works when everything works
Communication	Thinks out loud, checks in, drives the next step	Long silences, or waits to be told what to do

Level	What is expected	Where to practice
Intern / new grad	A correct, simple design; clear vocabulary; honest estimates; reacts well to hints	Warm-up problems and the quick reference
Mid-level	Drives the whole session; one or two solid deep dives; names failure modes without being asked	Medium problems and mock interviews
Senior	Finds the hard part fast, quantifies trade-offs, discusses operations, cost and evolution	Hard and advanced problems

How to answer

8 principles for turning what you know into a design that convinces.

1

Clarify before you draw Who uses it, for what, how many, how fast, and what must never go wrong? Two minutes of questions save twenty of redesign. A well-placed question ("can we ever sell the same seat twice?") shows more maturity than any diagram. Class 1 · Requirements →

2

Say your assumptions, with numbers "I'll assume 10 million daily active users, 20 requests each, a peak of 3× the average: about 7,000 requests per second at peak." The interviewer can correct you; silence they cannot. Round aggressively: one day is about 10⁵ seconds. Class 2 · Estimation →

3

Go broad, then deep First a design that covers every requirement end to end, even if naive; only then the deep dives. Polishing the cache before the write path exists is the most common way to run out of time.

4

Draw and narrate Boxes with names, arrows with what flows on them ("POST /holds", "OrderPlaced event"). Narrate a request as it travels through the diagram. Rehearse with the playground: same palette, same vocabulary.

5

Name the trade-off Every choice costs something. "I'll use a cache-aside Redis for the catalog: reads drop from 20 ms to 1 ms and the database sheds 95% of the load, at the cost of up to 60 s of stale prices, which is fine for a catalog but not for seat availability."

6

Find the bottleneck with numbers Which box hits its limit first? Compare the load you estimated with the capacity of each component (reference numbers). Utilization above about 70–80% is where latency explodes: that is the knee of the queueing curve. Class 2 · Queueing →

7

Design for failure For each arrow, ask: what if it is slow? what if it fails? what if the request arrives twice? Timeouts, retries with backoff and jitter, circuit breakers, idempotency keys and a dead-letter queue are the standard answers. Class 10 · Resilience patterns →

8

Drive, and check in You lead the session. Propose the next step ("the hard part is seat contention, shall I go deep there?") and check in every few minutes. If the interviewer raises a concern, take it seriously: "good point, let me revisit that".

Practical tip: make the transitions explicit. "That covers the requirements; now a quick estimate." "We have the high-level design; the risky part is the 10:00 spike, let me go deep there." The interviewer always knows where you are, and so do you.

The framework, step by step

Seven steps and a clock for a 45-minute interview. The framework page has templates for each step and BoxOffice worked through end to end.

1

Clarify the requirements · ~5 min Functional requirements (the 3–5 core features, and what is out of scope) and non-functional ones: scale, latency, availability, consistency, durability, cost. End with a short list written on the board. Class 1 →

2

Estimate · ~5 min Requests per second (average and peak), read/write ratio, storage per year, bandwidth, and the one number that decides the design (for BoxOffice: 2 M fans for 60 k seats). Class 2 · Numbers →

3

API and data model · ~5 min The main endpoints or events (with idempotency keys where money or inventory is involved) and the core entities with their keys and access patterns. The access patterns choose the database. Class 5 · API styles →

4

High-level design · ~10 min Clients → edge (DNS, CDN, load balancer, API gateway) → services → storage, with async work behind queues. Walk one read and one write through it. Keep it simple enough to fit on the board. Class 1 · Scaling journey →

5

Deep dives · ~12 min One or two components where the problem is hard: the seat hold (transactions and locks), the feed fan-out, the hot partition, the cache invalidation. This is where depth is assessed. Class 16 · Case studies →

6

Bottlenecks and trade-offs · ~5 min What breaks first at 10× the load? Where are the single points of failure? What did you trade (consistency for latency, cost for availability)? How would you observe and test it? Class 15 · Bottlenecks →

7

Wrap-up · ~3 min Summarize the design in four sentences, restate the key trade-offs, and list what you would do next with more time (multi-region, cost, observability, abuse). Leave the interviewer with a clear picture.

The two scoping questions that change everything

Always ask: "Is this read-heavy or write-heavy?" and "What must never be wrong?" A read-heavy system (catalog, feed, URL redirects) is solved with caches, replicas and a CDN. A write-heavy one (location updates, metrics, clicks) is solved with partitioning, queues and append-only storage. And the thing that must never be wrong (a seat, a balance, a payment) is where you pay for strong consistency, transactions and idempotency, while everything else can be eventually consistent.

Quick diagnosis

The symptoms that show up in troubleshooting and deep-dive questions, what they usually point to and what to do next.

In a troubleshooting question, the interviewer describes a symptom and expects a hypothesis, a way to confirm it and a fix. Before proposing a new architecture, rule out the cheap causes: a recent deploy, a saturated resource, a slow dependency, a cold cache, retries.

Symptom	Likely cause	Next step
p99 high but CPU idle	Threads or connections exhausted waiting on a slow dependency (Little's Law: concurrency = rate × time)	Trace a slow request; add a timeout and a circuit breaker; bulkhead the slow call; size the pool
p99 grows sharply as traffic rises, p50 almost flat	Utilization past the knee (~70–80%): queueing delay	Scale out or shed load before the knee; autoscale on utilization; find the saturated resource (USE method)
Latency spike right after a deploy or restart	Cold cache, cold JIT, connection pools refilling	Warm up before taking traffic (readiness probe), roll out gradually (canary), pre-warm the cache
Database CPU at 100%, mostly reads	Missing cache or index; N+1 queries	`EXPLAIN` the top queries; add an index; cache-aside; read replicas
Database melts the moment a popular key expires	Cache stampede (thundering herd)	Request coalescing / single flight, a lock per key, early probabilistic refresh, staggered TTLs
One shard or partition hot, the others idle	Hot key or a bad partition key (celebrity, flash-sale event)	Better key (add a suffix / salt), split the hot key, cache it, or give it a dedicated path
An outage gets worse when the dependency recovers	Retry storm: retries without backoff multiply the load	Exponential backoff with jitter, retry budgets, circuit breaker, idempotent endpoints
Consumer lag growing steadily	Consumers slower than producers, or fewer partitions than consumers	Scale consumers up to the partition count; add partitions; batch; find the slow handler; DLQ for poison messages
Users see stale data right after saving	Read from a lagging replica or a cache not invalidated	Read-your-writes: read from the leader after a write, or invalidate / version the cache entry
Same order charged twice	Retry after a timeout without idempotency	Idempotency key stored with the result; unique constraint; reconcile with the provider
Two fans got the same seat	Check-then-act race without a lock or constraint	`SELECT … FOR UPDATE`, a conditional update (`WHERE status = 'free'`), or a unique constraint on (event, seat)
Errors only in one availability zone or region	Zonal failure, bad host, network partition	Health checks and outlier detection; drain the zone; multi-AZ by default
Memory slowly grows until the pod restarts	Leak; unbounded cache or queue	Soak test; heap profile; bound every cache and queue
The LLM assistant is slow and expensive	Too many agent steps, no caching, large prompts	Fewer steps, a semantic cache, a smaller model for routing, streaming for time to first token (TTFT)

A complete example

"p99 is 4 s, CPU is at 15%"

Ask: since when? after a deploy? all endpoints or one? what does the service call?

Hypothesis: a downstream call got slow; every request now holds a thread for seconds, the pool of 200 threads is full and new requests wait in the queue. Little's Law: 100 req/s × 2 s = 200 requests in flight.

Confirm: a trace shows 3.8 s inside the call to the payment provider; thread-pool metrics show 200/200 busy.

Fix, cheapest first: a 500 ms timeout, a circuit breaker with a fallback ("payment pending"), a separate pool for that dependency (bulkhead); then make payment asynchronous.

Tools by symptom

What to use when

RED dashboard (rate, errors, duration): is the service healthy for its users?

USE method (utilization, saturation, errors): which resource is the bottleneck?

Distributed trace: where does one slow request spend its time?

Logs: what exactly happened to this request?

Profiler / flame graph: where does the CPU go?

EXPLAIN: why is this query slow?

Load test (k6) and fault injection (Toxiproxy): reproduce it before you fix it. Class 9 →

The key concepts

The concepts that come up most, in ten themes. Try answering out loud before opening each theme; each answer ends with the class where the topic is taught. For every palette component there is a full entry in the component catalog.

[1] Functional vs non-functional requirements?Functional: what the system does (buy a ticket, search events). Non-functional: how well (latency, availability, consistency, durability, scale, cost, security). The non-functional ones drive the architecture. Class 1
[2] Vertical vs horizontal scaling?Vertical: a bigger machine (simple, has a ceiling, single point of failure). Horizontal: more machines behind a load balancer (needs stateless services, shared state elsewhere). Start vertical, design for horizontal. Class 1
[3] How do you go from users to requests per second?Daily active users × requests per user ÷ 86,400 s (≈ 10⁵), times a peak factor (2–10×; for a flash sale, far more). 10 M DAU × 20 ÷ 10⁵ ≈ 2,000 req/s average. Class 2
[4] What is Little's Law?L = λ × W: items in the system = arrival rate × time in the system. 1,000 req/s × 0.2 s = 200 requests in flight, so you need about 200 threads or connections. Class 2
[5] Why does latency explode near 100% utilization?Queueing: in an M/M/1 queue, the time in system is S / (1 − ρ). At ρ = 0.5 it doubles the service time; at 0.9, ten times. Keep utilization below the knee (~70–80%). Class 2
[6] Why look at p99 rather than the average?The average hides the tail, and the tail is what users feel. With fan-out, the tail dominates: a page that calls 100 services, each slow 1% of the time, is slow 1 − 0.99¹⁰⁰ ≈ 63% of the time. Class 2
[7] How do availabilities combine?In series they multiply (99.9% × 99.9% ≈ 99.8%); in parallel the unavailabilities multiply (1 − 0.01² = 99.99%). 99.9% ≈ 8.8 h of downtime per year; 99.99% ≈ 53 min. Class 2

In class: 1 · What is system design · 2 · Estimation & performance fundamentals

[8] What does CAP actually say?During a network partition, a replicated system must choose between consistency (refuse or delay) and availability (answer, possibly stale). Without a partition there is no choice to make, which is why PACELC adds: else, latency vs consistency. Class 3
[9] Strong vs eventual consistency?Linearizable: every read sees the latest write, as if there were one copy. Eventual: replicas converge if writes stop. In between: read-your-writes, monotonic reads, causal. Pick per data item: seats strong, view counters eventual. Class 3
[10] What is a quorum?With N replicas, writing to W and reading from R with R + W > N guarantees the read overlaps the latest write. N = 3, W = 2, R = 2 is the usual choice. Class 3
[11] What does consensus (Raft) give you?A group of nodes agreeing on one ordered log despite failures, as long as a majority is up: leader election, replicated configuration, locks. etcd, ZooKeeper and Kafka's KRaft use it. Needs 2f + 1 nodes to tolerate f failures. Class 3
[12] Why can't you trust wall clocks?Clocks drift and NTP corrections jump; two machines can disagree by milliseconds or more. Use logical clocks (Lamport, vector) for ordering, or hybrid logical clocks; never "last write wins" on wall time for money. Class 3
[13] What is idempotency and why does it matter?Doing an operation twice has the same effect as once. Networks force retries, and retries duplicate requests; an idempotency key (stored with the result) makes "charge" and "hold seat" safe to retry. Class 3
[14] Name some fallacies of distributed computing.The network is reliable; latency is zero; bandwidth is infinite; the network is secure; topology doesn't change; there is one administrator; transport cost is zero; the network is homogeneous. Each one is a failure mode to design for. Class 3

In class: 3 · Distributed systems fundamentals

[15] What does a CDN do, and what can't it do?Serves cacheable content (images, JS, event pages) from edge locations near users, cutting latency and origin load. It cannot cache personalized or strongly consistent data, like seat availability at 10:00. Class 4
[16] L4 vs L7 load balancing?L4 routes TCP/UDP connections by address and port (fast, protocol-agnostic). L7 understands HTTP: routes by path or header, terminates TLS, retries, does canaries. Most web APIs use L7. Class 4
[17] Why consistent hashing?With hash(key) mod N, changing N moves almost every key; on a hash ring only about 1/N of the keys move. Virtual nodes even out the load. Used in caches, sharded stores and sticky load balancing. Class 4
[18] Token bucket vs sliding window?Token bucket: tokens refill at rate r up to burst b; allows bursts, cheap to store. Sliding window: counts requests in the last T seconds; smoother, a bit more state. Either way the state lives in a shared store (Redis) and the response is 429 with Retry-After. Class 4
[19] What does an API gateway do?A single entry point: routing, authentication, rate limiting, request shaping, TLS, observability. Keep business logic out of it; a BFF (backend for frontend) is a gateway tailored to one client. Class 4
[20] WebSockets, SSE or polling?WebSockets: bidirectional, long-lived (chat, games). Server-Sent Events: server-to-client stream over HTTP (live scores, LLM tokens). Long polling: the fallback. Each long-lived connection costs memory on a server and complicates load balancing. Class 4

In class: 4 · Traffic & edge · Catalog: edge components

[21] Why stateless app servers?Any replica can serve any request, so you can add, remove and replace replicas freely. Session and state go to a shared store (Redis, database) or into a token. Class 5
[22] Monolith or microservices?Start with a (modular) monolith; split a service out when a team, a scaling profile or a failure domain justifies it. Microservices buy independent deploys at the price of network calls, distributed data and operational cost. Class 5
[23] REST, gRPC or GraphQL?REST: simple, cacheable, public APIs. gRPC: binary, typed, streaming, service-to-service. GraphQL: clients choose fields, good for varied frontends, harder to cache and rate limit. Class 5
[24] When do you move work to a background worker?When the user does not need the result to continue (emails, thumbnails, reports) or when the work is slow or spiky. Respond 202 Accepted, enqueue, and let workers drain the queue at their own pace. Class 5
[25] When is serverless a good fit?Spiky or low, event-driven traffic with short tasks: you pay per invocation and scale to zero. Watch cold starts, concurrency limits, time limits and connection storms against the database. Class 5
[26] How do you integrate safely with a payment provider?Idempotency keys on every charge, timeouts and retries with backoff, webhooks for the final status (verified and deduplicated), and a daily reconciliation job against the provider's report. Class 5

In class: 5 · Compute & services · Catalog: compute, external

[27] SQL or NoSQL?Start from the access patterns. Relations, transactions and ad-hoc queries: SQL. Huge scale with simple key-based access, flexible schema or wide rows: key-value, document or wide-column. "NoSQL scales, SQL doesn't" is a myth worth avoiding. Class 6
[28] What does an index cost?Faster reads on the indexed columns; slower writes (every insert updates every index) and more storage. Index for the queries you actually run; check with EXPLAIN. Class 6
[29] B-tree vs LSM-tree?B-tree (PostgreSQL, MySQL): in-place updates, fast reads. LSM (Cassandra, RocksDB): append and compact, very fast writes, reads may check several files. Write-heavy workloads favor LSM. Class 6
[30] How do you prevent double booking?Make the database enforce it: a transaction with SELECT … FOR UPDATE on the seat, or an atomic conditional update (UPDATE seats SET status='held' WHERE id=? AND status='free'), or a unique constraint. Optimistic locking (a version column) when conflicts are rare. Class 6
[31] Isolation levels in one sentence each?Read committed: no dirty reads. Repeatable read / snapshot: a consistent snapshot, but write skew is possible. Serializable: as if transactions ran one at a time, at a cost in throughput and aborts. Class 6
[32] What does a read replica buy, and cost?More read capacity and a standby for failover. The cost is replication lag: a read right after a write may not see it. Writes do not scale with replicas. Class 6
[33] How do you choose a shard key?High cardinality, even distribution, and aligned with the main query so most queries hit one shard. Hash for even spread, range for range scans. Watch hot keys and resharding. Class 6
[34] OLTP vs OLAP?OLTP: many small transactions on current data (orders). OLAP: large scans and aggregates over history (sales per artist per month), in a columnar data warehouse fed by ETL/ELT or change data capture (CDC). Class 6

In class: 6 · Databases & data systems · Catalog: storage components

[35] Cache-aside, read-through, write-through, write-behind?Cache-aside: the app reads the cache, on a miss reads the database and fills the cache (the default). Read-through: the cache does that for you. Write-through: write to cache and database together. Write-behind: write to cache, flush later (fast, risks loss). Class 7
[36] How does hit ratio translate into database load?Database load = requests × (1 − hit ratio). Going from 90% to 99% hits cuts database reads by 10×. Average latency ≈ h × t_cache + (1 − h) × t_db. Class 7
[37] How do you invalidate?TTL (simple, bounded staleness), delete on write (cache-aside), events / CDC to evict, or versioned keys. "There are only two hard things…": decide how stale each item may be. Class 7
[38] What is a cache stampede and how do you stop it?A hot key expires and thousands of requests miss at once and hit the database. Fixes: request coalescing (single flight), a short lock per key, probabilistic early refresh, jittered TTLs, serving stale while revalidating. Class 7
[39] LRU, LFU or TTL?LRU evicts the least recently used (good default); LFU the least frequently used (keeps long-term favorites); TTL expires by age regardless of use. Modern caches mix them (e.g. W-TinyLFU). Class 7

In class: 7 · Caching · Catalog: Cache

[40] Queue, pub/sub or event log?Queue: each message to one consumer (work distribution). Pub/sub: each message to every subscriber (fan-out). Event log (Kafka): an ordered, retained log that many consumer groups read at their own offsets, and can replay. Class 8
[41] At-most-once, at-least-once, exactly-once?At-most-once can lose messages; at-least-once can duplicate them (the common default); "exactly-once" in practice means at-least-once delivery plus idempotent processing or transactions. Class 8
[42] How does Kafka scale consumers?A topic has partitions; within a consumer group each partition goes to one consumer, so parallelism is at most the number of partitions. Ordering is guaranteed only within a partition: key by order id or seat id. Class 8
[43] What is the transactional outbox?Write the business row and the event in the same database transaction (an outbox table), then a relay publishes the events. Avoids the dual-write problem where the database commits and the publish fails. Class 8
[44] Saga: choreography or orchestration?A saga splits a distributed transaction into local steps with compensations (release the seat if payment fails). Choreography: services react to events (loose, harder to follow). Orchestration: a coordinator drives the steps (explicit, one more component). Class 8
[45] What is backpressure, and a DLQ?Backpressure: slowing producers or rejecting work when consumers cannot keep up, instead of letting queues grow forever. A dead-letter queue (DLQ) parks messages that keep failing so they don't block the rest. Class 8

In class: 8 · Messaging & event-driven · Catalog: messaging components

[46] SLI, SLO, SLA and error budget?SLI: a measurement (share of requests under 300 ms). SLO: the target (99.9% over 28 days). SLA: a contract with penalties. Error budget: 100% − SLO, the unreliability you may spend on releases and experiments. Class 9
[47] Timeout, retry, circuit breaker, bulkhead?Timeout bounds the wait; retry (with backoff and jitter) handles transient errors; a circuit breaker stops calling a dependency that keeps failing; a bulkhead isolates resources so one slow dependency can't take them all. Class 10
[48] RPO vs RTO?RPO: how much data you can lose (time since the last good copy). RTO: how long until you are back. Backups give hours; replication gives seconds; multi-region active-active gets close to zero, at a high cost. Class 10
[49] Why chaos engineering?To find weaknesses in a controlled experiment rather than in an incident: state a hypothesis about steady state, inject a failure (latency, a crash, a zone outage), observe, fix. Start small, in staging, with a stop button. Class 10
[50] Metrics, logs and traces?Metrics: cheap aggregates over time (is something wrong?). Logs: detailed events (what exactly happened?). Traces: one request across services (where is the time going?). OpenTelemetry standardizes all three. Class 9
[51] RED, USE and the golden signals?RED for services: rate, errors, duration. USE for resources: utilization, saturation, errors. Google's four golden signals: latency, traffic, errors, saturation. Class 9
[52] What should page someone at 3 a.m.?Symptoms users feel, not causes: an SLO burning its error budget fast (burn-rate alerts over two windows). High CPU alone should not page anyone. Class 9

In class: 9 · Reliability · 10 · Observability · Catalog: observability components

[53] Public vs private subnets, and a NAT gateway?Only the load balancer lives in public subnets; services and databases live in private ones, with no inbound route from the internet. A NAT gateway lets private services call out (to a payment API) without being reachable. Class 12
[54] OAuth 2 / OIDC and JWT in one breath?OIDC authenticates the user and the auth server issues a signed token (often a JWT); services verify the signature and claims locally without calling the auth server. Short expiry plus refresh tokens; revocation is the hard part. Class 4
[55] What does a service mesh add?A proxy layer (sidecar or ambient) that gives every service mTLS, retries, timeouts, traffic splitting and telemetry without code changes, at the cost of latency per hop and operational complexity. Class 12
[56] Liveness vs readiness probes?Liveness: is the process stuck? (restart it). Readiness: can it take traffic now? (remove it from the load balancer). Checking a dependency in liveness causes restart storms. Class 11
[57] Rolling, blue/green or canary?Rolling replaces pods gradually; blue/green switches all traffic between two full environments (fast rollback, double cost); canary sends a small share to the new version and watches the metrics before going further. Class 11
[58] Which load test answers which question?Smoke: does it work? Load: does it meet the SLO at expected peak? Stress: where does it break? Spike: does it survive 10:00? Soak: does it degrade over hours? Breakpoint: what is the maximum? Class 15

In class: 11 · Network & security · 12 · Cloud-native · 14 · Performance

[59] Why put an LLM gateway in front of model providers?One place for routing between models, fallbacks when a provider fails, quotas and budgets per team, caching (exact and semantic), and token, cost and latency metrics. Class 13
[60] What is RAG?Retrieval-augmented generation: chunk your documents, embed them, store the vectors; at question time retrieve the most similar chunks (approximate nearest neighbors, e.g. HNSW), rerank, and put them in the prompt. Grounds the answer in your data. Class 13
[61] What is an agent loop, and why is it slow?The model decides on a tool call, the orchestrator runs it, the result goes back to the model, repeat until done. Each step is a full LLM call (often 0.5–2 s), so latency and cost grow with the number of steps; cap the steps. Class 13
[62] What is prompt injection and how do you defend?Untrusted text (a web page, an email, a review) carries instructions the model follows. Defenses: least-privilege tools, confirmation for side effects, separating instructions from data, input and output filters, never trusting model output as authorization. Class 13
[63] How do you size an LLM endpoint?Think in tokens: requests per second × tokens per request gives tokens per second; time to first token (TTFT) and tokens per second per stream set the latency; cost ≈ input tokens × price + output tokens × price. Class 13

In class: 13 · AI systems & agents · Catalog: AI & agents components

Quiz

Twenty-four interview-style questions, three per theme. Filter by theme to review only what you need.

Study plan

How to study, and a plan for the two weeks before an interview.

Out loud

Practice in pairs

System design is a spoken exam. Reading solutions feels like learning and is not. Do mock interviews with a classmate, with a timer and the rubric, and swap roles: being the interviewer teaches you what a good answer sounds like.

With numbers

Know a dozen by heart

Seconds in a day, the latency ladder (memory, SSD, same-datacenter round trip, cross-continent), what one database or one Redis can do, the nines. The quick reference and the numbers page have them.

Under load

Break it in the playground

For every design you practice, build it in the playground, run a spike or stress profile, and see which box turns red first. Then fix it and run again. Seeing the knee once is worth reading about it ten times.

Honestly

The checklist is your thermometer

Tick an item only when you can explain it to someone without notes, trade-off included. Space your reviews: a day, a week, a month. Whatever you forget between rounds is what needs attention.

Days	What to do
1–2	Fundamentals and estimation: themes 1–2 above, the quiz for both; estimate five systems from the problem list in ten minutes each
3	The framework: read the framework page; solve the URL shortener alone with a 45-minute timer, then compare with the case study
4–5	Edge, compute and data: CDN, load balancers, gateway, rate limiting; SQL vs NoSQL, indexes, transactions, replication, sharding; the rate limiter case study
6–7	Caching and messaging: patterns, stampede, Kafka, outbox, sagas, idempotency; the news feed and notification case studies
8	Reliability and observability: resilience patterns, SLOs, RED/USE; run the `cascading-failure` and `retry-storm` scenarios in the playground
9	Money and contention: the ticketing and payment case studies; explain double-booking prevention three ways
10	Real time and AI: chat, ride sharing, AI assistant
11–12	Two mock interviews per day in pairs, timed, with the rubric; the full quiz, going back only to what you got wrong
13	Behavioral: write three stories from your project (a trade-off, a failure, a measurement)
14	One last pass over the checklist, out loud; rest

To practice in the course's format: the workbook, with exercises at three levels and worked answers. For a quick review: the cheat sheet, the concept map and the study cards.

Checklist

The topics and habits of a good system-design interview. Your progress is saved in this browser.

Back to 10 a.m.

The full answer to "design a ticketing site for a stadium show".

Fans per seat

≈ 33

2 M fans for 60 k seats: most fans will not get a ticket, so fairness and a clear "sold out" matter as much as speed.

Peak at 10:00

~10⁵ req/s

Illustrative: 2 M fans refreshing every ~10–20 s in the first minutes. Mostly reads of the same event page.

Seat writes

~10³/s

Illustrative: if the 60 k seats go in the first minute with no waiting room. Small, but all on the same rows: contention, not volume. A waiting room lowers it to the rate you choose.

1

Clarify"Core features: browse the event, pick or be assigned seats, hold them for a few minutes, pay, get the ticket by email. Never sell a seat twice; the page must stay up at 10:00; fairness matters; bots are a real threat. Out of scope: resale, venue management." — topics [1], [9].

2

Estimate"2 M fans, 60 k seats, about 33 fans per seat. Reads at the peak are on the order of 10⁵ per second, almost all of the same page; seat writes are a thousand or so per second, all on the same event. The design problem is the spike and the contention, not storage." — topics [3], [4].

3

Absorb the spike at the edge"The event page and assets come from a CDN; seat-map availability is served from a cache refreshed every second or so, which is fine because the authoritative check happens at hold time. A WAF and bot protection, plus a per-user rate limit at the API gateway." — topics [15], [18], [19].

4

Turn the crowd into a line"A virtual waiting room: at 10:00 fans get a random or arrival-ordered position in a queue, and the gateway admits them at the rate the inventory service can sustain, say 2,000 per minute, with a signed admission token. The spike becomes a steady flow." — topics [24], [40], [45].

5

Make the seat the source of truth"Inventory in PostgreSQL, partitioned by event if needed. A hold is one atomic conditional update (status = 'held', hold_until = now() + 10 min WHERE status = 'free'), so two fans can never hold the same seat. An expiry job releases unpaid holds." — topics [30], [31].

6

Pay safely, confirm asynchronously"The order service calls the payment provider with an idempotency key, a timeout and a circuit breaker; the provider's webhook confirms. The order commits and writes an OrderConfirmed event to an outbox; Kafka fans it out to notification, analytics and search." — topics [13], [26], [43], [47].

7

Name the trade-offs and how you'd prove it"Strong consistency only for seats and orders; everything else eventually consistent. The waiting room trades some user patience for a system that does not fall over. I'd load-test with a spike profile and run a game day where the payment provider slows down." — topics [8], [49], [58].

Seven steps, a dozen of the concepts above, and every box justified by a number or a requirement. That is what the interviewer wanted to see: not the boxes you know, but why each one is there. Try it yourself in the playground with the flash-sale scenario, and compare with the ticketing case study.

References

To go deeper, grouped by kind.

The 10 a.m. question

What a system-design interview looks like

"Design a URL shortener"

"Make this rate limiter distributed"

"p99 tripled after the deploy"

"How many servers do we need?"

What the interviewer is assessing

How to answer

The framework, step by step

Quick diagnosis

"p99 is 4 s, CPU is at 15%"

What to use when

The key concepts

Quiz

Study plan

Practice in pairs

Know a dozen by heart

Break it in the playground

The checklist is your thermometer

Checklist

Back to 10 a.m.

References

Interview prep, in this course

Support material

Books

Free online resources