Quick reference
One page to review before an interview: the numbers, the formulas, one line and one trade-off for each component of the palette, the default moves for each shape of problem, and the phrases that help or hurt. Everything here is explained in the classes; follow the links when a line is not obvious to you anymore. For the full tables and sources, see the numbers cheat sheet and the component catalog.
Numbers to know
Latency ladder
Classic values popularized by Jeff Dean and Peter Norvig, rounded; modern hardware is faster on some rows (NVMe SSDs, newer CPUs), but the ratios are what you reason with.
| Operation | ā Time | Think of it as |
| L1 cache reference | 0.5ā1 ns | free |
| Main memory reference | 100 ns | 200Ć L1 |
| Compress 1 KB (fast codec) | ā 3 Āµs | |
| Send 1 KB over a 1 Gbps network | 10 Āµs | |
| Random 4 KB read from SSD | ā 20ā150 Āµs | NVMe at the low end |
| Read 1 MB sequentially from memory | ā 50ā250 Āµs | |
| Round trip within the same datacenter | ā 0.5 ms | 500 Āµs |
| Read 1 MB sequentially from SSD | ā 1 ms | |
| Disk (HDD) seek | ā 10 ms | avoid on hot paths |
| Round trip between AZs of a region | ā 1ā2 ms | ā illustrative |
| Round trip SĆ£o Paulo ā US East | ā 100ā150 ms | ā illustrative; light in fiber ā 200 km/ms |
| Round trip across the globe | ā 150ā300 ms | |
Typical service-level numbers (ā illustrative): Redis GET ā 0.2ā1 ms over the network; indexed SQL query ā 1ā10 ms; app server request ā 10ā50 ms; LLM answer ā 1ā10 s (time to first token 0.2ā2 s).
Rate conversions
| Per day | Per second (average) | Remember |
| 1 million | ā 12 | 1 day ā 86,400 s ā 10āµ s |
| 10 million | ā 115 | |
| 100 million | ā 1,150 | |
| 1 billion | ā 11,500 | |
Peak = average Ć peak factor (2ā5Ć for diurnal traffic; 10ā100Ć or more for a flash sale or a viral event). A month ā 2.6 Ć 10ā¶ s; a year ā 3.2 Ć 10ā· s.
Sizes
| Thing | ā Size |
| 2Ā¹ā° / 2Ā²ā° / 2Ā³ā° / 2ā“ā° | ā 10Ā³ (KB) / 10ā¶ (MB) / 10ā¹ (GB) / 10Ā¹Ā² (TB) |
| ASCII character / UTF-8 character | 1 B / 1ā4 B |
| int64, timestamp | 8 B |
| UUID | 16 B binary, 36 B as text |
| A short post or order row with metadata | ā 0.3ā1 KB |
| A compressed photo | ā 100 KB ā 2 MB |
| 1 minute of 1080p video at ~5 Mbps | ā 40 MB |
| An embedding of 1,536 float32 | ā 6 KB |
Availability (the nines)
| Availability | Downtime per year | per 30-day month | per day |
| 99% | ā 3.65 days | ā 7.2 h | ā 14.4 min |
| 99.9% | ā 8.76 h | ā 43.2 min | ā 1.44 min |
| 99.95% | ā 4.38 h | ā 21.6 min | ā 43 s |
| 99.99% | ā 52.6 min | ā 4.3 min | ā 8.6 s |
| 99.999% | ā 5.3 min | ā 26 s | ā 0.86 s |
| Formula | Use it toā¦ | Example |
| Little's Law L = Ī» Ć W | Size thread pools, connections, queues | 1,000 req/s Ć 0.2 s = 200 in flight |
| Utilization Ļ = Ī» / Ī¼ | Check headroom; keep Ļ below the knee (ā 0.7ā0.8) | 700 req/s on capacity 1,000 ā 0.7 |
| M/M/1 time in system W = S / (1 ā Ļ) | See why latency explodes near saturation | S = 10 ms: Ļ 0.5 ā 20 ms; Ļ 0.9 ā 100 ms |
| Servers needed = peak QPS / (QPS per server Ć target Ļ) | Capacity planning | 7,000 / (500 Ć 0.7) ā 20 servers |
| Series availability A = Aā Ć Aā Ć ā¦ | Synchronous chain of dependencies | 0.999Ā³ ā 99.7% |
| Parallel availability A = 1 ā (1 ā Aā)(1 ā Aā)ā¦ | Redundant replicas | two at 99% ā 99.99% |
| Cache: database load = Ī» Ć (1 ā h) | Value of the hit ratio | 10,000 req/s, h = 0.95 ā 500 req/s |
| Cache: average latency ā h Ć t_cache + (1 ā h) Ć t_db | Expected latency with a cache | 0.95 Ć 1 + 0.05 Ć 20 = 1.95 ms |
| Fan-out tail P(slow) = 1 ā (1 ā p)āæ | Why p99 matters with fan-out | p = 1%, n = 100 ā 63% |
| Quorum R + W > N | Read sees the latest write | N = 3, W = 2, R = 2 |
| Consensus fault tolerance n = 2f + 1 | Nodes needed to survive f failures | 5 nodes tolerate 2 |
| Amdahl speedup = 1 / ((1 ā p) + p / n) | Limit of parallelization | p = 0.9, n = 10 ā 5.3Ć |
| Error budget = (1 ā SLO) Ć window | How much unreliability you may spend | 99.9% over 28 days ā 40 min |
| Burn rate = error rate / (1 ā SLO) | Alert on fast budget consumption | 1.44% errors on 99.9% ā 14.4 |
Classes: Estimation āŗ Numbers Ā· Queueing Ā· Percentiles Ā· Availability math.
Components in one line
The trade-off column is what turns a component name into an argument.
Client
| Id | Component | One line | Trade-off to mention |
client | Client | Browser or program that sends requests | Retries and timeouts in the client shape your load; never trust client input |
mobile | Mobile | App on flaky networks, offline, slow to update | Old app versions live for months: version the API, keep it backward compatible |
Traffic & Edge
| Id | Component | One line | Trade-off to mention |
dns | DNS | Names to IPs; geo, weighted and failover routing | Cached answers: low TTL = agility but more queries; never fast failover |
cdn | CDN | Caches content near users | Huge offload vs staleness; only for shareable, cacheable content |
lb | Load Balancer | Spreads requests over healthy replicas | L7 features vs L4 speed; must itself be redundant |
waf | WAF | Filters malicious traffic and bots | Blocks attacks vs false positives on real fans |
apigw | API Gateway | Auth, rate limiting, routing at the front door | Central policy vs an extra hop and a shared dependency |
ingress | Ingress | Routes external HTTP into Kubernetes | Simple config vs fewer features than a full gateway (Gateway API) |
Compute
| Id | Component | One line | Trade-off to mention |
app | App Server | Stateless replicas running business logic | Horizontal scale requires state elsewhere |
worker | Worker | Drains queues in the background | Fast responses vs eventual completion and retries to handle |
serverless | Serverless | Functions scaled per request | No servers, scale to zero vs cold starts, limits, DB connection storms |
auth | Auth Service | Authenticates users, issues tokens | Stateless JWT validation vs hard revocation |
search | Search | Full-text search on an inverted index | Rich queries vs an eventually consistent copy of the data |
scheduler | Scheduler | Runs jobs on a schedule | Must run once: leader election or locks, idempotent jobs |
notifications | Notifications | Email/push/SMS through providers, async | Reliability via queues and retries vs duplicates (dedupe) |
analytics | Analytics | Collects and aggregates events | Off the request path; freshness vs cost |
Storage
| Id | Component | One line | Trade-off to mention |
sql | SQL Database | Relational data with transactions | Strong consistency and joins vs harder horizontal write scaling |
nosql | NoSQL DB | Partitioned key-value/document/wide-column | Scale and availability vs query flexibility; design around access patterns |
cache | Cache | In-memory hot data | Latency and DB offload vs staleness, invalidation, stampedes |
objstore | Object Store | Cheap, durable files and media | Very durable and cheap vs higher latency, no in-place updates |
dwh | Data Warehouse | Columnar analytics over history | Fast scans and aggregates vs not for transactions; data is minutesāhours behind |
vectordb | Vector DB | Nearest-neighbor search on embeddings | Semantic recall vs approximate results and index memory |
Messaging
| Id | Component | One line | Trade-off to mention |
queue | Message Queue | Each message to one consumer | Absorbs spikes vs async latency and at-least-once duplicates |
pubsub | Pub/Sub | Each message to every subscriber | Decoupled fan-out vs no replay (usually) and harder tracing |
stream | Event Stream | Ordered, retained, partitioned log | Replay and ordering per key vs partition planning and consumer lag |
kafka | Kafka | The reference event-streaming platform | Throughput and retention vs operational weight; parallelism ā¤ partitions |
Observability
| Id | Component | One line | Trade-off to mention |
metrics | Metrics | Time series: RED, USE, SLIs | Cheap aggregates vs cardinality limits; no per-request detail |
logs | Logs | Structured event records | Detail vs volume and cost; never log secrets |
tracing | Tracing | One request across services | Finds where time goes vs sampling may miss rare cases |
alerting | Alerting | Pages humans on symptoms | Fast detection vs alert fatigue; page on SLO burn |
healthcheck | Health Check | Liveness/readiness probes | Fast removal of bad replicas vs restart storms if liveness checks dependencies |
Network
| Id | Component | One line | Trade-off to mention |
vpc | VPC | Private network in a region | Isolation by default; plan non-overlapping ranges |
subnet | Subnet | Public or private slice in one AZ | Spread across AZs for availability vs cross-AZ traffic cost |
nat | NAT Gateway | Outbound internet for private services | Keeps services private vs per-GB cost and port limits |
vpn | VPN | Encrypted tunnel between networks | Private connectivity vs internet latency and limited bandwidth |
mesh | Service Mesh | mTLS, retries, telemetry between services | Uniform policy without code vs latency per hop and a platform to run |
AI & Agents
| Id | Component | One line | Trade-off to mention |
llmgw | LLM Gateway | One door to model providers | Fallbacks, budgets, caching vs another hop; semantic cache can be wrong |
orchestrator | Orchestrator | Runs the agent loop | Flexibility vs latency and cost Ć number of steps |
toolreg | Tool Registry | Tools the model may call (MCP) | Reuse and governance vs prompt bloat; authorize inside tools |
memory | Memory Fabric | Retrieval (RAG) and memory | Grounded answers vs retrieval misses and stale indexes |
safety | Safety Mesh | Guardrails on inputs, outputs, actions | Risk reduction vs latency and false positives; not a security boundary alone |
External
| Id | Component | One line | Trade-off to mention |
thirdparty | 3rd Party API | Someone else's API | Buy vs build; contain it with timeouts, breakers, queues |
payment | Payment | Payment provider | Idempotency keys, webhooks and reconciliation; timeout ā failure |
email | Email | Email delivery provider | Always async behind a queue; deliverability rules and quotas |
Default moves
| If the problem isā¦ | The standard move | Watch out for |
| Read-heavy (catalog, feed, redirects) | CDN + cache-aside + read replicas | Staleness, invalidation, stampede |
| Write-heavy (locations, clicks, metrics) | Partition by key, queue/stream, LSM/NoSQL, batch | Hot keys, ordering, consumer lag |
| Spiky (flash sale, launch) | Queue / waiting room + autoscaling + load shedding | Autoscaling is too slow for a cliff: pre-scale |
| Slow work in a request | 202 Accepted + queue + workers | Status endpoint, retries, DLQ |
| Money or inventory | Strong consistency, transactions, idempotency keys, reconciliation | Never treat a timeout as a failure |
| Global users | CDN, GeoDNS, multi-region reads | Cross-region writes: latency vs consistency |
| One celebrity / hot key | Split or replicate the key, cache it, pull for celebrities | Fan-out on write for millions of followers |
| Real time to clients | WebSockets/SSE + pub/sub between servers | Connection state, reconnection storms |
| Search / autocomplete | Inverted index or trie, fed asynchronously, cached | Index lag, ranking |
| Slow or flaky dependency | Timeout + retry with backoff and jitter + circuit breaker + bulkhead + fallback | Retry storms, non-idempotent retries |
| Unknown bottleneck | Measure: RED/USE, traces, load test to the knee | Optimizing without data |
| LLM feature | Gateway + RAG + few agent steps + guardrails + semantic cache | Cost per request, prompt injection |
Consistency and CAP in one table
| Data | Choice | Why |
| Seat hold, balance, order status | Strong (CP), transactions | An invariant users and money depend on |
| Event description, prices on the page | Eventual, cached secondsāminutes | Staleness is harmless; reads dominate |
| View counts, likes | Eventual, approximate counters | Exactness is not worth the coordination |
| Shopping cart (Dynamo classic) | Available (AP), merge on conflict | Better to accept the item and reconcile |
| User's own profile after editing | Read-your-writes | They must see their own change |
| Chat messages in a conversation | Ordered per conversation | Global order is unnecessary |
CAP: during a partition, choose consistency or availability. PACELC: else, choose latency or consistency. Most systems choose per data item, not once. Class 3 Ā· CAP & PACELC
SQL or NoSQL?
| Choose SQL whenā¦ | Choose NoSQL whenā¦ |
| You need multi-row transactions and constraints | Access is by key and must scale writes horizontally |
| Queries are varied or ad hoc; joins matter | The schema varies per item or rows are very wide |
| Data fits a few nodes (vertical scale + replicas goes far) | Data or write volume exceeds one primary |
| Correctness beats raw throughput (orders, seats) | Availability across regions beats strong consistency |
| You are not sure yet (the safer default) | You know the access patterns well and they are few |
NewSQL (Spanner, CockroachDB) offers SQL and transactions with horizontal scale, at a cost in latency and money. Class 6 Ā· Data models
Sync vs async, push vs pull
| Synchronous call | Asynchronous (queue / event) |
| User waits for | the result | an acknowledgement |
| Coupling | caller fails when callee fails | callee can be down; work waits |
| Latency | sum of the chain | fast response, eventual completion |
| Use for | reads, decisions needed now (hold a seat) | emails, reports, fan-out, slow or spiky work |
| Push (fan-out on write) | Pull (fan-out on read) |
| Work happens | when the item is created | when the reader asks |
| Good for | many readers, few writers per item; fast reads | celebrities with millions of followers; rarely read items |
| Cost | write amplification | slower, heavier reads |
| Usual answer | hybrid: push for most, pull for celebrities | |
Phrases that score and phrases that hurt
| Phrases that score | Phrases that hurt |
| "Before I draw, can I ask about scale and what must never be wrong?" | (Silence, then twenty boxes) |
| "I'll assume 10 M DAU; that's about 2,000 req/s on average, 7,000 at peak." | "It needs to handle a lot of traffic." |
| "I chose X over Y becauseā¦; the cost isā¦; I'd revisit ifā¦" | "X is the best." / "It scales." |
| "The hard part here isā¦, so I'd like to go deep there." | Going deep on the login page |
| "What happens if this call times out? It must be idempotent." | "We just retry." |
| "This is a single point of failure; I'd addā¦" | "It won't fail." |
| "I'd measure p99 and the error rate, and load-test to the knee." | "It should be fast enough." |
| "Good point ā let me revisit that." | Defending a flaw the interviewer found |