Numbers Every Engineer Should Know

How long does it take to read a seat from memory? From an SSD? From a database in the same data centre? From a server in Virginia, while the fan is in São Paulo? The answers span eight orders of magnitude — from a nanosecond to a hundred milliseconds — and most architectural rules of thumb ("cache it", "batch it", "don't call across regions synchronously") are just consequences of that spread. This page collects the numbers worth memorizing and shows how to turn them, together with a few assumptions about users, into requests per second, storage and bandwidth.

Powers of two (and ten)

Data sizes are powers of two; estimates are easier in powers of ten. Since 2¹⁰ = 1,024 ≈ 10³, you can switch freely for estimation purposes:

Strictly, 2¹⁰ bytes is a kibibyte (KiB) and 10³ bytes a kilobyte (kB); the difference (2.4% at KB, 10% at TB) does not matter for back-of-the-envelope work.

Time

A handy conversion table, using 1 day ≈ 10⁵ s (the exact factor is 86,400, which makes the real rates about 15% higher):

Latency numbers every engineer should know

In 2009, Jeff Dean (Google) popularized a list of "numbers everyone should know" — the cost of basic operations in a computer and a network¹. Hardware has changed since (SSDs replaced spinning disks on the hot path, networks got faster), but the ratios have stayed remarkably stable, because some limits — the speed of light, the physics of memory — do not improve. The table uses 2020s orders of magnitude, compiled from Dean's list, its later updates² and Gregg's Systems Performance³; your hardware may differ by 2–3×, which is fine for estimates.

What the table teaches

1. Memory is ~1,000× faster than SSD, and SSD ~100× faster than HDD for random reads. This is why caches (Redis keeps everything in RAM) and in-memory indexes exist.
2. Inside a data centre, the network costs more than the work. A Redis GET takes microseconds of CPU on the server; the round trip is what you wait for. A page that makes 50 sequential calls to Redis spends ~10–25 ms just on round trips — batch them (MGET, pipelining) or cache in process.
3. Across regions, physics wins. Light in optical fibre travels at about 200,000 km/s, i.e. ~5 µs per km, so every 100 km adds about 1 ms of round trip before any router or server touches the packet. São Paulo to Virginia is ~7,500 km in a straight line, so the theoretical minimum round trip is ~75 ms; real routes give ~110–130 ms. No technology removes this: avoid synchronous cross-region calls on the critical path and serve users from nearby (CDN, regional deployments).
4. Sequential beats random. Reading 1 MB sequentially from an SSD costs about as much as a few random 4 KB reads. Log-structured storage, Kafka's append-only log and batch processing all exploit this.
5. Round trips add up; bandwidth rarely limits small requests. For a 2 KB API response, latency is dominated by round trips (connection setup, TLS, the request itself), not by transfer time. Reusing connections (keep-alive, HTTP/2) matters more than a fatter pipe.

Throughput rules of thumb

Latency tells you how long one operation takes; capacity planning also needs how many operations per second a component can sustain. These vary much more with hardware, configuration and workload than latencies do, so treat the table as illustrative starting assumptions that you must replace with measurements (that is what lab 2 is for):

The pattern to remember: reads are cheaper than durable writes, memory is cheaper than disk, and anything that must flush to disk or cross the network to commit (durable writes, replication, consensus) has a lower ceiling.

Typical sizes

Two multipliers are easy to forget: replication (a database usually keeps 3 copies; object stores replicate internally) and indexes and overhead (often 1.5–2× the raw data in a relational database).

Estimating requests per second (QPS)

QPS (queries per second, used loosely for requests per second) is the starting point of every capacity estimate:

• Peak factor: traffic is not flat over the day. For consumer products the busiest hour is typically 2–3× the daily average; that is the number you size for. For event-driven businesses, the peak is a spike that has nothing to do with the daily average — estimate it separately.
• Read/write ratio: split the actions into reads and writes. Reads can be cached and replicated; writes go to a primary. A product at 100:1 and one at 1:1 are different designs.

Estimating storage

Estimating bandwidth

Estimating servers and memory

Once you know the peak load and the per-node capacity, the number of nodes follows — with headroom and redundancy:

• Target utilization of 60–75% for latency-sensitive services — the queueing page explains why running closer to 100% destroys p99.
• Spares (N+1, N+2): if one node fails or is being deployed, the others must absorb its load without crossing the knee.

For cache memory, a common rule of thumb is the 80/20 rule: a small fraction of the items receives most of the traffic, so caching the hot set — often ~20% of the items, sometimes far less — captures most of the reads.

catalog:
  build: ./services/catalog
  environment:
    DB_HOST: db
    DB_PORT: 5432
    DB_NAME: ${DB_NAME:-boxoffice}
    DB_USER: ${DB_USER:-boxoffice}
    DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
    DB_POOL_SIZE: ${DB_POOL_SIZE:-10}  # HikariCP pool size (lab 2 experiments with it)
  ports:
    - "8082:8082"
  depends_on:
    db:
      condition: service_healthy   # wait until PostgreSQL accepts connections
  deploy:
    resources:
      limits:
        cpus: "1.0"                # one core per replica: makes scaling visible in lab 3
        memory: 768M

How it connects

flowchart LR
    P2["powers of two and ten<br/>1 day ≈ 10⁵ s"] --> QPS["QPS: average and peak<br/>the spike estimated apart"]
    LAT["latency numbers<br/>RAM, SSD, network"] --> CACHE["cache the hot set<br/>→ Caching"]
    LAT --> RT["count round trips<br/>→ Edge protocols"]
    QPS --> NODES["nodes = peak / capacity × U<br/>+ spares → Compute"]
    QPS --> BW["bandwidth = QPS × size<br/>→ CDN"]
    SIZES["typical sizes"] --> STO["storage × retention<br/>× replication"]
    STO --> SHARD["shard or not?<br/>→ Databases"]
    STO --> OBJ["bulky files and logs<br/>→ object store"]
    NODES --> KNEE["target utilization<br/>→ Queueing"]

Palette components: app (instances = peak / capacity per node, plus spares) · sql (orders and seats stay under a terabyte; the durable-write ceiling) · cache (the hot set, ~0.6 GB for the catalog) · cdn (image bandwidth off the origin) · objstore (ticket PDFs, images, logs) · kafka (~10 MB/s per partition) · vectordb (~4 KB per embedding). Scenarios: monolith, read-heavy-cache, cdn-static, url-shortener. Continues in: Queueing, Edge — DNS & CDN, Caching, Databases — partitioning, Case study: URL shortener, Exercise: estimation.

Key takeaways