System Architecture · Insper · Lab 2

Lab 2 — Measure first

Load, stress and sweep tests with k6: find the knee, check Little's Law, name the bottleneck

The story so far

lab 1 · service timeone user: a few ms (detail), tens of ms (listing)
→
chapters 2–3 · lab 2load, stress, sweep: the knee
→
next · lab 3three replicas behind an edge

Class 2 estimated ~33,000 holds/s against ~1,100 for one database, on an assumed service time. Where does one real catalog leave its latency target?

  • Measured, not assumed: the knee and the resource that fills.

Measure before you scale

catalog gets slow under load. Cache? Replica? Ten more servers? Which one — and how do you know?

  • Guessing is how teams build a cache for a service limited by a pool of 10.
  • Three questions: how much load, where it breaks, why.
  • Tools: a load generator outside (k6), gauges inside (Actuator), docker stats.

Before you measure: a hypothesis

playground monolithbreakpoint profile: which box turns red first?
→
preset boxoffice-poolcatalog + pool of 10 under a ramp
→
your prediction"breaks at ___ req/s because ___ saturates"
→
the labsweep, gauges, docker stats
  • The playground models the catalog's CPU (800 req/s) and the pool as the only limits: it predicts that 30 connections move the knee past 600 req/s.
  • Write it down. Step 6 tests the prediction on the real service.

Closed vs open workload

Closed (vus: virtual users, VUs) Open (*-arrival-rate)
You set concurrent users requests per second
Server slows down k6 sends less k6 keeps the rate and starts more VUs
Overload ⚠️ hidden ✅ latency + dropped_iterations

Coordinated omission: a closed tester skips exactly the requests that would have suffered most.

The test family

Test Question
smoke does it work, how fast with 1 user?
load at X req/s, what are the percentiles p50, p95, p99 (median, 95th, 99th)?
sweep where is the knee?
stress what happens past it — and does it recover?
k6 run -e RATE=200 -e DURATION=1m load/load.js

Gauges from the inside

k6open model, RATE req/s
→
Tomcat threadstomcat.threads.busy · max 200
wait →
Hikari poolconnections.pending waiting · active ≤ 10
→
PostgreSQLits CPU in docker stats
  • A thread holds a connection only during its transaction: busy ≥ active + pending.
  • pending > 0 means threads are waiting for a connection, not short of work.
bash scripts/watch-metrics.sh http://localhost:8082

Three terminals during every test: k6, watch-metrics, docker stats.

The sweep

bash scripts/sweep.sh 50 100 200 300 400 500 600
uv run scripts/summarize.py results/lab2/rate-*.json
  • One minute per rate, warm-up first (a cold Java virtual machine, JVM, spoils the first point), 10 s of pause between steps.
  • One row per rate: throughput, p50, p95, p99, errors, dropped.

An example knee (1 central processing unit, CPU; pool of 10)

Offered Served p50 p99
200 200 0.5 ms 34 ms
300 300 0.6 ms 39 ms
400 400 1.5 ms 224 ms
500 490 15 ms 2,056 ms

One laptop, one run. Yours will differ — the shape will not.

Why latency explodes

  • Response ≈ S / (1 − ρ), so the wait ≈ ρ / (1 − ρ) × S.
  • ρ = 50% → wait ≈ 1×S.
  • ρ = 90% → ≈ 9×S.
  • ρ = 99% → ≈ 99×S.

Capacity = highest rate with p99 inside the target — not the highest throughput ever printed.

Little's Law: L = λ × W

  • 300 req/s × 10 ms = 3 requests inside → tomcat.threads.busy ≈ 3.
  • The pool is a server with 10 slots: it saturates when λ × (database time per request) = 10.
  • ~8 ms per request without load → 1,250 req/s. The knee is far lower: under load each query gets slower and holds its connection longer.
  • At the knee: db_active = 10, db_pending grows → the queue is at the pool.

200 busy threads, 10 active connections, 190 pending. More Tomcat threads would…?

Move the bottleneck

DB_POOL_SIZE=30 docker compose up -d catalog
OUT=results/lab2-pool30 bash scripts/sweep.sh 50 100 200 300 400 500 600
the knee moves upthe pool was the narrowest pipe — until the next resource saturates: the catalog's CPU or PostgreSQL
the knee barely movesdb at its CPU limit in docker stats: the pool was only where the queue showed
  • Either result answers the question: the evidence is which resource is full. Compare with your playground prediction.
  • Bigger pools cost database processes: 3 services × 3 replicas × 30 > max_connections.

Stress: past the knee and back

K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=results/lab2-stress.html \
  k6 run -e PEAK=600 load/stress.js
1 · the kneeon the way up, p95 leaves the floor
→
2 · the peak (600 req/s)VUs climb towards maxVUs: Little's Law on the client
→
3 · the recoveryafter the drop to 25%: seconds? never?

A service that recovers by itself is stable; one that stays slow needs load shedding (labs 3 and 9).

Without Docker: the twin

A model of the lab's stack (200 threads, the pool, 1 CPU, PostgreSQL with 7 cores) replays steps 1, 4, 6 and 7 with the page's k6 commands.

Step What the twin shows
4 · sweep p99 flat to 300 req/s, 159 ms at 400, a wall at 500: 463 req/s, 1,688 dropped
6 · pool of 30 the knee does not move; details faster, listings slower; db at 700%
1 · closed vs open 50 VUs: p99 266 ms, "all fine"; open at 500 req/s: 1,336 ms
7 · stress back under 300 ms 8 s after the drop

Service times chosen to match the shape of the page's run — not measured. Your laptop decides which case of step 6 you get.

Deliverable

  • Two sweeps (pool 10, pool 30), knee and capacity of each.
  • Little's Law table vs measured gauges.
  • Stress report with knee, peak and recovery.
  • One paragraph: the bottleneck of catalog and the evidence.

What breaks next

lab 2 · the kneeexample run: p99 224 ms at 400 req/s, 2 s at 500
→
a 30× gap10:00 asks for ~33,000 holds/s: no single machine
→
lab 3 · edgethree replicas behind Nginx
  • Between machines the network lies: class 3, Distributed Systems.

Takeaways

  1. Use an open model for public traffic; watch dropped_iterations.
  2. The knee: flat latency, then explosion — capacity lives just below it.
  3. Little's Law links throughput, latency and concurrency — check it with real gauges.
  4. The queue shows where requests wait (here: the pool); docker stats shows which resource is full.
  5. Fixing one bottleneck moves it; measure again after every change.

Chapters 2 and 3 of the story. Outline: (1) why measure before optimizing; (2) closed vs open models; (3) the scripts; (4) gauges from inside; (5) the sweep and the knee; (6) Little's Law and the pool; (7) moving the bottleneck; (8) stress.

The story of the course (2027.1 overview, "The story of the course"): every chapter opens with the number the previous one left. Lab 2 closes chapter 2 and opens chapter 3.

A model is a hypothesis. The deliverable has a "hypothesis check" line: right or wrong, and why.