Skip to content

a. Overview

In this lab

  • Write load, stress and sweep tests for catalog with k6, using an open workload model.
  • Find the knee: the arrival rate where latency stops being flat and explodes.
  • Check Little's Law with numbers taken from inside the service (busy threads, database connections) and identify the bottleneck.
  • Change one thing (the connection pool) and measure again: did the bottleneck move?

"Measure first" is the most useful habit in system design. Before adding a cache, a replica or ten more servers, you need to know how much load the current system takes, where it breaks and why. Guessing is how teams spend a month on a cache for a service whose real limit was a connection pool of 10. In this lab you treat catalog as a black box with a load generator in front (k6) and a few gauges inside (Actuator), and you produce the most important chart of the course: latency against throughput.

The story so far

Chapters 2 and 3 of the story of the course. Lab 1 built catalog and measured its service time with one user: a few milliseconds for a detail, tens for a listing (in lab 1's example run, ~11 ms and ~42 ms at the 95th percentile). Class 2 did the arithmetic for 10:00: ~33,000 holds/s of demand against ~1,100 that one database sustains at a sane utilization β€” an estimate built on an assumed service time. This lab writes load, stress and sweep tests with k6, measures the knee of one catalog instance and names its bottleneck with gauges from inside the service: the number that replaces the assumption.

Before you start

Time 2–3 hours (each sweep takes ~10 minutes of waiting: start it and read the next section)
You need the tools of lab 0 (k6, uv, jq); Docker with at least 4 GB; close heavy applications while measuring
Directory every command runs from the root of your boxoffice/ repository
docker compose version        # Docker Compose v2.x (or newer)
java -version                 # 21 or newer
mvn -v                        # Apache Maven 3.9+
k6 version                    # k6 v1.0+
uv --version                  # uv (runs the Python helper scripts)

Start state β€” the end of lab 1, with the stack running (docker compose up -d, catalog healthy):

boxoffice/ at the start of lab 2
πŸ“ boxoffice/
β”œβ”€β”€ YAML compose.yaml               # db + catalog (port 8082)
β”œβ”€β”€ JavaScript load/hello.js  load/smoke.js
β”œβ”€β”€ Markdown results/lab0.md  results/lab1.md
β”œβ”€β”€ GNU Bash scripts/check-env.sh
└── πŸ“ services/catalog/          # the service of lab 1

Catch up

Skipped or broke lab 1? Download boxoffice-after-lab-1.zip, unzip it, cd boxoffice and run docker compose up --build -d. This lab only adds files (k6 scripts and helpers); all of them are shown in full below and are in code/.

Slides

Click the slides and use ← β†’ to move through them; F goes full screen and P opens the presenter view, with the notes. Open the slides in a new tab

Theory connection

Before you measure: a hypothesis

The playground runs the same experiment in seconds. Form a hypothesis first, then check it against the real service.

Try it: the monolith scenario below starts with the breakpoint profile β€” the load grows until something gives. Watch which component turns red first and how the 99th percentile (p99) behaves just before. Then write down your guess for catalog: "it will break at about ___ req/s because ___ saturates first".

Then this lab's own preset: one catalog and its PostgreSQL with the default pool of 10 connections, under a ramp like stress.js. Find the knee and the component the Report blames, then follow the hints: the playground predicts that 30 connections move the knee past 600 req/s, because it models the catalog's CPU (800 req/s) and the pool as the only limits. Write that prediction down β€” step 6 tests it on the real service.

1. Closed or open workload?

In lab 0 each virtual user (VU) sent a request, waited for the answer, and sent the next one. That is a closed model: when the server slows down, the load generator slows down with it, and the test politely stops pushing. Real fans do not wait for each other: at 10:00 they arrive at whatever rate they arrive. That is an open model.

Closed model (vus, constant-vus) Open model (constant-arrival-rate, ramping-arrival-rate)
You set number of concurrent users arrival rate (requests per second)
When the server slows down k6 sends fewer requests k6 keeps the rate and starts more VUs
Good for "N users working in a back office" public websites, application programming interfaces (APIs), flash sales
Hides overload? ⚠️ yes β€” throughput drops quietly (coordinated omission) βœ… no β€” latency and queue grow visibly
Signal of saturation latency up, throughput flat latency explodes, dropped_iterations > 0

Coordinated omission

A closed-model tester that waits for slow responses omits the requests it would have sent meanwhile β€” exactly the ones that would have suffered the worst latency. The result looks better than reality. Prefer arrival-rate executors when the question is "can we take X requests per second?"1.

2. The scripts

The load test keeps a constant arrival rate (RATE) for a fixed time, with a realistic mix: 80% event details and 20% city listings.

load/load.js
// Lab 2: load test with an OPEN workload model. k6 starts RATE new requests per second,
// whether or not the previous ones have finished, like real fans arriving at a website.
//   k6 run -e RATE=100 -e DURATION=1m load/load.js
import http from 'k6/http';
import { check } from 'k6';

const BASE_URL = __ENV.BASE_URL || 'http://localhost:8082';
const RATE = Number(__ENV.RATE || 50);              // requests (iterations) per second
const DURATION = __ENV.DURATION || '1m';
const LIST_SHARE = Number(__ENV.LIST_SHARE || 0.2); // 20% listings, 80% event details
const CITIES = ['SΓ£o Paulo', 'Rio de Janeiro', 'Belo Horizonte', 'Porto Alegre', 'Curitiba',
  'Salvador', 'Recife', 'Fortaleza', 'BrasΓ­lia', 'Manaus'];

export const options = {
  scenarios: {
    open: {
      executor: 'constant-arrival-rate',
      rate: RATE,
      timeUnit: '1s',
      duration: DURATION,
      preAllocatedVUs: Math.max(20, RATE),   // VUs ready to start requests
      maxVUs: Math.max(200, RATE * 10),      // k6 may add more if requests get slow
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    http_req_failed: ['rate<0.01'],
    'http_req_duration{name:detail}': ['p(99)<500'],
    'http_req_duration{name:list}': ['p(99)<1000'],
  },
};

export default function () {
  if (Math.random() < LIST_SHARE) {
    const city = CITIES[Math.floor(Math.random() * CITIES.length)];
    const page = Math.floor(Math.random() * 5);
    const res = http.get(`${BASE_URL}/catalog/events?city=${encodeURIComponent(city)}&page=${page}`,
      { tags: { name: 'list' } });
    check(res, { 'list is 200': (r) => r.status === 200 });
  } else {
    const id = 1 + Math.floor(Math.random() * 200000);
    const res = http.get(`${BASE_URL}/catalog/events/${id}`, { tags: { name: 'detail' } });
    check(res, { 'detail is 200': (r) => r.status === 200 });
  }
}

The stress test climbs a staircase of rates past the knee, holds the peak and then drops back to a low rate. The question is not only "where does it break?", but "does it recover by itself when the load goes away?".

load/stress.js
// Lab 2: stress test. Climb a staircase of arrival rates past the knee, then come back
// down: does the service recover by itself when the load drops?
//   k6 run -e PEAK=600 load/stress.js
import http from 'k6/http';
import { check } from 'k6';

const BASE_URL = __ENV.BASE_URL || 'http://localhost:8082';
const PEAK = Number(__ENV.PEAK || 600);             // highest arrival rate (req/s)
const LIST_SHARE = Number(__ENV.LIST_SHARE || 0.2);
const CITIES = ['SΓ£o Paulo', 'Rio de Janeiro', 'Belo Horizonte', 'Porto Alegre', 'Curitiba',
  'Salvador', 'Recife', 'Fortaleza', 'BrasΓ­lia', 'Manaus'];

const step = (fraction, duration) => ({ target: Math.round(PEAK * fraction), duration });

export const options = {
  scenarios: {
    staircase: {
      executor: 'ramping-arrival-rate',
      startRate: 10,
      timeUnit: '1s',
      preAllocatedVUs: 50,
      maxVUs: 2000,
      stages: [
        step(0.25, '30s'), step(0.25, '30s'),   // warm-up and 1/4 of the peak
        step(0.50, '30s'), step(0.50, '30s'),
        step(0.75, '30s'), step(0.75, '30s'),
        step(1.00, '30s'), step(1.00, '1m'),    // peak, held longer
        step(0.25, '15s'), step(0.25, '1m'),    // back down: recovery?
      ],
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    http_req_failed: ['rate<0.05'],
    // a breakpoint test would abort here: { threshold: 'p(99)<2000', abortOnFail: true }
    http_req_duration: ['p(99)<2000'],
  },
};

export default function () {
  if (Math.random() < LIST_SHARE) {
    const city = CITIES[Math.floor(Math.random() * CITIES.length)];
    const res = http.get(`${BASE_URL}/catalog/events?city=${encodeURIComponent(city)}`,
      { tags: { name: 'list' }, timeout: '10s' });
    check(res, { 'list is 200': (r) => r.status === 200 });
  } else {
    const id = 1 + Math.floor(Math.random() * 200000);
    const res = http.get(`${BASE_URL}/catalog/events/${id}`, { tags: { name: 'detail' }, timeout: '10s' });
    check(res, { 'detail is 200': (r) => r.status === 200 });
  }
}
Test Executor Question
smoke (load/smoke.js, lab 1) 1 VU does it work, and how fast without load?
load (load/load.js) constant-arrival-rate at X req/s, what are p50/p95/p99 and the error rate?
sweep (scripts/sweep.sh) many load.js runs where is the knee?
stress (load/stress.js) ramping-arrival-rate what happens past the knee, and does it recover?

3. Watch from the inside

k6 sees the service from the outside. To find why latency grows you need gauges from the inside. Spring Boot Actuator already publishes them:

Metric Meaning Limit
tomcat.threads.busy requests being processed right now server.tomcat.threads.max = 200
hikaricp.connections.active database connections in use pool size (DB_POOL_SIZE, default 10)
hikaricp.connections.pending requests waiting for a connection β€” (should be 0)
sequenceDiagram
    participant K as k6 (one VU)
    participant T as Tomcat thread
    participant P as Hikari pool
    participant D as PostgreSQL
    K->>T: GET /catalog/events/42 (busy_threads + 1)
    T->>P: getConnection()
    Note over T,P: all 10 in use: the thread waits (db_pending + 1)
    P-->>T: a free connection (db_active + 1)
    T->>D: SELECT the event by id
    D-->>T: one row
    T->>P: close() returns the connection (db_active - 1)
    T-->>K: 200 and the JSON (busy_threads - 1)

A thread holds a connection only during its transaction, so busy_threads β‰₯ db_active + db_pending; when db_pending grows, the threads are not short of work β€” they wait for a connection.

scripts/watch-metrics.sh
#!/usr/bin/env bash
# Lab 2: print, once per second, how many requests are INSIDE the service right now.
#   bash scripts/watch-metrics.sh http://localhost:8082
URL=${1:-http://localhost:8082}
metric() {   # metric <name>: current value of an actuator metric
  curl -s "$URL/actuator/metrics/$1" | jq -r '.measurements[0].value // "?"'
}
printf '%-8s %13s %10s %11s\n' time busy_threads db_active db_pending
while true; do
  printf '%-8s %13s %10s %11s\n' "$(date +%T)" \
    "$(metric tomcat.threads.busy)" \
    "$(metric hikaricp.connections.active)" \
    "$(metric hikaricp.connections.pending)"
  sleep 1
done

Keep three terminals open during every test: k6, bash scripts/watch-metrics.sh, and docker stats (CPU of catalog and db).

You should see one line per second (at overload, one every few seconds: the request that reads the metrics waits for a free Tomcat thread too). At rest busy_threads is 1 (the request that reads the metric); under load it follows Little's Law, and at overload the pool fills up (lines from a run for this lab, at 50, 400 and 600 req/s):

time      busy_threads  db_active  db_pending
03:43:21           1.0        0.0         0.0
03:45:19          12.0       10.0         3.0
03:46:15         200.0        9.0       190.0

4. Find the knee

Run load.js at increasing rates, one minute each, and save each summary. The sweep script does the bookkeeping (and a warm-up first: a cold Java virtual machine, JVM, would spoil the first point):

scripts/sweep.sh
#!/usr/bin/env bash
# Lab 2: run load.js once per arrival rate and save one k6 summary per run.
#   bash scripts/sweep.sh 25 50 100 150 200 300 400
# Then:  uv run scripts/summarize.py results/lab2/rate-*.json
set -euo pipefail
OUT=${OUT:-results/lab2}
DURATION=${DURATION:-1m}
mkdir -p "$OUT"
echo ">>> warm-up: 30 s at RATE=$1 (not recorded)"
k6 run -q -e RATE="$1" -e DURATION=30s load/load.js > /dev/null || true
for rate in "$@"; do
  echo ">>> RATE=$rate req/s for $DURATION"
  # || true: a run that crosses a threshold still produces a summary; keep sweeping
  k6 run -q -e RATE="$rate" -e DURATION="$DURATION" \
     --summary-export "$OUT/rate-$rate.json" load/load.js > /dev/null || true
  sleep 10   # let queues drain before the next step
done
scripts/summarize.py
# /// script
# requires-python = ">=3.12"
# dependencies = []
# ///
"""Turn k6 --summary-export files into one Markdown table (one row per file).

    uv run scripts/summarize.py results/lab2/rate-*.json

The block above is inline script metadata (PEP 723): uv reads it and runs the
script with a suitable Python, no virtual environment to manage.
"""
import json
import re
import sys


def ms(metric, stat):
    value = metric.get(stat) if metric else None
    return f"{value:.1f}" if isinstance(value, (int, float)) else "–"


def row(path):
    m = json.load(open(path))["metrics"]
    offered = re.search(r"rate-(\d+)", path)
    dur = m.get("http_req_duration", {})
    failed = m.get("http_req_failed", {}).get("value", 0)
    dropped = m.get("dropped_iterations", {}).get("count", 0)
    return [
        offered.group(1) if offered else path,
        f"{m['http_reqs']['rate']:.0f}",
        ms(dur, "med"), ms(dur, "p(95)"), ms(dur, "p(99)"),
        ms(m.get("http_req_duration{name:detail}"), "p(99)"),
        ms(m.get("http_req_duration{name:list}"), "p(99)"),
        f"{failed * 100:.1f}%",
        str(dropped),
        str(m.get("vus_max", {}).get("value", "–")),
    ]


def sort_key(path):
    n = re.search(r"rate-(\d+)", path)
    return int(n.group(1)) if n else 0


header = ["offered (req/s)", "throughput (req/s)", "p50 (ms)", "p95 (ms)", "p99 (ms)",
          "p99 detail", "p99 list", "errors", "dropped", "max VUs"]
print("| " + " | ".join(header) + " |")
print("|" + "|".join("---:" for _ in header) + "|")
for path in sorted(sys.argv[1:], key=sort_key):
    print("| " + " | ".join(row(path)) + " |")
docker compose up -d                       # catalog + db from lab 1
bash scripts/sweep.sh 50 100 200 300 400 500 600
uv run scripts/summarize.py results/lab2/rate-*.json

sweep.sh prints one line per step (>>> warm-up: 30 s at RATE=50 (not recorded), then >>> RATE=50 req/s for 1m, …) and takes about 10 minutes for seven rates. Steps past the knee also print level=error msg="thresholds on metrics ... have been crossed": the latency thresholds of load.js failed, which is expected there β€” the summary is saved anyway. summarize.py then prints a Markdown table like the one below β€” paste it into results/lab2.md.

An example run on a laptop (catalog limited to 1 CPU, pool of 10 connections):

offered (req/s) throughput (req/s) p50 (ms) p95 (ms) p99 (ms) p99 detail p99 list errors dropped max VUs
50 50 1.3 29.2 41.3 6.5 89.7 0.0% 0 50
100 100 0.6 23.6 32.9 3.9 41.4 0.0% 0 100
200 200 0.5 21.9 33.5 5.5 59.3 0.0% 0 200
300 300 0.6 24.9 38.6 8.5 59.0 0.0% 0 300
400 400 1.5 49.6 223.7 199.1 294.4 0.0% 0 400
500 490 15.0 982.5 2056.2 1984.9 2238.8 0.0% 410 556

Past 500 req/s the numbers jump around from run to run β€” a sign that the service is overloaded and queues decide the outcome, not the code.

2026-11-18T00:40:16.007441 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 100 200 300 400 500 offered load (req/s) 100 200 300 400 500 throughput (req/s) Throughput follows the load... until it cannot offered = served served 100 200 300 400 500 offered load (req/s) 1 0 0 1 0 1 1 0 2 1 0 3 latency (ms, log scale) Latency is flat, then explodes p99 p50 knee

How to read it:

  • Below the knee, throughput equals the offered rate and p99 is roughly flat: requests rarely wait for each other.
  • At the knee, a resource reaches ~100% busy. Small increases in load produce large increases in latency β€” the queueing curve from class 2 (W β‰ˆ S / (1 βˆ’ ρ)).
  • Past the knee, throughput stops following the offered rate, dropped_iterations appears (k6 ran out of VUs to keep the rate), and p99 is measured in seconds. Users have already left.

Capacity is the knee, not the maximum

The useful capacity of this instance is the highest rate with p99 inside the target (say 300 ms), not the highest throughput the summary ever printed. Plan with a safety margin below the knee: the next spike will not ask permission.

5. Little's Law check

Little's Law holds for any stable system2: L = Ξ» Γ— W β€” the average number of requests inside (L) equals the arrival rate (Ξ») times the average time each one spends inside (W). Apply it to two boundaries:

  1. The whole service: L should match tomcat.threads.busy. At 300 req/s with an average latency of 10 ms, L = 300 Γ— 0.010 = 3 busy threads.
  2. The database pool: each request holds a connection for its database time. The pool is a server with 10 slots, so it saturates when Ξ» Γ— (database time per request) = 10.

That second line predicts the knee. Measure the database time per request with one user (lab 1): if the mix averages ~8 ms of database time per request without load, the pool would allow 10 / 0.008 β‰ˆ 1,250 req/s. The knee you measured is much lower β€” because under load the queries get slower (the database and the single catalog CPU are busy too), so each request holds its connection longer. Look at watch-metrics.sh around the knee: db_active sits at 10 and db_pending starts to grow. The pool is full; requests queue for it.

Think first: busy threads reach 200 and db_pending 190 at 600 req/s. What does that mean?

Tomcat has 200 threads, all busy; 10 hold a database connection and 190 are waiting for one. The service is not short of threads or CPU in the web layer β€” it is short of database connections (or of database speed). Adding Tomcat threads would only add waiters. This is the "find the narrowest pipe" exercise of the course: the bottleneck is the resource whose queue grows.

6. Move the bottleneck

Change one variable, measure again. The pool size is configurable without rebuilding (DB_POOL_SIZE, wired in lab 1):

DB_POOL_SIZE=30 docker compose up -d catalog        # recreate catalog with a bigger pool
OUT=results/lab2-pool30 bash scripts/sweep.sh 50 100 200 300 400 500 600
uv run scripts/summarize.py results/lab2-pool30/rate-*.json

Compare the two tables β€” and compare them with your playground prediction. If the pool was the narrowest pipe, the knee moves up, but not forever: with 30 connections the next resource saturates β€” the single CPU of catalog, or PostgreSQL itself. If the knee barely moves, the pool was only where the queue showed: docker stats shows db at its CPU limit, and 30 connections let more listings share the same cores (the twin at the end of step 7 plays this case; lab 3 meets it again with three replicas). Both outcomes answer the question; the evidence is which resource is full. A bigger pool also has a cost: every connection is a process in PostgreSQL, and in lab 6 three services Γ— three replicas Γ— 30 connections would exceed max_connections. Optimizing one resource always moves the bottleneck somewhere else; the job is to know where it went.

7. Stress: past the knee and back

docker compose up -d catalog       # back to the default pool (10): PEAK=600 is past its knee
K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=results/lab2-stress.html \
  k6 run -e PEAK=600 load/stress.js

Open http://localhost:5665 while it runs: the k6 web dashboard plots request rate, latency percentiles and VUs over time; the HTML (HyperText Markup Language) report is saved at the end. Look for three moments:

  1. The knee on the way up: p95 leaves the floor.
  2. The peak: VUs climb towards maxVUs β€” k6 needs more concurrent users to keep the rate because each request takes longer (Little's Law again, from the client side).
  3. The recovery: after the drop to 25% of the peak, how many seconds until p95 is back to normal? A service that recovers by itself is stable; one that stays slow (queues that never drain, a garbage collector that never catches up) needs load shedding β€” a topic for labs 3 and 9.

Without Docker: the twin. The simulator replays this lab with a model of its stack β€” the 200 Tomcat threads, the pool of DB_POOL_SIZE connections, one CPU for catalog, PostgreSQL with 7 cores (lab 3 saw it at ~670% CPU) β€” loaded the way load.js, sweep.sh and stress.js load it. Its service times are not measured: they are chosen so that the sweep has the shape of the example run of step 4 (an event detail costs almost nothing; a city listing keeps the database busy). Each script is one step of this page.

Try it

  1. 4 Β· sweep: p99 stays flat up to 300 req/s (83 ms), leaves the floor at 400 (159 ms) and hits the wall at 500: 1,364 ms, only 463 req/s served and 1,688 dropped iterations. The page's run gave 224 ms, 2,056 ms, 490 req/s and 410: the same shape, other numbers. At the end of the 600 req/s step, watch-metrics prints the line of step 3 β€” 200 busy threads, 10 active, 190 waiting β€” while docker stats shows db at 700% CPU β€” all its 7 cores β€” and catalog at 61%.
  2. 6 Β· pool of 30: the knee does not move β€” 464 req/s at 500. At 400 req/s the details' p99 falls from 93 ms to 9 ms (they no longer wait behind listings for a connection) and the listings' p99 rises from 210 ms to 289 ms (more of them share the database's cores). In this model the pool was where the queue showed and PostgreSQL was what was full; your machine decides which case you get.
  3. 1 Β· closed vs open: 50 VUs that wait for their answers get 459 req/s with p99 266 ms β€” inside a 300 ms target, nothing looks wrong. The open model at 500 req/s, on the same service, gives p99 1,336 ms and 1,558 dropped iterations.
  4. 7 Β· stress: at the peak k6 drops iterations and the VUs climb; after the drop to 150 req/s, the p99 of each second is back under 300 ms 8 s after the drop began β€” this catalog recovers by itself.

Measure it

Record everything in results/lab2.md:

results/lab2.md
# Lab 2 β€” catalog under load

Setup: catalog 1 CPU / 768 MB, Hikari pool = 10, PostgreSQL 17, k6 open model (80% detail, 20% list)

## Sweep (pool = 10)
<paste the table printed by summarize.py>

## Sweep (pool = 30)
<paste the table>

## Knee and capacity
- Knee (pool 10): ____ req/s β€” capacity with p99 < 300 ms: ____ req/s
- Knee (pool 30): ____ req/s β€” capacity with p99 < 300 ms: ____ req/s
- Bottleneck at the knee (evidence from watch-metrics / docker stats): ____

## Little's Law
| Offered (req/s) | avg latency (s) | Ξ» Γ— W | busy threads (measured) | db_active (measured) |
|---:|---:|---:|---:|---:|
|  100 |  |  |  |  |
|  300 |  |  |  |  |

## Stress test
- Peak reached: ____ req/s; p95 at the peak: ____ ms; dropped iterations: ____
- Recovery time after the drop: ____ s

## Hypothesis check
- My playground hypothesis was: ____ . It was right / wrong because ____ .

If your numbers differ

Your knee will not be at 300–400 req/s: it depends on your CPU, on Docker's resources and on what else runs. What matters is the shape (flat, then a sharp rise), the evidence for the bottleneck (db_active pinned at the pool size, db_pending growing) and the change when you resize the pool. If the knee is very low (< 100 req/s), check docker stats while the sweep runs: a container stuck at its CPU limit, or swap use, explains it. If k6 prints Insufficient VUs, the service is past its knee β€” that is a result, not an error.

Troubleshooting

Symptom Cause Fix
every request fails with connection refused the stack is not running, or BASE_URL points elsewhere docker compose ps; in lab 2 the target is http://localhost:8082 (the default)
level=warning msg="Insufficient VUs, reached 2000 active VUs and cannot initialize more" the arrival rate needs more concurrent requests than maxVUs expected past the knee; the run is still valid β€” note dropped_iterations
summarize.py prints only the header no JSON (JavaScript Object Notation) file matched the glob (wrong OUT folder) ls results/lab2/; pass the same folder you gave to sweep.sh
watch-metrics.sh prints ? jq missing, or catalog not reachable on 8082 install jq; curl localhost:8082/actuator/metrics must list tomcat.threads.busy
uv: command not found uv not installed lab 0, step 1 β€” or run python3 scripts/summarize.py … (it only uses the standard library)
the first point of a sweep is worse than the next ones cold JVM sweep.sh already warms up; repeat the run if your machine was busy
numbers vary a lot between two identical runs background load (browser, integrated development environment (IDE), other containers) close what you can, stop unrelated containers (docker ps), run twice and keep the second

Best practices in this lab

  • Load-test with an open workload model β€” load/load.js uses constant-arrival-rate, so a slow service cannot quietly lower the load it receives (Percentiles β€” measuring honestly).
  • Warm up, then sweep, with a pause between steps β€” scripts/sweep.sh discards a 30 s warm-up and sleeps 10 s between rates, so a cold JVM or a leftover queue does not spoil a point (Queueing β€” the knee).
  • Watch the inside while you load the outside β€” scripts/watch-metrics.sh prints busy threads and pending connections, the evidence that names the bottleneck (Little's Law).
  • Change one variable and measure again β€” only DB_POOL_SIZE changes between the two sweeps (Estimation β€” the method).
  • State capacity as the highest rate within the latency target β€” not the highest throughput ever printed; results/lab2.md records the knee and the capacity (Percentiles).

Clean up

docker compose up -d catalog     # if you changed DB_POOL_SIZE, recreate catalog with the default (10)
docker compose down              # when you are done for the day (data is kept)

Keep results/lab2* (the JSON summaries and your results/lab2.md): lab 3 compares against them. The HTML report is ignored by Git on purpose (it is large).

Deliverable

  • load/load.js, load/stress.js, scripts/sweep.sh, scripts/summarize.py and scripts/watch-metrics.sh in your repository.
  • Two sweeps (pool 10 and pool 30) with tables in results/lab2.md, the knee and the capacity of each.
  • The Little's Law table with at least two rates, compared with the measured gauges.
  • The stress test HTML report (results/lab2-stress.html, not committed) and its three moments described.
  • One paragraph: what is the bottleneck of catalog, and what is your evidence?
  • Repository tagged lab2; quiz answered.

End state

added in lab 2
πŸ“ boxoffice/
β”œβ”€β”€ JavaScript load/load.js                 # open-model load test (RATE, DURATION, LIST_SHARE)
β”œβ”€β”€ JavaScript load/stress.js               # staircase past the knee and back
β”œβ”€β”€ GNU Bash scripts/sweep.sh             # one load.js run per rate
β”œβ”€β”€ Python scripts/summarize.py         # k6 summaries -> Markdown table (uv run)
β”œβ”€β”€ GNU Bash scripts/watch-metrics.sh     # Tomcat threads and Hikari pool, once per second
└── πŸ“ results/lab2/  results/lab2-pool30/  results/lab2.md

No service changed: the stack is still db + catalog.

What breaks next

One instance leaves its latency target at a few hundred requests per second β€” in the page's example run, p99 is 224 ms at 400 req/s and 2,056 ms at 500 β€” and a bigger pool does not save it when PostgreSQL is what is full. At 10:00:00 the edge receives about 100,000 requests/s and the holds alone are ~33,000/s: a 30Γ— gap that no tuning of one machine closes. BoxOffice needs many machines and, between them, a network that loses, delays and duplicates messages (class 3 β€” Distributed Systems). Lab 3 β€” Edge takes the first step: three catalog replicas behind Nginx, measured against the knee you just found (class 4 β€” Traffic & Edge).


  1. Grafana k6 documentation, Open and closed models and Arrival-rate executors. ↩

  2. John D. C. Little, "A Proof for the Queuing Formula: L = Ξ»W", Operations Research 9(3), 1961. ↩