Load Testing

The test that failed to predict last year's outage used 200 virtual users, each looping "request, wait for the response, sleep one second". It reported p95 = 180 ms. On the real day the p99 was over 10 seconds. Both numbers were measured; only one described reality. The difference was not the tool — it was how the test generated load, what it measured, and what it asked. This page is about designing load tests that tell the truth: the six classic test types, the workload model behind each, thresholds that encode SLOs, and the measurement errors that make tests lie.

All examples use k6 (Grafana Labs), the load-testing tool of the course labs: tests are JavaScript, the engine is Go, and it supports both workload models through its executors¹. Equivalent snippets for Gatling, Locust, JMeter, wrk2 and vegeta are at the end.

Anatomy of a k6 test

A k6 script has an init part (imports, data, options), an optional setup() that runs once, the VU code (the default function or a named function per scenario) that runs many times, and an optional teardown(). BoxOffice's tests share one module with the user flows, so every test type below is just a different load shape over the same behaviour:

import http from 'k6/http';
import { check, sleep } from 'k6';
import { Trend, Counter } from 'k6/metrics';
import { pickEvent, pickUser } from './data.js';

export const BASE = __ENV.BASE_URL || 'http://localhost:8080';

// A 409 (seat already taken) is a valid business outcome, not a failed request.
http.setResponseCallback(http.expectedStatuses({ min: 200, max: 399 }, 409));

const holdTime = new Trend('hold_time', true);   // business metric, in ms
const seatsHeld = new Counter('seats_held');

const json = (user) => ({
  headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${user.token}` },
});

export function browse() {
  const ev = pickEvent();
  const res = http.get(`${BASE}/events/${ev.id}`, { tags: { name: 'GET /events/{id}' } });
  check(res, { 'event page 200': (r) => r.status === 200 });
}

export function search() {
  const res = http.get(`${BASE}/search?q=rock&city=sao-paulo`, { tags: { name: 'GET /search' } });
  check(res, { 'search 200': (r) => r.status === 200 });
}

export function hold() {
  const ev = pickEvent();
  const user = pickUser();
  const map = http.get(`${BASE}/events/${ev.id}/seats?available=true&limit=8`,
                       { tags: { name: 'GET /events/{id}/seats' } });
  if (!check(map, { 'seat map 200': (r) => r.status === 200 })) return;
  const free = map.json('seats') || [];
  if (free.length === 0) return;                       // sold out: a valid outcome
  sleep(1 + Math.random() * 3);                        // think time: the fan picks a seat
  const seat = free[Math.floor(Math.random() * free.length)];
  const res = http.post(`${BASE}/holds`, JSON.stringify({ eventId: ev.id, seatIds: [seat.id] }),
                        Object.assign(json(user), { tags: { name: 'POST /holds' } }));
  holdTime.add(res.timings.duration);
  if (res.status === 201) seatsHeld.add(1);
  check(res, { 'hold 201 or 409': (r) => r.status === 201 || r.status === 409 });
}

export function checkout() {
  const user = pickUser();
  const res = http.post(`${BASE}/orders`, JSON.stringify({ holdId: `test-${user.id}` }),
                        Object.assign(json(user), { tags: { name: 'POST /orders' } }));
  check(res, { 'order accepted': (r) => r.status === 201 || r.status === 202 });
}

Three details in this module prevent the most common reporting errors:

• The name tag. Without it, every URL with a different event id becomes a separate metric series — thousands of them — and per-endpoint thresholds become impossible.
• Expected statuses. By default k6 counts any 4xx/5xx as http_req_failed. A sold-out seat (409) is correct behaviour; a 503 is not. Decide explicitly.
• Business metrics. seats_held tells you whether the system did useful work; a test with perfect latency and zero seats held is testing an error page.

Open vs closed workload models

This is the single most important design decision in a load test².

• In a closed model, a fixed population of N users loops: send a request, wait for the response, think for Z seconds, repeat. A new request is only sent when a previous one finished. Throughput is a consequence: by the interactive response-time law, \(X = N / (R + Z)\).
• In an open model, requests arrive at a rate λ independent of how fast the system answers — like fans on the internet, who do not coordinate with each other or wait for strangers' requests to finish. If the system slows down, requests keep arriving and the queue grows.

sequenceDiagram
    participant C as Closed: 1 virtual user
    participant S as BoxOffice (slow phase)
    participant O as Open: arrivals every 100 ms
    C->>S: request 1
    O->>S: request A (t = 0)
    O->>S: request B (t = 100 ms)
    O->>S: request C (t = 200 ms)
    S-->>C: response 1 after 2 s
    Note over C: only now sends request 2 — the slow phase generated ONE sample
    O->>S: requests D…T keep arriving on schedule
    S-->>O: A…T answered late — 20 slow samples, a growing queue

The consequence is dramatic exactly when it matters. Take 200 virtual users with 1 s of think time:

The closed test reduced the load by 63 % at the moment the system struggled — a load generator that backs off is a gift no real on-sale gives you. Use the model that matches the real population:

k6 executors

In arrival-rate executors, VUs are just the workers that execute iterations. k6 needs enough of them to keep the schedule — Little's Law again: VUs needed ≈ arrival rate × iteration duration. A hold() iteration takes about 3 s (two requests plus think time); at 500 iterations/s you need ~1 500 VUs. If k6 runs out, it cannot start iterations on time and counts them in the dropped_iterations metric — which must be zero, or your "open" test silently became a closed one.

export const options = {
  scenarios: {
    open: {
      executor: 'constant-arrival-rate',
      rate: RATE,
      timeUnit: '1s',
      duration: DURATION,
      preAllocatedVUs: Math.max(20, RATE),   // VUs ready to start requests
      maxVUs: Math.max(200, RATE * 10),      // k6 may add more if requests get slow
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    http_req_failed: ['rate<0.01'],
    'http_req_duration{name:detail}': ['p(99)<500'],
    'http_req_duration{name:list}': ['p(99)<1000'],
  },
};

Thresholds: SLOs as pass/fail criteria

A threshold turns a metric into a verdict; k6 exits with a non-zero code when one fails, which is what lets a pipeline gate on it. Write the SLOs from the workload model as thresholds, per endpoint:

Thresholds on percentiles are evaluated over the whole test run (or over the tagged subset), which is why the warm-up must be a separate, untagged scenario (see Methodology — warm-up).

The six test types

Each type answers a different question. Their load shapes, relative to the expected peak:

Smoke test

The smoke test is the "does it even run?" check — run it after every change to the script or the system, and as the first step in CI. A closed model with one or two VUs is fine here.

import { browse, search, hold } from './lib/boxoffice.js';

export const options = {
  vus: 2,
  duration: '1m',
  thresholds: {
    http_req_failed: ['rate==0'],
    checks: ['rate==1'],
  },
};

export default function () {
  browse();
  search();
  hold();
}

Load test

The load test is the main verdict: the realistic mix, at the expected peak arrival rate, long enough to reach a steady state. Each flow gets its own open scenario with its share of the rate, so the mix stays right even when one endpoint slows down.

import { browse, search, hold, checkout } from './lib/boxoffice.js';
export { browse, search, hold, checkout };

const PEAK = Number(__ENV.PEAK || 1000);           // iterations/s at peak (scaled for the lab)

function flow(exec, share, iterSeconds) {
  const rate = Math.round(PEAK * share);
  return {
    executor: 'ramping-arrival-rate',
    exec,
    startRate: 0,
    timeUnit: '1s',
    preAllocatedVUs: Math.ceil(rate * iterSeconds),        // Little's Law: VUs = rate × duration
    maxVUs: Math.ceil(rate * iterSeconds * 3),
    stages: [
      { target: rate, duration: '5m' },                    // ramp up
      { target: rate, duration: '20m' },                   // plateau at the expected peak
      { target: 0, duration: '2m' },                       // ramp down
    ],
  };
}

export const options = {
  scenarios: {
    browse:   flow('browse',   0.55, 0.2),
    search:   flow('search',   0.15, 0.3),
    hold:     flow('hold',     0.20, 3.5),
    checkout: flow('checkout', 0.10, 1.0),
  },
  thresholds: {
    'http_req_duration{name:GET /events/{id}}':       ['p(99)<200'],
    'http_req_duration{name:GET /events/{id}/seats}': ['p(99)<300'],
    'http_req_duration{name:GET /search}':            ['p(99)<400'],
    'http_req_duration{name:POST /holds}':            ['p(99)<300'],
    'http_req_duration{name:POST /orders}':           ['p(99)<800'],
    http_req_failed: ['rate<0.001'],
    checks: ['rate>0.999'],
    dropped_iterations: ['count==0'],
  },
};

The shares are of iterations; since hold() makes two requests, calibrate the shares so that the resulting request mix matches the workload model (k6's end-of-test summary shows requests per name).

Stress test

The stress test goes beyond the expected peak in steps, holds each step long enough to observe, then drops back to see whether the system recovers by itself. The interesting output is not "pass/fail" but how it fails:

• Graceful: past capacity, the gateway or the service rejects the excess quickly (429/503 in milliseconds), latency of accepted requests stays near the SLO, and everything returns to normal when the load drops. This is what rate limiting and load shedding buy.
• Catastrophic: everything slows down, timeouts trigger retries, retries add load (a retry storm), the connection pool and threads are exhausted, and the system stays degraded after the load drops (memory pressure, a backlog of queued work, circuit breakers flapping).

import { browse, hold } from './lib/boxoffice.js';
export { browse, hold };

const PEAK = Number(__ENV.PEAK || 1000);
const steps = (share) => [
  { target: PEAK * share,       duration: '5m' },   // 1× peak
  { target: PEAK * share,       duration: '5m' },
  { target: PEAK * share * 1.5, duration: '2m' },   // 1.5×
  { target: PEAK * share * 1.5, duration: '5m' },
  { target: PEAK * share * 2,   duration: '2m' },   // 2×
  { target: PEAK * share * 2,   duration: '5m' },
  { target: PEAK * share * 0.3, duration: '2m' },   // back to normal: does it recover?
  { target: PEAK * share * 0.3, duration: '5m' },
];

export const options = {
  scenarios: {
    browse: { executor: 'ramping-arrival-rate', exec: 'browse', startRate: 0, timeUnit: '1s',
              preAllocatedVUs: 400, maxVUs: 4000, stages: steps(0.75) },
    hold:   { executor: 'ramping-arrival-rate', exec: 'hold', startRate: 0, timeUnit: '1s',
              preAllocatedVUs: 1000, maxVUs: 10000, stages: steps(0.25) },
  },
  // No pass/fail on latency: the stress test is read on the dashboards.
  thresholds: { checks: ['rate>0.5'] },
};

const step = (fraction, duration) => ({ target: Math.round(PEAK * fraction), duration });

export const options = {
  scenarios: {
    staircase: {
      executor: 'ramping-arrival-rate',
      startRate: 10,
      timeUnit: '1s',
      preAllocatedVUs: 50,
      maxVUs: 2000,
      stages: [
        step(0.25, '30s'), step(0.25, '30s'),   // warm-up and 1/4 of the peak
        step(0.50, '30s'), step(0.50, '30s'),
        step(0.75, '30s'), step(0.75, '30s'),
        step(1.00, '30s'), step(1.00, '1m'),    // peak, held longer
        step(0.25, '15s'), step(0.25, '1m'),    // back down: recovery?
      ],
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    http_req_failed: ['rate<0.05'],
    // a breakpoint test would abort here: { threshold: 'p(99)<2000', abortOnFail: true }
    http_req_duration: ['p(99)<2000'],
  },
};

Spike test

The spike test reproduces 10:00:00: from a quiet baseline to above the peak within seconds. It exercises exactly what the load test cannot — autoscaling lag, cold caches, connection storms against the database, and the waiting room.

import { hold } from './lib/boxoffice.js';
export { hold };

const PEAK = Number(__ENV.PEAK || 1000);

export const options = {
  scenarios: {
    onsale: {
      executor: 'ramping-arrival-rate', exec: 'hold', startRate: PEAK * 0.05, timeUnit: '1s',
      preAllocatedVUs: 2000, maxVUs: 12000,
      stages: [
        { target: PEAK * 0.05, duration: '2m' },   // quiet morning
        { target: PEAK * 1.5,  duration: '30s' },  // 10:00:00
        { target: PEAK * 1.5,  duration: '5m' },
        { target: PEAK * 0.05, duration: '30s' },
        { target: PEAK * 0.05, duration: '4m' },   // recovery window
      ],
    },
  },
  thresholds: {
    http_req_failed: ['rate<0.01'],                     // bounded errors during the jump
    'http_req_duration{name:POST /holds}': ['p(99)<1500'],
  },
};

Run it twice — with the autoscaler at its normal minimum and with pre-scaling — and compare the first two minutes. The difference is the case for pre-scaling (see Kubernetes — autoscaling).

Try it: the scenario below is a flash sale like BoxOffice's on-sale, and its load profile already contains the spike: a scripted event at t = 60 s (the sale opens) multiplies the traffic by 20 for 90 s (do not pick the spike profile on top of it — that would multiply it by 5 again). The design as drawn has no waiting room: users → lb → app → inventory-db, with buyers calling the payment provider. Run it and see where the excess load goes: app saturates and rejects it (errors), and p99 explodes. Then open it in the full playground (↗), press Show a possible solution (CDN, catalog cache, and a waiting-room queue drained by order workers) and run again. Compare p99, the error rate and the utilization of inventory-db: the excess now waits in the queue — the consumer lag — instead of failing.

Soak test

The soak (endurance) test holds a realistic load — typically the daily average — for hours. It finds problems that need time: memory leaks, connection or file-descriptor leaks, unbounded caches, log volumes filling disks, slowly growing tables without an index, certificate or token refreshes that fail after N hours, and GC behaviour after the heap has filled once.

import { browse, search, hold } from './lib/boxoffice.js';
export { browse, search, hold };

const RATE = Number(__ENV.RATE || 300);           // ~ average daily load

export const options = {
  scenarios: {
    browse: { executor: 'constant-arrival-rate', exec: 'browse', rate: RATE * 0.7, timeUnit: '1s',
              duration: '8h', preAllocatedVUs: 100, maxVUs: 1000 },
    search: { executor: 'constant-arrival-rate', exec: 'search', rate: RATE * 0.2, timeUnit: '1s',
              duration: '8h', preAllocatedVUs: 50, maxVUs: 500 },
    hold:   { executor: 'constant-arrival-rate', exec: 'hold', rate: RATE * 0.1, timeUnit: '1s',
              duration: '8h', preAllocatedVUs: 150, maxVUs: 1500 },
  },
  thresholds: {
    http_req_failed: ['rate<0.001'],
    'http_req_duration{name:POST /holds}': ['p(99)<300'],
  },
};

A soak test is read as trends, not totals: the p99 in hour 8 vs hour 1, and the slope of resource usage. A PromQL query such as the one below on the JVM's old-generation heap after garbage collections should be flat; a steady positive slope is a leak.

deriv(jvm_memory_used_bytes{service="inventory", area="heap", id=~".*Old Gen"}[1h])

Breakpoint test

The breakpoint (capacity) test ramps the arrival rate continuously, far beyond the expected peak, and stops itself when the system clearly breaks. Its output is a number: the knee (the highest rate at which the SLO still holds) and the breaking point — plus the name of the resource that saturated first.

import { hold } from './lib/boxoffice.js';
export { hold };

export const options = {
  scenarios: {
    breakpoint: {
      executor: 'ramping-arrival-rate', exec: 'hold', startRate: 10, timeUnit: '1s',
      preAllocatedVUs: 1000, maxVUs: 20000,
      stages: [ { target: 4000, duration: '40m' } ],          // ~ +100 req/s per minute
    },
  },
  thresholds: {
    'http_req_duration{name:POST /holds}': [
      { threshold: 'p(99)<1000', abortOnFail: true, delayAbortEval: '1m' },
    ],
    http_req_failed: [ { threshold: 'rate<0.05', abortOnFail: true, delayAbortEval: '1m' } ],
  },
};

Never run a breakpoint test against production, and always watch the load generator: at thousands of VUs a single k6 machine may become the bottleneck — check its CPU and dropped_iterations, and distribute the test (k6-operator) when needed. The Finding bottlenecks page shows how to read the resulting curve.

Coordinated omission

Coordinated omission is the measurement error that made last year's test lie³. It happens whenever the load generator waits for a response before sending the next request and measures latency from the moment it actually sent the request. When the server stalls, the tester stops sending — so it omits exactly the requests that would have experienced the stall, in coordination with the server's misbehaviour. The few samples it does record during the stall are drowned among thousands of fast ones.

The simulation below reproduces it. A server answers in 1 ms, but freezes for 1.5 s four times during a 60 s test — a stop-the-world GC pause or a table lock. The tester intends to send 100 requests per second. The closed loop waits for each response; the open model sends on schedule and measures from the intended send time:

The server and its four freezes are identical; only the measurement differs. The closed loop recorded ~600 fewer samples (the ones it never sent during the freezes) and reports a perfect p99 of 1 ms. In reality, about one request in ten during that minute would have waited between a few milliseconds and 1.5 s — the corrected p90 is ~150 ms and the p99 is over a second. An SLO checked with the first row passes; an SLO checked with the second fails. Users experience the second.

How to avoid it:

• Use open-model generators for open-world traffic: k6 constant-arrival-rate/ramping-arrival-rate, Gatling injectOpen, wrk2, vegeta. They start requests on schedule regardless of earlier responses (as long as they have free VUs/connections — watch dropped_iterations).
• Measure from the intended start time when you control the client (HdrHistogram's recordValueWithExpectedInterval, wrk2's correction).
• Count timeouts at their full duration and as errors; a test that discards timed-out requests omits the worst samples too.
• Beware server-side metrics as the only source: a server's own latency histogram starts the clock when the request is accepted, missing the time spent in the TCP accept queue or the load balancer. Compare client-side and server-side percentiles; a large gap means queuing in front of the service.

Think time and pacing

Real users pause between actions. Think time (sleep() between requests inside an iteration) makes a session realistic; pacing controls how often a whole iteration starts.

• In a closed model, think time reduces throughput for a given number of VUs (\(X = N/(R+Z)\)). Removing it to "generate more load" changes the population you are simulating.
• In an open model, the arrival rate fixes the throughput; think time only increases the number of VUs busy at once (VUs ≈ rate × iteration duration) — and therefore the number of concurrent sessions, open connections and held seats, which is realistic and often exactly where systems break.
• Randomize think times (1 + Math.random() * 3): constant pauses synchronize VUs into artificial waves.

Running, observing and correlating

A load test produces two sets of data that must be read together: the client-side results (k6) and the server-side telemetry (metrics, traces, logs, profiles). Run them on the same time axis:

# stream k6 metrics to Prometheus (remote write) and tag the run
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS="p(50),p(95),p(99),max" \
k6 run --out experimental-prometheus-rw --tag testid=onsale-2027-03-14-run2 \
       -e BASE_URL=https://perf.boxoffice.example -e PEAK=1200 k6/load.js

# or watch the built-in web dashboard and export an HTML report
K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=report.html k6 run k6/load.js

With k6 and the services' OpenTelemetry metrics in the same Grafana (the grafana/otel-lgtm stack of Hands-on 8), one dashboard shows the k6 arrival rate and p99 next to each service's RED metrics and each resource's USE metrics. Slow requests are found through trace exemplars, and their spans show where the time went. For tests beyond one machine, the k6-operator runs a test as several parallel pods in a Kubernetes cluster.

Load and chaos together

Failures rarely arrive on a quiet afternoon; they arrive at the peak, when retries, timeouts and pools are all under pressure. Testing resilience patterns without load proves little: a circuit breaker that never sees 2 000 concurrent requests has not been tested. Combine a load test with fault injection⁴, each with a hypothesis written in advance (Chaos engineering):

k6 run -e PEAK=800 k6/load.js &                       # steady load in the background
sleep 300                                             # 5 minutes of baseline
toxiproxy-cli toxic add --type latency --attribute latency=3000 --attribute jitter=500 \
              --toxicName slow-payment payment        # inject: provider answers in ~3 s
sleep 300                                             # 5 minutes degraded
toxiproxy-cli toxic remove --toxicName slow-payment payment   # recover
wait                                                  # compare the three phases in Grafana

With xk6-disruptor (a k6 extension), the fault is part of the test script and targets a Kubernetes Service directly:

import { ServiceDisruptor } from 'k6/x/disruptor';
import { checkout } from './lib/boxoffice.js';
export { checkout };

export const options = {
  scenarios: {
    load:  { executor: 'constant-arrival-rate', exec: 'checkout', rate: 200, timeUnit: '1s',
             duration: '15m', preAllocatedVUs: 400, maxVUs: 2000 },
    fault: { executor: 'shared-iterations', exec: 'disrupt', iterations: 1, vus: 1, startTime: '5m' },
  },
  thresholds: { 'http_req_duration{name:POST /orders}': ['p(99)<1500'] },
};

export function disrupt() {
  const payment = new ServiceDisruptor('payment', 'boxoffice');
  payment.injectHTTPFaults({ averageDelay: '3s', errorRate: 0.1, errorCode: 503 }, '5m');
}

export const options = {
  scenarios: {
    pay: {
      executor: 'constant-arrival-rate',
      exec: 'pay',
      rate: Number(__ENV.PAY_RATE || 20),
      timeUnit: '1s',
      duration: DURATION,
      preAllocatedVUs: 50,
      maxVUs: 600,
    },
    browse: {
      executor: 'constant-arrival-rate',
      exec: 'browse',
      rate: 20,
      timeUnit: '1s',
      duration: DURATION,
      preAllocatedVUs: 20,
      maxVUs: 300,
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    'http_req_duration{name:pay}': ['p(95)<1500'],
    'http_req_duration{name:browse}': ['p(95)<200'], // the collateral damage
    'http_req_failed{name:browse}': ['rate<0.01'],
  },
};

Try it: the scenario below has a slow third-party dependency and callers without timeouts, running with the chaos profile. Run it and watch threads (or concurrency) pile up in the callers until unrelated requests fail too — a cascading failure. Then enable the timeout and the circuit breaker on the call to the dependency and run again: the failure should stay contained to the feature that needs the dependency.

From the playground to real tools

The playground's load profiles are the same six test types (plus chaos), applied to a queueing simulation. Use the playground to build intuition and to form hypotheses quickly; confirm them with real tools on real services.

What the playground simplifies: it models capacity, service times, queues, timeouts, retries and failures, but not JIT warm-up, garbage collection, lock contention inside a database, network variance or the cost of your actual code. A design that fails in the playground will fail in reality; a design that passes in the playground still has to be measured.

Other tools

All the tools below can express the six test types; they differ in scripting language, support for the open model, distribution and reporting⁵.

public class HoldSimulation extends Simulation {
  HttpProtocolBuilder httpProtocol = http.baseUrl("http://localhost:8080");

  ScenarioBuilder holds = scenario("hold")
      .exec(http("GET /events/{id}/seats").get("/events/42/seats?available=true").check(status().is(200)))
      .pause(1, 4)
      .exec(http("POST /holds").post("/holds")
          .body(StringBody("{\"eventId\":42,\"seatIds\":[\"A-12\"]}")).asJson());

  {
    setUp(holds.injectOpen(
            rampUsersPerSec(0).to(500).during(300),
            constantUsersPerSec(500).during(1200)))
        .protocols(httpProtocol)
        .assertions(
            global().responseTime().percentile(99.0).lt(300),
            global().failedRequests().percent().lt(0.1));
  }
}

from locust import HttpUser, task, between

class Fan(HttpUser):
    wait_time = between(1, 4)          # think time

    @task(7)
    def event_page(self):
        self.client.get("/events/42", name="GET /events/{id}")

    @task(3)
    def hold(self):
        self.client.post("/holds", json={"eventId": 42, "seatIds": ["A-12"]}, name="POST /holds")

jmeter -n -t onsale.jmx -l results.jtl -e -o report/    # headless run + HTML report

# wrk2: constant 5 000 req/s, latency corrected for coordinated omission
wrk2 -t4 -c200 -d60s -R5000 --latency http://localhost:8080/events/42

# vegeta: constant 2 000 req/s, latency histogram
echo "GET http://localhost:8080/events/42" | vegeta attack -rate=2000/s -duration=60s \
  | vegeta report -type='hist[0,50ms,100ms,200ms,500ms,1s]'

How it connects

flowchart LR
    SLO["SLOs per endpoint"] --> THR["k6 thresholds:<br/>pass or fail"]
    WM{"workload model"} -->|"open: arrival rate"| OPEN["fans do not wait<br/>for each other"]
    WM -->|"closed: virtual users"| CLOSED["coordinated omission<br/>hides the queue"]
    OPEN --> TYPES["smoke, load, stress,<br/>spike, soak, breakpoint"]
    THR --> TYPES
    TYPES --> KNEE["breakpoint finds the knee<br/>→ Capacity planning"]
    TYPES --> LAG["spike shows autoscaling lag<br/>→ Cloud-native Kubernetes"]
    TYPES --> CHAOS["load + Toxiproxy faults<br/>→ Reliability chaos"]
    TYPES --> CORR["correlate with metrics<br/>and traces → Hands-on 8"]

Palette components: client and mobile (what the load generator imitates) · apigw and lb (where graceful rejection — 429/503 — should happen past capacity) · app (the replicas under test) · queue (the waiting room in the spike test) · thirdparty (the dependency Toxiproxy slows down) · metrics and tracing (correlating k6 results with the server side). Scenarios: flash-sale (spike), cascading-failure (load and chaos together); the profiles smoke, load, stress, spike, soak and breakpoint mirror the six test types. Continues in: Capacity planning, Cloud-native — Kubernetes, Reliability — chaos, Reliability — patterns, Estimation — percentiles, Hands-on 2.