Skip to content

3. Performance Gate

Tests catch wrong answers; they do not catch a change that makes the right answer ten times slower. A performance budget turns the latency you promised (lab 8's SLO, the capacity plan) into a build check: CI starts the service, runs a short k6 test with thresholds, and fails the pull request when a threshold is crossed.

A fixed environment

Comparisons only make sense if every run uses the same environment. CI starts a small, dedicated stack β€” PostgreSQL, Redis and catalog with the same CPU limit as in docker-compose.yaml β€” from its own Compose file:

.github/ci/compose.perf.yaml
# The performance-budget environment of CI (lab 11): the same small stack on every run, so that
# runs are comparable. Paths are relative to this file (.github/ci/).
name: perf

services:
  db:
    image: postgres:17
    environment:
      POSTGRES_DB: boxoffice
      POSTGRES_USER: boxoffice
      POSTGRES_PASSWORD: boxoffice
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U boxoffice -d boxoffice"]
      interval: 5s
      timeout: 3s
      retries: 20

  redis:
    image: redis:8-alpine
    command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru", "--save", ""]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 20

  catalog:
    build: ../../services/catalog
    environment:
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: boxoffice
      DB_USER: boxoffice
      DB_PASSWORD: boxoffice
      REDIS_HOST: redis
      REDIS_PORT: 6379
    ports:
      - "8082:8082"
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "1.0"          # the same limit as in docker-compose.yaml: budgets are per replica
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8082/actuator/health/readiness || exit 1"]
      interval: 5s
      timeout: 3s
      start_period: 60s
      retries: 30

docker compose … up --wait returns only when every healthcheck passes, so k6 never measures a JVM that is still starting.

The budget

load/budget.js
// Lab 11: performance budget of catalog, run by CI on every push and pull request.
// A crossed threshold makes k6 exit with code 99, which fails the "performance budget" job.
//   k6 run load/budget.js            (BASE_URL defaults to http://localhost:8082)
import http from 'k6/http';
import { check } from 'k6';

const BASE_URL = __ENV.BASE_URL || 'http://localhost:8082';

export const options = {
  scenarios: {
    warmup: { // JIT, connection pools and cache: not measured
      executor: 'constant-arrival-rate',
      rate: 20,
      timeUnit: '1s',
      duration: '20s',
      preAllocatedVUs: 20,
    },
    budget: { // the measured part: an open-model load the service must sustain
      executor: 'constant-arrival-rate',
      startTime: '20s',
      rate: Number(__ENV.RATE || 50),
      timeUnit: '1s',
      duration: '40s',
      preAllocatedVUs: 50,
      maxVUs: 200,
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    'http_req_duration{scenario:budget}': ['p(95)<300', 'p(99)<800'],
    'http_req_failed{scenario:budget}': ['rate<0.01'],
    'checks{scenario:budget}': ['rate>0.99'],
  },
};

export default function () {
  const id = 1 + Math.floor(Math.random() * 200000);
  const res = http.get(`${BASE_URL}/catalog/events/${id}`, { tags: { name: 'GET /catalog/events/{id}' } });
  check(res, { 'status 200': (r) => r.status === 200 });
}
Choice Why
a 20 s warmup scenario, not measured JIT compilation, connection pools and the cache warm up; thresholds only apply to scenario:budget
open model (constant-arrival-rate) the arrival rate does not slow down when the service does, as with real users
p95 < 300 ms and p99 < 800 ms the typical request and the tail; generous for one runner, so that only real regressions fail
failed < 1%, checks > 99% a fast 500 is not a success

The performance budget job of ci.yaml (step 1) installs k6 with grafana/setup-k6-action, runs the script with grafana/run-k6-action, and prints the service logs if it fails. A crossed threshold makes k6 exit with code 99 and the job fails.

Run it locally

docker compose -f .github/ci/compose.perf.yaml up -d --build --wait --wait-timeout 300
k6 run load/budget.js
docker compose -f .github/ci/compose.perf.yaml down -v

You should see all thresholds with βœ“. A test run on a laptop:

  β–ˆ THRESHOLDS

    checks{scenario:budget}
    βœ“ 'rate>0.99' rate=100.00%

    http_req_duration{scenario:budget}
    βœ“ 'p(95)<300' p(95)=6.7ms
    βœ“ 'p(99)<800' p(99)=9.13ms

    http_req_failed{scenario:budget}
    βœ“ 'rate<0.01' rate=0.00%

Break it on purpose

Prove the gate works. On a new branch, add an artificial delay to the event endpoint of services/catalog/src/main/java/boxoffice/catalog/EventController.java β€” the first line of the get method:

try { Thread.sleep(400); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }

Run the local commands again: p(95)<300 now shows βœ— and k6 ends with level=error msg="thresholds on metrics 'http_req_duration{scenario:budget}' have been crossed". Push the branch and open a pull request: the performance budget check is red and the merge button warns you. Close the pull request without merging (and delete the branch) β€” it was only an experiment.

A noisy gate is worse than no gate

Shared CI runners vary from run to run. Set thresholds with a safety margin over your normal numbers (here, p95 is ~7 ms against a 300 ms budget), keep the test short but long enough to have thousands of samples, and look at the trend of the numbers over many runs before tightening them. A gate that fails randomly trains people to re-run until green.

Next: GitOps with Argo CD