Capacity Planning

The business asks a simple question: "Can we put the stadium show on sale on March 14 at 10:00?" Behind it are engineering questions with numeric answers: how many replicas of each service, how big the database, how many connections, what admission rate for the waiting room, how early to scale up — and how much it costs. Capacity planning² turns the measurements of the previous pages into those numbers, with enough margin to survive what you did not predict.

From measurements to replicas

Capacity planning is arithmetic on three inputs you now know how to obtain:

1. Demand — the peak arrival rate per endpoint, from the workload model (forecast × safety factor).
2. Capacity per unit — the knee of one pod (or one database) from a breakpoint test: the highest rate at which it still meets the SLO.
3. Headroom — how far below the knee you want to run, and which failures you must survive.

Little's Law sizes everything that holds a request

Replicas are not the only resource. Every component that holds a request while it is in progress — threads, connections, sockets, VUs, queue slots — must be sized for the concurrency, and Little's Law gives it directly: concurrency = arrival rate × time held¹.

A capacity model for the on-sale

The model below combines the workload model's peak rates with per-pod knees measured in breakpoint tests (all values are illustrative assumptions — replace them with your measurements). It plans to run at most at 75 % of the knee and spreads replicas over three availability zones so that losing one zone still leaves enough capacity:

What the model says:

• The waiting room saves capacity where it is most expensive. Capping admitted holds at 3 000/s (below the database's knee) cuts inventory and order replicas by about a third — and, more importantly, keeps the load on the single primary database bounded.
• Zone resilience costs 50 % more replicas at the peak. That is the price of "a zone outage at 10:01 does not stop the sale". It may be worth it on on-sale day and not on a Tuesday — autoscaling bounds can differ by day.
• The connection count is a red flag. Close to a thousand client connections cannot go directly to PostgreSQL; PgBouncer (transaction pooling) multiplexes them onto a few dozen server connections — the number Little's Law actually requires.
• Pods become nodes become dollars. About a hundred pods at the peak is roughly the "+24 nodes for 2 hours" of the FinOps cost model.

Headroom: why not run at 100 % of the knee?

Typical targets run services at 50–75 % of their knee at the expected peak; the right number depends on how fast you can add capacity and how expensive a breach is. For BoxOffice's on-sale — minutes long, highly visible, impossible to repeat — generous headroom is cheap insurance.

Autoscaling lag: the minutes that matter

Autoscaling reacts to demand; it does not anticipate it. A rough timeline for a JVM service on Kubernetes when the spike starts:

During that window the backlog grows at the difference between demand and capacity. If holds jump from 500/s to 4 500/s while the deployment can serve 1 100/s for two minutes, the backlog is \((4\,500 - 1\,100) \times 120 \approx 408\,000\) requests — far beyond any timeout. They become errors, retries make it worse, and the fans who arrived first get the worst experience. Hence the rules for known peaks:

• Pre-scale before the event (a scheduled scale-up at 09:45; KEDA's cron scaler; raised minReplicas), and keep spare node capacity.
• Admission control (waiting room, rate limiting) so that excess demand waits outside the system, in order, instead of timing out inside it³.
• Warm up the new capacity: caches pre-loaded with the event, connection pools filled (minimum-idle = maximum-pool-size), a warm-up request to each new pod before it becomes ready.

Serverless platforms shorten the lag but do not remove it: new function instances have cold starts, and providers limit how fast concurrency may grow and how much concurrency an account may use (check the current quotas of your provider).

Try it: the scenario below sends a burst to a serverless component. Run it with the default settings and watch the cold-start latency at the start of the burst and the throttled (rejected) requests when the concurrency limit is reached. Then raise the limit or add provisioned (pre-warmed) concurrency, and compare the first seconds of the burst — the serverless version of pre-scaling.

Forecasting and the capacity review

Capacity planning is not done once. A simple, sustainable process:

• Forecast demand from history: trend (growth per month), seasonality (weekends, holidays) and events (on-sales are announced in advance — use the calendar, not only the trend).
• Re-measure capacity per unit after every significant change; a new feature can move the knee more than any infrastructure change. The performance budget in CI (below) catches most regressions before they reach production.
• Review monthly: peak utilization vs plan, headroom left, cost per unit of demand, upcoming events. Capacity, SLOs and cost belong in the same meeting.
• Prefer elasticity to precision: a system that scales out in one minute needs a less precise forecast than one that needs a week to order hardware.

Performance budgets in CI

A performance regression that reaches production is found by fans. A performance budget is a limit — per endpoint, per page, per build — that the pipeline enforces on every change, the same way tests enforce correctness⁴. Useful budgets mix noisy and deterministic measures:

The k6 budget is a short, open-model test against a production-like stack started inside the job. It is not a capacity test — CI runners are small and noisy — but it reliably catches the big regressions: a missing index, an N+1, a synchronous call added to the hot path.

import { textSummary } from 'https://jslib.k6.io/k6-summary/0.0.2/index.js';
import { browse, hold } from './lib/boxoffice.js';
export { browse, hold };

export const options = {
  scenarios: {
    browse: { executor: 'constant-arrival-rate', exec: 'browse', rate: 100, timeUnit: '1s',
              duration: '3m', preAllocatedVUs: 50, maxVUs: 200 },
    hold:   { executor: 'constant-arrival-rate', exec: 'hold', rate: 20, timeUnit: '1s',
              duration: '3m', preAllocatedVUs: 100, maxVUs: 300 },
  },
  thresholds: {
    'http_req_duration{name:GET /events/{id}}': ['p(95)<80'],
    'http_req_duration{name:POST /holds}':      ['p(95)<150'],
    http_req_failed: ['rate<0.001'],
    dropped_iterations: ['count==0'],
  },
};

export function handleSummary(data) {
  return {
    stdout: textSummary(data, { indent: ' ', enableColors: false }),
    'results/summary.json': JSON.stringify(data),
  };
}

name: perf-budget
on:
  pull_request:
  push: { branches: [ main ] }        # main's runs become the baseline

jobs:
  k6:
    runs-on: ubuntu-latest
    permissions: { contents: read, actions: read }
    steps:
      - uses: actions/checkout@v4
      - name: Start BoxOffice with production-like settings
        run: docker compose -f compose.yaml -f compose.perf.yaml up -d --build --wait
      - name: Seed a 60 000-seat event
        run: docker compose exec -T db psql -U postgres -d boxoffice -f /seed/onsale.sql
      - uses: grafana/setup-k6-action@v1
      - name: Warm up (not measured)
        run: k6 run --quiet -e BASE_URL=http://localhost:8080 --vus 5 --duration 30s k6/smoke.js
      - name: Performance budget (fails on threshold breach)
        run: mkdir -p results && k6 run -e BASE_URL=http://localhost:8080 k6/budget.js
      - name: Compare with the latest baseline from main
        if: github.event_name == 'pull_request'
        env: { GH_TOKEN: "${{ github.token }}" }
        run: |
          id=$(gh run list --workflow perf-budget --branch main --status success --limit 1 \
                 --json databaseId -q '.[0].databaseId')
          gh run download "$id" --name k6-summary --dir baseline || echo "no baseline yet"
          python3 k6/compare.py results/summary.json baseline/summary.json --max-regression 0.15
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: k6-summary, path: results/summary.json }

import argparse
import json
import os
import sys

parser = argparse.ArgumentParser(description="Compare k6 p95 per endpoint with a baseline")
parser.add_argument("current")
parser.add_argument("baseline")
parser.add_argument("--max-regression", type=float, default=0.15)
args = parser.parse_args()


def p95_by_endpoint(path):
    metrics = json.load(open(path))["metrics"]
    return {name: m["values"]["p(95)"] for name, m in metrics.items()
            if name.startswith("http_req_duration{name:")}


current = p95_by_endpoint(args.current)
baseline = p95_by_endpoint(args.baseline) if os.path.exists(args.baseline) else {}
failed = False
for name, value in sorted(current.items()):
    ref = baseline.get(name)
    if ref is None:
        print(f"{name}: {value:.1f} ms (no baseline)")
        continue
    change = (value - ref) / ref
    regression = change > args.max_regression
    failed |= regression
    print(f"{name}: {ref:.1f} -> {value:.1f} ms ({change:+.0%}){'  REGRESSION' if regression else ''}")
sys.exit(1 if failed else 0)

Two gates protect the budget: the absolute thresholds in budget.js (derived from the SLOs, with margin for the small runner) and the relative comparison with main (a 15 % slowdown fails even if the absolute threshold still passes). To keep the gate trusted rather than ignored:

• Fight noise: fixed arrival rate, warm-up step, the same runner type, and repeat once automatically before failing; a dedicated self-hosted runner reduces variance further.
• Prefer deterministic budgets where possible: a query-count assertion in an integration test never flakes, and catches the most common regression (N+1).
• Make failures actionable: upload the summary, link the trace of the slowest request, and say which endpoint regressed by how much.
• Run the full campaign elsewhere: the PR gate is a smoke alarm; the load, spike and breakpoint tests run nightly or before big events in the performance environment.

export const options = {
  scenarios: {
    warmup: { // JIT, connection pools and cache: not measured
      executor: 'constant-arrival-rate',
      rate: 20,
      timeUnit: '1s',
      duration: '20s',
      preAllocatedVUs: 20,
    },
    budget: { // the measured part: an open-model load the service must sustain
      executor: 'constant-arrival-rate',
      startTime: '20s',
      rate: Number(__ENV.RATE || 50),
      timeUnit: '1s',
      duration: '40s',
      preAllocatedVUs: 50,
      maxVUs: 200,
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    'http_req_duration{scenario:budget}': ['p(95)<300', 'p(99)<800'],
    'http_req_failed{scenario:budget}': ['rate<0.01'],
    'checks{scenario:budget}': ['rate>0.99'],
  },
};

performance:
  name: performance budget
  needs: test
  runs-on: ubuntu-latest
  timeout-minutes: 20
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
    - name: Start PostgreSQL, Redis and catalog
      run: docker compose -f .github/ci/compose.perf.yaml up -d --build --wait --wait-timeout 300
    - uses: grafana/setup-k6-action@43b9fc21641a76002687994433dd586f56e791b1 # v1.2.2
      with:
        k6-version: "1.8.2"
    - name: k6 budget (fails the job when a threshold is crossed)
      uses: grafana/run-k6-action@4082a08c7e40cf6d65863911077dfb60ae0a4f04 # v1.5.0
      env:
        BASE_URL: http://localhost:8082
      with:
        path: load/budget.js

How it connects

flowchart LR
    KNEE["measured knee per pod<br/>← Load testing"] --> REP["replicas = peak ÷<br/>(knee × target utilization)"]
    LL["Little's Law<br/>N = X × R"] --> CONN["threads, connections,<br/>PgBouncer"]
    REP --> HEAD["headroom:<br/>zone loss, 75 % of the knee"]
    HEAD --> LAG["autoscaling lag, cold starts<br/>→ pre-scale before 10:00"]
    LAG --> WR["waiting room admits what<br/>the database can take"]
    REP --> COST["pods → nodes → dollars<br/>→ Cloud-native FinOps"]
    FC["forecast and<br/>capacity review"] --> REP
    BUD["performance budget in CI<br/>guards the knee"] --> KNEE

Palette components: app (replicas per service) · sql (connections and the primary's limit) · queue (the waiting room's admission rate) · serverless (cold starts and concurrency limits) · metrics (utilization, the autoscaler's signal). Scenarios: serverless-burst, scale-out, flash-sale. Continues in: Estimation — queueing, Compute — serverless & autoscaling, Cloud-native — Kubernetes, Cloud-native — FinOps, Case study — ticketing.