3. Autoscaling

Lab 3 scaled catalog by hand: three replicas behind Nginx, whatever the load. The Horizontal Pod Autoscaler (HPA) closes the loop — it measures, compares with a target and changes the number of replicas — the autoscaling control loop of the compute class, running every 15 seconds.

The rule

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: catalog
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: catalog
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60     # % of the CPU *request* (250m): scale when pods average > 150m
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0  # react at once to load
      selectPolicy: Max
      policies:
        - type: Percent
          value: 100                 # at most double...
          periodSeconds: 15
        - type: Pods
          value: 2                   # ...or +2 pods, whichever is more, every 15 s
          periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 120  # wait 2 min of lower load before removing pods
      policies:
        - type: Pods
          value: 1
          periodSeconds: 30

The HPA's core formula is simple:

With 2 pods at 150% of their CPU request and a 60% target: ⌈2 × 150 / 60⌉ = 5 pods. Utilization is a percentage of the request (250m), not of the node: a pod using 150m is at 60%.

Load it

The k6 script ramps the arrival rate (open model) through the Ingress, holds it, and comes back down. The watcher logs what the HPA sees and decides every 5 seconds.

import http from 'k6/http';
import { check } from 'k6';

const BASE_URL = __ENV.BASE_URL || 'http://localhost'; // the Ingress
const PEAK = Number(__ENV.PEAK || 300);

export const options = {
  scenarios: {
    ramp: {
      executor: 'ramping-arrival-rate', // open model: arrivals keep coming even if pods are slow
      startRate: 20,
      timeUnit: '1s',
      preAllocatedVUs: 100,
      maxVUs: 1000,
      stages: [
        { target: PEAK, duration: '3m' }, // ramp up: watch the HPA react
        { target: PEAK, duration: '4m' }, // hold: does the latency come back down?
        { target: 20, duration: '1m' },   // ramp down
        { target: 20, duration: '3m' },   // quiet: watch the scale-down stabilization
      ],
    },
  },
  summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
  thresholds: {
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(95)<500'],
  },
};

export default function () {
  const id = 1 + Math.floor(Math.random() * 200000);
  const res = http.get(`${BASE_URL}/catalog/events/${id}`, { tags: { name: 'GET /catalog/events/{id}' } });
  check(res, { 'status 200': (r) => r.status === 200 });
}

#!/bin/sh
# Logs time, desired/current replicas, CPU and ready pods every 5 s. Usage: sh scripts/watch-hpa.sh > results/lab10-hpa.csv
echo "time,cpu,desired,current,ready"
while true; do
  HPA=$(kubectl -n boxoffice get hpa catalog -o jsonpath='{.status.currentMetrics[0].resource.current.averageUtilization},{.status.desiredReplicas},{.status.currentReplicas}')
  READY=$(kubectl -n boxoffice get deploy catalog -o jsonpath='{.status.readyReplicas}')
  echo "$(date +%T),$HPA,$READY"
  sleep 5
done

Start from warm pods. A JVM that has just started burns CPU compiling code, and the HPA scales on that CPU too: in a test run started right after a rollout, PEAK=250 took catalog to 8 replicas, liveness probes timed out on the busy CPUs, pods restarted, and k6 saw 26% errors. The same run with warm pods never left 2 replicas. Warm them up, then wait until the HPA is back at 2 replicas:

k6 run -q -e RATE=200 -e DURATION=1m load/catalog-otel.js > /dev/null   # lab 8's script
kubectl -n boxoffice get hpa catalog -w                    # wait for REPLICAS 2 and a low cpu, then Ctrl+C

Three terminals:

mkdir -p results && sh scripts/watch-hpa.sh | tee results/lab10-hpa.csv   # 1: the control loop
kubectl -n boxoffice get pods -l app=catalog -w            # 2: pods appearing
k6 run -e PEAK=1000 load/catalog-hpa.js                    # 3: the load (≈ 11 min)

Adjust PEAK to your machine. The HPA reacts to CPU, not to latency: PEAK must push two warm pods clearly above 60% of their CPU request (150m each, kubectl -n boxoffice top pods), but stay below what your laptop can generate. On a test laptop (8 CPUs for Docker), two warm pods were at about 40% with 250 req/s and crossed 60% at about 600 req/s, hence PEAK=1000.

Pitfalls

• No requests, no HPA. Utilization is relative to resources.requests.cpu; without requests the HPA reports <unknown> and does nothing.
• JVM warm-up looks like load. A starting JVM burns CPU (class loading, JIT): new pods can push the average up and trigger more scale-up. The startup probe keeps unready pods out of traffic but their CPU still counts; the stabilization window and the scale-up policy limit the damage.
• The database does not autoscale. Eight catalog pods × 10 connections = 80 connections to one PostgreSQL. Scaling the stateless tier moves the bottleneck to the stateful one — check max_connections (or use the PgBouncer of lab 6).
• Node capacity. If the pods requested do not fit on the nodes, they stay Pending. In the cloud a cluster autoscaler (or Karpenter) adds nodes, which takes minutes, not seconds.

Next: Ingress