Hands-on 7 — Load Balancer

One gateway for everything

• One process, one thread pool, no redundancy — the gateway becomes the bottleneck.
• If it crashes, the whole platform is unreachable.
• The fix: run several identical instances and spread the requests — horizontal scaling.

Forward × reverse proxy

Layer 4 × Layer 7

Load balancing algorithms

Architecture

Steps: 1 nginx.conf → 2 compose.yaml (lb + gateway replicas) → 3 run and verify.

1 · setup/nginx.conf

http {
    upstream gateways {
        server gateway:8080;
    }
    server {
        listen 80;
        location / {
            proxy_pass         http://gateways;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

Excerpt; the file also sets Host and X-Forwarded-Proto, plus events { worker_connections 1024; }.

Key directives

Algorithm inside upstream: none = Round Robin · least_conn; · ip_hash; · random; · server gateway:8080 weight=3;

How Nginx finds the replicas

1. Compose starts 3 replicas of gateway.
2. Docker's internal DNS resolves the name gateway to every replica IP.
3. Nginx queries that record once, at startup, and builds the upstream pool.

3 · Run and verify

docker compose up -d --build
docker compose ps -a        # lb + gateway-1, -2, -3, no ports on the gateways
for i in {1..9}; do curl -s http://localhost:8080/health-check; echo; done
docker compose logs gateway --tail 30

• /health-check answers the replica's hostname: the output cycles through the three gateways.
• All requests on one replica? Nginx started before the replicas were ready → docker compose restart lb.

Scale at runtime

docker compose up -d --scale gateway=5     # add replicas, no rebuild
docker compose exec lb nginx -s reload     # re-query Docker DNS
docker compose ps gateway                  # confirm the count
docker compose up -d --scale gateway=1     # scale down

Takeaways

1. A single gateway is a bottleneck and a single point of failure; replicas fix both.
2. Nginx works as a reverse proxy: the only exposed component, hiding the backends.
3. L4 routes by IP and port; L7 (Nginx) understands HTTP and can route by path, header or cookie.
4. The algorithm lives in the upstream block: Round Robin by default, or least_conn;, ip_hash;, random;.
5. Docker DNS turns gateway:8080 into all replica IPs — but Nginx resolves it only at startup.
6. After scaling, reload Nginx (nginx -s reload) so it sees the new replicas.