Skip to content

b. Docker Compose

The cluster is a standalone project: it does not need the microservices of the previous hands-on. Create a new folder with these files:

πŸ“ pgcluster/
β”œβ”€β”€ YAML compose.yaml
└── πŸ“ scripts/
    β”œβ”€β”€ GNU Bash node-entrypoint.sh
    β”œβ”€β”€ GNU Bash init-primary.sh
    └── GNU Bash rejoin-as-replica.sh

rejoin-as-replica.sh is used only in Failover; create it now so the folder is complete.

The files

Source

compose.yaml
name: pgcluster

x-pg-env: &pg-env
  POSTGRES_USER: ${DB_USER:-store}
  POSTGRES_PASSWORD: ${DB_PASSWORD:-devpass}
  POSTGRES_DB: ${DB_NAME:-store}
  REPLICATION_USER: replicator
  REPLICATION_PASSWORD: ${REPLICATION_PASSWORD:-replpass}

x-pg-node: &pg-node
  image: postgres:17
  entrypoint: ["bash", "/scripts/node-entrypoint.sh"]
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
    interval: 2s
    timeout: 3s
    retries: 30

services:

  pg-primary:
    <<: *pg-node
    hostname: pg-primary
    # cluster_name: the node's name in queries and the name a replica reports to its primary
    command: ["postgres", "-c", "cluster_name=pg-primary"]
    environment:
      <<: *pg-env
    ports:
      - 5432:5432
    volumes:
      - primary_data:/var/lib/postgresql/data
      - ./scripts:/scripts:ro

  pg-replica1:
    <<: *pg-node
    hostname: pg-replica1
    command: ["postgres", "-c", "cluster_name=pg-replica1"]
    environment:
      <<: *pg-env
      PRIMARY_HOST: pg-primary  # makes this node a replica (see node-entrypoint.sh)
    ports:
      - 5433:5432
    volumes:
      - replica1_data:/var/lib/postgresql/data
      - ./scripts:/scripts:ro

  pg-replica2:
    <<: *pg-node
    hostname: pg-replica2
    command: ["postgres", "-c", "cluster_name=pg-replica2"]
    environment:
      <<: *pg-env
      PRIMARY_HOST: pg-primary
    ports:
      - 5434:5432
    volumes:
      - replica2_data:/var/lib/postgresql/data
      - ./scripts:/scripts:ro

  client:
    image: postgres:17
    profiles: [tools]
    entrypoint: ["psql"]
    environment:
      PGUSER: ${DB_USER:-store}
      PGPASSWORD: ${DB_PASSWORD:-devpass}
      PGDATABASE: ${DB_NAME:-store}

volumes:
  primary_data:
  replica1_data:
  replica2_data:
scripts/node-entrypoint.sh
#!/bin/bash
# Entrypoint of every node of the cluster.
#   PRIMARY_HOST unset -> primary: the official entrypoint creates the database
#                         and runs /docker-entrypoint-initdb.d (none here: see below).
#   PRIMARY_HOST set   -> replica: on the first start, clones the primary with
#                         pg_basebackup and starts as a hot standby.
set -euo pipefail

PGDATA=${PGDATA:-/var/lib/postgresql/data}

if [ -s "$PGDATA/PG_VERSION" ]; then
  echo "node: data directory already initialised, starting PostgreSQL"

elif [ -z "${PRIMARY_HOST:-}" ]; then
  echo "node: empty data directory, initialising a new primary"
  # Runs once, right after initdb: the replication role and its pg_hba rule.
  cp /scripts/init-primary.sh /docker-entrypoint-initdb.d/10-init-primary.sh

else
  echo "node: empty data directory, cloning $PRIMARY_HOST"
  until pg_isready -h "$PRIMARY_HOST" -U "$POSTGRES_USER" -q; do
    echo "node: waiting for $PRIMARY_HOST..."
    sleep 1
  done
  mkdir -p "$PGDATA"
  chown postgres:postgres "$PGDATA"
  chmod 700 "$PGDATA"
  # -X stream: also copy the WAL written during the backup
  # -R: write standby.signal + primary_conninfo, so the node starts as a standby
  PGPASSWORD="$REPLICATION_PASSWORD" gosu postgres \
    pg_basebackup -h "$PRIMARY_HOST" -U "$REPLICATION_USER" \
                  -D "$PGDATA" -X stream -R --progress
fi

exec docker-entrypoint.sh "$@"
scripts/init-primary.sh
#!/bin/bash
# Runs once, when the primary creates its database (docker-entrypoint-initdb.d).
# Everything here lives in the data directory, so pg_basebackup copies it to every replica.
set -euo pipefail

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<SQL
-- A role that may only stream WAL: replicas connect with it, never the application.
CREATE ROLE $REPLICATION_USER WITH REPLICATION LOGIN PASSWORD '$REPLICATION_PASSWORD';
-- WAL kept on disk for a replica that falls behind (or is stopped for a while).
ALTER SYSTEM SET wal_keep_size = '256MB';
-- Lets pg_rewind turn an old primary into a replica after a failover.
ALTER SYSTEM SET wal_log_hints = on;
SQL

# "all" in pg_hba.conf does not match replication connections: they need their own rule.
echo "host replication $REPLICATION_USER all scram-sha-256" >> "$PGDATA/pg_hba.conf"
scripts/rejoin-as-replica.sh
#!/bin/bash
# Turns a stopped old primary into a replica of the new primary.
# Usage (from the folder of compose.yaml, with the node stopped):
#   docker compose run --rm --no-deps --entrypoint bash pg-primary \
#     /scripts/rejoin-as-replica.sh pg-replica1
set -euo pipefail

NEW_PRIMARY=${1:?usage: rejoin-as-replica.sh <new-primary-host>}
PGDATA=${PGDATA:-/var/lib/postgresql/data}

# pg_rewind copies back only the blocks that changed since the timelines split,
# discarding the writes the old primary made that never reached the new one.
gosu postgres pg_rewind --target-pgdata="$PGDATA" --progress \
  --source-server="host=$NEW_PRIMARY user=$POSTGRES_USER password=$POSTGRES_PASSWORD dbname=$POSTGRES_DB"

# Same as pg_basebackup -R: start as a standby, streaming from the new primary.
gosu postgres touch "$PGDATA/standby.signal"
echo "primary_conninfo = 'host=$NEW_PRIMARY user=$REPLICATION_USER password=$REPLICATION_PASSWORD'" \
  | gosu postgres tee -a "$PGDATA/postgresql.auto.conf" > /dev/null
echo "rejoin: $(hostname) will follow $NEW_PRIMARY on its next start"

Line endings on Windows

The scripts run inside Linux containers. If your editor saves them with Windows line endings (CRLF), bash fails with $'\r': command not found. Save them as LF (VS Code: bottom-right corner, CRLF β†’ LF).

How it works

compose.yaml

  • x-pg-env and x-pg-node are YAML anchors: the three nodes share the image, the environment, the entrypoint and the health check (<<: *pg-node). Only the name, the port and PRIMARY_HOST change.
  • cluster_name names each node. You will read it back with SELECT current_setting('cluster_name') to know which server answered, and a replica uses it as its name on the primary (application_name in pg_stat_replication) β€” synchronous replication refers to replicas by that name.
  • The health check uses -h 127.0.0.1 on purpose: while the image creates the database, a temporary server listens only on a Unix socket. Checking over TCP marks the node healthy only when the real server is up.
  • Each node has its own volume: a replica is a full copy, not a shared disk.
  • The replicas have no depends_on: node-entrypoint.sh already waits for the primary. With depends_on, docker compose start pg-replica1 would also restart a primary you stopped on purpose β€” which would ruin the failover experiments.
  • client is in the tools profile: docker compose up does not start it. docker compose run --rm client ... runs psql inside the cluster network, so you do not need PostgreSQL installed on your machine.

node-entrypoint.sh decides the role of a node when its data directory is empty:

flowchart TD
    start(["container starts"]) --> data{"data directory\nalready initialised?"}
    data -->|yes| run["start PostgreSQL\n(primary or standby, as it was)"]
    data -->|no| role{"PRIMARY_HOST set?"}
    role -->|no| init["initdb + init-primary.sh\n→ new primary"]
    role -->|yes| wait["wait for the primary"] --> clone["pg_basebackup -X stream -R\n→ copy of the primary"]
    clone --> standby["start as hot standby\n(read-only, streaming WAL)"]
    init --> run

pg_basebackup copies the whole data directory of the primary over a replication connection. -X stream also copies the WAL written during the copy, and -R writes standby.signal and primary_conninfo (where to stream from) into the new data directory: when PostgreSQL starts and finds standby.signal, it starts as a standby.

init-primary.sh runs once, when the primary creates its database. It creates the replicator role (allowed only to stream WAL), sets wal_keep_size and wal_log_hints with ALTER SYSTEM, and adds a pg_hba.conf rule for replication connections. All of this is stored in the data directory, so pg_basebackup copies it to every replica β€” any of them can be promoted later and already has the same configuration.

Start the cluster

From the pgcluster/ folder:

docker compose up -d --wait

--wait returns only when every node is healthy. Then:

docker compose ps

You should see three nodes, all healthy:

NAME                      SERVICE       STATUS                    PORTS
pgcluster-pg-primary-1    pg-primary    Up 20 seconds (healthy)   0.0.0.0:5432->5432/tcp
pgcluster-pg-replica1-1   pg-replica1   Up 20 seconds (healthy)   0.0.0.0:5433->5432/tcp
pgcluster-pg-replica2-1   pg-replica2   Up 20 seconds (healthy)   0.0.0.0:5434->5432/tcp

Look at how a replica was born:

docker compose logs pg-replica1
node: empty data directory, cloning pg-primary
...
LOG:  entering standby mode
LOG:  consistent recovery state reached at 0/30001D0
LOG:  database system is ready to accept read-only connections
LOG:  started streaming WAL from primary at 0/4000000 on timeline 1

Check the replication

On the primary, pg_stat_replication has one row per connected replica:

docker compose exec pg-primary psql -U store -c "SELECT application_name, client_addr, state, sync_state FROM pg_stat_replication;"
 application_name | client_addr | state     | sync_state
------------------+-------------+-----------+------------
 pg-replica1      | 172.18.0.3  | streaming | async
 pg-replica2      | 172.18.0.4  | streaming | async

On a replica, pg_is_in_recovery() is true (a standby is permanently "recovering": replaying WAL) and pg_stat_wal_receiver shows where it streams from:

docker compose exec pg-replica1 psql -U store -c "SELECT pg_is_in_recovery();" -c "SELECT status, sender_host FROM pg_stat_wal_receiver;"
 pg_is_in_recovery
-------------------
 t

  status   | sender_host
-----------+-------------
 streaming | pg-primary

Connecting from your machine

Each node is published on its own port: 5432 (primary), 5433 and 5434 (replicas), user store, password devpass, database store. You can open the three side by side in DBeaver, pgAdmin or the database view of your IDE.

Troubleshooting

Symptom Cause and fix
$'\r': command not found in the logs the scripts have CRLF line endings: save them as LF and run docker compose down -v && docker compose up -d --wait
a replica restarts in a loop with no pg_hba.conf entry for replication connection the primary was created before init-primary.sh existed: docker compose down -v and start again
port is already allocated on 5432 the db of a previous hands-on is running: stop it (docker compose down in that folder) or change the published ports
a replica keeps waiting for pg-primary... the primary is not up: docker compose logs pg-primary

Starting over

The volumes keep the cluster state, including any failover you did. To return to the initial state (one primary, two replicas), remove them:

docker compose down -v
docker compose up -d --wait