Outbox, Sagas & CQRS

A BoxOffice purchase touches three services that each own their data: order (the order), inventory (the seats) and payment (the charge, through an external provider). In a monolith with one database, the purchase would be one ACID transaction: hold the seats, charge, confirm — all or nothing. Split into services, there is no single database and no single transaction. The classic answer, a distributed transaction with two-phase commit (2PC), does not fit either: the payment provider will not join your transaction, and 2PC blocks every participant while a coordinator is unreachable.

This page collects the patterns that replace the distributed transaction in event-driven systems. Each one answers a precise question:

The dual-write problem

The simplest way to publish OrderConfirmed is to save the order and then send the event. That is two writes to two different systems — a dual write — and no transaction covers both:

sequenceDiagram
    participant O as order
    participant DB as order DB
    participant K as Kafka
    O->>DB: BEGIN · UPDATE orders SET status = 'CONFIRMED' · COMMIT
    Note over O: pod killed by a deploy
    O--xK: publish OrderConfirmed (never sent)
    Note over DB,K: the order is confirmed, but nobody is told: no e-mail, no ticket, no analytics

Swapping the order of the writes does not help:

Transactional outbox

The transactional outbox¹ removes the dual write by writing the event into the same database, in the same local transaction, as the business change. A separate relay later reads the outbox table and publishes to the broker.

sequenceDiagram
    participant O as order
    participant DB as order DB
    participant R as outbox relay
    participant K as Kafka
    O->>DB: BEGIN
    O->>DB: UPDATE orders SET status = 'CONFIRMED'
    O->>DB: INSERT INTO outbox (OrderConfirmed)
    O->>DB: COMMIT (both rows or neither)
    R->>DB: SELECT unpublished rows
    R->>K: publish OrderConfirmed (key o-7731)
    K-->>R: ack
    R->>DB: mark row as published

The local transaction guarantees that the order and its event exist together or not at all. The relay guarantees that every committed event is eventually published — at least once: if it crashes after publishing but before marking the row, it publishes the row again on restart. Consumers must be idempotent anyway (Delivery semantics).

The outbox table:

CREATE TABLE outbox (
    id              BIGSERIAL   PRIMARY KEY,           -- publication order
    event_id        UUID        NOT NULL UNIQUE,       -- consumers deduplicate on it
    aggregate_type  TEXT        NOT NULL,              -- 'order'
    aggregate_id    TEXT        NOT NULL,              -- 'o-7731', becomes the Kafka key
    type            TEXT        NOT NULL,              -- 'OrderConfirmed'
    payload         JSONB       NOT NULL,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at    TIMESTAMPTZ
);
CREATE INDEX outbox_unpublished ON outbox (id) WHERE published_at IS NULL;

The business code writes both rows in one transaction:

@Service
@RequiredArgsConstructor
public class OrderService {

    private final OrderRepository orders;
    private final OutboxRepository outbox;

    @Transactional
    public void confirm(String orderId) {
        Order order = orders.findByIdForUpdate(orderId);
        order.confirm();                                   // PENDING -> CONFIRMED
        orders.save(order);
        outbox.save(OutboxEvent.of("order", order.id(), "OrderConfirmed",
                                   OrderConfirmed.from(order)));   // same transaction
    }
}

The relay has two common implementations:

@Scheduled(fixedDelay = 200)
@Transactional
public void relay() {
    List<OutboxRow> rows = jdbc.query("""
        SELECT id, aggregate_id, payload FROM outbox
         WHERE published_at IS NULL
         ORDER BY id
         LIMIT 100
         FOR UPDATE SKIP LOCKED
        """, OutboxRow.MAPPER);
    for (OutboxRow row : rows) {
        kafka.send("order-events", row.aggregateId(), row.payload()).join();  // wait for the ack
        jdbc.update("UPDATE outbox SET published_at = now() WHERE id = ?", row.id());
    }
}

{
  "name": "order-outbox",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "order-db",
    "database.port": "5432",
    "database.user": "debezium",
    "database.password": "debezium",
    "database.dbname": "order",
    "topic.prefix": "order",
    "plugin.name": "pgoutput",
    "table.include.list": "public.outbox",
    "transforms": "outbox",
    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
    "transforms.outbox.table.field.event.id": "event_id",
    "transforms.outbox.table.field.event.key": "aggregate_id",
    "transforms.outbox.route.by.field": "aggregate_type",
    "transforms.outbox.route.topic.replacement": "${routedByValue}-events"
  }
}

@Transactional
public PlaceResult place(OrderRequest req) {
    int inserted = jdbc.sql("""
            INSERT INTO orders (id, show_id, hold_id, customer_email, amount, status)
            VALUES (?, ?, ?, ?, ?, 'PENDING_PAYMENT')
            ON CONFLICT (hold_id) DO NOTHING
            """)
            .params(UUID.randomUUID(), req.showId(), req.holdId(), req.customerEmail(), req.amount())
            .update();

    OrderView order = jdbc.sql(SELECT + " WHERE hold_id = ?")
            .param(req.holdId())
            .query(OrderView.class)
            .single();

    if (inserted == 0) {
        // same hold already ordered: idempotent replay, no second event
        return new PlaceResult(order, false);
    }

    OrderEvent event = OrderEvent.placed(order);
    if (props.notifyMode() == NotifyMode.SYNC) {
        notifications.send(event); // BEFORE: the fan waits for the e-mail
    } else {
        outbox.append(event.eventId(), "order", order.id(), event.eventType(), event);
    }
    return new PlaceResult(order, true);
}

@Scheduled(fixedDelayString = "${boxoffice.order.relay-interval-ms:200}")
@Transactional
public void relay() throws Exception {
    // SKIP LOCKED: several order replicas can relay in parallel without taking the same rows
    List<Pending> batch = jdbc.sql("""
            SELECT id, aggregate_id, event_type, payload::text AS payload
            FROM outbox
            WHERE published_at IS NULL
            ORDER BY created_at
            LIMIT 100
            FOR UPDATE SKIP LOCKED
            """)
            .query(Pending.class)
            .list();
    if (batch.isEmpty()) {
        return;
    }

    List<CompletableFuture<SendResult<String, String>>> acks = new ArrayList<>();
    for (Pending p : batch) {
        ProducerRecord<String, String> record =
                new ProducerRecord<>(topic, p.aggregateId().toString(), p.payload());
        record.headers().add("eventId", p.id().toString().getBytes(StandardCharsets.UTF_8));
        record.headers().add("eventType", p.eventType().getBytes(StandardCharsets.UTF_8));
        acks.add(kafka.send(record));
    }
    // wait for every ack; on failure the transaction rolls back and the rows are retried
    CompletableFuture.allOf(acks.toArray(new CompletableFuture<?>[0])).get(10, TimeUnit.SECONDS);

    jdbcTemplate.batchUpdate("UPDATE outbox SET published_at = clock_timestamp() WHERE id = ?",
            batch.stream().map(p -> new Object[] {p.id()}).toList());
    log.info("relayed {} event(s) to {}", batch.size(), topic);
}

Change data capture

Every relational database keeps a replication log — PostgreSQL's write-ahead log (WAL), MySQL's binlog — that records every committed change in commit order; that is how replicas stay in sync (see Replication). Change data capture (CDC) reads that log and turns each change into an event². The outbox relay above is one use of CDC; there are many others.

flowchart LR
    cat["catalog"] -->|"INSERT / UPDATE events"| db[("catalog DB<br/>PostgreSQL")]
    db -->|"WAL (logical decoding)"| dbz["Debezium"]
    dbz -->|"catalog.public.events"| k{{"Kafka"}}
    k --> s["search indexer"]
    k --> c["cache invalidator"]
    k --> w["warehouse loader"]

For BoxOffice, CDC on the catalog database keeps three derived systems in sync without changing the catalog code: the search index (new shows appear in search seconds after being created), the cache (entries are invalidated when an event changes; see Caching) and the data warehouse.

Sagas

A saga is a business transaction implemented as a sequence of local transactions, one per service; each step publishes an event or reply that triggers the next. If a step fails, the saga runs compensating transactions that semantically undo the steps already completed³¹. The idea comes from a 1987 database paper about long-lived transactions — older than microservices by decades.

The BoxOffice purchase saga

The pivot is the go/no-go point. Before it, every step can be compensated; after it, the saga never goes back — the remaining steps are retriable: they must eventually succeed, retried as many times as needed. That is why the seat hold must outlive the payment: if the hold could expire while the card is being charged, step 4 could fail permanently after the pivot and force a refund — a bad experience the design should make impossible (or at least rare and monitored).

There are two ways to coordinate the steps.

sequenceDiagram
    participant O as order
    participant I as inventory
    participant P as payment
    O->>I: OrderPlaced
    I->>I: hold seats
    I->>P: SeatsHeld
    P->>P: charge card
    alt payment succeeded
        P->>I: PaymentCaptured
        I->>I: mark seats SOLD
        I->>O: SeatsSold
        O->>O: CONFIRMED, publish OrderConfirmed
    else payment failed
        P->>I: PaymentFailed
        I->>I: release seats (compensation)
        I->>O: SeatsReleased
        O->>O: CANCELLED (compensation)
    end

sequenceDiagram
    participant S as purchase saga (order)
    participant I as inventory
    participant P as payment
    S->>I: HoldSeats
    I-->>S: SeatsHeld
    S->>P: ChargeCard (key o-7731)
    alt charged
        P-->>S: PaymentCaptured
        S->>I: MarkSeatsSold
        I-->>S: SeatsSold
        S->>S: CONFIRMED, publish OrderConfirmed
    else declined
        P-->>S: PaymentFailed
        S->>I: ReleaseSeats (compensation)
        I-->>S: SeatsReleased
        S->>S: CANCELLED
    end

The order's status is the saga's state, so the saga is a state machine:

stateDiagram-v2
    [*] --> PENDING
    PENDING --> SEATS_HELD : SeatsHeld
    PENDING --> CANCELLED : NoSeats
    SEATS_HELD --> PAID : PaymentCaptured (pivot)
    SEATS_HELD --> RELEASING : PaymentFailed / timeout
    RELEASING --> CANCELLED : SeatsReleased
    PAID --> CONFIRMED : SeatsSold (retried until success)
    CONFIRMED --> [*]
    CANCELLED --> [*]

A minimal orchestrator handles each reply in a local transaction, moves the state machine and sends the next command through the outbox:

@Component
@RequiredArgsConstructor
public class PurchaseSaga {

    private final SagaRepository sagas;
    private final CommandPublisher commands;          // writes commands to the outbox

    @Transactional
    public void handle(SagaReply reply) {
        PurchaseState s = sagas.lock(reply.orderId());  // SELECT ... FOR UPDATE
        if (s.alreadyApplied(reply.replyId())) return;  // duplicate reply: ignore
        switch (s.step()) {
            case HOLDING_SEATS -> {
                if (reply.ok()) { s.moveTo(CHARGING);  commands.send(new ChargeCard(s.orderId(), s.total())); }
                else            { s.moveTo(CANCELLED); commands.send(new CancelOrder(s.orderId())); }
            }
            case CHARGING -> {
                if (reply.ok()) { s.moveTo(CONFIRMING); commands.send(new MarkSeatsSold(s.orderId())); }
                else            { s.moveTo(RELEASING);  commands.send(new ReleaseSeats(s.orderId())); }
            }
            case CONFIRMING -> { s.moveTo(CONFIRMED); commands.send(new ConfirmOrder(s.orderId())); }
            case RELEASING  -> { s.moveTo(CANCELLED); commands.send(new CancelOrder(s.orderId())); }
            default -> throw new IllegalStateException("unexpected reply in " + s.step());
        }
        sagas.save(s.applied(reply.replyId()));
    }
}

In production, orchestrated sagas are often written on a durable execution / workflow engine — Temporal, AWS Step Functions, Azure Durable Functions, Camunda — which persists the state after each step, handles timers ("release the hold if no payment in 10 minutes") and retries, and survives restarts of the process running it.

Sagas are not transactions

A saga is ACD without I: it is atomic (eventually all steps or all compensations), consistent and durable, but not isolated. Other requests see intermediate states — seats held but not sold, an order paid but not yet confirmed. Design for it:

• Semantic lock: the intermediate state is explicit and respected by everyone. Seats in HELD are not offered to other fans; the order page says "processing payment".
• Compensations are business actions, not rollbacks. A refund is a new transaction with its own trace in the books (and a notification to the fan), not an erase.
• Every step and every compensation is idempotent, because commands and events are delivered at least once.
• Every step has a deadline. If the payment provider does not answer, do not compensate blindly — the charge may have succeeded. Query the provider (by idempotency key) and reconcile first (see Integrations).

CQRS

Command Query Responsibility Segregation (CQRS) separates the model that handles writes (commands) from the models that serve reads (queries)⁴. The write model is optimized for correctness and invariants (normalized tables, transactions, locks); each read model is a denormalized projection shaped for one screen or query, built from the write side's events.

flowchart LR
    fan["fan"] -->|"POST /orders (command)"| order["order<br/>write model"]
    order --> odb[("order DB<br/>normalized")]
    order -->|"outbox: order events"| k{{"Kafka"}}
    k --> p1["my-tickets projector"]
    k --> p2["search projector"]
    p1 --> v1[("my_tickets view<br/>one row per ticket")]
    p2 --> v2[("search index")]
    fan -->|"GET /me/tickets (query)"| v1

The "My tickets" page needs, for one fan, every ticket with show name, date, venue, seat and QR code — data owned by order, catalog and inventory. Joining across three services on every page view is slow and fragile. Instead, a projector consumes their events and maintains a my_tickets table with exactly the columns the page needs. Reads become a single indexed query, and the read side scales independently (replicas, cache, even a different database).

The price is eventual consistency between write and read: right after buying, the fan opens "My tickets" and the new ticket is not there yet, because the projector is a few hundred milliseconds behind. Common fixes for this read-your-writes problem (see Consistency):

• Return the new state in the command's response and let the client show it immediately.
• Pass a version token: the client sends "I need at least version 42" and the read side waits briefly (or falls back to the write model) until it has it.
• Read the user's own recent writes from the write model, everything else from the projection.

Event sourcing

In most systems the database stores the current state and events are a side product. Event sourcing inverts this: the sequence of events is the source of truth, and the current state is computed by replaying them⁵.

CREATE TABLE order_events (
    order_id    TEXT        NOT NULL,
    version     INT         NOT NULL,      -- 1, 2, 3... per order
    type        TEXT        NOT NULL,      -- 'OrderPlaced', 'SeatsHeld', ...
    payload     JSONB       NOT NULL,
    occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (order_id, version)        -- two writers cannot both append version 4
);

public record Order(String id, OrderStatus status, List<String> seats, long totalCents) {

    public static Order replay(List<OrderEvent> history) {
        Order order = null;
        for (OrderEvent e : history) {                // oldest first
            order = switch (e) {
                case OrderPlaced p     -> new Order(p.orderId(), PENDING, p.seats(), p.totalCents());
                case SeatsHeld h       -> order.with(SEATS_HELD);
                case PaymentCaptured c -> order.with(PAID);
                case OrderConfirmed c  -> order.with(CONFIRMED);
                case OrderCancelled c  -> order.with(CANCELLED);
            };
        }
        return order;
    }

    Order with(OrderStatus s) { return new Order(id, s, seats, totalCents); }
}

To change an order, the service loads its events, rebuilds the state, checks the business rules, and appends new events with the next version; the primary key turns a concurrent append into a conflict (optimistic concurrency). Long histories are loaded from a snapshot plus the events after it. Read models are projections, as in CQRS — which is why the two patterns usually appear together.

Choosing the patterns

How it connects

flowchart LR
    DW["dual write:<br/>database + broker"] --> OUT["transactional outbox<br/>one local transaction"]
    OUT --> RELAY["relay: polling or<br/>Debezium"]
    RELAY --> K["kafka topic"]
    WAL["WAL / binlog<br/>→ Databases replication"] --> CDC["change data capture"]
    CDC --> RELAY
    CDC --> DER["search index, cache,<br/>warehouse"]
    K --> SAGA["saga: local transactions<br/>+ compensations"]
    SAGA --> PIVOT["pivot = payment<br/>→ Integrations"]
    SAGA --> ISO["no isolation:<br/>semantic locks, deadlines"]
    SAGA --> TPC["2PC vs sagas<br/>→ Distributed transactions"]
    K --> CQRS["CQRS projections<br/>read-your-writes"]
    ES["event sourcing:<br/>events are the truth"] --> CQRS

Palette components: sql (business rows + outbox, WAL for CDC) · kafka · stream · search (CDC-fed index) · cache (CDC invalidation) · dwh (CDC-fed warehouse) · payment (the saga's pivot). Scenarios: payment-system, flash-sale. Continues in: Distributed — transactions (2PC vs sagas), Databases — replication (the log CDC reads), Caching — invalidation from CDC, Compute — integrations, Case study: payment system, Hands-on 7 — outbox, Exercise — event-driven.