2. Transactional Outbox
The order service must do two things when a fan places an order: store the order in PostgreSQL and announce it on Kafka. Two different systems, no shared transaction. This step shows why the obvious code is wrong and builds the standard fix, the transactional outbox.
The dual-write problem
sequenceDiagram
participant F as Fan
participant O as order
participant DB as PostgreSQL
participant K as Kafka
F->>O: POST /orders
O->>DB: INSERT order + COMMIT
DB-->>O: ok
Note over O: crash, deploy or Kafka timeout here
O--xK: publish OrderPlaced (never happens)
O-->>F: 500 (but the order exists!) Any order of the two writes has a failure window:
| Write order | Crash between the writes | Result |
|---|---|---|
| DB, then Kafka | order stored, event lost | no e-mail, and no other service ever hears of the order |
| Kafka, then DB | event published, DB insert fails | a ghost order: e-mail for an order that does not exist |
| Kafka inside a DB transaction | Kafka publish succeeds, commit fails | same ghost; Kafka cannot be rolled back by PostgreSQL |
The outbox removes the second system from the request: the service writes the event as a row in an outbox table in the same local transaction as the order. Either both rows exist or neither. A separate relay reads unpublished rows and publishes them, marking them as published only after Kafka acknowledges.
sequenceDiagram
participant F as Fan
participant O as order
participant DB as PostgreSQL
participant R as outbox relay
participant K as Kafka
F->>O: POST /orders
O->>DB: BEGIN; INSERT orders; INSERT outbox; COMMIT
O-->>F: 201 Created
loop every 200 ms
R->>DB: SELECT ... WHERE published_at IS NULL FOR UPDATE SKIP LOCKED
R->>K: send (key = orderId), wait for acks=all
K-->>R: ack
R->>DB: UPDATE outbox SET published_at = now()
end If the relay crashes after Kafka's ack but before the UPDATE, the row is published again on the next run: the outbox gives at-least-once publication. That is fine, because the consumer in step 3 is idempotent and each event carries a unique eventId.
Polling relay or CDC?
The relay here polls the table β simple, no extra infrastructure, a few hundred milliseconds of delay. The alternative is change data capture (CDC): a tool such as Debezium reads the PostgreSQL write-ahead log and publishes each outbox insert, with lower delay and no polling load, at the cost of running and operating Kafka Connect. See the patterns page of the messaging class.
The order service
A new Spring Boot service in services/order/, on port 8084, serving everything under /orders. It uses plain JDBC (JdbcClient) instead of JPA so that every statement of the transaction is visible.
services/order/
βββ .dockerignore
βββ Dockerfile
βββ pom.xml
βββ src/main/
βββ java/boxoffice/order/
β βββ OrderApplication.java
β βββ OrderProperties.java
β βββ OrderController.java
β βββ OrderService.java
β βββ OrderRequest.java
β βββ OrderView.java
β βββ OrderEvent.java
β βββ NotificationClient.java
β βββ outbox/
β βββ Outbox.java
β βββ OutboxRelay.java
βββ resources/
βββ application.yaml
βββ db/migration/V1__create_orders.sql
Build files
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>boxoffice</groupId>
<artifactId>order</artifactId>
<version>1.0.0</version>
<properties>
<java.version>21</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-restclient</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-flyway</artifactId>
</dependency>
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-database-postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
<build>
<finalName>app</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
# syntax=docker/dockerfile:1
# same Dockerfile as catalog (lab 1); only the port changes
FROM maven:3.9-eclipse-temurin-21 AS build
WORKDIR /src
COPY pom.xml .
COPY src ./src
RUN --mount=type=cache,target=/root/.m2 mvn -B -q package -DskipTests
FROM eclipse-temurin:21-jre
WORKDIR /app
RUN useradd --system --uid 10001 app
COPY --from=build /src/target/app.jar app.jar
USER app
EXPOSE 8084
ENTRYPOINT ["java", "-XX:MaxRAMPercentage=75", "-jar", "app.jar"]
server:
port: 8084
spring:
application:
name: order
mvc:
problemdetails:
enabled: true
datasource:
url: jdbc:postgresql://${DB_HOST:db}:${DB_PORT:5432}/${DB_NAME:boxoffice}?currentSchema=orders
username: ${DB_USER:boxoffice}
password: ${DB_PASSWORD:boxoffice}
flyway:
schemas: orders
data:
redis:
host: ${REDIS_HOST:redis}
port: 6379
kafka:
bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:kafka:9092}
producer:
acks: all
properties:
enable.idempotence: true
linger.ms: 5
max.block.ms: 5000 # Kafka down: fail the relay run in 5 s (default 60 s) and retry
boxoffice:
order:
notify-mode: ${NOTIFY_MODE:outbox}
notification-url: ${NOTIFICATION_URL:http://notification:8086}
topic: order-events
relay-interval-ms: 200
waiting-room:
admit-per-second: ${ADMIT_PER_SECOND:20}
admission-ttl: 5m
management:
endpoints:
web:
exposure:
include: health,info,metrics
endpoint:
health:
probes:
enabled: true # /actuator/health/liveness and /readiness
The producer uses Spring Boot's default StringSerializer for key and value: the outbox already stores the event as JSON text, so the relay publishes it byte for byte. acks: all plus enable.idempotence: true make the broker deduplicate the producer's own retries and keep their order within a partition.
Schema
CREATE TABLE orders (
id UUID PRIMARY KEY,
show_id VARCHAR(64) NOT NULL,
hold_id VARCHAR(64) NOT NULL UNIQUE,
customer_email VARCHAR(320) NOT NULL,
amount NUMERIC(10,2) NOT NULL,
status VARCHAR(20) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE outbox (
id UUID PRIMARY KEY,
aggregate_type VARCHAR(50) NOT NULL,
aggregate_id UUID NOT NULL,
event_type VARCHAR(50) NOT NULL,
payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
published_at TIMESTAMPTZ
);
-- the relay only ever looks at unpublished rows: keep that index tiny
CREATE INDEX outbox_unpublished ON outbox (created_at) WHERE published_at IS NULL;
hold_id is UNIQUE: one seat hold (from lab 6) can become at most one order. That constraint is what makes POST /orders idempotent β a fan who double-clicks, or a gateway that retries after a timeout, gets the same order back instead of a second one.
Domain records
package boxoffice.order;
import jakarta.validation.constraints.Email;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Positive;
import java.math.BigDecimal;
public record OrderRequest(
@NotBlank String showId,
@NotBlank String holdId,
@NotBlank @Email String customerEmail,
@NotNull @Positive BigDecimal amount) {
}
package boxoffice.order;
import java.math.BigDecimal;
import java.time.OffsetDateTime;
import java.util.UUID;
public record OrderView(
UUID id,
String showId,
String holdId,
String customerEmail,
BigDecimal amount,
String status,
OffsetDateTime createdAt) {
}
package boxoffice.order;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.UUID;
/** The fact published on order-events. eventId is unique per event: consumers dedupe by it. */
public record OrderEvent(
UUID eventId,
String eventType,
UUID orderId,
String showId,
String holdId,
String customerEmail,
BigDecimal amount,
Instant occurredAt) {
public static OrderEvent placed(OrderView order) {
return new OrderEvent(UUID.randomUUID(), "OrderPlaced", order.id(), order.showId(),
order.holdId(), order.customerEmail(), order.amount(), Instant.now());
}
}
package boxoffice.order;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.bind.DefaultValue;
@ConfigurationProperties(prefix = "boxoffice.order")
public record OrderProperties(
@DefaultValue("outbox") NotifyMode notifyMode,
@DefaultValue("http://notification:8086") String notificationUrl,
@DefaultValue("order-events") String topic,
@DefaultValue("200") long relayIntervalMs) {
public enum NotifyMode { SYNC, OUTBOX }
}
package boxoffice.order;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.ConfigurationPropertiesScan;
import org.springframework.scheduling.annotation.EnableScheduling;
@SpringBootApplication
@EnableScheduling
@ConfigurationPropertiesScan
public class OrderApplication {
public static void main(String[] args) {
SpringApplication.run(OrderApplication.class, args);
}
}
The transaction: order + outbox
The NOTIFY_MODE switch keeps the "before" version in the code so that you can measure both in step 5: sync calls notification over HTTP inside the request, outbox writes the event row.
package boxoffice.order;
import boxoffice.order.OrderProperties.NotifyMode;
import boxoffice.order.outbox.Outbox;
import java.util.Optional;
import java.util.UUID;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
public class OrderService {
public record PlaceResult(OrderView order, boolean created) {
}
private static final String SELECT = """
SELECT id, show_id, hold_id, customer_email, amount, status, created_at
FROM orders
""";
private final JdbcClient jdbc;
private final Outbox outbox;
private final NotificationClient notifications;
private final OrderProperties props;
public OrderService(JdbcClient jdbc, Outbox outbox, NotificationClient notifications,
OrderProperties props) {
this.jdbc = jdbc;
this.outbox = outbox;
this.notifications = notifications;
this.props = props;
}
@Transactional
public PlaceResult place(OrderRequest req) {
int inserted = jdbc.sql("""
INSERT INTO orders (id, show_id, hold_id, customer_email, amount, status)
VALUES (?, ?, ?, ?, ?, 'PENDING_PAYMENT')
ON CONFLICT (hold_id) DO NOTHING
""")
.params(UUID.randomUUID(), req.showId(), req.holdId(), req.customerEmail(), req.amount())
.update();
OrderView order = jdbc.sql(SELECT + " WHERE hold_id = ?")
.param(req.holdId())
.query(OrderView.class)
.single();
if (inserted == 0) {
// same hold already ordered: idempotent replay, no second event
return new PlaceResult(order, false);
}
OrderEvent event = OrderEvent.placed(order);
if (props.notifyMode() == NotifyMode.SYNC) {
notifications.send(event); // BEFORE: the fan waits for the e-mail
} else {
outbox.append(event.eventId(), "order", order.id(), event.eventType(), event);
}
return new PlaceResult(order, true);
}
public Optional<OrderView> find(UUID id) {
return jdbc.sql(SELECT + " WHERE id = ?").param(id).query(OrderView.class).optional();
}
}
package boxoffice.order;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
/** Only used in NOTIFY_MODE=sync: the "before" architecture. */
@Component
public class NotificationClient {
private final RestClient http;
public NotificationClient(RestClient.Builder builder, OrderProperties props) {
this.http = builder.baseUrl(props.notificationUrl()).build();
}
public void send(OrderEvent event) {
http.post().uri("/notifications").body(event).retrieve().toBodilessEntity();
}
}
package boxoffice.order;
import jakarta.validation.Valid;
import java.net.URI;
import java.util.UUID;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
@RestController
@RequestMapping("/orders")
public class OrderController {
private final OrderService orders;
public OrderController(OrderService orders) {
this.orders = orders;
}
@PostMapping
public ResponseEntity<OrderView> place(@Valid @RequestBody OrderRequest req) {
OrderService.PlaceResult result = orders.place(req);
if (!result.created()) {
return ResponseEntity.ok(result.order());
}
return ResponseEntity.created(URI.create("/orders/" + result.order().id())).body(result.order());
}
@GetMapping("/{id}")
public OrderView get(@PathVariable UUID id) {
return orders.find(id).orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));
}
}
ON CONFLICT (hold_id) DO NOTHING returns 0 rows when the hold was already ordered; the service then returns the existing order with 200 OK and writes no event. Under concurrency, PostgreSQL makes the second insert wait for the first transaction and then skip β no race, no duplicate.
The outbox and its relay
package boxoffice.order.outbox;
import tools.jackson.databind.ObjectMapper;
import java.util.UUID;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
@Component
public class Outbox {
private final JdbcClient jdbc;
private final ObjectMapper mapper;
public Outbox(JdbcClient jdbc, ObjectMapper mapper) {
this.jdbc = jdbc;
this.mapper = mapper;
}
/** MANDATORY: fails fast if someone calls it outside the business transaction. */
@Transactional(propagation = Propagation.MANDATORY)
public void append(UUID eventId, String aggregateType, UUID aggregateId, String eventType, Object payload) {
String json = mapper.writeValueAsString(payload); // Jackson 3: exceptions are unchecked
jdbc.sql("""
INSERT INTO outbox (id, aggregate_type, aggregate_id, event_type, payload)
VALUES (?, ?, ?, ?, ?::jsonb)
""")
.params(eventId, aggregateType, aggregateId, eventType, json)
.update();
}
}
package boxoffice.order.outbox;
import boxoffice.order.OrderProperties;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;
@Component
public class OutboxRelay {
private static final Logger log = LoggerFactory.getLogger(OutboxRelay.class);
record Pending(UUID id, UUID aggregateId, String eventType, String payload) {
}
private final JdbcClient jdbc;
private final JdbcTemplate jdbcTemplate;
private final KafkaTemplate<String, String> kafka;
private final String topic;
public OutboxRelay(JdbcClient jdbc, JdbcTemplate jdbcTemplate,
KafkaTemplate<String, String> kafka, OrderProperties props) {
this.jdbc = jdbc;
this.jdbcTemplate = jdbcTemplate;
this.kafka = kafka;
this.topic = props.topic();
}
@Scheduled(fixedDelayString = "${boxoffice.order.relay-interval-ms:200}")
@Transactional
public void relay() throws Exception {
// SKIP LOCKED: several order replicas can relay in parallel without taking the same rows
List<Pending> batch = jdbc.sql("""
SELECT id, aggregate_id, event_type, payload::text AS payload
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 100
FOR UPDATE SKIP LOCKED
""")
.query(Pending.class)
.list();
if (batch.isEmpty()) {
return;
}
List<CompletableFuture<SendResult<String, String>>> acks = new ArrayList<>();
for (Pending p : batch) {
ProducerRecord<String, String> record =
new ProducerRecord<>(topic, p.aggregateId().toString(), p.payload());
record.headers().add("eventId", p.id().toString().getBytes(StandardCharsets.UTF_8));
record.headers().add("eventType", p.eventType().getBytes(StandardCharsets.UTF_8));
acks.add(kafka.send(record));
}
// wait for every ack; on failure the transaction rolls back and the rows are retried
CompletableFuture.allOf(acks.toArray(new CompletableFuture<?>[0])).get(10, TimeUnit.SECONDS);
jdbcTemplate.batchUpdate("UPDATE outbox SET published_at = clock_timestamp() WHERE id = ?",
batch.stream().map(p -> new Object[] {p.id()}).toList());
log.info("relayed {} event(s) to {}", batch.size(), topic);
}
@Scheduled(cron = "0 0 * * * *")
@Transactional
public void purge() {
int n = jdbcTemplate.update(
"DELETE FROM outbox WHERE published_at < now() - interval '1 day'");
log.info("purged {} published outbox row(s)", n);
}
}
Read the relay slowly; each line answers a failure question:
| Line | Failure it handles |
|---|---|
FOR UPDATE SKIP LOCKED | two order replicas relaying at once: each locks different rows, none waits |
ORDER BY created_at + key = orderId | events of the same order reach the same partition in commit order |
allOf(...).get(10, SECONDS) | Kafka down or slow: the transaction rolls back, rows stay unpublished, the next run retries |
UPDATE ... published_at after the acks | a crash before it republishes the batch (at-least-once) β never loses it |
purge() | the table does not grow forever; published rows are kept one day for debugging |
Add order to Compose and test
order:
build: ./services/order
environment:
DB_HOST: db
DB_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
REDIS_HOST: redis
KAFKA_BOOTSTRAP_SERVERS: kafka:9092
NOTIFICATION_URL: http://notification:8086
NOTIFY_MODE: ${NOTIFY_MODE:-outbox} # outbox (after) | sync (before)
ADMIT_PER_SECOND: ${ADMIT_PER_SECOND:-20} # waiting-room admission rate
ports:
- "8084:8084"
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
kafka-init:
condition: service_completed_successfully
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8084/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 40s
retries: 3
Open Kafka UI (http://localhost:8089) β Topics β order-events β Messages: the event is there, with key = order id and the eventId and eventType headers.
Now prove the two guarantees:
- Idempotent request. Send the same
curlagain: the response is200(not201) with the sameid, and no new message appears in the topic. -
No lost event. Stop Kafka, place an order, start Kafka again:
docker compose stop kafka curl -s -X POST localhost:8084/orders -H 'Content-Type: application/json' \ -d '{"showId":"show-42","holdId":"hold-0002","customerEmail":"bia@example.com","amount":350.00}' docker compose exec db psql -U boxoffice -d boxoffice \ -c "SELECT event_type, created_at, published_at FROM orders.outbox ORDER BY created_at DESC LIMIT 2;" docker compose start kafkaYou should see
201for the order and, in the first query, the new row with an emptypublished_at. Wait untildocker compose ps kafkasayshealthyagain (10 s to a minute) and run the query once more:published_atis now filled. In a test run:event_type | created_at | published_at -------------+-------------------------------+------------------------------- OrderPlaced | 2026-10-18 18:02:40.865762+00 | 2026-10-18 18:03:51.614915+00 OrderPlaced | 2026-10-18 17:57:39.391694+00 | 2026-10-18 17:57:39.691802+00The order was accepted while Kafka was down and published 71 s later, when Kafka was back; the fan never saw the outage. While Kafka is down for more than 5 s,
docker compose logs ordershowsERROR β¦ Unexpected error occurred in scheduled taskevery few seconds: each relay run fails aftermax.block.ms(5 s) and the next one retries β expected.
Pitfalls
- Publishing inside the HTTP request "after commit" (e.g.
@TransactionalEventListener) still loses the event if the process dies between the commit and the send. Only a durable outbox row survives a crash. - Ordering across replicas. With several relays, two events of the same order could be published by different replicas out of order. Here each order has one event at a time; if you emit several per order, relay per aggregate (e.g.
ORDER BY aggregate_id, created_at) or use CDC. - The outbox is not a queue for the world. It is private to
order; other services read the Kafka topic, never the table.