1. Instrumente Tudo
Neste passo cada serviço Java passa a enviar traces, métricas e logs para um único contêiner, o otel-lgtm, sem nenhuma mudança no código de negócio. Depois você corrige o único lugar onde a instrumentação automática não enxerga a requisição: o salto pelo outbox do lab 7. Todos os comandos rodam na raiz do seu repositório boxoffice.
O backend: otel-lgtm
O grafana/otel-lgtm reúne um backend de observabilidade inteiro em uma imagem, para desenvolvimento e ensino: um OpenTelemetry Collector recebe OTLP (o OpenTelemetry Protocol) e encaminha cada sinal para o Prometheus (métricas), o Tempo (traces) e o Loki (logs), com o Grafana por cima, já ligado de modo que um exemplar de métrica abre o seu trace e um trace abre os seus logs. Em produção esses são sistemas separados e replicados (ou um serviço gerenciado); o protocolo que seus serviços falam — o OTLP — continua o mesmo.
otel-lgtm:
image: grafana/otel-lgtm:0.37.0
ports:
- "3000:3000" # Grafana (no login)
- "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP
volumes:
- ./infra/grafana/dashboards.yaml:/otel-lgtm/grafana/conf/provisioning/dashboards/boxoffice.yaml:ro
- ./infra/grafana/dashboards:/otel-lgtm/boxoffice-dashboards:ro
- ./infra/grafana/alerting:/otel-lgtm/grafana/conf/provisioning/alerting:ro
Os três volumes carregam o seu dashboard e as suas regras de alerta na inicialização; os arquivos são criados no passo 2 e no passo 3.
O agente: um jar, nenhum código
O agente Java do OpenTelemetry é conectado à JVM com -javaagent. No carregamento das classes ele reescreve o bytecode das bibliotecas que conhece — Spring MVC e WebFlux, os clientes HTTP, JDBC e HikariCP, Kafka, Redis, logback, a própria JVM — para que cada requisição vire um span, cada biblioteca reporte métricas com nomes padronizados (as convenções semânticas do OpenTelemetry) e cada linha de log carregue o trace id da requisição que a escreveu.
Baixe-o uma vez em infra/otel/ (são 26 MB: o .gitignore o mantém fora do Git, e cada clone roda o script):
#!/bin/sh
# Downloads the OpenTelemetry Java agent into infra/otel/ (mounted read-only into every Java service).
# Usage, from the repository root: sh scripts/get-otel-agent.sh
set -e
VERSION=2.32.0
mkdir -p infra/otel
curl -fsSL -o infra/otel/opentelemetry-javaagent.jar \
"https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v${VERSION}/opentelemetry-javaagent.jar"
ls -l infra/otel/opentelemetry-javaagent.jar
# build output
target/
*.class
# IDEs and OS
.idea/
.vscode/
*.iml
.DS_Store
# local secrets: never commit them
.env
# k6 HTML reports are big; commit the JSON summaries and the tables in results/*.md
results/*.html
# the OpenTelemetry Java agent is downloaded by scripts/get-otel-agent.sh (lab 8)
infra/otel/*.jar
As imagens não mudam: o Compose monta infra/otel/ somente leitura em cada serviço Java e liga o agente com variáveis de ambiente compartilhadas por uma âncora YAML:
# lab 8: OpenTelemetry settings shared by every Java service (merged into each "environment")
x-otel: &otel
JAVA_TOOL_OPTIONS: -javaagent:/otel/opentelemetry-javaagent.jar
OTEL_JAVAAGENT_ENABLED: ${OTEL_ON:-true} # false = same image, agent off
OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-lgtm:4318
OTEL_METRIC_EXPORT_INTERVAL: "10000" # 10 s instead of 60 s
OTEL_RESOURCE_ATTRIBUTES: service.namespace=boxoffice,deployment.environment.name=local
OTEL_TRACES_SAMPLER: ${OTEL_TRACES_SAMPLER:-parentbased_always_on}
OTEL_TRACES_SAMPLER_ARG: ${OTEL_TRACES_SAMPLER_ARG:-1.0}
LOGGING_PATTERN_CORRELATION: "[%X{trace_id:-},%X{span_id:-}] " # trace id in console logs
# ... and in every Java service, for example:
order:
build: ./services/order
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: order
| Variável | Efeito |
|---|---|
JAVA_TOOL_OPTIONS | a JVM a lê antes do main: conecta o agente sem mudar a imagem nem o ENTRYPOINT |
OTEL_JAVAAGENT_ENABLED | false desliga o agente com a mesma imagem — usado no passo Meça |
OTEL_SERVICE_NAME | o atributo service.name: o nome pelo qual você filtra em todo lugar. As réplicas catalog-1..3 usam todas catalog: um serviço, três instâncias |
OTEL_EXPORTER_OTLP_ENDPOINT | para onde enviar; o agente usa OTLP sobre HTTP (4318) por padrão |
OTEL_METRIC_EXPORT_INTERVAL | 10 s em vez dos 60 s padrão, para que os dashboards reajam durante o lab |
OTEL_RESOURCE_ATTRIBUTES | atributos extras em todo sinal; service.namespace agrupa os serviços do BoxOffice |
OTEL_TRACES_SAMPLER | quais requisições são rastreadas; todas por padrão (o experimento B do Meça muda isso) |
LOGGING_PATTERN_CORRELATION | o logging.pattern.correlation do Spring Boot: imprime [trace id,span id] em toda linha do console |
Pegue o arquivo completo deste lab (é o arquivo do lab 7 mais a âncora, os volumes, o otel-lgtm, um e um start_period maior nos health checks dos serviços Java):
curl -fsSL -o docker-compose.yaml https://insper.github.io/platform/2027.1/hands-on/8/code/docker-compose.yaml
docker-compose.yaml — o arquivo completo ao fim deste lab
name: boxoffice
# lab 8: OpenTelemetry settings shared by every Java service (merged into each "environment")
x-otel: &otel
JAVA_TOOL_OPTIONS: -javaagent:/otel/opentelemetry-javaagent.jar
OTEL_JAVAAGENT_ENABLED: ${OTEL_ON:-true} # false = same image, agent off
OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-lgtm:4318
OTEL_METRIC_EXPORT_INTERVAL: "10000" # 10 s instead of 60 s
OTEL_RESOURCE_ATTRIBUTES: service.namespace=boxoffice,deployment.environment.name=local
OTEL_TRACES_SAMPLER: ${OTEL_TRACES_SAMPLER:-parentbased_always_on}
OTEL_TRACES_SAMPLER_ARG: ${OTEL_TRACES_SAMPLER_ARG:-1.0}
LOGGING_PATTERN_CORRELATION: "[%X{trace_id:-},%X{span_id:-}] " # trace id in console logs
# settings shared by the three catalog replicas (a YAML anchor)
x-catalog: &catalog
build: ./services/catalog
image: boxoffice-catalog
volumes:
- ./infra/otel:/otel:ro # lab 8: the Java agent
environment:
<<: *otel
OTEL_SERVICE_NAME: catalog # three replicas, ONE service
DB_HOST: db
DB_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
DB_POOL_SIZE: ${DB_POOL_SIZE:-10}
REDIS_HOST: redis
REDIS_PORT: 6379
CACHE_ENABLED: ${CACHE_ENABLED:-true}
CACHE_TTL_EVENT: ${CACHE_TTL_EVENT:-60s}
CACHE_TTL_LIST: ${CACHE_TTL_LIST:-10s}
CACHE_STAMPEDE_PROTECTION: ${CACHE_STAMPEDE_PROTECTION:-false}
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8082/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s # lab 8: with the agent, JVMs starting together need up to ~90 s
retries: 3
services:
db:
image: postgres:17
command: ["postgres", "-c", "max_connections=${DB_MAX_CONNECTIONS:-100}"]
environment:
POSTGRES_DB: ${DB_NAME:-boxoffice}
POSTGRES_USER: ${DB_USER:-boxoffice}
POSTGRES_PASSWORD: ${DB_PASSWORD:-boxoffice}
REPLICATION_PASSWORD: ${REPLICATION_PASSWORD:-replicator}
ports:
- "5432:5432"
volumes:
- db-data:/var/lib/postgresql/data
# runs only when the volume is new: creates the "replicator" user
- ./infra/postgres/10-replication.sh:/docker-entrypoint-initdb.d/10-replication.sh:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 5s
timeout: 3s
retries: 10
# read-only streaming replica of db (hot standby)
db-replica:
image: postgres:17
user: postgres
entrypoint: ["/usr/local/bin/replica-entrypoint.sh"]
environment:
PGPASSWORD: ${REPLICATION_PASSWORD:-replicator} # used by pg_basebackup
REPLICA_DELAY: ${REPLICA_DELAY:-0} # e.g. 2s: artificial lag
ports:
- "5433:5432"
volumes:
- replica-data:/var/lib/postgresql/data
- ./infra/postgres/replica-entrypoint.sh:/usr/local/bin/replica-entrypoint.sh:ro
depends_on:
db:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "pg_isready -U boxoffice -d boxoffice"]
interval: 5s
timeout: 3s
retries: 20
pgbouncer:
image: edoburu/pgbouncer:v1.26.0-p0
ports:
- "6432:6432"
volumes:
- ./infra/pgbouncer/pgbouncer.ini:/etc/pgbouncer/pgbouncer.ini:ro
- ./infra/pgbouncer/userlist.txt:/etc/pgbouncer/userlist.txt:ro
depends_on:
db:
condition: service_healthy
redis:
image: redis:8-alpine
# a pure cache: bounded memory, evict least-recently-used keys, no persistence
command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru",
"--save", "", "--appendonly", "no"]
ports:
- "6379:6379"
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 10
# three identical replicas; no "ports:" — only the edge is reachable from outside
catalog-1:
<<: *catalog
hostname: catalog-1
catalog-2:
<<: *catalog
hostname: catalog-2
catalog-3:
<<: *catalog
hostname: catalog-3
inventory:
build: ./services/inventory
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: inventory
DB_HOST: ${INVENTORY_DB_HOST:-pgbouncer} # writes go through PgBouncer...
DB_PORT: ${INVENTORY_DB_PORT:-6432}
DB_MIGRATION_HOST: db # ...migrations straight to the primary
DB_REPLICA_HOST: ${DB_REPLICA_HOST:-db-replica}
DB_REPLICA_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
DB_POOL_SIZE: ${INVENTORY_POOL_SIZE:-20}
INVENTORY_LOCKING: ${INVENTORY_LOCKING:-pessimistic}
INVENTORY_HOLD_TTL: ${INVENTORY_HOLD_TTL:-PT10M}
ports:
- "8083:8083" # published for experiments; clients go through the edge
depends_on:
db:
condition: service_healthy
db-replica:
condition: service_healthy
pgbouncer:
condition: service_started
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8083/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s
retries: 3
auth:
build: ./services/auth
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: auth
DB_HOST: db
DB_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
JWT_ISSUER: ${JWT_ISSUER:-boxoffice-auth}
JWT_TTL: ${JWT_TTL:-PT1H}
ADMIN_EMAIL: ${ADMIN_EMAIL:-admin@boxoffice.dev}
ADMIN_PASSWORD: ${ADMIN_PASSWORD:-boxoffice-admin}
depends_on:
db:
condition: service_healthy
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8081/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s
retries: 3
gateway:
build: ./services/gateway
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: gateway
AUTH_URL: http://auth:8081
INVENTORY_URL: http://inventory:8083
AUTH_JWKS_URI: http://auth:8081/auth/.well-known/jwks.json
JWT_ISSUER: ${JWT_ISSUER:-boxoffice-auth}
ports:
- "8080:8080" # published for experiments; clients use the edge (port 80)
depends_on:
- auth
- inventory
- catalog-1
- catalog-2
- catalog-3
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8080/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s
retries: 3
edge:
image: nginx:1.30-alpine
ports:
- "80:80"
volumes:
- ./infra/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- gateway
# ---- lab 7: Kafka (KRaft), Kafka UI, Mailpit, order and notification ----
kafka:
image: apache/kafka:4.3.1
hostname: kafka
ports:
- "9094:9094"
environment:
KAFKA_NODE_ID: 1
KAFKA_PROCESS_ROLES: broker,controller
KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://localhost:9094
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
healthcheck:
test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 > /dev/null 2>&1"]
interval: 10s
timeout: 10s
start_period: 60s # a busy laptop can take a minute to start the broker
retries: 20
kafka-init:
image: apache/kafka:4.3.1
depends_on:
kafka:
condition: service_healthy
entrypoint: ["/bin/sh", "-c"]
command:
- |
/opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 --create --if-not-exists \
--topic order-events --partitions 6 --replication-factor 1 &&
/opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 --create --if-not-exists \
--topic order-events-dlt --partitions 6 --replication-factor 1
restart: "no"
kafka-ui:
image: kafbat/kafka-ui:v1.5.0
depends_on:
kafka:
condition: service_healthy
ports:
- "8089:8080"
environment:
KAFKA_CLUSTERS_0_NAME: boxoffice
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
mailpit:
image: axllent/mailpit:v1.31
ports:
- "8025:8025"
- "1025:1025"
order:
build: ./services/order
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: order
DB_HOST: db
DB_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
REDIS_HOST: redis
KAFKA_BOOTSTRAP_SERVERS: kafka:9092
NOTIFICATION_URL: http://notification:8086
NOTIFY_MODE: ${NOTIFY_MODE:-outbox} # outbox (after) | sync (before)
ADMIT_PER_SECOND: ${ADMIT_PER_SECOND:-20} # waiting-room admission rate
ports:
- "8084:8084"
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
kafka-init:
condition: service_completed_successfully
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8084/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s
retries: 3
notification:
build: ./services/notification
volumes:
- ./infra/otel:/otel:ro
environment:
<<: *otel
OTEL_SERVICE_NAME: notification
DB_HOST: db
DB_PORT: 5432
DB_NAME: ${DB_NAME:-boxoffice}
DB_USER: ${DB_USER:-boxoffice}
DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
KAFKA_BOOTSTRAP_SERVERS: kafka:9092
MAIL_HOST: mailpit
NOTIFY_CONCURRENCY: ${NOTIFY_CONCURRENCY:-3} # consumer threads (max useful: 6 partitions)
EMAIL_LATENCY: ${EMAIL_LATENCY:-50ms} # simulated e-mail provider latency
NOTIFY_IDEMPOTENT: ${NOTIFY_IDEMPOTENT:-true} # false = no dedupe (to see duplicates)
ports:
- "8086:8086"
depends_on:
db:
condition: service_healthy
mailpit:
condition: service_started
kafka-init:
condition: service_completed_successfully
deploy:
resources:
limits:
cpus: "1.0"
memory: 768M
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8086/actuator/health/readiness || exit 1"]
interval: 10s
timeout: 3s
start_period: 120s
retries: 3
# ---- lab 8: observability backend (Grafana, Prometheus, Tempo, Loki, OTel Collector) ----
otel-lgtm:
image: grafana/otel-lgtm:0.37.0
ports:
- "3000:3000" # Grafana (no login)
- "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP
volumes:
- ./infra/grafana/dashboards.yaml:/otel-lgtm/grafana/conf/provisioning/dashboards/boxoffice.yaml:ro
- ./infra/grafana/dashboards:/otel-lgtm/boxoffice-dashboards:ro
- ./infra/grafana/alerting:/otel-lgtm/grafana/conf/provisioning/alerting:ro
volumes:
db-data:
replica-data:
Crie as pastas de provisionamento (os passos 2 e 3 as preenchem) e salve o provider de dashboards, explicado no passo 2, antes do primeiro up: o Compose monta infra/grafana/dashboards.yaml como arquivo e, quando ele não existe, o Docker cria um diretório vazio com esse nome — o Grafana então não consegue lê-lo e o otel-lgtm termina.
apiVersion: 1
providers:
- name: BoxOffice
folder: BoxOffice
type: file
options:
path: /otel-lgtm/boxoffice-dashboards
Depois reinicie tudo — os serviços Java reiniciam para pegar o agente, e o --wait retorna quando todos os serviços estão saudáveis:
Com o agente, cada JVM sobe visivelmente mais devagar (a instrumentação acontece no carregamento das classes): com todas subindo juntas, o catalog levou cerca de 75–90 s numa execução de teste, e por isso os health checks dos serviços Java agora dão um start_period de 120 s. A primeira requisição pela borda também aquece o balanceador do gateway e as conexões do catalog com o Redis e o banco: numa stack fria ela pode passar do timeout de 5 s do gateway e voltar 504 — repita-a.
Abra o Grafana em http://localhost:3000 (sem login nesta imagem) → Explore. Você deve ver:
- Prometheus:
sum by (service_name) (http_server_request_duration_seconds_count)devolve uma linha por serviço —auth,catalog,gateway,inventory,notification,order. Cada série tem labels comohttp_route="/catalog/events/{id}",http_request_method="GET"ehttp_response_status_code="200"— a rota é o template, nunca a URL crua, o que mantém o número de séries limitado. - Tempo: busque pelo serviço
gateway. Uma requisição pela borda mostra spans dogatewaye docatalog, e dentro docatalog, quando o evento não está no cache Redis, um span por comando SQL. - Loki:
{service_name="order"}. Cada linha temtrace_idespan_id, e um botão que leva ao trace. O console também:docker compose logs orderimprime linhas como[14263dc50b861be8bcf0bc1f4dbad3c5,a1c96ae57fb41b45] o.s.web.servlet.DispatcherServlet ….
Os nomes das métricas seguem as convenções semânticas
O histograma de duração de servidor HTTP se chama http.server.request.duration no OpenTelemetry; o Prometheus o recebe como http_server_request_duration_seconds_bucket, _count e _sum (pontos viram sublinhados, a unidade vira sufixo). Atributos de recurso como service.name viram labels (service_name) e o job vira boxoffice/catalog (namespace/nome). Todos os dashboards e alertas deste lab dependem desses nomes.
O salto que o agente não enxerga: o outbox
Faça um pedido e procure o trace dele no Tempo. Com o código do lab 7 você encontra dois traces sem relação: um para o POST /orders (order → INSERTs) e outro que começa na tarefa agendada do relay do outbox e continua pelo Kafka até o notification. O agente propaga o contexto em chamadas HTTP e mensagens Kafka automaticamente, mas o outbox é uma linha do banco lida depois por outra thread: o contexto da requisição não está na linha, então se perde.
sequenceDiagram
participant O as order (thread HTTP)
participant DB as tabela outbox
participant R as relay (thread do agendador)
participant K as Kafka
participant N as notification
O->>DB: INSERT da linha do outbox + traceparent da requisição
R->>DB: SELECT da linha (com o seu traceparent)
Note over R: torna esse contexto o atual
R->>K: send: o agente injeta o MESMO trace id nos headers
K->>N: consumo: o agente continua o trace A correção é a mesma ideia que o W3C Trace Context usa no HTTP: levar o traceparent junto com o dado. Guarde-o na linha do outbox e torne-o o contexto atual quando o relay publicar. Quatro arquivos do order mudam:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>boxoffice</groupId>
<artifactId>order</artifactId>
<version>1.0.0</version>
<properties>
<java.version>21</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-restclient</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-flyway</artifactId>
</dependency>
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-database-postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<!-- lab 8: only the API (no SDK); the Java agent provides the implementation at runtime
and, without the agent, every call is a no-op. Version managed by Spring Boot. -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
</dependency>
</dependencies>
<build>
<finalName>app</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
package boxoffice.order.outbox;
import tools.jackson.databind.ObjectMapper;
import io.opentelemetry.api.trace.propagation.W3CTraceContextPropagator;
import io.opentelemetry.context.Context;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
@Component
public class Outbox {
private final JdbcClient jdbc;
private final ObjectMapper mapper;
public Outbox(JdbcClient jdbc, ObjectMapper mapper) {
this.jdbc = jdbc;
this.mapper = mapper;
}
/** MANDATORY: fails fast if someone calls it outside the business transaction. */
@Transactional(propagation = Propagation.MANDATORY)
public void append(UUID eventId, String aggregateType, UUID aggregateId, String eventType, Object payload) {
String json = mapper.writeValueAsString(payload); // Jackson 3: exceptions are unchecked
// capture the trace context of the current request (null without the agent)
Map<String, String> carrier = new HashMap<>();
W3CTraceContextPropagator.getInstance().inject(Context.current(), carrier, (c, k, v) -> c.put(k, v));
jdbc.sql("""
INSERT INTO outbox (id, aggregate_type, aggregate_id, event_type, payload, traceparent)
VALUES (?, ?, ?, ?, ?::jsonb, ?)
""")
.params(eventId, aggregateType, aggregateId, eventType, json, carrier.get("traceparent"))
.update();
}
}
package boxoffice.order.outbox;
import boxoffice.order.OrderProperties;
import io.opentelemetry.api.trace.propagation.W3CTraceContextPropagator;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
import io.opentelemetry.context.propagation.TextMapGetter;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;
@Component
public class OutboxRelay {
private static final Logger log = LoggerFactory.getLogger(OutboxRelay.class);
record Pending(UUID id, UUID aggregateId, String eventType, String payload, String traceparent) {
}
/** Reads the "traceparent" key from a Pending row. */
private static final TextMapGetter<Pending> ROW = new TextMapGetter<>() {
@Override
public Iterable<String> keys(Pending p) {
return List.of("traceparent");
}
@Override
public String get(Pending p, String key) {
return p == null || !"traceparent".equals(key) ? null : p.traceparent();
}
};
private final JdbcClient jdbc;
private final JdbcTemplate jdbcTemplate;
private final KafkaTemplate<String, String> kafka;
private final String topic;
public OutboxRelay(JdbcClient jdbc, JdbcTemplate jdbcTemplate,
KafkaTemplate<String, String> kafka, OrderProperties props) {
this.jdbc = jdbc;
this.jdbcTemplate = jdbcTemplate;
this.kafka = kafka;
this.topic = props.topic();
}
@Scheduled(fixedDelayString = "${boxoffice.order.relay-interval-ms:200}")
@Transactional
public void relay() throws Exception {
// SKIP LOCKED: several order replicas can relay in parallel without taking the same rows
List<Pending> batch = jdbc.sql("""
SELECT id, aggregate_id, event_type, payload::text AS payload, traceparent
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 100
FOR UPDATE SKIP LOCKED
""")
.query(Pending.class)
.list();
if (batch.isEmpty()) {
return;
}
List<CompletableFuture<SendResult<String, String>>> acks = new ArrayList<>();
for (Pending p : batch) {
ProducerRecord<String, String> record =
new ProducerRecord<>(topic, p.aggregateId().toString(), p.payload());
record.headers().add("eventId", p.id().toString().getBytes(StandardCharsets.UTF_8));
record.headers().add("eventType", p.eventType().getBytes(StandardCharsets.UTF_8));
// continue the trace of the request that created the event
Context origin = W3CTraceContextPropagator.getInstance().extract(Context.root(), p, ROW);
try (Scope ignored = origin.makeCurrent()) {
acks.add(kafka.send(record)); // the agent's producer span is a child of the request
}
}
// wait for every ack; on failure the transaction rolls back and the rows are retried
CompletableFuture.allOf(acks.toArray(new CompletableFuture<?>[0])).get(10, TimeUnit.SECONDS);
jdbcTemplate.batchUpdate("UPDATE outbox SET published_at = clock_timestamp() WHERE id = ?",
batch.stream().map(p -> new Object[] {p.id()}).toList());
log.info("relayed {} event(s) to {}", batch.size(), topic);
}
@Scheduled(cron = "0 0 * * * *")
@Transactional
public void purge() {
int n = jdbcTemplate.update(
"DELETE FROM outbox WHERE published_at < now() - interval '1 day'");
log.info("purged {} published outbox row(s)", n);
}
}
O pom.xml adiciona só io.opentelemetry:opentelemetry-api (a versão vem do Spring Boot): sem o agente, toda chamada da API é uma operação vazia, então o serviço continua rodando sem ele.
Reconstrua o order, faça um pedido e encontre o seu trace:
docker compose up -d --build --wait order
curl -s -X POST localhost:8084/orders -H 'Content-Type: application/json' \
-d '{"showId":"show-7","holdId":"trace-1","customerEmail":"t@example.com","amount":10}'
No Grafana → Explore → Tempo, rode a consulta TraceQL {resource.service.name="order" && name="POST /orders"} e abra o trace mais recente (um trace novo pode levar até meio minuto para aparecer na busca). Você deve ver um trace com spans dos dois serviços, como este de uma execução de teste (início relativo, serviço, span, duração):
0.0 ms order POST /orders 9.0 ms
5.3 ms order INSERT boxoffice.orders 0.4 ms
5.8 ms order SELECT boxoffice.orders 0.2 ms
6.6 ms order INSERT boxoffice.outbox 0.2 ms
22.3 ms order order-events publish 11.2 ms
36.4 ms notification order-events process 67.2 ms
38.1 ms notification INSERT boxoffice.processed_events 0.4 ms
99.7 ms notification INSERT boxoffice.notifications 0.4 ms
A requisição levou 9 ms; o relay publicou 22 ms depois do início (o período de 200 ms do polling põe isso em qualquer ponto entre 0 e 200 ms); o notification gastou 67 ms, a maior parte no provedor de e-mail simulado de 50 ms — a latência assíncrona do lab 7, visível de relance. O primeiro pedido depois de um reinício é bem mais lento (JVMs frias com o agente): olhe o segundo.
Na entrevista
"A propagação de contexto quebra em todo salto que não é uma chamada direta — filas, outboxes, jobs em lote. Eu levo o traceparent do W3C na mensagem ou na linha, e o consumidor continua o trace; para fan-in (um job processando muitas mensagens) eu uso span links em vez de um único pai."