1. O Serviço e o Gateway de LLM

Este passo cria o serviço assistant — Python 3.12 + FastAPI, gerenciado com uv — e o seu gateway de LLM: o único módulo que sabe quais modelos existem, qual responde por padrão, o que fazer quando um provedor falha e quanto custa cada resposta. Todos os comandos rodam na raiz do seu repositório boxoffice, a menos que um cd diga o contrário.

Infraestrutura: um LLM local e um PostgreSQL com vetores

Contêiner	Imagem	Porta	Papel
`ollama`	`ollama/ollama:0.40.6`	11434	roda os modelos locais: `qwen2.5:1.5b` (chat, com chamada de ferramentas) e `nomic-embed-text` (embeddings, 768 dimensões)
`ollama-init`	`ollama/ollama:0.40.6`	—	execução única: baixa os dois modelos (≈ 1,3 GB, uma vez)
`db` e `db-replica`	`pgvector/pgvector:0.8.7-pg17-trixie`	5432 / 5433	o mesmo PostgreSQL 17, agora com a extensão `vector` disponível
`assistant`	`services/assistant`	8088	o serviço FastAPI deste lab

A parte nova do docker-compose.yaml:

docker-compose.yaml (excerpt)

  ollama:
    image: ollama/ollama:0.40.6
    ports:
      - "11434:11434"
    volumes:
      - ollama-models:/root/.ollama      # models survive restarts (about 1.3 GB)
    healthcheck:                         # ollama-init must wait for the server, not just the container
      test: ["CMD", "ollama", "list"]
      interval: 5s
      timeout: 5s
      retries: 12

  # one-shot: download the chat model and the embedding model, then exit
  ollama-init:
    image: ollama/ollama:0.40.6
    environment:
      OLLAMA_HOST: ollama:11434
    entrypoint: ["/bin/sh", "-c"]
    command:
      - |
        ollama pull ${LOCAL_MODEL_TAG:-qwen2.5:1.5b} && ollama pull nomic-embed-text
    depends_on:
      ollama:
        condition: service_healthy
    restart: "no"

Baixe o arquivo completo (o do lab 9 mais a imagem do pgvector para db/db-replica, ollama, ollama-init e assistant):

curl -fsSL -o docker-compose.yaml https://insper.github.io/platform/2027.1/hands-on/12/code/docker-compose.yaml

docker-compose.yaml — o arquivo completo ao fim deste lab

docker-compose.yaml

name: boxoffice

# lab 8: OpenTelemetry settings shared by every Java service (merged into each "environment")
x-otel: &otel
  JAVA_TOOL_OPTIONS: -javaagent:/otel/opentelemetry-javaagent.jar
  OTEL_JAVAAGENT_ENABLED: ${OTEL_ON:-true}          # false = same image, agent off
  OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-lgtm:4318
  OTEL_METRIC_EXPORT_INTERVAL: "10000"              # 10 s instead of 60 s
  OTEL_RESOURCE_ATTRIBUTES: service.namespace=boxoffice,deployment.environment.name=local
  OTEL_TRACES_SAMPLER: ${OTEL_TRACES_SAMPLER:-parentbased_always_on}
  OTEL_TRACES_SAMPLER_ARG: ${OTEL_TRACES_SAMPLER_ARG:-1.0}
  LOGGING_PATTERN_CORRELATION: "[%X{trace_id:-},%X{span_id:-}] "   # trace id in console logs

# settings shared by the three catalog replicas (a YAML anchor)
x-catalog: &catalog
  build: ./services/catalog
  image: boxoffice-catalog
  volumes:
    - ./infra/otel:/otel:ro          # lab 8: the Java agent
  environment:
    <<: *otel
    OTEL_SERVICE_NAME: catalog       # three replicas, ONE service
    DB_HOST: db
    DB_PORT: 5432
    DB_NAME: ${DB_NAME:-boxoffice}
    DB_USER: ${DB_USER:-boxoffice}
    DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
    DB_POOL_SIZE: ${DB_POOL_SIZE:-10}
    REDIS_HOST: redis
    REDIS_PORT: 6379
    CACHE_ENABLED: ${CACHE_ENABLED:-true}
    CACHE_TTL_EVENT: ${CACHE_TTL_EVENT:-60s}
    CACHE_TTL_LIST: ${CACHE_TTL_LIST:-10s}
    CACHE_STAMPEDE_PROTECTION: ${CACHE_STAMPEDE_PROTECTION:-false}
  depends_on:
    db:
      condition: service_healthy
    redis:
      condition: service_healthy
  deploy:
    resources:
      limits:
        cpus: "1.0"
        memory: 768M
  healthcheck:
    test: ["CMD-SHELL", "wget -qO- http://localhost:8082/actuator/health/readiness || exit 1"]
    interval: 10s
    timeout: 3s
    start_period: 120s       # lab 8: with the agent, JVMs starting together need up to ~90 s
    retries: 3

services:

  db:
    image: pgvector/pgvector:0.8.7-pg17-trixie   # lab 12: PostgreSQL 17 + the pgvector extension
    command: ["postgres", "-c", "max_connections=${DB_MAX_CONNECTIONS:-100}"]
    environment:
      POSTGRES_DB: ${DB_NAME:-boxoffice}
      POSTGRES_USER: ${DB_USER:-boxoffice}
      POSTGRES_PASSWORD: ${DB_PASSWORD:-boxoffice}
      REPLICATION_PASSWORD: ${REPLICATION_PASSWORD:-replicator}
    ports:
      - "5432:5432"
    volumes:
      - db-data:/var/lib/postgresql/data
      # runs only when the volume is new: creates the "replicator" user
      - ./infra/postgres/10-replication.sh:/docker-entrypoint-initdb.d/10-replication.sh:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
      interval: 5s
      timeout: 3s
      retries: 10

  # read-only streaming replica of db (hot standby)
  db-replica:
    image: pgvector/pgvector:0.8.7-pg17-trixie   # lab 12: PostgreSQL 17 + the pgvector extension
    user: postgres
    entrypoint: ["/usr/local/bin/replica-entrypoint.sh"]
    environment:
      PGPASSWORD: ${REPLICATION_PASSWORD:-replicator}     # used by pg_basebackup
      REPLICA_DELAY: ${REPLICA_DELAY:-0}                   # e.g. 2s: artificial lag
    ports:
      - "5433:5432"
    volumes:
      - replica-data:/var/lib/postgresql/data
      - ./infra/postgres/replica-entrypoint.sh:/usr/local/bin/replica-entrypoint.sh:ro
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U boxoffice -d boxoffice"]
      interval: 5s
      timeout: 3s
      retries: 20

  pgbouncer:
    image: edoburu/pgbouncer:v1.26.0-p0
    ports:
      - "6432:6432"
    volumes:
      - ./infra/pgbouncer/pgbouncer.ini:/etc/pgbouncer/pgbouncer.ini:ro
      - ./infra/pgbouncer/userlist.txt:/etc/pgbouncer/userlist.txt:ro
    depends_on:
      db:
        condition: service_healthy

  redis:
    image: redis:8-alpine
    # a pure cache: bounded memory, evict least-recently-used keys, no persistence
    command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru",
              "--save", "", "--appendonly", "no"]
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 10

  # three identical replicas; no "ports:" — only the edge is reachable from outside
  catalog-1:
    <<: *catalog
    hostname: catalog-1
  catalog-2:
    <<: *catalog
    hostname: catalog-2
  catalog-3:
    <<: *catalog
    hostname: catalog-3

  inventory:
    build: ./services/inventory
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: inventory
      DB_HOST: ${INVENTORY_DB_HOST:-pgbouncer}      # writes go through PgBouncer...
      DB_PORT: ${INVENTORY_DB_PORT:-6432}
      DB_MIGRATION_HOST: db                         # ...migrations straight to the primary
      DB_REPLICA_HOST: ${DB_REPLICA_HOST:-db-replica}
      DB_REPLICA_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      DB_POOL_SIZE: ${INVENTORY_POOL_SIZE:-20}
      INVENTORY_LOCKING: ${INVENTORY_LOCKING:-pessimistic}
      INVENTORY_HOLD_TTL: ${INVENTORY_HOLD_TTL:-PT10M}
    ports:
      - "8083:8083"            # published for experiments; clients go through the edge
    depends_on:
      db:
        condition: service_healthy
      db-replica:
        condition: service_healthy
      pgbouncer:
        condition: service_started
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8083/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  auth:
    build: ./services/auth
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: auth
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      JWT_ISSUER: ${JWT_ISSUER:-boxoffice-auth}
      JWT_TTL: ${JWT_TTL:-PT1H}
      ADMIN_EMAIL: ${ADMIN_EMAIL:-admin@boxoffice.dev}
      ADMIN_PASSWORD: ${ADMIN_PASSWORD:-boxoffice-admin}
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8081/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  gateway:
    build: ./services/gateway
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: gateway
      AUTH_URL: http://auth:8081
      INVENTORY_URL: http://inventory:8083
      AUTH_JWKS_URI: http://auth:8081/auth/.well-known/jwks.json
      JWT_ISSUER: ${JWT_ISSUER:-boxoffice-auth}
    ports:
      - "8080:8080"          # published for experiments; clients use the edge (port 80)
    depends_on:
      - auth
      - inventory
      - catalog-1
      - catalog-2
      - catalog-3
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8080/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  edge:
    image: nginx:1.30-alpine
    ports:
      - "80:80"
    volumes:
      - ./infra/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - gateway

  # ---- lab 7: Kafka (KRaft), Kafka UI, Mailpit, order and notification ----

  kafka:
    image: apache/kafka:4.3.1
    hostname: kafka
    ports:
      - "9094:9094"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,EXTERNAL://localhost:9094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
    healthcheck:
      test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 > /dev/null 2>&1"]
      interval: 10s
      timeout: 10s
      start_period: 60s      # a busy laptop can take a minute to start the broker
      retries: 20

  kafka-init:
    image: apache/kafka:4.3.1
    depends_on:
      kafka:
        condition: service_healthy
    entrypoint: ["/bin/sh", "-c"]
    command:
      - |
        /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 --create --if-not-exists \
          --topic order-events --partitions 6 --replication-factor 1 &&
        /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 --create --if-not-exists \
          --topic order-events-dlt --partitions 6 --replication-factor 1
    restart: "no"

  kafka-ui:
    image: kafbat/kafka-ui:v1.5.0
    depends_on:
      kafka:
        condition: service_healthy
    ports:
      - "8089:8080"
    environment:
      KAFKA_CLUSTERS_0_NAME: boxoffice
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092

  mailpit:
    image: axllent/mailpit:v1.31
    ports:
      - "8025:8025"
      - "1025:1025"

  order:
    build: ./services/order
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: order
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      REDIS_HOST: redis
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
      NOTIFICATION_URL: http://notification:8086
      PAYMENT_URL: http://toxiproxy:18085            # lab 9: through Toxiproxy, not payment:8085
      PAYMENT_RESILIENCE: ${PAYMENT_RESILIENCE:-true}
      PAYMENT_READ_TIMEOUT: ${PAYMENT_READ_TIMEOUT:-1s}
      PAYMENT_RETRY_ATTEMPTS: ${PAYMENT_RETRY_ATTEMPTS:-3}
      PAYMENT_BULKHEAD: ${PAYMENT_BULKHEAD:-10}
      NOTIFY_MODE: ${NOTIFY_MODE:-outbox}            # outbox (after) | sync (before)
      ADMIT_PER_SECOND: ${ADMIT_PER_SECOND:-20}      # waiting-room admission rate
    ports:
      - "8084:8084"
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
      kafka-init:
        condition: service_completed_successfully
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8084/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  notification:
    build: ./services/notification
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: notification
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
      MAIL_HOST: mailpit
      NOTIFY_CONCURRENCY: ${NOTIFY_CONCURRENCY:-3}   # consumer threads (max useful: 6 partitions)
      EMAIL_LATENCY: ${EMAIL_LATENCY:-50ms}          # simulated e-mail provider latency
      NOTIFY_IDEMPOTENT: ${NOTIFY_IDEMPOTENT:-true}  # false = no dedupe (to see duplicates)
    ports:
      - "8086:8086"
    depends_on:
      db:
        condition: service_healthy
      mailpit:
        condition: service_started
      kafka-init:
        condition: service_completed_successfully
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8086/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  # ---- lab 8: observability backend (Grafana, Prometheus, Tempo, Loki, OTel Collector) ----

  otel-lgtm:
    image: grafana/otel-lgtm:0.37.0
    ports:
      - "3000:3000"   # Grafana (no login)
      - "4317:4317"   # OTLP gRPC
      - "4318:4318"   # OTLP HTTP
    volumes:
      - ./infra/grafana/dashboards.yaml:/otel-lgtm/grafana/conf/provisioning/dashboards/boxoffice.yaml:ro
      - ./infra/grafana/dashboards:/otel-lgtm/boxoffice-dashboards:ro
      - ./infra/grafana/alerting:/otel-lgtm/grafana/conf/provisioning/alerting:ro

  # ---- lab 9: payment adapter, fake provider and fault injection ----

  psp:
    image: wiremock/wiremock:3.13.2
    volumes:
      - ./infra/psp:/home/wiremock:ro

  toxiproxy:
    image: ghcr.io/shopify/toxiproxy:2.13.0
    command: ["-host=0.0.0.0", "-config=/config/toxiproxy.json"]
    volumes:
      - ./infra/toxiproxy/toxiproxy.json:/config/toxiproxy.json:ro
    ports:
      - "8474:8474"

  payment:
    build: ./services/payment
    volumes:
      - ./infra/otel:/otel:ro
    environment:
      <<: *otel
      OTEL_SERVICE_NAME: payment
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      PSP_URL: http://toxiproxy:18095                 # the provider, through Toxiproxy
    ports:
      - "8085:8085"
    depends_on:
      db:
        condition: service_healthy
      toxiproxy:
        condition: service_started
      psp:
        condition: service_started
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:8085/actuator/health/readiness || exit 1"]
      interval: 10s
      timeout: 3s
      start_period: 120s
      retries: 3

  # ---- lab 12: AI assistant (FastAPI + LiteLLM + pgvector) and a local LLM ----

  ollama:
    image: ollama/ollama:0.40.6
    ports:
      - "11434:11434"
    volumes:
      - ollama-models:/root/.ollama      # models survive restarts (about 1.3 GB)
    healthcheck:                         # ollama-init must wait for the server, not just the container
      test: ["CMD", "ollama", "list"]
      interval: 5s
      timeout: 5s
      retries: 12

  # one-shot: download the chat model and the embedding model, then exit
  ollama-init:
    image: ollama/ollama:0.40.6
    environment:
      OLLAMA_HOST: ollama:11434
    entrypoint: ["/bin/sh", "-c"]
    command:
      - |
        ollama pull ${LOCAL_MODEL_TAG:-qwen2.5:1.5b} && ollama pull nomic-embed-text
    depends_on:
      ollama:
        condition: service_healthy
    restart: "no"

  assistant:
    build: ./services/assistant
    environment:
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${DB_NAME:-boxoffice}
      DB_USER: ${DB_USER:-boxoffice}
      DB_PASSWORD: ${DB_PASSWORD:-boxoffice}
      OLLAMA_URL: ${OLLAMA_URL:-http://ollama:11434}   # macOS: http://host.docker.internal:11434 (native Ollama)
      LOCAL_MODEL: ollama_chat/${LOCAL_MODEL_TAG:-qwen2.5:1.5b}
      CATALOG_URL: http://gateway:8080                  # GET /catalog/** is public at the gateway
      ORDER_URL: http://order:8084
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}          # optional: empty = local model only
      OTEL_SERVICE_NAME: assistant
      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-lgtm:4318
      OTEL_METRIC_EXPORT_INTERVAL: "10000"
      OTEL_RESOURCE_ATTRIBUTES: service.namespace=boxoffice,deployment.environment.name=local
    ports:
      - "8088:8088"
    extra_hosts:
      - "host.docker.internal:host-gateway"             # reach a native Ollama on the host
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 1G
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8088/assistant/health')"]
      interval: 10s
      timeout: 5s
      start_period: 30s
      retries: 3

volumes:
  db-data:
  replica-data:
  ollama-models:

Onde o modelo deve rodar? (leia antes de baixar)

Um modelo de linguagem é muita aritmética; onde ele roda decide a sua latência.

macOS (Apple Silicon)Linux com GPU NVIDIASó CPU

O Docker no macOS não tem acesso à GPU: num contêiner o modelo roda na CPU e uma resposta curta pode levar um minuto. Rode o Ollama nativamente (ele usa a GPU pelo Metal) e aponte o assistente para ele:

brew install ollama && ollama serve &          # or the app from ollama.com
ollama pull qwen2.5:1.5b && ollama pull nomic-embed-text
export OLLAMA_URL=http://host.docker.internal:11434

e pule os contêineres ollama e ollama-init abaixo.

Instale o NVIDIA Container Toolkit e adicione ao serviço ollama: deploy: { resources: { reservations: { devices: [{ driver: nvidia, count: all, capabilities: [gpu] }] } } }.

Use os contêineres como estão. Funciona, devagar: numa execução de teste num notebook ocupado de 8 núcleos, o primeiro token levou dezenas de segundos. Prefira perguntas curtas, e compare com a rota opcional do Claude desta página.

Suba a infraestrutura (a imagem do db muda, então o db reinicia uma vez):

docker compose up -d db db-replica ollama ollama-initdocker compose wait ollama-initcontainer "c8d93485…" exited with status code 0docker compose exec ollama ollama listNAME ID SIZE MODIFIED
nomic-embed-text:latest 0a109f422b47 274 MB 7 seconds ago
qwen2.5:1.5b 65ec06548149 986 MB 29 seconds ago

O docker compose wait bloqueia até o ollama-init (one-shot) baixar os dois modelos (um ou dois minutos na primeira vez) e imprime o código de saída: 0. (O docker compose logs -f ollama-init não retornaria se o contêiner já tivesse terminado.) Você deve ver os dois modelos na lista (ollama list na sua máquina, se o Ollama roda nativamente). Com o Ollama nativo (macOS), suba só docker compose up -d db db-replica: um contêiner ollama falharia com port is already allocated na 11434, que o servidor nativo já usa.

O projeto: uv

O uv (da Astral) cria o ambiente virtual, resolve as dependências e grava um arquivo de lock com a versão e o hash exatos de cada pacote, para que o seu notebook, o CI e o contêiner instalem os mesmos bytes. O projeto foi criado com estes comandos (você pode rodá-los num services/assistant/ vazio, ou baixar os arquivos abaixo):

mkdir -p services/assistant && cd services/assistant
uv init --app --name assistant --python 3.12 --no-readme --vcs none .
rm main.py
uv add "fastapi>=0.146" "uvicorn[standard]>=0.54" "litellm>=1.106" "psycopg[binary,pool]>=3.3" \
  "pgvector>=0.5" "numpy>=2.3" "httpx>=0.28" "pydantic>=2.14" "mcp>=1.20" \
  "opentelemetry-sdk>=1.45" "opentelemetry-exporter-otlp-proto-http>=1.45"
cd ../..

O uv add edita o pyproject.toml e grava o uv.lock (≈ 3.000 linhas: faça commit dele, nunca o edite à mão; baixe-o de https://insper.github.io/platform/2027.1/hands-on/12/code/services/assistant/uv.lock se preferir as versões exatas deste roteiro). O uv não escreve o Dockerfile nem o .dockerignore: crie-os a partir das abas abaixo. Se você pular o uv init, baixe também o pyproject.toml do mesmo endereço e crie o .python-version a partir da aba dele (o site não serve arquivos cujo nome começa com ponto).

pyproject.toml.python-versionDockerfile.dockerignore

services/assistant/pyproject.toml

[project]
name = "assistant"
version = "1.0.0"
description = "BoxOffice AI support assistant (lab 12)"
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.146",
    "httpx>=0.28",
    "litellm>=1.106",
    "mcp>=1.20",
    "numpy>=2.3",
    "opentelemetry-exporter-otlp-proto-http>=1.45",
    "opentelemetry-sdk>=1.45",
    "pgvector>=0.5",
    "psycopg[binary,pool]>=3.3",
    "pydantic>=2.14",
    "uvicorn[standard]>=0.54",
]

services/assistant/.python-version

3.12

services/assistant/Dockerfile

# syntax=docker/dockerfile:1
FROM python:3.12-slim

# uv: the project's package and environment manager (lab 0); one static binary
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /app
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy

# dependencies first (cached layer): exactly the versions of uv.lock, without dev tools
COPY pyproject.toml uv.lock .python-version ./
RUN --mount=type=cache,target=/root/.cache/uv uv sync --frozen --no-dev --no-install-project

COPY app ./app
RUN useradd --system --uid 10001 app
USER app
EXPOSE 8088
CMD ["/app/.venv/bin/uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8088"]

services/assistant/.dockerignore

.venv/
__pycache__/
*.pyc
.pytest_cache/

O uv sync --frozen falha se o uv.lock não corresponde ao pyproject.toml — o build nunca resolve versões por conta própria. (O mcp está na lista porque o LiteLLM o importa quando uma requisição leva ferramentas.)

Rodar Python localmente (o passo 4 roda) grava pastas __pycache__/ ao lado do código; acrescente duas linhas ao .gitignore da raiz para que o git add -A as deixe de fora, e também qualquer .venv/:

.gitignore

# build output
target/
*.class
# IDEs and OS
.idea/
.vscode/
*.iml
.DS_Store
# local secrets: never commit them
.env
# k6 HTML reports are big; commit the JSON summaries and the tables in results/*.md
results/*.html
# the OpenTelemetry Java agent is downloaded by scripts/get-otel-agent.sh (lab 8)
infra/otel/*.jar
# Python (lab 12): bytecode caches and local environments
__pycache__/
.venv/

A configuração e o gateway

app/config.pyapp/llm.pyapp/init.py

services/assistant/app/config.py

"""All settings come from environment variables (the same names in Compose and in Kubernetes)."""
import os
from dataclasses import dataclass


@dataclass(frozen=True)
class Settings:
    db_dsn: str
    ollama_url: str
    local_model: str        # LiteLLM model name of the local chat model
    embed_model: str        # LiteLLM model name of the embedding model
    embed_dim: int          # dimension of the embedding vectors (nomic-embed-text: 768)
    claude_model: str | None  # set only when ANTHROPIC_API_KEY exists
    default_route: str      # "local" or "claude"
    catalog_url: str
    order_url: str
    max_steps: int          # agent loop: maximum LLM calls per question
    max_input_chars: int
    retrieve_k: int


def load() -> Settings:
    env = os.environ.get
    has_key = bool(env("ANTHROPIC_API_KEY"))
    return Settings(
        db_dsn=(
            f"host={env('DB_HOST', 'db')} port={env('DB_PORT', '5432')} "
            f"dbname={env('DB_NAME', 'boxoffice')} user={env('DB_USER', 'boxoffice')} "
            f"password={env('DB_PASSWORD', 'boxoffice')}"
        ),
        ollama_url=env("OLLAMA_URL", "http://ollama:11434"),
        local_model=env("LOCAL_MODEL", "ollama_chat/qwen2.5:1.5b"),
        embed_model=env("EMBED_MODEL", "ollama/nomic-embed-text"),
        embed_dim=int(env("EMBED_DIM", "768")),
        claude_model=env("CLAUDE_MODEL", "anthropic/claude-opus-5-5") if has_key else None,
        default_route=env("ASSISTANT_ROUTE", "claude" if has_key else "local"),
        catalog_url=env("CATALOG_URL", "http://gateway:8080"),
        order_url=env("ORDER_URL", "http://order:8084"),
        max_steps=int(env("MAX_STEPS", "4")),
        max_input_chars=int(env("MAX_INPUT_CHARS", "2000")),
        retrieve_k=int(env("RETRIEVE_K", "5")),
    )


settings = load()

services/assistant/app/llm.py

"""The LLM gateway: one place that knows models, routes, fallbacks, timeouts and costs."""
import litellm
from litellm import Router

from .config import settings

litellm.drop_params = True  # drop parameters a provider does not support instead of failing

model_list = [{
    "model_name": "local",
    "litellm_params": {"model": settings.local_model, "api_base": settings.ollama_url},
}]
fallbacks = []
if settings.claude_model:  # optional: only when ANTHROPIC_API_KEY is set
    model_list.append({"model_name": "claude", "litellm_params": {"model": settings.claude_model}})
    fallbacks = [{"claude": ["local"]}]  # provider down or out of quota: answer with the local model

router = Router(model_list=model_list, fallbacks=fallbacks, num_retries=1, timeout=120)


def real_model(route: str) -> str:
    """The provider model behind a route name (for metrics and costs)."""
    return settings.claude_model if route == "claude" and settings.claude_model else settings.local_model


async def embed(texts: list[str]) -> list[list[float]]:
    response = await litellm.aembedding(model=settings.embed_model, input=texts, api_base=settings.ollama_url)
    return [item["embedding"] for item in response.data]

services/assistant/app/__init__.py

"""BoxOffice AI support assistant (lab 12)."""

O LiteLLM dá a todo provedor a mesma interface no estilo da OpenAI, e o seu Router é o gateway da aula de gateway de LLM:

Preocupação	Aqui
Roteamento	duas rotas: `local` (Ollama) sempre, `claude` só quando `ANTHROPIC_API_KEY` está definida; `ASSISTANT_ROUTE` ou o campo `route` da requisição escolhe
Fallback	`claude` → `local` quando o provedor falha (queda, rate limit, sem crédito)
Timeouts e retries	120 s por chamada, 1 retry — uma chamada ao LLM é a dependência mais lenta que o BoxOffice tem
Custo	o `app/metrics.py` (passo 4) multiplica os tokens de cada resposta pelo preço do provedor

Por que um SDK dentro do processo e não o proxy do LiteLLM?

O LiteLLM também roda como um proxy separado (um servidor HTTP compatível com a OpenAI, com chaves, orçamentos e uma interface) compartilhado por muitos serviços. Um serviço só não precisa dele: o Router dentro do processo tem a mesma lógica de roteamento sem mais um contêiner. Com vários serviços de IA, o proxy vira o gateway compartilhado — o mesmo movimento que vai da resiliência por serviço (lab 9) para uma service mesh.

A rota opcional do Claude (custa dinheiro)

O lab funciona de ponta a ponta com o modelo local. Para comparar com um modelo de fronteira, crie uma chave de API em console.anthropic.com (pago por uso: a cobrança é por token — o padrão anthropic/claude-opus-5-5 custa US$ 4 por milhão de tokens de entrada e US$ 20 por milhão de tokens de saída; uma pergunta deste lab usa cerca de 1.000 tokens de entrada e 100 de saída, ≈ US$ 0,006) e exporte-a antes de subir o assistente:

export ANTHROPIC_API_KEY=sk-ant-...       # never commit it; docker-compose.yaml reads it from the shell

Com a chave, a rota padrão passa a ser claude; envie "route": "local" numa requisição para comparar. Sem ela, nada chama a API paga.

Suba o serviço

Os passos 2 e 3 explicam os demais arquivos de app/; baixe todos agora para que o serviço suba:

mkdir -p services/assistant/app
for f in __init__ config llm db rag tools guardrails agent metrics main; do
  curl -fsSL -o services/assistant/app/$f.py \
    https://insper.github.io/platform/2027.1/hands-on/12/code/services/assistant/app/$f.py
done
docker compose up -d --wait gateway order otel-lgtm      # what the assistant calls, with their dependencies
docker compose up -d --build --wait assistant
curl -s localhost:8088/assistant/health

Você deve ver {"status":"UP","documents":0,"route":"local"} ("route":"claude" se você exportou uma chave). documents é 0: a base de conhecimento fica vazia até o passo 2. Suba os serviços pelo nome, como acima: um docker compose up -d sem nomes também sobe o contêiner ollama, que falha ao lado de um Ollama nativo.

Rota claude sem querer?

Então a ANTHROPIC_API_KEY já está definida no seu shell (outras ferramentas usam a mesma variável), e toda pergunta vai para a API paga. Rode unset ANTHROPIC_API_KEY e docker compose up -d --wait assistant de novo: o health check deve mostrar "route":"local".

Próximo: RAG com pgvector