Compute

Compute is where your code runs: the services that receive a request, apply the business rules and call storage, messaging and external providers. The design questions are always the same — how much state does it keep, how does it scale, how long can a request take, and what happens when a dependency is slow? Stateless request handlers scale by adding replicas; long or heavy work moves to background workers; spiky, short work can go serverless; and some capabilities (identity, search, notifications, analytics) are so common that they become dedicated services.

In BoxOffice, catalog, inventory and order are app servers; e-mails and PDF tickets are generated by workers; the "doors open in 1 hour" reminders are fired by a scheduler; login is the auth service; event search is the search service; and every page view and purchase feeds analytics.

flowchart LR
    GW[API Gateway] --> AUTH[Auth Service]
    GW --> APP[App Server<br/>catalog / order]
    GW --> SR[Search]
    APP --> Q[(Queue)]
    SCH[Scheduler] --> Q
    Q --> W[Worker]
    W --> N[Notifications]
    APP -. events .-> AN[Analytics]

App Server

What it is. The process that runs your service code — a Spring Boot catalog service, a FastAPI assistant — handling synchronous requests: parse, authorize, run business rules, call the database or other services, respond. The key design property is being stateless: no user session or data kept in local memory between requests, so any replica can serve any request and replicas can be added, removed or replaced at will.

When to use.

• For every synchronous API that a user waits on: browsing events, holding seats, placing an order.
• Start with one deployable (a modular monolith) and split into services only along clear boundaries (bounded contexts) or when teams and scaling needs diverge.
• Scale horizontally behind a load balancer; autoscale on CPU, request rate or latency.

When not to (or watch out).

• Do not do slow work inside the request (sending e-mails, generating PDFs, calling a slow provider without a timeout): move it to a worker.
• Do not keep state in memory that must survive the replica (sessions, carts, rate-limit counters): use a cache or a database.
• Every synchronous call to another service adds latency and a failure point; long chains of services multiply both.

Key numbers.

Failure modes.

• Thread-pool exhaustion — a slow dependency holds every thread while CPU is idle; the service stops answering → timeouts, circuit breakers, bulkheads (scenario cascading-failure).
• Overload past the knee — at ρ → 1 queueing delay explodes → autoscaling with headroom, load shedding, rate limiting.
• Memory leak — capacity slowly drops over hours → soak tests, restarts, profiling.
• Bad deploy — the error rate jumps → canary releases, automatic rollback on SLO burn.
• Stateful replica dies — users lose their session → keep replicas stateless.

In the playground. app is a call node: each request calls every outgoing edge (sequentially by default, latencies add; parallelCalls makes them parallel, latency = max). Main parameters (defaults in parentheses): replicas (1), capacity per replica (500 req/s), latencyMs (median service time, 10 ms, lognormal), threads per replica (200, the Little's Law concurrency limit), queueLimit (200), autoscale with minReplicas, maxReplicas, targetUtil and warmupS (1, 20, 0.6, 60 s), and leakPerMin for soak tests. Edge parameters (timeoutMs, retries, backoff, circuitBreaker, fallback) protect its calls. Scenarios: monolith, scale-out, cascading-failure.

Real products.

Learn more: Compute › Services · Compute › Serverless & autoscaling · Estimation › Queueing · Reliability › Patterns · Hands-on 1 · Hands-on 2

Worker

What it is. A background process that pulls jobs from a queue or stream and processes them outside the user's request: sending e-mails, rendering PDF tickets, transcoding video, calling a slow partner API, recomputing a feed. Workers are the consumers of the async world: the producer returns immediately ("your order was received") and the work happens a few seconds later, at the pace the workers can sustain.

When to use.

• Work that is slow, heavy or unreliable and that the user does not need to wait for (BoxOffice: e-mail confirmation, PDF ticket, invoice).
• To absorb spikes: the queue holds the backlog while workers drain it at a steady rate (scenario async-queue).
• To retry failed work safely, with backoff and a dead-letter queue (DLQ) for the jobs that keep failing.
• Batch jobs: nightly reconciliation of payments, re-indexing the search catalog.

When not to (or watch out).

• When the user needs the result now (is my seat held?) — that is a synchronous call.
• Delivery is usually at least once: the same job can run twice → make the worker idempotent (dedupe by job/event id).
• A queue hides overload: if producers outpace consumers for long, the backlog and the end-to-end delay grow without bound → alert on consumer lag and autoscale workers on queue depth.
• Ordering is not guaranteed by most queues; when order matters per key, use a partitioned log (Kafka) with one consumer per partition.

Key numbers.

Failure modes.

• Poison message — one job always crashes the worker and is redelivered forever → max receive count + DLQ.
• Duplicate processing — the worker crashes after the effect but before the ack → idempotent consumers, dedupe table.
• Growing lag — consumers slower than producers → autoscale on queue depth (KEDA), more partitions, faster code.
• Downstream overload — 100 workers hammer a provider with a quota → limit worker concurrency to the provider's rate limit.

In the playground. worker is a call node that sits behind a messaging node: the queue holds a backlog and the workers drain it at replicas × capacity (defaults 2 × 100 messages/s). Watch backlog, consumer lag and end-to-end (async) latency in the metrics panel; with Kafka/streams, parallelism is capped by min(partitions, worker replicas). Parameters: replicas, capacity, latencyMs (50 ms), autoscale. Scenarios: async-queue, video-streaming, notification-system.

Real products.

Learn more: Compute › Async work · Messaging › Queues & pub/sub · Messaging › Semantics · Hands-on 7

Serverless

What it is. Functions as a Service (FaaS): you upload a function and the platform runs it per event (an HTTP request, a message, a file upload, a timer), scaling from zero to thousands of concurrent instances and billing per invocation and per millisecond of execution. You do not manage servers, but you accept the platform's limits: maximum duration, memory, concurrency quotas and cold starts (the delay when a new instance must be created).

When to use.

• Spiky or unpredictable, short workloads: image thumbnails after upload, webhooks from the payment provider, a nightly report.
• Glue code between managed services (a new object in storage → process it → write to a queue).
• Low-traffic APIs where paying for idle servers would dominate the cost.

When not to (or watch out).

• Latency-critical paths that cannot tolerate cold starts (hundreds of ms to seconds), unless you pay for provisioned/minimum instances.
• Long-running or stateful work (Lambda's maximum duration is 15 minutes).
• Constant high throughput: at steady high load, containers or VMs are usually cheaper per request.
• Scaling fast can kill your database: 1,000 concurrent functions each opening a connection exhaust PostgreSQL → connection pooler/proxy, concurrency limits.

Key numbers.

Failure modes.

• Throttling — the concurrency limit is reached and invocations fail (429) → reserved concurrency, queue in front, raise quotas.
• Cold-start tail latency — p99 dominated by new instances → provisioned/minimum instances, lighter runtimes, snapshots.
• Connection storms on the database → RDS Proxy / PgBouncer, limit concurrency.
• Retries of async invocations cause duplicates → idempotent handlers.
• Runaway cost — a loop (function writes to the bucket that triggers itself) → guards, budgets and alerts.

In the playground. serverless is a call node with autoscaling built in: requests that need a new instance pay a cold start (coldStartMs, default 600 ms) unless provisioned instances are kept warm, and above concurrencyLimit (default 1,000) requests are throttled (429). Cost is per invocation plus GB-seconds (memGB). Compare it with an app node under the spike profile. Scenarios: serverless-burst, video-streaming.

Real products.

Learn more: Compute › Serverless & autoscaling · Cloud-native › FinOps · Case study: video streaming

Auth Service

What it is. The service that answers who are you? (authentication) and issues credentials the rest of the system can verify cheaply: usually OAuth 2.1 / OpenID Connect (OIDC) with short-lived access tokens (often JWTs, signed so any service can validate them locally) and longer-lived refresh tokens. It manages users, passwords (hashed with Argon2 or bcrypt), multi-factor authentication, social login and sessions. What may you do? (authorization) is then checked by the gateway and the services using the token's claims.

When to use.

• Always, for any system with accounts. In BoxOffice, buying requires login; browsing does not.
• As a separate service (or a managed identity provider) so that login spikes and security fixes do not touch the business services.
• Centralize token validation at the API gateway; services still check fine-grained permissions (can this user cancel this order?).

When not to (or watch out).

• Do not build your own crypto or password storage; use a proven identity provider or library.
• Do not call the auth service on every request to validate a token — verify the JWT signature locally with the cached public keys (JWKS).
• Long-lived JWTs cannot be revoked easily → short expiry (minutes) + refresh tokens + a revocation list for emergencies.
• Login is a favorite attack target → rate limits, credential-stuffing protection, MFA.

Key numbers.

Failure modes.

• Auth outage = total outage for logged-in features → redundancy, local token validation keeps existing sessions working.
• Login storm at a fixed time, amplified by slow password hashing → autoscaling, encourage login before the sale opens.
• Key rotation breaks validation → publish new keys in the JWKS before signing with them, cache with refresh.
• Token leakage → short lifetimes, HTTPS only, secure cookie flags, PKCE for public clients.

In the playground. auth is a call node like app (parameters replicas, capacity (3,000 req/s), latencyMs (4 ms), threads, autoscale). Put it behind the gateway with an edge ratio < 1 to model that only some requests (logins, token refreshes) reach it, while the rest are validated locally. Build it into a blank design or extend flash-sale.

Real products.

Learn more: Network & security › Identity · Network & security › Zero trust · Compute › Services · Hands-on 4

Search

What it is. A search service answers text queries ("coldplay são paulo", with typos) ranked by relevance, plus filters and facets (date, city, price). It is built on an inverted index: for every term, the list of documents that contain it. The index is usually a secondary copy of data whose source of truth is the database, kept up to date by events or change data capture (CDC); updates become visible after a short refresh (near-real-time).

When to use.

• Full-text search, fuzzy matching, relevance ranking, autocomplete, faceted navigation — things a LIKE '%...%' in SQL does badly and slowly.
• In BoxOffice: searching events by artist, venue, city and date.
• Log search and analytics over semi-structured documents.
• Hybrid search (keywords + vector similarity) for AI assistants.

When not to (or watch out).

• Not a source of truth: no strong consistency, no multi-document transactions. Seat availability must come from inventory, not from the index.
• Keeping the index in sync is a design task of its own (dual writes are unsafe → outbox/CDC).
• Exact lookups by id or simple filters are cheaper in the primary database.
• Shards are fixed when an index is created in Elasticsearch/OpenSearch; resizing means reindexing.

Key numbers.

Failure modes.

• Index out of sync — an event cancelled in the DB still appears → CDC/outbox pipeline, periodic reconciliation, show availability from the source of truth.
• Expensive queries (leading wildcards, huge aggregations) saturate the cluster → query limits, timeouts, caching of popular queries.
• Hot shards → better routing/sharding, replicas for reads.
• Reindexing under load → build a new index in the background and switch an alias.

In the playground. search is a call node (parameters replicas, capacity (800 queries/s), latencyMs (25 ms), threads, autoscale). Put a cache in front of it for popular prefixes and watch the hit ratio cut its load. Scenario: search-autocomplete.

Real products.

Learn more: Databases › Specialized · Messaging › Patterns · Case study: search autocomplete

Scheduler

What it is. A component that triggers work by time rather than by a request: "every day at 03:00 reconcile payments", "every minute release expired seat holds", "send a reminder 24 h before each show". Simple schedulers run cron expressions; distributed ones make sure each job runs once even with several replicas (leader election or a lock), keep history, retry failures and handle many delayed jobs (one per order or per show).

When to use.

• Periodic maintenance: releasing expired holds in inventory, cleaning data, rotating keys, refreshing materialized views.
• Business timers: reminders before an event, abandoned-cart nudges, opening the sale at 10:00 sharp.
• Batch pipelines: nightly exports to the data warehouse, reports.

When not to (or watch out).

• Thundering herd at round times: thousands of jobs set for 00:00 fire together → spread start times (jitter).
• Running cron on every app replica executes the job N times → one scheduler, a distributed lock, or leader election.
• A job that runs longer than its interval overlaps itself → forbid concurrent runs.
• For "do X when Y happens", an event is better than polling on a timer.

Key numbers.

Failure modes.

• Double execution — two replicas both fire → leader election, locks with fencing tokens, idempotent jobs.
• Missed execution — the scheduler was down at 03:00 → catch-up policy, monitoring "job did not run" (dead-man's switch alert).
• Burst at the hour overloads queues and providers → jitter, rate-limited dispatch through a queue.
• Clock and time-zone bugs (daylight saving time, UTC vs local) → schedule in UTC, test DST transitions.

In the playground. scheduler generates its own periodic load: jobs every everyS seconds (defaults 1,000 every 60 s, all at once unless spreadS spreads them) into its targets (usually a queue that feeds workers), independent of the client traffic. Increase N to see the burst at each tick and how a queue smooths it. Scenario: notification-system.

Real products.

Learn more: Compute › Async work · Distributed › Consensus · Case study: notification system

Notifications

What it is. A service that turns "tell this user X" into actual messages on the right channel — push (APNs, FCM), e-mail, SMS, in-app — respecting user preferences and opt-outs, rendering templates, deduplicating, rate-limiting per user and retrying when providers fail. It sits behind a queue: producers publish "order confirmed" events and the notification service fans them out to channels at a pace the external providers accept.

When to use.

• Whenever several parts of the system need to reach users: order (confirmation), inventory (hold about to expire), scheduler (show reminders), marketing.
• To isolate the business services from the quirks of providers (quotas, outages, different APIs).
• To have one place for preferences, quiet hours, localization and auditing ("was this user notified?").

When not to (or watch out).

• Never send notifications synchronously inside the purchase request; publish an event and return.
• Push and SMS are best effort: important information must also be in the app or e-mail.
• Do not notify twice: at-least-once delivery requires a dedupe key per (user, notification).
• Respect laws and user consent (opt-in for marketing, unsubscribe links).

Key numbers.

Failure modes.

• Provider outage or throttling (429) → queue + retries with backoff, a second provider for critical channels.
• Duplicate notifications → idempotency key per message, dedupe store.
• Notification storm — a bug or a big event sends millions at once → per-user rate limits, global send rate, kill switch.
• Bad template breaks every message → template tests, staged rollout.

In the playground. notifications is a call node usually placed after a queue: it consumes the backlog and calls the external email provider, whose quota (default 100 sends/s) and random outages make retries and DLQs necessary. Parameters: replicas, capacity (1,000 req/s), latencyMs (15 ms), autoscale; on its edge to the provider, timeoutMs, retries, backoff. Scenarios: notification-system, flash-sale.

Real products.

Learn more: Compute › Integrations · Messaging › Queues & pub/sub · Case study: notification system · Hands-on 7

Analytics

What it is. The pipeline that turns raw events (page views, searches, holds, purchases, clicks) into numbers people can use: dashboards ("tickets sold per minute"), funnels (how many who held a seat actually paid?), fraud signals, recommendations. Events are collected asynchronously, usually through a stream, then processed in real time (stream processing) and/or in batch in a data warehouse. It is the read side of the business, kept away from the transactional databases.

When to use.

• Product and business metrics, A/B tests, funnels, sales dashboards for event organizers.
• Real-time counters during the flash sale (sold, held, waiting room size).
• Click analytics of a URL shortener, view counts of a video platform.

When not to (or watch out).

• Do not run analytical queries on the production OLTP database: a big GROUP BY can slow down purchases → replicate to a warehouse (CDC/ELT).
• Do not make the user request wait for the analytics write: fire-and-forget into a stream.
• Analytics numbers are usually approximate and slightly delayed; do not use them to enforce invariants (seat counts).
• Personal data: anonymize/aggregate and respect retention rules (LGPD/GDPR).

Key numbers.

Failure modes.

• Analytics backlog delays dashboards → it is async; scale consumers, accept lag, never block the purchase path.
• Double counting with at-least-once delivery → event ids and dedupe, or tolerate small errors.
• Schema drift — a producer changes the event shape and breaks pipelines → schema registry, versioned events.
• Hot partitions — one huge event floods one partition → partition by a well-distributed key.

In the playground. analytics is a call node normally attached asynchronously (through a stream, kafka or pubsub node) so it is off the request path: watch its consumer lag grow during spikes without affecting user latency. Parameters: replicas, capacity (2,000 events/s), latencyMs (20 ms). Scenarios: event-fanout, url-shortener.

Real products.

Learn more: Databases › Specialized · Messaging › Kafka · Case study: URL shortener · Case study: news feed