Queues & Pub/Sub

Every ticket sold on BoxOffice needs a PDF with a QR code. Generating it takes about 400 ms of CPU — far too slow to do inside the purchase request during a flash sale. The obvious fix is to put the job somewhere and let a pool of workers pick it up. That "somewhere" is the oldest and simplest messaging component: the queue. A few lines later the product team asks for an e-mail, a push notification, a search update and an analytics record for each purchase — four different services that each need their own copy. That is the second shape: publish/subscribe.

This page covers both shapes, how they behave under load, and the most famous queue of BoxOffice: the waiting room that keeps two million fans from crushing the seat inventory at 10:00.

Point-to-point queues

A message queue holds messages until a consumer takes them. Each message is delivered to one consumer; many consumers can read the same queue, and they compete for messages — the competing consumers pattern¹. Adding consumers increases throughput without any change to the producer.

flowchart LR
    order["order"] -->|"GenerateTicketPdf"| q[["queue: ticket-pdf"]]
    q -->|"msg 1"| w1["worker 1"]
    q -->|"msg 2"| w2["worker 2"]
    q -->|"msg 3"| w3["worker 3"]
    w1 --> os[("object store<br/>tickets/*.pdf")]
    w2 --> os
    w3 --> os

The life of a message

A queue cannot simply delete a message when a consumer receives it: the consumer might crash halfway through. So delivery is a two-step protocol — receive, then acknowledge:

1. The consumer receives a message. The broker does not delete it; it hides it from other consumers (SQS calls this the visibility timeout; RabbitMQ keeps it as unacknowledged on that consumer's channel).
2. The consumer processes it and sends an acknowledgement (ack). Only now is the message deleted.
3. If the consumer crashes, or the visibility timeout expires before the ack, the message becomes visible again and another consumer receives it.

sequenceDiagram
    participant Q as queue ticket-pdf
    participant W1 as worker 1
    participant W2 as worker 2
    Q->>W1: GenerateTicketPdf (order o-7731)
    Note over Q: message hidden for 30 s
    W1->>W1: render PDF...
    Note over W1: crashes before ack
    Note over Q: 30 s pass without ack: visible again
    Q->>W2: GenerateTicketPdf (order o-7731), delivery 2
    W2->>W2: render PDF, upload
    W2->>Q: ack
    Note over Q: message deleted

This protocol gives at-least-once delivery: no message is lost when a worker dies, but a message can be processed twice (worker 1 may have uploaded the PDF just before crashing). Every queue consumer must therefore be idempotent — see Delivery semantics. Here it is easy: the PDF is stored under a deterministic name (tickets/o-7731.pdf), so writing it twice gives the same result.

Ordering

With competing consumers, order is not preserved: message 2 may finish before message 1 because they run on different workers. For most jobs (PDFs, e-mails) this is fine. When it is not, you need a way to serialize related messages:

• FIFO queues with a group id (SQS FIFO MessageGroupId, Azure Service Bus sessions): messages with the same group are delivered in order, one at a time; different groups run in parallel.
• A partitioned log keyed by the entity — Kafka, next page — gives the same "ordered per key, parallel across keys" behaviour at high throughput.

Sizing the worker pool

Little's Law (see Queueing) tells you how many workers keep up with a steady arrival rate: the number of jobs in progress equals the arrival rate times the time each one takes.

Backpressure

Backpressure is any mechanism by which a slow consumer makes a fast producer slow down — or at least makes the system notice that it cannot keep up. Without it, a queue between a fast producer and a slow consumer is just a place where the problem grows quietly.

A queue that is allowed to grow without limit is a lie: memory or disk runs out eventually, and long before that the messages at the tail become stale (a "your seats are held" push that arrives two hours late is worse than none). Every queue needs an answer to "what happens when it is full?"²:

The flash-sale waiting room

At 10:00, two million fans arrive for 60,000 seats. inventory can hold seats safely (with row locks, see Transactions) at about 1,100 holds per second — the order of magnitude of class 2's estimate, against 33,000 that arrive; you would measure it with a load test. Letting everybody hit the purchase path at once does not sell tickets faster; it makes the database thrash and the whole site fail.

The virtual waiting room is a queue placed in front of the purchase flow, at the edge:

1. Fans who arrive are given a position in the waiting room instead of the purchase page.
2. An admission job lets fans out at a controlled rate, matched to what inventory can handle with margin.
3. An admitted fan receives a short-lived, signed admission token; the gateway rejects purchase calls without one.
4. When the inventory reaches zero, the waiting room tells everybody still waiting "sold out" — cheaply, from the edge, without touching the database.

sequenceDiagram
    participant F as Fan
    participant W as waiting room (edge)
    participant R as Redis (queue)
    participant G as gateway
    participant I as inventory
    F->>W: GET /events/ev-88/buy (10:00:01)
    W->>R: ZADD room:ev-88 score=arrival fan-123
    W-->>F: position 512,384 (about 17 min)
    loop every 15 s
        F->>W: GET /room/status
        W->>R: ZRANK room:ev-88 fan-123
        W-->>F: position 30,112
    end
    Note over R: admission job: 500 fans/s
    W-->>F: admitted + token (valid 10 min)
    F->>G: POST /holds (Authorization: admission token)
    G->>I: hold seats B-12, B-13
    I-->>F: held until 10:21

The queue itself can be very simple. A Redis sorted set ordered by arrival time gives positions in \(O(\log n)\):

# a fan joins (score = arrival time in ms; NX: joining twice keeps the first position)
ZADD room:ev-88 NX 1804406400123 fan-123

# where am I? (0-based rank)
ZRANK room:ev-88 fan-123

# admission job, once per second: let the next 500 fans in
ZPOPMIN room:ev-88 500

# how many are still waiting?
ZCARD room:ev-88

Design decisions worth discussing:

• Admission rate. It is a capacity number, not a guess: measured inventory throughput × a safety factor (say 50%): about 1,100 × 0.5 ≈ 500 fans/s, the rate used above. It can adapt: slow admissions down when inventory p99 or error rate rises — a feedback loop.
• Fairness. Strict FIFO rewards whoever has the lowest latency to the data center (and bots). Many ticketing sites give fans who arrived before the on-sale time a random position at 10:00, and FIFO only after that.
• Bots. The waiting room is the place to require a challenge and apply bot detection and per-account limits (see Traffic & edge).
• Hold expiry. Admission gives a fan a time-boxed right to buy; the seat hold expires (here, after 10 minutes) and the seats go back to the pool. That is a timer — another asynchronous job.
• Where it runs. Positions are served from the edge (CDN functions or a small, horizontally scaled service with Redis), so polling by two million browsers never reaches the core services.

public Ticket join(String showId) {
    String ticket = UUID.randomUUID().toString();
    Long arrival = redis.opsForValue().increment(seq(showId)); // the server decides the order
    redis.opsForZSet().add(queue(showId), ticket, arrival);
    return status(showId, ticket);
}
/** ZPOPMIN + SET in one Lua script: a polling fan never sees a ticket that is neither queued nor admitted. */
private static final RedisScript<Long> POP_AND_ADMIT = RedisScript.of("""
        local popped = redis.call('ZPOPMIN', KEYS[1], ARGV[1])
        for i = 1, #popped, 2 do
            redis.call('SET', ARGV[2] .. popped[i], '1', 'EX', ARGV[3])
        end
        return #popped / 2
        """, Long.class);

/** Runs on ONE instance only (see pitfalls): N replicas would admit N times the rate. */
@Scheduled(fixedRate = 1000)
public void admit() {
    Set<String> shows = redis.opsForSet().members(OPEN_SHOWS);
    if (shows == null) {
        return;
    }
    for (String showId : shows) {
        redis.execute(POP_AND_ADMIT, List.of(queue(showId)), String.valueOf(props.admitPerSecond()),
                admitted(showId, ""), String.valueOf(props.admissionTtl().toSeconds()));
    }
}

public boolean mayEnter(String showId, String ticket) {
    if (!isOpen(showId)) {
        return true; // no on-sale running for this show: no gate
    }
    return ticket != null && Boolean.TRUE.equals(redis.hasKey(admitted(showId, ticket)));
}

Publish/subscribe

In publish/subscribe (pub/sub) the producer publishes to a topic, and each subscription to that topic receives its own copy of every message³. Inside one subscription, consumers compete as in a queue; across subscriptions, every subscription sees everything. In effect, each subscription is a queue that the broker fills automatically.

flowchart LR
    order["order"] -->|"OrderConfirmed"| t{{"topic: order-events"}}
    t --> s1[["sub: notification"]]
    t --> s2[["sub: search"]]
    t --> s3[["sub: analytics"]]
    s1 --> n1["notification #1"]
    s1 --> n2["notification #2"]
    s2 --> se["search indexer"]
    s3 --> an["analytics loader"]

Adding a fourth consumer — say, a loyalty program that grants points — is a new subscription. The order service does not change, is not redeployed and does not even know. That is the main reason to use pub/sub: the producer is decoupled from the set of consumers.

Push or pull

Filtering and routing

Not every subscriber wants every message. Brokers let a subscription declare a filter on message attributes, so only matching messages are copied to it — for example, the SMS service subscribes only to OrderConfirmed events whose channel attribute is sms. RabbitMQ generalizes this with exchanges: a fanout exchange copies to every bound queue, a direct exchange routes by an exact routing key, and a topic exchange routes by patterns such as order.*.br.

Durable and ephemeral pub/sub

Some pub/sub systems do not store messages at all: Redis Pub/Sub and core NATS deliver a message only to subscribers connected at that moment. If a subscriber is down, it simply misses the message. That is perfect for "user is typing…" indicators in a chat and unacceptable for order confirmations. Durable pub/sub (Google Cloud Pub/Sub, SNS → SQS, NATS JetStream, Kafka) keeps each subscription's messages until they are acknowledged.

Queue, pub/sub or log?

The next page introduces the third shape, the log (Kafka, Kinesis). The table summarizes all three; the main question is what happens to a message after it is read.

Products

How it connects

flowchart LR
    P["producer: order"] --> Q["queue<br/>competing consumers"]
    Q --> ACK["receive, then ack<br/>visibility timeout"]
    ACK --> ALO["at-least-once<br/>→ Delivery semantics"]
    Q --> LL["worker pool sized by<br/>Little's Law → Estimation"]
    Q --> BP["backpressure: bound,<br/>scale, reject, drop"]
    BP --> AGE["age of oldest message<br/>→ freshness SLO"]
    WR["waiting room<br/>Redis sorted set"] --> TOK["admission token<br/>checked at the gateway"]
    TOK --> INV["inventory holds on SQL<br/>→ Databases"]
    P --> T["pubsub topic"]
    T --> SUBS["one subscription<br/>per consumer service"]
    T --> LOG["need replay or order per key?<br/>→ Kafka"]

Palette components: queue · pubsub · worker · objstore (the PDFs) · cache (Redis waiting room) · apigw (rejects calls without an admission token) · waf (bots at the waiting room) · sql (inventory) · notifications (one subscription). Scenarios: async-queue, flash-sale, notification-system, chat (ephemeral pub/sub). Continues in: Kafka and event logs, Delivery semantics, Compute — async work, Estimation — queueing, Observability — SLOs, Traffic & edge — WAF and bots, Hands-on 7 — waiting room.