Protocols

The BoxOffice seat map has a problem. During the sale, seats change colour every second as other fans hold them. The first version of the web app asked GET /api/events/42/seats every 2 seconds: 2 million fans × 0.5 requests per second = 1 million requests per second, most of them answering "nothing changed". Meanwhile, fans on a train in São Paulo complain that the app "freezes" every time the phone switches between cell towers. Both problems are protocol problems — the choice of how bytes travel between the phone and the edge — and both have good modern answers.

This page reviews the protocols that run on the edge: the three generations of HTTP, the TLS handshake that protects them, the options for real-time updates (polling, long polling, Server-Sent Events, WebSockets), and gRPC for service-to-service calls. The goal is not to memorise headers, but to know what each protocol costs in round trips and connections, and what it is good for.

Round trips are the currency

Bandwidth keeps growing; the speed of light does not. A round-trip time (RTT) is the time a packet takes to go to the server and back: around 1–10 ms inside a city, around 30–60 ms across a continent, around 100–200 ms between continents. Every protocol exchange that waits for an answer costs at least one RTT, and on a new connection there are several of them before the first byte of data: one to open TCP, one or two for TLS⁷.

That is why the history of web protocols is mostly a history of removing round trips and using connections better.

HTTP/1.1

HTTP/1.1 (1997, current specification RFC 9112) is a text protocol: the client writes GET /api/events/42 HTTP/1.1, headers, a blank line; the server answers with a status line, headers and a body¹.

• Persistent connections (keep-alive): one TCP connection can carry many requests one after the other, avoiding a new handshake for each.
• One request at a time per connection: a second request must wait until the first response has arrived completely. If the first response is slow, everything behind it waits — head-of-line (HOL) blocking. (Pipelining, which was meant to fix this, was never usable in practice.)
• Browsers work around it by opening about 6 parallel connections per host, and developers used to work around that with tricks like domain sharding, sprites and bundling.
• Headers are plain text and repeated on every request: cookies and user agents can add hundreds of bytes per request.

HTTP/1.1 is still everywhere — between a load balancer and its backends, in simple clients, in curl — and it is perfectly fine for low-concurrency, request-response traffic.

HTTP/2

HTTP/2 (2015, RFC 9113) keeps the same semantics — methods, status codes, headers — but changes how they travel²:

• Binary framing: messages are split into frames, each tagged with a stream id.
• Multiplexing: many requests and responses travel at the same time over one TCP connection, their frames interleaved. A slow response no longer blocks the others at the HTTP level.
• Header compression (HPACK): repeated headers are sent as small references to a table shared by both ends.
• Stream priorities and flow control per stream.

One connection per origin is enough, so the domain-sharding tricks become counterproductive. But one problem remains, one layer down: TCP delivers bytes in order. If a single packet is lost, TCP holds back all the bytes after it — of every stream — until the retransmission arrives. On a lossy mobile network, HTTP/2's single connection can be slower than HTTP/1.1's six. This is TCP head-of-line blocking, and it is the reason for HTTP/3.

HTTP/3 and QUIC

HTTP/3 (2022, RFC 9114) runs over QUIC (RFC 9000), a transport protocol built on top of UDP³. QUIC moves into user space what TCP did in the kernel, and redesigns it:

• Independent streams in the transport: a lost packet only delays the stream it belongs to; the others keep flowing. No TCP head-of-line blocking.
• TLS 1.3 built in: the transport and cryptographic handshakes are combined, so a new connection needs 1 RTT before sending data — and a resumed one can send data in the very first packet (0-RTT).
• Connection migration: a QUIC connection is identified by a connection ID, not by the (IP, port) pair. When the phone moves from Wi-Fi to 4G, its IP changes but the connection survives — no new handshake, no "freeze".
• Always encrypted, including most of the transport headers, which also makes it harder for middleboxes to interfere.

The costs: UDP is sometimes blocked or throttled by corporate networks (browsers fall back to HTTP/2 automatically), QUIC uses more CPU than kernel TCP, and load balancers must route by connection ID to keep migrating connections on the same server. Browsers, CDNs and the big platforms adopted it quickly; inside the data centre, HTTP/1.1 and HTTP/2 remain the norm.

TLS 1.3: the handshake

Every connection on the edge is encrypted with TLS. Before any HTTP data, client and server must agree on keys and the client must verify the server's certificate (that it really is boxoffice.com). TLS 1.2 needed two round trips for that; TLS 1.3 (2018, RFC 8446) needs one, by having the client guess the key-exchange parameters and send its key share immediately⁴.

sequenceDiagram
    participant C as Fan's phone
    participant S as CDN edge (boxoffice.com)
    C->>S: TCP SYN
    S-->>C: SYN-ACK
    C->>S: ACK + TLS ClientHello (key share, supported ciphers, SNI = boxoffice.com)
    S-->>C: ServerHello (key share) + certificate + Finished (encrypted)
    Note over C: verify certificate, derive keys
    C->>S: Finished + GET /api/events/42 (encrypted)
    S-->>C: 200 OK (encrypted)

With QUIC, the TCP exchange disappears: the first QUIC packet already carries the ClientHello, so the request goes out after one round trip. With 0-RTT resumption, a client that talked to the server recently reuses a pre-shared key and sends the request in the first flight — with a caveat: 0-RTT data can be replayed by an attacker, so servers only accept idempotent requests (GET) in 0-RTT.

The chart counts the round trips before the first response byte on a new connection (server processing time ignored), at three distances:

Two conclusions. First, the protocol matters most when the server is far away — 600 ms versus 150 ms intercontinental — which is also why CDNs terminate connections close to the user: with an edge in the same city, even TLS 1.2 costs only 40 ms. Second, the best handshake is the one you do not repeat: keep-alive and connection pooling (between the gateway and the services, too) make the setup cost disappear for every request after the first.

Real-time: getting updates to the client

HTTP is request-response: the client asks, the server answers. For the seat map, the waiting-room position or a chat, the server knows when something changed. There are four ways to get that information to the client.

Short polling

The client asks every N seconds: "anything new?". Trivial to implement and to cache, but wasteful (most answers are "no") and slow to react (up to N seconds of delay). Polling with micro-caching at the CDN (DNS & CDN) is a perfectly good design for the seat map when a few seconds of delay are acceptable.

Long polling

The client asks, and the server holds the request open until there is news (or a timeout of, say, 30 s), then answers; the client immediately asks again. Near real-time over plain HTTP, but every message costs a full request, and servers must hold many open requests.

Server-Sent Events (SSE)

The client opens one ordinary HTTP request with Accept: text/event-stream; the server keeps the response open and streams events as lines of text, for as long as it wants:

event: seat
id: 1842
data: {"seat": "B-12", "status": "held"}

event: seat
id: 1843
data: {"seat": "B-13", "status": "sold"}

SSE is one-way (server → client), text-only, and runs over normal HTTP — so it passes through proxies, CDNs and load balancers, works over HTTP/2 multiplexing, and the browser's EventSource API reconnects automatically, sending the last received id in a Last-Event-ID header so the server can resume. It is the simplest good answer for "push updates to many clients": live scores, notifications, the waiting-room position, and the token-by-token stream of an LLM response.

@app.post("/assistant/chat/stream")
async def chat_stream(q: Question, x_user_email: str | None = Header(default=None)):
    question = guarded(q)

    async def events():
        async for kind, data in agent.run(question, x_user_email, q.route):
            yield agent.sse(kind, {"text": data} if kind == "token" else data)

    return StreamingResponse(events(), media_type="text/event-stream", headers={"Cache-Control": "no-cache"})

WebSockets

A WebSocket (RFC 6455) starts as an HTTP/1.1 request that asks to upgrade the connection; after the 101 Switching Protocols answer, the TCP connection becomes a full-duplex channel of messages in both directions, with very low per-message overhead⁵:

sequenceDiagram
    participant C as Browser
    participant L as Load balancer (L7)
    participant W as ws service
    C->>L: GET /ws (Upgrade: websocket, Sec-WebSocket-Key)
    L->>W: forward the upgrade request
    W-->>C: 101 Switching Protocols (Sec-WebSocket-Accept)
    Note over C,W: same TCP connection, now a two-way message channel
    C->>W: {"type": "join", "room": "event-42"}
    W-->>C: {"type": "seat", "seat": "B-12", "status": "held"}
    C->>W: {"type": "ping"}
    W-->>C: {"type": "pong"}

WebSockets are the tool for interactive, bidirectional traffic: chat, multiplayer games, collaborative editing. They come with operational costs: connections are stateful and long-lived (a server restart disconnects thousands of users, who all reconnect at once), load balancers must support the upgrade and long idle timeouts, and you need a pub/sub backbone so that a message published on one server reaches clients connected to the others (the chat case study).

gRPC: calls between services

Between BoxOffice's own services, the priorities are different: no browsers, both sides under our control, many calls per user action, and a need for strict contracts. gRPC is an RPC (Remote Procedure Call) framework from Google that runs over HTTP/2 and encodes messages with Protocol Buffers (protobuf), a compact binary format generated from a schema⁶:

syntax = "proto3";
package boxoffice.inventory.v1;

service Inventory {
  rpc HoldSeats (HoldRequest) returns (Hold);                   // unary
  rpc WatchSeats (WatchRequest) returns (stream SeatEvent);     // server streaming
}

message HoldRequest {
  string event_id = 1;
  repeated string seat_ids = 2;
  string user_id = 3;
  string idempotency_key = 4;
}

message Hold {
  string hold_id = 1;
  int64 expires_at_epoch_ms = 2;
}

message WatchRequest { string event_id = 1; }
message SeatEvent { string seat_id = 1; string status = 2; }

From this file, the protobuf compiler generates client and server code in Java, Python, Go and others. gRPC offers four call types — unary, server streaming, client streaming and bidirectional streaming — and has deadlines built in: the caller says "I need an answer within 300 ms", and the deadline propagates through the chain of calls, so downstream services stop working on requests nobody is waiting for anymore.

The full comparison of API styles, with versioning and design advice, is in Compute — API styles.

Long-lived connections and load balancing

HTTP/2, gRPC, SSE and WebSockets all keep connections open for a long time. That has a consequence that surprises many teams: load is balanced when a connection is opened, not when a request is sent — unless an L7 proxy balances each request or stream.

• After a scale-out, the new replicas get no traffic: the existing clients are still attached to the old ones.
• After a deploy or a crash, all clients of a server reconnect at the same moment — a reconnection storm.
• An L4 load balancer sees one gRPC connection and sends all of the caller's requests to one replica (Load Balancing).

Remedies: L7 balancing per request (Envoy, service mesh) or client-side balancing for gRPC; a maximum connection age so clients rotate periodically; jittered reconnection backoff on the client; and draining that closes connections gradually during deploys.

Trade-offs at a glance

How it connects

flowchart LR
    RTT["round trips"] --> TLS["TLS 1.3, 0-RTT"]
    RTT --> QUIC["HTTP/3 over QUIC"]
    RTT --> LAT["latency budget<br/>→ Estimation"]
    H2["HTTP/2 streams"] --> GRPC["gRPC between services<br/>→ Compute: API styles"]
    RT["real-time push"] --> SSE["SSE: seat map, queue position"]
    RT --> WS["WebSockets: chat, games"]
    WS --> PS["pub/sub fan-out<br/>→ Messaging"]
    WS --> LL["long-lived connections<br/>→ load balancing"]
    TLS --> MTLS["mTLS inside the cluster<br/>→ service mesh"]

Palette components: client and mobile (choose the protocol) · cdn and lb (terminate TLS, speak HTTP/2 and HTTP/3) · pubsub (fan-out behind WebSockets) · mesh (mTLS between services). Scenarios: chat, video-streaming, ai-assistant (streamed tokens). Continues in: Compute — API styles, Case study: chat, Network & Security — service mesh.

Key takeaways