System Architecture · Insper · Lab 5 · Chapter 7

Lab 5 — Caching

Redis cache-aside on catalog: hit ratio, p99, invalidation — and a stampede, reproduced and fixed

The story so far

before · lab 4a gateway in front of 3 stateless replicas
→
chapter 7 · lab 5Redis cache-aside on catalog
→
next · lab 6chapter 6: seats, holds, locks

Every replica asks PostgreSQL, and the event pages are read ~100× more than they change. How many of those reads can memory answer?

  • Chapter 7 before 6: reads first, with the catalog you have; seats wait for lab 6.

The perfect cache candidate

How many times is "upcoming events in São Paulo, page 1" read for each time it changes?

  • Thousands of reads, a few writes per day.
  • Strong popularity skew: a few events get most views.
  • Lab 2: the database pool is the bottleneck.

What you will build

fan
→
edge → gatewayround robin
→
catalog-1..3EventCache · X-Cache header
→
shared by the three replicas
redis :6379256 MB, allkeys-lru, no persistence
dbPostgreSQL: only on a miss
  • First GET /catalog/events/42: X-Cache: MISS — the database answers, the JavaScript Object Notation (JSON) goes to Redis.
  • Second one, even on another replica: X-Cache: HIT — no database access.

Cache-aside in EventCache

GET key
hit →
return the JSONX-Cache: HIT
GET key
nil →
SELECT … in PostgreSQL
→
SET key json PX ttl + 0–10%
→
returnX-Cache: MISS
Pattern Who loads Writes
cache-aside ✅ the app, on miss database, then evict
read-through the cache —
write-through — cache + database, sync
write-behind — cache now, database later ⚠️

Keys and time to live (TTL)

Key TTL Invalidation
catalog:event:{id} 60 s evicted on update/delete
catalog:events:city=…:page=… 10 s expires only
  • Store the response JSON: a hit costs no database access and no serialization.
  • Jitter the TTL (+0–10%): keys written together do not expire together.

Invalidation

PUT /catalog/events/42admin token
→
UPDATE + COMMIT
→
DEL catalog:event:42next read: MISS, new data
listing keys untouchedstale for up to 10 s
  • Evict, don't overwrite: concurrent updates could cache an older version.
  • Every key keeps a TTL: the safety net for every race.

An admin changes a price. How long can a fan see the old one, on each page?

When Redis fails

GET key
→
Redis down or slow200 ms timeout
→
log, treat as a miss
→
200 from PostgreSQLX-Cache: MISS
  • Lettuce's default timeout is 60 s → set 200 ms.
  • Every Redis error becomes a miss: slower, but working.
  • No persistence: after a restart the cache is cold.

If the cache absorbed 95% of reads, its outage sends 20× the load to the database.

Hit ratio math

  • Database reads = (1 − h) × total reads.
  • h = 90% → the database sees 10%. h = 99% → it sees 1%.
  • Hit ratio depends on key space, skew and TTL — not on Redis speed.

Measure it

CACHE_ENABLED=false docker compose up -d --wait && k6 run -e RATE=300 load/cache.js
CACHE_ENABLED=true  docker compose up -d --wait && docker compose exec redis redis-cli FLUSHALL
k6 run -e RATE=300 load/cache.js     # cold
k6 run -e RATE=300 load/cache.js     # warm
  • 80% of the detail requests go to the 2,000 hottest events (1% of the catalog).
  • k6 reads X-Cache: cache_hit rate, latency_hit, latency_miss.
  • A run for this lab: ~72% cold, 72% again warm with the 60 s TTL; with CACHE_TTL_EVENT=5m, 83% warm vs 71% cold.

The stampede

  • A hot key expires.
  • Every request arriving during the reload misses.
  • 200 identical heavy queries hit the database at once.
CACHE_TTL_LIST=3s CACHE_STAMPEDE_PROTECTION=false docker compose up -d --wait
k6 run load/stampede.js       # cache_misses ≫ expirations

Fix: one rebuild per key

miss
→
SET key:lock token NX PX 2000
won →
re-check, load, SET, releaseone query per expiry
everyone else
lost →
poll the cache every 25 ms≤ 500 ms, then load anyway
→
HIT
  • A test run (200 virtual users, 60 s): misses 2,238 → 19; p99 ~1 s → ~0.3 s; db central processing unit (CPU) 234% → 4%.
  • Lock expires if the owner dies; release only your token (atomically: a Lua script).

Other cures

Technique Trade-off
TTL jitter does not help one hot key
probabilistic early refresh some duplicate work
stale-while-revalidate clients see stale data briefly
refresh-ahead must know the hot keys
edge request coalescing only cacheable Hypertext Transfer Protocol (HTTP) responses

Without Docker: the simulator

Check it (step 5)MISS on catalog-2, HIT on catalog-3, TTL 62
Listing after an updatea stale HIT for 7 s, ≤ CACHE_TTL_LIST
TTL and BYPASSone miss per expiry, every BYPASS a query
  • The Cache-aside page embeds lab 5's cache as a simulator: same keys, TTLs and outputs.
  • Stop Redis: still a 200; start it: empty, since there is no persistence.

What breaks next

lab 5 · readsthe database sees only 1 − h of the reads
→
the writesno cache may answer "is seat 42 free?" · 33,000 holds/s vs ~1,100
→
lab 6 · chapter 6inventory: which lock never sells a seat twice?
  • At 10:00 the fans stop reading and start holding seats.

Takeaways

  1. Cache-aside: the app loads on miss, evicts on write, every key has a TTL.
  2. A cache is only as good as its hit ratio — which feeds on skew.
  3. The cache is an optimization: fail open to the database, with short timeouts.
  4. Hot keys expiring cause stampedes: collapse concurrent misses into one load.
  5. State the staleness you accept — it is a product decision.

Chapter 7 of the story of the course, built before chapter 6. Outline: (1) why cache the catalog; (2) what you will build; (3) cache-aside and EventCache; (4) keys and time to live; (5) invalidation; (6) failure handling; (7) hit ratio math and measuring; (8) stampede and the lock; (9) other cures; (10) the simulator.

The story of the course (2027.1 overview, "The story of the course"): chapter 7 opens with the ~100x read ratio. Lab 4 measured the front door as cheap (a fraction of a ms to a few ms at p50), so the cost of a read is the database. The labs take chapter 7 before chapter 6.

With 60 s, the keys written early in the cold run expire early in the next one: each hot key misses about once per minute in both. A TTL longer than the run shows the warm cache win. Warm-ups before each run: the recreated replicas are cold JVMs.

The simulator sits at the end of step 5 of Cache-aside.