Lab 5 — Caching

The perfect cache candidate

• Thousands of reads, a few writes per day.
• Strong popularity skew: a few events get most views.
• Lab 2: the database pool is the bottleneck.

Cache-aside

GET key ──► hit?  ── yes ─► return (no DB)
              │
              no ─► SELECT … ─► SET key value PX ttl ─► return

Keys and TTLs

• Store the response JSON: a hit costs no DB and no serialization.
• Jitter the TTL (+0–10%).

When Redis fails

• Lettuce's default timeout is 60 s → set 200 ms.
• Every Redis error becomes a miss: slower, but working.

Invalidation

• Evict, don't overwrite: concurrent updates could cache an older version.
• Every key keeps a TTL: the safety net for every race.
• Listings: no eviction, bounded staleness of ~10 s.

Hit ratio math

• Database reads = (1 − h) × total reads.
• h = 90% → DB sees 10%. h = 99% → DB sees 1%.
• Hit ratio depends on key space, skew and TTL — not on Redis speed.

Measure it

CACHE_ENABLED=false docker compose up -d && k6 run -e RATE=300 load/cache.js
CACHE_ENABLED=true  docker compose up -d && docker compose exec redis redis-cli FLUSHALL
k6 run -e RATE=300 load/cache.js     # cold
k6 run -e RATE=300 load/cache.js     # warm

• k6 reads X-Cache: cache_hit rate, latency_hit, latency_miss.

The stampede

• A hot key expires.
• Every request arriving during the reload misses.
• 200 identical heavy queries hit the database at once.

CACHE_TTL_LIST=3s CACHE_STAMPEDE_PROTECTION=false docker compose up -d
k6 run load/stampede.js       # cache_misses ≫ expirations

Fix: one rebuild per key

miss ─► SET key:lock token NX PX 2000
          ├─ won  ─► re-check, load, SET, release
          └─ lost ─► poll the cache every 25 ms (≤ 500 ms)

• One query per expiry, across all replicas.
• Lock expires if the owner dies; release only your token.

Other cures

Takeaways

1. Cache-aside: the app loads on miss, evicts on write, every key has a TTL.
2. A cache is only as good as its hit ratio — which feeds on skew.
3. The cache is an optimization: fail open to the database, with short timeouts.
4. Hot keys expiring cause stampedes: collapse concurrent misses into one load.
5. State the staleness you accept — it is a product decision.