News Feed

"Design the home feed of a social network like Instagram or X: users follow other users and, when they open the app, see the latest posts of the people they follow."

The news feed is the classic read-heavy, fan-out problem. One write (a post) must become visible in the feeds of everybody who follows the author — two people for most users, fifty million for a pop star. The interview turns on one decision — do the work when the post is written, or when the feed is read? — and on what happens at the extremes of that decision: celebrities, inactive users, deleted posts and ranking.

BoxOffice has the same shape: fans follow artists and venues, and the "Following" tab shows tour announcements, new dates and presale codes. When a stadium-level artist with millions of followers announces a tour, BoxOffice faces exactly the celebrity problem of this page — at the worst moment, because everyone will open the app at once.

1. Clarify the requirements

(interview framework)

Functional requirements

1. A user publishes a post (text, optional images/video).
2. A user follows / unfollows other users (asymmetric: following is not friendship).
3. A user opens the home feed: posts of the people they follow, newest first (or ranked), paginated with infinite scroll.
4. Posts show author, counts (likes, comments) and media.

Non-functional requirements

Out of scope (say it explicitly)

Comments and likes as features (we only show counts), search, notifications, ads ranking, direct messages. Each is its own design problem.

2. Back-of-the-envelope estimates

# News feed: assumptions (illustrative)
dau            = 100_000_000
posts_per_dau  = 0.2          # most users read, few post
reads_per_dau  = 10           # feed opens / refreshes per day
peak_factor    = 3
avg_followers  = 200          # average fan-out per post (the tail is much larger)
timeline_len   = 800          # post IDs kept per cached feed
id_b           = 8            # one post ID (64-bit)
cache_overhead = 2            # data-structure overhead in the cache
post_b         = 1_000        # text + metadata
media_share    = 0.1          # posts with an image
image_b        = 500_000

posts_day      = dau * posts_per_dau
write_qps      = posts_day / 86_400
read_qps       = dau * reads_per_dau / 86_400
peak_read_qps  = read_qps * peak_factor
fanout_qps     = write_qps * avg_followers          # timeline inserts per second (push)
timeline_gb    = dau * timeline_len * id_b * cache_overhead / 1e9
posts_tb_year  = posts_day * post_b * 365 / 1e12
media_tb_day   = posts_day * media_share * image_b / 1e12

rows = [
    ("posts_day", posts_day, "posts"), ("write_qps", write_qps, "posts/s"),
    ("read_qps", read_qps, "req/s"), ("peak_read_qps", peak_read_qps, "req/s"),
    ("fanout_qps", fanout_qps, "inserts/s"), ("timeline_gb", timeline_gb, "GB"),
    ("posts_tb_year", posts_tb_year, "TB"), ("media_tb_day", media_tb_day, "TB/day"),
]
print("| quantity | value | unit |\n|---|---:|---|")
for name, value, unit in rows:
    v = f"{value:,.0f}" if abs(value) >= 100 else f"{value:,.2f}"
    print(f"| `{name}` | {v} | {unit} |")

What the numbers tell us:

• Reads (~12 k/s, ~35 k/s at peak) outnumber posts (~230/s) by 50:1. ⇒ make reads cheap, even at the cost of more work on writes.
• But push fan-out turns 230 posts/s into ~46 k timeline inserts/s on average — and the average hides the tail: one post by an account with 50 M followers is 50 M inserts by itself. ⇒ the write path must be asynchronous (queue + workers), and celebrities need special handling.
• Cached timelines of 800 IDs for every daily user take ~1.3 TB of RAM — a large but ordinary cache cluster (tens of nodes). Keeping only IDs, not posts, is what makes this affordable; caching for active users only cuts it further.
• Posts are ~7 TB/year (a partitioned store), media ~1 TB/day (object storage + CDN — never in the database).

3. API

POST /v1/posts                               → 201 {postId, createdAt}
     {"text": "Tour announced! Presale Friday 10:00", "mediaIds": ["m_81f…"]}
POST /v1/media                               → 201 {mediaId, uploadUrl}   (pre-signed PUT to object store)

GET  /v1/feed?limit=20&cursor=eyJ0cyI6…      → 200 {items: [Post…], nextCursor}
GET  /v1/users/{id}/posts?cursor=…           → 200 {items: [Post…], nextCursor}   (profile timeline)

PUT    /v1/users/{id}/followers/me           → 204   (follow)
DELETE /v1/users/{id}/followers/me           → 204   (unfollow)

• Cursor, not page number: cursor encodes (score or timestamp, postId) of the last item seen. New posts arriving at the top do not shift the pages (no duplicates, no gaps) and the query is an index seek, not OFFSET 4000.
• Media is uploaded directly to object storage with a pre-signed URL; the post only references mediaId.

4. Data model

-- Posts: partitioned by author, newest first (wide-column, e.g. Cassandra)
CREATE TABLE posts_by_author (
    author_id  BIGINT,
    post_id    BIGINT,          -- Snowflake-style: time-ordered, unique
    text       TEXT,
    media      LIST<TEXT>,
    visibility TEXT,
    PRIMARY KEY ((author_id), post_id)
) WITH CLUSTERING ORDER BY (post_id DESC);

-- Social graph, stored in both directions
CREATE TABLE followers (user_id BIGINT, follower_id BIGINT, PRIMARY KEY ((user_id), follower_id));
CREATE TABLE following (user_id BIGINT, followee_id BIGINT, PRIMARY KEY ((user_id), followee_id));

# Timeline cache (Redis sorted set, score = post timestamp), trimmed to 800 entries
feed:{user_id}   →  { (1767261600123, 918273645), (1767261598001, 918273511), ... }

# Post cache for hydration (hash per post) and counters
post:{post_id}   →  { author_id, text, media, created_at }
likes:{post_id}  →  1532

5. High-level design

flowchart LR
    u([App]) --> lb["Load balancer"]
    lb --> ps["Post service"]
    ps --> pdb[("Posts store<br/>by author")]
    ps --> k[["Kafka<br/>post-created"]]
    k --> fw["Fan-out workers"]
    fw --> g[("Social graph<br/>followers")]
    fw --> tl[("Timeline cache<br/>feed:{user} → IDs")]
    lb --> fs["Feed service"]
    fs --> tl
    fs --> pc[("Post cache")]
    pc --> pdb
    fs --> rk["Ranking"]
    u -. images .-> cdn["CDN"] --> obj[("Object store")]

Write path: the post service stores the post and publishes post-created; it returns 201 immediately. Fan-out workers read the author's followers in pages and push the post ID into each active follower's timeline (ZADD + ZREMRANGEBYRANK to trim).

Read path:

sequenceDiagram
    autonumber
    participant A as App
    participant F as Feed service
    participant T as Timeline cache
    participant C as Celebrity posts
    participant P as Post cache
    participant R as Ranking
    A->>F: GET /v1/feed?limit=20
    F->>T: ZREVRANGE feed:42 0 199
    T-->>F: 200 post IDs (pushed)
    F->>C: latest posts of followed celebrities (pull)
    C-->>F: 30 post IDs
    F->>F: merge, drop deleted / blocked
    F->>P: multi-get 230 posts (batched)
    P-->>F: post objects
    F->>R: score candidates
    R-->>F: top 20
    F-->>A: 20 posts + nextCursor

6. Deep dives

6.1 Fan-out on write, on read, or both

How much work does a threshold save? With a heavy-tailed distribution of followers, a tiny share of authors produces most of the fan-out:

# Who pays for push fan-out? Synthetic heavy-tailed follower counts (illustrative)
import random

random.seed(42)
authors = 200_000
followers = [min(50_000_000, int(60 / (1 - random.random()) ** 1.25)) for _ in range(authors)]
total = sum(followers)                  # timeline inserts if every author posts once
followers.sort()

print(f"median followers: {followers[authors // 2]:,}  ·  max: {followers[-1]:,}\n")
print("| threshold | celebrities | celebrities_pct | writes_avoided_pct |\n|---:|---:|---:|---:|")
for threshold in (10_000, 100_000, 1_000_000):
    celeb = [f for f in followers if f >= threshold]
    print(f"| {threshold:,} | {len(celeb):,} | {len(celeb) / authors:.2%} | {sum(celeb) / total:.1%} |")

Pulling the posts of a fraction of a percent of the authors removes most of the fan-out writes. The threshold is a tuning knob: lower it and reads get a bit more expensive (more celebrities to merge per user), raise it and celebrity posts take longer to fan out.

6.2 The timeline cache and hydration

• Store IDs, not posts. A post can be edited, deleted or have its counters change; copying it into 200 timelines would make every change a fan-out. A timeline is a list of (score, post_id); the content is hydrated at read time with one batched multi-get from the post cache.
• Trim each timeline to a few hundred entries (ZREMRANGEBYRANK). Nobody scrolls back 5 000 posts through the home feed; older history comes from the pull path.
• Active users only. Fan-out workers skip users who have not opened the app in, say, 30 days. When such a user returns, the feed service rebuilds the timeline with a pull (slow once, then cached) — a classic cache-miss path (caching patterns).
• Filter on read. Deleted posts, blocked authors and posts that became private are removed during hydration (the post cache says deleted), so the timeline itself never needs to be cleaned synchronously.
• Partition the timeline cache by user_id (consistent hashing); a user's feed lives on one shard, so the read is a single round trip.

6.3 Ranking

A chronological feed is "sort by time". A ranked feed is a small pipeline that must fit in the latency budget:

flowchart LR
    cand["Candidates<br/>~500 IDs<br/>(pushed + pulled)"] --> feat["Features<br/>author affinity, age,<br/>engagement, media type"]
    feat --> score["Scoring model<br/>(lightweight)"]
    score --> rerank["Re-rank<br/>diversity, dedupe,<br/>business rules"]
    rerank --> page["Top 20"]

• Candidate generation is the timeline (push + pull); features come from precomputed stores (affinity between user and author computed offline), never from expensive live queries.
• Precompute when possible: rank the first page when the user is likely to open the app (or right after fan-out) and cache it for a few minutes.
• Keep a fallback: if the ranking service is slow, return the chronological order. A feed that is less clever but on time is better than a spinner (reliability patterns).

6.4 Pagination and freshness

• Cursor pagination: the cursor is (score, post_id) of the last item; the next page is ZREVRANGEBYSCORE feed:42 (score -inf LIMIT 0 20. New posts arriving at the top do not shift the next page, unlike OFFSET.
• New posts while scrolling are shown with a "new posts" pill that reloads from the top (a separate request with since=), not inserted under the user's finger.
• Read-your-writes: after posting, the author must see the post in their own feed immediately even if fan-out takes seconds — insert it into the author's own timeline synchronously, or merge the author's latest posts on read (consistency models).

7. Bottlenecks, failure modes and trade-offs

The trade-offs to say out loud: write amplification vs. read latency (push pays at write time so reads are cheap), freshness vs. cost (asynchronous fan-out means seconds of delay), and relevance vs. latency (ranking must fall back to chronological when it is slow).

8. Try it in the playground

The news-feed scenario contains the post service, a fan-out path through a queue and workers, a timeline cache and the feed service.

Open the full playground

9. Follow-up questions an interviewer asks

10. Self-assessment rubric

Related: Messaging, Caching patterns, Partitioning, Hands-on 7 (asynchronous processing with Kafka), Hands-on 5 (caching).

How it connects

Which classes does this case test? Each branch is a class whose ideas the design relies on (blue), with the palette components where they show up (green):

flowchart LR
    CASE(["News feed"]) --> EST["Estimation<br/>fan-out write amplification"]:::tech
    CASE --> MSG["Messaging<br/>post events, fan-out workers"]:::tech
    CASE --> CAC["Caching<br/>timeline cache, hydration"]:::tech
    CASE --> DB["Databases<br/>partition posts and timelines"]:::tech
    CASE --> DIS["Distributed<br/>read-your-writes for the author"]:::tech
    CASE --> REL["Reliability<br/>fallback to chronological order"]:::tech
    MSG -.-> k1["kafka · worker"]:::comp
    CAC -.-> k2["cache"]:::comp
    DB -.-> k3["nosql · objstore · cdn"]:::comp
    REL -.-> k4["app · lb"]:::comp
    classDef comp fill:#1e9e5a22,stroke:#1e9e5a
    classDef tech fill:#2f6fed22,stroke:#2f6fed

Classes it tests: Messaging — Kafka · Caching — patterns · Databases — partitioning · Distributed — consistency · Reliability — patterns · Hands-on 5 and Hands-on 7. Palette components: lb · app · cache · nosql · kafka · worker · objstore · cdn. Scenarios: news-feed.

References

• XU, A. System Design Interview — An Insider's Guide, vol. 1, ch. 11 "Design a News Feed System"¹.
• KLEPPMANN, M.; RICCOMINI, C. Designing Data-Intensive Applications, 2nd ed. — the home-timeline example (fan-out on write vs. read) in the chapter on describing load².
• KRIKORIAN, R. Timelines at Scale — how Twitter combined push and pull³.
• BRONSON, N. et al. TAO: Facebook's Distributed Data Store for the Social Graph⁴.