Designing Twitter

The problem

Design Twitter/X: users post tweets, follow others, and see a home timeline of recent tweets from everyone they follow. The defining challenge is generating that timeline fast for hundreds of millions of users — the canonical fan-out problem.

Step 1 — Requirements

Functional: post a tweet; follow/unfollow; view a home timeline (tweets from followees, reverse-chronological or ranked); view a user’s profile timeline; likes/ retweets.

Non-functional: extremely read-heavy (timeline reads ≫ tweets), low-latency timeline (must feel instant), high availability, scale (100s of millions of users, billions of tweets), eventual consistency is fine (a tweet appearing a second late is OK).

Step 2 — Estimate

• Say 200M DAU, each posts ~2 tweets/day → ~400M tweets/day → ~5k writes/sec.
• Timeline reads dwarf that — users refresh constantly → ~hundreds of k/sec.
• A tweet is small (~300 B) but the fan-out is the cost: one tweet by someone with M followers must reach M timelines.

The read:write asymmetry says: precompute timelines so reads are cheap.

Step 3 — The core decision: fan-out on write vs read

How is a home timeline built?

• Fan-out on write (push) — when a user tweets, push the tweet id into each follower’s precomputed timeline (a per-user list in Redis). Reads are O(1) (just read your list). But a tweet costs O(followers) writes, and celebrities with millions of followers cause a write storm (“fan-out explosion”).
• Fan-out on read (pull) — store tweets per author; at read time, gather the latest tweets from everyone you follow and merge. Writes are cheap (O(1)), but reads are expensive (fetch + merge from hundreds of followees) — bad for the read-heavy hot path.

Step 4 — The hybrid (the senior answer)

Use both, split by case:

• Normal users → fan-out on write: push their tweets into followers’ timelines, so the common read is O(1).
• Celebrities (followers above a threshold) → fan-out on read: do not push; instead, at read time, merge in their recent tweets from a small per-celebrity store.

So a user’s timeline = their precomputed list (from normal followees) merged with the latest tweets of the few celebrities they follow. This caps the write storm while keeping reads fast. This hybrid is the key insight interviewers want.

Step 5 — Data model and storage

Tweets:     tweet_id (Snowflake), author_id, text, media_url, created_at
Followers:  user_id → [follower_ids]   (the social graph; sharded)
Timeline:   user_id → [tweet_ids]      (precomputed, in Redis, capped to ~800)

• Tweets in a sharded store (by tweet id / author); media in blob store + CDN.
• Timelines as capped lists in a cache (Redis) for O(1) reads.
• Social graph sharded by user id; “who follows X” and “who X follows” both queryable.

Step 6 — Serving and ranking

• Read timeline → read the precomputed list, merge celebrity tweets, hydrate tweet content from cache/store, return.
• Ranking — chronological is simplest; a ranked feed scores by engagement/recency/ affinity (like the news-feed lesson).

Trade-offs to raise

• Fan-out write (fast reads, write storm) vs read (cheap writes, slow reads) vs hybrid (best of both, more complex).
• Eventual consistency — timelines update within seconds; fine.
• Timeline cap — store only the latest ~N; older paginates from source.

The interview cue

“Read-heavy, so precompute timelines via fan-out-on-write for normal users (O(1) reads), but fan-out-on-read for celebrities to avoid the write storm — a hybrid, merged at read time. Tweets sharded + media on CDN, timelines in Redis, social graph sharded.” The push/pull hybrid + celebrity handling is exactly what this problem tests; implementation next.