Twitter / X (Social Media Timeline)

Metadata

Building Blocks Used

This case study builds upon the following foundational concepts:

• Caching Strategies (Redis home timelines)
• Message Queues (Fan-out worker pipelines)
• Database Sharding (Distributing massive user data)

1. System Requirements

Functional Requirements

• Users can post tweets (text and media).
• Users can follow other users.
• The Core Feature: Users have a Home Timeline displaying a chronological feed of tweets from people they follow.
• (Out of Scope for MVP): Algorithmic ranking, Search, Trending Topics.

Non-Functional Requirements

• High Availability: The timeline must always load.
• Fast Timeline Loading: The feed must generate in under 200ms.
• Massive Scale: Needs to handle viral events (e.g., the Super Bowl) where traffic spikes 100x.

2. Back-of-the-Envelope Estimation

• DAU (Daily Active Users): 250 Million.
• Average Tweets / User / Day: 2 (Total: 500 Million tweets/day).
• Average Feed Reads / User / Day: 20 (Total: 5 Billion reads/day).

QPS (Queries Per Second)

• Write QPS (Tweets): 500,000,000 / 86400 ≈ 6,000 Writes / second.
• Read QPS (Timelines): 5,000,000,000 / 86400 ≈ 60,000 Reads / second.
• Takeaway: A 10:1 Read-to-Write ratio. This is a read-heavy system, meaning we must pre-compute and cache data heavily to keep read latency low.

3. The Core Problem: Generating the Timeline

If Alice opens her app, she wants to see tweets from the 500 people she follows.

Approach 1: The SQL JOIN (Pull Model)

SELECT t.* FROM tweets t
JOIN follows f ON t.user_id = f.followee_id
WHERE f.follower_id = 'Alice'
ORDER BY t.created_at DESC LIMIT 20;

If we do this on the fly when Alice opens the app, we are asking a relational database to scan the tweets of 500 people, merge them, and sort them. At 60,000 Timeline Reads per second, this SQL query will bring the database to its knees instantly. We cannot compute timelines on the fly.

Approach 2: Fan-out on Write (Push Model)

Instead of computing the timeline when Alice reads, we compute it when her friends write.

Every user has an in-memory "Inbox" (a Redis Cache List). When Bob tweets, the system looks up all 100 of Bob's followers. It then takes Bob's tweet and pushes a copy of it into the Redis Inbox of all 100 followers.

When Alice opens her app, her timeline is already perfectly assembled in Redis. Generating her feed takes O(1) time. Reads are now blazing fast.

4. High-Level Architecture (Fan-out on Write)

graph TD
    Bob((Bob)) -->|POST /tweet| API[API Gateway]
    API --> TweetSvc[Tweet Service]
    
    TweetSvc -->|1. Save| DB[(PostgreSQL - Tweets)]
    TweetSvc -->|2. Publish| Kafka[(Kafka)]
    
    Kafka --> FanoutWorker[Fan-out Worker]
    
    FanoutWorker -->|3. Get Followers| GraphDB[(Graph DB - Follows)]
    FanoutWorker -->|4. Push to Inboxes| Redis[(Redis - User Timelines)]
    
    Alice((Alice)) -->|GET /timeline| API
    API --> TimelineSvc[Timeline Service]
    TimelineSvc -->|5. Instant Fetch| Redis

5. Deep Dive: The "Justin Bieber" Problem

The Fan-out on Write architecture is brilliant for normal users. But what happens when a celebrity with 100 Million followers (like Justin Bieber) tweets?

If the Fan-out Worker tries to push that tweet into 100 Million Redis Inboxes, it will take minutes (or hours), cause massive CPU spikes, and delay everyone else's tweets. This is known as the Celebrity Bottleneck or the Thundering Herd.

The Solution: Hybrid Fan-out

We classify users into two groups:

1. Normal Users (< 10,000 followers)
2. Celebrities (> 10,000 followers)

We use a Hybrid approach combining both Push and Pull:

• For Normal Users: Continue using Fan-out on Write (Push).
• For Celebrities: We use Fan-out on Read (Pull). When Bieber tweets, we do not push it to 100M inboxes. We simply save it to his personal database row.
• The Merge: When Alice opens her app, her Timeline Service grabs her pre-assembled Redis Inbox (which contains all her normal friends). Then, the service checks: "Does Alice follow any celebrities?" If yes, it fetches Bieber's latest tweets from a dedicated Celebrity Cache and merges them into Alice's feed on the fly.

Because fetching one tweet from a Celebrity Cache is fast, this merge operation takes milliseconds, completely avoiding the 100M write bottleneck.

6. Deep Dive: Data Modeling & Storage

1. The Social Graph (Followers)

Tracking who follows whom is a classic graph problem. While a relational DB can handle this with a junction table (follower_id, followee_id), at Twitter's scale, a dedicated Graph Database (like Neo4j) or a highly optimized NoSQL datastore (like Cassandra or FlockDB) is used to answer "Who does Bob follow?" in milliseconds.

2. Tweet Storage

Tweets are immutable (historically) and massive in volume. We can use a sharded SQL database (PostgreSQL) or a NoSQL database (DynamoDB). To handle the scale, we must implement Database Sharding.

• Shard by user_id: All of Bob's tweets are on Shard 3. Good for fetching a user's profile, but bad for the hybrid merge (since celebrities might crash a single shard).
• Shard by tweet_id: Tweets are evenly distributed. Requires a UUID Generator (Snowflake) where the ID contains a timestamp so tweets remain roughly chronological on disk.

3. Media Storage

Images and videos are stored in an Object Store (Amazon S3). The database only stores the URL pointing to S3. To ensure fast global loading, all media is served through a CDN.

7. Memory Optimization in Redis

Storing the timelines of 250 million users in RAM (Redis) is incredibly expensive.

Optimizations:

1. Don't store the whole tweet: The Redis list should only store [tweet_id, user_id], not the text payload. The client fetches the full text payload from a secondary cache or DB.
2. Limit the Inbox Size: Only keep the last 800 tweets in the Redis Inbox. If a user scrolls past 800 (very rare), fall back to the slow database query.
3. Active Users Only: Only keep timelines in Redis for users who have logged in within the last 14 days. If a dormant user logs in, they experience a slightly slower load time while their Redis Inbox is rebuilt from the DB.

8. Summary of Building Blocks Used

Website Metadata