Twitter

Assume 300M daily active users, each opening the home timeline 10 times per day on average, with each first page returning 50 tweet IDs and hydrated tweet objects. Assume 150M new tweets per day, a 10x peak traffic multiplier, an average tweet metadata record of 1 KB before replication, and 20 percent of tweets containing media.

For fanout math, assume an ordinary author has 200 followers on average. The follower distribution is heavy-tailed, so celebrity accounts are not pushed to all followers. Timeline caches keep about 1,000 recent tweet IDs for recently active users, not for every registered user forever.

• Home timeline reads: 300M DAU times 10 opens per day equals 3B timeline reads per day. 3B divided by 86,400 seconds is about 34,722 reads per second, rounded to 35K average QPS.
• Peak reads: applying a 10x multiplier for traffic bursts gives about 350K home timeline reads per second.
• Tweet writes: 150M tweets per day divided by 86,400 seconds is about 1,736 writes per second. With 10x peak, plan for about 17K tweet creates per second.
• Naive fanout: if every tweet were pushed to 200 followers, the system would create 30B home timeline entries per day. 30B divided by 86,400 seconds is about 347K timeline inserts per second on average, before peaks.
• Tweet storage: 150M tweets times 1 KB is 150 GB raw per day. With 3x replication, indexes, counters, and compaction headroom, plan for several hundred GB per day.
• Timeline cache: 1,000 tweet IDs per user at 16 bytes each is 16 KB raw. With scores, timestamps, Redis or cache overhead, and fragmentation, budget roughly 32 KB. Caching 60M recently active users requires about 1.9 TB.
• Media: if 20 percent of tweets have media, 30M media tweets per day at 1 MB average is 30 TB raw per day before transcoding variants, thumbnails, replication, and CDN cache copies.

• tweets.tweet_id is the primary key for hydration and delete checks.
• tweets.author_id, created_at supports profile timelines and author history.
• follow_edges.follower_id, followee_id supports home timeline pulls from accounts a user follows.
• follow_edges.followee_id, follower_id supports fanout workers finding followers of an author.
• home_timeline_entries.user_id, inserted_at supports recent home timeline reads and bounded trimming.
• Search terms, hashtags, and entities belong in a dedicated inverted index, not as secondary indexes on the tweet store.

Tweets belong to authors. Follow edges connect followers to followees and drive both push fanout and read-time pull. Home timeline entries reference tweets but are not source of truth; they can be deleted, trimmed, or rebuilt. Media records reference object storage and are linked from tweets through media_ids.

Use a distributed wide-column or key-value store such as Manhattan, Cassandra, DynamoDB, Bigtable, or FoundationDB for tweets and timeline entries. Partition tweets by tweet_id or author-time buckets, and partition home timelines by user_id with reverse-time ordering. Use a graph-optimized store or carefully sharded edge tables for follow relationships.

Use Elasticsearch, OpenSearch, Solr, or a custom inverted index for search. Use object storage such as S3, GCS, or Azure Blob for media and serve through CDN. Keep counters, analytics, and trend aggregates in streaming and OLAP systems rather than updating canonical tweet rows synchronously.

Fanout-on-write pushes a new tweet into follower home timelines when the tweet is created. It makes reads very fast because opening the home timeline is mostly a cache lookup plus hydration. It works well for ordinary accounts because follower counts are modest and writes are less frequent than timeline reads.

The downside is write amplification. An account with 50M followers would create 50M timeline inserts for one tweet. That can overwhelm queues, graph reads, cache shards, and storage, especially during live events when many high-follower accounts post together.

Fanout-on-read does the opposite. It stores the tweet once and, when a user opens the app, reads recent tweets from all followed accounts. It avoids write storms and handles celebrity accounts naturally, but read latency grows with the number of follows and requires many author timeline reads. Pure pull is hard for users who follow thousands of accounts.

The strong answer is hybrid. Push ordinary authors to active followers. Pull high-follower authors during reads. Optionally push celebrity tweets only to the most active or notification-enabled followers. This matches the skewed graph and keeps both writes and reads bounded.

Celebrity classification should not be a single static number. Use follower count, active follower count, posting rate, engagement, queue pressure, regional traffic, and recent virality. An account with 5M followers but few active followers may still be pushable during off-peak periods, while an account with 500K very active followers can be expensive.

A practical policy has bands. Small and medium authors use full push. Large authors use capped push to active followers and pull for everyone else. Mega celebrities use pull-only for home timelines, with optional notifications handled by a separate system. The Fanout Service should make this decision at event time and include enough metadata for timeline reads to pull the right accounts.

Timeline reads maintain a list of followed pull-author IDs. For each request, the Timeline Service fetches recent tweets from those authors, merges them with cached pushed IDs, deduplicates by tweet ID, and ranks the combined candidate set. This avoids fanout storms while preserving freshness for famous accounts.

The home timeline cache should store tweet IDs, timestamps, author IDs, source, and small rank hints, not full tweet objects. Full objects change when counters, labels, media processing, or deletes change. Storing IDs keeps cache entries compact and allows batch hydration from the Tweet Store and feature services.

Each user's cached list should be bounded, commonly hundreds to a few thousand IDs. Fanout workers append new IDs and trim old entries. Inactive users can have their cache expired and rebuilt lazily from recent follows when they return. This saves memory because caching every registered user's timeline forever is not economical.

Cache correctness requires final visibility checks. A cached ID may refer to a deleted tweet, a newly blocked author, a protected account, or a safety-limited tweet. The Timeline Service filters after hydration and can remove invalid entries asynchronously. For hard takedowns, maintain a small deny-list cache checked before returning results.

The graph has two access patterns with different hot spots. Home timeline reads need the list of accounts a user follows. Fanout workers need the followers of an author. Store both directions or maintain two materialized edge indexes because reversing one index at runtime is too expensive.

Shard by user ID for following lists and by followee ID for follower lists. Celebrity follower lists become huge partitions, so split them into pages or shards by follower ID ranges. Fanout workers can process pages independently and checkpoint progress. Follow and unfollow operations update both directions with idempotent writes.

Blocks, mutes, protected follows, and privacy rules are graph-adjacent state. Some can be stored as separate edge types, but timeline serving should receive a compact visibility decision or filter set so it does not perform many graph lookups per tweet.

A home timeline request should not rank the entire social graph. It should rank a bounded candidate set, such as a few hundred pushed IDs plus recent pull candidates from celebrity accounts. Candidate generation must be fast and predictable before the ranking model runs.

Hydration is often the hidden latency bottleneck. The service needs tweet text, author info, media metadata, counters, viewer relationship state, and safety labels. Use batch APIs, cache hot tweet and author objects, parallelize independent fetches, and enforce deadlines. Return a good enough timeline rather than waiting for every feature.

Ranking should have tiered fallbacks. If the model service is healthy, use personalized ranking. If it is slow, use cached rank hints and freshness. If feature fetches fail, use reverse chronological order. The user should still see a timeline.

Search and trends are not served from the home timeline cache. Tweet events flow into a search index for keyword, hashtag, author, and entity queries. Engagement and tweet streams feed trend detectors that compare recent volume against baselines by region and language.

Search freshness matters, but it should not block tweet creation. The create response can succeed before the tweet is searchable. Indexing systems should support retries, duplicate events, deletes, and safety labels so removed content disappears from search quickly.

Media follows a separate lifecycle. Uploads are scanned, transcoded, and stored in object storage. Timelines return metadata and CDN URLs. This prevents large media payloads from bloating tweet records, timeline caches, queues, and replication traffic.

Alternative one is pure fanout-on-write. It gives very fast reads and a simple home timeline cache, but it fails for celebrity accounts and produces enormous write amplification.

Alternative two is pure fanout-on-read. It minimizes write work and handles celebrities naturally, but home timeline latency grows with follow count and it becomes expensive for highly connected readers.

Alternative three is a fully ranked recommendation feed built from global candidate retrieval rather than only follows. This can improve engagement, but it is more complex and should be layered after the follow-based timeline is correct.

Do not use this full architecture for a small community feed, an internal activity log, or a product where users follow only a few accounts. A simpler pull model with relational storage and cache is easier to operate until follower graph skew and timeline read volume justify fanout infrastructure.