WhatsApp

Assume 3B registered users, 1.2B daily active users, 1.0B users concurrently online at peak, 70B user-authored messages per day, and an average recipient fanout of 1.7 after one-on-one and group traffic are blended.

Assume each encrypted text message envelope plus metadata is 1 KB before replication, 5 percent of messages include media, average encrypted media object size is 2 MB, each message produces two receipt events on average, and peak traffic is 4x the daily average. A connection gateway can safely hold 500K concurrent long-lived connections with headroom.

• Sends: 70B messages per day divided by 86,400 seconds is about 810K sends per second on average. With a 4x peak multiplier, plan for about 3.2M sends per second.
• Recipient deliveries: 70B messages times 1.7 average recipients is 119B recipient deliveries per day. 119B divided by 86,400 seconds is about 1.38M deliveries per second on average and about 5.5M at peak.
• Gateways: 1.0B concurrent sockets divided by 500K sockets per gateway is 2,000 gateways. Adding 30 percent headroom gives about 2,600 active gateways before disaster recovery spare capacity.
• Receipts: two receipt events per message gives 140B receipt events per day. 140B divided by 86,400 seconds is about 1.62M events per second on average and about 6.5M at peak.
• Message storage: 70B encrypted envelopes times 1 KB is 70 TB raw per day. With 3x replication and index overhead, reserve roughly 250 TB per day for the hot retention tier before compaction and deletion.
• Media: 5 percent of 70B messages is 3.5B media messages per day. 3.5B times 2 MB is about 7 PB raw media ingest per day, so media must use object storage and CDN-style delivery rather than the message database.
• Session registry writes: every connected device refreshes a lease periodically. If 1.0B sessions refresh every 30 seconds, that is about 33M lease refreshes per second, so refreshes must be batched, sharded, and handled by a specialized presence/session tier.

• messages.conversation_id, server_sequence is the primary read path for ordered conversation history.
• messages.sender_device_id, client_message_id must be unique enough to implement idempotent send retries.
• user_mailboxes.user_id, device_id, mailbox_sequence supports reconnect sync without scanning conversations.
• conversation_members.conversation_id, user_id supports membership checks and group fanout.
• connection_sessions.user_id, device_id maps an online recipient to the current gateway and connection id.

A conversation has many members and many ordered messages. Each accepted message creates one or more mailbox records, one per recipient device or per recipient logical inbox depending on the multi-device model. Connection sessions are short-lived leases, not the source of truth for message durability.

Receipts can be stored as compact per-user cursors such as delivered_up_to and read_up_to per conversation instead of one row per message when semantics allow. This reduces receipt storage from per-message state to per-conversation cursors.

Use a log-structured or wide-column store such as Cassandra, HBase, Bigtable, or DynamoDB for conversation logs and mailboxes. Partition conversation logs by conversation id plus sequence buckets to avoid oversized partitions in very large groups. Partition mailboxes by user id and device id because reconnect sync is recipient-oriented.

Use Redis, Aerospike, or a custom sharded in-memory store for the session registry because connection leases are extremely hot and short-lived. Use object storage for encrypted media and a separate streaming system such as Kafka, Pulsar, or Kinesis for fanout, receipts, and analytics.

A WhatsApp-like system is built around long-lived connections, not polling. Each online device has a socket to one gateway. The gateway periodically refreshes a lease in a session registry containing user id, device id, gateway id, connection id, region, and expiration time.

Delivery workers should not broadcast to all gateways. They query the registry for a recipient device and send to the owning gateway. Because registry entries can be stale, gateway delivery must be conditional: if the connection id no longer exists, the worker leaves the mailbox entry for later sync or retries after a fresh lookup.

Heartbeats are the scale hazard. Refreshing 1B sessions every few seconds would overload any ordinary cache. Use longer leases, gateway-level batching, sharded registries, delta heartbeats, and local gateway state. Presence freshness can be approximate; routing correctness comes from durable mailboxes and client acknowledgements.

Exactly-once delivery across mobile networks, retries, queues, and reconnects is not realistic. The server should provide at-least-once delivery and make every stage idempotent. The client sends a stable client message id. The message service stores a mapping from sender device plus client message id to server message id and sequence. If the client retries, it gets the same result.

Each recipient has a mailbox entry referencing the message id and conversation sequence. Delivery workers may retry the same mailbox item many times. Gateways may push duplicates after reconnect races. Clients deduplicate by message id and can acknowledge up to a mailbox cursor.

The sender can see sent after durable enqueue, delivered after a recipient device acknowledges receipt, and read after the recipient opens the conversation or advances a read cursor. These states are monotonic but eventually consistent.

Users expect a conversation to feel ordered even when messages are sent from multiple devices and regions. A common design assigns a monotonically increasing server sequence per conversation at acceptance time. The client displays messages by server sequence, with client timestamp only as secondary metadata.

The challenge is large groups and hot conversations. A single per-conversation sequencer can become hot. For ordinary chats, a leader shard per conversation is simple. For very large groups, use sequence allocation in ranges, conversation partitioning, or accept looser ordering with deterministic merge rules. The interview answer should explicitly state that global ordering across all conversations is unnecessary.

Retries must preserve sequence. If the same client message id is retried, the server returns the original server sequence. If failover occurs before acknowledgement, recovery must decide whether the sequence was committed by checking the idempotency record and message log.

One-on-one messaging creates one recipient delivery per send. Group messaging can create hundreds or thousands. Small and medium groups can use fanout-on-write: expand the recipient set at send time and enqueue one mailbox item per recipient. This gives fast offline sync because each user's mailbox already contains the message.

Very large communities may need hybrid fanout. Store one group message in the conversation log, enqueue for online active members first, and let inactive members pull from the group log on sync. This reduces write amplification but makes read and cursor logic more complex.

Membership version matters. A user who joins after a message should not receive old ciphertext unless history sharing is enabled, and a user who leaves should not receive future messages. Fanout tasks should include the membership version and clients should rotate group encryption keys when membership changes.

Offline support is a first-class requirement. When a device reconnects, it should not ask every conversation for missing messages. It should read a recipient mailbox or device sync log using a single cursor. The response contains encrypted messages, receipt updates, membership changes, and tombstones in a bounded page.

Multi-device increases complexity because a user may have phone, web, desktop, and companion devices with independent encryption sessions and cursors. The system can either create mailbox entries per device or maintain a logical user inbox plus device-specific delivery state. Per-device mailboxes are more expensive but make sync correctness clearer.

Retention policies matter. If a device is offline longer than the mailbox retention window, the server should return a cursor expired response and require a fuller resync from conversation logs or a client backup, depending on product policy and encryption constraints.

End-to-end encryption changes what servers can do. Servers cannot inspect message text for search, spam decisions, previews, or smart replies unless clients provide separate privacy-preserving signals. The server stores ciphertext, sender and recipient identifiers, timestamps, envelope size, and delivery state.

The Signal protocol family uses device identity keys, prekeys, sessions, and ratcheting so each device can decrypt only messages intended for it. For groups, clients can use sender keys or per-recipient key wrapping to avoid encrypting the full payload separately for every member.

Media should never flow through the message database. The client encrypts the media, uploads it to object storage, and sends a message containing a media id, content hash, size, and encrypted media key material. Recipients fetch the object and decrypt locally.

Alternative one is client polling against a REST API. It is simpler, but it wastes battery and bandwidth and cannot meet real-time latency at global scale.

Alternative two is a pure pub-sub system with no durable mailbox. It works for online-only chat, but offline users lose messages and reconnect sync becomes unreliable.

Alternative three is fanout-on-read for all groups. It reduces write amplification, but reconnect sync becomes expensive because each user must scan group logs and compute visibility on demand.

Alternative four is server-visible plaintext to support search, spam filtering, and previews. It simplifies product features but violates the privacy requirement and is not acceptable for a WhatsApp-like design.

Do not use this design for low-volume enterprise chat where compliance requires server-side retention, eDiscovery, and plaintext indexing; a Slack-like system may be more appropriate. Also do not use it for public broadcast feeds where follower fanout, ranking, and timeline storage dominate rather than private encrypted delivery.