"Design a real-time chat system" (WhatsApp / Messenger / Slack) is a common big-tech system design question. Its core challenges are three: ① real-time two-way communication, ② massive concurrent persistent connections, ③ reliable, ordered delivery (including offline). Beginners draw the database first, but this question's soul is "how connections are kept, and how a message gets from A to B with guaranteed delivery."
This post breaks it down layer by layer with interviewer follow-ups.
⏱️ Design through-line: chat isn't request-response — the server must proactively push to users. So the first problems are "how to keep a two-way connection" and "how does A's message know which server B is connected to."
Step 1 — Clarify Requirements
- Features: 1-on-1? group chat? Read receipts, presence, message history, file transfer, push notifications?
- Scale: DAU, concurrent connections (millions of persistent connections is the design crux), messages per second.
- Reliability: messages must not be lost, must be ordered, and deliverable offline.
- Latency: real-time (delivery in < a few hundred ms).
🎤 Interviewer follow-up: "How do 1-on-1 and group chat differ?" (Group needs fanout to N members; huge groups need special handling.) "Can messages be lost?" (No — persistence + delivery acks + offline re-delivery.)
Step 2 — Why WebSocket (the connection) ⭐
The server must proactively push to users, which classic HTTP request-response can't do. Options:
| Method | How | Problem |
|---|---|---|
| Short polling | client asks "any new messages?" every few seconds | high latency, wasted requests |
| Long polling | request held until a message arrives | better, but reconnects each time |
| WebSocket | one persistent two-way connection, server can push anytime | the choice; but managing massive connections |
Conclusion: use WebSocket for a persistent connection; the server pushes down this connection whenever there's a message.
🎤 Interviewer follow-up: "Why not HTTP polling?" (Poor real-time, wasted requests; one WebSocket pushes two-way most efficiently.) "How do you handle millions of connections?" (Multiple connection servers (WebSocket gateways) scaled horizontally, each holding a slice of connections.)
Step 3 — The Core Problem: How A's Message Finds B ⭐
Users connect to different connection servers. A is on server-1, B on server-3 — when A messages B, how does the system know "B is on server-3" and route it there?
- Session registry: when a user connects, write "user → which connection server" into a shared registry (e.g. Redis).
- Message flow: A's message → connection server → Chat Service → ① write to the message DB (persist) ② look up the registry to find B on server-3 → ③ push the message to server-3 via an internal message bus (or direct RPC) → server-3 pushes it to B over B's connection.
- B offline: registry has no B → the message is already persisted, and a push notification (APNs/FCM) notifies B; on reconnect, B pulls unread from the DB.
🎤 Interviewer follow-up: "How do you know which server a user is on?" (A Redis session registry
user → server, written on connect.) "How do connection servers pass messages to each other?" (An internal message queue/bus, or service discovery + RPC.)
Step 4 — Message Storage and Sharding
- Data model: a message table (message_id, conversation_id, sender, content, timestamp, sequence number).
- Storage choice: messages are write-heavy, read by conversation in time order — a fit for a wide-column store (Cassandra / HBase) or a dedicated store, sharded by
conversation_id. - Shard key:
conversation_id, so one conversation's messages land on one shard, easing time-ordered history reads. - History reads: paginate by conversation + time range (recent messages are hot and cacheable).
🎤 Interviewer follow-up: "Why Cassandra over MySQL?" (Huge message volume, write-intensive, queried by conversation time range — wide-column fits naturally.) "What's the shard key?" (conversation_id, to land a single conversation on one shard.)
Step 5 — Ordering, Read Receipts, Reliable Delivery ⭐
- Ordering: use a per-conversation incrementing sequence ID to guarantee order; the client sorts by sequence and fills gaps. (A global timestamp is unreliable across machines with clock skew.)
- Delivery status: sent → delivered → read three states, updated by acks.
- Reliable delivery (at-least-once + idempotency): deliver at least once, possibly duplicated → the client dedups by message_id (idempotent). Offline messages persist in the DB and are pulled on reconnect.
🎤 Interviewer follow-up: "How do you guarantee ordering?" (A per-conversation incrementing sequence ID; don't rely on cross-machine timestamps.) "What about duplicate delivery?" (The client dedups by a unique message_id — almost always asked.) "How do read receipts work?" (B sends an ack on read, updating state and notifying A.)
Step 6 — Group Fanout and Presence
- Group fanout: A sends to a group → Chat Service looks up members → push to each online member, route offline members to push/offline queues. Small groups fanout directly; very large groups (e.g. tens of thousands) need special strategies (e.g. fanout-on-read instead of real-time fanout).
- Presence: clients send periodic heartbeats; a presence service (Redis) records the last heartbeat and treats timeouts as offline. State changes are pushed to friends.
🎤 Interviewer follow-up: "How do you fanout a 10k-member group?" (Avoid pushing to everyone at once — use fanout-on-read or limit real-time push scope.) "How is presence maintained?" (Heartbeat + Redis TTL, offline on timeout.)
Architecture at a Glance
User A ──WS── ConnServer1 ┐ ┌ ConnServer3 ──WS── User B
├─► Chat Service ─┤
(Session registry: user→server, Redis) │
├─► Message DB (Cassandra, sharded by conversation)
├─► Internal message bus (cross-server delivery)
└─► Push service (APNs/FCM, for offline users)
One Table to Close
| Module | What it solves | Interview gimme |
|---|---|---|
| WebSocket | real-time two-way push | why not polling; scale connection servers |
| Session registry | A's message finds B | user→server in Redis |
| Message storage | huge volume, read by conversation | Cassandra, shard by conversation |
| Ordering/receipts | ordered, reliable | per-conversation sequence ID; dedup by message_id |
| Group/Presence | fanout, online status | large-group strategy; heartbeat + TTL |
The Answer Framework Cheat
When asked to "design a chat system," follow this line:
- Clarify: 1-on-1/group, scale, reliability needs
- Connection: why WebSocket (server pushes) + horizontally scaled connection servers
- Routing: the core problem — a session registry (user→server) so A's message finds B
- Storage: message DB (Cassandra, sharded by conversation)
- Reliability: per-conversation sequence ID for ordering + dedup by message_id + read receipts + offline push
- Advanced: group fanout (large-group strategy), presence (heartbeat)
Remember one line: a chat system's soul isn't the database — it's "how to keep massive two-way connections" and "how a message gets from A to B reliably and in order (including offline)." Nail connection routing and delivery guarantees and you've won the question.