Pushing real-time updates

A chat message, a new notification, a driver moving on a map. The user expects to see it now, without pressing refresh. The web was built for the opposite: the client asks, the server answers.

Crux: the client cannot be pushed to until it is connected, so a real-time design has two halves: how the update reaches the user's device, and how it reaches the right server first.

The problem

Split every real-time feature into two hops:

• Edge hop: server to client. Which protocol keeps the update flowing to the device?
• Inside hop: the event happens on one server, but the user's connection is on another. How does the update find the right machine?

Candidates often spend all their time on the first hop and forget the second. At scale, the second one is the harder one.

First ask how fresh "real time" must be. Five seconds is a different problem from 200 milliseconds. Ask how many updates per second a user receives, and whether the user must send data too, or only receive.

Options for the edge hop

Simple polling

The client asks every few seconds: "anything new?"

• What: plain HTTP requests on a timer.
• Use when: a delay of a few seconds is fine, or updates are rare (a job status page).
• Know: no new infrastructure, works everywhere, easy to scale, because every request is stateless. The cost is wasted requests (most answers are "no") and a delay of up to one polling interval.
• Careful: thousands of clients polling every second is a self-inflicted load test. Add jitter so they do not all fire together.

Long polling

The client asks, and the server holds the request open until it has something, or a timeout hits. Then the client asks again straight away.

• Use when: you want near-instant updates but cannot rely on anything beyond plain HTTP.
• Know: each client costs one open request. Proxies and load balancers may cut idle connections, so the timeout must be shorter than theirs.

Server-sent events (SSE)

One long-lived HTTP response, and the server writes events into it. One direction only, server to client.

• Use when: the client mostly listens: notifications, live scores, progress bars, streaming text.
• Know: the browser reconnects by itself and can tell the server the last event ID it saw, so you can resume. It is plain HTTP, so it works with existing proxies, auth and tooling.
• Careful: browsers limit the number of open connections per site on older HTTP versions. Check this if one page opens many streams.

WebSockets

A connection that starts as HTTP and is then upgraded to a two-way channel. Either side can send at any time.

• Use when: the client sends often and needs low delay too: chat, multiplayer games, collaborative editing.
• Know: the connection is stateful. The server holds memory for each user, deploys drop connections, and load balancers must be set up for long-lived connections.
• Careful: you must write reconnection, heartbeats (to notice dead connections) and re-sync yourself. Many teams use a library for this.

WebRTC

A direct peer-to-peer channel between browsers, mainly for audio, video and some data. It still needs a server for set-up (signalling). Mention it for video calls. Do not reach for it for chat or notifications.

Options for the inside hop

Once users hold connections, an event on any server must reach the server that holds the target connection.

Pub/sub in the middle

Each connection server subscribes to the channels its users care about (a chat room, a user ID). A sender publishes to the channel, and every subscribed server forwards the message to its local connections.

• Use when: messages are small and the audience is spread over servers: chat, notifications, live comments.
• Know: Redis pub/sub is simple and fast, but it is fire-and-forget: a server that is down or disconnected misses the message. A log such as Kafka keeps messages so a consumer can catch up, at the cost of more machinery.
• Careful: a server should subscribe to a channel only while it has a local listener. Otherwise every server receives every message.

Stateful servers with consistent hashing

For heavier work, such as a shared document where every edit must be merged in order, route every user of one document to the same server. A consistent hash ring maps a document ID to a server, so that adding a server moves only a small share of documents.

• Use when: one in-memory state per entity (document, game room) must see all its events in order.
• Know: you now own state on the server. You need a way to move or rebuild it when a server dies or the ring changes.

A worked example: chat

Say two people are in a room. Ana is connected to server A, Ben to server B. We assume WebSockets (both send and receive).

• 1. Ana sends a message over her socket to server A.
• 2. Server A writes it to the database first, with a room-scoped sequence number. Now it is safe.
• 3. Server A publishes it to the room's pub/sub channel.
• 4. Server B subscribes to that channel because Ben is connected. It forwards the message down Ben's socket.
• 5. If Ben was offline, nothing is pushed. When he reconnects he sends the last sequence number he saw, and the server replays what he missed from the database.

Notice what carries the correctness: the database and the sequence number, not the socket. The socket is only a fast delivery path that may drop a message at any time.

Choosing

• Start with polling if a few seconds of delay is acceptable. Say so, and say what would make you upgrade.
• Server-sent events when data flows one way and needs to feel instant.
• WebSockets when the client also sends often.
• Pub/sub the moment a second connection server exists.
• Consistent hashing only when one entity needs one owner and ordered events.

When to bring this up, and when not to

• Signals in the prompt: "live", "instantly", "notify", "see it as it happens", "collaborate", "track on a map".
• Not needed when the product tolerates a refresh or a delay of a few seconds. Say "polling every 10 seconds is enough here" and move on. Spending ten minutes on sockets for a daily report is a misread.

Deep dives interviewers use

The connection drops

Networks drop, phones sleep, servers deploy. Treat disconnection as normal.

• The client reconnects with exponential backoff and jitter, otherwise a restart of one server makes thousands of clients return in the same second.
• On reconnect it sends a cursor (the last event ID or sequence number it saw). The server replays from there, or says "too far behind, reload the whole state".
• Use heartbeats both ways, so each side notices a half-dead connection within a bounded time.
• Messages are delivered at least once, so the client drops duplicates by ID.

Ordering across servers

Two servers may deliver two messages of one room in different orders. Give each room (or document) one sequencer: either the database assigns the sequence number inside the write, or all writes for the room go through the one server that owns it. Clients order by that number and hold back any message that arrives early, until the gap fills.

One update, millions of listeners

A post by an account with ten million followers cannot be pushed over ten million sockets at once.

• Push to the online few, let the rest pull. Only followers who are connected get a push. Everyone else reads the new post when they open the app.
• Fan out in stages: the publisher sends to a few relay layers, and each relay forwards to its own connection servers. No single machine does all the sends.
• Accept that delivery to a huge audience is eventual: seconds, not milliseconds. State this as a deliberate trade, and keep the strict path for small groups.

A hot channel

One live event can make a single pub/sub channel the busiest in the system. Split the channel into several partitions (by viewer ID hash), and have each connection server subscribe to only the partitions it needs. Sizing the number of partitions comes from the measured rate, not a guess.

Presence and "who is online"

A heartbeat that refreshes a key with a short expiry gives presence cheaply. Do not broadcast every flip to everyone: coalesce changes, and send them only to the people who are looking at that user.

Common mistakes

• Choosing WebSockets for a feature that SSE or polling would serve.
• Forgetting the inside hop: two users on different servers.
• Treating the live channel as the source of truth. Store first, push second, and let clients catch up after a gap.
• No plan for reconnects. After a deploy or a network blip, every client reconnects at once. Use backoff with jitter, and expect a spike.
• Sending a huge room's every message to every member without thinking about fan-out. Large rooms may need batching, sampling or pull-based reads.
• Ignoring connection limits. Open connections cost memory and file descriptors per server, and the limit depends on your hardware and tuning, so measure it rather than quoting a number.
• Skipping auth. A WebSocket must be authenticated when it opens, and again when a token expires.

Check yourself

• What are the two hops in a real-time design?
• When would you choose SSE over WebSockets?
• Why must a long-polling timeout be shorter than the proxy's?
• Two users are connected to different servers. How does a message get across?
• A client was offline for a minute. How does it catch up?
• Why is Redis pub/sub alone not enough for guaranteed delivery?