WebSockets

Why it matters

Chat, multiplayer games, collaborative documents and trading screens all need both sides to send small messages often, with minimal latency. HTTP's request-response shape fights that; WebSockets replace it with a raw two-way pipe. The protocol itself is simple. The hard part, and the part interviews probe, is operating hundreds of thousands of long-lived stateful connections across a fleet of servers that each know only their own.

The idea

A WebSocket starts as an HTTP request with Upgrade: websocket. If the server agrees (101 Switching Protocols), the same TCP connection stops speaking HTTP and carries WebSocket frames in both directions until either side closes. Frames are small (two bytes of overhead for short messages), can be text or binary, and have no request-response pairing: either side sends whenever it wants.

What you must build yourself

HTTP gives you stateless requests, retries, caching and load balancing for free. A WebSocket gives you a pipe and nothing else.

1. 1

   Heartbeats

   Send ping frames (or application-level pings) every 20 to 30 seconds. Intermediaries kill idle connections, and TCP will not tell you a peer vanished until you try to send. A missed pong means close and reconnect.
2. 2

   Reconnect with backoff and resume

   Clients reconnect on close with exponential backoff and jitter. To avoid losing messages, the client sends its last-seen sequence number and the server replays from a short per-client buffer, or the client refetches state over HTTP.
3. 3

   Message protocol

   Frames have no types. Define an envelope (type, id, payload) and version it. Acknowledgements for anything that must not be lost.
4. 4

   Authentication

   Authenticate at the handshake (cookie or token in the URL or first message) and bind the connection to the principal. Re-check long-lived connections when tokens expire.
5. 5

   Backpressure

   A slow client cannot drain its socket; unbounded server-side buffers grow until memory runs out. Cap per-connection queues and drop or disconnect slow consumers.

Scaling across servers

A connection lives on exactly one server, so a message from user A to user B, who is connected to a different server, has to cross servers. The standard architecture:

Servers subscribe to channels for the rooms or users their connections care about and publish outgoing messages to those channels; the backbone fans out. A connection registry (user id to server id, in Redis) lets targeted messages go to one server instead of being broadcast. Load balancers need long-connection support and should balance on connect, with connection draining on deploys so clients reconnect gradually instead of all at once. Keep WebSocket servers stateless apart from the sockets themselves, so losing one just means its clients reconnect elsewhere.

Capacity

A well-tuned event-driven server holds hundreds of thousands of mostly-idle connections; memory per connection (buffers, TLS state) and file descriptor limits are the constraints, not CPU. Budget for the thundering herd on restart: a hundred thousand clients reconnecting within a second need backoff with jitter on the client and connection rate limiting on the server.

Trade-offs

WebSockets give the lowest latency and true bidirectionality with tiny per-message overhead, and in exchange you own heartbeats, reconnection, ordering, auth refresh, backpressure, cross-server fan-out and deploy choreography. For server-to-client-only traffic SSE is lighter; for occasional client messages, SSE plus HTTP is simpler. Use WebSockets when the client genuinely sends often and latency matters.

In an interview

Draw the fan-out, not just the socket: "Clients hold a WebSocket to one of N gateway servers; servers are stateless apart from sockets and subscribe to room channels on Redis pub/sub. A message is published once and delivered to each server with a member in that room. Heartbeats every 30 seconds, reconnect with jittered backoff and a sequence number for replay, per-connection send queues capped to protect memory." Then capacity: "Roughly 100k connections per server, so N scales with concurrent users, not messages."