Scaling WebSocket and Handling Slow Clients

Objectives

By the end of the lesson, you can explain why a connection is tied to one server instance, how an event reaches the right instance, and what can be done about a slow client.

Why is it needed?

For its lifetime a WebSocket connection is tied to a concrete server instance. With several instances it must therefore be known which event has to be forwarded to which connection. Load balancing (7.7) places a new connection; it does not move an already open connection unnoticed into another process.

Local and shared state

The connection object and the send buffer are in the memory of the receiving instance. The identity of the user, the subscriptions and part of the resume point can be durable or shared, but sharing does not mean that another process uses the same connection. Between the business event source and the instances a broker is therefore placed (8.1: pub/sub), which delivers the event to every instance, and each distributes it to its own connections.

flowchart TB
  C1["Client 1"] --> L["Load balancer"]
  C2["Client 2"] --> L
  L --> W1["WebSocket instance 1"]
  L --> W2["WebSocket instance 2"]
  E["Business event source"] --> B["Shared pub/sub"]
  B --> W1
  B --> W2
  W1 -->|"its own connections"| C1
  W2 -->|"its own connections"| C2

With many instances, channel- or target-based distribution is advisable, so that not every instance receives every event. A shared pub/sub is not necessarily a durable replay log (8.1): the two tasks are designed separately.

Sticky session and failure

According to the NGINX documentation, the IP-hash method directs the requests of the same client to the same server (sticky). This can be useful with solutions in which several requests belong together (for example the fallback long polling transport of a library). An already open WebSocket stays on its own instance in any case. Stickiness does not replace shared event distribution and the resync after a server failure (10.3).

If an instance fails, the client connects to another one. This must re-check authorisation and restore the subscriptions (10.5). Business state kept only in local memory is lost, so a durable command result should not live only in the memory of the connection.

Slow client and buffer

The server can produce events faster than the client receives them. According to MDN the browser WebSocket interface does not support backpressure: if messages arrive faster than the application can process them, memory can fill up or the processor can reach 100% load. WebSocketStream handles this with the Streams API, but according to MDN it is non-standard and supported by only one browser engine. On the server side, unlimited growth of the send buffer leads to memory exhaustion; the unsent bytes, the number of messages and the age of the oldest data are better indicators than the connection count.

For current-state data, several updates can be merged into the latest one. For important business events a durable log and later resumption are needed (8.1). Disconnecting a client that is too slow may be justified if the protocol clearly says how it recovers (10.3).

Taking out of service

On shutdown the instance should not accept new connections; it may close the existing ones within a limited time (close code 1001 "going away", RFC 6455), or steer them to another instance with a reconnect signal. Mass disconnection causes a reconnection wave, so gradualness and jitter are needed. The proxy configuration must support connection setup and the expected idle time; intermediate buffering, a short timeout and a maximum number of connections cause different faults.

Characteristics

Aspect	Scaled WebSocket
Direction	Two-way, between instances through a broker
Initiator	Client (connection), either party (message)
Connection persistence	Tied to an instance
Latency	An extra hop through the broker
Load	Connections × message rate × fan-out
Fault tolerance	Resync on instance failure; merging or disconnecting for a slow client
When it fits	Many parallel live clients with a shared event source
When it does not	When the number of clients is small and the need is rare (polling/SSE is simpler)

Common misconceptions

Statement	Clarification
"The load balancer moves the open connection."	It places only new connections.
"A sticky session solves scaling."	It solves neither event distribution nor failure.
"The browser WebSocket slows down automatically if the receiver is slow."	There is no backpressure (MDN).
"More connections = proportionally more load."	The message rate and the fan-out matter.

In short

• A connection is tied to an instance; a broker delivers events to the right instance.
• Local, shared and durable state must be handled separately.
• The browser WebSocket has no backpressure; for a slow client merging, a log or disconnecting is needed.
• On shutdown, gradual disconnection and jitter protect against a reconnection wave.

Review questions

1. Why is a load balancer not enough for scaling WebSocket?
2. Which state is local, shared and durable in a multi-instance system?
3. What can you do if the client receives more slowly than the server produces?

Learning resources

• WebSocket API (MDN) — the lack of backpressure, WebSocketStream.
• Using nginx as HTTP load balancer — IP-hash and sticky.
• RFC 6455, 7.4.1 — the 1001 closing code.