Building Ultra-Low Latency Real-Time APIs: WebSockets, SSE, and Redis Pub/Sub at Scale

Key Takeaway / TL;DR:

• WebSockets are ideal for bidirectional, low-latency communication, but SSE is simpler and more resilient for one-way server-to-client updates.
• Redis Pub/Sub is the backbone for horizontal scaling, but you must handle connection pooling and backpressure to avoid memory spikes and message loss.
• Graceful backoff and client reconnection strategies are critical for maintaining a stable real-time system under load.

The Problem & Industry Shift

Real-time features are no longer a luxury; they are table stakes for modern applications. From live dashboards and collaborative editing to chat and gaming, users expect sub-100ms updates. However, building a system that handles 100k+ concurrent connections is non-trivial. Traditional REST APIs with polling create unnecessary load and latency. The industry has shifted toward push-based protocols:

• WebSockets: Full-duplex, low-latency, but require persistent connections and more complex lifecycle management.
• Server-Sent Events (SSE): Simpler, one-way, auto-reconnect, but limited to server-to-client and HTTP/1.1 connection limits.
• Redis Pub/Sub: A lightweight message broker that enables horizontal scaling by broadcasting events across multiple application instances.

Each technology has trade-offs, and a robust architecture often combines them. This article dives into the engineering decisions required to build a real-time API that scales.

Architecture & Core Mechanics

A typical real-time architecture at scale looks like this:

[Client] <--WebSocket/SSE--> [Load Balancer] <--> [Node.js Instances] <--> [Redis Pub/Sub]
                                                                     |
                                                                     [Database]

• Clients connect via WebSocket or SSE to any instance behind a load balancer (sticky sessions are not required if using Redis).
• Each Node.js instance manages a set of connections and uses Redis Pub/Sub to broadcast messages to all instances.
• Redis acts as a message bus; when an event occurs (e.g., a new chat message), the producer publishes to a channel, and all instances subscribed to that channel forward the message to their local clients.

Connection Pooling

Each WebSocket or SSE connection consumes resources (memory, file descriptors). To handle 100k+ connections, you must:

• Use connection pooling for Redis clients (e.g., ioredis with cluster or redis with pool).
• Monitor memory usage per connection; a typical WebSocket connection can consume 20-50KB.
• Use backpressure mechanisms: if a client is slow, buffer messages or drop non-critical ones.

Horizontal Scaling with Redis Pub/Sub

Redis Pub/Sub is not persistent; if a subscriber is down, messages are lost. For critical updates, consider using Redis Streams or a message queue like RabbitMQ. However, for low-latency real-time, Pub/Sub is sufficient with a graceful degradation strategy.

Graceful Backoff

Clients will disconnect and reconnect. Implement exponential backoff with jitter to avoid a thundering herd on the server.

Production Code Example

Below is a production-grade Node.js example using ws for WebSockets, express for SSE, and ioredis for Redis Pub/Sub. It demonstrates connection pooling, broadcasting, and backpressure handling.

// server.js
const express = require('express');
const http = require('http');
const WebSocket = require('ws');
const Redis = require('ioredis');

const app = express();
const server = http.createServer(app);

// Redis connection pool (single connection is fine for demo; use cluster in prod)
const redis = new Redis({ host: 'localhost', port: 6379, maxRetriesPerRequest: 3 });
const pub = new Redis({ host: 'localhost', port: 6379 });

// WebSocket server with per-message deflate (for compression)
const wss = new WebSocket.Server({ server, perMessageDeflate: true });

// Map to track clients and their subscriptions (for targeted messages)
const clients = new Map(); // clientId -> ws

wss.on('connection', (ws, req) => {
  const clientId = req.headers['x-client-id'] || Math.random().toString(36);
  clients.set(clientId, ws);
  console.log(`Client connected: ${clientId}. Total: ${clients.size}`);

  // Subscribe to Redis channel for this client (unique channel for targeted messages)
  const channel = `client:${clientId}`;
  redis.subscribe(channel, (err, count) => {
    if (err) console.error('Subscribe error:', err);
  });

  ws.on('message', (message) => {
    // Handle incoming message (e.g., chat message)
    const parsed = JSON.parse(message);
    if (parsed.type === 'chat') {
      // Publish to global channel
      pub.publish('global', JSON.stringify({ from: clientId, text: parsed.text }));
    }
  });

  ws.on('close', () => {
    clients.delete(clientId);
    redis.unsubscribe(channel);
    console.log(`Client disconnected: ${clientId}. Total: ${clients.size}`);
  });

  // Send initial ack
  ws.send(JSON.stringify({ type: 'connected', clientId }));
});

// Redis subscriber for global channel
redis.on('message', (channel, message) => {
  if (channel === 'global') {
    const parsed = JSON.parse(message);
    // Broadcast to all local clients except sender (or include sender if needed)
    for (const [id, ws] of clients) {
      if (id !== parsed.from) {
        // Backpressure: if ws.bufferedAmount is high, skip or drop
        if (ws.bufferedAmount < 1024 * 1024) { // 1MB buffer limit
          ws.send(JSON.stringify({ type: 'chat', data: parsed }));
        } else {
          console.warn(`Client ${id} is slow; dropping message`);
        }
      }
    }
  } else if (channel.startsWith('client:')) {
    // Targeted message
    const clientId = channel.split(':')[1];
    const ws = clients.get(clientId);
    if (ws && ws.readyState === WebSocket.OPEN) {
      ws.send(message); // message is a string; you may want to parse and re-stringify
    }
  }
});

// SSE endpoint for one-way updates
app.get('/events', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  res.flushHeaders();

  const clientId = req.query.clientId || Math.random().toString(36);
  const channel = `client:${clientId}`;
  redis.subscribe(channel); // Note: in production, manage subscriptions carefully

  const sendEvent = (data) => {
    res.write(`data: ${JSON.stringify(data)}\n\n`);
  };

  // Send initial comment to establish connection
  sendEvent({ type: 'connected' });

  req.on('close', () => {
    redis.unsubscribe(channel);
    res.end();
  });
});

// Start server
const PORT = process.env.PORT || 3000;
server.listen(PORT, () => {
  console.log(`Server listening on port ${PORT}`);
});

Key Engineering Decisions:

• Redis connection pooling: Use separate connections for pub and sub to avoid blocking. In production, use a cluster and pool.
• Backpressure: Check ws.bufferedAmount to prevent memory exhaustion on slow clients.
• Client identification: Use a header or query param to maintain state across reconnects.

Performance, Cost & Trade-offs

Benchmarks

In a load test with 100k concurrent WebSocket connections on a single Node.js instance (with 8GB RAM), you can expect:

• Memory usage: ~3-5GB for connections (30-50KB per connection).
• Throughput: ~10k messages/sec broadcast with a single Redis instance.
• Latency: p99 < 50ms for message delivery across instances (network overhead).

Trade-offs

• WebSockets vs SSE: WebSockets offer lower latency for bidirectional communication but require more server resources and are harder to scale (need to maintain connection state). SSE is simpler and works over HTTP/2, allowing multiplexing, but is one-way.
• Redis Pub/Sub vs Message Queues: Pub/Sub is fast but not persistent; if a subscriber is down, messages are lost. For critical messages, use Redis Streams or Kafka.
• Cost: Redis is memory-intensive; a cluster with 10GB RAM can handle ~1M subscriptions. Horizontal scaling with more Node instances increases cost but improves fault tolerance.

Security Considerations

• Authentication: Use JWT or session tokens during the WebSocket handshake.
• Rate limiting: Prevent abuse by limiting messages per second.
• Input validation: Never trust client data; sanitize all messages.

Actionable Checklist / Summary

When adopting real-time APIs in production, follow these steps:

1. Choose the right protocol: Use WebSockets for bidirectional, SSE for one-way updates, or both.
2. Design for horizontal scaling: Use Redis Pub/Sub to decouple producers and consumers.
3. Implement connection pooling: Use dedicated Redis clients for pub and sub, and pool connections to avoid bottlenecks.
4. Handle backpressure: Monitor buffered amounts and drop non-critical messages when clients are slow.
5. Implement graceful backoff: Use exponential backoff with jitter for client reconnections.
6. Monitor and alert: Track connection counts, message latency, and Redis memory usage.
7. Plan for failure: Use Redis Sentinel or Cluster for high availability.
8. Test at scale: Load test with realistic concurrent connections and message rates.

By following these guidelines, you can build a real-time API that handles 100k+ concurrent connections with ultra-low latency and high reliability.