Load Balancing: Algorithms and Configuration

Load Balancing Algorithms

Round Robin:         Req1->S1, Req2->S2, Req3->S3, Req4->S1...
                     Simple but ignores server capacity differences

Weighted Round Robin: Server1 (w=3): 3 req, Server2 (w=1): 1 req
                      For heterogeneous servers

Least Connections:   Route to server with fewest active connections
                     Better for long-lived connections (WebSockets)

IP Hash:             hash(client_ip) % server_count
                     Same client always hits same server

Nginx Load Balancer

upstream api_servers {
    least_conn;

    server api-1.internal:3000 weight=3 max_fails=3 fail_timeout=30s;
    server api-2.internal:3000 weight=3 max_fails=3 fail_timeout=30s;
    server api-backup.internal:3000 backup;

    keepalive 32;
}

server {
    listen 443 ssl;
    ssl_certificate /etc/ssl/certs/cert.pem;
    ssl_certificate_key /etc/ssl/private/key.pem;

    location /api/ {
        proxy_pass http://api_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_connect_timeout 5s;
        proxy_read_timeout 60s;
        proxy_next_upstream error timeout http_503;
    }
}

Health Check Endpoint

app.get('/health/ready', async (req, res) => {
  const checks = await Promise.allSettled([checkDatabase(), checkRedis()]);
  const healthy = checks.every(c => c.status === 'fulfilled');

  res.status(healthy ? 200 : 503).json({
    status: healthy ? 'healthy' : 'degraded',
    checks: { db: checks[0].status, redis: checks[1].status },
  });
});

Session Persistence Options

1. Client-side sessions (JWT):
   - No affinity needed
   - Token contains session data
   - Preferred for stateless APIs

2. Sticky sessions (IP hash):
   - Client always hits same server
   - Problem: uneven load, failover loses sessions

3. Shared session store (Redis):
   - Any server handles any request
   - Best for horizontal scaling

Stateless application design with shared storage is the key to effortless horizontal scaling.