A load balancer, written from scratch

TCP and HTTP traffic spread across backends, in C++17 on plain POSIX sockets. This page reads live stats from the running server, pushed over Server-Sent Events the moment they change.

Run a load testSee the backends
Listen port
--
Healthy backends
--
Active connections
--
Updates received
0

Backends

Each server gets an HTTP HEAD every five seconds, and unhealthy ones leave the rotation until they answer again.

ConnectingConnecting…Polling every 2 seconds

Waiting for data

The first poll has not come back yet.

Traffic split

Share of active connections on each backend.

    Idle, no active connections right now.

    Connections over time

    Total active connections across all backends.

    Collecting data…

    Load test

    Fire concurrent HTTP requests at the load balancer and watch the traffic spread across the backends above.

    The load balancer frontend

    20

    Nothing fired yet. Results land here, and the backend connection counts above move as the requests arrive.

    Algorithms

    Four ways to pick a backend, switchable without a restart by sending SIGHUP.

    Least connections

    Best for
    Mixed workloads where some requests take much longer than others.
    Cost
    O(n) scan, atomic under a single mutex lock
    To run it
    Set algo = lc in lb.conf, then run kill -HUP $(pgrep load_balancer).
    Listen port--
    Stats port--
    Thread pool16 workers
    Health checkEvery 5 s
    Connection: closeInjected
    SIGHUP reloadAlgorithm and weights

    Under the hood

    One process, five kinds of thread, and no third-party libraries.

    16:47:27INFODashboard started. Connecting to stats endpoint…
    16:47:27INFOLive stream: https://lb-backend.namankundra.com/stats/stream (falls back to polling /api/stats)
    16:47:27INFOStart LB: ./scripts/start.sh
    16:47:27INFOTest: curl https://lb-backend.namankundra.com/

    Thread model

    Main thread
    accept() loop on :8080
    Worker pool (16)
    handle_client() for each connection
    Health check
    HTTP HEAD every 5 s
    Stats thread
    JSON and SSE on :8081
    SIGHUP thread
    sigwait(), then reload

    Design decisions

    TOCTOU-free
    One mutex covers select and increment
    atomic<Algorithm>
    Lock-free reads in worker threads
    SO_REUSEADDR
    Instant restart after a crash
    Retry on failure
    Next healthy backend when a connect fails
    sigwait()
    Safe async signal handling

    • Four algorithms

      Least connections, round robin, IP hash and rendezvous hashing.

    • Weighted backends

      Per-backend traffic ratios.

    • Health checks

      An HTTP HEAD to every backend, every 5 seconds.

    • Thread pool

      A fixed pool of 16 workers, no unbounded spawning.

    • Connection cap

      A hard limit per backend, skipped at capacity.

    • SIGHUP reload

      Live config changes with zero downtime.

    • Stats endpoint

      JSON on :8081/stats, streamed live from /stats/stream.

    • No libraries

      Pure C++17 on POSIX sockets.