Load Balancer + Rate Limiter Lab: HAProxy Stick-Tables for Connection Rate Limiting

Why Rate-Limit at the Load Balancer

Application-level rate limiting catches abuse after the request has already consumed backend resources -- TCP connection, TLS handshake, request parsing. HAProxy stick-tables let you shed load at the edge before any of that happens.

The Docker Compose Lab

# docker-compose.yml
version: '3.8'
services:
  haproxy:
    image: haproxy:2.9
    ports:
      - "80:80"
      - "8404:8404" # stats
    volumes:
      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    depends_on:
      - backend1
      - backend2
      - backend3
  backend1:
    build: ./backend
    environment:
      - BACKEND_ID=1
  backend2:
    build: ./backend
    environment:
      - BACKEND_ID=2
  backend3:
    build: ./backend
    environment:
      - BACKEND_ID=3

The backend is a minimal Go HTTP server:

// backend/main.go
package main
import (
  "fmt"
  "net/http"
  "os"
  "time"
)
func main() {
  id := os.Getenv("BACKEND_ID")
  http.HandleFunc("/api/data", func(w http.ResponseWriter, r *http.Request) {
    // Simulate 20ms of work
    time.Sleep(20 * time.Millisecond)
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"backend":"%s","status":"ok"}`, id)
  })
  http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(200)
    fmt.Fprint(w, "ok")
  })
  http.ListenAndServe(":8080", nil)
}

HAProxy Config with Stick-Table Rate Limiting

# haproxy.cfg
global
  log stdout format raw local0
defaults
  mode http
  log global
  timeout connect 5s
  timeout client 30s
  timeout server 30s
  option httplog
frontend http_front
  bind *:80
  # Stick-table: track request rate per source IP
  # 100k entries, 30s TTL, track HTTP request rate over 10s window
  stick-table type ip size 100k expire 30s store http_req_rate(10s)
  # Track the client
  http-request track-sc0 src
  # Deny if more than 50 requests in 10s (5 req/s sustained)
  http-request deny deny_status 429 if { sc_http_req_rate(0) gt 50 }
  # Add rate limit headers for observability
  http-response set-header X-RateLimit-Remaining %[sc0_http_req_rate(10s),sub(50)] if { sc_http_req_rate(0) gt 0 }
  default_backend app_servers
backend app_servers
  balance leastconn
  option httpchk GET /healthz
  http-check expect status 200
  server backend1 backend1:8080 check inter 3s fall 3 rise 2
  server backend2 backend2:8080 check inter 3s fall 3 rise 2
  server backend3 backend3:8080 check inter 3s fall 3 rise 2
frontend stats
  bind *:8404
  stats enable
  stats uri /stats
  stats refresh 5s

The key section is the stick-table. type ip size 100k creates an in-memory table keyed by source IP. store http_req_rate(10s) tracks a sliding window of request rates. The http-request deny line checks if the rate exceeds 50 requests per 10 seconds and returns 429 before the request ever reaches a backend.

Benchmarking with wrk

Start the lab and run wrk at increasing concurrency:

docker compose up -d
# Baseline: 10 concurrent connections
wrk -t4 -c10 -d30s http://localhost/api/data
# Expected: all 200s, ~20ms avg latency
# Over limit: 100 concurrent connections
wrk -t4 -c100 -d30s http://localhost/api/data
# Expected: mix of 200s and 429s
# Heavy load: 500 concurrent connections
wrk -t4 -c500 -d30s -L http://localhost/api/data
# Expected: mostly 429s, but p99 latency stays low
# because HAProxy rejects before forwarding

The -L flag prints the full latency distribution. Compare p50, p99, and p999 across the three runs.

Without stick-table rate limiting, the 500-connection test would show p99 latency climbing as backends queue requests. With it, p99 stays flat because excess requests never reach backends.

Checking the Stick-Table Live

HAProxy exposes stick-table contents via the stats socket or the stats page at :8404/stats. You can also query it from the command line:

# Show all tracked IPs and their request rates
docker compose exec haproxy \
  socat stdio /var/run/haproxy/admin.sock <<< "show table http_front"

This outputs each tracked IP with its current request rate, which is useful for debugging why a specific client is getting denied.

Key Differences from Application-Level Rate Limiting

• No backend resources consumed: HAProxy denies at the connection level before proxying
• Shared state by default: one HAProxy instance sees all traffic, no Redis needed for single-LB deployments
• No Retry-After header: stick-tables do not natively calculate when the window resets, so clients get a bare 429. Add application-level rate limiting behind this for richer error responses.
• Coarser granularity: stick-tables work on IP, not on user identity or API key. Use them as the first layer, with app-level per-user limits behind them.

This two-tier approach matches what Day 19 recommended: coarse limits at the edge, fine-grained limits in the application.