Networking, Load Balancing & Security

3. Load Balancing

Layer 4 vs Layer 7

Rendering diagram…

Algorithm per pool:

General API: Round Robin (stateless)
Realtime WS: Least Connections + IP Hash
AI endpoints: Least Connections (variable load)
Auth: Round Robin

Health Check Configuration

Setting	Value
Endpoint	`GET /health`
Interval	10 seconds
Timeout	3 seconds
Unhealthy	2 consecutive failures
Healthy	3 consecutive successes

Response checks:

HTTP 200 OK
Body: {"status":"healthy"}
Response time <500ms
DB connectivity confirmed

On failure:

Remove pod from rotation
Alert PagerDuty (if >30% pods down)
HPA triggers new pod immediately

Provider Load Balancer Comparison

Feature	AWS ALB	GCP Cloud LB	Hostinger LB	Hetzner LB	Heroku Router
Layer	L7	L7 (global)	L4/L7	L4/L7	L7 (dyno mesh)
WebSocket	✅	✅	✅	✅	✅ (60s timeout)
Path routing	✅	✅	⚠️ limited	⚠️ limited	❌
Global anycast	✅	✅ (best)	❌	❌	❌
SSL termination	✅	✅	✅	✅	✅
Cost/month	~$20 base	~$18 base	~$5	~$6	Included
Max connections	60,000	Unlimited	10,000	10,000	~25,000

4. Rate Limiting

Three-Layer Rate Limiting Strategy

Rendering diagram…

Layer 1 — Cloudflare Edge (Network Level)

Per-IP: 1000 req/min globally · 100 req/s burst Bot rules: JS challenge on suspicious patterns Countries: optional geo-blocking Cost: free (Cloudflare Free/Pro plan)

Layer 2 — Load Balancer (Connection Level)

Per-IP connections: 100 simultaneous max Connection rate: 50 new connections/sec per IP Request body size: 10MB hard limit WebSocket connections: 10 per IP

Layer 3 — Application (Stackhouse Governor Middleware)

Anonymous users: 100 req/min, burst 20 · Authenticated users: 1000 req/min, burst 100 Auth endpoints: 5 req/min (anti-brute-force) · AI endpoints: 20 req/min per user File upload: 10 req/min per user · Admin endpoints: 50 req/min State backend: Redis (distributed counter) · Algorithm: token bucket (Governor crate) Response on limit: HTTP 429 + Retry-After header

Rate Limit State — Redis Distributed Token Bucket

Field	Value
Key	`ratelimit:{user_id}:{endpoint_group}`
Value	`{tokens_remaining, last_refill_timestamp}`
TTL	120 seconds (auto-expire unused keys)

Example keys:

ratelimit:usr_1234:api → {tokens: 87, ts: 172...}
ratelimit:ip_1.2.3.4:auth → {tokens: 2, ts: 172...}
ratelimit:usr_1234:ai → {tokens: 15, ts: 172...}

Lua script (atomic check-and-decrement):

local tokens = redis.call('GET', key)
if tokens == false then tokens = max_tokens end
if tonumber(tokens) > 0 then
  redis.call('DECR', key)
  redis.call('EXPIRE', key, 120)
  return 1  -- ALLOWED
else
  return 0  -- BLOCKED
end