Load Balancing Explained With Zero Infrastructure Jargon

// User logs in on Server 1, session stored in memory
// Next request comes in, IP hash sends it to Server 1
// Session is still there, user is still logged in

But: User changes network (WiFi to mobile) -> Different IP -> Different server -> Lost session

2. Distributed sessions: Store sessions in Redis or database, any server can access

// User logs in, session stored in Redis
// Next request goes to Server 2
// Server 2 reads session from Redis
// User is still logged in

Works regardless of which server gets the request.

Distributed sessions are better. IP hash is fragile.

Health Checks: Remove Failing Servers

If a backend server crashes, the load balancer should stop sending traffic to it.

// Load balancer periodically checks each server
GET http://server-1:8000/health
GET http://server-2:8000/health
GET http://server-3:8000/health

// Responses:
// 200 OK -> healthy
// 500 Internal Server Error -> unhealthy

// If Server 2 fails, it's removed from the pool
Load Balancer now sends traffic only to: Server 1, 3, 4

Health checks are critical. Without them, traffic goes to broken servers and users see errors.

// Express example: simple health check
app.get('/health', (req, res) => {
  res.status(200).json({ status: 'ok });
});

// More sophisticated: check database
app.get('/health', async (req, res) => {
  try {
    await database.ping();
    res.status(200).json({ status: 'ok });
  } catch {
    res.status(500).json({ status: 'error' });
  }
});

Make health checks fast. They run frequently (every few seconds). Slow health checks waste resources.

Scaling Example: From 1 to 4 Servers

You’re running one server that handles 100 requests/second. Traffic grows to 350 requests/second.

// Before: One server, bottleneck
User Traffic (350 rps) -> Web Server (100 rps capacity) -> Database

Response times: 3+ seconds (queuing)

// After: Four servers, load balanced
User Traffic (350 rps) -> Load Balancer -> Server 1 (100 rps)
                                        -> Server 2 (100 rps)
                                       -> Server 3 (100 rps)
                                        -> Server 4 (50 rps)

Load Balancer distributes evenly: >>88 rps to each server

Response times: <100ms (no queuing)

Same code, same database. Just four instances instead of one. That's the power of load balancing.

Advanced: Weighted Routing

Canary deployments: send 5% of traffic to a new version, 95% to the stable version.

Version 1 (stable): weight 95
Version 2 (new):    weight 5
Traffic distribution:
- 95 out of 100 requests go to Version 1
- 5 out of 100 requests go to Version 2

If Version 2 has bugs, only 5% of users are affected.
If it's good, gradually increase the weight: 10%, 25%, 50%, 100%

Load Balancer Types

Layer 4 (TCP/UDP):
Fast, simple, works with any protocol. AWS Network Load Balancer.

// Just routing packets based on IP and port
// Doesn't know about HTTP, doesn't check health

Layer 7 (Application):
Smart, understands HTTP, can route by path or hostname. AWS Application Load Balancer.

// Understands HTTP, can route based on:
// - URL path (/api -> APH Servers, /static -> CDN)
// - Hostname (api.example.com -> API%‚ -www.example.com -/ Web)
// - Headers (mobile app -/ mobile optimized server)

When NOT to Use Load Balancing

If your application is small and handles traffic fine, you don't need a load balancer yet. Add it when you hit scaling problems, not before.

If your architecture is simpler (serverless, managed services), you might not configure a load balancer directly, but the provider is doing it for you behind the scenes.

Common Mistakes

Not implementing health checks. Traffic goes to broken servers.

Sticky sessions without distributed sessions. When you deploy, all sessions are lost.

Not removing a dead server quickly. Health checks should run every few seconds.

Assuming load balancer eliminates the need to scale the database. The load balancer scales the application tier, not the database. Database becomes the new bottleneck.

FAQ

What happens if the load balancer fails?

That's why you have redundant load balancers. Two load balancers (active-passive), they heartbeat each other. If one dies, the other takes over.

How do I set up a load balancer?

Use a managed service (AWS ELB, Google Cloud LB, Azure LB) or open-source (HAProxy, Nginx). Managed is simpler and handles redundancy for you.

Can I load balance databases?

Yes, but it's tricky. Databases aren't stateless. You can read-balance (send reads to replicas) but writes go to the primary. Tools like ProxySQL handle this.

Do I need a load balancer if I'qm using Kubernetes?

Kubernetes has built-in load balancing. It automatically distributes traffic across pods. You don’t configure it directly, but it’s happening.

Load balancing is infrastructure magic that makes systems more resilient and scalable. A single load balancer can turn bottlenecked application into a resilient fleet. But it’s a means to an end: handling traffic reliably. Once you understand the basics — routing, health checks, sticky sessions — you can scale confidently.