How to Learn Load Balancing Techniques from the system-design-notes Repository

The system-design-notes repository provides a comprehensive collection of real-world load balancing patterns, from basic horizontal scaling fundamentals in Chapter 1 to advanced techniques like WebSocket-aware routing, rate limiting, and graceful draining.

The liquidslr/system-design-notes repository is an open-source collection of system design chapters that documents how large-scale services implement load balancing techniques. By examining the markdown files across different design scenarios—from object storage to email services—you can understand the architectural trade-offs, placement strategies, and operational patterns that make load balancers critical infrastructure components.

Fundamental Load Balancing and Scaling

Start with Chapter 1 – Scaling (01. Scaling/Readme.md) to understand the core purpose of a load balancer in distributed systems. According to the source code, Section 4 (lines 55-71) illustrates a load balancer placed between clients and a pool of stateless servers.

This section establishes three primary benefits:

• Redundancy – eliminating single points of failure by distributing traffic across multiple instances
• Horizontal scalability – adding or removing servers behind the balancer without client awareness
• Request routing – acting as the entry point when moving from single-server to multi-server architectures

The repository also covers GeoDNS Routing (lines 62-63) in the same chapter, demonstrating how DNS-based geographic load balancing directs clients to the nearest data-center, reducing latency while maintaining high availability.

Load Balancer Placement in Multi-Tier Architectures

In Chapter 24 – S3-like Object Storage (24. S3-like Object Storage/README.md), the high-level design (line 115) explicitly lists the load balancer as the component that "distributes API requests across service replicas." This pattern shows how a front-end balancer fits into a complex multi-tier system that includes IAM, metadata stores, and data storage layers.

This placement demonstrates the API gateway pattern, where the load balancer serves as the unified entry point for RESTful operations before routing to specific service instances handling authentication, metadata lookup, or blob storage.

Traffic Protection and Rate Limiting

Chapter 23 – Distributed Email Service (23. Distributed Email Service/README.md) illustrates protective load balancing techniques at line 139. The documentation describes how the load balancer "rate limits excessive mail sends," acting as a gatekeeper that enforces per-user or per-IP quotas before traffic reaches SMTP servers.

This pattern highlights the edge protection responsibility of load balancers, using them to:

• Prevent downstream service overload
• Implement traffic shaping policies
• Block abusive clients at the perimeter

WebSocket-Aware and Connection Management

Advanced load balancing techniques appear in Chapter 17 – Nearby Friends (17. Nearby Friends/README.md), which addresses protocols with different connection lifetimes.

The repository describes WebSocket-aware balancing (line 77), showing how a load balancer routes both standard HTTP requests and long-lived socket connections across REST and WebSocket server pools. This requires sticky session support or protocol-aware routing to maintain persistent connections.

Additionally, the chapter covers draining and graceful shutdown (line 140). This technique involves marking a server as "draining" so the load balancer stops sending new connections to an instance slated for removal, while allowing existing sessions to complete. This ensures safe autoscaling and deployment without dropping active client connections.

Dynamic Scaling and Monitoring Integration

Chapter 20 – Metrics Monitoring and Alerting System (20. Metrics Monitoring and Alerting System/README.md) connects load balancing to operational automation. Line 230 describes the "Collector in an auto-scaling group behind a load balancer," demonstrating how monitoring data triggers scale-out events.

This integration shows how load balancers maintain an up-to-date target pool as auto-scaling groups add or remove instances in response to traffic spikes. The balancer dynamically registers new healthy instances and deregisters terminated ones, enabling fully automated capacity management.

Practical Implementation Examples

The design patterns in the repository align with these concrete implementation approaches:

AWS Application Load Balancer Configuration

This CloudFormation template mirrors the "distribute API requests across service replicas" pattern from Chapter 24:

# Example: AWS Application Load Balancer (ALB) target group

# Mirrors the “distribute API requests across service replicas” pattern from Chapter 24.

Version: '2012-10-17'
Resources:
  ApiTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: api-targets
      Port: 80
      Protocol: HTTP
      VpcId: vpc-xxxxxxx
      HealthCheckPath: /health
      TargetType: instance
  ApiALB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: api-alb
      Subnets:
        - subnet-111111
        - subnet-222222
      Scheme: internet-facing
      Type: application
  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref ApiALB
      Port: 80
      Protocol: HTTP
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref ApiTargetGroup

Nginx Reverse Proxy with Rate Limiting

This configuration demonstrates the rate-limiting and draining concepts from Chapters 17 and 23:

# Example: Nginx acting as a reverse‑proxy load balancer

# Shows “draining” and rate‑limiting concepts from Chapters 17 & 23.

http {
    limit_req_zone $binary_remote_addr zone=mail:10m rate=10r/s;

    upstream backend {
        server app1.example.com;
        server app2.example.com;
        server app3.example.com backup;   # “draining” can be achieved by moving a server to backup

    }

    server {
        listen 80;
        location / {
            proxy_pass http://backend;
        }
        location /sendmail {
            limit_req zone=mail burst=20 nodelay;
            proxy_pass http://backend;
        }
    }
}

Graceful Shutdown Implementation

This Go example implements the "mark a server as draining" pattern described in Chapter 17:

// Example: Go net/http server with graceful shutdown (draining)
// Mirrors the “mark a server as draining in the load balancer” idea from Chapter 17.
func main() {
    srv := &http.Server{Addr: ":8080", Handler: http.DefaultServeMux}
    go func() {
        if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatalf("listen: %s\n", err)
        }
    }()
    // Wait for termination signal...
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
    <-quit
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    if err := srv.Shutdown(ctx); err != nil {
        log.Fatal("Server forced to shutdown:", err)
    }
    log.Println("Server exiting")
}

Summary

• Start with fundamentals in 01. Scaling/Readme.md to understand why load balancers are required for horizontal scaling and redundancy.
• Study placement patterns in 24. S3-like Object Storage/README.md to see how balancers front complex API surfaces in multi-tier architectures.
• Explore protection mechanisms in 23. Distributed Email Service/README.md for rate limiting and traffic shaping implementations.
• Master connection management in 17. Nearby Friends/README.md for WebSocket-aware routing and graceful draining techniques.
• Integrate with auto-scaling using patterns from 20. Metrics Monitoring and Alerting System/README.md to dynamically manage target pools.

Frequently Asked Questions

What is the first chapter to read for load balancing basics?

Read Chapter 1 – Scaling (01. Scaling/Readme.md), specifically Section 4 (lines 55-71). This section explains the fundamental role of load balancers in providing redundancy and enabling horizontal scalability when moving from single-server to multi-server architectures.

How does the repository explain WebSocket load balancing?

Chapter 17 – Nearby Friends (17. Nearby Friends/README.md) covers WebSocket-aware balancing at line 77, showing how to route traffic across both REST and WebSocket server pools. The chapter also addresses the "draining" pattern (line 140) for safely managing long-lived connections during server maintenance.

Where can I find examples of rate limiting in load balancers?

Chapter 23 – Distributed Email Service (23. Distributed Email Service/README.md) describes rate limiting at line 139, where the load balancer acts as a gatekeeper to prevent excessive mail sends from reaching SMTP servers, protecting downstream infrastructure from overload.

How does the repository connect load balancers to auto-scaling?

Chapter 20 – Metrics Monitoring and Alerting System (20. Metrics Monitoring and Alerting System/README.md) demonstrates this integration at line 230, showing how collectors run in auto-scaling groups behind load balancers. The pattern illustrates how monitoring data triggers scaling events while the balancer dynamically updates its target pool.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}