This project implements a customizable load balancer designed to route asynchronous requests from multiple clients to multiple servers, ensuring an even distribution of the load. The load balancer uses consistent hashing to efficiently distribute requests and maintain system performance even in the event of server failures.
- Asynchronous Request Handling: Routes requests from several clients asynchronously among several servers.
- Consistent Hashing: Utilizes consistent hashing to distribute client requests evenly across server instances.
- Docker Deployment: Both the load balancer and server instances are containerized using Docker.
- Scalability: Supports dynamic scaling by adding or removing server instances.
- Fault Tolerance: Automatically spawns new server instances to handle requests in case of server failures.
- Endpoints:
/home(GET): Returns a unique identifier to distinguish among replicated server containers./heartbeat(GET): Sends heartbeat responses to check server health.
- Parameters:
- Number of Server Containers: 3
- Total Slots in Consistent Hash Map: 512
- Number of Virtual Servers for Each Server Container: log2(512) = 9
- Request Mapping Hash Function:
H(i) = i + 2i^2 + 17 - Virtual Server Mapping Hash Function:
Φ(i, j) = i + j + 2j^2 + 25
- Endpoints:
/rep(GET): Returns the status of the replicas managed by the load balancer./add(POST): Adds new server instances to the load balancer./rm(DELETE): Removes server instances from the load balancer./<path>(GET): Routes the request to a server replica based on consistent hashing.
- OS: Ubuntu 20.04 LTS or above
- Docker: Version 20.10.23 or above
- Languages: Python (preferred), C++, Java, or any other language
- Consistent Hashing: Maps each request to the appropriate server based on the hash value, reducing remappings when servers are added or removed. This technique distributes keys (or requests) uniformly across nodes, minimizing hotspots and ensuring balanced load distribution.
- Queue Sharding: Splits the queue into multiple shards, each managed by a different server instance. Consistent hashing ensures balanced mapping of requests to shards, even with server changes. Each shard operates independently, reducing contention and improving system performance.
Distributed Queue with Sharding Diagram:
- Network Reliability: Assumes a reliable network with minimal packet loss, designed to handle transient issues, though prolonged failures could impact performance.
- Uniform Load Distribution: Assumes client requests are uniformly distributed. If requests skew towards specific keys, additional mechanisms might be needed for effective load balancing.
- Server Homogeneity: Assumes all servers have similar capabilities and performance characteristics to ensure uniformity and predictability in handling requests.
Experiment:
Launched 10,000 asynchronous requests to a load balancer with 3 server containers.
Output:
Generated a bar chart showing the number of requests handled by each server instance.
Observations:
Bar Chart: The bar chart displays the distribution of requests among the three server containers.
Outcome: The load was distributed fairly evenly across the three servers.
Explanation:
Balanced Distribution: If the chart shows roughly equal bars, it indicates that the load balancer is effectively distributing requests across the available servers. Performance View: A balanced distribution suggests that the load balancer implementation is functioning as expected for a small number of servers. If there is significant imbalance, it could point to potential issues in the load balancing algorithm.
Experiment:
Incremented the number of server containers (N) from 2 to 6 and launched 10,000 requests for each increment.
Output:
Created a line chart showing the average load per server for different values of N.
Observations:
Line Chart: The line chart shows the average number of requests handled per server as N increases.
Expected Outcome: As N increases, the average load per server should decrease, assuming the load balancer distributes requests evenly.
Explanation:
Scalability: A decreasing trend in the average load per server with increasing N indicates good scalability. This means the load balancer effectively manages the load as more servers are added. Performance View: If the average load per server remains relatively constant or decreases as expected, it suggests that the load balancer scales well with the number of server containers.
Experiment:
Tested all endpoints of the load balancer and simulated a server failure by killing the process on a specific port.
Output:
Observed how the load balancer handles the failure and whether it quickly spawns a new instance to handle the load.
Observations:
Server Failure Handling: After simulating a failure, the load balancer should detect the unavailability of the server and redirect requests to available servers. Ideally, it should also spawn new instances if configured to do so.
Expected Outcome: The load balancer should continue to operate and handle requests, possibly with a brief period of reduced capacity during recovery.
Explanation:
Recovery Speed: A quick recovery and continued operation indicate that the load balancer is robust and responsive to server failures. Performance View: Effective failure handling demonstrates resilience and reliability of the load balancer in maintaining service availability.
- Clone the Repository:
git clone https://github.com/ogutubrenda/ds_project cd ds_project - Build Docker Images:
docker-compose build
- Run the Docker Containers:
docker-compse up
- Set up Environment Variables: Refer to the system components