Designing a load-balancer based on OS load

This is my first post after two years. Have been in an accident recently and had some spare time to think while taking rest for a month.

“Do we really consider OS load in the load-balancers?” I asked myself one-day while drowsy. There are myriad number of load-balancers which balance network traffic at various layers of TCP/IP. Some are TCP based, some IP, some can load-balance at application level like HTTP etc. But, most of them revolve around network latency etc. and almost none checks for OS load to schedule traffic to a particular node.

“Why?” probably because it is less cumbersome to keep track of the metrics that determine or influence load-balancer’s scheduler. But, what’s the harm in trying to make a tool which can really load-balance traffic based on the OS load metrics like memory usage, CPU usage, io-wait time, etc. “No harm at all” I replied to my own question (yes, I am talking to myself now-a-days after the head-eye injury during accident). So let’s start building a tool which can efficiently provide a “score card” to every node in the cluster and schedule traffic based on who has got a better score. As simple as that!

As a result, I started a project at GitHub, named “Themis” 

Every node in the cluster will have an agent running at a frequency depending upon the overall load of the cluster. The agent deducts (initial score is 100) scores the node based on available resources like CPU wait time, load-average, available RAM, swapiness etc. These can be tunable by predefining thresholds like 20% of the total CPU time is acceptible as a normal wait time for a particular node. The better the score is, the lower the load on that particular node. There is a plan to keep heuristics as well, to make the scoring process a bit smarter over the time by manipulating the thresholds automatically.
After the agent assigns a score to a particular node, it sends that score (along with the trend data, but probably less frequently) over to the load balancer through QPID (v0.30) messaging in JSON format
The load-balancer keeps a queue of all the nodes reverse-sorted based on their scores, i.e the node with highest score will be put in front of the queue and will be scheduled for the next connection, in case this node does not respond, next node in the queue will be considered.
Load-balancer will also keep track of a “participation-score” i.e how many times a particular node has been selected by the load-balancer. The lowest scorers will be reported by the load-balancer along with the different resource-limit/load data pin-pointing the reason why that node was not selected so frequently, thus giving a hint to the sysadmins on the bottlenecks
I have not yet started working on the load-balancer side, nor have I thought of a way to actually switch traffic to the nodes, well not yet. But then there are so many projects going on in that area and I can probably get inspired by one of them or just simply use my tool as an add-on to “influence” their scheduler policies.

So folks, if you are a Python devops/sysops guy and have some ideas to share with me or participate in the project, please drop a comment here, and we can catch up at GitHub.

Lets code it!!!

About admin_xor

Un*x/Linux junkie, loves to program, automate, automate and automate
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s