Hierarchical resource allocation and consolidation in a multi-core server cluster

What is it about?

The ever increasing demands for high-performance computing and large data storage have made distributed and cloud computing systems attract a lot of attentions from both industry and academia recently. There are some service level agreements (SLAs) made between clients and the cloud owner, which specify that the cloud owner's total profit depends on how these SLAs are satisfied. In this situation, resource allocation becomes one of the most important tasks of cloud owner. This study analyses an SLA-based resource allocation problem in a server cluster. The objective is to maximise the total profit, which is the total revenue obtained from servicing the clients minus the operational cost of the server cluster. The total revenue depends on the average request response time for each client as defined in his/her utility function by the SLAs, whereas the operating cost depends on the total energy consumption of the server cluster. A joint optimisation framework is proposed, comprised of request dispatching, dynamic voltage and frequency scaling (DVFS) for individual cores of the servers, as well as server- and core-level consolidations. Each DVFS-enabled core in the server cluster is modelled by using a continuous-time Markov decision process (CTMDP). A near-optimal solution is presented, which is comprised of a central manager and distributed local agents. In this algorithm, each local agent employs linear programming-based CTMDP solving method to solve the DVFS problem for the corresponding core, judiciously selecting the most appropriate execution frequency based on the number of waiting requests. On the other hand, the central manager solves the request dispatch problem and finds the optimal number of ON cores and servers for request processing, thereby achieving a desirable tradeoff between service request response time and power consumption. To reduce the computational overhead, the central manager uses a two-tier hierarchical solution, in which the first tier associates each client with a selected set of servers, and the second tier solves the request dispatch problem based on the results of the first tier solution. Experimental results demonstrate the consistent outstanding performance of the proposed near-optimal resource allocation and consolidation algorithm over the baseline algorithms.

Featured Image

Why is it important?

We consider the SLA-based resource allocation optimisation problem in the cloud computing framework. The objective is to maximise the total profit, which is the total revenue obtained from serving the clients subtracted by the energy cost of the server cluster. The total revenue depends on the average request response time for each client as defined in its utility function. Each core in the server cluster is modelled by a CTMDP. We propose a joint optimisation framework accounting for request dispatch, DVFS for individual cores in the server cluster, as well as core-level and server-level consolidations. The near-optimal solution is comprised of a central manager and distributed local agents. Each local agent employs linear programming-based CTMDP solving method to solve the DVFS problem for the corresponding core. On the other hand, the central manager solves the request dispatch problem and finds the optimal number of ON cores and servers for request processing. To reduce the computational overhead, the central manager is realised as a two-tier hierarchical controller. Experimental results demonstrate that the proposed near-optimal resource allocation and consolidation algorithm consistently outperforms the baseline algorithms.