برنامه ریزی کار موازی برای سیستم های محاسبات با کارایی بالا قدرت محدود

Power has become the primary constraint in high performance computing. Traditionally, parallel job scheduling policies have been designed to improve certain job performance metrics when scheduling parallel workloads on a system with a given number of processors. The available number of processors is not anymore the only limitation in parallel job scheduling. The recent increase in processor power consumption has resulted in a new limitation: the available power. Given constraints naturally lead to an optimization problem. We proposed MaxJobPerf, a new parallel job scheduling policy based on integer linear programming. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique that running applications at reduced CPU frequency/voltage trades increased execution time for power reduction. The optimization problem determines which jobs should run and at which frequency. In this paper, we compare the MaxJobPerf policy against other power budgeting policies for different power budgets. It clearly outperforms the other power-budgeting approaches at the parallel job scheduling level. Furthermore, we give a detailed analysis of the policy parameters including a discussion on how to manage job reservations to avoid job starvation.

Power consumption has become the primary concern in modern high performance computing (HPC) system design. The high power draw of modern systems leads to high operating costs, reliability issues and environment protection concerns. However, these are not the only factors that may limit available power in HPC environments. A power budget may be imposed by the existing power provisioning facilities as well as by any of the high power consumption issues mentioned before. It should be mentioned that power provisioning infrastructure is extremely costly. The cost of building a power provisioning facility ranges of $10–22 per deployed IT watt [2]. Nowadays, large scale computing centers typically consume few megawatts of power. According to some predictions, future exascale systems might consume more than 100 megawatts [14]. It is expected that in the future, the main goal of HPC, performance, will be limited even more by the available power. CPU power presents a significant portion of total system power when the system is under load [12]. Therefore, in this work we aim to maximize job performance in an HPC center under a given CPU power budget. The budget is given for all system processors. We have shown before that the DVFS technique can improve job wait times in a case of a power constrained system [5]. DVFS (Dynamic Voltage Frequency Scaling) is a widely used technique that trades processor performance for lower power consumption. With DVFS a processor can run at one of the supported frequency/voltage pairs lower than the nominal one. Lower frequency and voltage lead to significantly lower power consumption allowing more jobs to run simultaneously. In spite of the job runtime increase due to frequency scaling, overall performance measured in job performance metrics improves with DVFS application because of shorter job wait times. In traditional parallel job scheduling the available number of processors has been considered to be the only limiting factor preventing all queued jobs to start immediately. With the increase in processor power consumption, the available power has emerged as a new constraint. A policy that assigns CPU frequency to a job selected for execution in such a way that a given power budget is not violated has already been proposed [5]. Here we refer to this policy as PB-guided. It presents a power budgeting upgrade of the traditional EASY backfilling policy [28]. When the EASY scheduler is invoked, it selects jobs to run on the available processors from the wait queue one by one. Similarly, the PB-guided policy assigns CPU frequency to each job independently of the frequencies to be assigned to the other jobs that will be selected for execution. The PB-guided policy is designed to manage power in a conservative way in order to save some power for the other jobs selected for execution. Due to this conservatism, the power budget may not be fully exploited. In this paper we analyze in more detail a more sophisticated power budgeting parallel job scheduling policy, MaxJobPerf, based on integer linear programming [6]. Each time the scheduler is invoked, it solves an optimization problem to determine which jobs from the wait queue should start. The same optimization problem distributes the available power among the jobs assigning CPU frequency to each of the selected jobs. In this way the scheduler allocates both types of available resources, the processors and power, to all jobs at the same time. Linear programming has been already investigated for power unconstrained systems targeting specific types of scheduling such as scheduling of moldable jobs [4] or scheduling in heterogeneous environments [29]. We have evaluated the MaxJobPerf policy against an EASY backfilling based baseline without DVFS and the PB-guided policy. The policies have been tested for five workloads from production use with up to 9216 processors. The results clearly show that the MaxJobPerf policy outperforms the other two policies for all observed workloads. Due to its maximal use of the power budget and a relaxed constraint of job execution order, the MaxJobPerf significantly reduces the average job wait time, even when compared to the PB-guided policy. The average job wait time is even few times better for some workloads. The reduction in the average wait time leads to further improvements in other job performance metrics. The evaluation has been done for different power budgets and the MaxJobPerf policy performs consistently better than the other approaches over various power budgets. As we showed before [6], for some workloads it is of crucial importance that the MaxJobPerf scheduler manages job reservations. This is mandatory in order to control the job wait time. Without reservations, large jobs requesting a significant portion of the machine can experience very high wait times. In this paper, we performed an extensive study to determine an appropriate reservation assignment condition. The performed simulations show that wait time based reservations give better results than job size based reservations. We investigated how to determine an appropriate wait time threshold depending on the system load. The rest of the paper is organized as follows. The next section gives a description of the MaxJobPerf policy. In Section 3 we explain the models of frequency scaling impact on the job runtime and the average power consumption. The first part of Section 4 presents the evaluation methodology whilst the rest of the section discusses the obtained results. Section 5 gives an overview of the related work. It is followed by our conclusions.

In this paper we analysed a new parallel job scheduling policy that maximizes job performance under a given power budget, managing both processors and power as limited resources. The policy shows excellent performance for systems where CPU power is a critical resource meaning that there is not enough power to run all system processors fully loaded at the nominal frequency. Our MaxJobPerf policy uses integer linear programming to select which jobs from the wait queue should start. Their CPU frequencies are determined by the same optimization problem. We have compared the MaxJobPerf policy against the EASY based baseline without frequency scaling and the PB-guided policy. The MaxJobPerf policy achieves much better job performance measured in the Bounded Slowdown metric than the other two policies for various power budgets. This improvement in the BSLD metric is a result of a decrease in job wait times. The improvement in job wait times has come from two reasons. The MaxJobPerf policy uses all power available at the moment leading to higher average frequency compared to the PB-guided policy. Still, it uses frequency scaling allowing more jobs to run simultaneously under the same power budget compared to the baseline. The other reason for the performance improvement lies in the further relaxation of the FCFS order. Though the EASY based baseline allows jobs to execute before previously arrived ones under certain conditions, the execution order with the MaxJobPerf policy presents further arrival order relaxation allowing better job packing. However, the arrival order is indirectly respected by prioritization of jobs with higher wait times in the objective function and the use of the parameter WQChunkSize. A job scheduling policy deployed in a supercomputing center must not lead to job starvation. For this reason it was of crucial interest to discuss how the MaxJobPerf policy should manage job reservations. In this paper we showed that reservation assignment driven by the job wait time controls better high wait times characteristic for large jobs than reservations based on the job size. Furthermore, we investigated how to determine an appropriate wait time threshold necessary to acquire a reservation. This threshold can be assigned to each class of load thus that depending on the system load the threshold is automatically determined. The same threshold can be used to implemented various job priority levels through different values of the threshold.