Environment-conscious scheduling of HPC applications on distributed cloud-oriented data centers. (English) Zbl 1219.68070

Summary: The use of high performance computing (HPC) in commercial and consumer IT applications is becoming popular. HPC users need the ability to gain rapid and scalable access to high-end computing capabilities. Cloud computing promises to deliver such a computing infrastructure using data centers so that HPC users can access applications and data from a Cloud anywhere in the world on demand and pay based on what they use. However, the growing demand drastically increases the energy consumption of data centers, which has become a critical issue. High energy consumption not only translates to high energy cost which will reduce the profit margin of Cloud providers, but also high carbon emissions which are not environmentally sustainable. Hence, there is an urgent need for energy-efficient solutions that can address the high increase in the energy consumption from the perspective of not only the Cloud provider, but also from the environment. To address this issue, we propose near-optimal scheduling policies that exploit heterogeneity across multiple data centers for a Cloud provider. We consider a number of energy efficiency factors (such as energy cost, carbon emission rate, workload, and CPU power efficiency) which change across different data centers depending on their location, architectural design, and management system. Our carbon/energy based scheduling policies are able to achieve on average up to 25% of energy savings in comparison to profit based scheduling policies leading to higher profit and less carbon emissions.


68M20 Performance evaluation, queueing, and scheduling in the context of computer systems
68M14 Distributed systems


GrADS; Amazon EC2
Full Text: DOI


[1] Amazon, Amazon Elastic Compute Cloud (EC2), Aug. 2009. http://www.amazon.com/ec2/.
[2] Alpiron, Alpiron Suite, 2009. http://www.alpiron.com.
[3] Belady, C.: In the data center, power and cooling costs more than the it equipment it supports, Electronics cooling 13, No. 1, 24 (2007)
[4] Berman, F.; Casanova, H.; Chien, A.; Cooper, K.; Dail, H.; Dasgupta, A.; Deng, W.; Dongarra, J.; Johnsson, L.; Kennedy, K.; Koelbel, C.; Liu, B.; Liu, X.; Mandal, A.; Marin, G.; Mazina, M.; Mellor-Crummey, J.; Mendes, C.; Olugbile, A.; Patel, J. M.; Reed, D.; Shi, Z.; Sievert, O.; Xia, H.; Yarkhan, A.: New grid scheduling and rescheduling methods in the grads project, International journal of parallel programming 33, No. 2, 209-229 (2005)
[5] Bianchini, R.; Rajamony, R.: Power and energy management for server systems, Computer 37, No. 11, 68-74 (2004)
[6] Bradley, D.; Harper, R.; Hunter, S.: Workload-based power management for parallel computer systems, IBM journal of research and development 47, No. 5, 703-718 (2003)
[7] Braun, T.; Siegel, H.; Beck, N.; Boloni, L.; Maheswaran, M.; Reuther, A.; Robertson, J.; Theys, M.; Yao, B.; Hensgen, D.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems, Journal of parallel and distributed computing 61, No. 6, 810-837 (2001) · Zbl 0990.68013
[8] T. Burd, R. Brodersen, Energy efficient CMOS microprocessor design, in: Proceedings of the 28th Hawaii International Conference on System Sciences, HICSS’95, vol. 1060, 1995.
[9] J. Burge, P. Ranganathan, J.L. Wiener, Cost-aware scheduling for heterogeneous enterprise machines (CASH’EM), Technical Report HPL-2007-63, HP Labs, Palo Alto, Apr. 2007.
[10] Buyya, R.; Yeo, C. S.; Venugopal, S.; Broberg, J.; Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility, Future generation computer systems 25, No. 6, 599-616 (2009)
[11] Chase, J. S.; Anderson, D. C.; Thakar, P. N.; Vahdat, A. M.; Doyle, R. P.: Managing energy and server resources in hosting centers, SIGOPS operating systems review 35, No. 5, 103-116 (2001)
[12] Chen, Y.; Das, A.; Qin, W.; Sivasubramaniam, A.; Wang, Q.; Gautam, N.: Managing server energy and operational costs in hosting centers, ACM SIGMETRICS performance evaluation review 33, No. 1, 303-314 (2005)
[13] M. Chin, Desktop cpu power survey, Silentpcreview. com, 2006.
[14] US Department of Energy, Voluntary reporting of greenhouse gases: Appendix F. Electricity emission factors, 2007. http://www.eia.doe.gov/oiaf/1605/pdf/Appendix20F_r071023.pdf.
[15] K. Corrigan, A. Shah, C. Patel, Estimating environmental costs, in: Proceedings of the Ist USENIX Workshop on Sustainable Information Technology, San Jose, CA, USA, 2009.
[16] US Department of Energy, US Energy Information Administration (EIA) report, 2007. http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_a.html.
[17] Elyada, A.; Ginosar, R.; Weiser, U.: Low-complexity policies for energy-performance tradeoff in chip-multi-processors, IEEE transactions on very large scale integration (VLSI) systems 16, No. 9, 1243-1248 (2008)
[18] United States Environmental Protection Agency, Letter to enterprise server manufacturer or other interested stakeholder, Dec. 2006. http://www.energystar.gov/ia/products/downloads/Server_Announcement.pdf.
[19] United States Environmental Protection Agency, Report to congress on server and data center energy efficiency, Public Law 109-431, Aug. 2007. http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Datacenter_Report_Congress_Final1.pdf.
[20] M. Etinski, J. Corbalan, J. Labarta, M. Valero, A. Veidenbaum, Power-aware load balancing of large scale MPI applications, in: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, Rome, Italy, 2009.
[21] EUbusiness, Proposed EU regulation to reduce CO2 emissions from cars, Dec. 2007. http://www.eubusiness.com/Environ/co2-cars-eu-guide/.
[22] Fan, X.; Weber, W. -D.; Barroso, L. A.: Power provisioning for a warehouse-sized computer, , 13-23 (2007)
[23] D. Feitelson, Parallel workloads archive, Aug. 2009. http://www.cs.huji.ac.il/labs/parallel/workload.
[24] D.G. Feitelson, L. Rudolph, U. Schwiegelshohn, K.C. Sevcik, P. Wong, Theory and practice in parallel job scheduling, in: Proceedings of the 1997 International Workshop on Job Scheduling Strategies for Parallel Processing, London, UK, 1997.
[25] Feng, W.; Cameron, K.: The green500 list: encouraging sustainable supercomputing, Computer, 50-55 (2007)
[26] X. Feng, R. Ge, K.W. Cameron, Power and energy profiling of scientific applications on distributed systems, in: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, Los Alamitos, CA, USA, 2005.
[27] W. Feng, T. Scogland, The green500 list: year one, in: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, Rome, Italy, 2009.
[28] Freeh, V.; Lowenthal, D.; Pan, F.; Kappiah, N.; Springer, R.; Rountree, B.; Femal, M.: Analyzing the energy-time trade-off in high-performance computing applications, IEEE transactions on parallel and distributed systems 18, No. 6, 835 (2007)
[29] A. Gandhi, M. Harchol-Balter, R. Das, C. Lefurgy, Optimal power allocation in server farms, in: Proceedings of the 11th International Joint Conference on Measurement and Modeling of Computer Systems, Seattle, WA, USA, 2009.
[30] Gartner, Gartner estimates ict industry accounts for 2 percent of global CO2 emissions, Apr. 2007. http://www.gartner.com/it/page.jsp?id=503867.
[31] Garey, M. R.; Johnson, D. S.: Computers and intractability: A guide to the theory of NP-completeness, (1979) · Zbl 0411.68039
[32] S. Greenberg, E. Mills, B. Tschudi, P. Rumsey, B. Myatt, Best practices for data centers: results from benchmarking 22 data centers, in: Proceedings of the 2006 ACEEE Summer Study on Energy Efficiency in Buildings, Pacific Grove, USA, 2006.
[33] T. Hardware, Cpu performance charts, Toms Hardware, 2006.
[34] C. Hsu, U. Kremer, The design, implementation, and evaluation of a compiler algorithm for CPU energy reduction, in: Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation, Sweden, 2003.
[35] Ibarra, O.; Kim, C.: Heuristic algorithms for scheduling independent tasks on nonidentical processors, Journal of the ACM 24, No. 2, 280-289 (1977) · Zbl 0382.90048
[36] D. Irwin, L. Grit, J. Chase, Balancing risk and reward in a market-based task service, in: Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, Honolulu, USA, 2004.
[37] S.-H. Jang, V.E. Taylor, X. Wu, M. Prajugo, E. Deelman, G. Mehta, K. Vahi, Performance prediction-based versus load-based site selection: quantifying the difference, in: M. J. Oudshoorn, S. Rajasekaran (Eds.), ISCA PDCS, ISCA, 2005, pp. 148–153.
[38] K. Kim, R. Buyya, J. Kim, Power aware scheduling of bag-of-tasks applications with deadline constraints on dvs-enabled clusters, in: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, Rio de Janeiro, Brazil, 2007.
[39] B. Lawson, E. Smirni, Power-aware resource allocation in high-end systems via online simulation, in: Proceedings of the 19th Annual International Conference on Supercomputing, Cambridge, USA, 2005.
[40] J. Markoff, S. Hansell, Hiding in Plain Sight, google seeks more power, 2006. http://www.nytimes.com/2006/06/14/technology/14/search.html.
[41] Martello, S.; Toth, P.: An algorithm for the generalized assignment problem, Operational research 81, 589-603 (1981) · Zbl 0473.90047
[42] J. Moore, J. Chase, P. Ranganathan, R. Sharma, Making scheduling ”cool”: temperature-aware workload placement in data centers, in: Proceedings of the 2005 Annual Conference on USENIX Annual Technical Conference, Anaheim, CA, 2005.
[43] Mu’alem, A. W.; Feitelson, D. G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling, IEEE transactions on parallel and distributed systems 12, No. 6, 529-543 (2001)
[44] Nudd, G. R.; Kerbyson, D. J.; Papaefstathiou, E.; Perry, S. C.; Harper, J. S.; Wilcox, D. V.: Pace–a toolset for the performance prediction of parallel and distributed systems, International journal of high performance computing application 14, No. 3, 228-251 (2000)
[45] A. Orgerie, L. Lefèvre, J. Gelas, Save watts in your grid: green strategies for energy-aware framework in large scale distributed systems, in: Proceedings of the 2008 14th IEEE International Conference on Parallel and Distributed Systems, Melbourne, Australia, 2008.
[46] C. Patel, R. Sharma, C. Bash, M. Beitelmal, Energy flow in the information technology stack: coefficient of performance of the ensemble and its impact on the total cost of ownership, HP Labs External Technical Report, HPL-2006-55.
[47] C. Patel, R. Sharma, C. Bash, S. Graupner, Energy aware grid: global workload placement based on energy efficiency, Technical Report HPL-2002-329, HP Labs, Palo Alto, Nov. 2002.
[48] P. Pillai, K. Shin, Real-time dynamic voltage scaling for low-power embedded operating systems, in: Proceedings of the 18th ACM Symposium on Operating Systems Principles, Banff, Canada, 2001.
[49] R. Porter, Mechanism design for online real-time scheduling, in: Proceeding of the 5th ACM Conference on Electronic Commerce, New York, USA, 2004.
[50] Rivoire, S.; Shah, M. A.; Ranganathan, P.; Kozyrakis, C.: Joulesort: a balanced energy-efficiency benchmark, , 365-376 (2007)
[51] C. Rusu, A. Ferreira, C. Scordino, A. Watson, Energy-efficient real-time heterogeneous server clusters, in: Proceedings of the 12th IEEE Real-Time and Embedded Technology and Applications Symposium, Stockholm, Sweden, 2006.
[52] V. Salapura, et al. Power and performance optimization at the system level, in: Proceedings of the 2nd Conference on Computing Frontiers, Ischia, Italy, 2005.
[53] Sanjay, H. A.; Vadhiyar, S.: Performance modeling of parallel applications for grid scheduling, Journal of parallel distributed computing 68, No. 8, 1135-1145 (2008) · Zbl 1243.68124
[54] G. Singh, C. Kesselman, E. Deelman, A provisioning model and its comparison with best-effort for performance-cost optimization in grids, in: Proceedings of the 16th International Symposium on High Performance Distributed Computing, California, USA, 2007.
[55] Smith, W.; Foster, I.; Taylor, V.: Predicting application run times using historical information, Lect. notes comput. Sci. 1459, 122-142 (1998)
[56] Tang, Q.; Gupta, S. K. S.; Stanzione, D.; Cayton, P.: Thermal-aware task scheduling to minimize energy usage of blade server based datacenters, (2006)
[57] G. Tesauro, et al. Managing power consumption and performance of computing systems using reinforcement learning, in: Proceedings of the 21st Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2007.
[58] TOP500 Supercomputers, Supercomputer’s Application Area Share, 2009. http://www.top500.org/stats/list/33/apparea.
[59] G. Verdun, D. Azevedo, H. Barrass, S. Berard, M. Bramfitt, T. Cader, T. Darby, C. Long, N. Gruendler, B. Macarthur, et al. The green grid metrics: data center infrastructure efficiency (DCIE) detailed analysis, the green grid.
[60] L. Wang, Y. Lu, Efficient power management of heterogeneous soft real-time clusters, in: Proceedings of the 2008 Real-Time Systems Symposium, Barcelona, Spain, 2008.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.