UC BERKELEY

TECHNICAL REPORTS

EECS-2009-109.pdf
Conditions of Use

Archive Home Page

Towards Energy Efficient MapReduce

Authors:
Chen, Yanpei
Keys, Laura
Katz, Randy H.
Technical Report Identifier: EECS-2009-109
August 5, 2009
EECS-2009-109.pdf

Abstract: Energy considerations are important for Internet datacenters operators, and MapReduce is a common Internet datacenter application. In this work, we use the energy efficiency of MapReduce as a new perspective for increasing Internet datacenter productivity. We offer a framework to analyze software energy efficiency in general, and MapReduce energy efficiency in particular. We characterize the performance of the Hadoop implementation of MapReduce under different workloads. We also introduce quantitative models to guide operators and developers in improving the performance of MapReduce/Hadoop. A major, but somewhat unsurprising finding is that for workloads where the work rate is proportional to the amount of resources used, improving the performance as measured by traditional metrics such as job duration is equivalent to improving the performance as measured by lower energy consumed.