IO Cost Evaluation of OLAP Query Processing with MapReduce

Abstract

Google’s MapReduce has emerged as a popular framework for data-intensive computing. It is well-known by its elastic scalability and fine-grained fault tolerance. On the other hand, there are some debates in its efficiency. Especially, local and network I/Os can be a primary factor that degrades the performance of MapReduce, because it follows a data shipping paradigm where many partitioned data blocks move along distributed nodes. In this paper, we conduct a performance study to examine the I/O cost of MapReduce. Our results show that the I/O cost accounts for about 80% of the total processing cost when processing OLAP queries in the MapReduce platform.