Multitenant hadoop with advanced resource management향상된 자원관리를 지원하는 멀티테넌트 Hadoop

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 771
  • Download : 0
Multitenancy has gained growing importance with the development and evolution of cloud computing technology. In a multitenant environment, multiple tenants with different demands can share a variety of computing resources (e.g., CPU, memory, storage, network, and data) within a single system, while each tenant remains logically isolated. This useful multitenancy concept offers highly efficient and cost-effective systems without wasting computing resources to enterprises requiring similar environments for data processing and management. In this paper, we propose a novel approach supporting multitenancy features for Apache Hadoop, a large scale of distributed system commonly used for processing big data. Hadoop largely consists of HDFS responsible for storing and managing big data, YARN responsible for resource management and job execution management, and MapReduce as a programming model for data processing and analysis. We note that current Hadoop incurs many problems in system utilization due to its file-based management of metadata to which HDFS and YARN frequently refer for their operations, and also due to its limited resource (CPU and memory) management provided by YARN. These problems are also critical obstacles in providing multitenancy in Hadoop. To solve these problems, we propose a RDBMS-based metadata management scheme and an advanced resource management framework for improving the functional aspects of overall Hadoop. For this, we first analyze the Hadoop framework focusing on HDFS and YARN. We next define the problems for supporting multitenancy caused by its metadata and resource management and formally derive the requirements for resolving these problems. Based on the requirements, we then design the details of Multitenant Hadoop that replaces the original HDFS and YARN with the advanced HDFS and Multitenant YARN. Finally, through implementation of the proposed Multitenant Hadoop, we validate its correctness by experimental evaluation and we also show that Multitenant Hadoop satisfies all the derived requirements. The proposed Multitenant Hadoop significantly enhances the metadata and resource management scheme of Hadoop and, as a result, improves its stability, availability, and scalability. This can be much helpful in analyzing and proccessing data with high effectiveness for users who have difficulties with building large scale clusters.
Advisors
Whang, Kyu-Youngresearcher황규영researcher
Description
한국과학기술원 :전산학부,
Publisher
한국과학기술원
Issue Date
2016
Identifier
325007
Language
eng
Description

학위논문(박사) - 한국과학기술원 : 전산학부, 2016.8 ,[iv, 63 p. :]

Keywords

Multitenancy; Hadoop; Advanced HDFS; Multitenant YARN; Metadata Management; 멀티테넌시; 하둡; 향상된 HDFS; 멀티테넌트 YARN; 메타데이터 관리

URI
http://hdl.handle.net/10203/222427
Link
http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=663211&flag=dissertation
Appears in Collection
CS-Theses_Ph.D.(박사논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0