Runtime job scheduling for heterogeneous devices

Abstract

The utilization of GPUs is increasing as the use of various image processing and machine learning-based programs requiring large-capacity data operations increases. In the case of embedded systems, they are also used in various fields such as smartphones, autonomous driving systems, AI cameras, and smart home appliances. High-level libraries that support APIs that can utilize GPUs are also being developed simultaneously, but they do not support both chips and operating systems of embedded systems. Furthermore, the performance of GPU devices mounted in embedded system environments has not yet shown an extremely improved computational speed compared to CPUs. In particular, computational device performance in practical embedded system environments is poor considering the cost. We seek to address these issues by leveraging OpenCL, a library capable of applying the same parallel programming code to different types of devices. In this dissertation, we propose a HybridCL framework developed by hooking APIs in the OpenCL library. HybridCL supports the ability to select devices based on the status of real-time devices, as well as to split a task into multiple subtasks to distribute and run simultaneously on each device. HybridCL avoids bottlenecks that can occur in real time and ensures the performance of various tasks by leveraging surplus resources while balancing the load concentrated on a particular computational device. It also achieved performance improvements in existing kernel operations through concurrent execution capabilities.