Efficient task scheduling policy for heterogeneous edge computing

Abstract

With the proliferation of mobile platforms with constrained energy requirements, mobile asymmetricmulti-cores have emerged to provide both performance for interactive mobile applications with big cores,and energy reduction with small cores. Furthermore, mobile platforms are commonly equipped withheterogeneous computing processors such as GPU, DSP, and other accelerators for better computationof specific algorithms. This dissertation explores three critical aspects of asymmetric mobile systems,asymmetric hardware platform, application behavior, and the impact of scheduling and power manage-ment. This study shows that the current mobile applications are not fully utilizing the asymmetricmulti-cores due to the lack of TLP and low computational requirement for big cores. With the proliferation of applications with machine learning (ML), the importance of edge platformshas been growing to process streaming sensor data locally without resorting to remote servers. However,since an edge platform must perform the processing of multiple machine learning models concurrentlyfor various sensor data, its scheduling problem poses a new challenge to map heterogeneous machinelearning computation to heterogeneous computing processors. Furthermore, processing of each inputmust provide a certain level of bounded response latency, making the scheduling decision critical forthe edge platform. This dissertation proposes a new heterogeneity-aware ML inference scheduler foredge platforms. Based on the regularity of computation in common ML tasks, the scheduler uses thepre-profiled behavior of each ML model, and routes requests to the most appropriate processors. It alsoreduces the energy consumption while satisfying the service-level objective (SLO) requirement for eachrequest. For such SLO supports, the challenge of ML computation on GPUs and DSP is its inflexiblepreemption capability. To avoid the delay caused by a long task, the proposed scheduler decomposes alarge ML task to sub-tasks by its layer in the DNN model. Finally, as more sensors are equipped with edge platforms, the number of requests to be processedis increasing. Due to the limitations of the computing resources on the edge platform, all requests maynot be able to be processed. One of the ways to mitigate this is cloud offloading, which is a technologyto transfer from edge platform to resource-rich cloud platform. This dissertation extends the proposededge schedulers by combining cloud offloading. Proposed edge-cloud scheduler shows better performanceand SLO satisfaction than proposed edge-only schedulers.