SLO-Aware Inference Scheduler for Heterogeneous Processors in Edge Platforms

Cited 16 time in webofscience Cited 0 time in scopus
  • Hit : 500
  • Download : 137
DC FieldValueLanguage
dc.contributor.authorSeo, Wonikko
dc.contributor.authorKim, Yeonjaeko
dc.contributor.authorCha, Sanghoonko
dc.contributor.authorHuh, Jaehyukko
dc.contributor.authorPark, Jongseko
dc.date.accessioned2021-08-10T00:50:10Z-
dc.date.available2021-08-10T00:50:10Z-
dc.date.created2021-06-09-
dc.date.created2021-06-09-
dc.date.created2021-06-09-
dc.date.created2021-06-09-
dc.date.issued2021-07-
dc.identifier.citationACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION, v.18, no.4, pp.1 - 26-
dc.identifier.issn1544-3566-
dc.identifier.urihttp://hdl.handle.net/10203/287096-
dc.description.abstractWith the proliferation of applications with machine learning (ML), the importance of edge platforms has been growing to process streaming sensor, data locally without resorting to remote servers. Such edge platforms are commonly equipped with heterogeneous computing processors such as GPU, DSP, and other accelerators, but their computational and energy budget are severely constrained compared to the data center servers. However, as an edge platform must perform the processing of multiple machine learning models concurrently for multimodal sensor data, its scheduling problem poses a new challenge to map heterogeneous machine learning computation to heterogeneous computing processors. Furthermore, processing of each input must provide a certain level of bounded response latency, making the scheduling decision critical for the edge platform. This article proposes a set of new heterogeneity-aware ML inference scheduling policies for edge platforms. Based on the regularity of computation in common ML tasks, the scheduler uses the pre-profiled behavior of each ML model and routes requests to the most appropriate processors. It also aims to satisfy the service-level objective (SLO) requirement while reducing the energy consumption for each request. For such SLO supports, the challenge of ML computation on GPUs and DSP is its inflexible preemption capability. To avoid the delay caused by a long task, the proposed scheduler decomposes a large ML task to sub-tasks by its layer in the DNN model.-
dc.languageEnglish-
dc.publisherASSOC COMPUTING MACHINERY-
dc.titleSLO-Aware Inference Scheduler for Heterogeneous Processors in Edge Platforms-
dc.typeArticle-
dc.identifier.wosid000703450700004-
dc.identifier.scopusid2-s2.0-85116413818-
dc.type.rimsART-
dc.citation.volume18-
dc.citation.issue4-
dc.citation.beginningpage1-
dc.citation.endingpage26-
dc.citation.publicationnameACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION-
dc.identifier.doi10.1145/3460352-
dc.contributor.localauthorHuh, Jaehyuk-
dc.contributor.localauthorPark, Jongse-
dc.description.isOpenAccessY-
dc.type.journalArticleArticle-
dc.subject.keywordAuthorEdge computing-
dc.subject.keywordAuthorheterogeneous computing-
dc.subject.keywordAuthormachine learning-
dc.subject.keywordAuthorinference-
dc.subject.keywordAuthortask scheduling-
dc.subject.keywordPlusSMART CITIES-
dc.subject.keywordPlusBIG DATA-
dc.subject.keywordPlusINTELLIGENCE-
dc.subject.keywordPlusNETWORKS-
Appears in Collection
CS-Journal Papers(저널논문)
Files in This Item
120227.pdf(1.74 MB)Download
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 16 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0