Accelerated edge cloud system for stream data processing with incremental deep learning scheme

Abstract

Big data computing is a technology that enables low-cost, low-risk decision making for data-based industries by applying information embedded in data in real-time to traditional industrial processes through the production, storage, and analysis of increasing data. With the advance of the network infrastructure, smart sensors, and various monitoring technology, a variety of value creation services are available through a decision model based on high-level abstraction with the nonlinear transformation of deep learning. In this dissertation, we study the accelerated computing system considering time-variant properties for accurate energy demand prediction by processing the AMI stream data, which plays an important role in the next-generation energy system. In future urban environments, intelligent surveillance systems can also provide a variety of application services, such as illegal parking detection, by analyzing and processing video streams collected and generated from urban systems and vehicles. Analytic models in which objects are represented in a specific time and space from multiple observation sources have been studied through spatial feature vectors and spatial inverse projection of image objects. However, there are problems in collected multiple big data streams such as changes in data distribution over time, different characteristics of multi-class data, memory constraints and computational inefficiencies due to unbound characteristics of stream data, selection of suitable models based on training/inference performance and difficulty in the scheduling of resource corresponding to computational workload. Therefore, many researches have been studied to resolve these problems. In this dissertation, we proposed an edge cloud acceleration system based on an incremental learning scheme for efficient big data stream processing, a new task scheduling scheme for stream data training and inferencing task. And, we discussed a cloud workflow scheduling scheme and its application for DL acceleration in the edge-cloud environment. Firstly, we proposed an integrated system architecture with a deep learning training acceleration scheme through incremental deep learning of hyperparameter update scheduling and with heterogeneous accelerator (FPGU, GPU) resource scheduling for lightweight data types such as AMIs in edge-based stream processing environments. We resolved the parameter update scheduling problem to reflect the short-term non-stationary AMI data at a low latency while minimizing the decrease in prediction performance resulting from the partial training dataset. In the incremental deep learning, an unbounded data stream was handled as a bounded data stream. We proposed a utility function for adaptive incremental deep learning to improve model accuracy, through the rapid reflection of changes in concept drift, gain learning speed in batch learning. In addition, we proposed a heuristic to quickly find the decision vector that satisfied the optimization of the utility function through multivariate optimization of the utility function, and reduced the overhead in each training iteration. The proposed scheme showed a low error rate through the retraining process and reduced the training cost compared to the periodic retraining method. Also, heterogeneous accelerator (FPGU, GPU) resource scheduling through layer partitioning in the edge cloud showed that the performance of the CNN-LSTM model for AMI data processing could be improved. We implemented an incremental deep learning scheme for processing stream data in Apache Kafka and Spark environments. Secondly, we proposed the science gateway cloud architecture for advanced application processing in a cloud computing environment. In particular, we proposed a workflow scheduling model based on service level agreement and evaluated large scale scientific applications. A heuristic algorithm was proposed to solve the multi-constrained workflow scheduling problem that considers the heterogeneity of computing resources and tasks and resource availability in distributed computing. For a cost-effective scheduling of the deadline constrained workflows, we proposed a cost-effective resource scheduling scheme, which partition workflow based on critical paths, and assign sub-deadlines based on the relative workload between tasks for heterogeneous computing resources. We defined a service level violation cost model and proposed a cost-effective workflow scheduling scheme between resource allocation and service level violation. We also determined the degree of task division due to processing time violations and accelerated the processing through parallel processing of divided tasks. Through the performance evaluation of the proposed workflow scheduling scheme for a representative scientific application service, it showed that the proposed scheme guarantees deadline and reduces processing costs. Lastly, we proposed SIFT-CNN based complex event processing for surveillance/monitoring system for urban safety in smart city, and proposed workflow-based task scheduling scheme with FPGA-GPU resource. In order to interpret the semantic information existing in the image frames, we proposed a method for object re-identification, with feature extraction using SIFT-CNN scheme and the distance model between features collected from multiple videos. Also, by defining the resource scheduling problem in the edge-cloud collaboration structure, we showed that the tasks in object re-identification can be transformed into a workflow form and proposed the processing acceleration method for the transformed workflow. For the convolutional layer, we proposed an FPGA-GPU adaptive scheduling scheme based on the execution time estimation model of layer-specific characteristics of FPGA-GPU resources, for optimizing execution time considering the limited resources in the edge environment. In addition, by applying an adaptive DL pruning policy that increases computational performance by eliminating unnecessary computational parameters of deep learning models in case of processing time violations, a pruning degree determination scheme was integrated with the aforementioned scheduling problem that showed deadlines guarantee while maintaining maximum accuracy. Through the performance evaluation, we experimentally showed the relationship between acceleration and accuracy and show that effectiveness of processing acceleration of the proposed scheduling scheme. In conclusion, for effective processing of increasing data streams, we proposed deep learning acceleration schemes considering different characteristics from sensor stream data to video stream data, and finally, we showed our proposed schemes were outperform for computing acceleration in edge-cloud environments.