Accelerating supervised explainable object detection system for dl-based satellite imagery analysis

Abstract

Object detection from high-definition satellite imagery can be used in various areas such as military, disaster surveillance, and transportation planning. Due to the recent development of deep learning technology, image analysis performance has surpassed the level of human recognition, and thus the requirement for automatic object detection using deep learning on satellite imagery has increased. However, there are two obstacles when applying object detection schemes to satellite images. First, the object detection performance in terms of mAP is not high when applied to the satellite imagery. Actually, when we apply the SSD-inception-based object detector to xView satellite image dataset, the mAP value is observed as 10-20\%. It means that the object detection scheme is not applicable to satellite imagery. This is because objects in the satellite image have a low resolution to be recognized as satellite images are taken from hundreds and thousands of kilometers, observing thousand $km^2$ area. Therefore, objects in the satellite image are often ambiguous to be classified. Second, object detection in satellite images requires a huge amount of computation so it is hard to deploy real-time service on the satellite. Practically, object detection workload cannot be deployed on satellites as the satellite onboard performance is inferior in terms of performance and power. The object detection workload requires about 3200 GFlops of computation, whereas a typical satellite is only a few hundred GFlops. In addition, it is practically difficult to process data by sending data to the ground as a single general satellite image is about 3 GB. Therefore, it takes a long time to train or infer a large-scale object detection model using satellite data. Thus, the object detection workload needs to be accelerated for reliable service.

In Chapters 1 and 2 of this dissertation, we propose an object detection framework that improves the explainability and accuracy by using eXplainable AI (XAI) techniques. XAI explains the rationale for its decision to the supervisors so the ambiguity problem that occurred in satellite object detection can be alleviated. In addition, to provide a reliable service, the proposed framework should include an acceleration function for fast inferencing and training. Therefore, the proposed framework includes three main functions: (1) a posthoc explainability module that improves the object detection performance of satellite images and provides explanation to the supervisor, (2) model compression module to mount a complex deep learning model on satellite onboard, and (3) accelerating distributed training module for optimized large-scale training on high-performance cluster (HPC) environment.

In Chapter 3, we propose a post-hoc explainability schemes to provide a causal explanation to supervisors so that they can make correct decisions about ambiguous objects on satellite images. With the proposed scheme, AI provides the basis for object detection, and supervisors can make a correct decision based on the explanation. The proposed post-hoc explainability schemes analyzes the detailed object attributes and temporal information from past observation, providing sufficient explanation of decision to supervisors. Experiments showed that when the proposed scheme was applied to the real satellite images, the mAP performance improved by 1.3-3.4\% compared to the SSD-Inception baseline, and the object classification performance improved by about 10\%.

In Chapter 4, we propose a layer-wise channel pruning scheme on satellite onboard considering hardware and network specifications in order to overcome the hardware constraints (memory, power, etc.) of the satellite and to mount the trained model on the satellite onboard. By applying channel pruning, it is possible to deploy complex XAI applications on satellites and significantly reduce computation time. Therefore, considering the characteristics of the deep learning backbone, we propose a technique that determines the optimal pruning degree for each layer. In addition, we propose a technique to minimize the deep learning computation time by offloading the workload from the onboard to the central HPC system. Through experiments, we proved that the inference time can be greatly reduced when the offloading technique is applied, and when the channel pruning for each layer is applied, the inference time is accelerated by 15.64x while maintaining the accuracy of the inception backbone reference. It was confirmed that the memory consumption was reduced by 64\%.

In Chapter 5, we propose an optimized parallelization technique and resource allocation technique to accelerate the XAI application training time in the HPC environment. In general, there are two parallelization methods for distributed training: data and model parallelism. They are complementary to each other so determining the optimal parallelism method for a given situation is a challenging issue. In this dissertation, we propose a group hybrid parallelization(GHP) scheme to improve the shortcomings of the two parallelization methods and accelerate training by reducing the communication time, which is the bottleneck of S-SGD training. Through experiment and simulation, we showed the effectiveness of our proposed model. We observed that the proposed mathematical model accurately predicts the learning time with an average error of 4.7\% when executed in most cases. In addition, through a weak scaling experiment that increases the batch size and the number of workers together, the performance improved about 17 times compared to the case where empirical model parallelism was applied, and about 1.8 times compared to the case where data parallelization was applied.

In conclusion, we proposed a framework that explains the basis of object detection to the supervisors by applying the posthoc explainability model. The proposed framework can explain the satellite object detection to the supervisor by using XAI and thus increase the accuracy and explainability. Also, through the channel pruning scheme on satellite onboard and group hybrid parallelism scheme on HPC environment, we can provide reliable object detection service to supervisors. We expect the proposed system can be applied to monitoring applications for various purposes such as military and disaster.