(A) study on deep learning based automated aerial docking system

Abstract

In this study, an autonomous docking system for unmanned aerial vehicle (UAV) is proposed. The proposed autonomous docking system consists of two subsystems: docking mechanical system and deep learning based detection/tracking system. The fundamental challenges during an air-to-air docking phase are a robust securing between a leader and a follower aerial vehicles and effective target detection and tracking strategies. To confront those issues, this study not only presents the design of a robust aerial docking mechanism, but also proposes the effective deep learning based target detection and tracking algorithms. The design of the proposed docking mechanism is based on bi-stable characteristic for the robust mid-air docking. The proposed deep learning based target detection and tracking algorithms are developed for an onboard machine learning computer platform. For the reliable docking under various flight environments, a bi-stable characteristic based probe-and-drogue type mechanism was designed. The bi-stable characteristic of a docking mechanical system increases the transition time between two stable states. It makes that the proposed docking mechanism can lock/dock the probe with a low contact force. The drogue in the proposed mechanical system acts by itself to lock/dock the probe. To release the locked/docked probe, a shape memory alloy (SMA) spring is used for low-power consumption and simplicity. The proposed docking mechanism was built by a 3D printer with PLA material. The docking and releasing performance of the proposed docking mechanism had been intensively tested on ground using a specially devised robot arms and test rigs. To avoid the video latency and the excessive data transmission delay between a UAV in the air and a ground control station (GCS), a small and powerful AI performance onboard computer module was installed on the UAV. The proposed deep learning based detection/tracking system was developed to be operated by the onboard computer module. In order to overcome the critical challenges of the conventional target detection methods in the air and to achieve the robust drogue detection, deep convolution neural network (CNN) based single-stage detector algorithm, YOLOv4 tiny, was applied. To track and measure the 3D coordinates of the drogue, RGB-D camera is used and a point-cloud based algorithm is implemented. The implemented point-cloud based tracking algorithm measures real-time 3D coordinates (X, Y, Z) of the center point of the bounding box. The measured 3D coordinates are set as the target location for docking and fed to the designed PD controller. A hardware-in-the-loop testing was conducted for the performance validation before the mid-air docking test. For the ground test, the specially devised robot arms were used to simulate a leader and a follower aerial vehicles. After the ground test, the proposed autonomous docking system was integrated into a quadcopter drone hardware for the performance test. Autonomous mid-air detection and tracking tests were conducted in GPS-denied environment. To conduct indoor flight test, a visual inertial odometry (VIO) system was installed on the quadcopter drone.