Stereo matching technique for large-scale ship block using Guided Mask in Keypoint-based ROI

Abstract

The 3D precision measurement technology plays a crucial role in the quality control of products in the shipbuilding and manufacturing industries. In practical applications, Time-of-Flight optical measurement systems are commonly used for precise measurement of large ship blocks, but they are cumbersome and time-consuming. To address this challenge, research has been conducted on the 3D reconstruction of ship blocks using stereo cameras. Calibration is performed to determine camera parameters, and 3D shapes are reconstructed. However, the large size of ship blocks leads to significant perspective differences between images, making it difficult to apply conventional stereo matching techniques. Consequently, manual selection of measurement points in each image becomes necessary. In this study, we propose a novel stereo matching technique to address this issue and improve the identification of measurement points on captured ship blocks. Initially, a dataset of stereo images of ship blocks is constructed through various experiments. Subsequently, a feature matching network-based clustering process is applied to stereo images and the coordinates of query points. This process identifies keypoint clusters around query points and their centroids. Using these centroids as reference points, regions of interest (ROI) are obtained and preprocessed through co-visible region segmentation. A transformer-based algorithm predicts corresponding points on the target image, and their coordinates are obtained. Finally, these coordinates undergo a refinement process based on epipolar geometry. By utilizing this stereo matching technique, 3D coordinates for arbitrary points on the images can be obtained. This innovative approach is expected to overcome the inefficiencies of conventional measurement point identification, representing a groundbreaking technology in the field of stereo matching under wide baseline conditions.