6D pose and metric scale 3D shape estimation of objects for vision-based robotic grasping

Abstract

Recent advances in deep learning have led to a number of object recognition studies in images, but many problems have not been attempted on how to recognize objects placed in a real 3D world rather than a 2D plane. In this thesis, we propose methods for human-like 3D object pose estimation for two environments. To be specific, We propose the deep learning-based object pose estimation methods considering cases where objects that are already known and have never been seen before but have seen similar classes before. In recent years, methods for estimating the pose of known objects have been proposed. However, existing methods use the global features to estimate the pose, so if an object is obscured by other objects, performance is significantly degraded by estimating the pose using global features that are not related to the objects. To tackle this problem, we propose the pose estimation method using only local features and angle vectors of visible object regions. In particular, since it estimates using only the information most relevant to the pose of the object, it shows accurate pose estimation, unlike other methods. The angular vector we proposed is not affected by the scale information and can focus more on the direction, and is influenced less by the size information than other expressions. Unlike when grasping an object that is already known, to grasp a new object in general, information such as the pose of the object as well as what the metric scale object looks like is additionally required. In other to implement this, we propose the method of simultaneously estimate the pose and metric scale shape of a new object. Our method estimates by considering the shape of the part that is not visible from the sensor.When grasping, a metric scale object is very useful in estimation pose, preventing collision, and finding grasping the points. For this reason, we propose the object shape branch to estimate the metric scale shape effectively. Experiments show that the proposed shape is helpful for the pose of objects.