Model-based optimal hand posture recognition using KINECT sensor

Abstract

An articulated object, such as a human hand, moves in very complicated way and it is difficult to estimate its posture because of its high dimensionality and self-occlusion from a point of view. In this thesis, we use KINECT to observe a human hand. The color image is obtained from the RGB camera and the depth information from the IR sensor. By preprocessing two images from the observed hand and a 3D model, we obtain comparable information and the discrepancy between the two sides is defined to be a cost function of the optimization problem. The object being observed can have any shape of appearance and the system simply outputs the figure which is extracted by evaluating its degree of dissimilarity. Therefore, the system is robust to variation of hand appearances.

The entire process of the proposed system for estimating a hand posture is mainly based on the optimization algorithm finding the best 20 parameters of joint angles of 3D hand model. Since Particle Swarm Optimization algorithm has been proved to be an efficient search of near-optimal solution for high dimensional cost function with multiple local optima in many earlier studies, it is naturally adopted and modified to bring about better accuracy and performance. The inertia of the particles while searching on the global optimum is subject to be controlled by its “Phase Transitional Inertia”. Constriction can guarantee the stability of the PSO. Adaptive Constriction Method is make use of the two benefits together and proved to be positively effective on the accuracy of solutions.

A hand posture rendered with a set of 20 parameters of joint angles is determined with a 20 dimensional position of a particle. Hypothesis Pre-filtering is applied to improve performance and accuracy of the system. By reducing the number of hypotheses to be evaluated with the cost function, the same number of hypotheses can be operated in less time and more generations can be conducted in the same time.