Landing posture control of a robotic cat using fast converging reinforcement learning

Abstract

In this thesis, reinforcement learning controller for a robotic cat problem is introduced. When a cat is dropped in an upside down posture, it can rotate its body to land on its feet. Basically, this research is inspired from the posture control ability of cats. Robot`s posture control ability in the air is required when designing advanced robots that can run, jump and land, which can perform tasks in workplaces where ordinary robots cannot go. Also, when space robots operate in the space, it is hard for them to rotate the desired direction because there is no external forces. if the space robot has cat`s rotating ability, it can easily rotate its orientation using internal motion. In addition, the principle of a falling cat can be used in dextrous manipulation by multi-fingered robotic hands and path planning for mobile robots subject to nonholonomic constraints. In the previous research, they considered only the simple shape of robot, which consists of only two symmetric rigid bodies that enables easy dynamics calculation due to cancelations of many terms in the equation. Although their approaches provide accurate controller, it works only for a specific initial condition. However with proposed method, once algorithm is designed it can be applied to any kind of robot structures. Actually the complex asymmetric robot structure is used throughout the research. In addition, the learned controller can find solution with a random initial condition. The controller is based on gradient decent Sarsa($\lambda$), and Selective Experience Replay (SER) is added to it. By adding SER, the algorithm converges faster than the plain Sarsa($\lambda$). For simulation, the robotic cat simulator is developed, and all of these works are verified through the simulator.