Muscle-driven motion path planning using reinforcement learning

Abstract

In this paper, we introduce a novel motion planning method which is able to dynamically drive motions of a simple articulated body via reinforcement learning, especially Q-Learning. Given a goal position of operation space of the articulated body, static poses which satisfy the target have been readily calculated through the conventional inverse kinematics(IK) system. IK system has been also used to make a character pose in conjunction with keyframing by animators. In contrast, we explore how to automatically generate motion paths of operation space from a given initial position to a goal position without tedious keyframing tasks in a lot of frames or any motion capture data. In order to solve this problem, we apply a simple set of muscles into the articulated body and measure the rate of metabolic energy expenditure and comfort level of each muscle based on Hill`s muscle model. These two terms are crucially used to determine the optimal value functions of Q-Learning. As a result, we strongly believe that the result paths of operation space based on completely updated value functions can be regarded as physically-reliable motion paths since it is guided by Hill`s muscle model with physiologically-meaningful properties

the rate of metabolic energy expenditure and comfort level.