Development of a piano playing robot with motion-sound mapping

Abstract

This thesis presents the development of a piano playing robot with motion-sound mapping in order to provide the elderly a means of entertainment that can interact with them while it is in operation. A piano playing robot composed of a personal computer, a robotic arm, and a robotic hand is first developed to make a system that can play piano mechanically. Kinematic, dynamic equations, and Jacobian matrix for the robotic arm and the robotic finger are then built for realtime control and simulation of the system. A study on the actions underlying sound generation in acoustic piano is carried out to have an insight into the relations between piano sound and keys’ motion. These relations are later represented by neural networks. By using these relations, the piano-playing robot can produce fascinating music as a result of mimicking sample music. Experiment results demonstrate that the developed system outperforms existing systems in the sense of music attractiveness and make it a candidate to be used as a service robot in entertaining human being.