Emotional singing voice synthesis by changing duration, vibrato and timbre

Abstract

In this thesis, a novel emotional singing voice synthesis system is considered. There were various approaches to express emotion between human and machine or robot through varying facial expression, action and synthesized speech of a robot. Although singing is known as an effective way for expressing emotion, there is no research using singing to express emotion. To synthesize the singing voice with emotion, the statistical parametric synthesis system is used. The statistical parametric synthesis system uses a singing database which is composed of various melodies sung neutrally with restricted set of words and hidden semi-Markov models (HSMMs) of notes ranging from G3 to E5 to construct statistical information. The procedure of statistical parametric synthesis system is composed of mainly two parts, training and synthesis. In training part, both spectrum and excitation parameter are extracted from a singing database, and the statistical information of spectrum and excitation parameter for each note is constructed. Three steps are taken in the synthesis part: (1) Pitch and duration are determined according to the notes indicated by the musical score; (2) Features are sampled from appropriate HSMMs with the duration set to the maximum probability; (3) Singing voice is synthesized by the mel-log spectrum approximation (MLSA) filter using the sampled features as parameters of the filter. Emotion of a synthesized song is controlled by varying the duration, the vibrato parameters and the timbre according to the Thayer`s mood model which defines emotions in tense and energy axis. Perception test is performed to evaluate the synthesized song. The results show that the algorithm can control the expressed emotion of a singing voice given a neutral singing database.