We have developed a singing synthesis system named VocaListener that automatically estimates parameters (pitch and dynamics) for singing synthesis by mimicking a user's singing voice with the help of song lyrics. Since a natural voice is provided by the user, the synthesized singing voice mimicking it can be human-like and natural without time-consuming manual adjustments.

VocaListener iteratively estimates singing synthesis parameters so that the synthesized singing can become more similar to the user's singing in terms of pitch and dynamics. The iterative estimation provides robustness with respect to different singing synthesis systems and their singer databases. Moreover, VocaListener has a highly accurate lyrics-to-singing synchronization function, and we also provide an interface that lets a user easily correct synchronization errors just by pointing them out. In addition, VocaListener also has a function to improve synthesized singing as if the user's singing skills were improved.

Demonstration videos including examples of synthesized singing are available at http://staff.aist.go.jp/t.nakano/VocaListener/.