Vol.4 No.2 2011

Research paper : Toward the use of humanoid robots as assemblies of content technologies (S. Nakaoka et al.)−93−Synthesiology - English edition Vol.4 No.2 (2011) the framework, and resulted in the integrated software ChoreonoidNote 3).The Choreonoid framework is based on the C++ language, is compatible with programs written in C or C++, and is capable of realizing the algorithms and interface that require high-speed processing. Also, it is designed based on the architecture called Model View Controller and the signal mechanism, and while keeping the maintainability and expandability by increasing the independency of the objects, it is possible to execute complex associations between the objects[29]. New functions can be added as plug-ins and the new plug-ins can easily collaborate with the existing functions and other plug-in functions. Due to such characteristics, the Choreonoid surpasses the framework of the content technology, and is expected to be used widely as an environment to develop higher layer robot software.7 Experiment in content productionIn this research, to promote the technology, it was necessary to actually produce the contents using the Choreonoid and HRP-4C, and to verify and improve the technology. Considering the objective of this study, it was important to have the content created by professional creators rather than by us. By doing so, the practicality of the system could be verified, and new contents could be pioneered.As a joint activity with Dr. Masaru Ishikawa, Project Researcher of the Information and Robot Technology Research Initiative, The University of Tokyo, we asked Mr. SAM, a famous dancer and dance creator, to create an entertainment content using HRP-4C. This was supported by the DC-EXPO (DIGITAL CONTENT EXPO)Note 4) 2009 and DC-EXPO 2010, and the resulting performance was presented at the exposition. We shall now describe the efforts at the DC-EXPO 2010 that was done after the development of Choreonoid.At the DC-EXPO 2010, to fully verify the technologies described in this paper, we tried to develop a content where the HRP-4C would sing and dance, as shown in Fig. 7. In this content, HRP-4C gave a three-minute demonstration of a dance, choreographed by Mr. SAM, and a song “Deatta Koro No Youni (Like When We First Met)” by Every Little Thing, a Japanese music group.The dance motion was created entirely by Choreonoid based on Mr. SAM’s choreography. We told Mr. SAM the robot’s motion capacity, and then had him choreograph freely within the possible motion range. The dance motion included various poses and actions using the entire body as shown in Fig. 8, and the richness of expression was unseen in any previous performance by a robot. We changed the wig of HRP-4C and dressed HRP-4C in a costume to match the dance, and the stage lighting was also carefully planned. Mr. SAM’s choreography included the total combination performance with four human back dancers.For singing, we asked the cooperation of YAMAHA Corporation, and obtained the voice data and lip synch data through the VOCALOID. The VOCALOID sound source was CV-4C of the Crypton Future Media, Inc. Using the VocaListener technology of Nakano and Goto[30], tuning was based on the song by Ms. Kaori Mochida, the singer of the original song (voice track of the vocal part by Ms. Kaori Mochida supplied by avex trax). This ensured expressive singing close to that of a human being. The lip synch data was produced automatically and was created efficiently from the voice data by using the VOCALOID linkage function explained in chapter 5.This dance demonstration generated news mainly on the Internet since it was presented on October 16, 2010. As an example for its newsworthiness, the movies uploaded to YouTube by some audiences recorded over 2 million view counts and over 1,500 comments in 10 days after the presentation. One of the movies had the 6th highest view counts for YouTube Japan in the year 2010. Immediately after Fig. 7 Demonstration of singing and dancing by HRP-4CFig. 8 Example of various movements and poses actually done in the dance


page 22