AIST Stories No2

26From AIST to the Innovative Worldtechnologies were not enough to overcome the difficulties of understanding music. Toward achieving automatic music-understanding technologies, Goto and his colleagues have been researching and developing a wide range of core technologies for over 20 years and have become world leaders in the field.A web service "Songle" (launched in 2012) enables anyone to easily try some of such technologies. Songle is distinct from ordinary music players in that a user can view the entire structure of a song as a music map that is visualized by using music-understanding technologies. A user can jump and listen to the chorus with just a push of a button, which is useful not only for trial listening but also for comparison between repeated choruses.Aggregating the power of the crowd to improve usefulnessSongle has applied music-understanding technologies to over 700,000 songs that have already been open to the public on the Internet, and their results can be enjoyed by everyone in the form of music maps. There are some errors in the music maps, but this is to be expected. The reality is that AIST is conducting an important experiment in how to make imperfect technologies useful for society. Songle is provided with a function that enables anyone who can find and fix errors to contribute by making corrections. In this way, Songle can make use of a synergy between the capabilities of computers with their great processing power but imperfect understanding and humans with their ability to spot and fix errors. In other words, we are studying systems of harmonious cooperation between computers and humans that will be critical to society in the future.Another web service "Songrium" (launched in 2013) uses the automatic music-understanding results from Songle and enables a user to understand relationships between large numbers of songs from a variety of viewpoints. In Songrium, a user can also skip ahead to the chorus and can easily find songs with similar moods to their favorite songs, videos in which people are singing and dancing along with those favorite songs, and the like. In an era in which new original songs are first released on the Internet, such new Music-understanding technologies to realize the full potential of digital musicProposing New Ways to Enjoy Music for Supporting the Digital Content IndustryEngendering a "Content-Symbiotic Society" in which content can be created and published without hesitationMedia Interaction Group, Information Technology Research InstituteMaking computers "understand" musicMasataka Goto, a Prime Senior Researcher of the Information Technology Research Institute, knows that music-understanding technologies are necessary. He explains: "When I was shopping for CDs to encounter unfamiliar but interesting music in my twenties, I would go to a trial-listening corner and repeatedly press the fast-forward button to listen to chorus sections (“hook”) of a song. Since then, digital music has become the norm and, realizing that I would not be able to hear everything even if I spent my whole life listening to music, I decided that it would be helpful if computers could do the listening for me." From this point, he started on a quest to work out how computers could be made to understand music in a way that is useful for people.When humans listen to music, they do things like humming along to the melody, clapping in time with the beat, and instantly identifying the chorus. Although these things are easy for humans, it is difficult to implement automatic understanding of music by a computer if its mechanism is not clear. Since music is one of the most complicated signals with multiple sounds being mixed together, existing Music has been digitized and it is now possible to access a huge number of musical pieces. Technologies by which computers can automatically analyze and understand this vast body of music will make daily appreciation of music more enjoyable and enable anyone to easily create music. This change could lead to a drastic change of the culture around music. AIST's music-understanding technologies will open up new ways to enjoy music in the future and will provide a boost to content industries.AISTsupporting livelihoods! Our life and society will change in this way!