A Platform that Can Control Various Devices in Synchronization with Music Has Been Opened to the Public

– “Songle Sync” enables large-scale, music-driven multimedia performance via the Internet –

Researchers: Masataka Goto, Prime Senior Researcher, Takahiro Inoue, Technical Staff, Masayasu Ogata, Researcher, and Jun Kato, Researcher, the Information Technology Research Institute


The researchers have developed a production-ready platform that can control mobile devices, PCs, and the Internet of Things (IoT) devices, enabling a bring-your-own-device (BYOD) experience of music-driven multimedia performance via the Internet.

Figure 1
“Songle Sync” a platform that can control hundreds of a variety of devices synchronized with music playback

Figure 2
How to use Songle Sync


Music videos with synchronized sound and video have become commonplace, and a variety of visual effects synchronized with music playback in concert venues are now a frequent sight. However, such music-driven multimedia performances conventionally require dedicated devices or applications that were developed individually, resulting in poor expandability and high costs. Because of this, there has been strong demand to develop technology that enables easy and inexpensive development of music-driven applications, using a lot of various devices connected to the Internet.

Method developed by the AIST researchers

The researchers developed a large-scale web-based platform named “Songle Sync ( https://api.songle.jp/sync ),” which enables programmers to control hundreds of a variety of devices in synchronization with music playback. Using the results of analysis with the previously developed “Songle ( https://songle.jp ),” the researchers developed a technology that can drive multimedia effects in time with music playback by synchronizing smartphones, PCs, and other Internet-connected devices via intermittent compact communications with a Songle Sync server. In addition, they implemented the platform on top of web standard technologies so that a variety of devices can be controlled in a unified way, thereby improving scalability and reducing cost.

Future Research Plans

AIST plans to use this “Internet of Musical Things (IoMT)” technology that controls many mobile devices, PCs, and IoT devices simultaneously in various scenes including live events, shopping malls, and city streets, in cooperation with industry.

▲ ページトップへ