Emerging Technologies

Monsters in the Orchestra: A Surround Computing VR Experience

Monsters in the Orchestra is an interactive and immersive demonstration of surround computing as experienced in a VR environment. Participants are immersed in a stereoscopic world of monsters playing real-world instruments with positional audio and 3D x 360-degree gesture control. The original system, demonstrated at CES 2014, allowed participants to be fully immersed in a 30-foot-diameter dome driven by six HD projectors, creating a 360 degree view, from floor to apex. Real-time positional audio was introduced via hidden 32.4 speakers in three rings at different heights. Audio equalization and reverb were processed using AMD True Audio. The monsters were conducted or controlled by a human conductor using 3D x 360-degree gestures.

For SIGGRAPH 2014 Emerging Technologies, participants are immersed in the same experience using virtual reality headsets such as those from Sony and Oculus, and 3D spatialized audio delivered to each participant using stereo headphones. Unlike the dome experience, participants are able to interact with the monsters via simple gestures of both hands. Additionally, all participants share the same experience (concert) visually and aurally, but with unique perceptions based on their movements and positions. The net effect is that all participants share the same experience as a group as if they were all physically sitting in the same dome.

The performance of Monsters in the Orchestra is based on original visual and musical assets. Seventy-two original instruments are mapped to the 19 monster musicians. The monsters are able to move to any location in 3D space with the audio moving accordingly. Participants control and experience a five-minute concert presented by monsters. They can localize the monsters' positions based on instrument positions, despite using only stereo headphones.

  Download Video

Rémi Arnaud
AMD, Inc.

Emanuel Marquez
AMD, Inc.

Bill Herz
AMD, Inc.