Inside the Orchestra with Virtual Reality

Have you ever wondered what it would be like to follow a concert from the point of view of a violinist in an orchestra? Or from that of someone who plays the timpani, the cello, the flute or the horn. And from that of the conductor? The enjoyment and listening to music changes, and not by a little. The experience can be done “live” thanks to the beautiful initiative “Dentro l'Orchestra” by the Orchestra Milano Classica , which stages some concerts in which the audience can physically sit next to the conductors, or, more simply, with SOAVE – Stage Orchestra Audio Video Experience , an immersive experience in virtual reality that allows you to experience a symphonic concert directly from inside the orchestra , through the eyes and ears of its protagonists. Soave, a project funded by the Consorzio Meditech, is the result of the collaboration between ETT, a company that deals with technological innovation and digital communication part of Dedagroup, the Haydn Foundation of Trento and Bolzano and E-Volution.

Thanks to the creation of seven simultaneous audio-video recordings, the audience can experience the concert from different perspectives – from the conductor's podium to the string and wind sections, to the other instruments – immersing themselves, thanks to Virtual Reality viewers, in a totally immersive sound and visual environment. The 360-degree video footage and the audio that changes according to the position chosen within the orchestra, allow you to capture every detail, nuance and musical vibration, as if you were really part of the ensemble . orchestral.

Giovanni Verreschi, CEO of ETT said: “Soave combines art and technology, artistic sensitivity and digital innovation, with the aim of making symphonic music more accessible and immersive. We worked on the project thinking of a heterogeneous audience and the activity of conservatories, museums, theaters, experimentation centers and schools, active parts of the changes taking place in the cultural landscape” .

Equipped with a VR headset, users can freely choose from which position within the orchestra to experience the concert, perceiving in real time the acoustic (the sound reproduction is excellent) and visual changes generated by the perspective explored from time to time. The experience is possible thanks to advanced technologies developed by ETT , including a spatial audio engine and multilingual content, designed for versatile use in museum and educational contexts or in spaces dedicated to artistic performance. When the user changes position, the VR headsets automatically update the sound and video source, restarting the music from where it was interrupted.

In addition to the VR headset, the experience is also available in synchronous streaming and on the dedicated website .

A prototype of a physical immersive setup has also been designed, allowing the audience to physically move around a concert hall equipped with multi-projections and directional speakers. The system has been validated through the use of a digital twin, a technique with which the physical setup is digitally simulated to test its scenic and immersive effectiveness quickly and at low cost.