Immersive audio formats are offering us many new creative opportunities and tools as they simultaneously become more and more realistic. However, when mixing audio for multi-channel loudspeaker setups with height channels, traditional workflows using mixing desks with pan pots and joysticks often hinder us from having an intuitive production process. In this article, we want to show you how dearVR SPATIAL CONNECT helps you overcome these technological barriers by enabling you to intuitively mix audio for immersive linear and interactive audio formats.
Overview
In its essence, dearVR SPTAIAL CONNECT is a standalone application which enables you to directly control your DAW in a VR environment. Whether you want to deliver a binaural, an Ambisonics or a 7.1.4 mix, you can see and control your audio tracks within a VR headset. With dearVR SPATIAL CONNECT and a direct visual representation of your audio session, your workflow will speed up and you will feel more empowered to work creatively within immersive audio formats.
Use the VR controllers in your hand to control the spatializer plugins dearVR PRO2, dearVR AMBI MICRO, or the Nuendo VST MultiPanner. That way you can easily position audio objects in a 360° space and even draw complex automation lines. The virtual faders and the meter bridge allow smooth and individual levelling while the direct access to your DAW controls, like solo and mute, as well as transport and marker functions, enable a faster workflow.
dearVR SPATIAL CONNECT is a standalone application housing the VR control interface. Here, you can insert your 2D or 360° video or even decide to work without any video and use a virtual studio environment. dearVR SPATIAL CONNECT is designed to run alongside your DAW session and supports Steinberg’s Nuendo and Reaper.
Machine setup
dearVR SPATIAL CONNECT canals run on the same Windows machine which hosts your DAW.
Alternately, the application can be installed on a separate VR-ready computer. In this kind of two-machine setup, the DAW runs either on a Windows or a Mac machine and both computers are connected by a common network.
Headtracking
Using a VR headset offers immediate headtracking. Headtracking enables us to integrate the listener’s movements into binaural rendering in real-time. This results in a steady soundfield, which reflects a real world acoustic environment and contains enhanced externalization.
dearVR SPATIAL CONNECT supplies the DAW session with the head-tracking data derived from the VR headset. When you turn your head 180°, sound sources previously located in the front will now be perceived from behind. Headtracking is available for DAW tracks that hold a dearVR PRO2 or dearVR AMBI MICRO instance or for the NUENDO VST MultiPanner.
View modes
In addition to the visual representation in the VR headset, dearVR SPATIAL CONNECT can be operated on any common display in 3 different view modes:
VR-Mode: This mode presents the same field of view which is also shown in the connected VR hardware.
3D-Mode: In this mode, a cropped field of view of the 360° video is translated to a 2D screen. It enables you to change the field of view to the desired angle by clicking and dragging the mouse.
2D-Mode: This mode displays the 360°-video in its equirectangular view, essentially squashing the whole 360° field of view into a 2D representation. The 2D view enables you to use dearVR SPATIAL CONNECT even without a VR-headset. In this mode, you can use your mouse to directly position the audio sources without any connected VR goggles connected to dearVR SPATIAL CONNECT.
As already mentioned, you can also use dearVR SPATIAL CONNECT alongside 2D video content. This view-mode comes in handy if you want to judge your 3D-audio mix next to a 2D movie while still taking advantage of the whole 360° sphere for your audio mixing. You can also use dearVR SPATIAL CONNECT without any video content, thus simulating a virtual environment like a cinema or a high-end studio.
Representation
In dearVR SPATIAL CONNECT you can see both spatialized 3D dearVR sources as well as non-dearVR sources of your DAW session. Both sources are represented by visible objects, which can be grabbed, repositioned, and automated.
dearVR Sources
dearVR sources contain an instance of dearVR PRO2 in their Fx-slot. Therefore, the position in the 3D space will automatically be translated into the dearVR PRO2 rendering engine.
Non-dearVR sources
Non-dearVR sources contain no instance of dearVR PRO in their Fx-slots. Thus, the position in a 3D space will automatically be translated into the stereo panning of the audio track.
VR modules
When wearing the VR headset, different modules are available as interactive mixing panels. These modules provide you with the enhanced functionality of the VR-mixing workflow. You can individually position all these modules by simply grabbing and dragging them to your desired position.
Transport module
The transport module is the main module of dearVR SPATIAL CONNECT and is always visible in the VR space. It gives you the essential transport functions known from your DAW. From here, you can also activate all the other VR modules.
Minimap
As the mixing session grows in complexity, you might want to get yourself an overview of all our audio sources. This is where the Minimap comes into play. The Minimap is an interactive miniature view of the whole mixing session, which reflects the surrounding VR space like the 360° video, audio sources and the automation in real time.
Source List
The source list provides an overview of all the important data of your session within VR. It is divided into DAW-related parameters (left side) and 3D audio AAX/VST/AU spatializer plugin dearVR PRO2 (right side). You can also access basic DAW channel functions like solo and mute status, the level of each audio source, and the automation mode. In addition, within this view, you can control the most used features of dearVR PRO like Azimuth, elevation, distance as well as the selected room preset.
Meter bridge
The meter bridge brings the traditional mixing console to the VR world. You can directly monitor and level your audio sources from here. In addition, you can change the levels by selecting the channels and moving your hand up or down.
Controllers
dearVR SPATIAL CONNECT can be controlled by the HTC VIVE, Oculus Rift and Rift S. The controllers can switch into different modes, each of which changes the functionality of the controller. Based on the selected function, you can control the position of an object, change its level, solo and mute objects, and change the individual automation status.
To sum things up, we showed you different ways to integrate dearVR SPATIAL CONNECT into your mixing environment. dearVR SPATIAL CONNECT enables you to directly control the 3D audio spatializer dearVR PRO2, dearVR AMBI MICRO as well as the most important DAW features. In addition to that, it enables you to change the position of audio sources, draw automation data, insert markers, and access the transport control while staying in VR. With dearVR SPATIAL CONNECT you can also speed up your immersive audio production workflow and achieve accurate and creative results.