Mixed Reality

VIROO Studio provides a way to feed Mixed Reality content to HTC Elite XR headset using a new feature of the VIVE Business Streaming (VBS).

Steps to include Mixed Reality in a VIROO Scene

1. Configure VIROO Studio Project with Packages from HTC.
2. Add Mixed Reality Components to allow PassThrough Content.
3. Before publishing add references to Assembly Definition Files for HTC packages.

Configure VIROO Studio Project with Packages from HTC

Involves two sub-steps:

• Project Settings / Package Manager: Add scoped registry providing HTC information.

• Install packages from HTC through Package Manager

Add Mixed reality components

To generate a Mixed Reality content, we are going to make a composition of our 3D content over the Passthrough image provided by the Headset cameras. Our 3D generated frame buffer will be multiplied with the camera image using Alpha channel. The image from the cameras will be visible in areas where our Unity generated 3D frame buffer is transparent, and it will be hidden by 3D objects where they are opaque.

So the trick is to use a camera which renders a transparent background using Full Black and Transparent Color (0,0,0,0) as Clearing method (instead of a Skybox).

MR_Passthrough.cs component (listed below) provides a simple implementation to allow switching between Mixed and Virtual Reality with three methods.

Drag it to any GameObject to be present in the scene, and then:

• Make a call to ToggleMixedReality() to switch between mode.
• Make a call to SetMixedReality() to enter MR.
• Make a call to SetFullImmersiveVR() to enter MR.

Note: This script looks for GameObjects in the Scene tagged with the HideInMR Component, those decorative GameObjects containing HideInMR (like walls, ceiling, ...) will be deactivated automatically in Mixed Reality. This is useful to allow the same VIROO Scene to be enjoyed simultaneously in both Mixed Reality and Virtual Reality for several users, allowing them to decide the render mode in realtime.

HideInMR.cs Code listing.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class HideInMR : MonoBehaviour{}

In the example code provided there are public methods to enable/disable the MR or toggle between the two. You can call these methods from VIROO Actions.

In the example below it is being added to a menu with a RayInteraction and UnityEventNonBroadcastAction, since this should only happen on the local user's instance, but you could use other Actions or your custom code.

MR_Passthrough.cs Code listing.

using System.Collections;
using UnityEngine;
using UnityEngine.Rendering.PostProcessing;
using Viroo.Interaction.XRInteractionToolkit;
using Wave.Native;

public class MR_Passthrough : MonoBehaviour
{
    public bool inMR = false;

    private Camera cam;
    private Color savedColor;
    private CameraClearFlags savedClearFlags;

    // Toggle between modes
    public void ToggleMixedReality()
    {
        if (inMR)
        {
            SetFullImmersiveVR();
        }
        else
        {
            SetMixedReality();
        }
    }

    public void SetMixedReality()
    {
        ShowPassthroughUnderlay();
    }

    public void SetFullImmersiveVR()
    {
        HidePassthroughUnderlay();
    }

    // Unity Start Event Initialization
    IEnumerator Start()
    {
        cam = Camera.main;
        savedColor = cam.backgroundColor;
        savedClearFlags = cam.clearFlags;

        yield return null;

        if (inMR)
            ShowPassthroughUnderlay();
    }

    // internal private implementations

    void ShowPassthroughUnderlay()
    {
        cam.clearFlags = CameraClearFlags.SolidColor;
        cam.backgroundColor = Color.white * 0;

        // HTC provided methods to handle Passthrough functionality
        Interop.WVR_SetPassthroughImageQuality(WVR_PassthroughImageQuality.PerformanceMode);
        Interop.WVR_ShowPassthroughUnderlay(true);

        HideInMR[] hideInMRs = GameObject.FindObjectsOfType<HideInMR>(true);
        foreach (var hideInMR in hideInMRs)
        {
            hideInMR.gameObject.SetActive(false);
        }

        inMR = true;
    }

    void HidePassthroughUnderlay()
    {
        cam.clearFlags = savedClearFlags;
        cam.backgroundColor = savedColor;

        Interop.WVR_ShowPassthroughUnderlay(false); //Hide Passthrough Underlay

        HideInMR[] hideInMRs = GameObject.FindObjectsOfType<HideInMR>(true);
        foreach (var hideInMR in hideInMRs)
        {
            hideInMR.gameObject.SetActive(true);
        }
        inMR = false;
    }
}

Include References to Assembly definitions provided by HTC

In the VIROO Scene Publisher Window:

1. The first assembly definition includes MR components described previously, and other VIROO custom code for the Scene.
2. Wave.Native and Wave.XRSDK provide access to internal Interop functions to allow Passthrough Management through VBS, and therefore they must be included.