Building an XR Application in Unity with MRTK [Part 1] — Install the tools
Tool Requirements
Download the following tools to your development machine. This series assumes the user has a fundamental understanding of the Unity game engine and C# programming language. For building for HoloLens 2, a Windows 10 development machine is required. More information and spec requirements for HoloLens development can be found here.
[UPDATE] This guide has been updated to support Oculus devices. Most of the setup below is still required to get started with Unity & MRTK. For Oculus specific details, please see [Part 8] — Oculus Quest Support
Installing Mixed Reality Toolkit (MRTK)
Additional instructions for setting up the Mixed Reality Toolkit in Unity can be found here: Getting started with MRTK
- Download all .unitypackage files for the latest release (2.5.1 for this article)
- In Unity, right click Assets folder in your Project Window > Import Package > Custom Package… and select each unity package file downloaded, the import
- During import or under Mixed Reality Toolkit tab > Utilities > Configure Unity Project, the MRTK Project Configurator window may/will appear. This utility helps with auto-initializing a Mixed Reality project in Unity. It is generally recommended to enable most if not all settings. This configurator is particularly helpful for Windows Mixed Reality development on the UWP platform.
Furthermore, for HoloLens development, the targeted build platform must be Universal Windows Platform (UWP). Cntrl+shift+B will open the Build Settings window and then ensure the UWP platform is selected and applied. For Oculus Quest development, the targeted build platform will be Android.
Enabling XR SDK in Unity
The XR SDK is the latest edition from Unity for supporting the vast ecosystem of mixed reality platforms and devices. The XR SDK is essentially a suite of plugins provided by the top XR platform companies (ex: Microsoft, Facebook, etc) in conjunction with Unity so that the Unity game engine (Input, Rendering, etc) can run on these platforms/devices. These plugins enable Unity to interact with XR platform providers for tasks such as what device(s) is connected, what resolution is each eye, send/receive input information to controllers, and more which essentially abstracts you as the app developer from learning how to perform these tasks for every platform.
Going forward (i.e Unity 2019+) XR SDK will be the default XR pipeline for mixed reality development with Unity. In this next step, we need to instruct Unity to download all XR SDK packages to support our target platforms for this tutorial series, Windows Mixed Reality and Oculus.
- Go to Edit > Project Settings > XR Plugin Management in Unity
- Click Install XR Plugin Management
- After packages install, for each platform tab, enable the XR devices you wish to support for your application. This tutorial will enable Oculus and Windows Mixed Reality for Standalone, Windows Mixed Reality for UWP, and Oculus for Android.
NOTE: Additional detailed information for setting up Mixed Reality Toolkit with XR SDK from Unity can be found on their online documentation below.
Configuring the root scene
Getting started with MRTK is as simple as a few button clicks. We need to add the core MixedRealityToolkit script component to our scene and then create our own profiles to configure each of the systems provided by MRTK. A configuration profile in MRTK is effectively a ScriptableObject with various properties that act as levers to control how each system should run when initialized. MRTK provides default profiles which we will duplicate as a starting point and then make our own changes against.
- In Unity, under the Mixed Reality Toolkit tab, select Add to Scene and Configure…
- Select the Mixed Reality Toolkit GameObject and view in inspector
- Select the DefaultXRSDKConfigurationProfile at the top of the MixedRealityToolkit component.
- Click the Copy & Customize button which will open a pop-up
- Rename the new profile as desired and click Clone
- Save the scene to your scenes folder
In-Editor Input Simulation
MRTK comes auto-configured with the Input Simulation service to make it easy getting started developing and testing in the editor environment without a device.
After entering play mode with your scene, use WSAD keys to translate the camera and hold the right-mouse button while moving the mouse to rotate the camera.
Hold the space key to bring up the right simulated hand/controller and click the left-mouse button to select. Hold the left shift key to bring up the left simulated hand.
Furthermore, you can lock a simulated hand in the scene by pressing the Y key for the right hand and the T key for the left hand. Hold the space key to still move the right simulated hand but now once the space key is released, the right simulated hand will remain in scene. Press either Y or T key again to toggle the respective hands persistence in scene.
XR Performance
One will notice in the previous image from the Input Simulation section, the green-bar profiler at the bottom. The Diagnostics system built with MRTK is enabled by default. Performance is critical for XR applications. On HoloLens, application should always run at 60 FPS. This ensures a stable experience for the user as head tracking and visual quality will degrade with lower frame-rates.
The profiler though can be toggled on/off at runtime using the speech command “Toggle Profiler”. Furthermore, the entire system can be disabled for release builds using the MRTK profile configuration window. Select the MixedRealityToolkit GameObject in scene and view in the Inspector Window. Select the Diagnostics tab on the left-hand side and toggle Enable Diagnostics System.
Performance Recommendations for Unity
Final settings for the Unity Environment
The majority of environment and setup settings were covered above between the MRTK Project Configurator and the profile configuration. There are a few more items to review before moving on however.
- Open up the Project Settings Window in Unity via Edit tab > Project Settings
- Select the Tags and Layers tab on left-hand side. Set Layer 9 to Target and Layer 10 to Ball. Ensure Layer 31 is Spatial Awareness. These will be utilized in future articles.
- Select the Physics tab and modify the Layer Collision Matrix to match the image below. Effectively, we only want the Ball to be able to collide with anything else.
- Open the Quality tab and ensure all platforms are on Very Low configuration (click the black drop down arrow to change setting). This will ensure the Unity environment favors performance over quality since we want to ensure we hit 60 FPS on device.
- Finally, open the Player tab. Here you can set icon and app name etc.
- Ensure the Scripting Backend setting under Other Settings is set to IL2CPP.
- Under Publishing Settings and then Capabilities, ensure the Microphone and SpatialPerception items are checked. This will allow our UWP application to access the microphone for voice commands and the spatial mapping mesh data from the platform.
- Finally, ensure your XR Settings match the image below. Single Pass Instanced Rendering provides a more performant rendering pipeline for XR application. Depth Buffer Sharing will enable the platform to stabilize holograms more accurately on device to account for slight variations in head movement of the user.
Review
Now with your environment properly configured and initialized, we are ready to start making the first part of our XR application! In the next article, we will create the core mechanic of the game: the ball that the player will shoot.
Next section — [Part 2] —Building the ball
