DeveloLens 3 - HoloToolkit Spatial Mapping

<< DeveloLens 2 - HoloToolkit-Unity Init

HoloToolkit-Unity Spatial Mapping

So you've created a UWP app or two for your HoloLens, but you have an idea for an app that utilizes whatever room the user is in. There are a few tutorials in the Microsoft Academy that cover the concepts, and a few that show you how to use a version of the UnityToolkit that no longer exists. Unless you can reverse engineer the code they provide, and find new versions of classes they tell you to use, it can set you back in understanding how easy it is to use.

Here we will cover the latest version of the toolkit, and try to explain how it's used well enough so that it will remain relevant even as development continues. This will be about how to use the code they provide when you require specific functionality, and specifically spatial mapping for this post.

welcome, to the real world...

Egg Roll

Let's say we want to see what it would look like to have an egg roll around our floor. After creating a new scene, and saving it wherever you prefer, go ahead and add the toolkit. If you are not familiar with adding HoloToolkit to your Unity project, see my previous post on doing so. Since we'll be creating our own scene, we will not need the Examples or Tests folders.

If you're wanting to run this locally in Unity before deploying, you will need your room model. You can record your own from the device portal, or you can use the Test model currently provided in HoloToolkit-Tests > SpatialMapping > Prefabs which is usually nicer than the rooms I find myself developing in.

Remember that starting with a fresh project and scene, you must always run the HoloToolkit > Configure menu settings to make sure your scene is setup and the project is configured correctly for your app.

Delete the default the components, and then add the following components which are currently located in the following locations.

• HoloLensCamera: HoloToolkit > Input > Prefabs > HoloLensCamera.prefab
• SpatialMapping: HoloToolkit > SpatialMapping > Prefabs > SpatialMapping.prefab

Now we need a placeholder which I'll call Managers to add our, well, managers. Specifically the following managers.

• WorldAnchorManager: HoloToolkit > Utilities > Scripts > WorldAnchorManager.cs
• Wrapper around world anchor store to streamline some of the persistence api busy work.
• GesturesInput: HoloToolkit > Input > Scripts > InputSources > GesturesInput.cs
• Input source for gestures information from the WSA APIs, which gives access to various system-supported gestures. This is a wrapper on top of GestureRecognizer.
• InputManager: HoloToolkit > Input > Scripts > InputManager.cs
• Input Manager is responsible for managing input sources and dispatching relevant event to the appropriate input handlers. 
• RawInteractionSourcesInput: HoloToolkit > Input > Scripts > InputSources > RawInteractionSourcesInput.cs
• This input source only triggers SourceUp/SourceDown and SourceDetected/SourceLost. Everything else is handled by GesturesInput.
• Just remember this script is necessary to fill a gap for InputManager.
• GazeManager: HoloToolkit > Input > Scripts > Gaze > GazeManager.cs
• The gaze manager manages everything related to a gaze ray that can interact with other objects

These scripts will give us the functionality we need to use Spatial Mapping, and we can add just a little bit of code to roll our egg into it. Let's add our own script to our Managers called EggManager.

In a nutshell, this script enables the GestureRecognizer input functionality, which we then assign an event method to. This event is just going to place our egg at a predefined distance away from the user.

The important part of this is the public prefab that we call fill with any prefab we want, and the InputClicked event we will use to drop that prefab on the user click. This will give a project that looks like so:

I created an almost egg prefab by creating a sphere and altering it slightly, but you can drop anything like it's hot as long as it contains a rigidbody with the 'Use Gravity' enabled.

You can now build your Visual Studio solution from the HoloToolkit menu to whatever folder you'd like. Then open it up in Visual Studio for deployment to the emulator or the device.

Lets watch these eggs roll.