HoloLens 2 Emulator: Showing and Manipulating Hands in MRTK2 App

Last week, I wrote a first look at the new HoloLens 2 emulator and showed you how something of the hand movement in the HoloLens shell using an Xbox One controller. This was pretty hard to do as the hands were only intermittently displayed. It turns out that if you deploy an app made with the Mixed Reality Toolkit 2, you actually get a lot better graphics assisting you in manipulating. It takes some getting used to, but I was able to play the piano and press some buttons, just like Julia Schwarz was able to do in her now-famous MWC demo.

This then looks like this:

As you can see, the mere act of moving the hand past or through the piano keys or the buttons above actually triggers the buttons (if you turn the sound on you can hear the piano and some audio feedback on the buttons too).

This is simply the HandInteractionExamples scene from the MRTK2 dev branch, generated into a C++ app and deployed into the emulator.

To show you how the hands can be moved, I made another little captioned movie:

Using the Xbox controller is a lot easier this way, although I am not quite sure how to do a two-hand-manipulation yet, as the sticks can only control one hand at a time (the left or right bumpers determine which hand you control.