Friday, May 20, 2016

Hololens - Manipulate Gestures and Resizing Windows

I had a bit of trouble finding documentation for how to make a re-sizable window for Hololens and thus want to document my findings here.  As this is all fairly new please share your findings in the comments.

Here is the goal... in 2D form:

On the device you can click and drag (manipulate) any of the blue cubes to scale the window.  And you can click and drag the center circle to re-position the window in 3D space.

If you have a device you can find an example of this by opening the Photo app and clicking resize in the top right corner.

First and foremost make sure you are up-to-date with the latest Unity build which currently is 5.4b16:

Otherwise you may crash while trying to listen to the manipulate gesture.

If you aren't familiar with Gestures you can learn more about them in detail here:

Manipulate is going to be our focus since we are going to move/re-size based on the user's hand movements.

Microsoft's Holograms 211 tutorial will get you most of the way there. Namely the GestureManager code though their example uses both Navigation and Manipulation which adds a bit of complexity as you can't use both at the same time.  Since we only need Manipulate we can nix the Transition code and just add the GesturesSettings.ManipulationTranslate to our recognizer we are using for tap.  I'll share my code below.

Integrating with HoloToolkit:
Microsoft's HoloToolkit code base's GestureManager is different than the one in the tutorial. I decided to try to pick and choose bits from the 211 tutorial and drop it into the Toolkit code but ended up with a half baked solution.  You could manipulate items while gazing at them but as soon as the item lost your gaze the manipulation would stop. The trick is to make sure you Override the focusedObject. Otherwise when the focus object changes the gesture is cancelled.

GestureManager Class:
This class mixes the HoloToolkit code with the Holograms 211 Tutorial and makes adjustments to handle the change in focusedObject.

To handle these events you will need to wire something to catch the starts and then update while manipulating.  I have my corner buttons catching the start manipulate calls and then triggering StartScaleManipulation while my center button triggers StartManipulation. Code below:

Custom Hand Manager:
I am not sure if we added the FocusedGameObject or if that was originally in the HoloToolkit and since removed. In either case, this is the HandManager we are using. Feel free to rename to HandsManager or customize how you see fit.


Iuliu Blog said...

Hello there.

Can not find the HandsManagerInstance.FocusedGameObject ( used in the manipulation event handlers in your example).

Any ideas?

Ickydime said...

Thanks for the post. I updated the end of the blog to include the hands manager that we are using. I didn't realize the FocusedGameObject was not part of the Toolkit.

Anonymous said...

Hi there, thanks for your help on this - could you please let me know where we should be including the public enum ManipulationType code? Is that a separate cs script?

Thanks I am very new to Unity/HoloLens and C#.

Unknown said...

longchamp outlet
oakey sunglasses
michael kors outlet
coach factory outlet
harden vol 1
adidas shoes
coach factory outlet
michael kors outlet
canada goose outlet
christian louboutin outlet

Unknown said...

golden goose
reebok outlet
lebron 16
yeezy shoes
curry 4
air max 270
moncler jackets
moncler outlet
hogan outlet online

yanmaneee said...

yeezy supply
supreme clothing
irving shoes
yeezy 500
jordan retro
yeezy boost 350
jordan shoes
adidas yeezy
off white jordan 1

Unknown said...

o0o53g1m57 w6l84j3b39 b7i62p0m54 l3j95s8f93 u0b54r1i13 o6w96f0w33