Design / UI Guidelines Interaction Inputs: Hands & Mouse

Tracking hands is one of the most fundamental differences between the Meta 2 and VR stereoscopic headsets such as the Oculus Rift or HTC Vive. The HTC Vive and Oculus Rift provide users with hand-held controllers to interact with virtual objects. In contrast, Meta leverages hand-tracking technology to make it possible for continuous hand poses and movement to control holograms.

It is essential to design interactions that encourage your users to keep their hands in the Interaction Region so that they will have successful experiences. In addition, we recommend that you optimize your app to work with our gestures.

Hand Gesture Library

Below we have outlined the gestures the Meta 2 supports, as well as the various interaction states.


Figure 7. Graphics on the back of the hand will indicate whether your grab is registered
 


Interaction Indicator

As your hand approaches an interactive object, a white circle with a ring around it will appear on the back of your hand. That’s an indication that our system recognizes your hand. As you close your hand into a fist, the white ring becomes smaller and draws closer to the circle. When the ring touches the circle it will become green - indicating a successful grab interaction.

This gesture and the associated graphic indicator form the foundation of most all hand interactions within the Meta 2 SDK.

Out of Range Warning
When your hand moves near the edge of the sensor while engaging in any interaction, the graphic indicator will turn red and a warning message will appear stating that hand performance is getting weaker.

Note:
In its current state, the Meta Hand tracking detects a top point and center of the palm in order to estimate pose. We encourage that fingers are angled upwards, with palms facing the object the user wishes to control. The user may want to use only the index finger when pressing a button or doing any task requiring precision. This improves performance and system responsiveness when it comes to grabbing and clicking.

One-Hand Grab and Move Gesture


Figure 8. Example of the Grab and Move Gesture


As your hand approaches an interactive object, a white circle with a ring around it will appear on the back of your hand. As you close your hand into a fist, the white ring becomes smaller. When circle becomes green the grab is engaged. You can can move the object to a new position in world space.

When you reach the desired location, simply release the grab by opening your hand.

Two-hand Grab Gesture: Scale


Figure 9. Example of the Scale Gesture


For this gesture bring both hands up. White circles with rings will appear on the back of both hands. As you close your hands into fists, the white rings become smaller and the circle becomes green. Scaling the object larger is as simple as moving your hands in opposite directions. Make the object smaller by bring your hands towards each other. 

When you reach the desired size, simply release the grab by opening your hand.

Two-hand Grab Gesture: Rotate


Figure 10. Example of the Rotate Gesture


For this gesture bring both hands up. White circles with rings will appear on the back of both hands. As you close your hands into fists, the white rings become smaller and the circle becomes green. To rotate an object just move one hand forward while the other moves backward (in opposite directions). The object rotates on all axis.

When you reach the desired position, simply release the grab by opening your hands.

Two-hand Grab Gesture: Tilt


Figure 11. Example of the Tilt Gesture


For this gesture bring both hands up. White circles with rings will appear on the back of both hands. As you close your hands into fists, the white rings become smaller and the circle becomes green. You can tilt a panel by placing one hand on the top and the other on the bottom edge of the panel, and push in the direction you wish to tilt. The object rotates on the X axis.

When you reach the desired position, simply release the grab by opening your hands.

Point and Click Gesture


Figure 12. Example of the Point and Click Gesture


You can use your index finger to make a selection. For more precise interactions, using your mouse is best (see section below).

The Meta Cursor and the use of a Mouse or Stylus

The Mouse should be used for UIs where more precise interaction is needed. Note that as Meta’s hand tracking evolves, the need for the
mouse may dissipate over time. Keyboard control is also provided.
The following is a guide of the Mouse interactions.

 

Mouse Interaction: Select and Move (X,Y Axis)

Ui_Fig13_Image.jpg
Figure 13. Move object on the X and Y axis


Move an object on the X and Y axis by left-clicking and dragging your mouse.

 


Mouse Interaction: Select and Move (Z Axis)

Ui_Fig14_Image.jpg
Figure 14. Move object in Z space


Move an object in Z space by pressing the left mouse button while using your middle finger to move your mouse’s scroll wheel.

 


Mouse Interaction: Rotate

Ui_Fig15_Image.jpg
Figure 15. Rotate an object


To rotate an object, right-click while moving your mouse around.

 

Mouse Interaction: Rotate

Ui_Fig16_Image.jpg
Figure 16. Scale an object


To scale an object larger or smaller, press the left-mouse button while holding the CTRL key on your keyboard, and move your mouse back and forth.

 

Previous Next