Design patterns for Immersive Tech

Augmented Reality patterns

Hand menu triggered by rotation

User goals

  • Trigger ‘wearable’ menus with natural gestures
  • Avoid menus that clutter space when not needed
  • Access options when needed

Interaction

We constantly hear debates about gesture based interfaces, with many in the industry often arguing that we are a long way off a reliable gesture recognition engine. This all depends on what our definition of a gesture is. When we use an XR platform that constantly tracks the movement and position of the user’s head and hands (so any 6DoF VR system), each movement of the hand is already a gesture and the UI is able to respond to this. Traditional by updating the position of your hand, but also by using this as an input and updating the UI based on certain movements.

Here we will look at how a rotation of the wrist can trigger and affect the behaviour and state of interactive menus. Opening and closing menus can be a hassle, so why not have them appear as if by magic, following a certain degree of wrist rotation. This can work particularly well when combined with interactive wearable elements such as a virtual watch.

Elements to the interaction:

  • The user’s hands are being tracked either by hand tracking (Leap Motion, HL2) or 6DoF tacking on handheld controllers (Oculus Rift/Quest, Windows MR or  HTC Vive).
  • There may already be an interactive virtual watch attached to the position of the user’s wrist (but this is not essential)
  • As the user rotates their wrist inwards or outwards by a certain angle an arm menu appears. It is only visible within a certain range of the controller’s or wrist’s rotation.
  • As the user rotates the wrist back the other way the menu disappears
  • It’s recommended the appearance and disappearance is accompanied by a suitable sound (see video examples), as this firstly draws the user’s attention that something has been triggered and secondly helps the user to understand the exact mechanics of the interaction. The association of sound and muscle memory position can provide powerful reinforcement for learning.
  • It’s possible to have a different menu appear at different stages of rotation, so we get one menu when the wrist is fully extended inwards and another when it’s rotated outwards.

Linked to: AR hand menuWearable hand accessories,  Virtual hand menus

 

Examples

Threesixty Reality concept examples

Leap Motion concept examples

52 Comments