Design patterns for Immersive Tech

Augmented Reality patterns

Air Tap button selection

User goals

  • Target and select a button or option from a digital menu
  • Make a section by pressing with a finger, like on a touch screen or real physical button
  • Understand which option or button is being targeted and selected

Interaction

Air taping or using the finger to select options presented within a digital menu is already used in VR, particularly with Oculus touch controllers that allow simulation of hand and finger interactions. It’s also been a goal for AR for some time, often featuring in demo videos of how we’ll interact with spatial computers of the future. Today, Leap Motion and Hololens 2 are making this possible with fully articulated hand tracking.

Elements to the interaction:

  • Some kind of digital menu or panel with buttons and options is presented to the user in a space they can reach with their hand.
  • The user presses the button with the finger as if it was a touch screen or real button
  • The button reacts to the users finger press
  • It may highlight the cursor position as the finger hovers to indicate what button is about to be pressed (targeting)
  • It may animate to show the button being pressed in, to provide feedback that this option is now being selected
  • It may be accompanied by suitable audio or haptic feedback

 

Examples

Hololens 2: Air tapping buttons on digital menu

 

Leap Motion concept examples

Although the examples shown are from a VR demo, the same UX behaviour and interaction would work with AR that utilises full finger articulation.


 

Threesixty Reality concept examples

Note that here the user is using Oculus Touch controllers to simulate the hand. The user must grip the controller handle, but keep their index finger extended so it does not touch the trigger button. This is a standard Oculus convention and simulates the hand pointing with the index finger, which can now be used to also press virtual buttons.