- Target and select a button or option from a digital menu
- Make a section by pressing with a finger, like on a touch screen or real physical button
- Understand which option or button is being targeted and selected
Air taping or using the finger to select options presented within a digital menu is already used in VR, particularly with Oculus touch controllers that allow simulation of hand and finger interactions. It’s also been a goal for AR for some time, often featuring in demo videos of how we’ll interact with spatial computers of the future. Today, Leap Motion and Hololens 2 are making this possible with fully articulated hand tracking.
Elements to the interaction:
- Some kind of digital menu or panel with buttons and options is presented to the user in a space they can reach with their hand.
- The user presses the button with the finger as if it was a touch screen or real button
- The button reacts to the users finger press
- It may highlight the cursor position as the finger hovers to indicate what button is about to be pressed (targeting)
- It may animate to show the button being pressed in, to provide feedback that this option is now being selected
- It may be accompanied by suitable audio or haptic feedback
Hololens 2: Air tapping buttons on digital menu
Leap Motion concept examples
Although the examples shown are from a VR demo, the same UX behaviour and interaction would work with AR that utilises full finger articulation.
Threesixty Reality concept examples
Note that here the user is using Oculus Touch controllers to simulate the hand. The user must grip the controller handle, but keep their index finger extended so it does not touch the trigger button. This is a standard Oculus convention and simulates the hand pointing with the index finger, which can now be used to also press virtual buttons.