Targeting with ray casting

User goal

• Point at an object or option in the menu that the user would like to select or interact with (e.g. a menu button)
• Control pointer precisely with lowest amount of effort
• Allow the user to point and highlight an option whilst having the freedom to be looking elsewhere (decoupling of gaze and pointing)

Interaction

The user uses a handheld controller which is trackable and visible in the 3D VR environment. In certain modes, such as when a menu is called up, a ray resembling a laser, emits forwards from the end of the controller.

When the ray encounters an object in its path that has interactive properties the object will provide feedback to indicate possible interactions. For example the buttons and options in a virtual menu will change appearance when the raycast hovers over them, indicating they can be clicked.

The raycast can be blocked by 3D objects, often referred to as colliders or can shine right through them if they have been excluded by the developer, to make it easier for the raycast to land on interactive or interesting objects in the environment

1. The raycast is activated by the system, for example if a menu is opened
2. The user moves the controller to see where the raycast is pointing
3. Menu buttons and interactive elements are highlighted or change appearance when hit by the raycast to indicate they are interactive
4. Usually one of the other controller buttons (e.g. trigger) would select the menu item or trigger the interaction

Good

• Allows the user to quickly move the pointer with efficient hand movements
• Either hand can be used, supporting both left and right handed users
• The user can be highlighting an item in one location whilst looking elsewhere with their gaze. Minimises unnecessary head movements, as the user does not need to point their head directly at the item being highlighted.
• Much faster than using a D-pad controller, traditionally associated with game and interactive TV menus

Bad

• It can become fiddly or shaky if the user does not have steady hands
• Sometimes the user does not realise there is a raycast, for example if they don’t move the controller around, it may not be obvious there is an interactive ray being emitted from it
• Also the ray can sometimes be blocked by objects or behave in unexpected ways in the 3D environment

Examples

Onward, Oculus

Steam VR, Steam

Raw Data, HTC Vive