Design patterns for Immersive Tech

Virtual Reality patterns

Ray Casting with hand menus

User goal / Problem this is trying to solve

  • Allow the user to easily select something from a hand menu (or menu attached to controller)
  • Use both hands to make interaction with contextual menus efficient


This is an extension of the ‘display menu on controller‘ pattern and explores on of the ways to interact with the contextual menu.

  • The user triggers the contextual menu or ‘hand/arm menu’ in the usual way as described in the display menu on controller pattern
  • For this interaction to work the contextual menu is typically on the left hand and the pointer tool on the right hand, but this is a function of which hand is dominant so for left handed users the hands are switched (remember to provide options for this)
  • The other hand would provide the ray cast, which would typically appear and behave in the usual way as described by that pattern (and is it behaves in the rest of the application)
  • The user moves the ray cast hand and points to the options on the contextual menu
  • l process of highlighting targeted options and using a specific controller button to confirm the selection is used

Linked to: Targeting with ray casting & Display menu on controller & Virtual hands & hand menus


  • Uses the targeting and selection mechanism that is consistent with the rest of the app
  • Ray cast is typically intuitive to use and fairly accurate
  • Allows the user to bring up options and make selections without pausing or interrupting the main experience
  • Allows the user to control exactly which area of the screen the menu interaction takes place within


  • Ray Casting works best as a pointing device over longer distances. It can feel clunky over the shorter hand to hand distances


Raw Data: Using ray casting in combination with interactive hand menu

Google Blocks: Ray cast on augmented controller tool (note how the dot rather than full laser can help reduce noise, especially when dealing with complex multi-layered menus)