SUMMARY:
UCLA researchers in the Department of Electrical and Computer Engineering have developed software that allows users to interact with virtual and augmented reality via hand movements.
BACKGROUND:
Virtual and Augmented Reality (AR/VR) systems allow users to interact with a virtual environment in an immersive and intuitive way. Typically, a headset provides visual and auditory input of this virtual environment while physical controllers are used as means to interact with this virtual environment. These controllers are often expensive and are by far the least immersive and intuitive aspect of AR/VR. The use of interactive gloves has been explored as an alternative, and while they are intuitive, they are prohibitively expensive and often are uncomfortable to wear for extended periods. AR/VR systems are in desperate need for an interface which provides the intuitive aspect of the gloves with a significant price reduction.
One promising solution is known as free hand AR/VR which uses cameras to visually track the user’s hands so the user is not restricted by clunky controllers or restrictive gloves. Prior work has been focused on modeling the user’s hands in the virtual experience so that they may directly interact with objects as they would in the real world. While this has been promising, it is not very intuitive due to the lack of haptic feedback. One alternative approach is to treat the user’s hands as the objects rather than as hands interacting with virtual objects. The idea is rather than having a user hold a virtual wrench, the user could make a shape with their hands which will then act as the virtual wrench. There is a clear and present need for development of such an intuitive approach to interact with objects that is low cost and is significantly more comfortable than gloves and controllers.
INNOVATION:
Researchers at UCLA have developed software to allow freehand AR/VR interface that relies on using the user’s hands as objects within the virtual environment. In this instance the user can mimic an object by changing the orientation of their fingers which the software will then interpret. This allows the users hands to act as that object within the virtual environment and enables intuitive interaction with a virtual environment. The software eliminates the requirement of physical controllers and clunky gloves which reduces costs and barriers for widespread implementation.
POTENTIAL APPLICATIONS:
- Virtual Reality
- Augmented Reality
- Education
- Creative Design
ADVANTAGES:
- Low Cost
- No Physical Controller
- Intuitive Use
DEVELOPMENT-TO-DATE:
Software has been developed and successfully demonstrated.
DEMONSTRATION VIDEO:
Hand Interfaces
Hand Interfaces: Using Hands to Imitate Objects in AR/VR for Expressive Interactions