How Researchers Cracked Hand Tracking Technology on Quest

Oculus Blog
October 22, 2020

Hand tracking is one of many endeavors inside FRL Research designed to make interacting with technology feel more approachable. From machine perception and lifelike avatars to brain-computer interfaces, we're building the next computing platform to benefit as many different people as possible. Hand tracking technology opens immersive computing to more people by adopting a “come as you are” approach to human-computer interaction.

FRL’s work building more natural devices started with Touch controllers. Comfortable to hold and packed with sophisticated sensors, Touch controllers delivered life-like hand presence and made even the most basic interactions in VR (like grabbing a door handle) feel more like the real thing. But for all their considerable benefits, Touch controllers can’t replicate the expressiveness of a peace sign or the efficiency of typing. But more than five years ago, when FRL Research started working on hand tracking for VR, no one had ever shipped consumer-quality, controller-free hand tracking. The challenge was nothing less than to develop the technology from scratch to make VR more approachable by creating a new interface based on the human hand.

We explore how we accomplished that, as well as further potential of hand tracking, through two new publications. This week at UIST 2020, a symposium on user interface software and technology, FRL Research debuted a novel method that enables touch typing without a physical keyboard. We showed that hand tracking can potentially deliver the efficiency and familiarity of traditional typing, but on any flat surface. And at SIGGRAPH 2020, an annual conference on computer graphics, FRL researchers shared how they delivered practical hand tracking on Quest through numerous breakthroughs in mobile computing. Today, every Quest headset ships with hand tracking out-of-the-box.

Read the full story over at the Tech@ blog.