Ultraleap is a believer in hand-tracking and haptics for spatial computing.
Take a cursory glance at Ultraleap’s website and it becomes immediately obvious the company is doing similar work to Apple and Meta are with Quest and Vision Pro, respectively. Ultraleap states its mission to is to “make digital worlds feel more human,” noting that since its formation in 2019, it has been building “world-leading hand-tracking and haptics” that it boasts is the only such technology that “creates the sensation of touch in mid-air.” Ultraleap’s team comprises more than 150 people speed across the globe, with locations in Silicon Valley and the United Kingdom. The company employs experts in technical domains such as acoustics, computer vision, user interface design, and machine learning.
In recent weeks, I connected via email with Ultraleap’s senior vice president of engineering and operations Rowena Innocent for an interview on her work at the company and the company’s institutional philosophy on accessibility and mixed reality as an assistive technology. Innocent, who has more than 30 years of experience in what she described as “leading multidiscipline [research and development] teams to deliver novel scientific and engineering products for a wide range of industries such as biopharmaceuticals, nanomaterials, aerospace and energy,” explained her lifelong enthusiasm for solving problems and STEM subjects cultivated a passion for helping others “make new and exciting products.” She added she and her team is steadfastly committed to “committed to creating environments where people can explore and be their best,” whether navigating digital worlds or the physical one.
Innocent of course is familiar with competitors in the marketplace like Meta’s Quest and Apple’s Vision Pro. She told me she “still [gets] more of a kick from exploring how products work than from using them” as she did in childhood; working at Ultraleap is “perfect” for Innocent, as she gets to play with the latest mixed reality tech as part of her job.
Expounding on her work at Ultra, Innocent said Ultraleap is doing “exciting” work on creating virtual experiences that feel “as natural as the real world.” She pointed to Ulrealeap Hyperion as an example, telling me the hand-tracking software platform has capabilities which have the potential to make mixed reality intuitive and natural—not to mention accessible too. What makes this software stack stand out, according to Innocent, is its combination with haptic feedback; doing so, she said, enables people to reach out and touch virtual objects and the sensory feedback from the aforementioned haptics means the experiences makes it seem more realistic. In essence, Ultraleap aspires to augment the mixed reality experience with bimodal sensory input: not only are you experiencing something visually, you’re also experiencing it tactilely—which, in certain circumstances, can prove much more accessible depending on a person’s needs and tolerances in terms of their disability.
“Our software can now track fine, subtle microgestures in hands and fingers, with accuracy down to the millimeter,” Innocent said. “This is a game-changer for users who have mobility issues, [It sets] us far apart from VR systems in the past that required users to make exaggerated hand and arm movements to navigate a virtual space.”
Innocent acknowledged there remains work to be done in developing Ultraleap Hyperion, but noted her team is deeply committed to “learning and iterating on our solutions to ensure that as many people as possible can easily access and experience the magic of mixed reality.”
When specifically asked about accessibility, Innocent told me mixed reality technology has immense potential to help the disability community in ways smartphones and laptops never could. This is because content in confined to a fixed screen, which may not be accessible to many who are visually impaired, regardless of assistive technologies like screen readers and more. AR, Innocent said, represents a “paradigm shift.” Rather than adapt themselves to the technology at the risk of exclusion, augmented reality can be more human-centric. To wit, Innocent said AR can leverage senses such as hearing, sight, and touch can “immerse” a disabled person in such a way that the digital experience feels “natural and is augmented in such a way that it is assistive rather than restrictive.” Augmenting audio-visual experiences with haptics, Innocent went on to say, can help those with visual and/or hearing loss to still be able to experience things like music. (The Music Haptics feature new to iOS 18 is predicated on that very idea.) Innocent also said augmentation can be beneficial for the elderly, as smartphones and the like oftentimes are too complex and unapproachable because there’s a steep learning curve for those who aren’t technically minded. By contrast, Innocent said AR can prove “as intuitive and as natural as interacting with the real world” by using technology like haptic feedback.
“In combination with AI assistance, sensory augmentation of these new smart wearable [devices] can be a game-changer,” Innocent said of the benefits of haptics. “It’s an opportunity for everyone to have access and enjoy AR experiences. More importantly, it can make a significant positive impact on people’s health, independence, and quality of life.”
The sharing of my conversation with Innocent is timely, as it comes soon after Apple unveiled visionOS 2 at last week’s Worldwide Developers Conference. The forthcoming software update, which Apple says ships this fall, will bring with it new hand gestures that allow users to, for example, summon Control Center without needing to look upwards as they do today. According to Apple, the new gestures give people the ability to “see important information at a glance, like the current time and battery level, and perform actions like adjusting the volume.”
Feedback-wise, Innocent told me Ultraleap has garnered “amazing responses” from its customers, telling me people oftentimes provide ideas for future improvements and share anecdotes about how the company’s tech could possibly help others in their orbit live better.
Looking towards the future, Innocent said she believes it would be “amazing” if more companies leveraged augmented reality with an eye towards furthering accessibility and inclusivity. She added it’s equally amazing to see more open ecosystems and platforms crop up, saying she would “love it” if it were so open, anyone with a great idea could combine it with technologies from others doing similar work in this realm.
“The ultimate goal would be personalized solutions for the user making use of the best combination of assistive technologies,” Innocent said of her dreams for the future. “AR would then be truly human-centric.”