Niantic introduces Hand Tracking to its 8th Wall WebAR platform

In Augmented Reality News 

June 1, 2023 – Niantic, a provider of augmented reality (AR) technologies and the creator of Pokemon GO, has announced the introduction of Hand Tracking to its 8th Wall platform – the company’s suite of augmented reality (AR) tools for the web.

Hand Tracking on the 8th Wall platform will now allow developers to create WebAR experiences that incorporate the hand, wrist, and fingers all in the same experience. 8th Wall’s proprietary hand model extends just past the palm to include the region where a watch or bracelet would typically rest, according to the company. Developers can also fine-tune each experience with dozens of attachment points, adaptive hand mesh, and access to the camera pixels for skin smoothing.

Out of the box, the new hand model includes 36 attachment points distributed across the fingers, knuckles, wrist, and palm. Each finger contains up to seven attachment points on both the knuckles and the finger joints, letting developers control the position and rotation of items attached to the hand. Niantic stated that having these attachment points already available will help to save on development time and will enable developers to quickly anchor objects or experiences to a user’s hand.

Niantic’s adaptive hand mesh dynamically creates a personalized mesh of each user’s hand to enable objects to be appropriately attached to that hand. This feature allows developers to create experiences that automatically scale to fit the user’s approximate hand size and volume.

The company added that the new Hand Tracking functionality will unlock a wide range of experiences that use real-world physics, occlusion, or hand anchors, and will open several use cases, including:

Immersive Interactions: Developers can build AR experiences that use the hand as an interface, allowing audiences to pick up, move, push, and engage with virtual objects almost like they’re really there. This type of interaction can be used to create games, interactive campaigns, or even educational experiences.

Puppeteering and Hand Masking: A user’s hand can be transformed to create something entirely new, such as a monster-size hand, simple or intricate puppets, or even a foam finger for a sporting event. A user’s hands and fingers can then control the movement, providing a new level of immersion or personalization.

Special Effects: For making hands the foundation of experiences, from holding a orb of light, to unleashing laser beams from the tips of the fingers, to transforming thumbs into a lighter at a concert.

Virtual Try-on: For trying out rings, bracelets, and watches that automatically scale to fit the individual. Multiple items can even be tried on at once, according to the company.

Niantic added that because Hand Tracking in 8th Wall is built for the browser, developers can enhance their experiences by leveraging the power of the open web and by connecting real-time APIs to their experiences in order to create more engaging content.

The company noted that today’s announcement represents just the start of Hand Tracking tools for the 8th Wall platform, and Niantic will continue to iterate on this feature based on the feedback of the developer community.

For a sample project from Niantic’s project library, click here. To read the full release notes and documentation, click here. To learn more about Niantic, its 8th Wall platform, and its augmented reality solutions for the web browser, please visit the company’s website.

Image / video credit: Niantic / 8th Wall