Facebook announces new Spark AR capabilities to open up new Augmented Reality experiences on Instagram

In Augmented Reality News 

May 28, 2021 – Facebook has recently announced the roll out of two new augmented reality (AR) capabilities for Instagram, including multi-class segmentation and an improved target tracker. The company stated that both of these capabilities open up new AR experiences, and are available now with version 114 of Spark AR Studio.

Multi-Class Segmentation

Creators can now combine multiple segmentation textures, including hair, skin and person, together in the same effect. Facebook has also made it easier to control multiple texture properties, such as edge softness, which creates an outline around a person in a scene, and mask size, which adjusts the size of the outline.

This new capability will allow for the creation of more complex AR experiences, according to Facebook, ranging from character effects, to high fidelity clothing effects. 

There are apparently some limitations to consider when using segmentation in effects though, including:

• They often work better when identifying people from the chest upward;
• Low lighting tends to affect camera accuracy; and
• These types of effects usually perform best on newer mobile devices.

Facebook has provided updated documentation for reference on the use of segmentation, as well as a tutorial on how to create a hair segmentation effect. Lastly, the company is rolling out a new template that uses multi-class segmentation to produce a modern-art-inspired effect, which creators can use for inspiration, or simply as a shortcut for gathering up assets and organizing an original project.

Updated Target Tracker

Improvements have also been made to Spark AR Studio’s target tracker, which triggers an AR effect when a user points the camera at a fixed 2D image in the real world, like a poster or a sign. As part of this latest update, the target tracker can now detect multiple fixed or moving target images to trigger an AR effect.

When choosing to use a moving target image, the effect will move as the target image moves. If the user’s camera isn’t pointing at the target, the effect will simply disappear. Creators can still choose to use a fixed target image as well, and effects will appear in the same position where the user’s camera first detected it and will stay fixed in that position, even when the user moves their camera.

The target tracker is now able to track up to five unique images in a scene, however consideration for performance across mobile devices is recommended. Additionally, with the performance improvements in this latest target tracker release, Facebook also recommended updating previous effects that used target trackers. To do this, creators simply have to open a project file in the latest version of Spark AR Studio and re-publish their effect.

To learn more about how to optimally configure and use the target tracker, please view Facebook’s updated documentation. There is also a new template available that bundles everything needed to create an animated 3D poster.

For video examples of the new capabilities, click here. more information on Facebook’s Spark AR Studio for creating augmented reality experiences across the company’s family of apps, please click here.

Image credit: Facebook