Let’s get Aligned – Alignment UX methods for MR and AR in Construction

Let’s get Aligned!

Alignment UX methods for MR and AR in Construction

As the AEC industry continues to embrace digital tools and processes in their work, more and more construction professionals are finding real business value in bringing and experiencing 3-D model data on the job site with Mixed Reality (MR) and Augmented Reality (AR). 

To be successful using MR/AR on a project site, contractors face a handful of challenges, as well as bring to the table their own critical success criteria. In other words, a lot can go wrong, and some things must go right for construction professionals to realize true value from MR/AR. 

Chief among these critical success factors is that any MR/AR solution for AEC must enable true, accurate, and stable alignment between a digital model and the physical environment it augments and overlays. Additionally, any MR/AR solution must offer an easy and robust user experience (UX) to achieve alignment, enabling professionals to reliably and confidently match virtual to physical in three-dimensional space before they can proceed to compare, contrast, and analyze existing vs. proposed conditions, and react accordingly. 

I recently researched and surveyed several contemporary MR/AR software solutions for construction and found that there are multiple ways to achieve alignment between physical and virtual environments. Among the top solutions in the market, there are predominantly four approaches to alignment UX: visual, machine-readable, geometric, and algorithmic. I discuss the approach of each below, along with their advantages and disadvantages, like the trade-offs between accuracy, set-up, and ease of use.

Visual Alignment

Among the top MR/AR tools in construction, many adopt an approach that relies primarily on visual alignment with physical, manual adjustments. Arkio, in particular, has a very easy-to-use interface, enabling and requiring users to simply pick up a full-scale model and place it where it appears to align best with the physical environment. Users can grab and drag the model repeatedly until they are satisfied with the alignment, before spatially anchoring the model for reliable and repeatable use among multiple participants, persistent across multiple sessions.  

However, with such an easy, straightforward interface often comes a limitation in accuracy. Here, platforms including Argyle, BIM Holoview, and XYZ Reality offer a “gizmo,” which is a tool that enables the user to make fine adjustments to one or more axes to more precisely manipulate an object’s position (X, Y, Z) and orientation (roll, pitch, yaw) in 3-D space. These UXs vary subtly among click-and-drag interfaces and type/text interfaces, but all allow for relatively precise, if manual, alignment.  

This method can require initial effort up-front, but generally once the model is aligned, it stays anchored reliably and doesn’t depend on recalibration due to feature changes in the physical environment. 

Machine-Readable Alignment

Beyond relying on human perception to roughly align and then finely adjust alignment, many MR/AR solutions support using machine-readable fiducial markers as control points. The most common way to do so is with a pair of unique, machine-readable QR codes, placed at known points on the job site to correspond to known points in the model.  

Several solutions including BIM Holoview, Connect AR, vGIS, and XYZ Reality employ this method, with subtle variations. For example, BIM Holoview requires the user to place virtual QR-code markers in the model first, then align physical markers on the job site to match. AR Connect has some relative sophistication in the UX for creation and alignment of its markers and can start with just one marker rather than a pair. It also offers manual alignment to nudge a model back into position to mitigate for any drift after the QR marker has been placed. vGIS requires no placement of QR codes in the model at all, leveraging them to simply define the coordinate system and to position the device without model-tampering, reducing errors in placement. 

Whether using such precise, fixed points, or relying on more temporary and flexible QR codes affixed to various surfaces, the method of relying on multiple fiducial markers is generally inexpensive and reliable. For hardware to read QR codes, however, generally requires good lighting conditions, markers that can withstand adverse weather conditions (or limit to interior use), and comes with the risk that markers might be inadvertently moved or lost in the course of construction. In other words, there’s a little set-up and maintenance required to ensure alignment is reliable at each use. 

Geometric Alignment

A third means for aligning virtual to physical environments is one that relies on the human to identify and match key geometric features between the two. Not unlike “old school” perspective-matching tools once offered by rendering programs like Autodesk’s 3ds Max, the modern tools that employ this method, like Argyle, Gamma AR, and XYZ Reality, require a user to click on known and existing features, like the corner, edge, or surface of a wall or column in both the model and the real world.  

Argyle requires two spatial anchor points in the model, while Gamma AR requires two points defining a vector: its vertex and orientation in plan. Argyle then allows for the aforementioned gizmo-based fine tuning and adjustment. Gamma AR relies on parsing the BIM data (for example, a Revit view) to make sense of geometric features, like the corner of a wall intersection, so there is some initial set-up and data-management to be done, but it is very effective. XYZ Reality offers two geometric approaches. First, with defined control points using accurately placed markers permanent to the structure; and second, with a unique surface-alignment approach that requires selection of nine points on three unique, flat, non-coplanar surfaces to yield very precise, robust alignment with minimal need for any fine adjustments.  

The advantage to the geometric approach is that it doesn’t rely on extra reference markers that could be moved or lost, but comes with the disadvantage of requiring permanent geometric features built in-place and some general fussiness in first-time setup. 

Algorithmic Alignment

Finally, the fourth general approach to alignment UX relies on some algorithmic “magic” in the software or display hardware. Most contemporary software solutions require some algorithmic calculations for alignment, but a few lean heavily on this approach: vGIS, BIM Holoview, and XYZ Reality. BIM Holoview, for example, leverages automatic anchor placements for world-locking, following an initial alignment using QR codes and gizmos. XYZ Reality enhances its QR-code alignment with laser-tracking. 

While in many ways, algorithmic alignment seems the most promising of the approaches due to the reduced need for manual human effort and the growing maturity of computer vision, it is also the least mature and least pervasive technique employed today. 

Within the next few years, as more sophisticated immersive display devices reach the consumer and enterprise markets, and as contemporary MR/AR software tools evolve and mature, it is anticipated that all four of these methods of alignment UX will continue to be used, either individually or in various combinations. There is also the possibility of new methods to be invented and deployed in the market. 

However, the potential growth of algorithmic alignment, fueled by AI, computer vision, and scene-parsing, could influence and may ultimately diminish the popularity of the other three methods. Moreover, more sophisticated software tools will likely support multiple alignment UX methods which all work together to be able to address several scenarios on construction sites, not unlike what solutions such as XYZ Reality and vGIS already offer today. Different environmental conditions demand different approaches, so having a choice of methods is important. 

No matter which methods become most popular in time, alignment between the physical and the virtual worlds should be seen as a fundamental technology that must be completely reliable, effortless, and repeatable, allowing construction professionals to leverage MR and AR to enable effective decisions and get the most from real-time, immersive experience of spatial data on the job site.