US Army researchers develop Soldier decision-making system that populates immersive XR environments with complex data

In Augmented Reality, Virtual Reality and Mixed Reality News

October 27, 2020 – The US Army has today announced that its researchers have developed a system that populates immersive environments with complex, heterogeneous data, allowing warfighters to “retrieve relevant information quickly in support of mission-critical decisions.”

According to Dr. Mark Dennison, a research psychologist with the US Army Combat Capabilities Development Command’s Army Research Laboratory (CCDC ARL), the system is inspired by the potential of extended reality (XR), with many scientists and engineers throughout the Department of Defense (DoD), commercial and academic spaces having developed systems to explore what is possible in the XR domain.

Augmented, virtual and mixed reality (collectively known as extended reality/XR) present opportunities to redefine how users interact with computers and information, Dennison said. The work by the laboratory is focused on creating an XR common operating environment that enables interoperability between commercial, off the shelf XR systems and program of record systems, heterogeneous sensors and big data analytics over a secure, encrypted network.

Dennison added: “This work aims to create a framework for enabling interoperability between XR technologies and big data analytics to accelerate decision-making and enable shared understanding between co-located and distributed users.”

The Army stated that by linking the ‘Accelerated User Reasoning: Operations, Research, and Analysis for Cross Reality’, or ‘auroraXR’, system with the ‘Elastic Stack’, a big-data analytics architecture used throughout the DoD, industry and academia, scientists and engineers are now able to populate immersive environments with complex, heterogeneous data.

“These rich datasets enable CCDC ARL scientists to investigate how complex decision-making processes are enhanced or suffer detriments in XR environments, especially in comparison to existing, fielded information systems,” Dennison stated.

The system has been developed by Stormfish Scientific Corporation, which is under contract with the laboratory and has contributed to the ARL-funded research.

Dennison continued: “Military decision-making requires the convergence of large volumes of data from a wide spectrum of sources that cross domains. Much of this data may be irrelevant or redundant when attempting to filter down to critical elements necessary to answer a specific question. Therefore, the Army requires novel information meditation tools to assist human decision makers with identifying and retrieving relevant data quickly enough to support mission-critical decisions for which the windows of opportunity may be less than a few minutes.”

For example, an Army brigade commander may need to leverage intelligence, surveillance and reconnaissance (ISR) assets from the Air Force while simultaneously tracking Army ground units co-located with Marines.

“This information space is only exacerbated by gray asset data, such as social media feeds, which may dynamically change mission planning within complex urban terrain,” Dennison noted. “Technologies that identify, transmit and display only that information which is relevant to a particular unit’s mission are critical for Combined Joint All-Domain Command and Control, or CJADC2, decision-making.”

The army highlighted that its research has also helped to address the lack of a secure XR networking solution that meets DoD networking and security requirements, and provides a near turn-key solution for linking foundational research to prototyping systems. The effort also supports the Adaptive Cross Reality Information Mediation, or AXRIM, task under the Robust, Resilient, and Intelligent C3I, or RRIC3I, project in the Army’s Network Cross-Functional Team.

The Army noted that the work is helping to shape its understanding of how an immersive common operating picture will augment decision-making in the mobile, mounted and command post computing environments across echelons. Dennison closed by stating that: “Researchers are eager to continue working in this area to operationalize science for transformational overmatch and provide Soldiers with unmatched capabilities on the battlefield.”

Image credit: US Army