On February 1, Northrop Grumman Corporation announced that it has been awarded a Defense Advanced Research Projects Agency (DARPA) Perceptually-enabled Task Guidance (PTG) program contract to develop a prototype artificial intelligence (AI) assistant. The prototype will be integrated into an augmented reality (AR) headset to assist UH-60 Blackhawk helicopter pilots in their decision making.

The Defense Advanced Research Projects Agency is an agency of the U.S. Department of Defense responsible for the research and development of new technologies for military use. Its program aims to explore “advances in augmented reality for human-computer interfaces and deep learning for speech and video analysis to develop “task guidance” virtual assistants “that can provide users with audio and visual feedback.” Bruce Draper, program manager at DARPA’s Information Innovation Office states:

“We are increasingly looking to develop technologies that make AI a true collaborative partner with humans, developing virtual assistants that can provide substantial assistance to human users when performing tasks requiring advances in a number of areas of interest in machine learning and AI technology, including knowledge acquisition and reasoning.”

According to DARPA, the program seeks to determine how task guidance assistants could be used in battlefield medicine, pilot guidance and mechanical repair.

Northrop Grumman, formed from the 1994 merger between Northrop and Grumman, based in Los Angeles, is the world’s third-largest weapons company. It has just won a contract under this program to develop an augmented reality helmet augmented with AI.

Combining AI and augmented reality

Northrop Grumman, in partnership with the University of Central Florida (UCF), will develop an operator and intuitive context adaptive reasoning assistant (OCARINA) that will support UH-60 Blackhawk pilots, who fly both visually and with instruments, in a complex environment. Most of the time, they only have simple warning systems, such as audible alerts to increase altitude.

The helmet will have sensors that allow the AI to“observe what the user perceives and know what the user knows.” Through advanced information processing and an AR interface, the AI assistant will provide verbal feedback and guidance as well as graphics that will optimize the crew’s piloting and skills. Erin Cherry, senior autonomy program manager at Northrop Grumman, states:

“The goal of this prototype is to expand a pilot’s skills. This will help teach new tasks, recognize and reduce errors, improve task completion time and, most importantly, prevent catastrophic events.”

Translated from USA : Les pilotes d’hélicoptère bientôt équipés d’un casque de réalité augmentée assisté par l’IA