S2RC has been awarded a Phase I SBIR in response to Air Force Research Laboratory’s (AFRL’s) AF222-0014 topic “Just-in-Time, Multi-Mission Airmen/Warfighters (JITMMA/W) Natural User/Technology Interfaces”. S2RC proposed Multi-Mission Airman Training Research for Interface Experience (MATRIX) for this effort. The objective of this topic is to develop a set of tools and processes for enabling a useful and usable user interface when the learning material, interaction model, and the target hardware may not be known in advance. This topic is in support of the USAF’s Agile Combat Employment (ACE) doctrine.
Figure 1. Multi-Mission Airmen/Warfighters are “Airmen/ Warfighters capable of accomplishing tasks outside of their core Air Force Specialty. Specifically, these personnel are often trained as a cross-functional team to provide combat support and combat service support to ACE force elements” - Agile Combat Deployment Doctrine.
S2RC is proposing a multi-modal Human Machine Interface (HMI) approach JITMMA/W user and technology interfaces. The multi-modal HMI will need to address Augmented Reality (AR), Virtual Reality (VR), tablets, smart phones, and potentially another hardware solution. To achieve this S2RC is taking a Cognitive Systems Engineering (CSE) and approach to this challenge and relying on our expertise in novel HMIs and domain knowledge of USAF ACE operations. S2RC intends to explore a multi-modal HMI Primary Alternate Contingency and Emergency (PACE) plan to determine if this approach offers multi-mission Airmen/Warfighters the flexibility needed to conduct JIT training, perform the action they received training on, and switch between the two actions as needed.
To accomplish this effort, S2RC is partnered with Rogue Wolf Advanced Research LLC and is receiving support from Enduvo who is providing access to their Immersive Virtual Classroom and Raytheon BBN which is proving access to its Cyrus Speech Recognizer.
Figure 2. S2RC is excited to continue working with Rogue Wolf Advanced Research, Enduvo, and Raytheon and provide MATRIX to our Airmen and Warfighters.
What does it all mean? Below is an example of the multi-modal HMI PACE:
Scenario: ACE Forward Operating Location (FOL) Echo has been hit with cluster munitions. A team of two multi-mission Airmen/Warfighters have been tasked to scan the North side of the runway for Unexploded Ordinance (UXOs) and mark them for later actions by EOD. EOD is currently disarming UXOs on the runway. The team is equipped with Samsung S20TE EUDs, ATAK, PRC-152 radios, rifles, gesture HMI device, and speech HMI through their radio push to talk, and they are carrying flags to mark the area. EOD has told them to be on the lookout for SPBE-D sub-munitions.
Actions:
The multi-mission Airmen/Warfighters conduct a UXO recognition course through JIT training to determine what the SPBE-D sub-munitions looks like. They use the touch screen as they are not moving or under duress.
The multi-mission Airmen/Warfighters spot what they believe is a sub-munition. The spotter uses gesture and speech HMI to plot a position in ATAK and marks a flag at a safe distance. The data is shared on the TAK serve and they continue mission.
The multi-mission Airmen/Warfighters come across an unconscious Airmen. They quickly pull up some medical training. While applying pressure to the wound with both hands, the team leader utilizes speech HMI to bring up a tourniquet video for their teammate to apply a tourniquet. They plot the point on ATAK for medical teams to respond and use JIT training to go through the rest of the medical steps. *Although they have basic medical training it is important to note they have been operating for a long period of time and this is their first time seeing an injured human. The JIT training offers reassurance they are doing the right things. *
The pair of multi-mission Airmen/Warfighters turn over the patient to the medical team and continue their mission. Upon completing the mission, they return to their original jobs until their shift is over.
Figure 3. DAF personnel are doing more with less, Multi-Mission Airmen/Warfighters need effective HMI options to accomplish JIT and succeed against our adversaries.
Contact S2RC to learn more about MATRIX or if you are interested in providing your feedback, input, and experience as an ACE Multi-Mission Airmen/Warfighter. Email MATRIX’s Principal Investigator Richard Stone directly at rstone@s2rc.com or the S2RC team at info@s2rc.com.