From tablets to voice commands, Raytheon’s Department 22 is working out how best to join fighter pilots and drones into a collaborative team.
From tablets to voice commands, Raytheon’s Department 22 is working out how best to join fighter pilots and drones into a collaborative team.
If all goes according to plan, Air Force fighter pilots will one day fly into contested airspace accompanied by as many as five drones under their control to serve as weapons trucks, electronic warfare platforms, intelligence-gathering devices, and even decoys.
But research is already underway to figure out how to reduce the drone-controlling workload on pilots already confronted with a wide range of inputs and decisions to make in dynamic combat environments.
“The biggest thing [pilots] don’t want is to layer on an additional level of information management they have to deal with in the cockpit,” said Jason “Tex” Clark, a retired Air Force lieutenant colonel who leads the autonomous missions systems group at Raytheon Intelligence & Space’s new Department 22 strategic business unit. “We’re trying to put the human interface into the autonomy and make that as seamless, and as minimally burdensome as we can make it.”
Making that happen is one of the lines of effort of Department 22, launched in May to focus ”on rapidly developing next-generation military technologies aimed at widening the capability gap over the nation’s near-peer adversaries,” according to the company.
When it comes to teaming pilots and drones, the goal is to create a system that will make it so that future pilots will only need the same level of effort to communicate with the drones as they currently have now talking with human wingmen, said Clark, who served in leadership roles at various command levels and was an weapon systems officer instructor in the F-111 Aardvark and the F-15E Strike Eagle, accumulating over 2,500 hours of flight time.
“We are trying to think about autonomy just the way a four-ship lead deals with his three wingmen. We’re really trying to drive autonomy to get to the point where he can execute mission-level tasking into his CCA autonomous platforms, just like he interfaces with a wingman.”
That development work has been underway for a while, Steve Fino, a retired Air Force colonel who is now the Lead Engineer for Autonomy at Department 22, told The War Zone.
“In the program that we’re working on, we have a single pilot that’s been essentially controlling four UAVs in a combat-relevant mission,” said Fino, a former F-15C Eagle pilot who also served as an operational test pilot, flew the RQ-4 Global Hawk and served as the squadron commander for a joint U-2 and RQ-4 squadron in the Middle East.
Fino, in charge of efforts to make the autonomy work, said Department 22 calls the drones “little buddies.”
In a program for a customer he would not name, Fino said the pilots teamed with BQM-34S Firebee drones.
“We’ve done multiple design reference missions within that space. We’re tackling the most challenging components of that, which is ‘when I’m in the battlespace, how do I actually get the little buddies to do the things that I need them to do to make me as a pilot more combat effective?’”
The current interface has pilots using a tablet device to communicate with the drones, which Fino acknowledges is less than ideal because pilots don’t want to have to look at something else in the cockpit.
But Clark and Fino said it is also faster and easier, especially during the testing phase, to give a pilot a tablet as opposed to upgrading an airplane’s operational flight program (OFP) software that controls its avionics and also enables integration with external weapons. Kratos, another defense contractor that is working and competing directly in the CCA space has also leveraged tablets for command of ‘loyal wingmen’ type drones going back to the mid-2010s.
Division 22 is also researching a voice command system, similar to common AI-powered devices like Alexa or Siri, that has a lot of promise because pilot commands are fairly scripted.
“We’ve looked at doing voice control as well,” said Fino. “In reality, it’s pretty easy, because really what you’re mirroring is the way that pilots communicate with the human wingman right now, right? And so it’s actually a much simpler problem than you have with your Alexa or Siri, right? Because a pilot’s dictionary, their language, is very defined. It’s written in a tactics manual. And so I don’t have to worry about random words.”
There will of course be challenges, Fino acknowledged, such as when a pilot in a dogfight gets excited and doesn’t use the exact terminology or in situations with degraded communications.
“There is some complexity there, but the voice control is not a very challenging problem. And it provides an additional modality for the pilots to potentially control their little buddies.”
Whether future systems use a tablet, voice command, or ultimately an updated OFP where the commands show up on the pilot’s cockpit displays is up to the requirements of whatever program is selected, said Clark.
“That’s not a technical problem,” he said. “That’s a business relations program. It’s more programmatic.”
During last month’s Air and Space Forces Air, Space and Cyber conference at National Harbor, Maryland, Gen. Mark D. Kelly, head of Air Combat Command, talked about how he would like to see industry take an “iterative approach” to develop CCAs.
Clark said that is true not just for the technology, but also for building the trust of pilots worried about working with drones, including in life-or-death situations.
One iterative step forward for both technology and pilot trust will come next week, said Clark, as ACC hosts a “technical interchange meeting” to investigate “taking autonomous vehicles to replace the contract-funded companies that they pay to present red air forces for training for F-22 and F 35″ pilots, known as blue teams.
“I do not want to speak for [ACC], but our understanding of their view is that they see this [adversary air] capability as a great first incremental step to get operators comfortable from a trust standpoint because we could put autonomous vehicles up as a red air adversary in the same airspace against blue.”
The reason, he said, is that “now you at least have these autonomous vehicles living and flying and operating within the same confined airspace as the blue guys, and they can begin to see that [the drones] at least behave rationally. They’re not going to be a safety threat. And they can operate together.”
That, said Clark, is “a logical increment. And I think ACC is onto something there. And recently, quite frankly, they’re focused on that as the first step to engender the trust element.”
ACC told The War Zone it is not looking to replace contracted adversary air, or ‘ADAIR,’ capabilities with drones, but to augment them.
“There are no plans to replace contracted ADAIR with uncrewed platforms,” Capt. Erica L. Feehan, a command spokesperson, told The War Zone Friday morning. “Uncrewed ADAIR is an opportunity to improve in-flight training for Combat Air Force fighter aircrew by increasing the quantity and quality of adversary air platforms.”
The command is hosting the industry days “to build market research and discuss how semi-autonomous adversary air can complement existing adversary air capabilities and be a foundational building block for collaborative combat aircraft prototypes.”
Feehan concurred with Clark that having drones fly against human pilots will help them get a better understanding of those capabilities.
Having pilots compete against “initial operational prototypes with ADAIR functions” will put “new CCA technology into the hands of pilots to determine what works and what doesn’t,” said Feehan. “They will be able to understand control laws and variables therein, improve human-machine interfacing based on cognitive workload feedback, and determine information exchange requirements for assured command, control and communications. By accomplishing this through a rapid series of iterations, operators will help inform the future of CCA, gaining familiarity and trust in the performance of uncrewed platforms.”
But for defense contractors like Raytheon, work is already well underway.
“It is and it has been,” said Clark. Kendall “has publicly stated that he believes that technology has matured to the point where the government can go directly into an [engineering and manufacturing development] program on CCA, without a risk reduction phase or whatever else that might precede that. There has been ongoing work that supports that conclusion. So it’ll be interesting to see what happens next year.”
Contact the author: howard@thewarzone.com
