On March 2, the US Navy pulled an F-35C out of the ocean. The $94.4 million jet came in hard for a landing on January 24, then skidded on the deck, injuring sailors before collapsing and falling into the sea. The pilot ejected and survived , but the incident raises an ominous question about flight operations: What can the military do to ensure pilots land safely as many times as they take off and help all airmen avoid the mistakes of their peers?
The F-35C is a navy plane, and aircraft carrier landings are notoriously difficult. But no branch of the military is immune to accidents, and both Marines and Aviation crashed planes this year. Post-crash investigations, drawn from aircraft recorded avionics and telemetry data, can reveal specific causes of error, from mechanical failure to choices made by pilots.
In 2020, the Air Force turned to artificial intelligence to detect unusual flight patterns during training, before they became a costly, even tragic mistake. To better understand outliers in flight patterns, the Air Force is working with Crowdbotics, an artificial intelligence/machine learning company, to analyze and process data that planes are already collecting. This data processing and analysis is done with custom software, which the company and the Air Force call a specific tool.
“Fighter jets are one of the biggest investments of the US military. They are super advanced technologies that a human being is attached to,” says Anand Kulkarni, CEO of Crowdbotics. The fighters are “extremely well equipped with data-producing machines, and all that data is more or less discarded at the end of each flight”.
Planes capture this data, avionics and flight telemetry, several times per second, creating a record of time, speed and position. This is a massive dataset produced by every flight, and difficult for humans to process without the aid of data analysis tools. Currently, this data can be used in debriefings, where pilots sit down after a mission and watch the flights unfold on a monitor over the course of a few hours. That’s enough time to detect any significant changes, like a jet suddenly breaking up with the formation, but the data has the potential to reveal much more.
“Usually at the end of the debriefing, the student keeps a few notes, but we erase our tapes,” says Major Mark Poppler of the 4th F-15E Training Squadron. “We clear our shot sheets and all data is cleared. This is how this project was born. My predecessor recognized this and thought, given the advances we’ve made in computing over the past few decades, how can we automate much of this process to make debriefing more efficient and then eliminate less data? »
Currently, Crowdbotics’ work with the Air Force is limited to the F-15E training squadron at Seymour Johnson Air Force Base in North Carolina. The program is in the Phase 2 stage, with much more of its potential awaiting success and expansion to other aircraft.
Take-offs and landings
Even limited to F-15E pilot training flights alone, Crowdbotics’ prototype scan tool appears to be able to catch performance deviations before they become a major problem. Consider the work of landing an airplane, something every flight should include, which are ideally routine events, not disasters. Data collected by planes and processed through the prototype tool built by Crowbotics can see if something is wrong.
“How fast you execute an approach and landing in a Strike Eagle [an F-15E] depends on your weight of fuel,” says Poppler. “And so [the prototype tool] can actually calculate the final approach speed based on your fuel weight. And then you grade by the same standards that an evaluator pilot would grade you: was your approach quick? Did you land quickly? Did you land on the correct part of the runway? »
[Related: Everything to know about the Air Force’s new fighter jet, the F-15EX Eagle II]
These are all questions that are easy to answer with avionics data, but difficult for an instructor to understand in another aircraft or on the ground. During training, if a pilot consistently takes too steep a landing angle, the data could catch him before the instructor, and the instructor could adjust accordingly. By processing flight data from the same pilots over time and across a program, the Air Force could use the tool to gauge how an individual is improving. By examining the flight data collected in a squadron, the data can detect if a pilot is doing something different from everyone else.
“The way I look at it and the way the software looks at it, whenever we see outliers, we don’t necessarily start out saying it’s good or bad,” says Kulkarni, of Crowdbiotics. “We say ‘it’s different from the book’ or different from the norm of what most pilots do.” But different in this case could mean worse or better.
Error detection is important to maintain pilot and aircraft safety. Spotting innovation allows new techniques to spread much faster than waiting for a pilot to finish their career and return as an instructor. It has the potential to shift knowledge transfer from a generational exchange to a peer-to-peer exchange.
The execution of all things
Crowdbotics’ contract with the Air Force is formally for “Standardizing and optimizing USAF pilot training with machine learning and in-depth maneuver analysis.” The tool can analyze flights in a simulator in the same way it can analyze data from flight recorders and provide a better understanding of normal operations and flight analysis deviations. With optimization, it can also break away from a one-size-fits-all approach to teaching pilots. Each year, the training squadron hosts 40 to 50 pilots, hoping to get as many as possible to serve on 10-year Air Force engagements. It’s a kind of batch training that lives on in giant bureaucratic organizations like the military.
By using specific data from each pilot, the Air Force can instead better allocate instructor time among all those pilots, perhaps identifying those who need more help, and then focusing on them rather than teaching top performers.
The data can help find which pilots need help, which pilots have already demonstrated proficiency, and which trainees may be best suited to a different aircraft. It’s still early in the program, but having the data means the Air Force can use real flight analytics to assess pilot readiness, supplement instructor evaluations, and hopefully promise better results. than existing methods.
This can also apply to preparing pilots for missions. If a specific mission calls for the F-15Es to be used as ground bombers, a commander could consult their squadron’s record and choose pilots based on how well they have flown those missions in the past. (The F-15 was originally designed as a purely air-to-air fighter, but the two-seat F-15E variant is designed to attack ground targets, while retaining the class’s combat capability.)
Right now, the prototype “proves that they can recognize and create maneuvers,” says Poppler. “What we chose to start with is really one ship [one aircraft]unclassified, maneuvers, takeoffs, landings, instrument approaches, loops, a bit more aerobatics, things like that.
In the future, the tool could be used to examine the more complex maneuvers that appear in air-to-air combat training. For now, the Crowdbiotics tool is building a dataset to capture what regular flight looks like, what outlier flights look like, and if anything can be done to train pilots to achieve the best results in their planes.
“So if you’re flying and you come in too hot, if your angle of attack is too high an angle of attack at the end, or if in a combat situation you’ve responded poorly to an attacker or a defender , or in a way that was sub-optimal, the system will tell you. It will identify what you should have done, and then tell you exactly what the deviation from it was,” Kulkarni explains. it is simulated or real data, the only difference between simulator data and real data is that you get more data points in the real data stream.”
If the technology proves useful to the Air Force, it could expand to other aircraft types, and Crowdbotics is also willing to explore it for commercial flight analysis. Flight is a data-rich activity, and pilots can earn by learning from that data before they are in an accident.