Washington, DC — While the U.S. Department of Defense has an established testing and evaluation enterprise, it is not well-suited to the intricacies of artificial intelligence, WestExec Advisors’ Managing Partner and former Under Secretary of Defense Michèle Flournoy said during a webinar hosted Tuesday by the Center for Security and Emerging Technology (CSET) at Georgetown University.
Flournoy was joined by Avril Haines and Gabrielle Chefitz, along with Johns Hopkins Applied Physics Laboratory’s Ashley Llorens and the Joint Artificial Intelligence Center’s Dr. Jane Pinelis, in a panel discussion led by Dr. Richard Danzig, who served as secretary of the U.S. Navy. The speakers discussed the need for a modern Test, Evaluation, Verification and Validation infrastructure that accounts for the unique challenges in fielding emerging technologies, particularly in artificial intelligence, also known as AI.
WestExec released its new report, “Building Trust Through Testing: Adapting DOD’s Test & Evaluation, Validation & Verification (TEVV) Enterprise for Machine Learning Systems, Including Deep Learning Systems,” on October 5.
The report cites brittleness, data availability and complex system integration as some of the challenges DOD faces in fielding new AI technologies.
Flournoy pointed out that no matter how well-designed an AI system may be, “The adversary gets a vote. And adversaries will try to poison the data, cause the system to malfunction, and so forth.” To remedy this reality, she noted, “DOD and the intelligence community need to invest in red-teaming” that will help uncover potential vulnerabilities.
The authors called on the Defense Department to invest in priority research areas with industry and academia, establish a TEVV coordinating body within the Office of Secretary of Defense and develop a tailored, risk-based framework for machine learning and deep learning testing and safety.
Chefitz noted, “DOD should really look at some of the private-sector best practices” including the use of modeling and simulation and standards for safety-critical systems, learning from experiences that, in some cases, the U.S. military may not wish to repeat – such as incidents involving commercially-developed self-driving vehicles.
As head of Testing and Evaluation at JAIC, Dr. Pinelis highlighted the Pentagon’s efforts to ensure that AI-enabled systems and the people working with them are aligned in support of the right values. “We’re frequently designing ways to automate tools to help humans,” she noted, “but it’s important as well to consider how the humans might behave, and ensure that they do so in a way that’s consistent with ethical principles.”
Llorens addressed both the matters of safety and trust, a key factor in developing and fielding successful AI-enabled technology: “One of the things that people don’t realize is that it can be hard to safely test autonomous systems, even on a test range,” he said, adding that recent technological advancements are making it more feasible. “It’s not magic, but you do have to figure out which failsafe systems need to be built in.”
In summing up, Haines noted that the Defense Department’s TEVV enterprise will need to be carried in a way that both enables effectiveness and efficiency along with allowing a certain amount of experimentation.
“This is a space where we’re really going to have to meet high standards,” Haines said. “The question is, can we do this in a way that actually promotes innovation as opposed to stifling it through the process?”