CSET Events

Building Trust Through Testing

CSET hosted WestExec Advisors' Michèle Flournoy and Gabrielle Chefitz, together with Avril Haines, for a discussion of their new report outlining how the Department of Defense can adapt its test, evaluation, validation and verification (TEVV) infrastructure for artificial intelligence. The authors were joined by Johns Hopkins Applied Physics Laboratory's Ashley Llorens, the Joint Artificial Intelligence Center's Jane Pinelis, and moderator Richard Danzig.

Download Full Report

Advances in artificial intelligence  and machine learning (AI/ML) have the potential to transform fundamentally how the U.S. military deters and fights, and how the Department of Defense operates. Critical to realizing this potential is DOD’s ability to adapt its Test, Evaluation, Validation and Verification (TEVV) enterprise for AI/ML. In a new report, WestExec Advisors’ Michèle Flournoy and Gabrielle Chefitz, together with Avril Haines, argue that a robust system for AI/ML TEVV is crucial to increasing trust in and, consequently, accelerating the deployment of these systems on a timeline consistent with the rate of innovation, operational need and U.S. ethics and principles. The authors outline how the Defense Department’s TEVV enterprise – processes, policies, research, infrastructure and personnel — can be adapted to support the deployment of trustworthy, effective and values-based AI-enabled systems in the United States and abroad. CSET hosted a panel discussion with the authors, alongside Johns Hopkins Applied Physics Laboratory’s Ashley Llorens, the Joint AI Center’s Dr. Jane Pinelis, and moderator Richard Danzig, on the report and the broader questions framing the AI TEVV challenge.

Remarks and Discussion

Participants

Michèle Flournoy is Co-Founder and Managing Partner of WestExec Advisors, and former Co-Founder and Chief Executive Officer of the Center for a New American Security (CNAS), where she currently serves on the board. Michèle served as the Under Secretary of Defense for Policy from February 2009 to February 2012. She was the principal advisor to the Secretary of Defense in the formulation of national security and defense policy, oversight of military plans and operations, and in National Security Council deliberations. In the mid-1990s, she served as Principal Deputy Assistant Secretary of Defense for Strategy and Threat Reduction and Deputy Assistant Secretary of Defense for Strategy. Michèle earned a bachelor’s degree in Social Studies from Harvard University and a master’s degree in International Relations from Balliol College, Oxford University, where she was a Newton-Tatum scholar.

Avril Haines served as Assistant to the President and Principal Deputy National Security Advisor to President Obama and as Deputy Director of the Central Intelligence Agency. Avril also held a number of senior legal positions in the government, including Legal Adviser to the National Security Council and Assistant Legal Adviser for Treaty Affairs. Avril received her bachelor’s degree in Physics from the University of Chicago and a law degree from Georgetown University Law Center.

Gabrielle Chefitz serves as a Senior Associate at WestExec Advisors. Previously, she was a Research Assistant to the Director of the Project on the Middle East Peace Process at the Washington Institute for Near East Policy. Prior to joining the Washington Institute, Gabrielle supported the work of the Senate Foreign Relations Committee as a member of the legislative team covering the Middle East and North Africa. Gabrielle is also the co-author of “Debtbook Diplomacy: China’s Strategic Leveraging of its Newfound Economic Influence and the Consequences for U.S. Foreign Policy,” published by Harvard’s Belfer Center for Science and International Affairs. Gabrielle earned a master’s degree in Public Policy from the Harvard Kennedy School of Government and a bachelor’s degree from Northwestern University’s Medill School of Journalism.

Ashley J. Llorens is chief of the Intelligent Systems Center at the Johns Hopkins Applied Physics Laboratory, where he directs research and development in artificial intelligence, robotics and neuroscience. Previously, he led projects for the Office of Naval Research that designed, developed and transitioned machine learning algorithms for U.S. Navy systems. Ashley has served as a subject matter expert for AI in strategic studies for the Defense Science Board, the Secretary of Energy Advisory Board and the Naval Studies Board. He earned an M.S. in Electrical Engineering from the University of Illinois at Urbana-Champaign. 

Dr. Jane Pinelis is the Chief of Test, Evaluation and Assessment at the Department of Defense Joint Artificial Intelligence Center. Previously, Jane served as the Director of Test and Evaluation for the Under Secretary of Defense for Intelligence’s Algorithmic Warfare Cross-Functional Team, better known as Project Maven. Jane also previously worked for the Institute for Defense Analyses and the United States Marine Corps. She earned a B.S. in Statistics, Economics and Mathematics, an M.A. in Statistics and a Ph.D. in Statistics, all from the University of Michigan. 

Dr. Richard Danzig is a Senior Advisor to the Johns Hopkins Applied Physics Laboratory and Chair of the Advisory Panel for the Idaho National Laboratories’ Innovation Center. He previously served as Chairman of the Board of the Center for Strategic and Budgetary Assessments, a member of the President’s Intelligence Advisory Board and the Secretary of Defense’s Defense Policy Board. Dr. Danzig served as Under Secretary of the Navy from 1993 to 1997 and Secretary of the Navy from 1998 to January 2001.

Media Coverage

Former national security leaders want DOD to work more collaboratively on AI testing, FedScoop, October 7, 2020.