CSET Events

One Size Does Not Fit All

Building Trust Across a Diverse Range of AI Systems

CSET Senior Fellow Dr. Heather Frase discussed her research on effectively evaluating and assessing AI systems across a broad range of applications.

Recent advances in AI have garnered enormous public interest, with policymakers and citizens alike marveling at new systems’ capabilities and expressing concern about their risks. As society considers not only how to use these systems, but also how much to trust them, understanding how to effectively evaluate new AI will become even more critical. 

However, AI systems have a wide variety of functionality, capabilities, and outputs. They are also created using different tools, data modalities, and resources, which further complicates their assessment. Senior Fellow Dr. Heather Frase leads CSET’s research examining how to best evaluate AI systems and build trust among relevant stakeholders across a variety of sectors and applications. Dr. Frase and John Bansemer, Director of CSET’s CyberAI Project and Senior Fellow, held a webinar discussing her recently published research agenda outlining issues relevant to AI assessment and related hot-button topics.  

Recording and Discussion

Participants

Heather Frase, PhD is a Senior Fellow at Georgetown’s Center for Security and Emerging Technology (CSET), where she works on AI Standards and Testing. Prior to joining CSET, Heather spent eight years providing data analytics, computational modeling, Machine Learning (ML), and Artificial Intelligence (AI) support for Intelligence, Defense, and Federal contracts. Additionally, Heather spent 14 years at the Institute for Defense Analyses (IDA), supporting Director, Operational Test and Evaluation (DOT&E). At IDA she led analytic research teams to apply scientific, technological, and statistical expertise to develop data metrics and collection plans for operational tests of major defense systems, analyze test data, and produce assessments of operational effectiveness and suitability. She has a Ph.D. in Material Science from the California Institute of Technology and a BS in Physics from Miami University in Oxford Ohio.

John Bansemer is the Director of the CyberAI Project and Senior Fellow at Georgetown’s Center for Security and Emerging Technology (CSET). In addition to his work at CSET, he is an adjunct professor at Georgetown University’s School of Foreign Service.  Prior to joining CSET, John served in a variety of cyber, space and intelligence positions within the U.S. Air Force before retiring as a Lieutenant General. His last role was serving as the Assistant Director for National Intelligence, Partner Engagement, within the Office of the Director of National Intelligence. Prior to that assignment, he served as the Deputy Chief, Central Security Service, at the National Security Agency. He also held a variety of staff positions including on the Air Staff and the National Security Council staff. His joint experience includes serving as the director of intelligence at European Command. John holds a master’s degree in computer science from James Madison University and was a national defense fellow at Harvard University. He received his B.S. in Computer Science and Statistics from Roanoke College.