Explainability and interpretability are often cited as key characteristics of trustworthy AI systems, but it is unclear how they are evaluated in practice. This report examines how researchers evaluate their explainability and interpretability claims in the context of AI-enabled recommendation systems and offers considerations for policymakers seeking to support AI evaluations.

This follow-up report builds on the foundational framework presented in the March 2024 CSET issue brief, “An Argument for Hybrid AI Incident Reporting,” by identifying key components of AI incidents that should be documented within a mandatory reporting regime. Designed to complement and operationalize our original framework, this report promotes the implementation of such a regime. By providing guidance on these critical elements, the report fosters consistent and comprehensive incident reporting, advancing efforts to document and address AI-related harms.

This paper presents two new methods for identifying research relevant to emerging technology. The authors developed and deployed technology topic classification and targeted research field scoring over a corpus of scientific literature to identify research relevant to cybersecurity, LLM development, and chips fabrication and design—expanding CSET’s existing set of topic classifications for AI, computer vision, NLP, robotics, and AI safety. The paper summarizes motivation, methods, and results.

Data Snapshots are informative descriptions and quick analyses that dig into CSET’s unique data resources. This three-part series uses data from a variety of sources to track how three cloud providers—Amazon, Alphabet, and Microsoft—distribute their financial resources to create and sustain demand for their cloud services. By investing in data centers & workforce training, the large tech platforms of Amazon, Google, and Microsoft draw developers, companies, and governments to their tools & services.

Data Snapshots are informative descriptions and quick analyses that dig into CSET’s unique data resources. This three-part series uses data from a variety of sources to track how three cloud providers—Amazon, Alphabet, and Microsoft—distribute their financial resources to create and sustain demand for their cloud services. By investing in data centers & workforce training, the large tech platforms of Amazon, Google, and Microsoft draw developers, companies, and governments to their tools & services.

Data Snapshots are informative descriptions and quick analyses that dig into CSET’s unique data resources. This three-part series uses data from a variety of sources to track how three cloud providers—Amazon, Alphabet, and Microsoft—distribute their financial resources to create and sustain demand for their cloud services. By investing in data centers & workforce training, the large tech platforms of Amazon, Google, and Microsoft draw developers, companies, and governments to their tools & services.

The U.S. Army’s 18th Airborne Corps can now target artillery just as efficiently as the best unit in recent American history—and it can do so with two thousand fewer servicemembers. This report presents a case study of how the 18th Airborne partnered with tech companies to develop, prototype, and operationalize software and artificial intelligence for clear military advantage. The lessons learned form recommendations to the U.S. Department of Defense as it pushes to further develop and adopt AI and other new technologies.

CSET submitted the following comment in response to a Request for Information (RFI) from the Department of Commerce regarding 89 FR 27411.

How to govern artificial intelligence is a concern that is rightfully top of mind for lawmakers and policymakers.To govern AI effectively, regulators must 1) know the terrain of AI risk and harm by tracking incidents and collecting data; 2) develop their own AI literacy and build better public understanding of the benefits and risks; and 3) preserve adaptability and agility by developing policies that can be updated as AI evolves.

Policy and research communities strive to mitigate AI harm while maximizing its benefits. Achieving effective and trustworthy AI necessitates the establishment of a shared language. The analysis of policies across different countries and research literature identifies consensus on six critical concepts: accountability, explainability, fairness, privacy, security, and transparency.