Karson Elmgren is a Research Analyst on the AI Assessment team.

Problems of AI safety are the subject of increasing interest for engineers and policymakers alike. This brief uses the CSET Map of Science to investigate how research into three areas of AI safety — robustness, interpretability and reward learning — is progressing. It identifies eight research clusters that contain a significant amount of research relating to these three areas and describes trends and key papers for each of them.

A new CSET report by Tim Rudner and Helen Toner introduces specification in machine learning as a key element in greater AI safety.

This paper is the fourth installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” outlined three categories of AI safety issues—problems of robustness, assurance, and specification—and the subsequent two papers described problems of robustness and assurance, respectively. This paper introduces specification as a key element in designing modern machine learning systems that operate as intended.

CSET's Director of Strategy Helen Toner sat down with National Defense to discuss AI failures from her and Zachary Arnold's CSET report "AI Accidents: An Emerging Threat."

Looking at AI in the automotive industry, a CSET report "AI Accidents: An Emerging Threat" identifies AI failures and calls for a greater emphasis on research and development to improve safety.

Senior Fellow Andrew Lohn discusses the threats and vulnerabilities AI systems, especially the Pentagon's, are susceptible to in his recent CSET report Poison in the Well

As the use of AI technology becomes more common, more problems arise prompting the need for policy and government responses, according to CSET's Helen Toner.

This paper is the third installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” described three categories of AI safety issues: problems of robustness, assurance, and specification. This paper introduces interpretability as a means to enable assurance in modern machine learning systems.

This paper is the second installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” described three categories of AI safety issues: problems of robustness, assurance, and specification. This paper introduces adversarial examples, a major challenge to robustness in modern machine learning systems.