Problems of AI safety are the subject of increasing interest for engineers and policymakers alike. This brief uses the CSET Map of Science to investigate how research into three areas of AI safety — robustness, interpretability and reward learning — is progressing. It identifies eight research clusters that contain a significant amount of research relating to these three areas and describes trends and key papers for each of them.
This paper is the fourth installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” outlined three categories of AI safety issues—problems of robustness, assurance, and specification—and the subsequent two papers described problems of robustness and assurance, respectively. This paper introduces specification as a key element in designing modern machine learning systems that operate as intended.
CSET's Director of Strategy Helen Toner sat down with National Defense to discuss AI failures from her and Zachary Arnold's CSET report "AI Accidents: An Emerging Threat."
Looking at AI in the automotive industry, a CSET report "AI Accidents: An Emerging Threat" identifies AI failures and calls for a greater emphasis on research and development to improve safety.
Senior Fellow Andrew Lohn discusses the threats and vulnerabilities AI systems, especially the Pentagon's, are susceptible to in his recent CSET report Poison in the Well
As the use of AI technology becomes more common, more problems arise prompting the need for policy and government responses, according to CSET's Helen Toner.
This paper is the third installment in a series on “AI safety,” an area
of machine learning research that aims to identify causes of
unintended behavior in machine learning systems and develop
tools to ensure these systems work safely and reliably. The first
paper in the series, “Key Concepts in AI Safety: An Overview,”
described three categories of AI safety issues: problems of
robustness, assurance, and specification. This paper introduces
interpretability as a means to enable assurance in modern machine
learning systems.
This paper is the second installment in a series on “AI safety,” an
area of machine learning research that aims to identify causes of
unintended behavior in machine learning systems and develop
tools to ensure these systems work safely and reliably. The first
paper in the series, “Key Concepts in AI Safety: An Overview,”
described three categories of AI safety issues: problems of
robustness, assurance, and specification. This paper introduces
adversarial examples, a major challenge to robustness in modern
machine learning systems.
This website uses cookies.
To learn more, please review this policy. By continuing to browse the site, you agree to these terms.
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.