Artificial intelligence is vulnerable to cyber attacks. Machine learning systems—the core of modern AI—are rife with vulnerabilities. Attack code to exploit these vulnerabilities has already proliferated widely while defensive techniques are limited and struggling to keep up. Machine learning vulnerabilities permit hackers to manipulate the machine learning systems’ integrity (causing them to make mistakes), confidentiality (causing them to leak information), and availability (causing them to cease functioning).
These vulnerabilities create the potential for new types of privacy risks, systemic injustices such as built-in bias, and even physical harms. Developers of machine learning systems—especially in a national security context—will have to learn how to manage the inevitable risks associated with those systems. They should expect that adversaries will be adept at finding and exploiting weaknesses. Policymakers must make decisions about when machine learning systems can be safely deployed and when the risks are too great.
Attacks on machine learning systems differ from traditional hacking exploits and therefore require new protections and responses. For example, machine learning vulnerabilities often cannot be patched the way traditional software can, leaving enduring holes for attackers to exploit. Even worse, some of these vulnerabilities require little or no access to the victim’s system or network, providing increased opportunity for attackers and less ability for defenders to detect and protect themselves against attacks.
Accordingly, this paper presents four findings for policymakers’ consideration:
- Machine learning introduces new risks: Using machine learning means accepting new vulnerabilities. This is especially true in the context of national security, but also in critical infrastructure, and even in the private sector. However, this does not mean machine learning should be prohibited. Rather, it is incumbent upon policymakers to understand the risks in each case and decide whether they are outweighed by the benefits.
- New defenses may only offer short-term advantage: Attackers and defenders of machine learning systems are locked in a rapidly evolving cat-and-mouse game. Defenders appear to be losing; their techniques are currently easily defeated and do not seem well-positioned to keep pace with advances in attacks in the near future. Still, defensive measures can raise the costs for attackers in some narrow instances, and a proper understanding of machine learning vulnerabilities can aid defenders in mitigating risk. Nonetheless, the effectiveness of defensive strategies and tactics will vary for years and will continue to fail at thwarting more sophisticated attacks.
- Robustness to attack is most likely to come from system-level defenses: Given the advantages that attackers have, for machine learning systems to function in high-stakes environments, they must be built in with greater resilience than is often the case today. To aid this effort, policymakers should pursue approaches for providing increased robustness, including the use of redundant components and ensuring opportunities for human oversight and intervention when possible.
- The benefits of offensive use often do not outweigh the costs: The United States could employ the types of attacks described in this primer to good effect against adversaries’ machine learning systems. These offensive techniques could provide another valuable arrow in the U.S. national security community’s quiver and might help prevent adversaries from fielding worrisome AI weapons in the first place. On the other hand, the United States can lead by setting norms of restraint. The United States must also be cautious to ensure its actions do not alienate the community that is developing these technologies or the public at large who rely on machine learning.
Machine learning has already transformed many aspects of daily life, and it is easy to see all that the technology can do. It likewise offers the allure of reshaping many aspects of national security, from intelligence analysis to weapons systems and more. It can be hard, however, to perceive machine learning’s limitations, especially those—like its susceptibility to hacking—that are most likely to emerge in highly contested environments. To better understand what the technology can and cannot do, this primer introduces the subject of machine learning cybersecurity in a detailed but non-technical way. It provides an entry point to the concepts and vocabulary needed to engage the many important issues that arise and helps policymakers begin the critical work of securing vital systems from malicious attacks.