The Replicator effort by the U.S. Department of Defense (DoD) is intended to overcome some of the military challenges posed by China’s People’s Liberation Army (PLA). This blog post intends to both identify tradeoffs for the Department to consider as it charts the path for Replicator, and to provide a sense for the state of industry readiness to support.

Drawing from prior CSET research, this blog post describes different domestic and international initiatives the Chinese government and companies are pursuing to shore up investment in AI and meet China’s strategic objectives, as well as indicators to track their future trajectories.

How can we measure the reliability of machine learning systems? And do these measures really help us predict real world performance? A recent study by the Stanford Intelligent Systems Laboratory, supported by CSET funding, provides new evidence that models may perform well on certain reliability metrics while still being unreliable in other ways. This blog post summarizes the study’s results, which suggest that policymakers and regulators should not think of “reliability” or “robustness” as a single, easy-to-measure property of an AI system. Instead, AI reliability requirements will need to consider which facets of reliability matter most for any given use case, and how those facets can be evaluated.