Worth Knowing
Governor Newsom Vetoes Sweeping AI Regulation, SB 1047: California Governor Gavin Newsom vetoed a closely watched AI regulation, SB 1047, that would have implemented some of the country’s most extensive safety protocols for powerful AI systems. As we covered last month, California’s status as home to many of the world’s top AI developers meant the bill’s progress was closely watched and hotly contested. In a statement, Newsom wrote that while the bill was “well-intentioned,” it was too focused on the largest models and ignored the risks posed by smaller models or systems deployed in particularly risky environments. But observers also pointed to robust lobbying efforts by tech and venture capital firms, as well as opposition from prominent members of California’s congressional delegation, as key factors in Newsom’s decision. The bill had undergone significant changes since its introduction in response to industry feedback, earning it the support of some major AI developers like Anthropic and Tesla, SpaceX, and xAI CEO Elon Musk. But others, like San Francisco-based OpenAI, raised concerns about the bill’s impact on innovation and argued that AI regulation was best left to the federal government. While Newsom vetoed SB 1047, he did sign a number of more targeted AI bills, including AB-2013, which will require generative AI companies to disclose information about their training data, and SB-942, a law that will require watermarking for AI-generated content.
- More: Senator Wiener Responds to Governor Newsom Vetoing Landmark AI Bill | Governor Newsom announces new initiatives to advance safe and responsible AI, protect Californians
- More: A.I. Pioneers Call for Protections Against ‘Catastrophic Risks’ | OpenAI’s Advantage over Rival AI Companies Vanishes, Report Finds
Government Updates
DOD Announces Replicator 2 — Counter-Drone Defenses the Focus: In a September 27 memo, Secretary of Defense Lloyd Austin announced a second “Replicator” initiative aimed at rapidly developing and fielding counter-drone technologies. The move comes just over a year after the launch of the original Replicator program, which sought to deploy thousands of attritable autonomous systems across multiple domains within 24 months (according to Austin’s memo, Replicator 1 is on track). While Replicator 1 focused on offensive capabilities, Replicator 2’s focus points to the DOD’s growing concern about the difficulty of defending against the small, cheap drones that have proven particularly effective in recent conflicts, including in Ukraine and the Middle East. In January, an Iranian-made drone killed three U.S. service members and injured dozens more in an attack on a base in Jordan. According to the Washington Post, a U.S. military assessment indicated that the low-flying drone had likely gone undetected, underscoring the need for more robust Counter-Small Unmanned Aerial Systems (C-sUAS) capabilities. Austin’s memo directs the Deputy Secretary of Defense and Vice Chairman of the Joint Chiefs of Staff to develop a Replicator 2 plan for inclusion in the FY2026 budget request, with the expectation to deliver “meaningfully improved” C-sUAS protection for critical assets within 24 months of congressional funding approval.
Commerce Considering Country-Specific Chip Export Caps: Last month, the Commerce Department’s Bureau of Industry and Security created a mechanism to ease exports of advanced AI chips to vetted overseas data centers, but reports indicate that country-specific export license caps could be on the horizon. Under the updated rule, data centers can apply for Validated End User (VEU) status, allowing them to receive chips — following a “rigorous review process” and after agreeing to facilitate on-site inspections and abide by new reporting requirements — under a general authorization rather than requiring suppliers to obtain individual licenses for each shipment. While the updated rule would ease chip shipments in some cases, Bloomberg reports that country-specific chip export caps could soon be coming soon. According to the report, the Biden administration is particularly focused on managing exports to Persian Gulf countries, which have had a growing appetite for AI chips in recent years. But policymakers have become increasingly concerned that some of those Middle East-bound chips are actually destined for China, one of the primary targets of the Commerce Department’s stringent semiconductor controls. Taken together, the updated VEU process and the rumored country-specific caps appear to be an attempt to thread the needle: maintaining the lucrative flow of high-end chips while ensuring they don’t end up in the wrong hands.
FTC Cracks Down on AI Over-Promising: The Federal Trade Commission took action against several companies it accused of making deceptive AI claims to consumers, the agency announced. The five targets of the law enforcement sweep — dubbed “Operation AI Comply” — include multiple companies that promised to use AI to help consumers build passive income-generating online storefronts. Also targeted was DoNotPay, a company that made headlines last year when it tried to get a defendant to use its service — hooked up via wireless headphones and smart glasses — to fight a traffic ticket in court. The FTC accused the company of misleading consumers with its claim to offer “the world’s first robot lawyer.” The company agreed to pay nearly $193,000 to settle the case. Misleading AI claims have been on the FTC’s radar for some time — as we covered last year, the agency warned companies to “keep [their] AI claims in check” and outlined the types of deceptive marketing practices it was on the lookout for. FTC Chair Lina Khan continues to cite AI as one of the areas her agency is watching most closely, so it wouldn’t be surprising to see similar crackdowns take place in the future.
OMB Issues Guidance on Responsible AI Acquisition: Earlier this month, the Office of Management and Budget issued guidance on responsible AI acquisition by the federal government. The guidance (fact sheet available here) lays out specific requirements and suggested best practices for federal agencies to follow when acquiring AI. The memo emphasizes three strategic goals — “managing AI risks and performance,” “promoting a competitive AI market with innovative acquisition,” and “ensuring collaboration across the federal government” — and builds on OMB guidance released earlier this year that detailed how federal agencies could and should use AI. Both sets of guidelines were mandated as part of President Biden’s October 2023 executive order on AI. The earlier memo placed particular emphasis on mitigating risks posed by “safety-impacting” and “rights-impacting” AI and required agencies to develop minimum practices for deploying these types of systems. Agencies have until December 1, 2024, to institute those practices.
What’s New at CSET
REPORTS
- Fueling China’s Innovation: The Chinese Academy of Sciences and Its Role in the PRC’s S&T Ecosystem by Cole McFaul, Hanna Dohmen, Sam Bresnick, and Emily S. Weinstein
- Securing Critical Infrastructure in the Age of AI by Kyle Crichton, Jessica Ji, Kyle Miller, John Bansemer, Zachary Arnold, David Batz, Minwoo Choi, Marisa Decillis, Patricia Eke, Daniel M. Gerstein, Alex Leblang, Monty McGee, Greg Rattray, Luke Richards, and Alana Scott
- Through the Chat Window and Into the Real World: Preparing for AI Agents by Helen Toner, John Bansemer, Kyle Crichton, Matthew Burtell, Thomas Woodside, Anat Lior, Andrew Lohn, Ashwin Acharya, Beba Cibralic, Chris Painter, Cullen O’Keefe, Iason Gabriel, Kathleen Fisher, Ketan Ramakrishnan, Krystal Jackson, Noam Kolt, Rebecca Crootof, and Samrat Chatterjee
PUBLICATIONS
- Foreign Policy: Russia’s Global Information Operations Have Grown Up by Josh Goldstein and Renée DiResta
- Lawfare: Could AI Lead to the Escalation of Conflict? PRC Scholars Think So by Sam Bresnick
- FedScoop: AI Won’t Pause for the Election, and AI Regulation Shouldn’t Either by Owen Daniels and Jack Corrigan
- Council on Foreign Relations: Advancing Inclusive Innovation: Meeting the Surge in Demand for STEM Jobs by Matthias Oschinski
- CSET: How I Won DEF CON’s Generative AI Red-Teaming Challenge by Colin Shea-Blymyer
- CSET: Revisiting AI Red-Teaming by Jessica Ji and Colin Shea-Blymyer
- CSET: The Future of Drones in Ukraine II: A Report from the NATO-Ukraine Defense Innovators Forum by Kyle Miller
EMERGING TECHNOLOGY OBSERVATORY
- The Emerging Technology Observatory is now on Substack! Sign up for the latest updates and analysis.
- AI Leaders in the S&P 500: Perspectives from PARAT
- Beyond TSMC and EUV: Five Key Points on Chip Chokepoints
EVENT RECAPS
- On September 19, 2024, CSET’s Helen Toner hosted OpenMined’s Andrew Trask and Irina Bejan for a discussion about how privacy-enhancing technology infrastructure can better support AI innovation.
IN THE NEWS
- Associated Press: In global game of influence, China turns to a cheap and effective tool: fake news (Didi Tang and David Klepper quoted Jaret Riddick)
- Marketplace: Tech Bytes – Week in Review: Senate weighs AI regulation, Instagram launches teen accounts and AirPods aid the hard of hearing (MarketPlace quoted Helen Toner )
- Politico: The Nobels throw AI a coming-out party (Derek Robertson quoted Helen Toner)
- Roll Call: China hawks prep tech investment screening measure (Gopal Ratnam cited the CSET report U.S. Outbound Investment into Chinese AI Companies)
- Semafor: America’s emerging AI leaders: Amazon, Walmart and Wells Fargo? (Mizy Clifton quoted Zachary Arnold and cited the ETO blog post, AI Leaders in the S&P 500: Perspectives from PARAT)
- The Atlantic: The AI Doomers Are Licking Their Wounds (Damon Beres quoted Helen Toner)
- The National Law Review: Governor Newsom Vetoes SB 1047, Rejecting AI Whistleblower Protections (Jesse Broad-Cavanagh quoted Helen Toner)
- The Wire China: Helen Toner on Setting the Rules for AI (Eliot Chen spoke to Helen Toner)
What We’re Reading
Report: State of AI Report 2024, Nathan Benaich and Air Street Capital (October 2024)
Essay: The Intelligence Age, Sam Altman (September 2024)
Essay: Machines of Loving Grace: How AI Could Transform the World for the Better, Dario Amodei (October 2024)