Worth Knowing
OpenAI’s New Reasoning Model and the Next Stage in AI Development: Last Thursday, OpenAI introduced its latest model, dubbed “OpenAI o1.” While notable for its performance on a range of difficult tasks and benchmarks — the San Francisco-based AI lab says the model significantly outperformed its flagship GPT-4o model on competitive math and coding benchmarks — o1 has many excited because of what it could portend for the next stage in AI development. According to o1’s accompanying system card, OpenAI used large-scale reinforcement learning to train o1 to perform “chain of thought” reasoning, enabling it to deliberate its way through complex problems and execute more sophisticated strategies. While OpenAI hasn’t published much explicit information on how this training worked, observers think the model was trained on human-produced examples of step-by-step reasoning, enabling the model to better understand the process of complex problem-solving (this kind of process-focused training was detailed in a 2023 paper by OpenAI researchers). Importantly, this reasoning process, which occurs when the model is generating outputs (known as “inference”), corresponds to the amount of time and computing power dedicated to it — the more time and computing power, the better the quality of the output. Much of the recent investment in AI is due to a similar scaling property: the more computing power AI developers devote to training an AI model, the more sophisticated the model. With o1 demonstrating a similar scaling law for inference-time, OpenAI seems to have set the stage for the next phase in AI development: massive up-front investment in AI training coupled with heavy continuous spending on inference. Both o1 and o1-mini — a faster, cheaper-to-use version of the reasoning model — are available to paid ChatGPT users and through OpenAI’s API.
- More: Scaling: The State of Play in AI | OpenAI co-founder Sutskever’s new safety-focused AI startup SSI raises $1 billion
- More: Mark Zuckerberg and Daniel Ek on Why Europe Should Embrace Open-Source AI | How the Draghi report could affect future EU-U.S. relations | Far-left, far-right MEPs slam Draghi report, while center offers support
- TSMC’s new Arizona chipmaking facility has achieved production yields comparable to its fabs in Taiwan, according to a Bloomberg report. TSMC — incentivized in part by billions in CHIPS Act subsidies — has committed more than $65 billion to build out its Arizona operations. Over the last year, concerns seemed to be growing that the Arizona fabs might fail to meet their production targets, with observers pointing to worker training and cultural differences as potential obstacles. The news that TSMC’s U.S. facilities are matching Taiwanese yield rates at a relatively advanced (though not bleeding edge) 4 nm process node should help ease those concerns and bolster confidence in the potential of high-end U.S. chipmaking.
- Intel plans to establish its semiconductor foundry business as an independent subsidiary, the company announced earlier this week. The U.S. chipmaker is one of the few companies that still designs and manufactures its own semiconductors. Once the dominant model, the rise of contract chipmakers like TSMC gave rise to so-called “fabless” companies like Nvidia, which design specialized chips and contract out their manufacturing to highly specialized foundries. While Intel’s foundry business will still be under the Intel umbrella, the move could be a precursor to spinning off Intel Foundry into a separate company — something Intel competitor AMD did with its chipmaking arm (which became GlobalFoundries) in 2009.
- More: Intel delays Germany, Poland chip factories for two years | U.S. and Japan near deal to curb chip technology exports to China
Government Updates
California Governor Weighs Signing Far-Reaching AI Regulation SB 1047: Last month, the California State Legislature passed SB 1047, a hotly debated AI regulation that, if signed into law by Governor Gavin Newsom, could become one of the most impactful in the United States. As home to leading AI companies like Google, OpenAI, and Anthropic, California’s regulations could have global implications. The bill would set standards for developers of the most powerful AI models — those that use computing power in excess of 10^26 floating-point operations or that cost more than $100 million to train — and mandate safety precautions, including pre-deployment testing and post-deployment monitoring. It also establishes whistleblower protections and empowers California’s Attorney General to take legal action against negligent AI developers. But in response to input from AI developers, the bill was stripped of some of its most controversial features, including criminal penalties and a new regulatory body that would have overseen so-called “frontier models.” While Anthropic has offered tentative support following these modifications, the bill still faces opposition from industry groups and some prominent California politicians, including Speaker Emerita Nancy Pelosi and other key members of the state’s Congressional delegation. Governor Newsom has not said whether he will sign the bill but voiced concern about a potential “chilling effect” in public comments earlier this week. The governor has until September 30 to sign or veto the bill.
DIU Looks to Generative AI for Joint Planning and Wargaming Help: The Pentagon’s Defense Innovation Unit (DIU) wants to use generative AI for joint planning and wargaming capabilities, according to a solicitation issued last month. The initiative, dubbed “Thunderforge,” aims to accelerate joint planning — a “complex, time-consuming, and resource-intensive” process — by leveraging AI to rapidly process information, produce draft planning products, and generate options for human planners. DIU is looking for solutions that can synthesize large amounts of data, develop courses of action, and conduct automated wargaming against likely adversary actions. Since the launch of ChatGPT, the Pentagon and defense contractors have been interested in finding ways to incorporate generative AI in a military context. But the technology’s drawbacks — especially its propensity to hallucinate and make plausible-sounding (and therefore difficult-to-identify) mistakes — have given senior DOD officials pause. Until reliable fixes are made or sufficient guardrails are identified, the usefulness of generative AI in the military will likely be limited to areas where 100% accuracy is not a necessity. Wargaming could very well be one of those areas, so it will be interesting to see whether Thunderforge yields any successes. Responses to DIU’s solicitation closed on September 6.
In Translation
CSET’s translations of significant foreign language documents on AI
CSET’s translations of significant foreign language documents on AI
Beijing Municipal AI Plan: Beijing Municipal Action Plan to Promote “AI+” (2024-2025). This Beijing Municipal industrial policy describes how the city government plans to integrate AI into a wide variety of industries in 2024 and 2025. These industries and sectors include robotics, education, healthcare, scientific research, spatial computing, digital marketing, Party propaganda, the power grid, surveillance, and censorship, among others.
Guangdong AI Policy: Certain Measures of Guangdong Province for Empowering Thousands of Industries with Artificial Intelligence. This document outlines the near-term AI industrial policy of Guangdong Province, a technologically advanced region in southern China. The policy’s main objective is to increase Guangdong’s total compute by 50% between 2025 and 2027 to support the AI industry. The policy also emphasizes strengthening AI-related industries and resources such as specialized AI chips, smart sensors, industrial design software, Chinese-language datasets, and domestically developed algorithms.
If you have a foreign-language document related to security and emerging technologies that you’d like translated into English, CSET may be able to help! Click here for details.
What’s New at CSET
PUBLICATIONS
- Harvard Kennedy School Misinformation Review: How spammers and scammers leverage AI-generated images on Facebook for audience growth, Renée DiResta and Josh A. Goldstein
- Council on Foreign Relations: Don’t Reinvent the Wheel to Govern AI by Jack Corrigan and Owen J. Daniels
- Lawfare: AI Regulation’s Champions Can Seize Common Ground—or Be Swept Aside by Helen Toner and Zachary Arnold
- TIME: What Google’s Antitrust Defeat Means for AI by Jack Corrigan
- Atlantic Council: Stay ahead together: Identifying strategies to compete with China on global 5G by Ngor Luong
- Atlantic Council: Assessing China’s AI development and forecasting its future tech priorities by Hanna Dohmen
EMERGING TECHNOLOGY OBSERVATORY
- The Emerging Technology Observatory is now on Substack! Sign up for the latest updates and analysis.
- Editors’ picks from ETO Scout: volume 14 (7/19/24-8/19/24)
- New research tools for Chinese science and tech: Celebrating Scout’s first anniversary
- Editors’ picks from ETO Scout: volume 15 (8/20/24-9/13/24)
EVENT RECAPS
- On August 29, CSET hosted a half-day conference on the military’s application of AI and software. The conference covered the results of a joint project with the Army’s 18th Airborne Corps, Johns Hopkins Applied Physics Laboratory, and CSET, and the resulting paper: “Building the Tech Coalition: How Project Maven and the U.S. 18th Airborne Corps Operationalized Software and Artificial Intelligence for the Department of Defense.”
- On September 11, Dr. Jeffrey Ding of George Washington University joined CSET Research Analyst Hanna Dohmen to discuss his new book, Technology and the Rise of Great Powers. Watch a recording of the event.
IN THE NEWS
- TIME: Time100 AI 2024 (TIME named Helen Toner to its list of the “100 Most Influential People in AI”)
- Breaking Defense: ‘Success begets challenges’: NGA struggles to meet rising demand for Maven AI (Sydney J. Freedberg Jr. covered the CSET event Building the Tech Coalition)
- GZero Media: Europe adopts first “binding” treaty on AI (Scott Nover quoted Mina Narayanan)
- Interesting Engineering: 1,000 hits per hour: US Army’s AI system to get smarter, speedier, deadlier (Abhishek Bhardwaj cited the CSET report Building the Tech Coalition)
- Lawfare: Lawfare Daily: Helen Toner and Zach Arnold on a Common Agenda for AI Doomers and AI Ethicists (Kevin Frazier and Jen Patja hosted Helen Toner and Zachary Arnold)
- Politico: Digital Future Daily: California tackles digital superintelligence — maybe (Derek Robertson hosted Helen Toner)
- TechCrunch: Equity Podcast: Is there a right way to regulate AI? A conversation with Helen Toner (Becca Szkutak spoke to Helen Toner)
What We’re Reading
Paper: Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Model Parameters, Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar (August 2024)
Paper: Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers, Colin Wei, Yining Chen, and Tengyu Ma (July 2021)
Paper: Consent in Crisis: The Rapid Decline of the AI Data Commons, Shayne Longpre et al., Data Provenance (July 2024)
Upcoming Events
- September 19: CSET Webinar, Privacy, Security, and Innovation – Friends Not Foes, featuring Andrew Trask and Irina Bejan of OpenMined and CSET’s Helen Toner
What else is going on? Suggest stories, documents to translate & upcoming events here.