Firms crack down on AI propaganda operations, Anthropic and OpenAI try to open the black box, and California AI bill grabs headlines

Sign up
Worth Knowing

Companies Report AI-Enabled Propaganda Campaigns and Disinformation in Year of Elections: Late last month, leading AI companies publicly detailed efforts to use AI to shape political discourse and opinion. In its May 2024 quarterly threat report, Meta reported on campaigns by networks in numerous countries, including Bangladesh, China, Croatia, Iran, and Israel, to carry out a range of operations aimed at influencing their own domestic politics or other countries’ political discourse. Many of these efforts centered around using generative AI to create fake images, content, and profiles to coordinate influence operations. For instance, Meta reported that a Chinese network was targeting Sikh communities across Australia, Canada, India, New Zealand, Pakistan, the United Kingdom, and Nigeria with AI-generated content intended to stoke criticism of the Indian government and express support for a Sikh separatist movement. These efforts may aim to inflame opinion in these countries amid reporting of assassination attempts against Sikh activists living abroad, allegedly supported by the Indian government, referenced in Meta’s threat report.
Looking into the black box? The opacity of neural networks has long presented limitations to understanding how certain AI models really work. While recent years have seen some limited headway in interpretability as a means of better understanding how these systems function, progress has remained limited. However, researchers at Anthropic and OpenAI have both reported recent successes in identifying features of large models that are interpretable by humans.
  • According to reporting by Wired, Anthropic’s mechanistic interpretability team was able to map clusters of neurons within the neural net to identify features—or concepts—that the clusters would predictably generate using a technique Anthropic calls “dictionary learning.” Beginning by examining a simple single-layer neural net and eventually progressing on to multilayer nets, researchers identified clusters that represented features like the Golden Gate Bridge, and used clusters of similar neurons to identify related concepts, like Alcatraz or California governor Gavin Newsom. They were also able to identify clusters that produced negative features, like hate speech or unsafe code, and could manipulate the behavior and outputs of Claude, Anthropic’s LLM. The researchers believe that conducting this “brain surgery” on models could help AI safety efforts by allowing them to excise dangerous features.
  • Shortly after Wired’s report on Anthropic’s mechanistic interpretability progress, OpenAI released its own post and corresponding research paper and GitHub code about interpreting patterns within its GPT-4 model. OpenAI presented its methodology for using a sparse autoencoder algorithm to highlight the connections between firing neural activations in a neural network and features, like those described by Anthropic’s interpretability team, that are easier for humans to understand. OpenAI’s team highlighted that while this progress is promising, interpretability work is still far from being a silver bullet; other researchers have echoed this sentiment, with scalability of current interpretability methods a likely limitation.
Government Updates

California SB 1047 Passes State Senate with Assembly Vote Upcoming in August: Last month, California Senate Bill 1047 (referred to as SB 1047), introduced by State Senator Scott Weiner, passed a vote in the Senate and will be voted on in the state’s Assembly. The bill is a state-level attempt to regulate risks from large frontier AI models, and its mandatory requirements would stand in contrast to federal-level efforts around AI safety—like President Biden’s White House summit with tech CEOs or the National Institute of Standards and Technology’s AI risk management framework—which have largely entailed voluntary compliance from developers. The bill has received national attention for originating in tech companies’ own backyard and for its potential to influence other state and federal regulations in a year where as many as 600 pieces of AI legislation are reportedly being considered in the United States. According to Weiner, SB1047’s purpose is to introduce uniformity in risk mitigation and safety approaches for large AI models, creating baseline self-reporting standards around risks that companies must follow to be legally compliant. Under the proposed legislation, developers must provide reasonable assurances to a new Frontier Model Division in California’s Department of Technology that their models are unlikely to cause critical harms, defined as those that could cause over $500,000,000 in damages, threaten public safety, or lead to the creation of nuclear or other mass casualty weapons. The bill applies to AI models trained by developers using computing power greater than 10^26 FLOPS (aligning it with a standard laid out in the Biden Administration’s October 2023 AI Executive Order), and whose computing training costs exceed $100,000,000. In other words, it applies to models likely larger than today’s most advanced, and to the largest tech developers.

It has drawn mixed reactions from different members of the AI community:
  • AI developers are not enthusiastic about the bill, and many have opposed it. Opponents of the bill suggest that it will stifle innovation, particularly norms of open model development and fine-tuning, and that reporting and risk assessment requirements could become onerous. TechNet, a technology company network that counts Anthropic, Apple, Google, Meta, OpenAI, and others among its ranks, opposes the bill, as do a number of groups with ties to industry or civil society like the California Chamber of Commerce and the Civil Justice Association of California.
  • Individuals and groups concerned with AI safety have mixed feelings about the bill. Some leading AI safety voices believe that it is a common sense first step in safety regulation, while others believe it does not do enough to meaningfully protect against AI risks.
Given the furor around the bill, the Assembly vote in August will be one to watch.

Palantir Dives Deeper into Data in Support of DOD Initiatives: Palantir, who has been leading work on the Department of Defense’s evolved Project Maven efforts and was recently awarded a $480,000,000 DOD contract to build the Maven Smart System Prototype, was also awarded a contract to develop the Pentagon’s data-sharing ecosystem earlier this month, according to reporting by NextGov. Palantir’s ecosystem, known as Open DAGIR (Data and Applications Government-owned Interoperable Repositories), will play a key role in supporting the DOD’s Combined Joint All Domain Command and Control initiative, better known as CJADC2. The CJADC2 initiative is intended to use AI, among other capabilities, to enable command and control by connecting personnel, systems, and platforms across the U.S. military services, with the goal of enabling joint military commanders to more effectively manage the battlefield. Deputy Secretary of Defense Kathleen Hicks announced earlier this year that the Pentagon’s Chief Digital and Artificial Intelligence Office (CDAO) had delivered a minimal viable capability of CJADC2. Breaking Defense reported in February that CDAO planned to test information sharing over CJADC2 with Five Eyes partners after doing the hard work of cleaning the DOD’s data resources. CJADC2 has been one of the DOD’s highest-profile AI efforts, and the award of the data contract to Palantir likely signifies a next step in building on this minimum viable product.

In Translation
CSET’s translations of significant foreign language documents on AI

AI Standardization Guidelines: Guidelines for the Construction of a Comprehensive Standardization System for the National Artificial Intelligence Industry (Draft for Feedback). This document, released by the Science and Technology Department of the Chinese Ministry of Industry and Information Technology, lays out a draft Chinese government framework for a system of standards for AI. Its authors invited feedback and comments on the draft from the Chinese public from January 17 to 31, 2024.

Governance of Large Models: Research Report on Governance Modernization in the Digital Age: Practice and Prospects for the Application of Large Models in the Government Domain (2023). This report by a Chinese state-run think tank describes how the Chinese government and foreign governments are using large AI models. The authors list more than 50 large models developed by Chinese tech companies that provincial and local governments in China have deployed for various purposes. While these AI systems could improve Chinese local governments’ provision of public services, the focus of many of these governments on early detection and suppression of social disturbances suggests that these AI models are being used to enhance and intensify China’s surveillance state.

If you have a foreign-language document related to security and emerging technologies that you’d like translated into English, CSET may be able to help! Click here for details.

Job Openings

Please bookmark our careers page to stay up to date on all active job postings. You can also subscribe to receive job announcements by updating your subscription preferences in the footer of this email.

What’s New at CSET

REPORTS

PUBLICATIONS

EMERGING TECHNOLOGY OBSERVATORY

EVENT RECAPS

  • On June 6, CSET hosted experts to discuss where AIxBio policymakers go next after the 180-day mark following the Biden AI Executive Order. Watch a full recording of the event.

TESTIMONY

  • On May 23, CSET Non-Resident Senior Fellow Kevin Wolf testified before the US-China Economic and Security Review Commission on economic competition with China. Read his full testimony here.

IN THE NEWS

What We’re Reading

Report: Open-Endedness is Essential for Artificial Superhuman Intelligence, arXiv (June 2024)