OpenAI vs. NYT — the copyright fight and the future of AI training, AI in the FY2024 NDAA, and two GAO reports on AI

Sign up
Worth Knowing

EU Leaders Reach Agreement on AI Act — Ratification Still in the Works: In December, EU leaders reached an agreement on the bloc’s landmark AI Act. The sweeping regulatory package will set important new standards for AI development and deployment within the EU, and — much like earlier EU regulatory efforts such as the General Data Protection Regulation (GDPR) — will likely have a significant impact on AI development abroad. While the official compromise text has not yet been released, an EU press release laid out the key pillars of the agreed-upon regulation. The AI Act will:
  • Prohibit AI systems for untargeted facial image scraping, emotion recognition in workplaces and schools, social scoring, manipulation of human behavior, and sensitive biometric categorization (though the act includes law enforcement exemptions for some uses of biometric identification systems).
  • Include obligations for systems classified as “high risk,” such as those used in healthcare, environmental applications, and elections. Such systems will be subjected to mandatory fundamental rights impact assessment among other requirements.  
  • Impose transparency and compliance requirements on “general-purpose AI” systems (also known as foundation models). These requirements include publishing technical documentation, publishing information about content used for training, and compliance with EU copyright law.  
Originally proposed by the European Commission in 2021, the AI Act nearly collapsed just before the finish line, reportedly over concerns about its handling of foundation models. Marathon negotiation sessions proved enough to salvage a deal, but apparently not to the complete satisfaction of all parties: after the deal was announced, French President Emmanuel Macron publicly criticized the bill, citing concerns about its impact on Europe’s innovation ecosystem. Macron’s critique is more than just sour grapes — the AI Act still must be ratified by member states, and if France can rally at least three allies, it will be able to delay the law’s adoption. Observers say the final text of the AI Act should become available early next month, at which point we should have a clearer sense of the ratification timeline, as well as the odds.
A Heavyweight Copyright Battle — The New York Times Sues OpenAI: Late last month, the New York Times sued OpenAI and its biggest investor, Microsoft, for copyright infringement, setting up what could be the most significant fight yet over generative AI training practices. The Times’ lawsuit alleges that OpenAI’s large language models (LLMs) were trained by copying the Times’ copyrighted content, as evidenced by their ability to “generate output that recites Times content verbatim, closely [summarize] it, and [mimic] its expressive style.” The Times’ lawsuit is far from the first copyright case OpenAI has faced — since June, the company has been sued a dozen times on copyright grounds. And as we’ve covered, AI image-generation companies like Stability AI and Midjourney have also faced suits from artists over their training practices. While OpenAI maintains that its training practices constitute fair use, the company has also begun pursuing licensing deals with media companies to train its models on their content. It reached a deal with German media giant (and Politico owner) Axel Springer in December and had been in talks with the Times before the suit was filed. Apple, which is angling to join the LLM race, has reportedly pursued licensing deals worth more than $50 million with a number of publishers. For those LLM developers without deep pockets, however, the copyright cases working their way through the courts will be closely watched — if the claims are successful and licensing becomes the norm, LLM development could become much more difficult for all but the richest tech companies.
Using AI to Speed Up Scientific Research: A handful of recent papers show how AI can be used to accelerate scientific discovery:
Government Updates

AI and Emerging Tech in the FY2024 NDAA: Before breaking for the holidays, Congress wrapped up its consideration of the Fiscal Year 2024 National Defense Authorization Act, passing the critical authorization bill for the 63rd year running. The bill contains a number of AI provisions:
  • It includes provisions supporting the use of AI for optimizing aerial refueling in contested environments (Sec. 346) and maintenance operations at shipyards serving the U.S. Navy (Sec. 350).
  • It authorizes the Chief Digital and Artificial Intelligence Officer to access and control any DOD data and establishes a CDAO Government Council to oversee the ethical collection and use of such data (Sec. 1521).
  • It directs CDAO to develop and maintain data assets that support cyber operations preparations (Sec. 1523).
  • It directs the DOD to develop a bug bounty program for foundational models used by the department (Sect. 1542) and a prize competition to develop watermarking and other detection technologies for identifying content created by generative AI (Sec. 1543).
  • It tasks the DOD with defining guidance for near- and long-term strategies for the adoption of AI, as well as policies to support its ethical and secure use (Sec. 1544).
On December 13, the Senate approved the final conference report negotiated between the two chambers by a vote of 87 to 13. The House of Representatives followed a day later, approving by a vote of 310 to 118 and the President signed it into law on December 22.

NSF to Stand Up NAIRR Pilot Next Week: Next week, the National Science Foundation will launch a pilot program for the National AI Research Resource, as directed by President Biden’s November executive order on AI. The NAIRR — envisioned as “a shared research infrastructure” that would provide computing power, access to open government and non-government datasets, and training resources to students and AI researchers — has been in the works since 2020, when the National AI Initiative Act of 2020 (passed as part of the FY2021 NDAA) established a task force to research its feasibility. The task force published its final report early last year, in which it recommended a $2.6 billion investment over six years to build out the NAIRR in full, as well as a pilot program to make computing resources available sooner. The president’s November EO directed the head of the NSF to identify and enlist computational, data, software, and training resources from across the Federal government and private sector for the pilot program. NSF officials told reporters that the initial effort will be a “modest… proof-of-concept effort” comprising “resources that we have in hand” and “in-kind contributions from different technology companies.” It’s not clear exactly how robust the resourcing will be, but it will likely be a far cry from the computing power recommended by the task force for the full NAIRR. NSF officials said the goal of the pilot is to prove the value of the NAIRR concept. Should it prove a success, it will be up to Congress to fund the full project.        

GAO Reports Explore DOD’s AI Workforce and Federal Agencies’ AI Adoption: Two recent Government Accountability Office reports highlight the steps the federal government still needs to take to fully realize its AI ambitions:
  • A study on the Pentagon’s AI workforce found that the DOD had not sufficiently defined and identified its AI workforce, hindering its ability to effectively meet its strategic goals and objectives. As the GAO report noted, the DOD has previously identified the cultivation of AI expertise as a strategic focus area, but without consistent definitions, the DOD can’t effectively assess progress or confidently set future goals. GAO recommended that the CDAO should be tasked with sufficiently defining and identifying the DOD’s AI workforce.
  • A review of 23 federal agencies’ AI adoption found that the agencies had identified approximately 1,200 AI use cases, most of which were in the planning phase. While agencies like NASA and the Department of Commerce reported the highest number of AI use cases, GAO found that overall AI implementation remains inconsistent. Many agencies have not fully met federal AI requirements, with incomplete or inaccurate data in their AI use case inventories and a lack of comprehensive implementation planning. GAO recommends that agencies update their AI inventories, align them with guidelines, and implement AI requirements in federal law and policy to enhance management and oversight of AI applications​​.
In Translation
CSET’s translations of significant foreign language documents on AI

PRC Computing Power Policy: Action Plan for the High-Quality Development of Computing Power Infrastructure. This document is a Chinese government policy for the near-term development of computing power. The plan urges the expansion of compute in China, particularly of supercomputing and “intelligent compute” optimized for AI applications. But the policy also emphasizes improving the energy efficiency and lowering the carbon footprint of computing power infrastructure such as data centers. An appendix includes various compute-related metrics for China to strive for in 2023, 2024, and 2025.

PRC S&T Challenges: CAST Announces its Major Scientific Questions, Engineering Technology Challenges, and Industrial Technology Questions for 2023 | CAST 25th Annual Conference. This announcement by a Chinese Communist Party-run association for scientists names 28 questions as China’s outstanding S&T issues and challenges of 2023. The questions span a wide range of topics, including microchips, sustainable agriculture, renewable energy, rail transit, coal mining, manned Mars exploration, and many more. 

Beijing Municipal Plan: Beijing Municipal Implementation Plan for Promoting the Innovative Development of Future Industries. This Beijing municipal government plan identifies 20 “future industries” that Beijing is targeting with favorable industrial policies, so as to build up world-class companies in these industries by 2035. The industries that Beijing — home of China’s best universities and a disproportionately high number of its leading tech companies — is boosting are in the AI, healthcare, manufacturing, energy, materials, and space sectors.

If you have a foreign-language document related to security and emerging technologies that you’d like translated into English, CSET may be able to help! Click here for details.

What’s New at CSET

REPORTS

PUBLICATIONS

EMERGING TECHNOLOGY OBSERVATORY

PROJECTS

EVENT RECAPS

IN THE NEWS

What We’re Reading

Report: Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, Apostol Vassilev, Alina Oprea, Alie Fordyce, and Hyrum Anderson, NIST (January 2024)

Strategy: National Defense Industrial Strategy, U.S. Department of Defense (2023)

Paper: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, Evan Hubinger et al., arXiv (January 2024)

Paper: Textbooks Are All You Need, Suriya Gunasekar et al., arXiv (June 2023)

Upcoming Events


What else is going on? Suggest stories, documents to translate & upcoming events here.