GPT-4, Bard, Claude, Copilot and more — an avalanche of generative AI news, AI and emerging tech in Biden’s 2024 budget request, and the FTC issues another AI warning

Sign up
Worth Knowing

OpenAI Debuts GPT-4 but Stays Mum About What’s Under the Hood: Last week, OpenAI announced GPT-4 — its long-anticipated next-generation large language model (LLM). In release materials, the company detailed GPT-4’s improvements over GPT-3.5, the model that powers the free version of the uber-popular ChatGPT. Those advances include multimodal capabilities (GPT-4 can accept both text and images as inputs, though it can only output text), improved performance across a range of academic and professional tests and other benchmarks, better guardrails against generating “disallowed content,” a higher likelihood of producing factual responses, and the capability to process and respond to much longer strings of text. But while OpenAI was eager to discuss GPT-4’s capabilities, it shared little about what was under the hood, beyond a section in the technical report that describes GPT-4 as a pre-trained “Transformer-style” model fine-tuned using Reinforcement Learning from Human Feedback. The tight-lipped debut differs significantly from the GPT-3 release, when OpenAI shared detailed information about how its model was built and trained. The GPT-4 technical paper credited this shift to changes in the broader “competitive landscape” and “the safety implications of large-scale models like GPT-4.” In an interview with The Verge’s James Vincent, OpenAI co-founder and Chief Scientist Ilya Sutskever said the company’s earlier approach to sharing its research had been “flat out” wrong. Despite keeping the finer details of the model’s architecture and training under wraps, OpenAI seems to have fewer reservations about unleashing the model itself. GPT-4 is already available for use by developers through the company’s API and to paying subscribers through ChatGPT.

Expert Take: “Many of the important questions about large language models are still open. One debate to watch closely concerns how openly available increasingly powerful language models should be. Machine learning has benefited from its culture of extreme openness, but as the state of the art comes closer to systems that could enable dangerous actions — or even wreak havoc on their own — researchers and policymakers will need to think seriously about how to prevent proliferation while promoting equitable access.” — Helen Toner is CSET’s Director of Strategy and Foundational Research Grants and a member of OpenAI’s Board of Directors
AI Is Coming to the Office —  A Generative AI News Roundup: Beyond GPT-4, the biggest news came from Microsoft and Google, which both announced plans to add generative AI features to their popular office software tools. Microsoft’s AI-powered “Copilot” will be integrated across its suite of 365 products, including Word, Excel, Teams, and Outlook. Like the company’s new AI-powered Bing chatbot, the new Copilot is powered by OpenAI’s GPT-4 (not to be confused with GitHub’s Copilot, which also got a GPT-4-powered upgrade this week). Google, meanwhile, announced last week that it would begin incorporating generative AI into a number of its Workspace products, such as Gmail, Docs, Sheets and Slides. Google will also give developers access to its powerful PaLM language model, one of the world’s biggest and most powerful LLMs (though the lack of detailed information about GPT-4 makes direct comparisons difficult). Then on Tuesday, the company entered the AI chatbot race, releasing its “Bard” chatbot to a limited number of users in the United States and UK. It’s been a quick pivot for Google, which until late last year had been wary of launching generative AI products due to concerns about the “reputational risk” biased or untrustworthy systems could pose. That all changed when ChatGPT’s surging popularity caused Google executives to declare “code red” and shift more resources toward generative AI products. For now, access to Google’s new tools is mostly limited to select “trusted testers,” but the company says it will begin rolling them out more broadly this year. Other generative AI highlights include:
Without doubt, it’s been one of the most frenetic periods in the history of AI, and many things remain unsettled. But one of the clearest takeaways is that AI will be incorporated into the work products of millions of knowledge workers around the world in the very near future. The effect this will have on those workers is still anyone’s guess.
Stanford Researchers Train A GPT-3.5 Knockoff That Can Run Locally: Researchers at Stanford University fine-tuned a language model — dubbed “Alpaca” — that performs comparably to OpenAI’s GPT-3.5 but can run on consumer hardware. In a post outlining their process, the researchers show how they used the 7 billion-parameter version of Meta’s recently released LLaMA language model and a dataset of inputs and outputs from OpenAI’s text-davinci-003 (the model — also known as GPT-3.5 — powering the free version of ChatGPT) to fine-tune the 7B LLaMA model. The result was an instruction-following model (a chatbot like ChatGPT or Anthropic’s Claude) that the authors say performed as well as the original text-davinci-003 model. Unlike OpenAI’s model, however, Stanford’s Alpaca can be run on consumer hardware, such as consumer-grade GPUs (and even on smartphones, according to some users). While the Stanford team pulled a demo of Alpaca offline, LLaMA’s weights have already leaked and Stanford’s dataset and training code are freely available, meaning anyone with sufficient skill can copy Stanford’s fine-tuning process for less than $100 in cloud computing costs. As observers have pointed out, this method could potentially allow almost anyone with access to a model’s API to substantially replicate it outside the control of the original developer — making it available for AI-powered disinformation or other nefarious purposes.
Government Updates

AI and Emerging Tech in the FY2024 Budget Request: Last week, the Biden Administration submitted its Fiscal Year 2024 budget request. It includes significant proposed investments to bolster U.S. science and technology activities, among them $4.4 billion for the Defense Advanced Research Projects Agency, $796.5 million for National Science Foundation-supported AI research, and $1.2 billion for the NSF’s new Directorate of Technology, Innovation and Partnerships. Additional highlights include:
The overall DOD Research, Development, Test and Evaluation funding request represents a 4 percent increase over 2023 enacted levels. If met, the NSF request alone would boost the agency’s overall budget by more than 18 percent. Congress has begun its annual budget hearings, but it remains to be seen how final spending will match or differ from the administration’s submission.

The Commerce Department Proposes Guardrails on CHIPS Funds: On Tuesday, the Commerce Department announced proposed rules that would significantly limit the ability of CHIPS Act fund recipients to invest in China or expand their operations there. As we covered earlier this month, the Commerce Department will soon begin accepting applications for the $39 billion in semiconductor manufacturing incentives provided by the CHIPS and Science Act of 2022. In addition to boosting domestic semiconductor manufacturing, one of the act’s primary objectives (going back to its earlier iterations as the Endless Frontier Act and the United States Innovation and Competition Act of 2021) is countering China. The proposed rules would limit CHIPS funding recipients from investing more than $100,000 or expanding existing production capacity by more than 5 percent for facilities producing “leading edge” or “advanced” chips in “foreign countries of concern,” including China and Russia. A specific list of “semiconductors critical to national security,” including those used in quantum computing and for specialized military purposes, will also be subject to the 5 percent limit — regardless of the technology used to manufacture them. For facilities producing non-critical “legacy” chips, the expansion limit would be set at 10 percent. The proposed rules will be open for public comment for 60 days, beginning on March 23.

The FTC Issues Another Warning About AI: In a blog post earlier this week, the Federal Trade Commission warned AI developers against making and selling products that can be used for fraud, scams, or other harmful activities. The post is the second AI-related guidance issued by the FTC in the last month, a sign — together with its new Office of Technology — of the agency’s growing focus on tech-related enforcement efforts. The new guidance encourages companies to think through the “reasonably foreseeable” ways their generative AI systems could be misused for fraudulent purposes, and warns them to implement “durable, built-in” protections against consumer harm if they want to avoid FTC enforcement actions. The post seems to be related to a reported surge in scam calls during which fraudsters used AI-powered voice-mimicking software to impersonate the relatives of (often elderly) targets — indeed, the FTC issued a consumer alert about such calls the same day as the blog post. Beyond addressing low-level scams, the agency seems to be considering potential enforcement actions related to the “potentially dangerous” risks AI poses to children, teens and other vulnerable groups; the blog post concludes by emphasizing that FTC staff are closely monitoring concerns around these issues.

In Translation
CSET’s translations of significant foreign language documents on AI

PRC Domestic Demand Strategy: Outline of the Plan for the Strategy to Expand Domestic Demand (2022-2035). This is a translation of China’s short- to mid-term strategy for expanding domestic demand in its economy. Although its theme is increasing Chinese consumer demand, the strategy is wide-ranging and includes calls to increase the quantity and quality of the supply of goods and services and to improve China’s social safety net. It does not, however, set any metrics to measure how well the strategy is being implemented. Although this document is merely the “outline” of the plan to expand domestic demand, it is likely that this “outline” will be the fullest version of the strategy that China makes public.

If you have a foreign-language document related to security and emerging technologies that you’d like translated into English, CSET may be able to help! Click here for details.

What’s New at CSET

PUBLICATIONS

IN THE NEWS

What We’re Reading

Report: Report and Recommendations, U.S. Chamber of Commerce Commission on Artificial Intelligence Competitiveness, Inclusion, and Innovation (March 2023)

Article: Three Easy Ways to Make AI Chatbots Safer, Noah Giansiracusa, Scientific American (March 2023)

Upcoming Events


What else is going on? Suggest stories, documents to translate & upcoming events here.