OpenAI Debuts GPT-4 but Stays Mum About What’s Under the Hood: Last week, OpenAI announced GPT-4 — its long-anticipated next-generation large language model (LLM). In release materials, the company detailed GPT-4’s improvements over GPT-3.5, the model that powers the free version of the uber-popular ChatGPT. Those advances include multimodal capabilities (GPT-4 can accept both text and images as inputs, though it can only output text), improved performance across a range of academic and professional tests and other benchmarks, better guardrails against generating “disallowed content,” a higher likelihood of producing factual responses, and the capability to process and respond to much longer strings of text. But while OpenAI was eager to discuss GPT-4’s capabilities, it shared little about what was under the hood, beyond a section in the technical report that describes GPT-4 as a pre-trained “Transformer-style” model fine-tuned using Reinforcement Learning from Human Feedback. The tight-lipped debut differs significantly from the GPT-3 release, when OpenAI shared detailed information about how its model was built and trained. The GPT-4 technical paper credited this shift to changes in the broader “competitive landscape” and “the safety implications of large-scale models like GPT-4.” In an interview with The Verge’s James Vincent, OpenAI co-founder and Chief Scientist Ilya Sutskever said the company’s earlier approach to sharing its research had been “flat out” wrong. Despite keeping the finer details of the model’s architecture and training under wraps, OpenAI seems to have fewer reservations about unleashing the model itself. GPT-4 is already available for use by developers through the company’s API and to paying subscribers through ChatGPT.
Expert Take: “Many of the important questions about large language models are still open. One debate to watch closely concerns how openly available increasingly powerful language models should be. Machine learning has benefited from its culture of extreme openness, but as the state of the art comes closer to systems that could enable dangerous actions — or even wreak havoc on their own — researchers and policymakers will need to think seriously about how to prevent proliferation while promoting equitable access.” — Helen Toner is CSET’s Director of Strategy and Foundational Research Grants and a member of OpenAI’s Board of Directors
Anthropic, a Google-backed AI company started by OpenAI alumni (among them CSET Non-Resident Fellow Jack Clark), launched its AI-powered chatbot “Claude.” Anthropic has announced partnerships with several companies to incorporate Claude into their products and is giving some users early access to the chatbot on a limited basis.
Without doubt, it’s been one of the most frenetic periods in the history of AI, and many things remain unsettled. But one of the clearest takeaways is that AI will be incorporated into the work products of millions of knowledge workers around the world in the very near future. The effect this will have on those workers is still anyone’s guess.
Stanford Researchers Train A GPT-3.5 Knockoff That Can Run Locally: Researchers at Stanford University fine-tuned a language model — dubbed “Alpaca” — that performs comparably to OpenAI’s GPT-3.5 but can run on consumer hardware. In a post outlining their process, the researchers show how they used the 7 billion-parameter version of Meta’s recently released LLaMA language model and a dataset of inputs and outputs from OpenAI’s text-davinci-003 (the model — also known as GPT-3.5 — powering the free version of ChatGPT) to fine-tune the 7B LLaMA model. The result was an instruction-following model (a chatbot like ChatGPT or Anthropic’s Claude) that the authors say performed as well as the original text-davinci-003 model. Unlike OpenAI’s model, however, Stanford’s Alpaca can be run on consumer hardware, such as consumer-grade GPUs (and even on smartphones, according to some users). While the Stanford team pulled a demo of Alpaca offline, LLaMA’s weights have already leaked and Stanford’s dataset and training code are freely available, meaning anyone with sufficient skill can copy Stanford’s fine-tuning process for less than $100 in cloud computing costs. As observers have pointed out, this method could potentially allow almost anyone with access to a model’s API to substantially replicate it outside the control of the original developer — making it available for AI-powered disinformation or other nefarious purposes.
The overall DOD Research, Development, Test and Evaluation funding request represents a 4 percent increase over 2023 enacted levels. If met, the NSF request alone would boost the agency’s overall budget by more than 18 percent. Congress has begun its annual budget hearings, but it remains to be seen how final spending will match or differ from the administration’s submission.
The Commerce Department Proposes Guardrails on CHIPS Funds: On Tuesday, the Commerce Department announced proposed rules that would significantly limit the ability of CHIPS Act fund recipients to invest in China or expand their operations there. As we covered earlier this month, the Commerce Department will soon begin accepting applications for the $39 billion in semiconductor manufacturing incentives provided by the CHIPS and Science Act of 2022. In addition to boosting domestic semiconductor manufacturing, one of the act’s primary objectives (going back to its earlier iterations as the Endless Frontier Act and the United States Innovation and Competition Act of 2021) is countering China. The proposed rules would limit CHIPS funding recipients from investing more than $100,000 or expanding existing production capacity by more than 5 percent for facilities producing “leading edge” or “advanced” chips in “foreign countries of concern,” including China and Russia. A specific list of “semiconductors critical to national security,” including those used in quantum computing and for specialized military purposes, will also be subject to the 5 percent limit — regardless of the technology used to manufacture them. For facilities producing non-critical “legacy” chips, the expansion limit would be set at 10 percent. The proposed rules will be open for public comment for 60 days, beginning on March 23.
The FTC Issues Another Warning About AI: In a blog post earlier this week, the Federal Trade Commission warned AI developers against making and selling products that can be used for fraud, scams, or other harmful activities. The post is the second AI-related guidance issued by the FTC in the last month, a sign — together with its new Office of Technology — of the agency’s growing focus on tech-related enforcement efforts. The new guidance encourages companies to think through the “reasonably foreseeable” ways their generative AI systems could be misused for fraudulent purposes, and warns them to implement “durable, built-in” protections against consumer harm if they want to avoid FTC enforcement actions. The post seems to be related to a reported surge in scam calls during which fraudsters used AI-powered voice-mimicking software to impersonate the relatives of (often elderly) targets — indeed, the FTC issued a consumer alert about such calls the same day as the blog post. Beyond addressing low-level scams, the agency seems to be considering potential enforcement actions related to the “potentially dangerous” risks AI poses to children, teens and other vulnerable groups; the blog post concludes by emphasizing that FTC staff are closely monitoring concerns around these issues.
In Translation CSET’s translations of significant foreign language documents on AI
PRC Domestic Demand Strategy:Outline of the Plan for the Strategy to Expand Domestic Demand (2022-2035). This is a translation of China’s short- to mid-term strategy for expanding domestic demand in its economy. Although its theme is increasing Chinese consumer demand, the strategy is wide-ranging and includes calls to increase the quantity and quality of the supply of goods and services and to improve China’s social safety net. It does not, however, set any metrics to measure how well the strategy is being implemented. Although this document is merely the “outline” of the plan to expand domestic demand, it is likely that this “outline” will be the fullest version of the strategy that China makes public.
If you have a foreign-language document related to security and emerging technologies that you’d like translated into English, CSET may be able to help! Click here for details.
CSET:Blog post: Women’s History Month, a (non-exhaustive) list of information and resources compiled by CSET team members to help us learn from, support and honor the women who’ve transformed our world
CBS: In a Nicole Keller story on CBS Mornings about the potential national security threat posed by TikTok, which was tied to today’s House hearing, Puglisi addressed the platform’s potential for illicit tech transfer to China.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.