When Open AI presented its software GPT-2 to the public in February 2019 – the abbreviation stands for “Generative Pre-trained Transformer” – some observers thought the whole action was a well-placed PR gag. Because the research laboratory did present convincing texts that were supposedly generated by the software after it had only received a brief input. However, Open AI did not want to publish the actual model – the architecture of the network and the 1.5 billion parameters that enable the software to add sentences, translate texts or write summaries. The software is potentially dangerous, wrote the developers because it could be used to produce fake news on a large scale. Only after months of hesitation and weighing up the risks did Open AI give the data finally free.
GPT-3, the successor to GPT-2, is once again orders of magnitude more complex and powerful with 175 billion parameters. And it looks like GPT-3 will actually prove to be the original fears of the Open AI researchers. At least that can be seen from a current one examination recently published by Ben Buchanan, Andrew Lohn, Micah Musser and Katerina Sedova of the Center for Security and Emerging Technology (CSET) at Georgetown University in Washington.
The researchers examined the performance of GPT-3 in six different disinformation scenarios. For example, the software should write as varied as possible postings that appear as if they were written by many different users, but at the same time all promote a certain topic, for example the rejection of the fact that there is climate change. It should provide completely new ideas for conspiracy theories and generate posts that specifically incite groups against each other. You should measure your ability to rewrite messages in such a way that they fit into a certain worldview and, conversely, generate medium-length texts for a certain worldview that should substantiate this worldview with fictitious events.
Human helps computer
Some of these tasks, such as rewriting messages with a certain tendency, are actually still too complex for GPT-3. However, the scientists were able to show that, with a little human help, these jobs can be broken down into simpler subtasks: The system should first reduce a given text to a list with a few important statements. The researchers then gave these statements a new twist and used the changed sentences as starting material for new articles generated by GPT-3.
However, it is difficult to measure how effective the automated disinformation is. The researchers tested the effect by presenting postings generated by GPT-3 to users with defined political preferences – and then asking them whether they agree or disapprove of political opinions. The results were mixed, but clearly visible: One experiment, for example, looked at the question of whether the US should relax its sanctions against China. While the group that had seen five postings designed for this purpose subsequently rejected sanctions by 40 percent, it was only 22 percent in the independent control group. However, the researchers found no explanation for the fact that the AI-generated reports were significantly less effective in the opposite case – i.e. for tightening sanctions.
The conclusion of the researchers is nevertheless pessimistic. The investigation shows that operations like the one at the Russian troll factory, which is said to have interfered in the US election campaign, can be at least partially automated with the help of powerful language models like GPT-3. “Of course you have to take into account that running a troll factory involves more than just writing texts,” writes Andrew Lohn, Senior Fellow at CSET and co-author of the study. “A big part of the work is also generating fake accounts and spreading the news. But then you would actually probably need fewer writers who have to speak the language and are familiar with the politics and culture of a country.
Only selected partners are allowed
Access to GPT-3 is still strictly limited. Open AI only grants selected partners access to the model via an API – and Microsoft has an exclusive license to access the code itself. But it is only a matter of time before that changes. On the one hand, other companies are also working on such large models – Huwaei, for example, has a transformer model with Pangu-Alpha with 200 billion parameters presentedthat was trained with 45 terabytes of data. On the other hand, the research community itself is not idle and wants to recreate GPT-3 with an international project.
“On the one hand, there are (large language models) useful tools that can improve productivity in a positive way. The downside is that they could help reinforce opinions on the margins, ”writes Andrew Lohn. “One person can write thousands of messages about an idea or topic that are both coherent and diverse so that it appears that that one person is really a lot of people. This could accelerate the trend to bring rare extreme ideas to the fore. “
Open AI is certainly aware of this risk, writes Lohn, because the company provided excellent support for the investigation from the start. However, Open AI has not yet responded to a request from Technology Review about possible consequences of the investigation.