OpenAI reported that it stopped five covert influence operations designed to misuse its AI programs to engage in deceptive activity, such as spreading mass misinformation.
OpenAI shut down operations in Russia, China, Iran, and Israel, where AI tools were used to quickly manufacture comments and news articles en-masse that contained misleading information. The campaign’s purpose was to steer political outcomes and manipulate public beliefs while masking malicious activities.
“Over the last three months, our work against IO actors has disrupted covert influence operations that sought to use AI models for a range of tasks,” writes OpenAI in a blog post. “(This includes) generating short comments and longer articles in a range of languages, making up names and bios for social media accounts, conducting open-source research, debugging simple code, and translating and proofreading texts.”
According to OpenAI, these operations were taken down between 2023 and 2024 and those that remain haven’t seen any meaningful increases in audience engagement thanks to AI use. Researchers even observed AI models refusing to produce the material hackers asked for as a result of their new defenses.
The operations include:
- Bad Grammar. A Russian threat actor that mostly spreads misinformation through Telegram. Bad Grammar was recently discovered.
- Doppelganger. A persistent Russian threat actor that spreads anti-Ukrainian propaganda.
- Camouflage. A persistent Chinese threat actor that creates content to praise China and disparage its enemies.
- International Union of Virtual Media (IUVM). An Iranian threat actor that posts content criticizing the US and Israel. They combine misleading information with legitimate criticisms to obfuscate which claims are factual.
- STOIC. An Israel-based commercial company generation content about the conflict in Gaza and the Histradut trade routes to show Israel in a positive light. Researchers nicknamed it “Zero Zeno,” referencing the founder of Stoic philosophy … as well as highlighting the company’s inability to gain any traction.
After OpenAI shut down these operations, the companies haven’t been able to spread into any authentic communities.