Anthropic has said it disrupted attempts to use its AI models in ways that could support the development of biological weapons.

In a report titled ‘Detecting and countering misuse of AI’, the San Francisco-based AI company listed five case studies of potential biological misuse of its models which it says “is one of the most serious risks of frontier AI models”.

The scientific research in those case studies involved work around the mosquito-born chikungunya virus, a “highly-pathogenic” strain of the bird flu, a family of viruses that include smallpox and mpox, venoms and other toxins.

In the examples identified, Anthropic said the actors circumvented controls imposed to prevent users from unsupported regions accessing their models, while the actors also “engaged in other efforts to obfuscate the purpose of their research to evade our safeguards”.

“When we detected and investigated these cases, we banned the users’ accounts and incorporated our investigative findings into our frontier model safeguards, enforcement, and threat intelligence processes to better prevent, detect, and disrupt these activities in the future,” the report said.

Anthropic said the intent of the actors was not clear-cut and the AI lab did not identify them.

“The individuals implicated in these case studies are working scientists. We do not assert that they intended harm, and identifying them or their labs could expose them to harm,” the report said.

Anthropic said its aim in sharing the examples is to “spark conversation within the AI industry, and with governments, about emerging biological risks and how best to counter them”.

Anthropic says China, Iran used its AI to aid spying

Also in the report, Anthropic said it shut down multiple cases of state-sponsored surveillance operations that used its AI models, and warned that governments and state-aligned actors are increasingly using AI to spy on ethnic minorities and dissidents.

The incidents were found and disrupted between January and July, and they originated in China, Iran and west Africa, Anthropic said in a report about misuse of its models.

These surveillance campaigns “targeted the same diaspora and dissident communities these regimes have historically targeted”.

“These include pro-democracy figures in Hong Kong, Tibetan and Falun Gong communities across Asia, and Iranian minority communities and opponents of the Iranian regime abroad,” the report said.

In one case, Iranian actors developed a way to identify people through their social media accounts; in another, a contractor working for Malian national security authorities used Claude “to design the underlying software that enabled the intelligence gathering”.

“AI is now being used in place of an engineering workforce,” the report said.

Anthropic also disrupted efforts by users to design weapons and create dating scams.

The San Francisco-based AI lab also accused Chinese developers of deceptively using Claude to produce responses to user queries while simultaneously “distilling” that data to improve their own models.

Distilling is an industry term that refers to a process where one AI model is used to train another one.

Chinese developers have previously been accused of using that technique without permission, essentially stealing resources from American labs such as Anthropic and OpenAI.

Now Anthropic alleges that China’s Moonshot and Deepseek have also secretly used Claude to generate answers given to its users.

“In one instance, over a 10-day period, Moonshot relayed almost 300,000 customer requests to Anthropic” through a “network of 5,380 fraudulent accounts, most of which appeared to be located in Singapore and Japan,” the report said.

Some of that data contained sensitive user information, potentially in violation of privacy agreements.

“We do not know if Moonshot notified their customers that their requests were being rerouted to Anthropic and exposed to a third party,” the report said.

Deepseek used similar techniques.

Earlier this week, an artificial intelligence researcher who left OpenAI to join Anthropic has decided to leave the industry, accusing both US companies of “gambling with our lives” in the race to develop AI models capable of self-improvement.

Additional reporting AFP