Anthropic says Claude blocked attempt to develop biological weapons

One of the world’s largest AI models has said it blocked an attempt to develop biological weapons using its technology.

Anthropic, operator of Claude, said in its latest threat intelligence report that Russian-based cyber criminals and Iranian propaganda actors tried to use the AI model for malicious reasons.

Biological ‘misuse’ is one of the most severe risks facing the AI model, they added. 

Other cases of misuse included attempts to develop software for ‘conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them’.

‘The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,’ Anthropic said.

But the head of threat intelligence at Anthropic stressed that the attempt to create a biological weapon was ‘incredibly nuanced’.

‘You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” he told the New York Times.

Anthropic has safeguards in place and was able to stop the malicious attempts on their LLM.

Earlier this week, a researcher at Anthropic told his colleagues he was leaving by saying they were ‘causing human extinction’.

Jacob Coxon, 27, joined Anthropic earlier this year to train its bots by giving them huge amounts of data after leaving ChatGPT-maker OpenAI.

Coxon quit on Tuesday and told his colleagues on Slack that he was sad to say goodbye to his fellow ‘Ants’, a term for Anthropic employees.

‘Like many of you,’ his message continued, ‘I think that without international coordination we will incur a risk of causing human extinction.

‘I’ve recently become more concerned about the degree and imminence of this risk. I also find myself viscerally fearing the intelligence explosion where previously those concerns were unemotional.’

The messages, obtained by NBC News, came hours before Coxon posted on X, warning that the AI arms race threatens human civilisation.

‘The people building AI earnestly believe that it could kill us all by the end of the decade,’ he wrote.

‘This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.’