Anthropic warns AI is now orchestrating major parts of cyberattacks

Rather than running bulk queries, Kimi chatbot creator Moonshot and DeepSeek allegedly routed live customer conversations, which sometimes included sensitive information, through Claude and used its responses as training data, Anthropic alleged.

A hacking group whose tradecraft was consistent with Russia-based threat actor Midnight Blizzard allegedly ran phishing, hotel Wi-Fi hijacking and WhatsApp-takeover operations against targets in the Ukrainian government, military and diplomatic sectors, using AI at nearly every stage, Anthropic said.

The US government has previously linked Midnight Blizzard, a tracking term coined by Microsoft, to Russia’s SVR foreign intelligence service.

The Russian Embassy in Washington did not immediately respond to a request for comment.

The group allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote the code until it evaded detection again.

Anthropic also identified what it called "new categories of threat actors" misusing Claude, including those using the platform to "develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them."

The report detailed incidents of operators using Claude to develop software for weapons design and development, or to support intelligence gathering and procurement related to weapons programs, in China, Russia, and Yemen.

The big tech firm also outlined five case studies in which scientists used its AI models "in ways that could support biological weapons development."

Older models of Claude were "well below the threshold" of being able to assist users in carrying out biological research, however, Anthropic says that it cannot say the same for newer models.

To counter any potential threats, they say recent models were launched with "stronger safeguards that restrict access to a wide range of dual-use biological research queries."

It says that without proper safeguards, the misuse of AI to create bioweapons could have "catastrophic consequences".

Anthropic noted: "Biological capabilities are dual use: they can be used for beneficial or harmful purposes, and it is often difficult to distinguish between them.

"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease.

"Sophisticated threat actors are aware that we (and other AI providers) are attempting to detect dangerous uses of our models, and they use the dual-use nature of biology to maintain a kind of “plausible deniability” about their research."

The company said that it detected and disrupted activity linked to affiliates of the ShinyHunters cybercrime collective, one of the most prolific cybercrime enterprises in recent months linked to attacks on major corporations around the world.

Jacob Klein, head of threat intelligence at Anthropic, said in an interview that models have become more capable over the last year, raising new risks.

"A year ago, let’s say you wanted to optimize a drone or optimize the software on a missile, the models just wouldn’t be as good at that task as they are now," he said.