Chinese AI models give hackers powerful cybersecurity tools with fewer restrictions

372     0
Chinese AI models give hackers powerful cybersecurity tools with fewer restrictions
Chinese AI models give hackers powerful cybersecurity tools with fewer restrictions

When local cybersecurity start-up DepthFirst sought artificial intelligence software to help hunt computer bugs, it turned to a Chinese company, not the top U.S. AI labs in its hometown.

American AI firms such as Anthropic and OpenAI restrict their chatbots, Claude and ChatGPT, respectively, from helping with cybersecurity tasks for all but certain, vetted users. Their leading Chinese rivals, however, such as DeepSeek and Z.ai, allow anyone to download and modify their AI models for nothing.

Earlier this year, DepthFirst heavily modified a version of Z.ai’s free GLM AI model and combined it with other tools to make a system that can ferret through software to find flaws that could be exploited by malicious hackers.

In a video demonstrating its prowess that was shared with The Washington Post, a DepthFirst employee remotely accessed the camera and photo roll on a smartphone browsing TikTok by exploiting a series of bugs inside the popular video app. The hack was for demonstration purposes and wasn’t carried out on real users.

The flaws that allowed remote access to a TikTok user’s phone were found by DepthFirst’s AI bug hunter in a piece of open-source software used by the social video company. DepthFirst disclosed the vulnerability to TikTok, which confirmed it and fixed the problem, according to messages from the company’s security team reviewed by The Post.

 “This issue was reported through TikTok’s bug bounty program months ago and quickly patched. We have no evidence it was ever exploited against TikTok users,” TikTok said in a statement received after this article published. “We routinely work with independent security researchers to identify and resolve potential vulnerabilities and value their contributions.”

DepthFirst uses its bug-hunting AI to help clients shore up their defenses, and it has found dozens of serious vulnerabilities in software used by millions of people, said Qasim Mithani, the company’s chief executive. Its power has also left him convinced that the world faces a reckoning from the growing power of AI as a tool for cyberattackers.

In April, Anthropic warned that the latest version of its Claude AI model was skilled at both cyberdefense and cyberoffense, prompting a scramble from the Trump administration and other governments to understand the risks. Anthropic and OpenAI have limited their AI models’ hacking skills and only permit vetted companies to access their full power, for defensive purposes.

Months later, free Chinese AI models that have far fewer restrictions also have potent cybersecurity skills. They give just about anyone in the world access to high-level hacking knowledge that can be used for defense or attack.

“It’s something that keeps me up at night,” said Mithani. “You’re going to see these open-weight models get really good,” he said, using an industry term for models that anyone can use or modify.

‘You shouldn’t trust any app’

Cybersecurity firms report that hackers around the world have already supercharged their operations with AI tools. Defenders can use AI, too. But widening access to the technology is making sophisticated hackers more capable and lowering the bar for people with more limited skills to try their hand at cybercrime, said John Hultquist, chief analyst with Google’s Threat Intelligence Group.

“There’s a lot of technical changes that we’ve experienced over the years that have changed the nature of these threats,” said Hultquist. “This is by far the most significant.”

Hamza Chaudhry, the head of AI and national security at the nonprofit Future of Life Institute, said that the expected arrival in a few months of open Chinese models that rival the power of the best currently offered by Anthropic and OpenAIwill create significant new security vulnerabilities.

In the past, relatively few hackers could breach the most important targets, he said. “If these systems are better than or just as good as those hackers, now a lot more people can hit things that have never been hit before,” said Chaudhry, whose organization advocates for AI safety regulations.

The Pentagon’s top cyberdefense officer said last week that it has seen a “tenfold” increase in security vulnerabilities as AI has emerged as a powerful tool for attackers.

Although Mithani says defensive work like that done at DepthFirst will continue to become more powerful thanks to AI, he says consumers may need to change how they think about digital security. Well-worn advice about looking out for phishing emails and maintaining good passwords might not be enough, he said.

“Consumers have to be very careful,” said Mithani. He recommends people regularly update their apps and limit app access to photos, location and their phone’s camera. “You shouldn’t trust any app,” he said.

Z.ai said when it announced the latest version of GLM last month that it worked with outside security experts to evaluate the AI model’s cybersecurity skills and would publicly disclose vulnerabilities found by the software. The company suggested in a post on X that GLM could help improve computer security. “An open world cannot have only open attack surfaces. It must also have an open shield,” the company said.

‘Nonstop in the red’

Alarm bells about AI-powered hacking began to ring in April, when Anthropic announced a major breakthrough. Its latest AI model, Mythos, was so capable at understanding and navigating computer code that it could find software bugs that human developers and hackers had missed for years.

Anthropic’s leaders said the technology was so dangerous that it could be provided only to a small number of cybersecurity and internet infrastructure companies.

Mythos set off a panic among national security officials around the world. It spurred the Trump administration to stray from its laissez-faire approach to AI regulation and move to require companies to submit new, high-end models for government review before release. ChatGPT-maker OpenAI has said its own AI models are also powerful at finding security flaws. (The Washington Post has a content partnership with OpenAI.)

In recent months, cybersecurity companies have been working overtime trying to use AI to find potential bugs and help their clients get them fixed.

“Our team has been incredibly busy, nonstop in the red,” said Sam Rubin, senior vice president of threat intelligence at Unit 42, which is part of the cybersecurity company Palo Alto Networks.

The company reported last month that when Unit 42 used AI-powered tools to analyze almost 4,000 open-source software projects, it found more than 14,000 confirmed vulnerabilities. More than 99 percent were previously unreported, it said.

Rubin said the company has also observed that attackers can move faster. In one incident analyzed by his team, attackers used more than 50 different techniques to break into a European software company with the aim of ransoming its data. An operation of that sophistication would normally take more than 10 days to pull off, Rubin estimated, but took place in just 10 hours.

Even companies building AI aren’t safe from hacks enabled by the technology. On Thursday, a group of independent cybersecurity researchers said they had in July used their access to a version of Anthropic’s AI without limits on cyber use to find a way to hack into OpenAI’s computer systems. The hackers disclosed the exploit to OpenAI, and the company patched it. The Wall Street Journal first reported on the OpenAI breach.

Nation state-backed hackers are also turning to AI. Anthropic said last week that a group affiliated with Russian intelligence services that at times used its Claude chatbot had built a set of malware tools that automatically rebuilt themselves when they ran into defensive software that detected them.

The spies used it to break into the computers of a military drone maker and stole blueprints, component lists and proprietary software, according to a report from Anthropic, which said it cut off the group from using its technology.

 The drone operation is just one of dozens of attacks that take advantage of the latest AI models reported by cybersecurity organizations in the past several months. Open Chinese models have been used in many of them. Chinese companies DeepSeek and Z.ai currently outrank all U.S. models except for OpenAI’s restricted-access model for cybersecurity work on CyberGym, a series of tests meant to evaluate how capable an AI model is at cybersecurity tasks.

Although they can enable more sophisticated operations, free AI models aren’t perfect hackers. Unit 42 reported in July that a China-based attacker had used DeepSeek’s free AI model to power an autonomous AI “agent” that searched for software vulnerabilities, selected potential hacking targets and then tried to break in using the bugs it found.

The attack was revealed after the AI agent appeared to respond to a command from its operator by installing a software in the wrong place, exposing target lists and other details of the hacking campaign.

“Open-weight models … still do lag the flagship frontier cyber models,” said Rubin, “but they’re rapidly getting better.”

Editorial Team

Elizabeth Baker

Technology & Business Editor

Print page

Comments:

comments powered by Disqus