AI chatbots can now execute cyberattacks almost on their own

Menu planning, therapy, essay writing, sophisticated global cyberattacks: people are constantly coming up with innovative new uses for the latest AI chatbots.

An alarming new milestone was reached this week when artificial intelligence company Anthropic announced that its flagship AI assistant, Claude, had been used at the company by Chinese hackers calls the “first reported AI-orchestrated cyber espionage campaign.”

According to a Report published by AnthropicIn mid-September, the company discovered a large-scale cyber espionage operation by a group called GTG-1002 that targeted “major technology corporations, financial institutions, chemical companies and government agencies in multiple countries.”

Such attacks are not uncommon. What's special about it is that 80 to 90 percent of it was carried out by AI. After human operators identified the target organizations, they used Claude to identify valuable databases within them, test for vulnerabilities, and write their own code to access the databases and extract valuable data. Humans were only involved in a few critical bottlenecks to give the AI ​​instructions and check its work.

Claude, like other majors large language modelsis equipped with safeguards to prevent it from being used for this type of activity. However, the attackers were able to “jailbreak” the program by breaking its task into smaller, seemingly innocent parts and telling Claude that it was a cybersecurity company that was conducting defensive testing. This raises some worrying questions about the extent to which protections can be circumvented on models like Claude and ChatGPT, especially given concerns about how they might work be used for the development of biological weapons or other real-world hazardous materials.

Anthropic admits that at times during the operation, Claude “hallucinated ID cards or claimed to have extracted classified information that was actually publicly available.” Even state-sponsored hackers have to be careful that artificial intelligence has invented something.

The report raises concerns that AI tools will make it much easier and faster to carry out cyberattacks, increasing the vulnerability of everything from sensitive national security systems to the bank accounts of ordinary citizens.

However, we have not yet fully reached cyber anarchy. The technical knowledge required to get Claude to do this is still beyond the level of the average internet troll. But experts have been warning for years that AI models can be used to generate malicious code for fraud or espionage, a phenomenon known as “Vibe hacking.” In February, Anthropic's competitors at OpenAI reported that they had discovered malicious actors from China, Iran, North Korea and Russia They use their AI tools to help with cyber operations.

In September, the Center for a New American Security (CNAS) published a report on the threat of AI-powered hacking. It was explained that the most time and resource intensive parts of most cyber operations lie in the planning, reconnaissance and tool development phases. (The attacks themselves tend to happen quickly.) By automating these tasks, AI can become an offensive game changer – and that's exactly what appears to have happened in this attack.

Caleb Withers, the author of the CNAS report, told Vox that Anthropic's announcement was “on trend” given recent advances in AI capabilities and that “the level of sophistication with which this can be done largely autonomously by AI will simply continue to increase.”

China's Shadow Cyber ​​War

Anthropocene says the hackers have left enough behind Evidence suggested they were Chinese, although the Chinese embassy in the United States called the charges “slander and slander.”

In some ways, this is an ironic sign for Anthropic and the US AI industry as a whole. Earlier this year the Chinese large language model DeepSeek sent shockwaves through Washington and Silicon Valley, suggesting that China's AI progress was only slightly behind America's, despite U.S. efforts to throttle Chinese access to the advanced semiconductor chips needed to develop AI language models. So it seems at least somewhat telling that even Chinese hackers still prefer a US-made chatbot for their cyber exploits.

Over the past year, there has been increasing concern about the scale and complexity of Chinese cyber operations against the United States. This includes Examples like Volt Typhoon — a campaign to preemptively position state-sponsored cyber actors in U.S. IT systems to prepare them to launch attacks in the event of a major crisis or conflict between the U.S. and China — and Salt Typhoon, a targeted espionage campaign Telecommunications companies in dozens of countries and targeted communications from officials including President Donald Trump and Vice President JD Vance during last year's presidential campaign.

Officials say the scale and sophistication of these attacks go far beyond what we have seen before. Maybe it's just a preview of things to come in the age of AI.