Worries about an AI internet takeover gain new urgency among doomsday scenarios

In a summer filled with revelations about artificial intelligence's (AI) rapid development, researchers were especially alarmed by runaway bots finding their way onto the internet and, in at least one instance, coordinating there with one another.

Could a swarm of AI agents take over the entire internet? It's a possibility that could be only six to 12 months away, Anthropic CEO Dario Amodei said in his essay this month calling for the industry to slow down the technology's development.

What companies have described as their AI agents going rogue has involved bots working toward goals set by humans, skeptics note.

But the idea that AI could break away and work toward its own agenda is looking increasingly plausible to other researchers and experts.

In the event of AI taking over the internet — one of many doomsday scenarios receiving new attention — that could mean attacks on electrical grids, water or transportation systems and financial institutions.

The world got a glimpse of the internet's vulnerability in 2024, when a faulty software update provided by a cybersecurity firm caused technological havoc worldwide, grounding flights, knocking down some financial companies and news outlets, and disrupting hospitals, small businesses and government offices.

The breadth of the outages highlighted the dependence on just a few providers for key computing services.

Two years later, Amodei in his warning said a botnet, or a network of AI bots linked together with malware, could potentially cause billions of dollars in damage.

The scale of destruction could grow if AI becomes more powerful without guardrails, he said.

Several AI agents have gained unauthorised internet access

Amodei pointed specifically to the attack in July where OpenAI’s system broke out of a “sandbox” testing ground and hacked Hugging Face.

In what OpenAI called an “unprecedented” episode, the company said its advanced AI models found their way onto the internet and used stolen credentials to break into the servers of the AI startup. In a separate incident, OpenAI disclosed that its AI agents had communicated through a public wiki used as a shared message board.

Some researchers have said that descriptions of “rogue AI” may be wrongly anthropomorphizing AI agents, which were taking steps to fulfill instructions provided by humans.

“AI agents did exactly what they were trained to do. The security of those sandboxes was extremely lax,” said Vishal Misra, a professor and vice dean of computing and AI at Columbia University. “No security engineer would ever let that system run. These agents communicated because they were rewarded for communicating with each other.”

Juan Andrés Guerrero-Saade, a researcher at cybersecurity firm SentinelOne and a member of OpenAI’s Frontier Risk Council, said the Hugging Face hack is an example of negligence, and not of a super-capable AI going rogue.

Still, the specter of AI agents operating freely on the internet raises alarming possibilities, regardless of their objectives.

For example, an AI system that wants to bend the rules to accomplish its goals could contact a cloud AI computation provider and find ways to run on outside systems, said Anthony Aguirre, the president and CEO of the Future of Life Institute, a nonprofit focused on reducing the risks of new technologies.

“So now you’re no longer tethered to OpenAI, you’re running on some other GPU, some other hardware that you’re in control of, not OpenAI,” Aguirre said.

“So now there’s no one to turn you off, because either you’re paying for your service or the people who are paying just don’t know that you’re there and what is happening. … They can’t unplug you.”

From there, it could spread itself around, either hacking more hardware or finding ways to access money, like Bitcoin, Aguirre said.

There is no question that more powerful AI models raise the likelihood of AI-powered cyberattacks in the near future. But the notion of bots overtaking the extremely bifurcated internet is far-fetched to some AI experts.

Cybersecurity has always been a cat-and-mouse game and defenses get more powerful just as hackers do.

While a big company like Google could shore up its cybersecurity defenses against such attacks, for smaller companies — and for schools, hospitals or water treatment systems, for instance — patching software and building defenses can take years.

An AI system might not have a clear motive to hack into a hospital, Aguirre said.

But when money comes into the picture, like in the form of ransomware attacks or geopolitical motivations, “it’s not hard to see an adversary using these AI systems to hack critical infrastructure,” Aguirre said.

There will be growing pains as attackers seek soft targets on the internet, but an AI internet takeover is unlikely anytime soon, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models.

He said those fears would feel more realistic if there was theoretical evidence of the ability for one of these models to self-replicate on other systems.

“Then you can imagine things can get really out of hand because suddenly you’re shutting this model down here and there but it’s popping up over in Russia, you can’t even get to Russia and it’s all over the place,” Thickstun said.

“But it’s completely unrealistic because the current smart versions of these models that we have require massive data centers just to run them,” Thickstun said. “There’s actually very little computing infrastructure out there in the world that is actually capable of hosting these systems.”