I’m not going to lie — I don’t typically jump or get riled up by claims that AI is going to take over the world or threaten humanity, beyond the insane energy use and emissions issue, that is. I first heard such warnings 23 years ago. Yes, 23 years ago. That’s a couple of decades before ChatGPT, Claude, or Grok were a thing. The speaker, a renowned environmentalist author and speaker who was giving a keynote presentation at an event I was attending at an Ivy League school, had apparently read the same kind of sci-fi stuff as various AI guys, like Elon Musk, who have been warning about AI taking over society. Now, though, we’ve got some more specific threats and warnings about where AI is headed, from a top AI CEO and backed by other AI CEOs.
Dario Amodei, the CEO of Anthropic, just wrote the article “We Must Pace the Frontier.” Here’s his short summary of it: “I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.” Sam Altman and Elon Musk backed him up in agreement, something these three don’t typically do.
Here’s a key line from the article that I saw highlighted by Anatoli Kopadze: “My second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the ‘grader’ responsible for evaluating their performance. It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.” (Emphasis via bolding added.)
Are we on the verge of AI creating AI that takes over the internet and causes hundreds of billions of dollars of damage? Are these companies going to voluntarily find ways to avoid such disaster? (I think any hope of government adequately regulating them is out the window.)
I definitely don’t have the expertise to answer either question, and it seems no one does. These AI CEOs are also prone to ridiculous, fantastical, techno-utopia dreams from what I’ve seen. In that same article, Amodei also wrote: “I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.” I’m sorry, but some of these hopes or even expectations are ridiculous, and they just repeat fantasies that pop up generation after generation. So, maybe it’s the same for the dire, dramatic warnings Amodei and crew have been coming up with. However, in this case, it’s a more specific, clear warning and concerns things these guys actually, really have expertise in. So, the forecast and warning feel much more realistic. We will see….
This is what Anatoli Kopadze tweeted on the matter:
“Ok this is starting to feel like a f*cking disaster.
“The CEO of Anthropic just published an article admitting AI is already building the next generation of AI by itself.
“He says within 6 to 12 months a rogue swarm could take over the entire internet and cause hundreds of billions of dollars in damage.
“And what makes it scarier, Elon Musk just backed up everything Dario said.
“All of this dropping just days after Jacob Coxon went viral with his warning about AI and the extinction of humanity.
“Tell me this timing isn’t strange.”
It is an interesting collection of people and warnings. We’ll see where things head.
UPDATE: Oh, also note the following recent warning from Jacob Coxon, a former Anthropic and OpenAI researcher who resigned from Anthropic out of concern about where these AI leaders are heading. Coxon said that they are “racing straight to self-improving superintelligence and gambling with our lives.” That doesn’t sound good. “The people building AI earnestly believe that it could kill us all by the end of the decade,” he added. “This is not a marketing stunt.” In response, some in the industry were critical and dismissive while others backed up the concerns. “Jacob is correct here—we really do earnestly believe AI could kill all humans!” said Evan Hubinger, the alignment science lead at Anthropic. “I personally think it is >10% within the next decade.” …
Featured image from cottonbro studio on Pexels.