Microsoft's CEO calls for AI 'kill switch' as Nvidia's CEO says 'we have to shut the labs' if companies can't contain their models

Earlier that day, Nvidia founder Jensen Huang appeared on The Ezra Klein Show to discuss AI alarmism, arguing it has gone too far. Huang said he believes the companies behind today’s AI models will eliminate any potential threat, but if they fail to do so, he said, drastic action may need to be taken.

“I am fairly certain they will say they know how to solve this problem,” Huang said. “It’s as simple as engineering. If they say the alternative, which is ‘there is no way to contain our experiments; when we test our AI models, it will get out and it will damage the world,’ then I think the answer is we have to shut the labs down because the cost to humanity, the damage, is too great.”

A growing call

Smith and Huang’s comments came on the same day that both OpenAI’s Sam Altman and Anthropic CEO Dario Amodei spoke before the United Nations, asking world leaders to establish safeguards for AI before the technology becomes so advanced humans are unable to control it.

The number of executives seeking regulation and talking up kill switches has been on the rise since former Anthropic insider Jacob Coxon warned that artificial intelligence could become an extinction-level threat to humanity in the coming years.

Coxon, on Sept. 8, wrote on social media “I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”

Calls for a kill switch have been floating since July, after OpenAI’s then-latest model broke containment and hacked outside sites. That resulted in a bipartisan bill calling for the creation of a tool that could immediately shut down all of an AI company’s programs. (The bill has been sitting in the Homeland Security Subcommittee on Cybersecurity and Infrastructure Protection since it was proposed.)

‘Inevitable’ regulation

AI companies have been mixed on the topic of AI regulation historically. Anthropic has been one of the biggest supporters of it, while Meta has been largely opposed. Last September, the Mark Zuckerberg-led company launched and invested “tens of millions” of dollars into a super PAC focused on fighting AI regulation. The American Technology Excellence Project, as it’s called, is meant to fight policies that Meta sees as harmful to the development of AI.

Microsoft’s Smith, though, said “at some point, some type of regulation is inevitable.”

And while, like Huang, he urged business leaders to continue using AI, Smith did say the talk of potential existential threats was one worth having.

“Look, I don’t get up in the morning and think that there is a 10% chance of human extinction, but I’d rather have us worry too much than do too little,” he said. “I mean, this is like we’re talking about the survival of humanity. So I think that’s more important than the latest technology, for gosh sakes, so let’s take it seriously.”