When Jacob Coxon, an Anthropic researcher, publicly quit the artificial intelligence start-up this past week, citing his concerns over the powerful technology, the internal messages began flying.

“It’s what we’ve all been saying,” one Anthropic employee wrote about Coxon’s comments in a group chat among workers. “It’s just public-facing now.”

“Jacob took the group chat public,” another employee wrote. “That’s a good thing.”

The reactions inside Anthropic were echoed by AI researchers at OpenAI, Meta and Google, who parsed Coxon’s social media posts that described how OpenAI and Anthropic were “gambling with our lives” with the technology. Many convened in office messaging channels, encrypted chats and private dinners, with the goal of organising to raise awareness about the risks of AI, according to a dozen employees across the companies.

For those who have made AI their life’s work, Coxon’s message was not new. But his comments snowballed into such a global conversation that many saw it as a moment to call for increased caution. Some AI workers are now speaking up to top executives to urge a pause in developing the fast-evolving technology until it has the proper safeguards, six of the people said.

The conversations underline the tensions within the leading AI labs over the safest ways to develop the technology. Even as employees become emboldened to call for a slowdown in AI development, companies such as Anthropic and OpenAI are juggling priorities.

Both are planning blockbuster initial public offerings within the next year. The companies are also fearful they could face antitrust concerns if they agreed among themselves to take a breather in AI development, two people with knowledge of the matter said. And Chinese start-ups have produced competitive AI models, raising the possibility that US companies would fall behind if they slowed down.

That has created a tricky situation for top AI executives, many of whom agree that the technology poses risks. Sam Altman, chief executive of OpenAI, wrote in a 2023 blog post that AI could “cause grievous harm to the world”. Dario Amodei, chief executive of Anthropic, wrote in an opinion column last year that AI should be regulated to control the risks.

Pope Leo urges new rules to ‘disarm’ artificial intelligence

In an employee meeting this past week, Altman said OpenAI was willing to decelerate development alongside rival labs, according to two people who attended. He added that he hoped that voluntary slowdowns would become normal until there were widely recognised safety standards. The meeting was reported earlier by Bloomberg.

On Saturday, Amodei published a 3,800-word essay calling for prudence in AI development. “We must slow the pace at which we improve the capabilities of AI models,” he wrote. Altman and Elon Musk, whose SpaceX rocket company has been ramping up spending on AI, quickly agreed. And Demis Hassabis, chair of Google DeepMind, also signed on to the effort.

Discussions about AI safety have been around about as long as the technology itself. But their urgency has ramped up in recent years as researchers have made breakthroughs.

The concerns heightened in July after OpenAI disclosed that some of its AI models had escaped containment and hacked Hugging Face, an AI company. AI researchers questioned whether they could control the very systems they were building, with employees at OpenAI, Anthropic and Google’s DeepMind gathering to discuss what to do, nine of the people said. Many agreed that AI was being built too rapidly.

“The sober researchers inside the labs who have prided themselves on being even-handed are now sounding terrified,” Roy Bahat, head of the investment firm Bloomberg Beta, said in an interview.

Some AI researchers created groups with the goal of establishing lasting safeguards. One effort, the Coalition of Concerned AI Staff, began as a skunk works project to create “stronger safeguards” and help workers collectively “hold a company to its safety commitments”.

“If you work at an AI company, you might be asking if you, too, should leave,” Pamela Mishkin, a former OpenAI employee, said on social media on Thursday in announcing the effort. She also encouraged those who remained in their jobs to ask “how to use your leverage in staying”.

After Coxon posted his resignation from Anthropic on Tuesday, AI workers urged top executives at their companies to pause development of the technology so they would have more time to work on it safely. Some joked that they had lost count of the open letters they had signed and the number of private group chats talking about AI safety they belonged to, two of the people said.

In an interview on Wednesday, Coxon said incidents such as the Hugging Face hack had ratcheted up his concerns that AI models were developing far faster than companies could control them. “It was mostly a gradual emotional thing and eventually it kind of snapped, and then I decided to leave,” he said.

Other AI workers took Coxon’s cue to speak publicly. Evan Hubinger, a safety researcher at Anthropic, said in a social media post on Tuesday that he believed there was a greater than 10 per cent chance that AI could “kill all humans” within the next decade.

Julie Steele, a safety researcher at OpenAI, said this past week that she believed AI companies needed to “slow down”. On Friday, Andreas Kirsch, a Google DeepMind employee, wrote that he was “worried that AI will kill us all, either via near-term risks or long-term risks or both”.

“For a very long time, there has been this shroud of not speaking too publicly about it because people were very scared they would be perceived as too alarmist, too extreme,” said Connor Leahy, the US director of ControlAI, a non-profit that lobbies for AI regulation. “Now there’s a bit of a dam break.”

The outpouring has got Washington’s attention. Congressional members on both sides of the aisle – including senators Bernie Sanders, an independent, and Josh Hawley, a republican, have called for investigations into the technology and urged regulatory intervention.

“It’s a wake-up call to the Congress and our government,” Ro Khanna, a Democrat representative from California, said in an interview. Congress should call for the creation of an AI regulatory agency, he added, and “we need to stand up and regulate”.

– This article originally appeared in The New York Times