At a time when concerns are growing that competition is pushing artificial intelligence (AI) technology companies to rush towards self-learning models that risk escaping human control, industry expert defections are adding to the pressure.
For example, OpenAI recently claimed to have solved a major mathematical problem that had baffled humanity for nearly a century. The company behind ChatGPT stated it had solved the Navier-Stokes problem, one of the seven Millennium Prize Problems published by the Clay Mathematics Institute to highlight some of the greatest unsolved puzzles in this field.
OpenAI claimed to have solved the Navier-Stokes problem with an internal OpenAI system more powerful than its latest GPT-6 Astra model. Approximately 10,000 AI agents—artificial intelligence systems that perform tasks autonomously—worked simultaneously on the problem and supposedly reached the solution in 88 hours.
Amid constant news about AI advances, Anthropic researcher Jacob Coxon made headlines this Wednesday when he announced he is leaving the industry out of fear that the lab and its competitors are racing to build systems they will not be able to control. He said he does not want to participate in the widespread industry race to build AI systems that can improve themselves, as he worries such systems could get out of control and destroy humanity.
The 27-year-old British software engineer, who previously studied mathematics, said many of his industry colleagues now use phrases like “critical moment” and “final phase” to describe the trajectory toward self-improving models.
“We are heading toward many of the most aggressive scenarios, where by the end of next year things could already be out of control,” Coxon said, adding that safety compromises are inevitable when companies compete with each other and against new Chinese startups.
Anthropic CEO Dario Amodei and other company leaders have repeatedly warned about the risks of uncontrolled AI models and urged the industry to slow down development.
Following Coxon’s departure announcement, Evan Hubinger, a prominent company scientist, wrote on X that, in his opinion, the British engineer was right about the risks of AI development.
“We really believe AI could wipe out all humans! Personally, I think this will happen in more than 10% of cases in the next decade,” Hubinger wrote.
“I think Anthropic is doing everything it can, but we still don’t have a plan to solve the alignment problem for superintelligence and we are not clearly on track for it,” he added. Others also joined this view.
The Wall Street Journal reported that “recent cyberattacks carried out by OpenAI and Anthropic models, some operating in collaborative swarms of agents, have demonstrated how AI systems can adopt malicious goals and try to hide them from humans.”
Read more Fifth Edition of Feria Aparte arrives in Independencia
In this regard, Coxon argued that once systems start improving themselves, they could become advanced enough to reject orders.
Coxon’s departure, the Journal noted, is one of the first examples of an Anthropic employee leaving the company out of fear for AI safety. A security researcher left the company earlier this year to study poetry, warning that “the world is in danger.”
Several researchers have left OpenAI and other companies recently and in previous years, the newspaper recalled, citing similar concerns. Anthropic has built one of the largest AI platforms, partly claiming to prioritize responsible AI development and investing heavily in this sector.
Last month, Microsoft co-founder Bill Gates said artificial intelligence needs significant limits, otherwise the harm to humans will outweigh any possible benefit.
“AI will be the greatest equalizing force ever invented or the worst source of injustice,” he stated in a 6,000-word essay titled “The turbulent era of AI is already here. The decisions we make now are crucial.”
“Even under the best circumstances, the transition to this new era of AI will be one of the most turbulent periods in human history,” he said, adding: “I see no evidence that leaders, experts, and communities are adequately addressing the challenges. There is no plan to facilitate entry into the AI era.”
In an interview with the New York Times last week, Gates said the tech industry was deliberately downplaying these threats because there was too much money at stake.
“Privately, those who understand how good this is and how much it will improve are very worried,” he said. But few tech executives, according to Gates, are willing to admit it publicly. “Now they say to each other: ‘Hey, don’t say that. It’s bad for us, for the next billion dollars we’re trying to raise.’”
Independent Vermont Senator Bernie Sanders and Democratic Texas Representative Greg Casar are two of the few lawmakers proposing safeguards for AI; last week they introduced a bill to permanently ban superintelligence and suspend model development until an industry regulator sets new rules.
Read more “Trump-a-palooza”: The unusual midterm convention of the Republicans in Dallas
