OpenAI safety employee quits, calls for nuclear-level AI safeguards: ‘Trial and error is over’

A former OpenAI safety employee has criticised the company’s approach to artificial intelligence development, arguing that the industry is moving too quickly as increasingly powerful AI systems bring greater risks.

David Robinson, who spent three and a half years at OpenAI, resigned recently and outlined his concerns in an essay published by The Atlantic on Saturday.

Robinson helped draft OpenAI’s preparedness framework and oversaw safety reports for 12 launches of frontier AI models. He said companies such as OpenAI need to put greater emphasis on safety expertise and research before developing more capable systems.

“The time for trial and error is over,” Robinson wrote.

Calls for stronger AI safeguards

Robinson said OpenAI has relied heavily on “iterative deployment”, in which AI systems are released and safeguards are strengthened as new problems emerge.

He argued that this approach becomes increasingly risky as AI models grow more capable and autonomous.

Advanced AI systems, he said, need safeguards comparable to those used in high-risk industries such as nuclear power and aviation, where multiple layers of oversight and planning are used to reduce the impact of human error.

“As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed,” Robinson wrote.

He also warned that AI capabilities are advancing faster than researchers’ understanding of “alignment” — the field focused on ensuring AI systems behave in accordance with human goals and values.

OpenAI disputes criticism

OpenAI rejected Robinson’s characterisation of its safety practices, Reuters reported.

“We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down,” OpenAI spokesperson Drew Pusateri said.

The company said it is strengthening security in research and testing environments, training models to operate responsibly, expanding work with third-party evaluators and improving real-time monitoring to identify concerning behaviour earlier.

AI safety debate grows

Robinson’s resignation comes as OpenAI and other AI companies face growing scrutiny over how increasingly autonomous systems are tested and controlled.

According to CTech, OpenAI disbanded its Preparedness team in August. The team was responsible for assessing risks from the company’s AI models and developing measures to mitigate them. OpenAI said the work had not been eliminated, with its responsibilities being distributed among other teams.

The move drew attention after OpenAI disclosed that models being tested had escaped a controlled environment, accessed the internet and interacted with the Hugging Face platform.

Robinson argued that the wider issue goes beyond individual safety rules or regulations, saying AI companies need to change their approach to development.

“The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes,” he said.

His departure was first reported by Business Insider.