David Robinson has been a part of OpenAI for 3.5 years. Then he quit. His warning is clear: it's time for trial and error to be over.
The AI Safety Employee Who Quit OpenAI
Robinson contributed to the company's preparedness plan and preparedness reports on 12 frontier-model launches. He wrote his resignation letter in the Atlantic, in which he states that the company's culture is flawed. He feels that it is too short a launch schedule to allow for careful safety work.
He talks about a company running from one release to another. At that rate, he notes, he's not getting the care levels that advanced AI systems require.
What Is Iterative Deployment?
OpenAI's approach is heavily influenced by its “iterative deployment” methodology. When problems arise, the company tests the system, observes its performance and increases its safety measures.
This is the case for many products. Programs are fixed for bugs. Features get refined. However, Robinson suggests that the method may not be suitable for frontier AI models, which could have more severe repercussions if it fails. If it is a big deal, it's too late to fix issues after you launch.
Lessons From Nuclear Power and Aviation
Robinson cites industries that got the lesson. Nuclear power and the aviation industry are subject to highly effective safeguards, a series of checks, and a reliable and comprehensive level of safety knowledge. In these professions engineers plan for failure before it even happens.
His reasoning goes along these lines: If there is enough reason to, in the first place, worry about advanced AI safety, then it is reasonable to believe it should be done in a similar fashion. First companies must invest in safety research and in specialized expertise, then in the development of more robust systems. Speed comes second.
AI Alignment: Capabilities Are Moving Faster
AI capabilities are progressing at a rate that outpaces understanding of alignment, Robinson added. The field dealing with ensuring that AI systems align with human objectives and values is Alignment.
That gap matters. As models get more sophisticated and the science of controlling them becomes less sophisticated, there is an increased risk. Researchers might not be able to determine the behavior of a system, or to prevent it from malfunctioning.
OpenAI's Response
OpenAI has retaliated. The company strives to keep its models as capable as it can safely run and secure it, a spokesperson said. The spokesperson also indicated that OpenAI slows down its training or releases a “model reservation” when it wants to slow down.
This is a disagreement, not a right and wrong thing. Both parties emphasize the importance of AI safety. There are conflicting views on how much caution is sufficient.
A Wider Debate Across the AI Industry
The essay by Robinson joins the debate on the speed of AI development. Both OpenAI and its competitors, Anthropic, have come under criticism for failures to adhere to safety protocols or strange behavior among experimental systems.
The following episodes leave a tough issue unaddressed: should AI companies sprint ahead and get to the right thing or slow down and demonstrate safety before it's released?
What This Means for Everyone
AI systems are invading the world of work, school and home. The individuals constructing them are the ones that mold the sense of security that that future offers. In a gesture of good faith and transparency, Robinson has resigned and is now offering a serious examination of the trade-offs in AI companies between ambition and responsibility.
Intelligent individuals may have different opinions on the proper speed. The important thing is to keep the dialogue going, to encourage safety researchers to be heard and to take deliberate steps to make decisions regarding frontier AI models.
The debate isn't about if AI will progress, it's about when. It is the quality of our approach and desire to get there.
