A group of current and former OpenAI employees is urging the company and other AI firms to protect employees who raise safety concerns about artificial intelligence. They published an open letter on Tuesday, which calls for stronger whistleblower protections to allow researchers to flag AI risks without fear of retaliation. As AI systems develop rapidly, there are strong incentives to proceed without sufficient caution, said Daniel Ziegler, a former OpenAI engineer and one of the letter’s organizers. Ziegler worked at OpenAI between 2018 and 2021, contributing to some of the techniques that made ChatGPT successful. He now worries that the push to commercialize AI technology is leading companies to overlook potential dangers.
Concerns Over Responsible AI Development
Another co-organizer, Daniel Kokotajlo, left OpenAI this year due to concerns about the company’s approach to building advanced AI systems, known as artificial general intelligence. Kokotajlo criticized the industry’s “move fast and break things” mentality, arguing it’s unsuitable for a technology as powerful and poorly understood as AI. OpenAI responded to the letter, emphasizing that it has mechanisms in place for employees to express concerns, such as an anonymous integrity hotline. The company highlighted its commitment to developing safe AI systems and engaging in rigorous debate about the technology.
Key takeaways
- Current and former OpenAI employees published an open letter calling for stronger whistleblower protections so researchers can raise AI safety concerns without retaliation.
- The letter carries 13 signatories including former OpenAI staff and people affiliated with Google DeepMind, with four current OpenAI employees listed anonymously.
- It is endorsed by Yoshua Bengio, Geoffrey Hinton and Stuart Russell, all of whom have warned about risks from future AI systems.
- A central demand is ending non-disparagement agreements that can strip departing employees of their equity for criticising the company; OpenAI had recently exempted former employees from such terms after public backlash.
- OpenAI responded by pointing to existing channels including an anonymous integrity hotline, and to its stated commitment to safe development.
Support From AI Pioneers
The open letter has 13 signatories, including former OpenAI employees and individuals affiliated with Google’s DeepMind. Four current OpenAI employees are listed anonymously. The letter calls for companies to eliminate “non-disparagement” agreements that can penalize departing employees by revoking their equity if they criticize the company. In response to social media backlash, OpenAI recently exempted former employees from such agreements. The letter is endorsed by prominent AI scientists Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, who have all warned about the existential risks posed by future AI systems.
Industry-Wide Implications
This call for increased oversight comes as OpenAI begins developing the next generation of AI technology behind ChatGPT. The company formed a new safety committee after losing key leaders focused on the safe development of powerful AI systems. The AI research community has long debated the risks associated with AI and how to balance them with commercial interests. These ongoing conflicts have influenced leadership decisions at OpenAI and continue to create distrust in its management.
Effective Altruism and AI Ethics
Many signatories, including Ziegler, are associated with effective altruism—a movement focusing on addressing significant global challenges such as AI safety. Ziegler stated that the letter’s authors are concerned not just with out-of-control AI systems, but also with issues of fairness, product misuse, job displacement, and the potential for AI to manipulate people without adequate safeguards. He emphasized that the aim is to foster greater oversight, transparency, and public trust in AI companies, rather than to criticize OpenAI specifically.
Frequently asked questions
Who organised the letter and why?
Former OpenAI engineer Daniel Ziegler, who worked there from 2018 to 2021 on techniques behind ChatGPT, and Daniel Kokotajlo, who left over concerns about the company's approach to artificial general intelligence. Kokotajlo argued a 'move fast and break things' culture is wrong for a technology this powerful and poorly understood.
Is the concern only about runaway AI systems?
No. Ziegler listed fairness, product misuse, job displacement and the potential for AI to manipulate people without adequate safeguards alongside loss-of-control scenarios.
Was this aimed specifically at OpenAI?
Ziegler framed it as the opposite — a call for greater oversight, transparency and public trust across AI companies generally, rather than a criticism of OpenAI alone.
