OpenAI’s Superalignment team, liable for developing ways to control and steer “superintelligent” AI systems, was promised 20% of the corporate’s compute resources, in response to an individual from that team. But requests for a fraction of that compute were often denied, blocking the team from doing their work.
That issue, amongst others, pushed several team members to resign this week, including co-lead Jan Leike, a former DeepMind researcher who while at OpenAI was involved with the event of ChatGPT, GPT-4, and ChatGPT’s predecessor, InstructGPT.
Leike went public with some reasons for his resignation on Friday morning. “I actually have been disagreeing with OpenAI leadership in regards to the company’s core priorities for quite a while, until we finally reached a breaking point,” Leike wrote in a series of posts on X. “I imagine far more of our bandwidth needs to be spent preparing for the following generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact, and related topics. These problems are quite hard to get right, and I’m concerned we aren’t on a trajectory to get there.”
OpenAI didn’t immediately return a request for comment in regards to the resources promised and allocated to that team.
OpenAI formed the Superalignment team last July, and it was led by Leike and OpenAI co-founder Ilya Sutskever, who also resigned from the corporate this week. It had the ambitious goal of solving the core technical challenges of controlling superintelligent AI in the following 4 years. Joined by scientists and engineers from OpenAI’s previous alignment division in addition to researchers from other orgs across the corporate, the team was to contribute research informing the protection of each in-house and non-OpenAI models, and, through initiatives including a research grant program, solicit from and share work with the broader AI industry.
The Superalignment team did manage to publish a body of safety research and funnel hundreds of thousands of dollars in grants to outside researchers. But, as product launches began to take up an increasing amount of OpenAI leadership’s bandwidth, the Superalignment team found itself having to fight for more up front investments — investments it believed were critical to the corporate’s stated mission of developing superintelligent AI for the advantage of all humanity.
“Constructing smarter-than-human machines is an inherently dangerous endeavor,” Leike continued. “But over the past years, safety culture and processes have taken a backseat to shiny products.”
Sutskever’s battle with OpenAI CEO Sam Altman served as a serious added distraction.
Sutskever, together with OpenAI’s old board of directors, moved to abruptly fire Altman late last 12 months over concerns that Altman hadn’t been “consistently candid” with the board’s members. Under pressure from OpenAI’s investors, including Microsoft, and lots of the company’s own employees, Altman was eventually reinstated, much of the board resigned, and Sutskever reportedly never returned to work.
In accordance with the source, Sutskever was instrumental to the Superalignment team — not only contributing research but serving as a bridge to other divisions inside of OpenAI. He would also function an envoy of sorts, impressing the importance of the team’s work on key OpenAI decision makers.
Following the departures of Leike and Sutskever, John Schulman, one other OpenAI co-founder, has moved to go up the sort of work the Superalignment team was doing, but there’ll not be a dedicated team — as an alternative, it can be a loosely associated group of researchers embedded in divisions throughout the corporate. An OpenAI spokesperson described it as “integrating [the team] more deeply.”
The fear is that, because of this, OpenAI’s AI development won’t be as safety-focused because it could’ve been.
