A high-profile ex-OpenAI coverage researcher, Miles Brundage, took to social media on Wednesday to criticize OpenAI for “rewriting the historical past” of its deployment strategy to doubtlessly dangerous AI techniques.
Earlier this week, OpenAI printed a doc outlining its present philosophy on AI security and alignment, the method of designing AI techniques that behave in fascinating and explainable methods. In the doc, OpenAI mentioned that it sees the event of AGI, broadly outlined as AI techniques that may carry out any process a human can, as a “steady path” that requires “iteratively deploying and studying” from AI applied sciences.
“In a discontinuous world […] security classes come from treating the techniques of in the present day with outsized warning relative to their obvious energy, [which] is the strategy we took for [our AI model] GPT‑2,” OpenAI wrote. “We now view the primary AGI as only one level alongside a collection of techniques of accelerating usefulness […] In the continual world, the best way to make the following system protected and useful is to be taught from the present system.”
But Brundage claims that GPT-2 did, in actual fact, warrant considerable warning on the time of its launch, and that this was “100% constant” with OpenAI’s iterative deployment technique in the present day.
“OpenAI’s launch of GPT-2, which I used to be concerned in, was 100% constant [with and] foreshadowed OpenAI’s present philosophy of iterative deployment,” Brundage wrote in a publish on X. “The mannequin was launched incrementally, with classes shared at every step. Many safety specialists on the time thanked us for this warning.”
Brundage, who joined OpenAI as a analysis scientist in 2018, was the corporate’s head of coverage analysis for a number of years. On OpenAI’s “AGI readiness” staff, he had a selected give attention to the accountable deployment of language era techniques similar to OpenAI’s AI chatbot platform ChatGPT.
GPT-2, which OpenAI introduced in 2019, was a progenitor of the AI techniques powering ChatGPT. GPT-2 might reply questions on a subject, summarize articles, and generate textual content on a stage generally indistinguishable from that of people.
While GPT-2 and its outputs could look fundamental in the present day, they have been cutting-edge on the time. Citing the danger of malicious use, OpenAI initially refused to launch GPT-2’s supply code, opting as a substitute of give chosen information retailers restricted entry to a demo.
The determination was met with blended critiques from the AI trade. Many specialists argued that the menace posed by GPT-2 had been exaggerated, and that there wasn’t any proof the mannequin could possibly be abused within the methods OpenAI described. AI-focused publication The Gradient went as far as to publish an open letter requesting that OpenAI launch the mannequin, arguing it was too technologically vital to carry again.
OpenAI ultimately did launch a partial model of GPT-2 six months after the mannequin’s unveiling, adopted by the complete system a number of months after that. Brundage thinks this was the best strategy.
“What a part of [the GPT-2 release] was motivated by or premised on considering of AGI as discontinuous? None of it,” he mentioned in a publish on X. “What’s the proof this warning was ‘disproportionate’ ex ante? Ex publish, it prob. would have been OK, however that doesn’t imply it was accountable to YOLO it [sic] given information on the time.”
Brundage fears that OpenAI’s purpose with the doc is to arrange a burden of proof the place “issues are alarmist” and “you want overwhelming proof of imminent risks to behave on them.” This, he argues, is a “very harmful” mentality for superior AI techniques.
“If I have been nonetheless working at OpenAI, I might be asking why this [document] was written the best way it was, and what precisely OpenAI hopes to attain by poo-pooing warning in such a lop-sided approach,” Brundage added.
OpenAI has traditionally been accused of prioritizing “shiny merchandise” on the expense of security, and of speeding product releases to beat rival corporations to market. Last 12 months, OpenAI dissolved its AGI readiness staff, and a string of AI security and coverage researchers departed the corporate for rivals.
Competitive pressures have solely ramped up. Chinese AI lab DeepSeek captured the world’s consideration with its overtly accessible R1 mannequin, which matched OpenAI’s o1 “reasoning” mannequin on plenty of key benchmarks. OpenAI CEO Sam Altman has admitted that DeepSeek has lessened OpenAI’s technological lead, and mentioned that OpenAI would “pull up some releases” to raised compete.
There’s some huge cash on the road. OpenAI loses billions yearly, and the corporate has reportedly projected that its annual losses might triple to $14 billion by 2026. A sooner product launch cycle may benefit OpenAI’s backside line near-term, however presumably on the expense of security long-term. Experts like Brundage query whether or not the trade-off is value it.
