On Thursday, weeks after launching its strongest AI mannequin but, Gemini 2.5 Pro, Google printed a technical report exhibiting the outcomes of its inner security evaluations. However, the report is mild on the small print, specialists say, making it troublesome to find out which dangers the mannequin may pose.
Technical stories present helpful — and unflattering, at instances — data that firms don’t at all times extensively promote about their AI. By and huge, the AI neighborhood sees these stories as good-faith efforts to assist unbiased analysis and security evaluations.
Google takes a unique security reporting strategy than a few of its AI rivals, publishing technical stories solely as soon as it considers a mannequin to have graduated from the “experimental” stage. The firm additionally doesn’t embody findings from all of its “harmful functionality” evaluations in these write-ups; it reserves these for a separate audit.
Several specialists TechCrunch spoke with have been nonetheless upset by the sparsity of the Gemini 2.5 Pro report, nevertheless, which they famous doesn’t point out Google’s Frontier Safety Framework (FSF). Google launched the FSF final 12 months in what it described as an effort to establish future AI capabilities that would trigger “extreme hurt.”
“This [report] could be very sparse, comprises minimal info, and got here out weeks after the mannequin was already made obtainable to the general public,” Peter Wildeford, co-founder of the Institute for AI Policy and Strategy, instructed TechCrunch. “It’s inconceivable to confirm if Google resides as much as its public commitments and thus inconceivable to evaluate the protection and safety of their fashions.”
Thomas Woodside, co-founder of the Secure AI Project, mentioned that whereas he’s glad Google launched a report for Gemini 2.5 Pro, he’s not satisfied of the corporate’s dedication to delivering well timed supplemental security evaluations. Woodside identified that the final time Google printed the outcomes of harmful functionality exams was in June 2024 — for a mannequin introduced in February that very same 12 months.
Not inspiring a lot confidence, Google hasn’t made obtainable a report for Gemini 2.5 Flash, a smaller, extra environment friendly mannequin the corporate introduced final week. A spokesperson instructed TechCrunch a report for Flash is “coming quickly.”
“I hope this can be a promise from Google to begin publishing extra frequent updates,” Woodside instructed TechCrunch. “Those updates ought to embody the outcomes of evaluations for fashions that haven’t been publicly deployed but, since these fashions might additionally pose severe dangers.”
Google could have been one of many first AI labs to suggest standardized stories for fashions, nevertheless it’s not the one one which’s been accused of underdelivering on transparency recently. Meta launched a equally skimpy security analysis of its new Llama 4 open fashions, and OpenAI opted to not publish any report for its GPT-4.1 sequence.
Hanging over Google’s head are assurances the tech big made to regulators to keep up a excessive normal of AI security testing and reporting. Two years in the past, Google instructed the U.S. authorities it might publish security stories for all “important” public AI fashions “inside scope.” The firm adopted up that promise with related commitments to different international locations, pledging to “present public transparency” round AI merchandise.
Kevin Bankston, a senior adviser on AI governance on the Center for Democracy and Technology, referred to as the pattern of sporadic and obscure stories a “race to the underside” on AI security.
“Combined with stories that competing labs like OpenAI have shaved their security testing time earlier than launch from months to days, this meager documentation for Google’s high AI mannequin tells a troubling story of a race to the underside on AI security and transparency as firms rush their fashions to market,” he instructed TechCrunch.
Google has mentioned in statements that, whereas not detailed in its technical stories, it conducts security testing and “adversarial pink teaming” for fashions forward of launch.
