If you possibly can’t seize what you wish to seek for with only a image, Google Lens will now allow you to take a video — and even use your voice to ask about what you’re seeing. The characteristic will floor an AI Overview and search outcomes based mostly on the video’s contents and your query. It’s rolling out in Search Labs on Android and iOS in the present day.
Google first previewed utilizing video to look at I/O in May. As an instance, Google says somebody curious concerning the fish they’re seeing at an aquarium can maintain up their telephone to the exhibit, open the Google Lens app, after which maintain down the shutter button. Once Lens begins recording, they will say their query: “Why are they swimming collectively?” Google Lens then makes use of the Gemini AI mannequin to supply a response, just like what you see within the GIF beneath.
When talking concerning the tech behind the characteristic, Rajan Patel, the vp of engineering at Google, advised The Verge that Google is capturing the video “as a collection of picture frames after which making use of the identical pc imaginative and prescient strategies” beforehand utilized in Lens. But Google is taking issues a step additional by passing the data to a “customized” Gemini mannequin developed to “perceive a number of frames in sequence… after which present a response that’s rooted within the internet.”
There isn’t help for figuring out the sounds in a video simply but — like should you’re attempting to establish a hen you’re listening to — however Patel says that’s one thing Google has been “experimenting with.”
Google Lens can be updating its photograph search characteristic with the flexibility to ask a query utilizing your voice. To attempt it, goal your digicam at your topic, maintain down the shutter button, after which ask your query. Before this alteration, you would solely kind your query into Lens after snapping an image. Voice questions are rolling out globally on Android and iOS, however it’s solely accessible in English for now.
