At Google I/O 2024, there were several exciting announcements about Google’s advancements in generative AI and multi-modal technologies. One of the standout features coming to Google’s suite of apps is a new video search tool that will revolutionize Google Lens.
The new “Search with Video” feature in Google Lens lets you record videos of anything you’re curious about and share them with Google. Using Gemini, it will analyze the video to understand the context and provide you with the answers you’re seeking.
Rose Yao, Senior Director of Google Product Management, gave a brief live demonstration of this feature. She recorded a video of a vinyl record player and inquired why its tonearm was drifting without actually mentioning the word “tonearm.” Nevertheless, Gemini recognized that she wanted to address the tonearm issue and provided relevant results.
The custom Gemini model was able to capture her voice from the uploaded video. It analyzed the video frame-by-frame to identify the device’s make and model and detect motion to understand the issue. This feature will be available within Google Lens, and you should be able to try it out in the coming weeks. Users will need to opt in through Google Search Labs.
This feature will make web searches much more accessible, especially for users with hand impairments and our grandparents who struggle with navigating Google searches. What are your thoughts on this feature? Are you excited to try it out? Let us know in the comments below.
