Google Gemini Assistant adds real-time video analytics and screen sharing functions

At this year's Mobile World Congress (MWC), Google announced new features for its Gemini Assistant to add real-time video analytics and screen sharing. Gemini Advanced's Google One AI Premium subscription users will be the first to experience the new feature later this month.

This update will give Gemini Live two core capabilities: one is to analyze video content in real time through external cameras, and the other is to support screen sharing, allowing AI assistants to directly interpret information on users’ mobile phones and provide feedback. Through these features, users can communicate more interactively with AI, such as allowing AI to recognize objects, parse screen content, or provide real-time suggestions.

The new features will be launched first on Android devices and will be supported in multiple languages. Google demonstrated the integration of these features in major Android devices live at MWC, further strengthening its competitiveness in the field of AI assistants.

The update also marks a key step for AI assistants to move towards real-world interactions. Google's long-term goal is Project Astra, a universal multimodal AI assistant that can process text, video, audio in real time and has short-term memory capabilities. Astra is expected to be deeply integrated with tools such as Google Search, Lens and Maps in the future.

With the launch of Gemini Live, competition between Google and OpenAI is becoming increasingly fierce. Since December last year, ChatGPT has supported real-time video and screen sharing in advanced voice mode, and Google's update is undoubtedly a step forward in the face of it. Whether Gemini can further consolidate its AI leadership with the help of this new feature is worthy of continuous attention.