Google I/O 2024 witnessed the unveiling of a significant upgrade to their AI-powered chatbot, Gemini.
This new iteration, called Gemini Live, promises a more natural and interactive experience for users through real-time voice chat and multimodal capabilities.
Gemini Live departs from traditional chatbots by incorporating real-time voice interaction. Users can converse with Gemini on their smartphones, ask clarifying questions mid-sentence, and experience a dynamic dialogue that adapts to their speech patterns. Furthermore, Gemini Live leverages smartphone cameras, allowing it to “see” the user’s surroundings and respond accordingly.
While seemingly familiar at first glance, Google emphasizes substantial advancements in core technologies. Gemini Live harnesses the power of Google Lens’ image analysis and Google Assistant’s speech recognition to deliver superior results. New generative AI techniques contribute to more accurate image analysis and a more realistic, error-free dialogue experience.
The technical innovations behind Gemini Live are rooted in Project Astra, a DeepMind initiative focused on creating AI-powered applications with real-time multimodal understanding. Oriol Vinyals, a DeepMind principal scientist, highlights the project’s potential: “Imagine how that combination [real-time voice and multimodal capabilities] will feel very powerful.”
DeepMind CEO Demis Hassabis envisions AI agents that seamlessly integrate into our lives, understanding our environment and responding in a natural conversational flow. Gemini Live exemplifies this vision, functioning as a virtual assistant for everyday tasks. Imagine receiving interview coaching, brainstorming ideas, or getting help planning a trip – all through Gemini Live.
Gemini Live’s ability to “remember” is facilitated by the underlying model, Gemini 1.5 Pro (the flagship in Google’s generative AI family). This model boasts a longer-than-average context window, enabling it to process and reason over a significant amount of data – roughly an hour of video – before crafting a response. Vinyals emphasizes, “That’s hours of video that you could have interacting with the model, and it would remember all that has happened before.”
Comparisons are inevitable, with Gemini Live resembling Meta’s Ray-Ban AI glasses and OpenAI’s revamped ChatGPT. However, a key differentiator is cost. Gemini Live will be exclusive to Gemini Advanced, a premium tier requiring a $20 per month Google One AI Premium Plan subscription. Interestingly, Google showcased a concept for AR glasses featuring Gemini Live-like functionality but remained tight-lipped on their commercial prospects.
The improvements extend beyond Live. Gemini Advanced users in over 150 countries and 35 languages can now leverage Gemini 1.5 Pro’s capabilities to analyze, summarize, and answer questions about lengthy documents (up to 1,500 pages). Document imports can be done from Google Drive or directly uploaded from mobile devices.
Later this year, Gemini Advanced will benefit from an even larger context window (2 million tokens) and the ability to handle video uploads (up to two hours) and complex codebases (over 30,000 lines). This expanded context will enhance Gemini’s image understanding and problem-solving abilities. Additionally, a “planning experience” is on the horizon, generating custom travel itineraries based on user preferences and data from Gmail, Google Search, and Maps.
Taking a cue from OpenAI’s GPTs, Google is introducing Gems – custom chatbots generated from natural language descriptions. Users can create private Gems for personal use or share them with others. Whether Google plans a marketplace similar to OpenAI’s GPT Store remains to be seen.
Gemini and Gems will soon integrate with various Google services, including Calendar, Tasks, Keep, and YouTube Music, to automate routine tasks. Imagine using Gemini to effortlessly add school event details from a flyer directly to your calendar.
While generative AI has a history of inaccuracies and shortcomings, Google’s vision for Gemini and Gemini Advanced is undoubtedly intriguing. If these advancements deliver on their promises, they could significantly enhance user productivity and streamline daily tasks. However, it’s important to maintain a cautious approach until real-world performance can be evaluated.
