-
Weak La Niña and dry conditions likely in the Southwest this winter - 7 mins ago
-
Donald Trump Urges Women To Get ‘Fat Pig’ Husbands To Vote Early - 9 mins ago
-
How a 102-year-old woman is defying the odds as a musician, volunteer and more - 12 mins ago
-
iQOO 13 Design Revealed in Leaked Live Images; Could Feature Narrow Bezels, Flat Edges - 15 mins ago
-
NFL Week 7 picks, schedule, odds, injuries, fantasy tips - 18 mins ago
-
Bayer Leverkusen vs Frankfurt Prediction: Bundesliga - 19 mins ago
-
Edwardian doctor in Thetford was UK’s first black mayor - 22 mins ago
-
Rohit Sharma Receives Plea To Join RCB In IPL 2025 During India vs NZ Test In Bengaluru. Ex-MI Captain Did This - 37 mins ago
-
Manchester United vs Brentford Prediction: Ten Hag desperate for win - 38 mins ago
-
Pop Idol Mark Rhodes’ advice to Liam Payne on coping with fame - 40 mins ago
Google DeepMind Is Integrating Gemini 1.5 Pro in Robots That Can Navigate Real-World Environments
Google DeepMind shared new advancements made in the field of robotics and vision language models (VLMs) on Thursday. The artificial intelligence (AI) research division of the tech giant has been working with advanced vision models to develop new capabilities in robots. In a new study, DeepMind highlighted that using Gemini 1.5 Pro and its long context window has now enabled the division to make breakthroughs in navigation and real-world understanding of its robots. Earlier this year, Nvidia also unveiled new AI technology that powers advanced capabilities in humanoid robots.
Google DeepMind Uses Gemini AI to Improve Robots
In a post on X (formerly known as Twitter), Google DeepMind revealed that it has been training its robots using Gemini 1.5 Pro’s 2 million token context window. Context windows can be understood as the window of knowledge visible to an AI model, using which it processes tangential information around the queried topic.
For instance, if a user asks an AI model about “most popular ice cream flavours”, the AI model will check the keyword ice cream and flavours to find information to that question. If this information window is too small, then the AI will only be able to respond with the names of different ice cream flavours. However, if it is larger, the AI will also be able to see the number of articles about each ice cream flavour to find which has been mentioned the most and deduce the “popularity factor”.
DeepMind is taking advantage of this long context window to train its robots in real-world environments. The division aims to see if the robot can remember the details of an environment and assist users when asked about the environment with contextual or vague terms. In a video shared on Instagram, the AI division showcased that a robot was able to guide a user to a whiteboard when he asked it for a place where he could draw.
“Powered with 1.5 Pro’s 1 million token context length, our robots can use human instructions, video tours, and common sense reasoning to successfully find their way around a space,” Google DeepMind stated in a post.
In a study published on arXiv (a non-peer-reviewed online journal), DeepMind explained the technology behind the breakthrough. In addition to Gemini, it is also using its own Robotic Transformer 2 (RT-2) model. It is a vision-language-action (VLA) model that learns from both web and robotics data. It utilises computer vision to process real-world environments and use that information to create datasets. This dataset can later be processed by the generative AI to break down contextual commands and produce desired outcomes.
At present, Google DeepMind is using this architecture to train its robots on a broad category known as Multimodal Instruction Navigation (MIN) which includes environment exploration and instruction-guided navigation. If the demonstration shared by the division is legitimate, this technology might further advance robotics.
Source link