Google's AI Brain for Robots: Gemini Robotics ER 2
Real-time spatial reasoning and multi-robot collaboration for smarter physical world agents.
1What is Gemini Robotics ER 2?
Google's new AI model Gemini Robotics ER 2 serves as a high-level brain for robots. It enables real-time spatial reasoning, multi-step task planning, and collaboration between different robots. Developers can start using the model through Gemini API, Google AI Studio, or Gemini Enterprise Agent Platform. This technology aims to make robots safer and more helpful in the physical world by understanding their environment and adapting to changes instantly.
2Video Understanding and Self-Correction
Gemini Robotics ER 2 allows robots to track their own progress by watching continuous video feeds. It can adapt when things go wrong and know exactly when to move to the next step. The model achieves 57.4% accuracy in progress classification tasks and 91.3% accuracy in identifying critical moments in video frames. These capabilities ensure robots complete complex tasks according to specifications before proceeding to next actions.
3Multi-Robot Collaboration
The new model enables different robots to work together in shared spaces, completing complex workflows that a single robot couldn't handle alone. For example, it can integrate with Boston Dynamics' Spot robot to perform interactive tasks like fetching objects. This collaboration capability allows diverse machines to communicate and leverage each other's specialized skills. Code examples are available on GitHub for developers.
4Real-Time Performance
Gemini Robotics ER 2 uses a bidirectional streaming endpoint optimized for latency-sensitive tasks. This allows the model to manage action models and robotics APIs without jarring 'stop-and-think' pauses during multi-step tasks. ER 2 consistently outperforms ER 1.6 across three control modes (real VLA, sim VLA, and human tele-op) while delivering 4x faster execution speed - crucial for safe physical world operations.
5Advanced Task Monitoring
The model addresses one of robotics' hardest challenges: knowing when a task is complete. It can verify complex tasks (like tightening a light bulb or tying a trash bag) meet specifications before moving to the next task. Progress is classified into five levels (0-20%, 20-40%, 40-60%, 60-80%, 80-100%), providing robots with real-time situational awareness and the ability to adjust actions or retry failed steps without restarting entire workflows.
6Tool Integration
Developers can declare low-level control interfaces (like VLA models or navigation APIs) as tools and stream multimodal video, audio, or text directly into the model. Gemini Robotics ER 2 enhances this tool orchestration workflow. Its performance can be evaluated with simulated robots, real-world robot control, or even paired with a human remotely controlling the robot, demonstrating versatile application potential.
7Boston Dynamics Integration
Google developed a demo with partners at Boston Dynamics using their Spot robot. Gemini Robotics ER 2 orchestrates Spot APIs (including navigation and manipulator movement) to create an interactive robot that fetches objects on natural language commands. This integration showcases ER 2's potential in real-world applications, from simple tasks like retrieving popcorn to more complex industrial operations.
8A New Era in Robotics
Gemini Robotics ER 2 represents a significant leap in robots' ability to assist humans in the physical world. With real-time decision making, multi-robot collaboration, and self-correction capabilities, it has potential to revolutionize applications ranging from industrial manufacturing to everyday life. By making this technology available to developers, Google could accelerate the evolution of physical AI agents across multiple industries.
Original article: DeepMind — Google'dan Robotlar için Yapay Zeka Beyni: Gemini Robotics ER 2