Embodied AI explained
Gemini Robotics ER 2 Explained: What Google’s Robot AI Does
Google’s new preview model is designed to be a robot’s high-level reasoning layer, not its muscles or safety system.
Gemini Robotics ER 2 is designed to help a robot understand a scene, plan a task, call tools, watch what happens, and check whether the job is complete. Google released two preview endpoints on July 30, 2026: a turn-based model for embodied reasoning and a streaming version for continuous audio and video context through the Gemini Live API.
The important word is reasoning. Google describes ER 2 as a high-level brain that sits above the robot’s motion system. It can decide what should happen and request actions through tools, but a separate robot API, controller, or vision-language-action model still has to move the hardware.
Want the quick version next time?
Follow @besttechinsight for fast, source-backed explanations of new products, chips, AI, robots, EVs, and science. The detailed guide stays here on BTI.
Source check: July 31, 2026. This guide uses Google’s July 30 Gemini API documentation, release notes, launch article, model card, and robotics safety page. Both ER 2 endpoints are public previews, details can change, and BTI has not tested the models on physical hardware. Google’s benchmark and capability descriptions are provider evidence, not independent BTI validation.
Gemini Robotics ER 2: the fast answer
ER 2 takes multimodal context such as text, images, video, and audio. It reasons about physical space and time, can write or call code and tools, and can coordinate a sequence that involves more than one robot. Google also highlights two capabilities that matter when a plan meets the real world: finding the important moment in a video and classifying how far a task has progressed.
That means the model is meant to do more than issue a command and hope. A robot agent can observe the current state, ask a controller to act, inspect new visual evidence, and decide whether to continue, recover, or stop. It is still a probabilistic generative model, so that loop does not turn every decision into a guaranteed fact.
The five-part robot reasoning loop
| Stage | What Google documents | What it means in plain English |
|---|---|---|
| 1. Observe | Accept text, images, video, and audio describing the robot and its environment. | The model first needs a current view of the workspace, task, and available tools. |
| 2. Plan | Use spatial, temporal, and physical reasoning to decide what should happen next. | It turns the goal and the scene into a sequence of higher-level actions. |
| 3. Orchestrate | Call robot APIs and other tools across multi-step and multi-robot workflows. | The model requests actions from the robot system; it does not directly become the motor controller. |
| 4. Watch | Find relevant moments in video and classify task progress while work is underway. | It can look for the point where a step started, changed, failed, or appeared complete. |
| 5. Verify | Use success detection and progress signals to decide whether to continue, recover, or stop. | It is designed to check the result instead of assuming that a requested action succeeded. |
1. ER 2 is the planner, not the motor controller
A physical robot normally has several layers. Cameras and sensors describe the world. A reasoning system decides what the task requires. A robot API or lower-level controller translates requested actions into motion. Hardware limits, collision checks, emergency stops, and human operating procedures protect the system when software is wrong.
ER 2 lives mainly in the reasoning and orchestration layer. Google’s launch article calls it a high-level brain. The model can break a goal into steps, select tools, generate code, and use visual feedback to revise the plan. It does not replace the real-time control system that determines joint angles, grip force, or certified stopping behavior.
This separation is useful because one model can reason across different robot platforms when each platform exposes a clear tool interface. It is also a boundary developers cannot skip. A sensible plan can still become unsafe if the controller, tool schema, permissions, workspace, or physical safeguard is poorly designed.
2. There are two ER 2 preview endpoints
| Endpoint | Best fit | Boundary to remember |
|---|---|---|
gemini-robotics-er-2-preview |
Turn-based reasoning, video moment finding, progress classification, code execution, and multi-step tool orchestration. | It returns text instructions or tool calls; a separate robot API, controller, or vision-language-action system performs physical motion. |
gemini-robotics-er-2-streaming-preview |
Low-latency robot agents that need continuous audio and video context through the Gemini Live API. | It is a distinct streaming preview endpoint, not a drop-in replacement for every turn-based integration. |
The standard gemini-robotics-er-2-preview endpoint fits workflows where the application sends a task and context, receives reasoning or tool calls, performs an action, and sends back the result. Google lists improved spatial reasoning, video moment finding, progress classification, multi-step tool use, agentic code execution, and multi-robot orchestration among its capabilities.
The gemini-robotics-er-2-streaming-preview endpoint is optimized for continuous interaction through the Live API. A stream of audio and video can keep the model updated while a task unfolds, reducing the delay between seeing a change and adapting the plan. Developers still have to design interruption, timeout, permissions, and physical-stop behavior around that stream.
3. Progress classification answers a practical robot problem
Software tools usually return clean success or error messages. Physical actions often do not. A gripper may miss an object, a drawer may stop halfway, a container may be in the wrong place, or a human may move something after the plan begins. An agent needs evidence about the actual scene rather than only the command it sent.
Google says ER 2 can classify task progress and detect success. In plain English, the model can inspect visual context and estimate whether the job has not started, is underway, is stuck, or appears complete. Video moment finding complements that by locating the point where an important event happened.
These are model judgments, not certified sensors. The safest design uses them as one signal among several. A completion check can combine ER 2’s visual assessment with controller telemetry, force limits, fixed workspace rules, deterministic state checks, and human review for consequential actions.
4. Multi-robot orchestration is about tool coordination
Google demonstrates ER 2 coordinating tasks across different robots. The general idea is not that one model directly drives every motor. Each robot exposes a bounded set of capabilities, and the reasoning layer decides which tool should act, in what order, and with what handoff between steps.
That pattern could support a mobile base carrying an object to a workcell, a robot arm manipulating it, and a camera station checking the result. The model needs current context and reliable tool descriptions, while each robot remains responsible for the lower-level action its interface promises.
Orchestration also increases the number of failure paths. One stale camera frame, ambiguous tool name, overbroad permission, or missed completion signal can affect later steps. Logs, scoped tools, confirmation points, and a clear owner for the physical workspace matter more as the chain gets longer.
5. Preview access is not a production safety certificate
Google makes the models available through the Gemini API and Google AI Studio, and describes a private-preview path for enterprise robotics work. Public preview means developers can evaluate the capability; it does not mean every robot, environment, latency target, or operational policy has been validated.
The model card asks developers to use discretion in production, commercial, and public environments. It excludes safety-critical healthcare, transportation, and other vital contexts from intended use. Google’s broader safety page describes semantic, physical, and operational safeguards, but also says model-level proximity features are not guaranteed safety-rated systems.
Google’s API documentation warns that generative models can make mistakes and that physical robots can cause damage. The practical response is defense in depth: constrained tools, minimum permissions, limited speed and force, restricted work zones, independent collision handling, emergency stops, monitoring, and people who can interrupt the system.
What to check before building with ER 2
- Choose the endpoint by interaction pattern. Use turn-based requests when the workflow has clear steps; evaluate streaming only when continuous audio or video context materially helps.
- Keep tools narrow and explicit. Give the model only the robot actions required for the task, with validated arguments, timeouts, and bounded workspaces.
- Verify physical results independently. Combine model progress judgments with telemetry, deterministic checks, and human confirmation where errors matter.
- Design failure and stop behavior first. Decide what happens on uncertain perception, tool failure, lost connectivity, stale video, or conflicting state before increasing autonomy.
- Minimize personal data. Google’s privacy notice calls for consent and data minimization when identifiable people appear in audio, images, or video.
These are conservative implementation lessons inferred from Google’s published documentation. They are not a claim that BTI audited the model, a robot platform, or a complete deployment.
Frequently asked questions
What is ER 2?
It is Google’s preview embodied-reasoning model for robotics. It accepts multimodal context, plans tasks, calls tools, watches progress, and checks results while leaving physical execution to robot APIs and lower-level controllers.
Can Gemini Robotics ER 2 tell when a robot is done?
Google says ER 2 is designed for progress classification and success detection. That can help an agent judge whether a task appears complete, but it is a model estimate rather than a guaranteed or safety-rated measurement.
What is the difference between the standard and streaming models?
The standard preview fits turn-based reasoning and tool workflows. The streaming preview uses the Gemini Live API for continuous audio and video context, which can support lower-latency adaptation while a robot task unfolds.
Does ER 2 directly control robot motors?
No. Google positions it as a high-level reasoning layer. A robot API, controller, or vision-language-action model still performs the physical action and must enforce hardware and safety limits.
Can ER 2 be treated as a finished production system?
No. Google labels both endpoints as previews. The model card advises caution in production, commercial, and public settings and excludes safety-critical healthcare, transportation, and other vital contexts from intended use.
What happens to Gemini Robotics ER 1.6?
Google’s current API documentation says the ER 1.6 preview endpoint is scheduled to shut down at the end of August 2026. Developers should verify the latest migration guidance before changing an integration.
Editorial and affiliate disclosure
This guide contains no affiliate links and makes no hands-on testing, private-preview access, robot-performance, safety-certification, price, rating, review, availability, investment, or endorsement claim. The hero is a BTI-original conceptual illustration, not a Google screenshot, product photograph, benchmark artifact, or deployment record.
Sources and evidence boundaries
Capability, access, and safety statements come from Google and Google DeepMind. They describe the provider’s preview release and evaluations. BTI does not treat those statements as independent proof for every robot, environment, or use case.
- Gemini API: Gemini Robotics overview: Google documents the two ER 2 preview endpoints, supported inputs, tool workflow, progress capabilities, privacy notice, and ER 1.6 shutdown.
- Gemini API: July 30 release notes: The changelog records the public-preview launch and names the standard and streaming endpoints.
- Google DeepMind: Gemini Robotics ER 2 announcement: Google describes ER 2 as a high-level reasoning layer that watches progress, adapts plans, and hands execution to lower-level robot systems.
- Google DeepMind: Gemini Robotics ER 2 model card: The model card states intended uses, evaluation scope, production cautions, and safety-critical exclusions.
- Google DeepMind: robotics safety approach: Google explains its layered semantic, physical, and operational safeguards while warning that model-level features are not guaranteed safety-rated systems.
The BTI takeaway
ER 2 matters because it closes more of the loop between a plan and the messy physical result. The model can observe, reason, request actions, watch progress, and check completion. The real engineering work is still in the boundaries around it: narrow tools, dependable controllers, independent safety systems, and a clear way to stop when the model is uncertain or wrong.