Google’s Genie 4 Adds Sound and Dialogue to Interactive Worlds

Google DeepMind announced Genie 4 on October 27 (opens in new tab), adding native sound and interactive dialogue to its real-time world simulations. Users can navigate a generated scene, change it with text prompts, and hear the environment and its characters respond.

The model is already rolling out to trusted testers. Google AI Ultra subscribers in Project Genie and early-access partners using the Gemini API and Google AI Studio are next, but DeepMind describes those releases as coming soon (opens in new tab). A waitlist is open; this is not a generally available developer service.

For agent builders, the more interesting change is behind the scenery: Gemini models help plan events and monitor the world as Genie renders it. Google is experimenting with that combination for robotics scenarios, though it has not established that visually convincing simulations are reliable substitutes for physical testing.

The world can now talk back

According to DeepMind's announcement (opens in new tab), Genie 4 generates audio simultaneously with the world. That includes ambient noise and music, plus dialogue that changes with user input and events in the scene. Google says the accompanying sounds in its launch examples come directly from Genie 4.

Users navigate and control the camera with standard keyboard controls. Text prompts can change the weather, introduce a person or object, or alter the camera while the simulation continues. Gemini adapts the request to the surrounding scene and helps steer what happens between inputs.

DeepMind also claims better physical realism, including aerodynamics and collisions, and more natural human movement. Those claims matter for interactive worlds: an object has to respond plausibly to an intervention, not merely look convincing in a single frame.

The launch examples demonstrate the intended range. They include wind moving fabric across a meadow, a falling maple seed, a busy train station, and a musician playing on a platform in a lake. Google describes the demonstrations as recordings of real-time interactions, with sequences shortened for illustration.

Gemini plans; Genie renders

DeepMind assigns different roles to the Gemini family. Gemini Omni helps anchor the simulations in real-world appearance and behavior. In the agent loop described in the launch post, Gemini 3.8 Flash observes the scene, divides goals into steps and reviews incoming frames to check progress. It revises the plan when the user intervenes or the scene changes.

Google has also experimented with Gemini 4 Argon and Genie 4 alongside Gemini Robotics (opens in new tab). In one example, Argon breaks a checkers task involving Apptronik's Apollo humanoid into steps that Genie simulates. The steps are supplied as text captions while the world is being generated.

DeepMind calls closer integration between structured reasoning and pixel-level generation a path toward hybrid world models. The practical aim is to make events follow a plan without requiring every visual detail to be specified in advance.

A few minutes is not a persistent world

The Genie 4 product page (opens in new tab) puts a useful boundary on the demonstrations: the model supports “a few minutes of continuous interaction,” rather than extended sessions.

DeepMind lists several other unresolved problems:

• Visual details and object permanence can drift, particularly during longer runs, after complex prompt edits or while navigating through the scene.
• Interactions involving users, multiple independent agents and shared environments remain difficult to model accurately.
• Real locations cannot be reproduced with perfect accuracy.
• Legible text is often generated only when it appears in the input world description.
• Real-time audio accuracy remains an area of research.

The launch post also identifies complex physical interactions, spatial accuracy and long-term consistency as unfinished work. Better-looking collisions do not remove those limits.

Neither the launch announcement nor the product page publishes comparative benchmark scores for Genie 4.

What testers are trying

DeepMind says filmmakers are using the preview as a rapid draft for scenes, changing camera angles, lighting, dialogue and action during exploration. Game developers are testing explorable concepts before building them in an engine. Other testers are investigating historical reconstructions and rapid scenario generation for engineering.

TechCrunch's launch reporting (opens in new tab) describes film pre-visualization as one of the emerging uses and reports that the team still considers the end-to-end product experience unsettled. That fits the current access model: experiments with selected partners before a broader release.

Access and safeguards

DeepMind has opened a Project Genie waitlist (opens in new tab). The announced next groups are Google AI Ultra subscribers and early-access Gemini API and AI Studio partners. The launch materials do not specify a public Genie 4 API price, fixed usage allowance or wider-release date.

Google says it conducted internal safety evaluations before the preview and applied content guardrails. It also says all generated video and audio will carry SynthID watermarks.

Genie 4 combines native audio, live scene editing and Gemini-guided planning, but remains a short-session research preview. Wider access through Project Genie, the Gemini API and Google AI Studio is still coming soon.