VR Meets AI: How Generative Models and Built-In Assistants Are Rewriting Virtual Reality

For most of its history, virtual reality had a content problem. Headsets got lighter and sharper, but every world, every character and every line of dialogue still had to be built by hand. Over the past year that has started to change. Artificial intelligence is now showing up at three levels of the VR stack at once: as an assistant that sees what you see, as a tool that builds worlds from a sentence, and as the brain behind characters you can actually talk to.

This piece looks at where that convergence stands today, what the evidence says, and what still stands in the way.

1. The AI assistant moves into the operating system

The clearest sign of the shift is Samsung's Galaxy XR, launched in October 2025 as the first headset on Google's Android XR platform. Google describes Gemini as part of the operating system itself: it can understand what is on screen and what the cameras see in the room, then answer questions or act across apps by voice, gaze and gesture. At launch the headset cost $1,799 in the US and Korea, and Google says more than 50 XR experiences from partners such as Adobe, Calm, Fox Sports and MLB were ready on day one.

Meta has taken a similar route on Quest. Meta AI arrived on Quest 3 headsets as an experimental feature, and Meta said its vision-capable version, first seen on Ray-Ban Meta glasses, would come to Horizon OS in passthrough mode and eventually replace the older voice commands. Experimental AI features were initially held back in EU countries, an early reminder that regulation shapes how fast these tools spread.

The point is not that a chatbot now floats in your field of view. It is that the headset is becoming a device that understands context. A flat screen only knows what you type; a mixed reality headset knows where you are looking, what objects are on the table and what you are doing with your hands. That makes it one of the richest inputs an AI model can get.

2. Worlds generated from a prompt

The second front is content creation, and here the numbers are already meaningful. At Meta Connect 2025, Meta said its generative AI creator tools for Horizon Worlds, which produce 3D meshes, textures, skyboxes, sound and TypeScript code, had been used by nearly 60% of eligible creators in the previous 30 days, and that roughly one in three new worlds published in the prior month contained AI-generated assets.

Outside the big platforms, a new class of "world models" is pushing further. World Labs, the startup led by Fei-Fei Li, commercially launched Marble in late 2025. Marble turns text, images or video into editable 3D environments that can be exported as Gaussian splats or standard meshes, and the company says those worlds can be viewed directly in VR on Apple Vision Pro and Meta Quest 3. Marble 1.1, released in April 2026, removed the fixed spatial footprint that limited earlier scenes. Google DeepMind's Genie 3 takes a different approach, generating explorable worlds in real time as you move through them, though access is still restricted to a small group of Google AI Ultra subscribers in the US.

Capture is converging with generation too. Meta's Hyperscape Capture lets a Quest 3 or Quest 3S owner scan a room with the headset's cameras; the scene is processed in the cloud over a few hours into a photorealistic Gaussian splat that streams back to the standalone headset. Up to eight people can now visit a shared capture together. UploadVR called it some of the most realistic VR it had seen, while noting the usual splat artifacts on fine text and in areas the cameras never reached.

3. Characters that talk back

The third shift is less visible but may matter most for engagement. Scripted non-player characters, with their fixed dialogue trees, have long been one of the weakest parts of VR immersion. Meta shared one concrete data point: when the Horizon world Citadel replaced scripted NPCs with AI-generated characters, time spent increased by nearly 15%. Meta has also said Horizon creators will be able to add voice-driven AI NPCs.

The same idea has strong potential in training. A widely cited PwC study of new managers found that VR learners completed soft-skills training up to four times faster than classroom learners and felt up to 275% more confident applying what they learned, with VR reaching cost parity with classroom training at around 375 learners. That study predates modern language models and relied on pre-scripted scenarios. Pairing the same immersive format with AI role-play partners, such as a difficult customer or a nervous patient who responds naturally, is the logical next step, and one many enterprise training vendors are now pursuing.

4. Apple's quieter take: AI as polish

Apple has been more restrained. In visionOS 26 the most striking AI feature is spatial scenes, which Apple says uses a new generative AI algorithm to add lifelike depth to ordinary 2D photos so they can be viewed in three dimensions. Redesigned Personas, the digital avatars used for calls, rely on volumetric rendering and machine learning and are created on device in seconds. Apple Intelligence features such as Image Playground are present, but the company frames AI as something that improves the experience rather than as the headline.

The market context: AI is pulling attention toward glasses

The broader numbers explain why every major player is leaning on AI. According to IDC, global XR device shipments grew 44.4% in 2025, but that growth came mainly from smart glasses, while Meta Quest shipments fell 42.3% year over year. Meta still held 72.2% of the overall market. IDC forecasts a further 33.5% growth in 2026 and expects display-enabled glasses to overtake VR and MR headsets in shipments by 2027.

In other words, the AI assistant is becoming the main reason people buy face-worn devices, and lightweight glasses are winning that race on comfort and price. For VR headsets to keep their place, AI has to make them meaningfully better at what only they can do: fully immersive worlds, rich social presence and hands-on training.

What still stands in the way

• Privacy. An assistant that understands your room needs constant access to cameras and sensors in your home. How that data is processed, stored and used for training will decide how far users and regulators let these features go.
• Compute and latency. Many of the best results, from Hyperscape processing to Genie's real-time worlds, depend on the cloud. Standalone headsets have limited power, and VR is unforgiving of delay.
• Quality and interactivity. Generated worlds still look better than they play. Reviewers note that Marble scenes are navigable but offer limited interactivity, and splat-based captures break down on fine detail.
• Creative and economic questions. If one in three new worlds already includes AI assets, platforms will need clear rules on attribution, quality and how human creators are paid.

The bottom line

AI is not a single feature being added to VR. It is solving the medium's oldest bottlenecks at once: content that is expensive to build, characters that feel lifeless, and interfaces that were never as natural as they should be. The evidence so far, from creator adoption in Horizon to engagement gains from AI NPCs, suggests the combination works. The next two years will show whether it is enough to make headsets a mainstream product, or whether the same AI pulls users toward lighter glasses instead.

Sources

• Google: Introducing Galaxy XR, the first Android XR headset
• Meta: Connect 2025 Day 2 keynote recap
• Meta: Introducing Meta AI on Meta Quest
• UploadVR: Meta Horizon Hyperscape hands-on
• Radiance Fields: World Labs formally launches Marble
• Apple Newsroom: visionOS 26
• IDC: XR market grew 44.4% in 2025
• PwC: VR training study