UK Startup Worldmodeldata Uses Video Game Data to Train AI World Models

Worldmodeldata, a UK-based startup advised by AI researcher Yann LeCun, is pioneering a new approach to training world models for AI. By licensing nearly 1 million hours of data from video game studios, the company aims to address the shortage of action-oriented training material. However, experts are divided on whether video game data can effectively replicate real-world physics for tasks like robotics.

Startup Turns to Video Game Data for AI Training

Worldmodeldata, a UK-based startup, is addressing a major challenge in AI development: the shortage of training data for world models. According to WIRED, the company is curating datasets from video game studios, packaging controller inputs and other in-game data to help AI systems understand real-world physics and actions.

The startup, advised by Meta’s chief AI scientist Yann LeCun, has licensed nearly 1 million hours of data from video game studios. This approach aims to fill a critical gap, as world models require both visual and action data—unlike large language models (LLMs), which are trained solely on text.

Why World Models Need Different Data

The limitations of LLMs in navigating physical environments have driven researchers like Fei-Fei Li and Yann LeCun to focus on world models. These models are designed to simulate real-world interactions, but their development has been constrained by the lack of high-quality datasets. According to WIRED, there is currently no existing corpus of material for training world models comparable to the vast text datasets used for LLMs.

Worldmodeldata’s strategy leverages the structured data generated by video games, where player actions, environmental interactions, and physics are already recorded. This data could help AI systems learn cause-and-effect relationships in dynamic environments.

Skepticism and Potential Limitations

While the approach is innovative, some experts question its effectiveness for certain applications. Nvidia’s lead for world model development, Ming-Yu Liu, told WIRED that the company prefers using a custom physics engine tailored to replicate real-world conditions. This raises doubts about whether video game data—where physics are often simplified for gameplay—can train models for tasks requiring fine motor control, such as manipulating objects.

Associate Professor Xiatian Zhu of the University of Surrey noted that video game physics are frequently eccentric, with developers prioritizing the illusion of realism over accuracy. As a result, models trained on such data may struggle with precision tasks but could excel in generating hyperrealistic video or 3D environments.

Background: The Push for World Models

World models represent the next frontier in AI research, aiming to overcome the limitations of LLMs by enabling machines to understand and interact with physical environments. Unlike LLMs, which rely on static text data, world models require dynamic datasets that capture actions, physics, and real-time decision-making.

The lack of such datasets has been a significant hurdle, prompting researchers and startups to explore unconventional sources. Video games, with their rich interaction data, offer a promising but unproven alternative.

What this means

Lazyfounder analysis — our interpretation, not reported fact.

Worldmodeldata’s approach highlights a critical gap in AI development: the lack of high-quality, action-oriented training data for world models. While video game data offers a novel solution, its limitations—particularly in replicating real-world physics—could restrict its usefulness to specific applications like simulation or content generation. For founders and operators in AI, this underscores the importance of aligning data sources with the intended use case. If the goal is robotic control or precision tasks, synthetic or custom-engineered data may still be necessary. However, for applications like virtual environments or AI-driven storytelling, video game data could prove invaluable.

Key takeaways

• Worldmodeldata is licensing video game data to train world models, a departure from traditional text-based training for LLMs.
• The startup has secured nearly 1 million hours of data from video game studios, addressing a key bottleneck in world model development.
• World models require both visual and action data to understand real-world physics, unlike LLMs.
• Nvidia and other experts caution that video game data may not be suitable for tasks requiring fine motor control.
• Video game data could be more effective for generating hyperrealistic video or 3D environments than for robotics.

FAQ

What are world models in AI?

World models are AI systems designed to simulate and understand real-world physics and interactions. Unlike large language models (LLMs), which are trained on text, world models require visual and action data to learn cause-and-effect relationships in dynamic environments.

Why is video game data being used to train world models?

Video games generate vast amounts of structured data, including player actions, environmental interactions, and physics simulations. Startups like Worldmodeldata are licensing this data to fill the gap in training material for world models, as there is currently no comparable corpus of real-world action data.

What are the limitations of using video game data for AI training?

Video game physics are often simplified or eccentric, prioritizing the illusion of realism over accuracy. Experts warn that models trained on such data may struggle with precision tasks like robotic control but could be effective for generating hyperrealistic video or 3D environments.

Who is advising Worldmodeldata?

Worldmodeldata is advised by Yann LeCun, Meta’s chief AI scientist and a prominent researcher in the field of artificial intelligence.

Related on Lazyfounder

• Nvidia
• More technology news
• ASUS Launches ExpertCenter Pro ET900N G3: A Desktop AI Powerhouse