Microsoft has partnered with Teachable AI Experiences and Ninja Theory to launch Muse, an AI model designed for game developers. Muse can generate game visuals and controller actions using its 3D understanding, trained on 1.6 billion parameters and one billion gameplay images and actions. It has been tested with Ninja Theory’s Bleeding Edge. The model can create sequences up to 2 minutes long, adapting to game physics, and is already used to develop real-time playable AI for other titles. Additionally, it might help modernize retro games. Muse’s resources are available on Azure AI Foundry, with interactive demos set to launch via Copilot Labs.
Microsoft has collaborated with Teachable AI Experiences and the Ninja Theory from Xbox Games Studios to unveil a new AI model designed to assist game developers. The model is named Muse.
What capabilities does Muse possess? It can create game visuals and simulate controller actions through a 3D understanding of a game environment. Trained on 1.6 billion parameters, it utilized 1 billion images and controller actions, equivalent to approximately 7 years of uninterrupted human gameplay, using Ninja Theory’s Bleeding Edge as a reference.
The dataset for the model was gathered from gameplay data provided by consenting users. Muse operates with visuals and captured controller actions at a resolution of 300 x 180px. Consequently, it can generate sequences lasting up to 2 minutes and forecast game development in world model mode. Demonstrations reveal the model’s proficiency in executing camera shifts, making path selections, and even creating new characters, all while adhering to the game’s physics.
So, what role does Muse play in contemporary game development? According to Microsoft, Muse is already being employed to create real-time playable AI models based on other games. It assists developers in experimenting with new ideas and potentially enhancing existing titles.
One intriguing application is the simplified adaptation of retro games that originally ran on older hardware. The AI model can be utilized to adapt classic games for contemporary systems.
The model weights, sample data, and WHAM (World and Human Action Model) are all accessible on Azure AI Foundry. Additionally, interactive AI game demos will be made available through Copilot Labs later this year.
You can explore some of the demos at the source link below.
Source