Artificial intelligence uses less than two minutes of videogame footage to recreate game engine

Their AI system watches less than two

minutes of gameplay video and then builds its own model of how the game operates by studying the frames and making predictions of future events, such as what path a character will choose or how enemies might react.

To get their AI agent to create an accurate predictive model that could account for all the physics of a 2D platform-style game, the team trained the AI on a single "speedrunner" video, where a player heads straight for the goal. This made "the training problem for the AI as difficult as possible."

Their current work uses Super Mario Bros. and they've started replicating the experiments with Mega Man and Sonic the Hedgehog as well. The same team first used AI and Mario Bros. gameplay video to create unique game level designs.

The researchers found that their game engine predicted video frames significantly more similar to those in the original game when compared to the same test on a neural network. This gave them an accurate, general model of a game using only the video footage.