World model

Predict what may happen next#

Ha and Schmidhuber's World Models learns a compressed representation of observations and a recurrent model of how that representation evolves. A controller can use features from the model, and the authors also train an agent inside a generated environment.

For a robot, a corresponding use would be predicting how an object might move after a push. The prediction could concern a compact state representation instead of a photorealistic future image.

Prediction and control remain different roles#

A world model predicts outcomes; a policy chooses actions. One system can contain both, but a predictive video model does not automatically provide a robot controller.

The cited work evaluates simulated reinforcement-learning environments. Its generated-environment training demonstrates the method in that setting, rather than establishing that any visually plausible model accurately predicts physical robot contact.

Controllers can exploit model errors#

The authors describe a controller finding behavior that exploits mistakes in its learned environment. Those opportunities did not behave the same way in the original environment.

This makes model fidelity relevant to the task and to the actions considered. A model that reconstructs familiar observations well can still make poor predictions for unfamiliar actions. System identification and world-model learning overlap where both estimate dynamics from data, although world-model architectures need not use explicit mechanical equations.