Temporal Predictive Coding as World Model for Reinforcement Learning

Аннотация

Partially observable environments pose a fundamental challenge for reinforcement learning (RL), requiring agents to infer hidden states from incomplete sensory input. We propose incorporating Temporal Predictive Coding (TPC) as a world model within RL agents to address this problem. By continuously predicting future observations, TPC builds robust latent representations that capture essential state information and temporal dependencies. We evaluate this approach in grid-world environments with varying levels of perceptual ambiguity. Across multiple tasks, TPC-augmented agents consistently outperform or match strong baselines, including LSTM, RWKV, Clone-Structured Cognitive Graphs (CSCG), and episodic control agents. Analysis of the learned representations shows that TPC effectively disentangles underlying state structure, resolving perceptual aliasing and supporting generalization across time …