Enhancing Autonomous Driving with Spatial Memory and Attention in Reinforcement Learning

Abstract

Reinforcement learning in environments with visual observations presents challenges due to incomplete individual observations. The lack of complete information leads to increased uncertainty in decision-making, which requires agents to be supplemented with a memory module to retain information about previous observations. Our paper proposes a novel spatial memory mechanism with a flexible access system based on the multihead attention mechanism. Through experiments in the Atari benchmark and multiple autonomous driving environments, our approach outperforms agents using classical convolutional and recurrent neural networks. Further analysis reveals repeated interpretive patterns in attention distribution among trained agents. This study highlights the effectiveness of spatial memory and attention mechanisms in improving the efficiency of deep reinforcement learning in partially observable environments.