Facilitating Multiagent Coordination Relying on Graph Information Representation

Abstract

The popular multiagent reinforcement learning (MARL) methods primarily focus on exploring the capability of value functions to facilitate multiagent coordination. These MARL methods, following the centralized training with decentralized execution (CTDE) paradigm, tend to design ingenious network architectures while overlooking the impact of coordination through expanding local observation information. To tackle this deficiency, we model the multiagent systems (MASs) as a graph and use a graph neural network (GNN) to extract rich information between one agent and the others efficiently. Moreover, we propose a multigraph-neural-network information representation (MGIR) method that uses the power of GNN in local observation information extraction, enabling the acquisition of higher quality information. Specifically, multiple GNNs are used during centralized training to characterize the MAS from different perspectives and extract representations of latent variables. During distributed execution, these latent variables are leveraged to expand local observation information. Extensive comparative experiments substantiate that our proposed MGIR demonstrates superior coordination performance when compared with baseline methods. In addition, it can be flexibly integrated into various value function decomposition methods of MARL.