Distributed Knowledge-Enhanced Multi-Agent Reinforcement Learning for Internet of Drones

Abstract

With the growing demand for low-altitude transportation, the application of the Internet of drones (IoD) in urban logistics has become increasingly significant. However, the complex obstacles present in urban environments, such as tall buildings and no-fly zones, pose numerous challenges for the IoD, including low data-driven efficiency and difficulties in representing implicit knowledge. To address these challenges, this paper proposes an IoD swarm collaborative scheduling method based on distributed knowledge-enhanced multi-agent reinforcement learning (DKEMARL). The method leverages prior environmental knowledge to design a knowledge embedding and expansion module, which provides comprehensive and detailed environmental observation data during training and introduces a reward mechanism that balances task timeliness with flight safety. Within a centralized training and decentralized execution framework, the multi-agent deep deterministic policy gradient (MADDPG) approach is employed to facilitate efficient cooperation among the IoD. Specifically, we develop a scenario model for low-altitude transportation tasks involving an IoD swarm, considering factors such as task timeliness, flight distance cost, and safety constraints, and propose an optimization objective to balance timely task completion with flight safety. Simulation results demonstrate that the proposed DKEMARL algorithm can significantly enhance task completion efficiency compared to baseline methods that do not incorporate knowledge enhancement.