Cai, Yuxin (Nanyang Technological University), Liu, Zhengxuan (Tianjin University), He, Xiangkun (Nanyang Technological University), Zuo, Zhiqiang (Tianjin University), Lv, Chen (Nanyang Technological University)

Interaction-Aware Hierarchical Representation of Multi-Vehicle Reinforcement Learning for Cooperative Control in Dense Mixed Traffic

Scheduled for presentation during the Invited Session "Enhancing Trustworthiness and Resilience of Connected and Autonomous Vehicles in Adversarial Environments" (FrAT7), Friday, September 27, 2024, 11:30−11:50, Salon 15

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on October 3, 2024

Abstract

Improving traffic performance in dense mixed traffic scenarios such as bottlenecks, presents significant chal-lenges due to complex interactions and the unpredictable behaviors of human drivers. These challenges are compounded by varying human driving styles and different proportions of Connected and Automated Vehicles (CAVs) within the traffic flow. Our research focuses on developing cooperative control strategies for CAVs to enhance generalization across diverse traffic scenarios. To address these challenges, we introduce an Interaction-Aware Hierarchical Representation (IAHR) module, integrated into Multi-Agent Reinforcement Learning (MARL) framework. The IAHR module hierarchically processes inter-actions between CAVs and Human-Driven Vehicles (HDVs), effectively extracting essential features to facilitate generaliza-tion across various traffic scenarios. Additionally, we design an effective reward function that balances individual interests with overall traffic performance, guiding CAVs to improve their driving efficiency and safety while also enhancing overall traffic flow. The model is rigorously trained and zero-shot evaluated in various bottleneck scenarios. Results demonstrate the model’s capability to significantly improve traffic performance under dense conditions and generalize across different CAV penetration rates, vehicle numbers, and HDV driving style distributions.