Feng, Jialong (Tongji University), ZHU, Hong (Tongji University), TANG, Keshuang (Tongji University), HAN, Tianyang (Amap), Tang, Zhixian (The Hong Kong Polytechnic University)

Intersection Dynamics-Aware Continuous Learning in Adaptive Traffic Signal Control Featuring Fast Startup and Adaptation

Scheduled for presentation during the Invited Session "Cooperative Driving Technology for Connected Automated Vehicles" (WeAT10), Wednesday, September 25, 2024, 11:30−11:50, Salon 18

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on October 7, 2024

Abstract

Reinforcement learning (RL) methods have been intensively investigated for Adaptive Traffic Signal Control (ATSC) problems. Considering the impracticality of the conventional train-test two-phase process in existing research, this paper adopts a continuous learning approach and addresses two challenges: startup and adaptation. To address these challenges, an Intersection Dynamics-Aware DQN (IDA-DQN) algorithm together with a Backward Intersection Dynamics Model (BIDM) is proposed. IDA-DQN extends the Deep Q-Network (DQN) by incorporating the learning and utilizing of BIDM to reduce reliance on real-world interactions. The proposed BIDM is designed to replenish lane-level vehicle counts using a backward approach, aiding in understanding the dynamics of isolated intersections and in supplementing synthetic samples in addition to real-world samples. Synthetic samples are then used by IDA-DQN to enhance its startup efficiency and adaptability. Through simulation experiments across various demand scenarios, the efficacy of IDA-DQN is validated, particularly demonstrating a fast startup effect with a speed increase of up to 47.53% at moderate traffic levels.