Zhang, Zhiyao (Vanderbilt University), Gunter, George (University of Illinois), Quinones Grueiro, Marcos (Vanderbilt University), Zhang, Yuhang (Vanderbilt University), Barbour, William (Vanderbilt University), Biswas, Gautam (Vanderbilt University), Work, Daniel (Vanderbilt University)

Phase Re-Service in Reinforcement Learning Traffic Signal Control

Scheduled for presentation during the Regular Session "Traffic signal control" (WeAT9), Wednesday, September 25, 2024, 12:10−12:30, Salon 17

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on December 26, 2024

Abstract

This article proposes a novel approach to traffic signal control that combines phase re-service with reinforcement learning (RL). The RL agent directly determines the duration of the next phase in a pre-defined sequence. Before the RL agent's decision is executed, we use the shock wave theory to estimate queue expansion at the designated movement allowed for re-service and decide if phase re-service is necessary. If necessary, a temporary phase re-service is inserted before the next regular phase. We formulate the RL problem as a semi-Markov decision process (SMDP) and solve it with proximal policy optimization (PPO). We conducted a series of experiments that showed significant improvements thanks to the introduction of phase re-service. Vehicle delays are reduced by up to 29.95% of the average and up to 59.21% of the standard deviation. The number of stops is reduced by 26.05% on average with 45.77% less standard deviation.