QI, JIAJU (University of Guelph), Lei, Lei (University of Guelph), Jonsson, Thorsteinn (EthicalAI)

Hierarchical Deep Reinforcement Learning for Charging Scheduling of Electric Buses with Uncertainties

Scheduled for presentation during the Regular Session "Electric Vehicles - Charging and Scheduling I" (WeAT3), Wednesday, September 25, 2024, 11:30−11:50, Salon 6

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on December 26, 2024

Abstract

The increased adoption of Electric Buses (EBs) marks a significant step towards a more environmentally sustainable transportation system. A critical concern for bus companies is reducing the operational costs associated with charging these vehicles. This task is particularly challenging due to the uncertainties in travel time, energy consumption, and fluctuating electricity prices, compounded by the constraints of limited charging infrastructure. Deep Reinforcement Learning (DRL) techniques overcome these challenges by learning an optimal decision process from data but are difficult to design for an efficient and stable learning process. In this paper, we conceive two augmented Markov Decision Processes (MDPs) and propose a novel Hierarchical DRL (HDRL) algorithm called Double Actor-Critic Multi-Agent Proximal Policy Optimization (DAC-MAPPO). The proposed algorithm enhances learning efficiency and convergence speed by integrating the MAPPO algorithm into the DAC architecture. Specifically, a centralized high-level agent is responsible for making charger allocation decisions, while multiple decentralized low-level agents determine the charging power for each EB at every time step. Experimental evaluations using real-world data demonstrate the superior performance and effectiveness of the DAC-MAPPO algorithm.