Kou, Yuran (Shenzhen Institutes of Advanced Technology Chinese Academy of Sc), Yin, Jianwen (University of Chinese Academy of Sciences), Peng, Lei (Shenzhen Institutes of Advanced Technology,Chinese Academy of Sc), Sun, Tianfu (Shenzhen Institutes of Advanced Technology, Chinese Academy of S), Liu, Jia (Shenzhen Institute of Advanced Technology Chinese Academy of Sci), Li, Huiyun (Shenzhen University of Advanced Technology)

Uncertainty-Aware Reinforcement Learning for Autonomous Driving in High-Dynamic Traffic Scenarios

Scheduled for presentation during the Regular Session "S12c-Safety and Risk Assessment for Autonomous Driving Systems" (WE-LA-T12), Wednesday, November 19, 2025, 16:20−16:40, Broadbeach 3

2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC), November 18-21, 2025, Gold Coast, Australia

This information is tentative and subject to change. Compiled on October 19, 2025

Abstract

Safe decision-making for autonomous driving in high-dynamic environments relies heavily on estimating multi-source uncertainty, which can help mitigate associated risks. In this paper, we quantify both aleatoric and epistemic uncertainty in driving decisions and develop a risk-sensitive driving policy. We propose Uncertainty-Aware Reinforcement Learning (UA-RL), a novel framework that integrates Ensemble Quantile Networks (EQN) into the CrossQ architecture to jointly model these two types of uncertainty. Specifically, UA-RL incorporates the quantile regression into the Actor-Critic structure to estimate return distributions for aleatoric uncertainty, while employing an ensemble of Q-networks to capture epistemic uncertainty. Additionally, we introduce a unified thresholding mechanism to monitor both uncertainties and trigger a conservative candidate policy when uncertainty exceeds safe constraints. Experimental results on the MetaDrive simulation platform—under high-dynamic and highly interactive mixed traffic scenarios—demonstrate that UA-RL significantly reduces collision rates compared to baseline methods, highlighting its safety and effectiveness in driving decision-making.