Kang, Letian (Beihang University), Xing, Jiandong (Beihang University), Cui, Zhiyong (Beihang University), Lan, Zhengxing (beihang university), Wang, Zihe (Beihang University), Ren, Yilong (Beihang University), Yu, Haiyang (Beihang University)

Worst-Case Ready: A Policy-Constrained RL Framework for Robust Autonomous Driving

Scheduled for presentation during the Regular Session "S26b-Motion Planning, Trajectory Optimization, and Control for Autonomous Vehicles" (TH-EA-T26), Thursday, November 20, 2025, 14:50−14:50, Broadbeach 1&2

2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC), November 18-21, 2025, Gold Coast, Australia

This information is tentative and subject to change. Compiled on October 18, 2025

Abstract

Reinforcement learning (RL) shows great potential in autonomous vehicle decision-making. However, real-world disturbances like weather and unknown factors can distort environmental perception, leading to inaccurate decisions and compromised safety. Conventional safety barriers or uncertainty prediction methods often fall short in ensuring driving safety under such perturbations. To address this issue, this paper proposes a method named Worst-Case Aware Soft Actor-Critic (WCA-SAC) for autonomous vehicles' robust decision-making. By formulating a constrained Markov Decision Process and confining the difference in policy distributions before and after perturbations, the vehicle can gain the expectation of environmental variations. A worst-case perturbation generation method is introduced to simulate extreme perturbations within a certain range, enabling the vehicle to recognize the worst-case scenario. The Lagrangian Multiplier method is employed to iteratively optimize the policy under the given constraints. The proposed WCA-SAC facilitates a balanced exploration of the policy space, while simultaneously ensuring the policy’s robustness against environmental perturbations. Extensive experiments under various perturbation intensities and traffic flow conditions demonstrate that our approach outperforms other baseline algorithms, ensuring safe and robust autonomous driving decision-making.