Doroudian, Erfan (Concordia University), Taghavifar, Hamid (Concordia University)

Balancing Individual and Collective Interests: Integrating Transformer-Based PPO and Social Intelligence for Autonomous Vehicles at Unsignalized Intersections

Scheduled for presentation during the Regular Session "Autonomous vehicles - intersection management" (FrAT8), Friday, September 27, 2024, 11:50−12:10, Salon 16

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on October 8, 2024

Abstract

Autonomous vehicles (AVs) promise a transformative shift in transportation paradigms, yet their integration into mixed-traffic environments poses complex challenges. In this paper, we propose a novel framework that combines Proximal Policy Optimization (PPO) with the Self-attention mechanism from transformers and Social Value Orientation (SVO) to address the delicate balance between individual and collective interests at unsignalized intersections. By integrating SVO, a psychological concept characterizing individuals’ preferences for cooperation and competition, into PPO with Self-attention mechanism, our framework enables AVs to make ethically informed decisions that consider both individual and collective welfare. Transformers, known for their ability to capture long-range dependencies and model complex relationships within data, enhance the reinforcement learning capabilities by providing more effective and context-aware decision-making. We present a detailed formulation of the PPO-Transformer-SVO framework, discussing how SVO influences AV behavior and cooperation strategies at intersections. Through simulation experiments, we demonstrate the effectiveness of our approach in optimizing traffic flow, minimizing collisions, and ensuring collective and individual benefits among road users. Our results highlight the potential of integrating human-inspired social preferences and advanced reinforcement learning techniques into AV decision-making algorithms to enhance the safety, efficiency, and ethicality of future transportation systems.