Keywords: Reinforcement Learning for Planning, Deep Learning Based Approaches, Human Factors Analysis in Vehicle Design
Abstract: Existing autonomous driving systems are optimized for egocentric efficiency metrics, which are in fundamental conflict with the socialized expectations and habitual patterns of human drivers. This contradiction stems from the traditional approach's dual neglect of the trade-offs between self and other in driving decisions, and the culturally rooted social qualities of traffic interactions. To this end, this paper proposes a dual-adaptation framework that integrates social value orientation (SVO) and human-in-the-loop(HITL) guidance, modeling vehicular interactions as competitive-cooperative agents by quantifying the social utility function, and dynamically calibrating the SVO parameters with the help of real-time human feedback. The method innovatively transforms abstract social preferences into mathematically tractable decision boundaries, enabling the human-vehicle co-evolutionary mechanism to contextualize self-adaptation according to the regional driving etiquette, and thus cracking the inherent contradiction between individual trajectory optimization and group traffic harmony. Empirical studies based on HighwayEnv driving scenarios show that compared with pure reinforcement learning methods, this method reduces human-vehicle interaction conflicts while maintaining self-vehicle efficiency. The research results provide a quantifiable interaction paradigm and a verifiable training architecture for the construction of culturally-aware autonomous driving systems through the deep coupling of computational social value modeling and human social intelligence.

Keywords: Real-Time Control Strategies
Abstract: Usually, a controller for path- or trajectory tracking is employed in autonomous driving. Typically, these controllers generate high-level commands like longitudinal acceleration or force. However, vehicles with combustion engines expect different actuation inputs. This paper proposes a longitudinal control concept that translates high-level trajectory-tracking commands to the required low-level vehicle commands such as throttle, brake pressure and a desired gear. We chose a modular structure to easily integrate different trajectory-tracking control algorithms and vehicles. The proposed control concept enables a close tracking of the high-level control command. An anti-lock braking system, traction control, and brake warmup control also ensure a safe operation during real- world tests. We provide experimental validation of our concept using real world data with longitudinal accelerations reaching up to 25 m/s^2. The experiments were conducted using the EAV24 racecar during the first event of the Abu Dhabi Autonomous Racing League on the Yas Marina Formula 1 Circuit.

Keywords: Real-Time Control Strategies, Motion Planning Algorithms for Autonomous Vehicles, Predictive Trajectory Models and Motion Forecasting
Abstract: Autonomous racing is an emerging research field that has gained significant interest in recent years. Controlling a vehicle at the limit of tire grip presents a challenging control problem. This paper demonstrates the implementation of a randomized optimization based Model Predictive Control (MPC), referred to as Randomized Model Predictive Control (RMPC), to address the challenge of autonomous racing. The random samples in the proposed RMPC are generated using a novel generation technique in the frequency domain. This approach prevents undesirable oscillations, which are critical to the smoothness required for vehicle control. The proposed RMPC is implemented on a Graphics Processing Unit (GPU), minimizing data transfer to the Central Processing Unit (CPU). The results demonstrate improved control performance and lap times that surpass those achievable with state-of-the-art nonlinear control methods. The proposed scheme proved fast and efficient in driving a full-scale autonomous race car built on a Dallara SF23 Superformula chassis. As part of the Abu Dhabi Autonomous Racing League (A2RL) race of 2024, the proposed implementation set the fastest autonomous lap of the series at the iconic Yas Marina Circuit, achieving a lap time within 10% of a Formula 1 driver in the human-driven version of the same car.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Adaptive Vehicle Control Techniques, Real-Time Control Strategies
Abstract: This paper presents a novel contingency planning framework that integrates learning-based multi-modal predictions of traffic participants into Branch Model Predictive Control (MPC). Leveraging reachability analysis, we address the computational challenges associated with Branch MPC by organizing the multitude of predictions into driving corridors. Analyzing the overlap between these corridors, their number can be reduced through pruning and clustering while ensuring safety since all prediction modes are preserved. These processed corridors directly correspond to the distinct branches of the scenario tree and provide an efficient constraint representation for the Branch MPC. We further utilize the reachability for determining maximum feasible decision postponing times, ensuring that branching decisions remain executable. Qualitative and quantitative evaluations demonstrate significantly reduced computational complexity and enhanced safety and comfort.

Keywords: Real-Time Control Strategies
Abstract: Motion control of autonomous vehicles (AVs) that considers nonlinear dynamics and multidimensional motion coupling characteristics represents a critical research direction, particularly for vehicles operating under extreme conditions. However, the nonlinearity of model and the time-varying characteristics of reference states make control law design challenging, often resorting to computationally expensive approaches such as model predictive control (MPC). This paper proposes an explicit nonlinear control design framework for optimal trajectory tracking of AVs. Specifically, taking the three-degree-of-freedom vehicle dynamics model as an example, we first augment the system state to yield an affine representation, followed by input-output feedback linearization. The discrete-time linearized system is then augmented with reference states from the preview horizon, and a linear quadratic regulator is designed to provide an analytical solution for the virtual control inputs. Finally, the original control inputs are computed using the feedback linearization control law. We performed simulations to evaluate the effectiveness of our proposed approach and compared it against MPC. Results indicate that our proposed approach achieves tracking accuracy, smoothness, and robustness comparable to MPC, while significantly reducing computational requirements, with a computing time of nearly 0 ms.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Motion Forecasting, Ethics in Driving Decision Making
Abstract: In urban intersection scenarios, the traditional reachable set-based planning lacks the capability to account for traffic rules, while the spatiotemporal coupling characteristics lead to poor trajectory tracking performance during sharp turns. To address these issues, this paper introduces traffic rule semantics to constrain the coarse trajectories and drivable corridors during the reachable set-based coarse planning stage. Additionally, a decoupled planning approach is adopted to replace the coupled optimization method for fine planning in curved sections. The simulation results demonstrate that the proposed method could make autonomous vehicle almost in accordance with the traffic rules at urban intersections. Furthermore, the deviation error from the lane centerline during sharp turns is reduced by at least 46% compared to the coupled approach.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Real-Time Control Strategies, Adaptive Vehicle Control Techniques
Abstract: Autonomous vehicles (AVs) face significant decision-making challenges in unprotected left-turn scenarios due to complex game-theoretic interactions with human-driven vehicles. The coexistence of velocity constraints and multiple Nash equilibria (NEs) exacerbates the dilemma between decisiveness and safety. This paper proposes a game-theoretic framework addressing three key aspects: 1) generating velocity-constrained trajectories to simplify closed-loop equilibrium derivation, 2) resolving equilibrium multiplicity through spatiotemporal safety analysis, and 3) developing a conflict-aware selection mechanism for safety-critical decisions. Extensive simulations demonstrate 10.46% higher traffic efficiency and 15.24% safety rate compared to conventional game-theoretic baselines, effectively balancing interaction ambiguity resolution with operational constraints.

Keywords: Real-Time Control Strategies, Adaptive Vehicle Control Techniques, Level 2 ADAS Control Techniques
Abstract: This work presents a methodology to estimate tire parameters and their uncertainty using a Bayesian optimization approach. The literature mainly considers the estimation of tire parameters but lacks an evaluation of the parameter identification quality and the required slip ratios for an adequate model fit. Therefore, we examine the use of Stochastical Variational Inference as a methodology to estimate both - the parameters and their uncertainties. We evaluate the method compared to a state-of-the-art Nelder-Mead algorithm for theoretical and real-world application. The theoretical study considers parameter fitting at different slip ratios to evaluate the required excitation for an adequate fitting of each parameter. The results are compared to a sensitivity analysis for a Pacejka Magic Formula tire model. We show the application of the algorithm on real-world data acquired during the Abu Dhabi Autonomous Racing League and highlight the uncertainties in identifying the curvature and shape parameters due to insufficient excitation. The gathered insights can help assess the acquired data's limitations and instead utilize standardized parameters until higher slip ratios are captured. We show that our proposed method can be used to assess the mean values and the uncertainties of tire model parameters in real-world conditions and derive actions for the tire modeling based on our simulative study.

Keywords: Control Strategies for Autonomous UAVs, Real-Time Data Processing for UAVs
Abstract: In drone missions, accurate estimation of remaining flight time is crucial for enhancing mission duration and reducing the risk of losing drones due to power outages. This paper addresses the challenge of predicting future power consumption for a quadcopter UAV, focusing on the energy demands of planned future maneuvers. Existing physics-based energy consumption models assume the drone hovers or moves with constant speed. While orientation and velocity measurements are typically used, acceleration measurements are often ignored, despite being identified as important for power usage prediction in recent machine learning literature. This paper proposes a physics-based, lightweight energy consumption model based on measurements of the quadcopter’s motion, including acceleration. Our model also includes a synthetic wind estimation scheme, enhancing prediction accuracy in outdoor and windy conditions. The model’s predictions for power consumption are compared to real-life flight data, demonstrating the effectiveness of the presented approach despite its relative simplicity.

Keywords: Decision Making, Multi-Objective Planning Approaches
Abstract: This paper introduces COR-MCTS (Conservation of Resources - Monte Carlo Tree Search), a novel tactical decision-making approach for automated driving focusing on maneuver planning over extended horizons. Traditional decision-making algorithms are often constrained by fixed planning horizons, typically up to 6 seconds for classical approaches and 3 seconds for learning-based methods, limiting their adaptability in particular dynamic driving scenarios. However, planning must be done well in advance in environments such as highways, roundabouts, and exits to ensure safe and efficient maneuvers. To address this challenge, we propose a hybrid method integrating Monte Carlo Tree Search (MCTS) with our prior utility-based framework, COR-MP (Conservation of Resources Model for Maneuver Planning). This combination enables long-term, real-time decision-making, significantly enhancing the ability to plan a sequence of maneuvers over extended horizons. Through simulations across diverse driving scenarios, we demonstrate that COR-MCTS effectively improves planning robustness and decision efficiency over extended horizons.

Keywords: Decision Making, Multi-Objective Planning Approaches, Motion Forecasting
Abstract: Autonomous vehicles (AVs) often display overly conservative behavior in mixed traffic environments with human-driven vehicles (HVs). This is primarily due to the complexity of interactions, diverse conflict points, and the heterogeneity of human driving behaviors. These factors pose significant challenges to AVs' decision-making. To effectively solve the problem, this paper proposes a virtual game-theoretic framework based on the interaction influence weight quantification for multi-vehicle interaction. The upper layer of the framework employs an attention mechanism based on Graph Convolutional Network (GCN) and Long Short-Term Memory (LSTM) networks to capture the interaction effects between vehicles. The lower layer estimates the Social Value Orientation (SVO) of each interactive vehicle according to the historical trajectories to identify heterogeneity and models the interaction process as a virtual game among vehicles to solve the optimal decisions. Finally, simulation experiments show that the proposed algorithm outperforms existing benchmark decision-making algorithms in terms of safety and efficiency, showing good robustness in interactions with surrounding vehicles of different driving styles.

Keywords: Decision Making, Motion Forecasting, Multi-Objective Planning Approaches
Abstract: Connected Autonomous Vehicles (CAVs) achieve efficient information sharing through V2V and V2I communication, enabling effective collaborative driving at unsignalized intersections and beyond. However, when unexpected issues such as communication blockages or failures occur, significant challenges occur to allow reliable and efficient cooperative decision-making. This paper introduces the Predicted Inter-Distance Profile based on Probabilistic Uncertainty Interval (PIDP-PUI) method and a cooperative optimized decision process under uncertain situations. Simulation results demonstrate that the proposed methods enable collision-free decision-making and efficient planning at unsignalized intersections, even under communication loss.

Keywords: Reinforcement Learning for Planning, End-to-End Neural Network Architectures and Techniques, Decision Making
Abstract: Reinforcement Learning (RL) is a promising approach for achieving autonomous driving due to robust decision-making capabilities. RL learns a driving policy through trial and error in traffic scenarios, guided by a reward function that combines the driving objectives. The design of such reward function has received insufficient attention, yielding ill-defined rewards with various pitfalls. Safety, in particular, has long been regarded only as a penalty for collisions. This leaves the risks associated with actions leading up to a collision unaddressed, limiting the applicability of RL in real-world scenarios. To address these shortcomings, our work focuses on enhancing the reward formulation by defining a set of driving objectives and structuring them hierarchically. Furthermore, we discuss the formulation of these objectives in a normalized manner to transparently determine their contribution to the overall reward. Additionally, we introduce a novel risk-aware objective for various driving interactions based on a two-dimensional ellipsoid function and an extension of Responsibility-Sensitive Safety (RSS) concepts. We evaluate the efficacy of our proposed reward in unsignalized intersection scenarios with varying traffic densities. The approach decreases collision rates by 21% on average compared to baseline rewards and consistently surpasses them in route progress and cumulative reward, demonstrating its capability to promote safer driving behaviors while maintaining high-performance levels.

Keywords: Decision Making, Human-Machine Interface (HMI) Design Principles, Predictive Trajectory Models and Motion Forecasting
Abstract: In interactions between automated vehicles (AVs) and crossing pedestrians, modeling implicit vehicle communication is crucial. In this work, we present a combined prediction and planning approach that allows to consider the influence of the planned vehicle behavior on a pedestrian and predict a pedestrian’s reaction. We plan the behavior by solving two consecutive optimal control problems (OCPs) analytically, using variational calculus. We perform a validation step that assesses whether the planned vehicle behavior is adequate to trigger a certain pedestrian reaction, which accounts for the closed-loop characteristics of prediction and planning influencing each other. In this step, we model the influence of the planned vehicle behavior on the pedestrian using a probabilistic behavior acceptance model that returns an estimate for the crossing probability. The probabilistic modeling of the pedestrian reaction facilitates considering the pedestrian’s costs, thereby improving cooperative behavior planning. We demonstrate the performance of the proposed approach in simulated vehicle-pedestrian interactions with varying initial settings and highlight the decision making capabilities of the planning approach.

Keywords: Decision Making, Reinforcement Learning for Planning, Predictive Trajectory Models and Motion Forecasting
Abstract: Generating optimal trajectories in dynamic environments is crucial for advanced autonomous driving. Analyzing multi-level processes individually can obscure the interdependencies between levels, resulting in suboptimal trajectories. Furthermore, short-term planning often fails to anticipate dynamic road conditions, thereby limiting hazard identification. This leads to steering or speed control errors due to high computational demands, which ultimately compromise smooth driving. To address these challenges, this study proposes a robust framework that integrates multi-level modules to generate optimal trajectories and to execute predicted state-based long-term planning. In particular, we employ Hierarchical Reinforcement Learning (HRL): the upper level makes high-level driving decisions, and the generated trajectory serves as an objective function for the lower-level motion planner, which is executed by a low-level controller. Additionally, the framework incorporates dynamic state prediction of surrounding vehicles, enabling long-term planning based on predicted state vectors. To evaluate the proposed framework, various scenarios were simulated using the CARLA autonomous driving simulator. Results show that the framework significantly outperforms baseline models in trajectory smoothness, computational efficiency, hazard avoidance, adaptability, and learning performance. These improvements demonstrate its effectiveness in dynamic multi-lane environments for autonomous driving.

Keywords: Vehicle-to-Infrastructure (V2I) Communication, V2X Communication Protocols and Standards
Abstract: This paper introduces a centralized approach for fuel-efficient urban platooning by leveraging real-time Vehicle-to-Everything (V2X) communication and Signal Phase and Timing (SPaT) data. A nonlinear Model Predictive Control (MPC) algorithm optimizes the trajectories of platoon leader vehicles, employing an asymmetric cost function to minimize fuel-intensive acceleration. Following vehicles utilize a gap- and velocity-based control strategy, complemented by dynamic platoon splitting logic communicated through Platoon Control Messages (PCM) and Platoon Awareness Messages (PAM). Simulation results obtained from the CARLA environment demonstrate substantial fuel savings of up to 41.2%, along with smoother traffic flows, fewer vehicle stops, and improved intersection throughput.

Keywords: End-to-End Neural Network Architectures and Techniques, Reinforcement Learning for Planning, Adaptive Vehicle Control Techniques
Abstract: Developing robust, efficient navigation algorithms is challenging. Rule-based methods offer interpretability and modularity but struggle with learning from large datasets, while end-to-end neural networks excel in learning but lack transparency and modularity. In this paper, we present MIND-Stack, a modular software stack consisting of a localization network and a Stanley Controller with intermediate human interpretable state representations and end-to-end differentiability. Our approach enables the upstream localization module to reduce the downstream control error, extending its role beyond state estimation. Unlike existing research on differentiable algorithms that either lack modules of the autonomous stack to span from sensor input to actuator output or real-world implementation, MIND-Stack offers both capabilities. We conduct experiments that demonstrate the ability of the localization module to reduce the downstream control loss through its end-to-end differentiability while offering better performance than state-of-the-art algorithms based on traditional path tracking approaches. We showcase sim-to-real capabilities by deploying the algorithm on a real-world embedded autonomous platform with limited computation power and demonstrate simultaneous training of both the localization and controller towards one goal. While MIND-Stack shows good results, we discuss the incorporation of additional modules from the autonomous navigation pipeline in the future, promising even greater stability and performance in the next iterations of the framework.

Keywords: Decision Making, Real-Time Control Strategies, Multi-Objective Planning Approaches
Abstract: Achieving high-level autonomous driving poses significant challenges, primarily due to the extensive and time-consuming testing required for low-probability abnormal behavior scenarios. Existing simulation platforms are limited by microscopic traffic flow models based on car-following and lane-changing theories (focused on the behavior of individual vehicles), which limits the implementation of complex and diverse abnormal behaviors such as road deviation and cutting in. This paper proposes an Abnormal behavior Generation model based on Event Probability triggers and Static paths (AGEPS model), which is capable of continuously and automatically generating abnormal behaviors without disrupting normal traffic flow. The AGEPS model is computationally efficient and scalable, as it decouples path optimization and trajectory tracking from obstacle avoidance, using explicit control law for both tracking and avoidance. The paper demonstrates three typical abnormal behaviors: Overtaking on the Right (OOR), Driving on the Lane Line (DOL), and Sudden Braking (SUB), indicating that the AGEPS model effectively generates these behaviors by selecting and tracking target static paths while adjusting lateral offsets and desired speeds. Experimental results validate the AGEPS model's effectiveness and its ability to generate abnormal behaviors continuously. Simulation results indicate that the AGEPS model reduces single-step computation time (including decision-making and control) by over 92.8% compared to the MPC controller, with a single-step execution time of just 4 ms.

Keywords: UAV Motion Planning Algorithms, Deep Learning Based Approaches, Real-Time Control Strategies
Abstract: Emergency search and rescue (SAR) operations often require rapid and precise target identification in com- plex environments where traditional manual drone control is inefficient. In order to address these scenarios, a rapid SAR system, UAV-VLRR (Vision-Language-Rapid-Response), is developed in this research. This system consists of two aspects: 1) A multimodal system which harnesses the power of Visual Language Model (VLM) and the natural language processing capabilities of ChatGPT-4o (LLM) for scene interpretation. 2) A non-linear model predictive control (NMPC) with built- in obstacle avoidance for rapid response by a drone to fly according to the output of the multimodal system. This work aims at improving response times in emergency SAR operations by providing a more intuitive and natural approach to the operator to plan the SAR mission while allowing the drone to carry out that mission in a rapid and safe manner. When tested, our approach was faster on an average by 33.75% when compared with an off-the-shelf autopilot and 54.6% when compared with a human pilot.

Keywords: Decision Making, Reinforcement Learning for Planning
Abstract: One of the unresolved challenges for autonomous vehicles is occlusion-aware decision-making under the high uncertainty of various occlusions. Recent occlusion-aware decision-making methods encounter issues such as overly conservative behavior, high computational complexity, or scenario scalability challenges. Benefiting from automatically generating data by exploration randomization, we uncover that reinforcement learning (RL) may show promise in occlusion-aware decision-making. However, previous occlusion-aware RL faces challenges in expanding to various dynamic and static occlusion scenarios, low learning efficiency, and lack of predictive ability. To address these issues, we introduce Pad-AI, a self-reinforcing framework to learn occlusion-aware decision-making through active perception. Pad-AI utilizes vectorized representation to represent occluded environments efficiently and learns over the semantic motion primitives to focus on high-level active perception exploration. Furthermore, Pad-AI integrates prediction and RL within a unified framework to provide risk-aware learning and reliable policy optimization. Our framework was tested in challenging scenarios under both dynamic and static occlusions and demonstrated efficient and general perception-aware exploration performance to other strong baselines in closed-loop evaluations.

Keywords: Reinforcement Learning for Planning, Decision Making, Motion Planning Algorithms for Autonomous Vehicles
Abstract: In complex real-world traffic environments, autonomous vehicles (AVs) need to interact with other traffic participants while making real-time and safety-critical decisions accordingly.The unpredictability of human behaviors poses significant challenges, particularly in dynamic scenarios, such as multi-lane highways and unsignalized T-intersections.To address this gap, we design a bi-level interaction decision-making algorithm (BIDA) that integrates interactive Monte Carlo tree search (MCTS) with deep reinforcement learning (DRL), aiming to enhance interaction rationality, efficiency and safety of AVs in dynamic key traffic scenarios.Specifically, we adopt three types of DRL algorithms to construct a reliable value network and policy network, which guide the online deduction process of interactive MCTS by assisting in value update and node selection.Then, a dynamic trajectory planner and a trajectory tracking controller are designed and implemented in CARLA to ensure smooth execution of planned maneuvers.Experimental evaluations demonstrate that our BIDA not only enhances interactive deduction and reduces computational costs, but also outperforms other latest benchmarks, which exhibits superior safety, efficiency and interaction rationality under varying traffic conditions.

Keywords: Decision Making, Level 3 Driving Systems Architecture and Techniques, Real-Time Control Strategies
Abstract: In complex environments, Intelligent Vehicles (IVs) require reliable decision-making to ensure safe and efficient navigation. To guarantee the proper operation of the IV, it must operate within its Operational Design Domain (ODD). This means that, in a specific context, the vehicle must have the necessary capabilities to guarantee the efficient and robust working of its functions. Current decision-making approaches primarily address dynamic constraints but often fail to consider the full vehicle's ODD. When the ODD is considered, it is monitored to make only an immediate decision. For instance, if it starts raining, the vehicle decides to reduce speed and increase the following distance to ensure safe braking on wet roads. However, neglecting future long-term states could lead the vehicle to an imminent departure from its ODD, and this could increase the appearance of risky situations. This paper presents a decision-making architecture, focused on the tactical level, designed to address these gaps. By evaluating the vehicle's ODD across both current and future states, the proposed framework constructs a reachable horizon that supports long-term decision-making. The proposed architecture, called ODD-aptive, formalizes the decision-making process as a Markov Decision Process (MDP), enabling a systematic analysis of vehicle capabilities and reachable states confined to the vehicle's ODD. By focusing on long-term decision-making, this approach ensures IVs to remain functional, adaptable, and safe even under dynamic and evolving conditions, supporting a reliable autonomy of IVs.

Keywords: Feedback Systems for Driver Interaction, Deep Learning Based Approaches
Abstract: Voice interaction is one of the important development directions of intelligent cabins. However, various noise interferences inside and outside the vehicle pose significant challenges to human-vehicle interaction. Speech enhancement technology can extract clear speech signals from mixed speech signals, thereby significantly improving the quality and intelligibility of voice commands, making it a current research hotspot. To address the issue of low intelligibility of voice commands in vehicle environment, we propose a speech enhancement method based on 3D tensor representation. Specifically, through short-time Fourier transform, the real and imaginary parts are concatenated in a new dimension to form a 3D tensor as the input feature. Based on convolutional neural networks, we conducted systematic research, building UNet speech enhancement models based on 1D, 2D, and 3D convolutions respectively, and compared the performance of the proposed 3D complex domain model with that of the time-domain and frequency-domain models. Additionally, we incorporated an attention mechanism and developed a time-frequency domain joint model by leveraging its information filtering and focusing capabilities, thereby significantly enhancing the speech enhancement effect. Finally, we conducted extensive experiments on the Voicebank+Demand dataset. The results show that although the time-frequency domain joint model outperforms the 3D complex domain model on the test set, in the vehicle environment, the 3D complex domain model achieved the highest scores of 3.67, 97.94, and 4.36 respectively in the three metrics of PESQ, STOI, and COVL.

Keywords: Driver State Detection Algorithms, Automotive Datasets
Abstract: Field sobriety tests and breathalyzers are commonly used to prevent alcohol-impaired driving, but they are expensive and time-consuming to administer. We propose a set of sobriety tests which, in contrast, can feasibly be automated and deployed to modern vehicles equipped with a driver monitoring camera. Our tests are inspired by research on the physiological effects of alcohol, with particular focus on eye movements and gaze behavior. We run an exploratory in-lab study with N=50 subjects (20 alcohol-impaired, 30 control), and train a variety of models to detect alcohol impairment. We find that, using only 10 seconds of observations of the driver, one of the four proposed tests performs comparably to existing non-breathalyzer field sobriety tests. We make our code and data available to support further research efforts to combat alcohol-impaired driving: https://toyotaresearchinstitute.github.io/IV25-beyond-breathalysers/

Keywords: User Experience in Autonomous Vehicles, User-Centric Intelligent Vehicle Technologies, Safety Verification and Validation Techniques
Abstract: In-car conversational systems bring the promise to improve the in-vehicle user experience. Modern conversational systems are based on Large Language Models (LLMs), which makes them prone to errors such as hallucinations, i.e., inaccurate, fictitious, and therefore factually incorrect information. In this paper, we present an LLM-based methodology for the automatic factual benchmarking of in-car conversational systems. We instantiate our methodology with five LLM-based methods, leveraging ensembling techniques and diverse personae to enhance agreement and minimize hallucinations. We use our methodology to evaluate CarExpert, an in-car retrieval-augmented conversational question answering system, with respect to factual correctness—specifically, factual relevance and factual consistency with the vehicle’s manual. We produced a novel dataset specifically created for the in-car domain, and tested our methodology against an expert evaluation. Our results show that the combination of GPT-4 with the Input Output Prompting achieves over 90% factual correctness agreement rate with expert evaluations, other than being the most efficient approach yielding an average response time of 4.5s. Our findings suggest that LLM-based testing constitutes a viable approach for the validation of conversational systems regarding their factual correctness.

Keywords: Human Factors Analysis in Vehicle Design, User Experience in Autonomous Vehicles, User-Centric Intelligent Vehicle Technologies
Abstract: Drivers' perception of risk determines their acceptance, trust, and use of Automated Driving Systems (ADSs). However, perceived risk is subjective and difficult to evaluate using existing methods. To address this issue, a driver's subjective perceived risk (DSPR) model is proposed, regarding perceived risk as a dynamically triggered mechanism with anisotropy and attenuation. 20 participants are recruited for a driver-in-the-loop experiment to report their real-time subjective risk ratings (SRRs) when experiencing various automatic driving scenarios. A convolutional neural network and bidirectional long short-term memory network with temporal pattern attention (CNN-Bi-LSTM-TPA) is embedded into a semi-supervised learning strategy to predict SRRs, aiming to reduce data noise caused by subjective randomness of participants. The results illustrate that DSPR achieves the highest prediction accuracy of 87.91% in predicting SRRs, compared to three state-of-the-art risk models. The semi-supervised strategy improves accuracy by 20.12%. Besides, CNN-Bi-LSTM-TPA network presents the highest accuracy among four different LSTM structures. This study offers an effective method for assessing driver's perceived risk, providing support for the safety enhancement of ADS and driver's trust improvement.

Keywords: Human-Machine Interface (HMI) Design Principles, Profile Extraction and Discovery from Datasets, User Experience in Autonomous Vehicles
Abstract: Data dashboards can visually present data for filtering, interaction and data analysis in a dynamic and intuitive way. In the automotive sector, test engineers are interested in monitoring and analyzing vehicle measurement data from test drives. Data sections recorded under specific conditions are especially relevant, showing important scenarios of the driving operation. This work explores Large Language Models (LLM) assisting in data filtering requests that could help the user identify relevant dashboard filter settings for their domain-specific questions. We use function calling to select functions and corresponding parameter values from a provided set of analysis functions. We tested our method using GPT-3.5-Turbo and zero-shot generalization with a generated test data set of user prompts. The results show that correct functions and parameters can be selected by the model, but iterative and use case-specific adaptations and fine-tuning might be required to achieve a reliable automatic functionality in a dashboard application. We propose possible further approaches, with user studies being relevant for the exploration of productive usage.

Keywords: Human Factors Analysis in Vehicle Design
Abstract: Understanding how human drivers handle interactions with each other can aid the development of automated vehicles capable of operating in mixed traffic. Interactions between human drivers are often complex, so driver behavior models are needed to better understand them. However, existing models mostly focus on the behavior of one driver, which limits their ability to explain complex reciprocal interactions between multiple drivers. At the same time, the prior research that does focus on interactive behaviors of two or more drivers is typically limited to describing drivers' tactical decisions, limiting the understanding of how these decisions are related to operational aspects of behavior (safety margins and control inputs). In this work, we address this gap, focusing specifically on highway merging interactions. We build upon the Communication-Enabled Interactions (CEI) framework - a previously proposed holistic approach to interaction modeling. We develop a CEI-based model of highway merging that captures both tactical and operational aspects of the behavior of two drivers interacting in a highway merging scenario. Our model exhibits human-like behavior aligned with empirical observations of high-level decisions (i.e., who goes first?), safety margins (headways), and position and velocity profiles. Based on our model, we identify key mechanisms regarding drivers' beliefs, velocity perception, and planning, which can potentially generalize beyond highway merging to other interactive human driving behaviors. Our findings highlight the potential of the CEI framework in modeling reciprocal traffic interactions in realistic traffic scenarios, and contribute to understanding the complexities of interactions between human drivers.

Keywords: Driver State Detection Algorithms
Abstract: Driver drowsiness detection (DDD) prevents road accidents caused by driver fatigue. Vehicle dynamics-based DDD has been proposed as a method that is both economical and high performance. However, there are concerns about the reliability of performance indicators and the reproducibility of many of the existing methods. This paper aims to develop a vehicle dynamics-based DDD method whose implementation and experimental results are transparent to related work. We first formalize a framework for extracting features from an open dataset and performing DDD with lightweight ML models; the framework is carefully designed to support a variety of configurations. Second, we implement three existing representative methods and a concise method in the framework. Finally, we report the results of experiments to verify the reproducibility and clarify the performance of DDD based on common metrics. Our findings imply the issues inherent in DDD methods developed in a non-standard manner, and demonstrate a high performance method implemented appropriately.

Keywords: Driver State Detection Algorithms, Feedback Systems for Driver Interaction, Human Factors Analysis in Vehicle Design
Abstract: In this paper, we introduce a conditional transformer-based U-net architecture for speech emotion recognition (SER). The proposed architecture consists of convolutional transformer (CTR)-based encoder, feature decoder and emotion decoder. The CTR-based U-net encoder is designed to extract compressed bottleneck features for recognition. The emotion is then predicted by the CTR-based emotion decoder, where we use additional trainable emotion query to better capture the characteristics of different emotional states. The CTR-based feature decoder reconstructs the given input features from the bottleneck features. This auxiliary decoder allows us to use additional information while training the model, which further improves the recognition performance. Specifically, the proposed feature decoder is performed through sophisticated embedding of the predicted emotional states and the features of the encoder layers. In the proposed framework, we consider the channel attention for each time frame in the CTRs to better enable a real-time processing of emotion recognition. Experimental results showed that the proposed emotion classification method provided better recognition performance than the selected benchmark algorithms.

Keywords: Foundation Models Based Approaches, Driver State Detection Algorithms, Feedback Systems for Driver Interaction
Abstract: Driver monitoring systems face critical challenges in modern transportation, including limited multitasking capabilities and a lack of interpretability. These limitations hinder the accurate and comprehensive assessment of driver states such as distraction, drowsiness, and emotions, which are essential to ensure road safety. This paper introduces visual language models for multitask domain adaptation in driver monitoring (VLM-DM), a novel framework that addresses these challenges by leveraging advanced visual language models for the simultaneous execution of multiple driver monitoring tasks. By employing parameter-efficient training methods such as Low-Rank Adaptation (LoRA) and integrating dynamic prompt tuning, VLM-DM achieves superior performance compared to state-of-the-art methods. Our experiments on three benchmark datasets across different driver states, demonstrating significant improvements in multitask accuracy and interpretability. This work highlights the potential of advanced multitask and multimodal architectures in developing robust, scalable, and interpretable driver monitoring systems for real-world applications.

Keywords: Level 2 ADAS Control Techniques, Deep Learning Based Approaches
Abstract: The visual attention of human drivers in traffic scenarios is often affected by subjective and objective factors. Unfortunately, the previous work on visual attention prediction of drivers rarely considered the impact of subjective intention on the selection of focus area and object. Therefore, we propose a cognitively guided modeling for the visual attention selection of the driver. Firstly, we use the driving experience of human drivers to construct a cognitive map that depicts the relationship between driving intention and the focus area and object in different situations (including driving scene and vehicle status). Subsequently, we combine the object information recommended by the graph in the current situation with the object information in the environment to determine the object list that needs attention. Finally, class activation mapping (CAM) is used to map the object into a heat map, which is then fused with the attention area weight to generate the final visual attention prediction. To verify the effectiveness of the method, we constructed the Intention-Related Driving Attention Dataset (IRDA) and conducted experiments. The experimental results show that our method is superior to the classic saliency prediction model and deep network-based visual attention in main indicators. In addition, we also conducted experiments on the public datasets BDDA and DADA. The experimental results show that the method of this paper obtains results that are as competitive as the state-of-the-art method.

Keywords: User Experience in Autonomous Vehicles, Human-Machine Interface (HMI) Design Principles, Trust and Acceptance of Autonomous Technologies
Abstract: With the accelerating embrace of automated vehicles, lack of ‎public trust has become a ‎critical concern worldwide. ‎Nonetheless, the literature shows a significant and thorough ‎effort ‎to increase trust in automated vehicles. Biophilia is one ‎strategy being utilized to establish a ‎trustworthy environment, ‎although it has not yet been applied in the automotive industry, ‎thus, a ‎biophilic interior design was developed and applied in a ‎static driving simulator using a ‎mixed reality headset for ‎investigating the effect on the driving experience. A gender-‎‎balanced sample (11 F,11 M) was asked to perform an in-lab ‎experiment and experienced four ‎different driving scenarios, ‎with and without Biophilic interventions. The regression ‎results ‎showed that initial trust is the only effective parameter ‎on affective trust in ‎automated ‎vehicles is (P=0.05), and no ‎significant relationship was found between gender and ‎affective ‎trust. As conclusion, the study allowed to assess that ‎initial trust in self-driving cars ‎plays a ‎more important role ‎than gender or driving experience contrary to initial ‎expectations.

Keywords: Human Factors Analysis in Vehicle Design, Human-Machine Interface (HMI) Design Principles, User-Centric Intelligent Vehicle Technologies
Abstract: Autonomous Personal Mobility Vehicles (APMVs) are designed to address the ``last-mile'' transportation challenge for everyone. When an APMV encounters a pedestrian, it uses an external Human-Machine Interface (eHMI) to negotiate road rights. Through this interaction, passengers are also passively exposed to the process. This study examines passengers' gaze behavior toward pedestrians during such interactions, focusing on whether passengers' personality traits influence their gaze patterns towards pedestrians when using different eHMI designs. When using a visual-based eHMI, which caused passengers to struggle in perceiving the communication content, the results suggested that passengers with higher Neuroticism scores, who were more sensitive to communication details, might seek cues from pedestrians' reactions. In addition, a multimodal eHMI (visual and voice) using neutral voice did not significantly affect the gaze behavior of passengers toward pedestrians, regardless of personality traits. In contrast, a multimodal eHMI using affective voice encouraged passengers with high Openness to Experience scores to focus on pedestrians' heads. In summary, this study revealed how different eHMI designs influence passengers' gaze behavior and highlighted the effects of personality traits on their gaze patterns toward pedestrians, providing new insights for personalized eHMI designs.

Keywords: Driver State Detection Algorithms, Feedback Systems for Driver Interaction, Human-Machine Interface (HMI) Design Principles
Abstract: Cognitive distraction (CD) is one of the major causes of traffic accidents, but there remains room to improve its detection. Most prior research on CD detection has commonly used basic statistical measures (e.g., mean, standard deviation) of driver-facing camera signals such as gaze and pupil size. However, these signals often exhibit subtle and complex patterns that conventional approaches cannot fully capture. In this paper, we evaluate a wide range of machine learning models and feature extraction methods using data from 52 participants in a driving simulator under two cognitive distraction inducing tasks (n-back and statement tasks). Our results demonstrate that combining gaze, pupil, and features derived from physiological signals (e.g., fixation saccade ratio and gaze entropy) and comprehensive time-series feature extraction boosts detection performance. While deep neural networks (Transformers) excel at modeling intricate relationships, our results show that tree-based ensemble methods (e.g., CatBoost) achieve comparable or higher detection performance while maintaining their advantage of better interpretability. Cross-task experiments further show that models trained on one type of task can generalize to another task. Feature analyses (via SHAP and Sobol) reveal that nonlinearity in vertical gaze movements, baseline pupil size, and greater minimum gaze distance are related to CD. These findings suggest that integrating multiple modalities, sophisticated feature engineering, and employing models capable of capturing nonlinear interactions are effective strategies for detecting CD. To support future research in this field, we release our code and preprocessed data: https://toyotaresearchinstitute.github.io/IV25-cognitive-distraction/

Keywords: Synthetic Data Generation for Training, Trust and Acceptance of Autonomous Technologies
Abstract: Machine learning models, particularly those used in open-world settings like autonomous driving, require extensive and diverse datasets for effective training. However, the overabundance of standard situations in large datasets often leads to underrepresentation of rare scenarios and edge cases, which are crucial for achieving high performance across all situations. This work proposes a simulation framework for capturing human-in-the-loop simulation data, enabling the creation of realistic and diverse datasets for machine learning models as well as human behavior studies. The framework combines full body motion capture with virtual reality to record scene-relevant interactions between humans and virtual objects, allowing for the generation of various ground truths in simulation. To validate the effectiveness of this approach, an action recognition model is trained on simulated data generated by the proposed framework. From the point of view of a vehicle, the model needs to determine if the vehicle is the intended target of a waving pedestrian. The investigation into different keypoint representations and spatial encodings reveals the importance of integrating spatial and contextual data for accurate action recognition. Furthermore, the results show that human-in-the-loop simulations can effectively capture complex human behaviors and interactions, enabling the creation of more realistic and diverse datasets for machine learning models.

Keywords: Teleoperation Control Systems for Vehicles, Human Factors Analysis in Vehicle Design
Abstract: Remote driving allows to control a vehicle from a distance by a remote operator. Remote drivers use video feeds and sensor data to understand the current driving situation and to generate control commands. Remote driving raises several human-factor challenges. One of these challenges is remote drivers’ ability to deal with time delays in the communication between teleoperated vehicle and operator workstation. By using a teledriving simulator, a subject study with N = 20 participants was conducted to examine the effect of different up- and downlink latencies on remote drivers’ behavior, workload, and driving performance. The results show that latencies of 200 ms increase driving effort to keep the vehicle in the lane and cause higher workload. Effects on driving performance are weak due to compensation behavior (braking and speed reduction). Different effects of up- and downlink latencies do not become apparent. The implications of the results regarding human-factors research of remote driving are discussed. Furthermore, future approaches and research tools to empirically analyze remote driving are considered.

Keywords: Human Factors Analysis in Vehicle Design, Trust and Acceptance of Autonomous Technologies, Ethics in Driving
Abstract: As highly automated vehicles (AVs) are deployed in various countries, mixed-autonomy traffic will become common and persist for the foreseeable future. In such environments, AVs must operate in ways which are both predictable and acceptable to human drivers, particularly in complex intersections where negotiation is crucial. However, how human drivers expect AVs to interact with them, particularly in scenarios where the right-of-way is ambiguous, remains unclear. In this research, we conducted a simulation-based video survey of UK drivers (N = 87), investigating their perception of aggressive and defensive AV driving styles under unclear right-of-way scenarios. The analysis indicates that a defensive driving style is generally preferred by human drivers, while an aggressive style can also be acceptable at lower-speed interaction zones. These findings provide empirical evidence for algorithm engineers seeking to design motion control and negotiation strategies that align with human expectations in mixed traffic.

Keywords: Driver State Detection Algorithms, Feedback Systems for Driver Interaction, User Experience in Autonomous Vehicles
Abstract: Personalized gaze estimation in in-vehicle settings is crucial for driver monitoring but remains challenging due to individual anatomical differences and environmental variability. Existing approaches often rely on fully supervised fine-tuning, which requires labeled calibration data and limits practical deployment. To overcome this limitation, we propose a Test-Time Personalized Gaze Estimation framework that enables in-vehicle gaze personalization with minimal supervision. Our method builds upon GazeDPTR_V2, which extends gaze estimation to gaze zone classification by leveraging positional features from point projection and visual attributes from images. We introduce a two-stage adaptation strategy: (1) Self-Supervised Test-Time Adaptation, where augmentation-based self-supervision is applied to adapt the model to freely available unlabeled test-time data, facilitated by Model-Agnostic Meta-Learning (MAML) for effective unsupervised adaptation; (2)Weakly-Supervised Personalization, where a small number of gaze zone samples collected from the driver during an initial setup phase are used for fine-tuning. We evaluate our approach on the IVGaze dataset, which provides both gaze direction vectors and gaze zone labels. Experimental results show that our method significantly outperforms baseline models, demonstrating the effectiveness of self-supervised meta-learning and weak supervision for real-world in-vehicle gaze estimation.

Keywords: Human Factors Analysis in Vehicle Design, Vulnerable Road User Protection Strategies
Abstract: This study assesses a Vehicle-to-Pedestrian (V2P) collision warning system compared to conventional vehicle-issued auditory alerts in a real-world scenario simulating a vehicle on a fixed track, characterized by limited maneuverability and the need for timely pedestrian response. The results from analyzing speed variations show that V2P warnings are particularly effective for pedestrians distracted by phone use (gaming or listening to music), highlighting the limitations of auditory alerts in noisy environments. The findings suggest that V2P technology offers a promising approach to improving pedestrian safety in urban areas

Keywords: Data Annotation and Labeling Techniques, Deep Learning Based Approaches, Techniques for Dataset Domain Adaptation
Abstract: Deep learning models for object detection in autonomous driving have recently achieved impressive performance gains and are already being deployed in vehicles worldwide. However, current models require increasingly large datasets for training. Acquiring and labeling such data is costly, necessitating the development of new strategies to optimize this process. Active learning is a promising approach that has been extensively researched in the image domain. In our work, we extend this concept to the LiDAR domain by developing several inconsistency-based sample selection strategies and evaluate their effectiveness in various settings. Our results show that using a naive inconsistency approach based on the number of detected boxes, we achieve the same mAP as the random sampling strategy with 50% of the labeled data.

Keywords: 3D Scene Reconstruction Methods, Automotive Datasets, Dataset Augmentation Using Neural Field
Abstract: Accurate 3D reconstruction of vehicles is vital for applications such as vehicle inspection, predictive maintenance, and urban planning. Existing methods like Neural Radiance Fields and Gaussian Splatting have shown impressive results but remain limited by their reliance on dense input views, which hinders real-world applicability. This paper addresses the challenge of reconstructing vehicles from sparse-view inputs, leveraging depth maps and a robust pose estimation architecture to synthesize novel views and augment training data. Specifically, we enhance Gaussian Splatting by integrating a selective photometric loss, applied only to high-confidence pixels, and replacing standard Structure-from-Motion pipelines with the DUSt3R architecture to improve camera pose estimation. Furthermore, we present a novel dataset featuring both synthetic and real-world public transportation vehicles, enabling extensive evaluation of our approach. Experimental results demonstrate state-of-the-art performance across multiple benchmarks, showcasing the method’s ability to achieve high-quality reconstructions even under constrained input conditions.

Keywords: Advanced Multisensory Data Fusion Algorithms, User-Centric Intelligent Vehicle Technologies, Static and Dynamic Object Detection Algorithms
Abstract: Vehicle-following presents significant advantages in flexible scenarios such as platooning and valet parking, where a human-driven leader navigates complex environments, and an autonomous follower replicates the leader's trajectory. The key objective is accurate, robust, long-term pose tracking of the leader. This paper proposes a framework using mass-produced sensors, including a front fisheye camera, front millimeter-wave radar, and wireless communication. The framework consists of three modules: 1) a Visual Feature Tracking module using a DiMP tracker for robust taillight feature tracking; 2) a Visualradar Fusion module to estimate the rear center of the leader vehicle for reference point tracking; 3) a Leader Pose Tracking module combining sensor data and vehicle-to-vehicle communication in a particle filter framework for precise pose tracking in three degrees of freedom. Real-world experiments show lateral and longitudinal errors of 0.056 meters and 0.165 meters, respectively.

Keywords: Deep Learning Based Approaches, High-Resolution Image Sensing Techniques, Synthetic Data Generation for Training
Abstract: This paper presents a real-time AI-based stereo vision system for automotive applications. The model is capable of handling cameras with different lens types such as pinhole and fisheye, as well as arbitrary camera poses. A coarse to fine model is implemented, which can provide depth information at various resolutions. The cost volume is generated using spherical/depth sweeping method and based on the intrinsic and extrinsic parameters. The model is trained and tested for different camera arrangements using synthetic datasets generated in CARLA. The results show that the proposed model is camera-agnostic, allowing models trained on specific camera pairs to transfer to other setups without retraining. Thus possible misalignment can be compensated by recalibrating extrinsic parameters and modifying the cost volume creation. This system is well-suited for modern vehicles equipped with multiple different cameras, and offers a robust depth estimation for applications like low speed maneuver and parking assistance.

Keywords: Infrastructure Requirements for Automated Vehicles, Static and Dynamic Object Detection Algorithms, Lidar-Based Environment Mapping
Abstract: The deployment of roadside LiDAR sensors plays a crucial role in the development of Cooperative Intelligent Transport Systems (C-ITS). However, the high cost of LiDAR sensors necessitates efficient placement strategies to maximize detection performance. Traditional roadside LiDAR deployment methods rely on expert insight, making them time-consuming. Automating this process, however, demands extensive computation, as it requires not only visibility evaluation but also assessing detection performance across different LiDAR placements. To address this challenge, we propose a fast surrogate metric, the Entropy-Guided Visibility Score (EGVS), based on information gain to evaluate object detection performance in roadside LiDAR configurations. EGVS leverages Traffic Probabilistic Occupancy Grids (TPOG) to prioritize critical areas and employs entropy-based calculations to quantify the information captured by LiDAR beams. This eliminates the need for direct detection performance evaluation, which typically requires extensive labeling and computational resources. By integrating EGVS into the optimization process, we significantly accelerate the search for optimal LiDAR configurations. Experimental results using the AWSIM simulator demonstrate that EGVS strongly correlates with Average Precision (AP) scores and effectively predicts object detection performance. This approach offers a computationally efficient solution for roadside LiDAR deployment, facilitating scalable smart infrastructure development.

Keywords: Radar Object Detection and Tracking, Static and Dynamic Object Detection Algorithms, Deep Learning Based Approaches
Abstract: Accurate 3D multi-object tracking (MOT) is vital for autonomous vehicles, yet LiDAR and camera-based methods degrade in adverse weather. Meanwhile, Radar-based solutions remain robust but often suffer from limited vertical resolution and simplistic motion models. Existing Kalman filter–based approaches also rely on fixed noise covariance, hampering adaptability when objects make sudden maneuvers. We propose Bayes-4DRTrack, a 4D Radar-based MOT framework that adopts a transformer-based motion prediction network to capture nonlinear motion dynamics and employs Bayesian approximation in both detection and prediction steps. Moreover, our two-stage data association leverages Doppler measurements to better distinguish closely spaced targets. Evaluated on the K-Radar dataset (including adverse weather scenarios), Bayes4DRTrack demonstrates a 5.7% gain in Average Multi-Object Tracking Accuracy (AMOTA) over methods with traditional motion models and fixed noise covariance. These results showcase enhanced robustness and accuracy in demanding, realworld conditions.

Keywords: Foundation Models Based Approaches, Deep Learning Based Approaches
Abstract: This paper proposes novel methods to enhance the performance of monocular 3D object detection models by leveraging the generalized feature extraction capabilities of a vision foundation model. Unlike traditional CNN-based approaches, which often suffer from inaccurate depth estimation and rely on multi-stage object detection pipelines, this study employs a Vision Transformer (ViT)-based foundation model as the backbone, which excels at capturing global features for depth estimation. It integrates a detection transformer (DETR) architecture to improve both depth estimation and object detection performance in a one-stage manner. Specifically, a hierarchical feature fusion block is introduced to extract richer visual features from the foundation model, further enhancing feature extraction capabilities. Depth estimation accuracy is further improved by incorporating a relative depth estimation model trained on large-scale data and fine-tuning it through transfer learning. Additionally, the use of queries in the transformer's decoder, which consider reference points and the dimensions of 2D bounding boxes, enhances recognition performance. The proposed model outperforms recent state-of-the-art methods, as demonstrated through quantitative and qualitative evaluations on the KITTI 3D benchmark and a custom dataset collected from high-elevation racing environments. Code is available at url{https://github.com/JihyeokKim/MonoDINO-DETR}.

Keywords: Lidar-Based Environment Mapping, Smart City Mobility Integration Strategies
Abstract: Overgrown roadside vegetation poses a danger to road users by obstructing the visibility of the road and potentially obscuring signs or other traffic participants. Thus, regulations clearly define the height above the road that must not be obscured. As manually identifying such incidents is time-consuming, we propose 3D Clearance Control: a pipeline that automatically detects vegetation in need of trimming. Our system is based on LiDAR point clouds, which give access to accurate position and height information. It comprises four main steps: the semantic segmentation of the point cloud, the aggregation of scans within a scene, the estimation of road boundaries, and the creation of the volume representing the clearance gauge. We developed a modular process to perform a comprehensive evaluation combining different segmentation models and road boundary approximation methods. We measure the accuracy and computing times on three widely used street-level datasets: SemanticKITTI, NuScenes, and PandaSet. We achieved an mIoU of 67.6 on our annotated test scenes and a speed increase of 52.7% compared to previous systems.

Keywords: Application of Neural Fields in Autonomous Driving, Techniques for Dataset Domain Adaptation, Static and Dynamic Object Detection Algorithms
Abstract: Autonomous vehicles often have varying camera sensor setups, which is inevitable due to restricted placement options for different vehicle types. Training a perception model on one particular setup and evaluating it on a new, different sensor setup reveals the so-called cross-sensor domain gap, typically leading to a degradation in accuracy. In this paper, we investigate the impact of the cross-sensor domain gap on state-of-the-art 3D object detectors. To this end, we introduce CamShift, a dataset inspired by nuScenes and created in CARLA to specifically simulate the domain gap between subcompact vehicles and sport utility vehicles (SUVs). Using CamShift, we demonstrate significant cross sensor performance degradation, identify robustness dependencies on model architecture, and propose a data-driven solution to mitigate the effect.

On the one hand, we show that model architectures based on a dense Bird’s Eye View (BEV) representation with backward projection, such as BEVFormer, are the most robust against varying sensor configurations. On the other hand, we propose a novel data-driven sensor adaptation pipeline based on neural rendering, which can transform entire datasets to match different camera sensor setups. Applying this approach improves performance across all investigated 3D object detectors, mitigating the cross-sensor domain gap by a large margin and reducing the need for new data collection by enabling efficient data reusability across vehicles with different sensor setups.

The CamShift dataset and the sensor adaptation benchmark are available at https://dmholtz.github.io/camshift/.

Keywords: Deep Learning Based Approaches
Abstract: In this paper, we introduce a novel method for creating appearance embeddings to identify individual persons using an object re-identification (ReID) framework. We present CLFormer (Camera LiDAR Transformer), a transformer-based architecture that incorporates multi-modal data from both camera and LiDAR sensors. We introduce the 3D Cuboid-Inclusive Point Embedding (3D-CIPE), which leverages rich data from LiDAR point clouds and 3D cuboids to add a learnable embedding into the transformer structure. Additionally,through ablation studies, we explore and analyze various strategies for the early and late fusion of multi-modal input data. To evaluate our proposed CLFormer, we reinterpret the nuScenes dataset [1] for ReID purposes and use it for our experiments. Our method demonstrates a significant improvement in performance, outperforming the image-only baseline with an increase of 2.3 in mean Average Precision (mAP).

Keywords: Advanced Multisensory Data Fusion Algorithms, Deep Learning Based Approaches
Abstract: Perception is a crucial aspect of an autonomous driving system. One essential task is represented by multi-camera 3D object detection, which allows an intelligent vehicle to detect the surrounding obstacles using a camera-only setup. Currently, there are a lot of different approaches trying to solve this task, with many of them being transformer-based. Specifically, most of these make use of object queries instead of a Bird’s Eye View plane to directly represent the set of possible detections and avoid any post-processing operation, like non-maxima suppression. However, the ambiguous supervision caused by the bipartite matching loss typically leads to training instability. To overcome this limitation, we propose an additional module able to “push” the object queries toward the locations that more likely contain obstacles, providing both better insights into their position to the detection module and stabilizing the bipartite matching during training. We evaluate our proposal against different objet queries-based baselines both on the nuScenes dataset test and validation sets. Specifically, compared to the lightweight PETR architecture, we highlight an increase of 1.6% both in NDS and mAP under the same configuration settings.

Keywords: Integrating Diverse Data Sources (e.g HD maps, LIDAR) in Neural Scene Representations, Scalable Neural Scene Representation
Abstract: Scene reconstruction and novel view synthesis are extensively utilized in domains such as autonomous driving, facilitating the creation of more comprehensive datasets and accelerating algorithm development. Current techniques have achieved promising results in image-based synthesis. Nonetheless, they encounter difficulties including limited dynamic range, motion blur, and suboptimal performance under challenging illumination. Furthermore, these techniques depend significantly on precise camera poses, which are difficult to acquire in practical situations. Event cameras, characterized by their high dynamic range and temporal resolution, demonstrate advantages in challenging illumination and rapid motion scenarios and have been utilized in several autonomous driving applications. This study is the first to combine image and event camera for neural radiance field (NeRF) reconstruction in driving scenarios. The proposed method leverages the unique properties of event camera and adopts a progressive reconstruction strategy to jointly optimize camera poses during training, reducing reliance on pose precision and enabling more robust and accurate scene reconstruction under challenging illumination. Experimental results demonstrate that the proposed approach attains enhanced performance in novel view synthesis, exhibiting a 3.5% increase in PSNR and a 2% increase in SSIM on the DSEC dataset compared to baseline method.

Keywords: 3D Scene Reconstruction Methods, Integration Methods for HD Maps and Onboard Sensors, Advanced Multisensory Data Fusion Algorithms
Abstract: Maps play an important role in autonomous driving systems. The recently proposed 3D Gaussian Splatting (3D-GS) produces rendering-quality explicit scene reconstruction results, demonstrating the potential for map construction in autonomous driving scenarios. However, due to the time and computational costs of generating Gaussian scenes, how to update the map becomes a significant challenge. In this paper, we propose LT-Gaussian, a map update method for 3D-GS-based maps. LT-Gaussian consists of three main components: Multimodal Gaussian Splatting, Structural Change Detection Module, and Gaussian-Map Update Module. Firstly, the Gaussian map of the old scene is generated using our proposed Multimodal Gaussian Splatting. Subsequently, during the map update process, we compare the outdated Gaussian map with the current LiDAR data stream to identify structural changes. Finally, we perform targeted updates to the Gaussian-map to generate an up-to-date map. We establish a benchmark for map updating on the nuScenes dataset to quantitatively evaluate our method. The experimental results show that LT-Gaussian can effectively and efficiently update the Gaussian-map, handling common environmental changes in autonomous driving scenarios. Furthermore, by taking full advantage of information from both new and old scenes, LT-Gaussian is able to produce higher quality reconstruction results compared to map update strategies that reconstruct maps from scratch. Our open-source code is available at https://github.com/ChengLuqi/LT-gaussian.

Keywords: Deep Learning Based Approaches, Advanced Multisensory Data Fusion Algorithms
Abstract: Radar is a key sensing modality to obtain scene understanding in automotive driving applications. However, automotive radars that are based on the widely used frequency-modulated continuous-wave (FMCW) signal waveforms are vulnerable to signal spoofing attacks. These attacks can distort the resulting radar images at the victim radar receiver via a false positive by introducing artificial objects, or due to a false negative by concealing real objects. In this work, we propose a deep learning method to detect radar points compromised by spoofing attacks and to classify the type of attack. Leveraging mid-fusion of radar and camera sensors, as well as a multi-head architecture, our method simultaneously identifies the specific spatial locations where spoofing attacks distort the radar image, which enables the use of undistorted portions of the data for partial functionality. Evaluation on real data with simulated cases of spoofing attacks shows that our approach achieves mean classification F1 scores of 0.93 for false positive and 0.82 for false negative attacks, demonstrating its effectiveness in enhancing radar image integrity.

Keywords: Deep Learning Based Approaches, Advanced Multisensory Data Fusion Algorithms, Automotive Datasets
Abstract: Precise vehicle state estimation is crucial for safe and reliable autonomous driving. The number of measurable states and their precision offered by the onboard vehicle sensor system are often constrained by cost. For instance, measuring critical quantities such as the Vehicle Sideslip Angle (VSA) poses significant commercial challenges using current optical sensors. This paper addresses these limitations by focusing on the development of high-performance virtual sensors to enhance vehicle state estimation for active safety. The proposed Uncertainty-Aware Hybrid Learning (UAHL) architecture integrates a machine learning model with vehicle motion models to estimate VSA directly from onboard sensor data. A key aspect of the UAHL architecture is its focus on uncertainty quantification for individual model estimates and hybrid fusion. These mechanisms enable the dynamic weighting of uncertainty-aware predictions from machine learning and vehicle motion models to produce accurate and reliable hybrid VSA estimates. This work also presents a novel dataset named Real-world Vehicle State Estimation Dataset (ReVStED), comprising synchronized measurements from advanced vehicle dynamic sensors. The experimental results demonstrate the superior performance of the proposed method for VSA estimation, highlighting UAHL as a promising architecture for advancing virtual sensors and enhancing active safety in autonomous vehicles.

Keywords: Sensor Fusion for Accurate Localization, Advanced Multisensory Data Fusion Algorithms, Synthetic Data Generation for Training
Abstract: Precise covariance estimation is critical in intelligent vehicles and autonomous systems, as many algorithms rely on it to optimize performance. However, covariance values are often arbitrarily defined due to a lack of reliable information. Many systems provide scores that do not directly reflect positional or dimensional uncertainty, limiting the interpretability and reliability of these systems in real-world applications.

This paper proposes a novel Gaussian Process-based algorithm to enhance 3D detection outputs by estimating the covariance of inferred attributes. Unlike previous methods, the proposed approach directly quantifies uncertainty using detections in a sensor-agnostic manner, offering robust and interpretable covariance estimates. The method integrates statistical tools, including interval segmentation strategies and variance analysis, to refine error estimation across different detection classes and spatial dimensions. The proposed framework is evaluated using a KITTI-based dataset under diverse scenarios, demonstrating that ground-truth detections consistently fall within the inferred uncertainty regions. The experimental results highlight the method’s high performance in uncertainty estimation, leading to more reliable sensor fusion, object tracking, and localization in intelligent systems.

Keywords: Representation Learning for Driving Scenarios, Deep Learning Based Approaches
Abstract: Recent studies have suggested that potential shortcomings of certain perception modules can be discovered by analyzing the performance of worst scenarios. However, finding the worst perception scenario (WPS) requires datasets with rich semantic annotations of the scenes in visual perception tasks and it is time-consuming to label all the scenario data. To address this, we proposed a method of prediction for WPS, which utilized prior information to predict the model performance under the absence of annotations. Specifically, this paper introduced a scenario matcher based on hybrid re-ranking, which combined the labeled and unlabeled data to generate the pseudo-sample set. In addition, we designed a sample reorganization module to update this sample set through the nearest neighbor retrieval. We also discussed the distribution relationship between labeled and unlabeled data, categorizing it into three cases, and validated the effectiveness of the proposed method on the KITTI and ApolloScape datasets.

Keywords: Self-Diagnostic Systems for Vehicle Safety, Automotive Datasets, Real-World Testing Methodologies for Safety Systems
Abstract: Construction equipment has important roles in industries such as construction and mining. Any downtime because of failures increase cost. Traditional diagnostic systems detect failures only after they occur, making it difficult to take precautions and prolonging repair times. This paper is the first to address the analysis of machine learning-powered Prognostic and Health Management (PHM) systems specifically for predicting failures in diesel engine air intake systems, focusing on two common issues: air leakage and Exhaust Gas Recirculation (EGR) blockage. This study compares various machine learning and deep learning models for anomaly detection and fault classification using real-world sensor data from controlled engine tests. The results demonstrate that ensemble and neural network-based machine learning methods, such as Random Forest, XGBoost, and LSTM, achieve highly successful predictions for anomaly detection and fault classification.

Keywords: Safety Verification and Validation Techniques, Real-World Testing Methodologies for Safety Systems, Vulnerable Road User Protection Strategies
Abstract: Simulation models of law-abiding pedestrians are all too prevalent. However, the models of dangerous pedestrians are frighteningly limited. This leads to the unreliability of autonomous driving in human habitats as they learn to deal with pedestrians exhaustively in a simulated world. There exists a shortage of data and analysis on dangerous pedestrians. Decision-making factors and behavior identification are not sufficient; a dangerous pedestrian model should consist of a collection of behaviors that represent their natural behavior pattern. On this basis, we propose to define these collections of behaviors in the form of pedestrian archetypes. Each archetype suggests a distinct pedestrian personality with their own special take on crossing the road, giving autonomous driving an opportunity to significantly improve their testing strategies and reliability against pedestrians.

Keywords: Safety Verification and Validation Techniques
Abstract: The range of vision of vehicle sensors used by automated driving functions is considerably influenced by the lateral movement of vehicles within their lane. With the increasing relevance of simulations for the validation of automated driving functions, realistic modeling of this lateral movement gains in importance. Different stochastic models addressing this task have been proposed in literature. The used datasets and performed evaluations, however, are diverse, hampering the comparison of the models. Further, in some cases the evaluated features are limited, restricting the assessment of the model's applicability for a wide range of use cases. This work therefore starts with the identification of a suitable uniform data basis and evaluation strategy enabling the latter. Based on this, the models are compared in terms of qualitative and quantitative results and runtime. The results reveal strengths and limitations of the models, indicate suitable use cases, and reveal potentials for improvement.

Keywords: Safety Verification and Validation Techniques, Real-World Testing Methodologies for Safety Systems, Automotive Datasets
Abstract: The deployment of automated functions that can operate without direct human supervision has changed safety evaluation in domains seeking higher levels of automation. Unlike conventional systems that rely on human operators, these functions require new assessment frameworks to demonstrate that they do not introduce unacceptable risks under real-world conditions. To make a convincing safety claim, the developer must present a thorough justification argument, supported by evidence, that a function is free from unreasonable risk when operated in its intended context. The key concept relevant to the presented work is the intended context, often captured by an Operational Design Domain specification (ODD) specification. ODD formalization is challenging due to the need to maintain flexibility in adopting diverse specification formats while preserving consistency and traceability and integrating seamlessly into the development, validation, and assessment. This paper presents a way to formalize an ODD in the Pkl language, addressing central challenges in specifying ODDs while improving usability through specialized configuration language features. The approach is illustrated with an automotive example but can be broadly applied to ensure rigorous assessments of operational contexts.

Keywords: Real-World Testing Methodologies for Safety Systems, Safety Verification and Validation Techniques, Real-Time Control Strategies
Abstract: In this paper, a control barrier function (CBF) is constructed for the lane-keeping problem which is applicable to both human-driven and automated vehicles. Based on the resulting CBF, a safety filter is developed that prevents the vehicle from crossing the lane boundaries, while only modifying the nominal steering input when necessary. The effectiveness of the proposed control approach is demonstrated in a series of numerical simulations and real vehicle experiments with a human driver. The experimental results show that the safety filter can successfully keep the vehicle inside the lane boundaries by seamlessly modifying the steering input of the human driver in a minimally invasive manner.

Keywords: Safety Verification and Validation Techniques, Decision Making, Real-World Testing Methodologies for Safety Systems
Abstract: Recent years have brought a surge of efforts in rethinking the vehicle’s electrical and/or electronic (E/E) architecture as well as the development process to reduce complexity and enable automation, connectivity, and electromobility. Resource allocation is an important step of the development process that can influence the quality of the designed system. As the design space is large and complex, intuitive design can turn into a time consuming process with sub-optimal solutions. Here, we present an approach to automatically map software components to available hardware resources. Compared to existing frameworks, our method provides a wider range of safety analyses in compliance with the ISO 26262 standard, encompassing aspects such as reliability, task scheduling, and automotive safety integrity level (ASIL) compatibility. We propose an integer linear programming (ILP)-based approach to perform ASIL decomposition technique specified by the standard. This technique proves beneficial when suitable hardware resources are unavailable for a safety critical application. We formulate a multi-objective optimization problem to minimize both the development cost and the maximum execution times of critical function chains.The effectiveness of the proposed approach is investigated on an exemplary case study, and the results of the performance analysis are presented and discussed.

Keywords: Trust and Acceptance of Autonomous Technologies, Vulnerable Road User Protection Strategies, Safety Verification and Validation Techniques
Abstract: Beyond safety considerations for pedestrians in autonomous driving, fairness in detection promotes societal acceptance and builds public trust. This work introduces FAIR-PED, a novel framework to examine the impact of pedestrian visual attributes on the fair performance of ML-based detection models. The attribute labels are automatically generated using a pedestrian attribute recognition method that leverages contrastive language-image pretraining (CLIP) models, requiring little to no manual annotation. Over 100 pretrained CLIP models are evaluated based on recall balance and accuracy. The ECP dataset is automatically labeled using the best-performing CLIP model for selected attributes. Bootstrap resampling is employed to validate the results. Fairness is measured using the Equal Opportunity Difference (EOD) metric, which compares the recall rates of subgroups. Evaluations reveal detection rate differences across attributes, with insignificant biases toward women and pedestrians carrying bags (EOD = -1.12% and -0.34%, respectively) and a significantly lower detection rate for laterally viewed pedestrians (EOD = +4.65%). Since people crossing the road are often seen as laterally viewed pedestrians, the consistent presence of view bias across all detectors is particularly concerning.

Keywords: Safety Verification and Validation Techniques
Abstract: The generation of unknown unsafe scenarios is crucial for the verification and validation of automated vehicles. However, the complexity of traffic and the sensitivity of testing costs pose significant challenges in reducing the unknown unsafe region. To address this problem, a framework is proposed for generating unknown unsafe scenarios based on hierarchical classification. Firstly, the critical scenario library is obtained using an evolving simulation system. Secondly, the scenarios in the scenario library are hierarchically classified based on the maneuver, position, and motion of the vehicles, resulting in abstract scenarios. Finally, a concretization method is employed to search for potential unsafe scenarios within these abstract scenarios. This methodology was applied to the testing of the Stackelberg algorithm. The evolving system generated 6142 scenarios. These were then classified using the abstraction method, yielding 35 abstract scenarios. For the abstract scenario with the highest hazard rate of 19.47%, the concretization method was used to design logical scenarios. Based on an optimization algorithm, 443 potential concrete unsafe scenarios were identified.

Keywords: Advanced Passive Safety Systems
Abstract: In highly automated vehicles, the conventional roles of driver and passenger are transformed. This paradigm shift presents novel opportunities for the design of the vehicles interior. Recent research has demonstrated that occupants prefer to sit facing each other in highly automated vehicles, e.g., in living room configuration, in addition to conventional seating arrangements. However, studies have indicated that the restraint systems currently employed in new interior concepts do not achieve the desired effect. The shuttle, developed as part of the AWARE2ALL project funded by the European Union, was simulated in a sled test environment, undergoing various modifications to assess the necessary vehicle structures for ensuring occupant restraint. It was determined that reducing the forward movement of the occupant leads to a successful reduction in loads on the occupant. Incorporating a footrest within the design has emerged as a key element in reducing loads among occupants.

Keywords: Real-World Testing Methodologies for Safety Systems, Safety Verification and Validation Techniques
Abstract: We present a post-hoc approach for scenario-based testing of automated driving systems, enabling the analysis of safety and correctness for (cooperative) automated driving systems in many scenarios without conducting tests for individual scenarios. The system under test is operated in its physical environment, and data is recorded during operation. Then, driving scenarios are identified in this data and functional requirements are checked, yielding pass or fail verdicts for individual scenarios. We validate the envisioned post-hoc approach in a single-case mechanism experiment by the example of a platooning controller, identifying a previously unknown bug in the tested system, as well as a functional insufficiency concerning the intended operational design domain.

Keywords: Safety Verification and Validation Techniques
Abstract: Autonomous vehicles (AVs) face significant threats to their safe operation in complex traffic environments. Adversarial policy for scenario generation has been established as a robust paradigm for enhancing AV resilience against adversarial perturbations through proactive exposure to synthetically engineered safety-critical scenarios. Training an attacker within an adversarial policy, allowing the target AV to expose vulnerabilities through interaction with this attacker. However, adversarial policies in existing methodologies often get stuck in a loop of over-exploiting established vulnerabilities, resulting in poor exploration for AVs. To overcome the limitations, we introduce a pioneering framework termed the vulnerability-aware and curiosity-driven adversarial reinforcement learning policy. Specifically, during the traffic vehicle attacker training phase, a surrogate network is employed to fit the value function of the AV victim, providing dense information about the victim's inherent vulnerabilities. Subsequently, random network distillation is used to characterize the novelty of the scenario, constructing an intrinsic reward to guide the attacker in exploring unexplored territories. Experimental results demonstrated that the adversarial policy embedded within the attacker exhibited robustness in convergence and significantly enhanced the activation of policy exposure in learning-based AVs, outperforming both other adversarial modalities and alternative reinforcement learning approaches, with a notable reduction in crash rates. The code is available at https://github.com/caixxuan/VCAT.

Keywords: Advanced Passive Safety Systems, Motion Forecasting, Automotive Datasets
Abstract: Lane-changing maneuvers, particularly those executed abruptly or in risky situations, are a significant cause of road traffic accidents. However, current research mainly focuses on predicting safe lane changes. Furthermore, existing accident datasets are often based on images only and lack comprehensive sensory data. In this work, we focus on predicting risky lane changes using the CARLA Risky-lane-change Anticipation in Simulated Highways (CRASH) dataset (our own collected dataset specifically for risky lane changes), and safe lane changes (using the HighD dataset). Then, we leverage Knowledge Graphs (KGs) and Bayesian inference to predict these maneuvers using linguistic contextual information, enhancing the model’s interpretability and transparency. The model achieved a 91.5% f1-score with anticipation time extending to four seconds for risky lane changes, and a 90.0% f1-score for predicting safe lane changes with the same anticipation time. We validate our model by integrating it into a vehicle within the CARLA simulator in scenarios that involve risky lane changes. The model managed to anticipate sudden lane changes, thus providing automated vehicles with further time to plan and execute appropriate safe reactions. Finally, to enhance the explainability of our model, we utilize Retrieval Augmented Generation (RAG) to provide clear and natural language explanations for the given prediction.

Keywords: Safety Verification and Validation Techniques, Infrastructure Requirements for Automated Vehicles, Self-Diagnostic Systems for Vehicle Safety
Abstract: Autonomous vehicles are typically developed using a two-layer architecture, consisting of high and low levels. However, communication between these layers has not been widely standardized. For example, in the Autoware Core/Universe architecture, the vehicle interface module interacts with the low-level system defining only the necessary information, not the way to do this. Focusing on this high-low level interface, this work proposes the microAutoware package, a micro-ROS-based framework for STM32 microcontrollers that integrates Autoware Core/Universe vehicle interface with the embedded system. The proposed framework uses ROS resources to create a transparent, efficient, and easy-customizable interface for high and low levels. Due to the complexity and risk of testing the embedded system on real systems, we also developed a hardware-in-the-loop interface, enabling testing of the whole system with the CARLA simulator. With the experiments made in the testbed, the microAutoware supplies all the capabilities of Autoware's vehicle interface, keeping the control performance and providing fail-safe features while still working in real-time. The open-source repository of this work is available at: https://github.com/LMA-FEM-UNICAMP/microAutoware.

Keywords: Safety Verification and Validation Techniques, Automotive Datasets, Infrastructure Requirements for Automated Vehicles
Abstract: The development and deployment of autonomous driving technology are crucial for enhancing future traffic safety. However, road safety challenges persist due to the behavioral differences between autonomous vehicles and human drivers in mixed traffic conditions. A comprehensive understanding of the distinct behaviors of autonomous vehicles is essential to improve traffic safety. This study aims to evaluate the driving volatility of autonomous vehicles and identify volatility hotspots using real-world data. Various volatility indicators commonly used in existing studies were further processed to derive an integrated driving volatility measure based on principal component analyses. Driving volatility was analyzed by comparing autonomous and manual driving modes. Volatility hotspots were identified by comparing driving volatility of each data point. The findings indicate that the average driving volatility in autonomous mode was approximately 45% lower than in manual mode. Factors such as uphill grades were found to increase the instability of autonomous vehicles more significantly than in manual driving. Complex road alignments, such as reverse horizontal curves, increased manual driving volatility. The results of this study provide insights for designing road environments that are more compatible with autonomous vehicles in the future.

Keywords: Synthetic Data Generation for Training, Safety Verification and Validation Techniques, Data Annotation and Labeling Techniques
Abstract: This paper presents an infrastructure for defining scenarios and utilizing them to validate automated driving systems. It addresses various aspects of scenario-based testing, with a focus on lane keeping, lane changing, and traffic light scenarios. We define the scenarios using the OpenSCENARIO 2.0 format, as well as directly through Python scripts. These scenarios are integrated into two distinct simulators: an in- house simulator based on the Intelligent Driver Model (IDM), and the CARLA simulator. In these simulators, two agents are subjected to a range of challenging conditions, and the risk of failure is assessed. This evaluation provides insights into the agents’ performance and their safety compliance, acting as a benchmark for safety assessment in the different scenarios.

Keywords: Static and Dynamic Object Detection Algorithms, Safety Verification and Validation Techniques, Automotive Datasets
Abstract: Precise perception of the environment is essential in highly automated driving systems, which rely on machine learning tasks such as object detection and segmentation. Compression of sensor data is commonly used for data handling, while virtualization is used for hardware-in-the-loop validation. Both methods can alter sensor data and degrade model performance. This necessitates a systematic approach to quantifying image validity. This paper presents a four-step framework to evaluate the impact of image modifications on machine learning tasks. First, a dataset with modified images is prepared to ensure one-to-one matching image pairs, enabling measurement of deviations resulting from compression and virtualization. Second, image deviations are quantified by comparing the effects of compression and virtualization against original camera-based sensor data. Third, the performance of state-of-the-art object detection and semantic segmentation models is analyzed to determine how altered input data affects perception tasks, including bounding box accuracy and reliability. Finally, a correlation analysis is performed to identify relationships between image quality and model performance. As a result, the LPIPS metric achieves the highest correlation between image deviation and machine learning performance across all evaluated machine learning tasks.

Keywords: Safety Verification and Validation Techniques, Techniques for Dataset Domain Adaptation, Deep Learning Based Approaches
Abstract: Color fidelity is often overlooked in simulation-based validation for autonomous vehicles, yet even minor color mismatches can undermine the reliability of AI-driven perception systems. In this paper, we systematically examine how controlled deviations in color reproduction—quantified by DeltaE—affect object detection accuracy across 32 variants of YOLO. Using a Macbeth ColorChecker, we derive calibrations for key color transforms (brightness, contrast, hue, gamma, saturation and color bias) and apply these to the COCO validation set. Our evaluations demonstrate that increasing DeltaE yields significant drops in detection metrics, especially for safety-critical categories such as pedestrians and cyclists. Based on these findings, we propose DeltaE thresholds that define acceptable color fidelity in camera simulations. Furthermore, we contribute these transformed datasets and scripts as a publicly available benchmark, enabling reproducible comparisons and guiding future research on color-based vulnerabilities in automated driving and other safety-critical domains.

Keywords: Real-World Testing Methodologies for Safety Systems, Real-Time Data Processing for UAVs, Self-Diagnostic Systems for Vehicle Safety
Abstract: Autonomous driving is a complex undertaking. A common approach is to break down the driving task into individual subtasks through modularization. These sub-modules are usually developed and published separately. However, if these individually developed algorithms have to be combined again to form a full-stack autonomous driving software, this poses particular challenges. Drawing upon our practical experience in developing the software of TUM Autonomous Motorsport, we have identified and derived these challenges in developing an autonomous driving software stack within a scientific environment. We do not focus on the specific challenges of individual algorithms but on the general difficulties that arise when deploying research algorithms on real-world test vehicles. To overcome these challenges, we introduce strategies that have been effective in our development approach. We additionally provide open-source implementations that enable these concepts on GitHub. As a result, this paper's contributions will simplify future full-stack autonomous driving projects, which are essential for a thorough evaluation of the individual algorithms.

Keywords: Profile Extraction and Discovery from Datasets, Data Annotation and Labeling Techniques, Vulnerable Road User Protection Strategies
Abstract: Imbalanced datasets are a common challenge in machine learning, where rare situations can significantly impact model performance, especially in safety critical applications like autonomous driving. Addressing these imbalances during the recording stage of a dataset is crucial to ensure that models generalize well to new data and real-world scenarios. This paper proposes a novel approach to improve pose dataset distribution live during dataset production by detecting and mitigating rare human poses interactively. Our live out-of-distribution mitigation (LOODM) method uses dimensionality reduction to transform high-dimensional human pose data into a low-dimensional latent space, allowing for efficient analysis of human poses. We analyze the latent space using density and distance to identify unique rare poses and visualize them to a recorded human subject, who can then repeat each rare pose until it is better represented in the dataset. As such, our work focuses on addressing dataset imbalances at their point of origin, namely during recording. LOODM demonstrates significant improvements in the performance of a human pose estimation model on rare poses, while also reducing PCKh metric fluctuations between rare and non-rare poses, making it a promising dataset production tool, e.g., for autonomous driving.

Keywords: Global vs. Local Localization Techniques, End-to-End Neural Network Architectures and Techniques, Deep Learning Based Approaches
Abstract: Accurate camera pose estimation is crucial for autonomous driving and vehicle networking. Traditional pipelines based on geometric models and feature matching struggle in dynamic, featureless environments which are common in many environments. Inspired by the success of vision transformers (ViT), our approach uses a ViT backbone with an attention-based mask to extract a global image descriptor, which is then passed through fully connected layers for pose regression. The multi-headed self-attention in ViT helps the model learn scene layouts and focus on relevant features. We introduce an attention mask to improve performance in challenging scenes, especially dynamic or featureless ones. We compare three backbones: ViT (multi-headed self-attention throughout), ConViT (self-attention in the last two layers, gated positional self-attention elsewhere), and ResNet (pure convolution). We evaluate our model on two commonly used benchmarks for outdoor and indoor localization and we show that our model which uses ViT backbone achieves the state of the art results for both indoor and outdoor multi-scene absolute localization benchmarks.

Keywords: Dynamic Object Tracking, Instance and Panoptic Segmentation Techniques, Automotive Datasets
Abstract: Bird’s Eye View (BEV) maps have gained popularity in the autonomous driving field due to their ability to offer an information-rich and easily interpretable representation of the environment, playing a crucial role in various tasks such as motion planning, perception, and sensor fusion. In this paper, we propose a novel framework that integrates deep learning with grid filtering and feature engineering to achieve multi-object tracking and panoptic segmentation. The framework also validates object detections and removes spurious instances, resulting in a more robust environmental representation that includes tracked objects and semantic segmentation in both perspective and BEV domains. Furthermore, we introduce an original hybrid data association function that combines deep learning-based and handcrafted features to enhance object tracking accuracy. To support this approach, we present a dedicated dataset designed for training the deep learning model used in data association comprising four object categories: cars, pedestrians, riders, and animals. The proposed solution, implemented in C++, is integrated with contemporary deep learning frameworks and has been evaluated on international benchmarks such as Cityscapes and KITTI.

Keywords: Synthetic Data Generation for Training, Motion Planning Algorithms for Autonomous Vehicles, Reinforcement Learning for Planning
Abstract: Motion planning is a fundamental problem in autonomous driving and perhaps the most challenging to comprehensively evaluate because of the associated risks and expenses of real-world deployment. Therefore simulations play an important role in efficient development of planning algorithms. To be effective, simulations must be accurate and realistic, both in terms of dynamics and behavior modeling, and also highly customizable in order to accommodate a broad spectrum of research frameworks. In this paper, we introduce SMARTS 2.0, the second generation of our motion planning simulator which, in addition to being highly optimized for large-scale simulation, provides many new features, such as realistic map integration, vehicle-to-vehicle (V2V) communication, traffic and pedestrian simulation, and a broad variety of sensor models.

Moreover, we present a novel benchmark suite for evaluating planning algorithms in various highly challenging scenarios, including interactive driving, such as turning at intersections, and adaptive driving, in which the task is to closely follow a lead vehicle without any explicit knowledge of its intention. Each scenario is characterized by a variety of traffic patterns and road structures. We further propose a series of common and task-specific metrics to effectively evaluate the performance of the planning algorithms. At the end, we evaluate common motion planning algorithms using the proposed benchmark and highlight the challenges the proposed scenarios impose. The new SMART 2.0 features and the benchmark are publicly available at the SMARTS repository.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Multi-Objective Planning Approaches, Level 2 ADAS Control Techniques
Abstract: Current Autopilot systems struggle in highly dynamic traffic scenarios due to their lack of human-like competitive capability. This research proposes a human-like Autopilot planner with right-of-way priority acquisition capability. It bears the following features: i) human-like Autopilot with proactive right-of-way acquisition capability; ii) with enhanced maneuvering optimality; iii) aggressive but not unsafe; iv) with real-time implementation capability. The proposed approach can identify surrounding vehicles’ driving styles in real time and plan motions based on Stackelberg competition. Simulation results show that the proposed approach acquires right-of-way priority proactively. The proposed approach enhances driving efficiency by up to 3.57% across different congestion levels and improves driving safety by up to 13.52% compared to the conventional Autopilot planner. The proposed approach shows remarkable flexibility in dense traffic. Additionally, the proposed approach enables real-time implementation by maintaining less than 100 milliseconds of computation time.

Keywords: Predictive Trajectory Models and Motion Forecasting, Feedback Systems for Driver Interaction, User-Centric Intelligent Vehicle Technologies
Abstract: We introduce a rear-view camera system designed for trailers, comprising multiple cameras, to address the challenges of limited visibility and the awkward maneuverability of a trailer when performing a reverse driving task (eg parking). The cameras are placed on the top edge of the trailer looking downwards, while the different camera feeds are transformed and stitched to create a top-down view. The actual steering angle of the vehicle and the hitch angle are measured using IoT hardware. The path of the trailer is determined using inverse kinematics of the vehicle and trailer, and lines are overlaid on the top-down view to provide drivers with dynamic, real-time guiding lines on their smartphone. We describe a practical implementation of our system on one instance of a vehicle and a trailer, and extrapolate the concept to different types of trailers in simulations.

Keywords: Motion Forecasting, Self-Diagnostic Systems for Vehicle Safety, Real-Time Control Strategies
Abstract: On-line availability of instantaneous values of joint angles in (automated/intelligent) articulated vehicles is crucial for the supervision and diagnostics, localization, feedback control and motion planning purposes. Direct precise measuring of articulation angles in the joints of multi-trailer vehicles is usually expensive and mechanically challenging. In this paper, we derive, analyze, and compare two alternative (algebraic- and integral-type) on-line reconstruction methods of joint angles using only speed information of the vehicle's segments. Thanks to a modular representation of vehicle kinematics, the on-line reconstruction methods are highly scalable with respect to a number of trailers and applicable to N-trailer vehicles comprising an active tractor and arbitrary number of N passive trailers with fixed (non-steerable) wheels. The considered reconstruction methods require only information about the angular and longitudinal speeds of all the vehicle segments, making the joint angles reconstruction relatively inexpensive and effective in low-speed motion conditions. The methods are experimentally validated in forward and backward motion scenarios using a motion control system implemented on a board of a full-scale intralogistic (tugger-train) 3-trailer vehicle.

Keywords: Collision Avoidance Algorithms, Motion Planning Algorithms for Autonomous Vehicles, Multi-Objective Planning Approaches
Abstract: Nonlinear Model Predictive Control (NMPC) has established itself as a robust framework for autonomous navigation problems by effectively handling nonlinear dynamics and complex constraints in real-time. Despite its success in various applications, challenges remain in integrating robust obstacle avoidance while navigating in dynamic and uncertain environment. To enhance safety, risk assessment metrics like Time-To-Collision (TTC) are widely used for evaluating collision risks. Extensions to these metrics allow their application across diverse road scenarios, making them valuable as cost function components or constraints within risk-aware NMPC formulations In this work, we propose novel obstacle constraint formulations that account for uncertainties in obstacle positions using confidence thresholds, along with TTC-based constraints to ensure safe trajectory planning. Our approaches are validated through extensive simulations in urban scenarios, including overtaking maneuvers and intersections, demonstrating their effectiveness in mitigating risks. Detailed implementations and results are publicly available to support further research and development.

Keywords: Adaptive Vehicle Control Techniques, Predictive Trajectory Models and Motion Forecasting, Real-Time Control Strategies
Abstract: Vehicle mobility and control performance on deformable terrains is governed by the complex interaction that occurs at the tire-terrain interface. Unfortunately, on deformable terrains, accurately measuring terrain information is challenging, and discrepancies between assumed and actual parameters can degrade control performance and cause a loss of vehicle mobility. To address these challenges, this paper proposes an online adaptive Model Predictive Control (MPC) framework for autonomous vehicles operating in off-road environments with deformable terrains. First, we develop a physics-informed learning tire model for deformable terrains that is adaptable online and compatible with MPC. A novel Model Predictive Control formulation is presented for autonomous vehicles operating on deformable terrains and the efficacy of the formulation and proposed tire model is evaluated in simulation with Project Chrono. Comparative experiments, with and without online adaptation, highlight improved speed and path tracking performance through online adaptation when a mismatch between assumed and actual terrain parameters is present.

Keywords: Level 4-5 Autonomous Driving Systems Architecture, Adaptive Vehicle Control Techniques, Sensor Fusion for Accurate Localization
Abstract: This study investigates the application of Online Dynamic Mode Decomposition (Online DMD) for real-time system identification and control in an autonomous vehicle lane-keeping system. The Online DMD algorithm dynamically updates a linear state-space model of lateral vehicle dynamics, enabling continuous adaptation to changing road conditions. To test the robustness and predictive capabilities of these models, Model Predictive Control (MPC) and Linear Quadratic Regulator (LQR) strategies are designed and implemented in MATLAB/Simulink. The system is evaluated under constant longitudinal velocity across diverse road sections in simulation environment. The results demonstrate that combining data-driven system identification with optimal control frameworks achieves robust lane tracking and adaptability, while also revealing that the short-term prediction capability of Online DMD may pose limitations in certain dynamic scenarios.

Keywords: Adaptive Vehicle Control Techniques, Real-World Testing Methodologies for Safety Systems, Real-Time Control Strategies
Abstract: The path tracking performance of global vehicle targets (GVTs) is crucial in closed-field testing for autonomous vehicles (AVs). However, the uncertain latency of 5G communication, when GVTs receive control commands from the cloud server, is likely to adversely impact their path tracking performance. To address this issue, we propose a cloud-based predictive path tracking control strategy considering the uncertain latency of 5G communication. Firstly, the 5G communication dataset from the closed test field is analyzed, and an LSTM-attention latency prediction model is established and validated. Subsequently, a nonlinear model predictive control (NMPC) algorithm that incorporates communication latency is formulated. Finally, real-world tests are conducted, and the results demonstrate that the proposed method significantly improves latency prediction accuracy and path tracking performance of GVTs.

Keywords: Decision Making, Motion Planning Algorithms for Autonomous Vehicles, Real-Time Control Strategies
Abstract: Scientific development often takes place in the context of research projects carried out by dedicated students during their time at university. In the field of self-driving software research, the Formula Student Driverless competitions are an excellent platform to promote research and attract young engineers. This article presents the software stack developed by BME Formula Racing Team, that formed the foundation of the development that ultimately led us to full-scale autonomous racing. The experience we gained here contributes greatly to our successful participation in the Abu Dhabi Autonomous Racing League. We therefore think it is important to share the system we used, providing a valuable starting point for other ambitious students. We provide a detailed description of the software pipeline we used, including a brief description of the hardware-software architecture. Furthermore, we introduce the methods that we developed for the modules that implement perception; localisation and mapping, planning, and control tasks.

Keywords: End-to-End Neural Network Architectures and Techniques, Trust and Acceptance of Autonomous Technologies, Self-Diagnostic Systems for Vehicle Safety
Abstract: The ability to quantify and understand uncertainty is crucial for improving the safety and reliability of autonomous vehicle systems. In this work, we introduce a novel domain awareness mechanism for end-to-end (E2E) autonomous driving algorithms by integrating Spectral-normalized Neural Gaussian Processes (SNGP) for deterministic uncertainty quantification into an E2E trainable autonomous driving framework. The goal is to enable the model trained on simulated data from CARLA to distinguish between unseen CARLA and real-world nuScenes scenarios during inference. Our results demonstrate that, after re-calibration, the model can effectively quantify the domain gap between simulated and real-world data. We found a 13% increase in throttle uncertainty when giving our model nuScenes instead of CARLA data. Additionally our experiments show that the quality of the predicted probability distributions is not influenced by the input domain. We further highlight, that the predictive ability of the E2E model is not affected by the network alterations introduced by SNGP.

Keywords: Multi-Agent Coordination Strategies, Adaptive Vehicle Control Techniques, Predictive Trajectory Models and Motion Forecasting
Abstract: The deployment of Connected and Autonomous Vehicles (CAVs) offers significant potential to improve traffic safety, efficiency, and environmental sustainability. Among the key applications of CAV technology is vehicle platooning, where multiple vehicles cooperate to travel at close distances with high efficiency. However, the implementation of such systems in mixed-traffic conditions, where CAVs coexist with Human-Driven Vehicles (HDVs), introduces substantial challenges. These challenges stem from communication delays, unpredictable human driving behaviors, and the complexities of real-world traffic dynamics. To address these issues, this study proposes a novel car-following control framework based on Model Predictive Control (MPC), which integrates interaction-aware motion prediction with adaptive gap regulation under communication delays, referred to as IAG-MPC. This framework allows CAV platoons to dynamically adjust inter-vehicle spacing in response to the behaviors of surrounding vehicles. Furthermore, the desired gap is dynamically adjusted based on the relative states of vehicles, and it explicitly incorporates communication delays into the control objectives, ensuring robustness and reliability under real-world conditions. Experimental evaluations using the HighD dataset demonstrate that IAG-MPC outperforms baseline approaches in maintaining safe and efficient platoon operation, even in the presence of unpredictable HDV behavior.

Keywords: Adaptive Vehicle Control Techniques, Predictive Trajectory Models and Motion Forecasting, Motion Planning Algorithms for Autonomous Vehicles
Abstract: Extreme cornering in racing often leads to large sideslip angles, presenting a significant challenge for vehicle control. Conventional vehicle controllers struggle to manage this scenario, necessitating the use of a drifting controller. However, the large sideslip angle in drift conditions introduces model mismatch, which in turn affects control precision. To address this issue, we propose a model correction drift controller that integrates Model Predictive Control (MPC) with Gaussian Process Regression (GPR). GPR is employed to correct vehicle model mismatches during both drift equilibrium solving and the MPC optimization process. Additionally, the variance from GPR is utilized to actively explore different cornering drifting velocities, aiming to minimize trajectory tracking errors. The proposed algorithm is validated through simulations on the Simulink-Carsim platform and experiments with a 1:10 scale RC vehicle. In the simulation, the average lateral error with GPR is reduced by 52.8% compared to the non-GPR case. Incorporating exploration further decreases this error by 27.1%. The velocity tracking Root Mean Square Error (RMSE) also decreases by 10.6% with exploration. In the RC car experiment, the average lateral error with GPR is 36.7% lower, and exploration further leads to a 29.0% reduction. Moreover, the velocity tracking RMSE decreases by 7.2% with the inclusion of exploration.

Keywords: Foundation Models Based Approaches, Automotive Datasets, Decision Making
Abstract: This paper presents PrefDrive, a novel framework that integrates driving preferences into autonomous driving models through large language models (LLMs). While recent advances in LLMs have shown promise in autonomous driving, existing approaches often struggle to align with specific driving behaviors (e.g., maintaining safe distances, smooth acceleration patterns) and operational requirements (e.g., traffic rule compliance, route adherence). We address this challenge by developing a preference learning framework that combines multimodal perception with natural language understanding. Our approach leverages Direct Preference Optimization (DPO) to fine-tune LLMs efficiently on consumer-grade hardware, making advanced autonomous driving research more accessible to the broader research community. We introduce a comprehensive dataset of 74,040 sequences, carefully annotated with driving preferences and driving decisions, which, along with our trained model checkpoints, is made publicly available url{https://github.com/LiYun0607/PrefDrive/} to facilitate future research. Through extensive experiments in the CARLA simulator, we demonstrate that our preference-guided approach significantly improves driving performance across multiple metrics, including distance maintenance and trajectory smoothness. Results show up to 28.1% reduction in traffic light violations and 8.5% improvement in route completion while maintaining appropriate distances from obstacles. The framework demonstrates robust performance across different urban environments, showcasing the effectiveness of preference learning in autonomous driving applications.

Keywords: Adaptive Vehicle Control Techniques, Real-Time Control Strategies, Level 3 Driving Systems Architecture and Techniques
Abstract: Parameter tuning for vehicle controllers remains a costly and time-intensive challenge in automotive development. Traditional approaches rely on extensive real-world testing, making the process inefficient. We propose a multi-fidelity Bayesian optimization approach that efficiently learns optimal controller parameters by leveraging both low-fidelity simulation data and a very limited number of real-world experiments. Our approach significantly reduces the need for manual tuning and expensive field testing while maintaining the standard two-stage development workflow used in industry. The core contribution is the integration of an auto-regressive multi-fidelity Gaussian process model into Bayesian optimization, enabling knowledge transfer between different fidelity levels without requiring additional low-fidelity evaluations during real-world testing. We validate our approach through both simulation studies and real- world experiments. The results demonstrate that our method achieves high-quality controller performance with only very few real-world experiments, highlighting its potential as a practical and scalable solution for intelligent vehicle control tuning in industrial applications.

Keywords: Adaptive Vehicle Control Techniques, Cooperative Planning Strategies in Vehicle Networks, Motion Planning Algorithms for Autonomous Vehicles
Abstract: Cooperative Adaptive Cruise Control of Vehicle Platoon is considered with bi-directional communication and heterogeneous parameters in the mathematical model. A model reference adaptive control algorithm is developed with the help of a homogeneous reference platoon (RP). Unlike past works, this work deals with unknown engine efficiency, which leads to bi-linear parametric uncertainty. Classical adaptive law is augmented with a prediction error based term to design a composite adaptation technique for the bi-linear uncertain platoon model. Lyapunov based analysis along with the novel concept of virtual platoon (VP) establishes stability and convergence properties of the controller. Simulation results further validate the efficacy of the proposed algorithm.

Keywords: Decision Making, Reinforcement Learning for Planning
Abstract: The ability to rapidly adapt to unseen scenarios for safe driving has long been a core challenge in autonomous driving. Rule-based methods are heavily reliant on labeled data and suffer from data biases. Many current reinforcement learning(RL)-based methods, on the other hand, are restricted to certain training scenarios, making it difficult for them to adapt to more complex and diverse traffic scenarios. In contrast, human drivers can quickly adapt to new driving situations based on their accumulated driving skills. Inspired by this, we propose the Vehicle Skill Graph (VSG), a novel framework for autonomous driving decision-making. By accumulating 1,000 diverse skills using RL and adopting knowledge graph embedding (KGE) techniques, we build a skill graph that offers a structured understanding of driving knowledge and discovers the potential relations between driving skills and new traffic scenarios. This enables rapid adaptation to new driving environments and addresses the issue of training scenario dependency in RL learning-based approaches. Experimental results demonstrate that VSG effectively captures the latent connections between traffic scenes and driving skills, facilitating efficient sequential decision-making in complex driving situations.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Real-Time Control Strategies, Multi-Agent Coordination Strategies
Abstract: The Combined Alternate-Direction Lane Assignment and Reservation-Based Intersection Control (CADLARIC) concept is a novel approach designed to optimize urban traffic flow by managing directionally unrestricted traffic to improve efficiency while reducing the number of conflicts. It leverages connected and automated vehicles to distribute conflicts and allows vehicles to use opposite-direction lanes, minimizing turn-related conflicts. A main feature of this concept allows vehicles to utilize lanes typically reserved for the opposite direction, thus minimizing conflicts primarily for left- and right-turning vehicles. While previous research focused on vehicular flows, this study incorporates pedestrian accommodation and evaluates the balance between pedestrian and vehicle service. However, prioritizing pedestrians too highly disrupts this balance in autonomous intersections. To better evaluate the multi-modal capabilities of CADLARIC, this study extends its functionality to identify an optimal balance between pedestrian and vehicle service. The results show that, under low pedestrian and vehicle demands, pedestrian control variables have a minimal impact due to limited interactions. However, with higher demands, frequent interactions between vehicles and pedestrians enable the identification of an optimal point that balances service for both modes, improving overall system performance.

Keywords: End-to-End Neural Network Architectures and Techniques
Abstract: A safe and interpretable end-to-end autonomous driving system is essential for real-world applications. However, existing methods struggle with incomplete feature understanding, the black box problem, and poor interpretability, making it hard to adapt to complex environments and be accepted by users. In this study, we propose an end-to-end autonomous driving framework, PA-TCP, which enhances safety and interpretability through a hybrid attention mechanism and efficient state representation. Specifically, we introduce a parallel-weighted compound attention module that dynamically captures and prioritizes critical environmental features for vehicle driving. This module leverages a parallel architecture to simultaneously combine spatial and channel attention mechanisms through learned adaptive weights, enabling fine-grained feature selection and robust scene understanding in challenging scenarios. Next, we integrate vehicle dynamics, navigation commands, and contextual information through a linear-based Squeeze-and-Excitation attention framework, which systematically identifies and emphasizes the most task-relevant features while achieving a balance between representation capability and computational overhead. Extensive experiments on the CARLA simulation platform demonstrate the superiority of our approach over the baseline method TCP, including a 25.08% increase in driving score, a 16.2% increase in route completion, and a 6.8% increase in infraction score. We also demonstrate its effectiveness regarding generalization capabilities.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Multi-Objective Planning Approaches
Abstract: This paper presents a solution to an on-ramp merging trajectory planning problem within a multiple target vehicles environment by employing model predictive control principles. The shrinking horizon MPC (SHMPC) scheme is used to guarantee that the optimal trajectories are selected among the available gaps into which the ego vehicle can merge. For each trajectory, a time-varying terminal set is used to ensure recursive feasibility and to fulfill the safety and obstacle avoidance constraints. To enhance computational efficiency, a procedure is proposed for defining the terminal set to be as large as possible, derived from the offline-calculated maximum controlled invariant set. We present a simple decision mechanism, accounting for safety, comfort, and efficiency. Then, we demonstrate the effectiveness of the proposed solution by applying it to two different scenarios

Keywords: Application of Neural Fields in Autonomous Driving, Static and Dynamic Object Detection Algorithms
Abstract: Real-time traffic light recognition is fundamental for autonomous driving safety and navigation in urban environments. While existing approaches rely on single-frame analysis from onboard cameras, they struggle with complex scenarios involving occlusions and adverse lighting conditions. We present ViTLR, a novel video-based end-to-end neural network that processes multiple consecutive frames to achieve robust traffic light detection and state classification. The architecture leverages transformer-like design with convolutional self-attention modules, optimized specifically for deployment on the Rockchip RV1126 embedded platform. Extensive evaluations on two real-world datasets demonstrate that ViTLR achieves state-of-the-art performance while maintaining real-time processing capabilities (>25 FPS) on RV1126's NPU. The system shows superior robustness across temporal stability, varying target distances, and challenging environmental conditions compared to existing single-frame approaches. We have successfully integrated ViTLR into an ego-lane traffic light recognition system using HD maps for autonomous driving applications. The complete implementation, including source code and datasets, is made publicly available to facilitate further research in this domain.

Keywords: Advanced Multisensory Data Fusion Algorithms, Vehicle-to-Infrastructure (V2I) Communication
Abstract: Many data fusion approaches in autonomous driving assume or require data to arrive in an in-sequence order. However, this can generally not be guaranteed for systems with multiple sensors having different transmission and processing latencies. Additionally, latency itself is performance and safety-critical. To ensure safe and efficient driving, latency must be kept minimal. State-of-the-art approaches artificially delay incoming data in an effort to overcome the sequence issue, which drastically increases latency by waiting. Detailed a-priori information about the sensor and transmission characteristics is required, yet, out-of-sequence data cannot be prevented, leading to data loss. In this work, making statistical assumptions reasonable for autonomous driving sensor configurations, we propose an optimal solution providing both guaranteed upper bounds on the data loss and the induced delay. Based on these assumptions, our approach maintains the optimal latency required to ensure in-sequence data ordering. Additionally, by estimating sensor and system characteristics online, our method adaptively adjusts according to the current situation. We demonstrate the superiority of our approach with an extensive evaluation based on both simulated and real-world data. For the latter, using data from an autonomous vehicle emphasizes the importance of our work for intelligent vehicles.

Keywords: Integration Methods for HD Maps and Onboard Sensors, Lidar-Based Environment Mapping
Abstract: This paper explores the use of automotive radars for 3D mapping in inland waterways (IW), a domain where radar-based mapping remains largely underexplored. A methodology is proposed for constructing 3D maps using a network of four automotive radars mounted on a water surface vessel (WSV), with data collected from diverse IW environments in Germany. The performance of the radar-based maps is evaluated in comparison to LiDAR-generated maps using the Iterative Closest Point (ICP), with metrics such as fitness score and inlier correspondence RMSE analyzed. The results demonstrate that radar maps effectively capture the overall structure of water channels and the infrastructure surrounding it, but exhibit reduced detail compared to LiDAR. Despite these limitations, radar's robustness to adverse environmental conditions, suitability for detecting moving targets and cost-effectiveness highlight its potential for enhancing autonomous navigation and transport operations in IW environments.

Keywords: Automotive Datasets, Sensor Fusion for Accurate Localization, Static and Dynamic Object Detection Algorithms
Abstract: Autonomous vehicles often rely on high-definition (HD) maps for navigation; however, these maps are not fre- quently updated and often lack semi-static information, such as temporary roadwork zones, which can significantly alter the road network. This limitation underscores the urgent need for an accurate global position of roadwork zones. However, the absence of publicly available datasets for evaluating roadwork zone detection and geo-localization models has hindered the development of reliable autonomous driving systems. To address this challenge, we propose the Roadwork Zone Detection and Geo-localization (RZDG) dataset, which includes both simulated and real-world data, providing multimodal sensor inputs along with comprehensive annotations. The dataset supports multiple perception tasks, including image semantic segmentation, 3D object detection, and object geo-localization. In addition, we introduce a tracker-based roadwork zone detection and geo-localization (RZDG) pipeline, an extension of AB3DMOT, for accurate object geo-localization in roadwork zones. We benchmark our approach on the RZDG dataset, demonstrating its effectiveness in detecting roadwork zones and transforming object positions from the local coordinate system to the global coordinate system. A prediction is considered a true positive (TP) if its estimated position falls within one meter of the ground truth. Our experimental results show that our approach achieves high accuracy on both real and simulated data. Specifically, we report: Precision: 0.565 (real) / 0.615 (simulated) Recall: 0.898 (real) / 0.809 (simulated) F1-score: 0.597 (real) / 0.665 (simulated). The RZDG dataset and code can be found at: https://github.com/chrisyan/RZDG.

Keywords: Advanced Multisensory Data Fusion Algorithms, Deep Learning Based Approaches, Semantic Segmentation Techniques
Abstract: Sensor fusion plays a crucial role in accurate and robust environment perception for autonomous driving. Recent works utilize Bird's-Eye-View (BEV) grid as a 3D representation, however, only using a partial set of multimodal signals. This paper introduces Simple-Multimodal-Attention-BEV (SMAB), a novel and simple approach to multimodal sensor fusion in BEV perception. We propose an attention mechanism called BEV feature aggregation that effectively enhances BEV feature representations. It integrates bilinearly interpolated semantic data from cameras with rasterized distance information from radars and/or lidars, and facilitates training with full-modality data or partial-modality data without modification of the method. In addition to the simplicity of the design, we demonstrate that using all sensor modalities improves segmentation accuracy. Meanwhile, SMAB is resilient to sporadic sensor signal loss, which enhances the robustness of the perception system. The proposed method outperforms state-of-the-art methods while simplifying the model.

Keywords: Radar Object Detection and Tracking, Deep Learning Based Approaches
Abstract: Object detection from radar data is a crucial task for understanding dynamic environments, particularly in adverse weather conditions where other sensors may struggle. However, predicting bounding boxes (BBoxes) from radar point clouds poses significant challenges due to the sparsity and high noise levels inherent in radar data. In this paper, we introduce RPT^2, a novel radar BBox prediction framework that integrates a radar point transformer with a point-based tracker to effectively leverage spatial and temporal information. The key innovation lies in using a lightweight tracker as a query for a transformer-based decoder to retrieve relevant spatial features from the encoded representation while leveraging temporal information, including 2D velocity beyond range velocity. This design enables the decoder to align the orientation and velocity of the object, resulting in more accurate predictions. Furthermore, the tracker identifies points of interest, significantly reducing the number of queries processed by the decoder and improving computational efficiency. We evaluated RPT^2 on the nuScenes dataset, where it delivers outstanding performance while maintaining efficiency, demonstrating its potential as a robust solution for radar-based object detection in real-world scenarios.

Keywords: Collision Avoidance Algorithms, Radar Object Detection and Tracking, Automotive Datasets
Abstract: Autonomous vehicles are increasingly utilized in diverse industries, relying heavily on perception systems to interpret their surroundings for decision-making and control. While Line-of-Sight perception technologies have advanced significantly, Non-Line-of-Sight (NLoS) perception remains a critical challenge. Current systems struggle to detect objects in NLoS scenarios, such as pedestrians or vehicles suddenly appearing from behind obstacles, leading to accidents, particularly at narrow T-junctions in urban environments. To address this, mmWave radar has emerged as a promising sensor for NLoS perception due to its ability to capture reflections and estimate the location of dynamic objects in occluded areas. However, previous researches are limited to controlled settings or single objects, with challenges like multipath reflections requiring precise spatial analysis for real-world use. In this paper, we propose a localization method for multi-dynamic NLoS pedestrians using ray tracing on 2D radar point clouds obtained from mmWave radar in outdoor environments. The approach involves inferring spatial information from static points, performing ray tracing for dynamic points, and applying noise filtering and clustering to estimate pedestrian locations. Validation on a custom-built test bed demonstrates the effectiveness of the method, establishing a foundation for advanced NLoS perception technologies in real-world driving.

Keywords: Safety Verification and Validation Techniques, Representation Learning for Driving Scenarios, Deep Learning Based Approaches
Abstract: The ability to operate safely in increasingly complex traffic scenarios is a fundamental requirement for ac{ADS}. Ensuring the safe release of ac{ADS} functions necessitates a precise understanding of the occurring traffic scenarios. To support this objective, this work introduces a pipeline for traffic scenario clustering and the analysis of scenario category completeness. The ac{CVQ-VAE} is employed for the clustering of highway traffic scenarios and utilized to create various catalogs with differing numbers of traffic scenario categories. Subsequently, the impact of the number of categories on the completeness considerations of the traffic scenario categories is analyzed. The results show an outperforming clustering performance compared to previous work. The trade-off between cluster quality and the amount of required data to maintain completeness is discussed based on the publicly available highD dataset.

Keywords: Dynamic Object Tracking, Advanced Passive Safety Systems, Perception Algorithms for Adverse Weather Conditions
Abstract: A novel approach to detect road surface anomalies by visual tracking of a preceding vehicle is proposed. The method is versatile, predicting any kind of road anomalies, such as potholes, bumps, debris, etc., unlike direct observation methods that rely on training visual detectors of those cases. The method operates in low visibility conditions or in dense traffic where the anomaly is occluded by a preceding vehicle. Anomalies are detected predictively, i.e., before a vehicle encounters them, which allows to pre-configure low-level vehicle systems (such as chasis) or to plan an avoidance maneuver in case of autonomous driving. A challenge is that the signal coming from camera-based tracking of a preceding vehicle may be weak and disturbed by camera ego-motion due to vibrations affecting the ego vehicle. Therefore, we propose an efficient method to compensate camera pitch rotation by an iterative robust estimator. Our experiments on both controlled-setup and normal traffic conditions show that road anomalies can be detected reliably at distance even in challenging cases where the ego-vehicle traverses imperfect road surfaces. The method is effective and performs in real time on standard consumer hardware.

Keywords: Cooperative Perception and Localization Techniques, Sensor Fusion for Accurate Localization, Advanced Multisensory Data Fusion Algorithms
Abstract: Roadside perception has become a critical component for connected and automated vehicles (CAVs), enhancing safety and offering a comprehensive view of the traffic environment that onboard detection systems alone cannot provide. By supplementing the limitations of onboard sensors, roadside perception systems improve the accuracy and reliability of detecting and localizing vehicles and pedestrians in challenging locations. Currently, a variety of cameras, including fisheye and regular cameras, are deployed along roadsides for surveillance purposes. These sensors have significant potential to improve vehicle and pedestrian detection.

This paper extends our previous work on single image sensor vehicle detection by developing a comprehensive multiple sensor fusion framework. We take advantage of the complementary strengths of multiple fisheye and regular cameras to enhance the accuracy and robustness of the perception system. The proposed system has been extensively tested in Mcity, a controlled urban testing environment, through numerous field tests. The results demonstrate the effectiveness of our approach, showcasing promising improvements in vehicle and pedestrian detection and tracking accuracy.

Keywords: Deep Learning Based Approaches, Semantic Segmentation Techniques, Static and Dynamic Object Detection Algorithms
Abstract: Autonomous vehicles drive millions of miles on the road each year. Under such circumstances, deployed machine learning models are prone to failure both in seemingly normal situations and in the presence of outliers. However, in the training phase, they are only evaluated on small validation and test sets, which are unable to reveal model failures due to their limited scenario coverage. While it is difficult and expensive to acquire large and representative labeled datasets for evaluation, large-scale unlabeled datasets are typically available. In this work, we introduce label-free model failure detection for lidar-based point cloud segmentation, taking advantage of the abundance of unlabeled data available. We leverage different data characteristics by training a supervised and self-supervised stream for the same task to detect failure modes. We perform a large-scale qualitative analysis and present LidarCODA, the first publicly available dataset with labeled anomalies in real-world lidar data, for an extensive quantitative analysis.

Keywords: Deep Learning Based Approaches, Sensor Fusion for Accurate Localization, Advanced Multisensory Data Fusion Algorithms
Abstract: Multi-sensor fusion plays a critical role in enhancing perception for autonomous driving, overcoming individual sensor limitations, and enabling comprehensive environmental understanding. This paper first formalizes multi-sensor fusion strategies into data-level, feature-level, and decision-level categories and then provides a systematic review of deep learning-based methods corresponding to each strategy. We present key multi-modal datasets and discuss their applicability in addressing real-world challenges, particularly in adverse weather conditions and complex urban environments. Additionally, we explore emerging trends, including the integration of Vision-Language Models (VLMs), Large Language Models (LLMs), and the role of sensor fusion in end-to-end autonomous driving, highlighting its potential to enhance system adaptability and robustness. Our work offers valuable insights into current methods and future directions for multi-sensor fusion in autonomous driving.

Keywords: Advanced Multisensory Data Fusion Algorithms
Abstract: Multi-source sensors, such as LiDAR, cameras, Inertial Measurement Units (IMU), and suspension displacement sensors, can describe road surface characteristics from different dimensions. Sensor fusion, which incorporates vehicle dynamic characteristics, is a key to improving the accuracy of road surface identification. Therefore, we propose a road surface identification method that combines segmented image features, statistically analyzed and extracted LiDAR features, and vehicle state features, with suspension displacement serving as a supervisory signal. The environmental and vehicle state features are fused using the Transfuser as the backbone, where the vehicle state features are extracted using Fourier Feature Mapping and a Multilayer Perceptron (MLP) network. The accuracy of road surface identification is further improved through the use of a state feature dropout module and a suspension displacement supervision module during the training process. Experimental results show that our method effectively combines multi-source sensor information and achieves higher accuracy in road surface identification compared to single-sensor-based methods and other multi-sensor fusion approaches. Furthermore, comparative road surface identification tests under constant and variable vehicle speeds, conducted under the same road conditions, demonstrate that our method is not affected by the type of vehicle motion, due to the supervision module based on the decoupled suspension displacement signal.

Keywords: Integration Methods for HD Maps and Onboard Sensors, Integrating HD Maps and Perception Data in Neural Networks
Abstract: High-definition (HD) maps are crucial for autonomous vehicles, but their creation and maintenance is very costly. This motivates the idea of online HD map construction. To provide a continuous large-scale stream of training data, existing HD maps can be used as labels for onboard sensor data from consumer vehicle fleets. However, compared to current, well curated HD map perception datasets, this fleet data suffers from localization errors, resulting in distorted map labels. We introduce three kinds of localization errors, Ramp, Gaussian, and Perlin noise, to examine their influence on generated map labels. We train a variant of MapTRv2, a state-of-the-art online HD map construction model, on the Argoverse 2 dataset with various levels of localization errors and assess the degradation of model performance. Since localization errors affect distant labels more severely, but are also less significant to driving performance, we introduce a distance-based map construction metric. Our experiments reveal that localization noise affects the model performance significantly. We demonstrate that errors in heading angle exert a more substantial influence than position errors, as angle errors result in a greater distortion of labels as distance to the vehicle increases. Furthermore, we can demonstrate that the model benefits from non-distorted ground truth (GT) data and that the performance decreases more than linearly with the increase in noisy data. Our study additionally provides a qualitative evaluation of the extent to which localization errors influence the construction of HD maps.

Keywords: Driver State Detection Algorithms, Deep Learning Based Approaches, Level 3 Driving Systems Architecture and Techniques
Abstract: Ensuring safe transition of control in automated vehicles requires an accurate and timely assessment of driver readiness. This paper introduces Driver-Net, a novel deep learning framework that fuses multi-camera inputs to estimate driver take-over readiness. Unlike conventional vision-based driver monitoring systems that focus on head pose or eye gaze, Driver-Net captures synchronised visual cues from the driver’s head, hands, and body posture through a triple-camera setup. The model integrates spatio-temporal data using a dual-path architecture, comprising a Context Block and a Feature Block, followed by a cross-modal fusion strategy to enhance prediction accuracy. Evaluated on a diverse dataset collected from the University of Leeds Driving Simulator, the proposed method achieves an accuracy of up to 95.8% in driver readiness classification. This performance significantly enhances existing approaches and highlights the importance of multimodal and multi-view fusion. As a real-time, non-intrusive solution, Driver-Net contributes meaningfully to the development of safer and more reliable automated vehicles and aligns with new regulatory mandates and upcoming safety standards.

Keywords: Deep Learning Based Approaches
Abstract: Autonomous driving systems require robust object detection in complex environments. Event cameras outperform RGB cameras under challenging lighting conditions, but face limitations due to the scarcity of available datasets and lack of specialized training. To narrow the gap between RGB- and event-based detection accuracy and avoid the high complexity of real-time RGB-event fusion, in this paper, we propose a knowledge distillation framework. Our approach uses both modalities during training but relies solely on sparse event data at inference and transfers knowledge from a robust RGB-based teacher model. We build on the success of DETR (DEtection TRansformer) and we leverage an event-aware masked knowledge distillation mechanism, to boost event-based detection accuracy. Experiments on the DSEC-DET dataset demonstrate that our method not only excels in challenging driving scenarios where RGB images are unreliable, but also surpasses the state-of-the-art in event-based object detection.

Keywords: Static and Dynamic Object Detection Algorithms, Deep Learning Based Approaches
Abstract: Modern sensor and image acquisition technologies have enabled Automated Vehicles (AVs) to leverage image data for enhanced on-board visual perception. To achieve this, AVs deploy Computer Vision (CV) deep learning tools for safe and accurate object detection. These systems typically rely on Image Signal Processing (ISP) pipelines to convert RAW sensor data into human-perceivable RGB images, as state-of-the-art deep learning models are optimised for such inputs. However, in safety-critical real-time applications such as AVs, CV systems must not only be reliable and accurate but also fast. In this work, we demonstrate that a lightweight ISP comprising simple processing steps can effectively operate on RAW image data while maintaining strong object detection performance across diverse scenes. Our experiments show that our proposed simple ISP maintains a balanced trade-off between accuracy and latency.

Keywords: Safety Verification and Validation Techniques, Level 3 Driving Systems Architecture and Techniques
Abstract: The accelerated development of autonomous vehicle software necessitates the implementation of an agile approach. As real-world testing is insufficient from an economical and technical point of view, it is necessary to provide credible virtual test systems, such as Hardware-in-the-Loop simulations, during this iterative development process. This paper demonstrates the application of the DevOps methodology to the development of HiL simulations to facilitate the delivery of incremental enhancements while ensuring the reliability through comprehensive testing in order to address the challenges posed by increasing demands for speed, flexibility, documentation, credibility assessment and traceability to the superior vehicle software development. The application of the process and its key technologies results in a DevOps platform as an end-to-end solution with high automation.

Keywords: Continuous Localization Solutions, Level 4-5 Autonomous Driving Systems Architecture, Safety Verification and Validation Techniques
Abstract: The application of radar sensors for motion estimation has recently been discussed in the research community. However, challenging environments such as garages and tunnels can lead to erroneous motion estimation results. Since the estimate serves as an input for trajectory control in automated driving functions, it can pose a safety hazard, e.g., if undesired acceleration is applied. This work presents a framework to address unsafe controller actions caused by motion estimation failures. A monitoring module continuously observes controller actions and feeds back critically evaluated measures to the motion estimation module. We propose three additional algorithms that adapt both the estimation module and its input data—namely, the filtered radar point cloud. First, a method that incorporates previous motion states into the point cloud filtering process. Second, a ridge regression algorithm that generates alternative estimates and compensates for erroneous conclusions arising from an unfavorably selected point set. Third, an adapted cluster selection approach that increases the number of true positive detection points. Experiments show that critical estimates can be identified in most cases, with a success rate of 97.7%. Additionally, we found that the estimation process becomes significantly more robust. In summary, this work emphasizes the importance of safe motion state measurement for the operation of automated vehicles. It introduces a method for determining the criticality of motion states and presents approaches to mitigate erroneous estimations.

Keywords: Self-Diagnostic Systems for Vehicle Safety
Abstract: Reliability is an important objective in vehicle design and should be assured during vehicle use, as vehicle components degrade over their lifetime. Among other components, suspension dampers deteriorate, which can affect ride comfort and vehicle safety.

Diagnosing the health state of suspension dampers in real-world scenarios is challenging because sensor signals are limited, and the measurable effects of degradation appear as complex signal patterns. To address these challenges, machine learning methods are increasingly used to develop diagnostic models. To reduce the need for high-quality and large-scale training data, this work proposes a physics-informed machine learning model for diagnosis. The model incorporates physics-based knowledge into a standard machine learning model through auxiliary losses. These losses are motivated by insights gained from a physics-based simulation model that describes the coupling between damper degradation and its effects on measurement signals.

The diagnostic model and its performance have been successfully tested within a real vehicle, with and without degraded dampers. We show that the incorporation of auxiliary losses greatly improves the diagnostic performance and requires less training data for a comparable performance.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Vulnerable Road User Protection Strategies, Collision Avoidance Algorithms
Abstract: Autonomous vehicles (AVs) must navigate dynamic urban environments where occlusions and perception limitations introduce significant uncertainties. This research builds upon and extends existing approaches in risk-aware motion planning and occlusion tracking to address these challenges. While prior studies have developed individual methods for occlusion tracking and risk assessment, a comprehensive method integrating these techniques has not been fully explored. We, therefore, enhance a phantom agent-centric model by incorporating sequential reasoning to track occluded areas and predict potential hazards. Our model enables realistic scenario representation and context-aware risk evaluation by modeling diverse phantom agents, each with distinct behavior profiles. Simulations demonstrate that the proposed approach improves situational awareness and balances proactive safety with efficient traffic flow. While these results underline the potential of our method, validation in real-world scenarios is necessary to confirm its feasibility and generalizability. By utilizing and advancing established methodologies, this work contributes to safer and more reliable AV planning in complex urban environments. To support further research, our method is available as open-source software at https://github.com/TUM-AVS/OcclusionAwareMotionPlanning.

Keywords: Safety Verification and Validation Techniques, Collision Avoidance Algorithms, Adaptive Vehicle Control Techniques
Abstract: Cut-in maneuvers in high-speed traffic are critical challenges that could cause abrupt braking and collisions, demanding safe and efficient lane change strategies. We propose a Dynamic Bayesian Network (DBN) framework to integrate lateral evidence with safety assessment models to predict lane changes and to ensure safe cut-in maneuvers effectively. Our proposed framework consists of three key probabilistic hypotheses (lateral evidence, lateral safety, and longitudinal safety) that handle the decision-making process through dynamic data processing and assessments of vehicle positions, lateral velocities, relative distance, and Time-To-Collision (TTC) computations. The DBN model's performance compared with other conventional rule-based approaches demonstrates superior performance in crash reduction, especially in critical high-speed scenarios, while maintaining a competitive performance in low-speed scenarios. This paves the way for robust, scalable, and efficient safety validation in automated driving systems.

Keywords: Foundation Models Based Approaches, Vulnerable Road User Protection Strategies
Abstract: Prediction of pedestrian crossing intention is a critical function in autonomous vehicles. Conventional vision-based methods of crossing intention prediction often struggle with generalizability, context understanding, and causal reasoning. This study explores the potential of vision-language foundation models (VLFMs) for predicting pedestrian crossing intentions by integrating multimodal data through hierarchical prompt templates. The methodology incorporates contextual information, including visual frames, physical cues observations, and ego-vehicle dynamics, into systematically refined prompts to guide VLFMs effectively in intention prediction. Experiments were conducted on three common datasets—JAAD, PIE, and FU-PIP. Results demonstrate that incorporating vehicle speed, its variations over time, and time-conscious prompts significantly enhances the prediction accuracy up to 19.8%. Additionally, optimised prompts generated via an automatic prompt engineering framework yielded 12.5% further accuracy gains. These findings highlight the superior performance of VLFMs compared to conventional vision-based models, offering enhanced generalisation and contextual understanding for autonomous driving applications.

Keywords: Safety Verification and Validation Techniques
Abstract: Scenario-Based Testing (SBT) is the most widely used method for safety assurance of Automated Driving System equipped Vehicles (ADS-Vs). To comply with ISO 21448, a key standard for ADS-V safety assurance, SBT can be formulated as an optimization problem to identify unknown hazardous scenarios within the ADS-V’s Operational Design Domain. To identify hazardous scenarios through optimization, an appropriate cost function for scenario hazardous-ness is needed that satisfies specific requirements. Existing criticality measures in the literature do not satisfy these requirements. We interpret scenario hazardous-ness as the proximity to a collision for the ADS-V. We propose a Criticality Measure (CM) using the set of alternative futures of a Reference Vehicle (RV) following the trajectory of the ADS-V. The CM indicates the proximity to a collision, both in time and space. The proposed CM provides a posteriori knowledge about the criticality of a scenario already generated by the optimization where the future evolution of every scene is available. To showcase the developed CM, multiple scenarios in two road networks, an intersection, and a highway segment are evaluated for criticality. We briefly discuss how to incorporate the severity of collisions/potential future collisions in the CM. The possible extensions and limitations of the proposed CM are discussed.

Keywords: Safety Verification and Validation Techniques, Vulnerable Road User Protection Strategies, Real-World Testing Methodologies for Safety Systems
Abstract: Motorcycles, bicycles, and other single-track vehicles are popular but dangerous methods of transportation. While some are piloted by only a single rider, many powered two-wheelers are ridden with a passenger, who may also significantly influence the vehicle's dynamics. Because simulations are a critical component of vehicle safety research, this paper asks whether a simulated, active "virtual passenger" has stabilizing or destabilizing effects on a rider-vehicle-passenger system. This virtual passenger exerts its control effort by moving an inverted pendulum to simulate the motion of a human passenger's torso without explicit knowledge of rider inputs. A battery of simulations in the nonlinear, multi-body Webots robot simulator show that the passenger's control efforts have mixed effects on both rider effort and vehicle stability over abrupt transitions in pavement height. This indicates that the inclusion of passenger motion may be critical when vetting the safety of roadway designs and/or emerging motorcycle technologies like Advanced Rider Assist Systems.

Keywords: Self-Diagnostic Systems for Vehicle Safety, Collision Avoidance Algorithms, Automotive Datasets
Abstract: Highly Automated Vehicles (HAVs) present complex challenges for regulators and industry stakeholders due to their susceptibility to diverse faults and the risks posed by their black-box nature during real-world operations. Regulators play a critical role in ensuring the safety and reliability of HAV systems, which necessitates robust fault detection and oversight mechanisms. A critical regulatory task is to define a minimal yet sufficient set of data (i.e., a Minimal Grey Box) that regulators should collect to trace the causal chain of any collision or near-collision events. However, conventional approaches, such as Event Data Recorders (EDRs) and Data Storage Systems for Automated Driving (DSSAD), fall short in detecting HAV-specific faults, mainly due to the vast volume of data generated by these systems. Furthermore, existing fault detection methods are often inadequate in identifying software-related issues, largely because of the opaque, black-box characteristics of HAVs. To address the challenges posed by the sheer size of the datasets and the opacity of the systems under oversight, this study introduces a fault detection method that extracts a Minimal Grey Box (MGB). The proposed approach first learns the causal structure of HAV faults using Bayesian Networks, which are subsequently mapped onto a fault tree. The fault detection results demonstrate the superior performance of the detection model based on the concise MGB, achieving high F1-scores (>95%) across 5 Apollo sub-modules. This approach highlights a promising solution to the efficient oversight and accurate fault detection in HAVs.

Keywords: Safety Verification and Validation Techniques, Trust and Acceptance of Autonomous Technologies
Abstract: The opacity of decision-making in autonomous vehicles, rooted in the use of accurate yet complex AI models, has created barriers to their societal trust and regulatory acceptance, raising the need for explainability. We propose a post-hoc, model-agnostic solution to provide teleological explanations of vehicle behaviour in urban environments. Based on an existing explainability method called Intention-aware Policy Graphs, our approach enables the extraction of interpretable and reliable explanations of vehicle behaviour in the nuScenes dataset from global and local perspectives. We demonstrate how these explanations can be used to verify whether the vehicle operates within acceptable legal boundaries and to reveal potential vulnerabilities in autonomous driving datasets and models.

Keywords: End-to-End Neural Network Architectures and Techniques, Safety Verification and Validation Techniques, Deep Learning Based Approaches
Abstract: Autonomous robots and self-driving cars rely on deep neural networks to perceive the environment and plan their actions. In particular, Convolutional Neural Networks (CNNs) became the defacto standard approach in computer vision. However, it was shown that they are also exposed to adversarial attacks: once small perturbations are added to the original image, a CNN will misinterpret the data. We introduce CyberDet, which is a simple, yet effective technique, based on boldsymbol{k}th order differences computed from the input image. The operator allows a CNN--based binary classifier to discriminate between attacked and genuine images. The method ia agnostic of the attack method, and of the data used for train or inference, and it is used online to guard the data acquisition streams of the perception system the autonomous robot RovisLab AMTU. We have experimentally shown that CyberDet is effectively discloses the Fast Gradient Sign attack, two variants of the Projected Gradient Descent, and Additive Uniform Noise attacks. The experiments have been performed on the benchmarking datasets CIFAR-10, CIFAR-100 and the Tiny ImageNet. The CyberDet code is publicly available at href{https://github.com/lmsasu/adversarial_attack}{https://github.com/lmsasu/adversarial_attack}.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Collision Avoidance Algorithms, Vulnerable Road User Protection Strategies
Abstract: Risk assessment and management in urban scenarios are difficult for automated vehicles due to perception uncertainties and the latent stochastic and high-dynamic behaviors of other agents e.g., Personal Light Electric Vehicles (PLEVs). Although the Predictive Inter-Distance Profile (PIDP) provides a continuous assessment of the risk between multiple agents, it fails when there are significant uncertainties in the estimated states of the agents. In this paper, we propose a Stochastic PIDP (sPIDP) to handle the uncertainties in the motion of the agents. sPIDP projects the uncertainties to the inter-distance between the agents. Furthermore, an Uncertainty-aware MPC is proposed to perform risk management. Statistical results considering multiple traffic scenarios show that our method is efficient for safe navigation.

Keywords: Safety Verification and Validation Techniques, Collision Avoidance Algorithms, Trust and Acceptance of Autonomous Technologies
Abstract: Autonomous Vehicles (AVs) must be thoroughly tested to meet high safety requirements. Scenario-based testing using simulation is a common approach for validating them. Usually, scenarios test only one ego vehicle against road users with pre-defined behaviors called Non-Playable Characters (NPC). Such scenarios ensure reproducibility but are not always relevant and realistic, as they do not capture interactions between (e.g., non-cooperative) AVs. Consequently, they are unsuitable for testing safety-critical emerging behaviors like those happening in the real world. To tackle this problem, we propose TIAV, an approach for generating interactive critical scenarios that allows developers to study how AVs influence each other. Experiments on the reference CommonRoad simulation framework show that TIAV can identify scenarios leading to collisions and disengagements and trigger significantly more failures than a random baseline. Thanks to its ability to expose unsafe AV interactions, TIAV allows developers to validate AVs' functional correctness and check the effects of AVs' simultaneous deployment. TIAV is available as open-source software: https://github.com/parcaini/TIAV

Keywords: Advanced Passive Safety Systems, Predictive Trajectory Models and Motion Forecasting
Abstract: This paper presents an integrated approach for en- hancing crash probability prediction in critical driving situations by combining synthetic data generation, machine learning tech- niques, and naturalistic driving studies (NDS). The study bridges the gap between active and passive vehicle safety and involves two main workstreams: Trajectory Prediction and Trajectory Weight- ing. The first workstream focuses on simulating driving scenarios to predict vehicle trajectories and assess collision risks using a machine learning model, specifically Gradient Boosting. The second workstream utilizes data from naturalistic driving studies to compute realistic weightings for different trajectories based on historical driving data. By integrating these workstreams, the approach provides a detailed and accurate estimation of crash probabilities, considering context-specific driving behaviors. The results indicate that the individual weighting model offers a more realistic and practical estimation of crash probabilities compared to a uniform weighting scheme. The proposed algorithm’s run- time averages 9.2 ms for 185 trajectories, making it suitable for real-time applications. This enhanced crash probability calcula- tion can be used to preemptively activate passive safety systems when the collision risk is high or an accident is unavoidable.

Keywords: Reinforcement Learning for Planning, Collision Avoidance Algorithms, Safety Verification and Validation Techniques
Abstract: Reinforcement learning (RL), known for its self-evolution capability, offers a promising approach to training high-level autonomous driving systems. However, handling constraints remains a significant challenge for existing RL algorithms, particularly in real-world applications. In this paper, we propose a new safety-oriented training technique called harmonic policy iteration (HPI). At each RL iteration, it first calculates two policy gradients associated with efficient driving and safety constraints, respectively. Then, a harmonic gradient is derived for policy updating, minimizing conflicts between the two gradients and consequently enabling a more balanced and stable training process. Furthermore, we adopt the state-of-the-art DSAC algorithm as the backbone and integrate it with our HPI to develop a new safe RL algorithm, DSAC-H. Extensive simulations in multi-lane scenarios demonstrate that DSAC-H achieves efficient driving performance with near-zero safety constraint violations.

Keywords: Decision Making, Motion Planning Algorithms for Autonomous Vehicles, Safety Verification and Validation Techniques
Abstract: In the trajectory planning of automated driving, data-driven statistical artificial intelligence (AI) methods are increasingly established for predicting the emergent behavior of other road users. While these methods achieve exceptional performance in defined datasets, they usually rely on the independent and identically distributed (i.i.d.) assumption and thus tend to be vulnerable to distribution shifts that occur in the real world. In addition, these methods lack explainability due to their black box nature, which poses further challenges in terms of the approval process and social trustworthiness. Therefore, in order to use the capabilities of data-driven statistical AI methods in a reliable and trustworthy manner, the concept of TrustMHE is introduced and investigated in this paper. TrustMHE represents a complementary approach, independent of the underlying AI systems, that combines AI-driven out-of-distribution detection with control-driven moving horizon estimation (MHE) to enable not only detection and monitoring, but also intervention. The effectiveness of the proposed TrustMHE is evaluated and proven in three simulation scenarios.

Keywords: Motion Planning Algorithms for Autonomous Vehicles, Real-World Testing Methodologies for Safety Systems, Level 3 Driving Systems Architecture and Techniques
Abstract: This paper presents a methodology that is automatically performed by an Automated Driving System (ADS) to bring the vehicle into a Minimum Risk Condition (MRC), in case the driver does not take the driving task after a take-over demand or after a severe system failure. This procedure is commonly known as Minimum Risk Maneuver (MRM). The interest behind MRM is to ensure vehicle's safety within an automated mode. The current work proposes an MRM methodology for automated vehicles equipped with level 3-5 ADS. It is implemented as a state machine specifying high-level maneuver decisions that are used in the vehicle motion control. The system is simulated in a CarMaker environment, demonstrating its feasibility in ensuring the safety of the vehicle in several failure scenarios. In addition, experimental results on an automated vehicle, robotized by Valeo, are shown to approve the proposed system.

Keywords: User-Centric Intelligent Vehicle Technologies
Abstract: Tire wear prediction plays a critical role in ensuring vehicle safety, optimizing tire lifespan, and reducing environmental impact. Traditional methods often rely on specialized hardware sensors or monitoring devices. While effective, these approaches are costly and vendor-locked. This research proposes a purely data-driven approach, leveraging existing sensors of connected vehicles and their historical readouts. By utilizing a machine learning model, the framework identifies patterns and correlations in highly multidimensional data space to estimate tire wear progression and yield end-of-life alerts while requiring only a single initial tread depth measurement. In addition, adopting explainable AI techniques makes it possible to score vehicle users with respect to how adverse to tire longevity their behaviors are and thus open prospects for tire health coaching. The proposed methodology is being integrated into existing connected vehicle platforms to enable proactive maintenance.

Keywords: Multi-Agent Coordination Strategies, Driver State Detection Algorithms, Multi-Objective Planning Approaches
Abstract: A lack of understanding of interactions and the inability to effectively resolve conflicts continue to impede the progress of Connected Autonomous Vehicles (CAVs) in their interactions with Human-Driven Vehicles (HDVs). To address this challenge, we propose the Recognize then Resolve (RtR) framework. First, a Bilateral Intention Progression Graph (BIPG) is constructed based on CAV-HDV interaction data to model the evolution of interactions and identify potential HDV intentions. Three typical interaction breakdown scenarios are then categorized, and key moments are defined for triggering cooperative conflict resolution. On this basis, a constrained Monte Carlo Tree Search (MCTS) algorithm is introduced to determine the optimal passage order while accommodating HDV intentions. Experimental results demonstrate that the proposed RtR framework outperforms other cooperative approaches in terms of safety and efficiency across various penetration rates, achieving results close to consistent cooperation while significantly reducing computational resources. Our code and data are available at: https://github.com/FanGShiYuu/RtR-Recognize-then-Resolve/

Keywords: Human Factors Analysis in Vehicle Design, Vulnerable Road User Protection Strategies, Multi-Agent Coordination Strategies
Abstract: As automated vehicles (AVs) become integral to urban traffic, understanding multi-pedestrian interaction dynamics is crucial for ensuring safety and efficiency. This study examines how pedestrians’ start positions, mutual influence, and AV communication strategies impact crossing behavior and the formation of informal shared spaces. A Virtual Reality (VR) experiment with 30 participants (15 pairs) was conducted, where two participants interacted simultaneously in the same virtual environment alongside a simulated pedestrian and an AV. Pedestrians were assigned different start positions (roadside vs. distant), and the AV employed implicit (rolling stop maneuver) and explicit (eHMI) communication strategies. Results show that distant pedestrians had significantly lower certainty and were less likely to cross in front of the AV than roadside pedestrians. However, over half engaged in joint crossing, exhibiting a significantly shorter crossing time, reinforcing the emergence of informal shared spaces. The rolling stop maneuver effectively discouraged further crossings, enhancing traffic flow but causing some confusion. These findings highlight the need to consider multi-pedestrian interaction dynamics in AV design. Further research should refine AV strategies for navigating informal shared spaces and optimizing pedestrian-AV interactions in complex urban environments.

Keywords: Motion Forecasting, Cooperative Perception and Localization Techniques, Predictive Trajectory Models and Motion Forecasting
Abstract: Traditional perception systems in automated driving have different constraints that do not allow for complete environmental awareness. Cooperative Perception (CP) addresses these limitations by sharing information between vehicles and/or infrastructure through Vehicle-to-Everything (V2X) communications. This collaborative approach mitigates occlusions and extends sensor coverage, proving essential for Cooperative Driving Automation (CDA). However, there are remaining challenges about its application in online real-world scenarios, such as CP information transmission and communication degradations. In this cooperative context, Motion Prediction (MP) proves to be crucial, since it provides a scene representation of all the agents with their positions, velocities and future trajectories. Thus, shared information between agents can improve each agent understanding of the overall scene.

This paper introduces LFF-V2V, A Late Fusion Cooperative Framework in V2V Scenarios. It combines two state-of-the-art late fusion methods, Non-Maximum Suppression (NMS) and Weighted Box Fusion (WBF), with a map-less Hierarchical Vector Transformer (HiVT) motion prediction model. We have conducted an extensive evaluation in two environments: CARLA simulator and the real-world V2X-Real dataset, analyzing different communication strategies. Our results demonstrate the effectiveness of CP in improving object detection and motion prediction, even with degraded communications.

Keywords: Multi-Agent Coordination Strategies, Cooperative Planning Strategies in Vehicle Networks, Real-Time Control Strategies
Abstract: The rise of autonomous and electric vehicles has revolutionized the logistics sector, with autonomous systems poised to address critical challenges such as charging station placement, task allocation, and inter-agent collaboration. This paper introduces CoRA (Connected Robot Automation), a research platform designed to tackle logistics problems across strategic, tactical, and operational levels, which haven’t been addressed by other research so far.

CoRA integrates design, planning and operational problems via cutting-edge technologies, such as distributed Model Predictive Control (DMPC). Its hardware and software stack is built upon a foundation of equally complex and research-worthy systems, including perception, localization, and planning. The system operates within a dual framework, combining physical experiments and virtual simulations. This paper provides a comprehensive overview on the research platform to highlight key optimization problems like the Facility Location Problem (FLP) and Vehicle Routing Problem (VRP), contributing to scalable, fault-tolerant, and efficient logistics solutions for autonomous systems.

Keywords: Multi-Agent Coordination Strategies, V2X Communication Protocols and Standards, Real-Time Control Strategies
Abstract: In this study, we propose a novel negotiation-based cooperative maneuvering strategy to assist connected automated vehicles (CAVs) in resolving conflicts under different traffic scenarios. We introduce conflict charts to determine when negotiation is necessary, along with a request and response protocol to facilitate traffic conflict resolution. Additionally, we propose an easy-to-implement controller that allows CAVs to resolve conflicts based on the agreement reached through negotiation. Simulation results using real vehicle data are used to demonstrate that the proposed negotiation protocol helps to ensure safety while improving time efficiency compared to cooperations that rely on other communication strategies.

Keywords: Cooperative Planning Strategies in Vehicle Networks, Real-Time Control Strategies, Motion Planning Algorithms for Autonomous Vehicles
Abstract: In contemporary society, autonomous driving systems face enormous challenges from various aspects, including the environment, traffic participants, and communication. The primary task of achieving vehicle autonomy is to ensure that Connected and Automated Vehicles (CAVs) can cooperate safely and efficiently with Human-Driven Vehicles (HDVs). Previous research mostly focused on strategies for purely CAV scenarios or treated HDVs as random factors in traffic, with relatively few studies considering the synchronous inducement and control of both types of vehicles within a unified framework. This research focuses on mixed traffic flow that includes both HDVs and CAVs, specifically addressing the merging problem at highway on-ramp entrances. We propose an inducement control method and planning framework that targets both vehicle types simultaneously. For upstream HDVs, speed inducement is implemented based on time series predictions, along with behavior modeling including both internal and external uncertainties using a Gaussian mixture model. For ramp vehicles, merging decisions are made while considering the uncertainties of upstream vehicles. Additionally, based on the speed inducement strategy and decision module, a cooperative uncertainty-aware planning model is constructed to achieve motion planning. The research demonstrates that the proposed speed inducement strategy can enhance traffic safety and driver experience, and the cooperative control framework designed for both types of vehicles exhibits real-time functionality and performs excellently in terms of computation speed and planning success rates.

Keywords: Reinforcement Learning for Planning, Semantic Segmentation Techniques, Real-Time Control Strategies
Abstract: Deep Reinforcement Learning (DRL) for traffic light control adaptively adjusts signals based on real-time traffic conditions to alleviate urban congestion. However, transmitting image data from intersection cameras results in high communication overhead and latency in practical deployments. Traditional DRL methods lack interpretability when selecting some common traffic features (e.g., traffic densities, speed) as state input. Inspired by 6G semantic communication that transmits semantic information rather than raw image data, this paper proposes xTL, an eXplainable AI (XAI)-guided DRL development approach to achieve semantic state design and address communication overhead. Unlike traditional DRL that heavily depends on empirical tuning, xTL allows human experts to efficiently guide and interpret model design by using XAI-generated explanations to distill lightweight semantic traffic features from image data. Utilizing SHapley Additive exPlanations (SHAP)-generated saliency maps, we identify a new critical feature: the location of the last vehicle in the first platoon on each incoming road, which can interpret intersection traffic dynamics and effectively improve traffic. Experiments on two urban intersections in SUMO demonstrate that xTL slashes communication costs by over 90% and shortens DRL training duration by about 21%, without compromising traffic control effectiveness.

Keywords: Adaptive Vehicle Control Techniques, Multi-Agent Coordination Strategies
Abstract: Switching communication topologies can cause instability in vehicle platoons, as vehicle information may be lost during the dynamic switching process. This highlights the need to design a controller capable of maintaining the stability of vehicle platoons under dynamically changing topologies. However, capturing the dynamic characteristics of switching topologies and obtaining complete vehicle information for controller design while ensuring stability remains a significant challenge. In this study, we propose an observer-based distributed model predictive control (DMPC) method for vehicle platoons under directed Markovian switching topologies. Considering the stochastic nature of the switching topologies, we model the directed switching communication topologies using a continuous-time Markov chain. To obtain the leader vehicle's information for controller design, we develop a fully distributed adaptive observer that can quickly adapt to the randomly switching topologies, ensuring that the observed information is not affected by the dynamic topology switches. Additionally, a sufficient condition is derived to guarantee the mean-square stability of the observer. Furthermore, we construct the DMPC terminal update law based on the observer and formulate a string stability constraint based on the observed information. Numerical simulations demonstrate that our method can reduce tracking errors while ensuring string stability

Keywords: Multi-Agent Coordination Strategies, Decision Making, Safety Verification and Validation Techniques
Abstract: The safe deployment of autonomous vehicles (AVs) into real-world traffic requires robust interaction with human drivers exhibiting heterogeneous behavioral tendencies, spanning from rational cooperation to adversarial aggression. Existing simulation frameworks often lack the capacity to systematically model such behavioral diversity, limiting their applicability for rigorous AV evaluation. To address this challenge, we propose a multi-agent reinforcement learning framework that generates dynamically controllable traffic through Tendency-Guided Policy Optimization (TGPO). Central to TGPO is the Adversary-Rationality-Tendency (ART), a continuous hyperparameter that enables fine-grained control over the spectrum of driving behaviors by fusing separately learned adversarial and rational value functions. Furthermore, we design an ART-guided policy network incorporating multi-head mechanisms to resolve high-dimensional multi-agent observations, adaptively prioritizing context features aligned with assigned driving tendencies. Extensive experiments across urban and highway scenarios demonstrate that TGPO generates traffic with enhanced behavioral controllability and diversity, which provides a scalable solution for simulating realistic interactions with various driving tendencies, thereby facilitating the development of AV systems capable of handling complex real-world corner cases.

Keywords: Multi-Agent Coordination Strategies, Decision Making, Multi-Objective Planning Approaches
Abstract: The increased complexity of vehicle testing can be attributed to the rapid development of technical advancements within the automotive industry, thereby prolonging the time to market of a product. The process of allocating and coordinating vehicle tests at proving grounds (PGs) is a complex and time-consuming task. Currently, this process is still performed manually, which is inefficient. This study proposes a methodology for assigning scenarios to designated sites, taking into account travel aspects between locations and fulfilling participant requirements. The allocation procedure is formulated as an Open Job Shop Scheduling problem with temporal synchronisation and skill matching, and is solved by the Constraint Programming tool Google OR-Tools. The efficacy of the approach is demonstrated by its ability to generate a close-to-optimal schedule to fulfil customer requests. Case studies demonstrate that a combination of two distinct objectives are essential to meet the demands of compactness and time efficiency. The findings of this study provide a solid foundation for enhancing automation at a PG, thereby improving efficiency and optimising testing

Keywords: Cooperative Planning Strategies in Vehicle Networks, Adaptive Vehicle Control Techniques, Real-Time Control Strategies
Abstract: We explore the advantages of using deep learning-based intent sharing for platooning of connected automated vehicles (CAVs). Unlike traditional platooning algorithms that rely on status-sharing — exchanging current position, speed, and acceleration—our approach focuses on intent-sharing, where each CAV shares its predicted future trajectory with other CAVs. We introduce a deep learning model to generate the intent for each CAV and integrate this intent into a receding horizon control framework. Our approach aims to minimize spacing errors with the leading vehicle while improving energy efficiency and maintaining string stability. Through microscopic simulations using real-world highway data, we demonstrate that our intent messages significantly enhance energy efficiency compared to conventional status and intent-based platooning algorithms. Moreover, we show that this improvement is particularly pronounced when reducing the frequency of intent message transmission.

Keywords: Reinforcement Learning for Planning, Multi-Agent Coordination Strategies
Abstract: Supervised learning has demonstrated to be an effective strategy in training neural networks for vehicle navigation. However, it requires labeled data, which may not be available when a large number of vehicles need to be controlled simultaneously. In contrast, Deep Reinforcement Learning (DRL) circumvents the necessity for ground truth labels through environmental exploration. However, most concurrent DRL approaches either tend to operate in the discrete action/state space or do not consider the vehicle kinematics. In this paper, we use DRL to control multiple vehicles while also considering their kinematics. The task is for all the vehicles to reach their desired destination/target while avoiding collisions with each other or static obstacles in an unconstrained environment. For this, we propose a decentralized Proximal Policy Optimization (PPO) based DRL agent that independently provides control commands to each vehicle. The agent is based on two separate PPO models. The first is used to drive each vehicle to the proximity of its target. Once within the target's proximity, the second model is used to park that vehicle at the correct position and orientation. The decentralized nature of the algorithm allows each agent to rely only on information about its current state and target, along with details regarding the closest obstacle/agent. By scaling this approach to all vehicles, simultaneous navigation of multiple vehicles can be achieved. Experimental results show a collective strategy that allows consistent results across a wide range of scenarios while scaling to situations with up to 20 vehicles and 12 stationary obstacles.

Keywords: Cooperative Perception and Localization Techniques, Radar Object Detection and Tracking, Deep Learning Based Approaches
Abstract: In V2X collaborative perception, the domain gaps between heterogeneous nodes pose a significant challenge for effective information fusion. Pose errors arising from latency and GPS localization noise further exacerbate the issue by leading to feature misalignment. To overcome these challenges, we propose V2X-DGPE, a high-accuracy and robust V2X feature-level collaborative perception framework. V2X-DGPE employs a Knowledge Distillation Framework and a Feature Compensation Module to learn domain-invariant representations, effectively reducing the feature distribution gap between vehicles and roadside infrastructure. Historical information is utilized to provide the model with a more comprehensive understanding of the current scene. Furthermore, a Collaborative Fusion Module leverages a heterogeneous self-attention mechanism to extract and integrate heterogeneous representations from vehicles and infrastructure. To address pose errors, V2X-DGPE introduces a deformable attention mechanism, enabling the model to capture feature misalignments by dynamically offsetting sampling points. Extensive experiments on the real-world DAIR-V2X dataset demonstrate that the proposed method outperforms existing approaches, achieving state-of-the-art detection performance. The code is available at https://github.com/wangsch10/V2X-DGPE.

Keywords: Cooperative Perception and Localization Techniques, Static and Dynamic Object Detection Algorithms, Advanced Multisensory Data Fusion Algorithms

Keywords: Crowdsourced Localization and Mapping, Cooperative Perception and Localization Techniques, Application of Neural Fields in Autonomous Driving
Abstract: Autonomous driving faces safety challenges due to a lack of global perspective and the semantic information of vectorized high-definition (HD) maps. Information from roadside cameras can greatly expand the map perception range through vehicle-to-infrastructure (V2I) communications. However, there is still no dataset from the real world available for the study on map vectorization onboard under the scenario of vehicle-infrastructure cooperation. To prosper the research on online HD mapping for Vehicle-Infrastructure Cooperative Autonomous Driving (VICAD), we release a real-world dataset, which contains collaborative camera frames from both vehicles and roadside infrastructures, and provides human annotations of HD map elements. We also present an end-to-end neural framework (i.e., V2I-HD) leveraging vision-centric V2I systems to construct vectorized maps. To reduce computation costs and further deploy V2I-HD on autonomous vehicles, we introduce a directionally decoupled self-attention mechanism to V2I-HD. Extensive experiments show that V2I-HD has superior performance in real-time inference speed, as tested by our real-world dataset. Abundant qualitative results also demonstrate stable and robust map construction quality with low cost in complex and various driving scenes. As a benchmark, both source codes and the dataset have been released at OneDrive for the purpose of further study.

Keywords: Safety Verification and Validation Techniques, Real-World Testing Methodologies for Safety Systems, Behavior Assessment Using Cooperative Data
Abstract: Forward collision warning (FCW) systems are available in many new vehicles and are becoming increasingly popular. It is important to clearly understand their effectiveness in reducing collision risks, as well as their potential negative impacts on driving safety and volatility. This research aims to answer this question using real-world connected vehicles (CV) data collected under naturalistic settings. We extract 2,332 FCW events from the New York City CV Pilot Deployment dataset, including 1,326 events in the treatment group (warning issued) and 1,006 in the control group (no warning issued). From these FCW events, eleven volatility and nine safety variables are proposed and calculated, considering the interactions between following and leading vehicles. These variables are further filtered using an ensemble variable elimination and selection method based on Variance Inflation Factors, Person correlation, and Akaike Information Criterion. Three binary logit models are developed for modeling volatility, safety, and both volatility and safety, respectively. These models are compared based on their log-likelihood values and the margin effects (MEs) of variables. The results show that FCW systems significantly enhance driving safety by reducing the time vehicles spend in high-risk conditions (ME: -4.59%) and the period of harsh braking to avoid collisions (ME: -5.27%) through issuing audible warnings. Despite increases in driving volatility, the overall benefits of FCW on safety outweigh the risks, with a benefit-to-risk ratio of 48.72%, resulting in a favorable net effect on driving dynamics from a statistical perspective.

Keywords: Over The Air (OTA) Update Security Protocols, UAV Swarm Intelligence, Cooperation between UAVs and Ground Vehicles
Abstract: Over-the-Air (OTA) updates present significant challenges for autonomous vehicle fleets and UAV swarms, particularly in maintaining system integrity while minimizing operational disruption. This paper proposes a hierarchical update management framework that addresses the key challenges of coordinated software deployment across heterogeneous autonomous systems. We implement a distributed consensus protocol based on a modified Raft algorithm, combined with a lightweight validation mechanism that operates on edge computing nodes. Our approach introduces a two-phase update strategy: first, a leader-follower architecture validates updates on a subset of vehicles under controlled conditions, followed by gradual fleet-wide deployment with automated rollback capabilities. We evaluate our framework on a testbed of 12 autonomous vehicles and 24 UAVs across urban and suburban virtualized environments, comparing against three baseline update strategies. Results demonstrate a 47% reduction in update completion time compared to traditional sequential deployment, while maintaining a 99.9% success rate for update validation. Our system achieves these improvements while requiring only 23% of the bandwidth utilized by existing fleet-wide update solutions. We also present a comprehensive security analysis, identifying potential attack vectors in V2X communication during updates and implementing countermeasures through PKI-based authentication and real-time anomaly detection.