Keywords: V2X Communication, Security, Privacy
Abstract: Vehicular Ad Hoc Network (VANET) is a pervasive network, where vehicles communicate with nearby vehicles and infrastructure nodes, such as Road-side unit (RSU). Information sharing among vehicles is an essential component of an intelligent transportation system (ITS), but security and privacy concerns must be taken into consideration. Security of the network can be improved by granting access only to authenticated vehicles and restricting or revoking access for vehicles involved in misbehavior. In this paper, we present a novel blockchain based approach to authenticate vehicles and notify other vehicles about any unauthorized messages in real time. This helps protect other vehicles in the network from making critical decisions based on false or inaccurate information. In the proposed architecture, vehicles communicate with each other using pseudonyms or pseudo IDs and the Blockchain is used to securely maintain the real identity of all vehicles, which can be linked to the pseudo IDs if needed. The goal is to protect privacy or individual vehicles, while still ensuring accountability in case of misbehavior. The performance of the proposed approach is evaluated for different vehicle and attacker densities, and results demonstrate it has lower authentication delay and communication overhead compared to existing approaches.

Keywords: Security, Privacy, Intelligent Ground, Air and Space Vehicles
Abstract: Automatic Dependent Surveillance Broadcast (ADS-B) is one of the most prominent protocols in Air Traffic Control (ATC). Its key advantages derive from using GPS as a location provider, resulting in better location accuracy while offering substantially lower deployment and operational costs when compared to traditional radar technologies. ADS-B not only can enhance radar coverage but also is a standalone solution to areas without radar coverage. Despite these advantages, a wider adoption of the technology is limited due to security vulnerabilities, which are rooted in the protocol's open broadcast of clear-text messages. In spite of the seriousness of such concerns, very few researchers attempted to propose viable approaches to address such vulnerabilities. Even fewer attempted to develop ways of automatically classifying ADS-B attacks using advanced techniques such as machine learning. In this paper, we propose a new module to our ADS-Bsec framework capable of classifying ADS-B attacks using Support Vector Machines (SVM), Decision Tree, and Random Forest (RF). To illustrate and evaluate our ideas, we designed several experiments using a flight dataset from Lisbon to Paris that includes attacks from three categories. Our experimental results demonstrated that machine learning-based models provide high performance in terms of accuracy, sensitivity, and specificity metrics.

Keywords: Security, Cooperative Systems (V2X), V2X Communication
Abstract: Federated learning (FL) is a machine learning technique that aims at training an algorithm across decentralized entities holding their local data private. Wireless mobile networks allow users to communicate with other fixed or mobile users. The road traffic network represents an infrastructure-based configuration of a wireless mobile network where the Connected and Automated Vehicles (CAV) represent the communicating entities. Applying FL in a wireless mobile network setting gives rise to a new threat in the mobile environment that is very different from the traditional fixed networks. The threat is due to the intrinsic characteristics of the wireless medium and is caused by the characteristics of the vehicular networks such as high node-mobility and rapidly changing topology. Most cyber defense techniques depend on highly reliable and connected networks. This paper explores falsified information attacks, which target the FL process that is ongoing at the RSU. We identified a number of attack strategies conducted by the malicious CAVs to disrupt the training of the global model in vehicular networks. We show that the attacks were able to increase the convergence time and decrease the accuracy of the model. We demonstrate that our attacks bypass FL defense strategies in their primary form and highlight the need for novel poisoning resilience defense mechanisms in the wireless mobile setting of the future road networks.

Keywords: Human-Machine Interface, Automated Vehicles, Hand-off/Take-Over
Abstract: It is essential to help drivers have appropriate understandings of level 2 automated driving systems for keeping driving safety. A human machine interface (HMI) was proposed to present real time results of image recognition by the automated driving systems to drivers. It was expected that drivers could better understand the capabilities of the systems by observing the proposed HMI. Driving simulator experiments with 18 participants were preformed to evaluate the effectiveness of the proposed system. Experimental results indicated that the proposed HMI could effectively inform drivers of potential risks continuously and help drivers better understand the level 2 automated driving systems.

Keywords: Human-Machine Interface, Automated Vehicles, Novel Interfaces and Displays
Abstract: To maximize safety and driving comfort, autonomous driving systems can benefit from implementing foresighted action choices that take different potential scenario developments into account. While artificial scene prediction methods are making fast progress, an attentive human driver may still be able to identify relevant contextual features which are not adequately considered by the system or for which the human driver may have a lack of trust into the system’s capabilities to treat them appropriately. We implement an approach that lets a human driver quickly and intuitively supplement scene predictions to an autonomous driving system by gaze. We illustrate the feasibility of this approach in an existing autonomous driving system running a variety of scenarios in a simulator. Furthermore, a Graphical User Interface (GUI) was designed and integrated to enhance the trust and explainability of the system. The utilization of such cooperatively augmented scenario predictions has the potential to improve a system’s foresighted driving abilities and make autonomous driving more trustable, comfortable and personalized.

Keywords: Automated Vehicles, Situation Analysis and Planning, Sensor and Data Fusion
Abstract: Research on connected vehicles represents a continuously evolving technological domain, fostered by the emerging Internet of Things paradigm and the recent advances in intelligent transportation systems. In the context of assisted driving, connected vehicle technology provides real-time information about the surrounding traffic conditions. In this regard, we propose an online and adaptive scheme for parking availability mapping. Specifically, we adopt an information-seeking active sensing approach to select the incoming data, thus preserving the onboard storage and processing resources; then, we estimate the parking availability through Gaussian Process Regression. We compare the proposed algorithm with several baselines, which attain lower performance in terms of mapping convergence speed and adaptation capabilities.

Keywords: Mapping and Localization, Sensor and Data Fusion, Information Fusion
Abstract: High Definition (HD) maps enable autonomous vehicles to not only navigate roads but also localize. Using perception sensors such as cameras or lidars, map features can be detected and used for localization. The accuracy of vehicle localization is directly influenced by the accuracy of the features. It is therefore essential for the localization system to be able to detect erroneous map features. In this paper, an approach using Kalman smoothing with observation residuals is presented to address this issue. A covariance intersection of the residuals is proposed to manage their unknown correlation. The method also leverages the information of multiple runs to improve the detection of small errors. The performance of the method is evaluated using experimental data recorded on public roads with erroneous road signs. Our results allow to evaluate the gain of detection brought during successive drives.

Keywords: Mapping and Localization, Sensor and Data Fusion, Automated Vehicles
Abstract: In the world of autonomous driving, high resolution maps play a fundamental role. Such maps are highly accurate representations of the environment and are essential for all the algorithms of strategy and path planning operations. Unfortunately, it is not always possible to guarantee the total reliability of these maps and therefore it is necessary to design a procedure for their validation. In this paper, we introduce a framework for validating map data at run-time based on probabilistic graphical models. Results from simulations show the capabilities of the proposed approach and highlight the need to find an appropriate balance between model accuracy and complexity.

Keywords: Automated Vehicles, Mapping and Localization
Abstract: Typically, automated driving functions rely on high-definition (HD) maps for modeling the stationary environment (SE). However, outdated or erroneous maps pose a risk to both safety and performance of such a driving function. To address the issue of false map data provided to the vehicle, deviations ahead of the vehicle must be detected and corrected, preferably within the vehicle. To enable the continued operation of the driving function, a SE model as input to the driving function has to be generated on the fly. Moreover, to reduce the probability to encounter deviations in the first place, map update hypotheses have to be provided, e.g., to compute an update in an external server.

In this paper, we present a concept for integrating prior knowledge, e.g., regarding rule-compliant lane configurations, into the generation of the SE model. Prior knowledge enables the evaluation of undetected elements, the interpretation of connections between elements, and an overall plausibility check. Last, we provide an example for SE modeling for which we demonstrate the benefit of incorporating prior knowledge. The main novelity of this work is to show a way of deriving and representing required knowledge for SE modeling. Instead of focussing on individual infrastructure entities (e.g., intersection) as typically discussed in related works, we establish our derivation by analyzing traffic regulations and exemplary critical scenarios that arise due to the presence of map deviations.

Keywords: Advanced Driver Assistance Systems, Hand-off/Take-Over, Human-Machine Interface
Abstract: Recent developments in advanced driving assistance systems (ADAS) that rely on some level of autonomy have led the automobile industry and research community to investigate the impact they might have on driving performance. However, most of the research performed so far is based on simulated environments. In this study we investigated the behavior of drivers in a vehicle with automated driving system (ADS) capabilities in a real life driving scenario. We analyzed their response to a take over request (TOR) at two different driving speeds while being engaged in non- driving-related tasks (NDRT). Results from the performed experiments showed that driver reaction time to a TOR, gaze behavior and self-reported trust in automation were affected by the type of NDRT being concurrently performed and driver reaction time and gaze behavior additionally depended on the driving or vehicle speed at the time of TOR.

Keywords: Mapping and Localization, Automated Vehicles, Autonomous / Intelligent Robotic Vehicles

Keywords: Intelligent Vehicle Software Infrastructure, Automated Vehicles, Vehicle Environment Perception
Abstract: Innovation cycles in the automotive industry are shortening due to influences from the IT world and trends such as automated driving. In contrast, the life cycles of the vehicles remain substantially longer. This gives rise to problems such as the lack of availability of essential components. In addition, new business cases are emerging, such as retrofitting functionality at the customer's site. However, the electrical/electronical and software architectures of today's vehicles do not offer the required degree of flexibility during runtime.

This paper therefore presents a plug-and-play approach to dynamically reconfigure vehicle architectures. This includes the registration of new components such as ECUs or intelligent sensors/actuators in the vehicle, including the transfer of required software components to the vehicle's software repository. The focus here is on the exchange of components based on so-called capabilities: As a consequence, it is not only possible to dynamically switch to another capability in the event of a failure, but also when a capability with higher-rated properties becomes available after adding new components. This approach is demonstrated using two cameras that are switched between at runtime. Our findings indicate that feasible switching times can be achieved and that the approach is therefore suitable for productive use.

Keywords: Self-Driving Vehicles, Automated Vehicles, Reinforcement Learning
Abstract: Autonomous driving problems related to vehicle control using deep reinforcement learning (DRL) techniques are still unsolved. DRL approaches have achieved notable results, its dependability on reward functions and defining the type of control actions are dominating factors of the objective, that controls its success. Several DRL approaches applied in the past consider a finite set of available actions to be controlled by the agent hence, it performs sharp actions. While real driving requires precision control capabilities that tend to apply safer and smoother actions. For incorporating such precision control capabilities, this paper considers the driving problem as a continuous control problem. For this, the gym-highway environments are used as these environments are controllable and customizable to simulate diverse driving scenarios. The simulation setup for parking is updated to resemble the complex scenario and for highway driving a novel reward function is designed to handle continuous actions. Dual critic based DRL approaches are applied as these approaches have shown remarkable performance in robotic locomotion control problems. The video results demonstrate the way different policies fulfil the objective.

Keywords: Radar Sensing and Perception, Vehicle Environment Perception, Automated Vehicles
Abstract: Testing of advanced driver-assistance systems (ADAS) is complex, time-consuming and expensive. Therefore, new methods for the validation of such applications are required. A common solution is the use of virtual validation via environment simulation tools, but their reliability must first be confirmed. For this purpose, a comparison of a real and a simulated radar sensor under adverse weather conditions is performed in this work. To quantify the deviation between reality and the virtual test environment, a complex scenario with multiple traffic objects is set up on the proving ground and in the simulation. The data is measured for clear weather, rain and fog and afterwards compared to validate the performance of the sensor model under those non-ideal conditions.

The implemented method shows that both sensors neglect the same traffic objects. Compared to the real radar, the variation of the measured parameters according to changing weather conditions in the simulation tends to be correct, but the values are not completely realistic.

Though, the validation method is implemented successfully, in further work the comparison of different sensors is recommended. Furthermore, an in-depth examination of the impact due to varying intensity of rain and fog has to be undertaken.

Keywords: Vehicle Environment Perception, Vision Sensing and Perception, Image, Radar, Lidar Signal Processing
Abstract: For visual perception in automated driving, a reliable detection of so-called corner cases is important. Corner cases appear in many different forms and can be image frame- or sequence-related. In this work, we consider a specific type of corner case: collective anomalies. These are instances that appear in unusually large amounts in an image. We propose a detection method for collective anomalies based on a comparison of a test (sub-)set instance distribution to a training (i.e., reference) instance distribution, both distributions obtained by an instance-based semantic segmentation. For this comparison, we propose a novel so-called earth mover’s deviation(EMDEV) measure, which is able to provide signed deviations of instance distributions. Further, we propose a sliding window approach to allow the comparison of instance distributions in an online application in the vehicle. With our approach, we are able to identify collective anomalies by the proposed EMDEV measure, and to detect deviations from the instance distribution of the reference dataset.

Keywords: Cooperative Systems (V2X), Automated Vehicles, Vehicle Control
Abstract: This research proposes a Cooperative Adaptive Cruise Control Lane Change (CACCLC) controller. It is designed for making space to change lane successfully. The proposed controller has the following features: i) capable of making space to change lane by adopting a new Backward-Looking (BL) information topology; ii) with string stability; iii) with consideration of vehicle dynamics. The proposed CACCLC controller is evaluated on a joint simulation platform consisting of PreScan and Matlab/Simulink. Results demonstrate that: i) a lane-change gap for a single vehicle could be utilized for CACCLC maneuver; ii) the proposed CACCLC controller is with string stability and could eliminate 66.76% gap error from the end to the start of a platoon.

Keywords: Cooperative Systems (V2X), V2X Communication, Autonomous / Intelligent Robotic Vehicles
Abstract: In terms of Vehicle to Everything (V2X) testing and evaluation, Hardware-in-the-loop (HIL) simulation has become an indispensable technology. In the research of HIL testing, it is necessary to use micro-traffic simulation software to build scenarios and simulate traffic objects to meet the testing requirements of complex traffic scenarios for the Internet of Vehicles (IOV). However, the performance of the micro-simulation model deadly influences simulation accuracy. Hence, in this paper, an improved micro-simulation model is constructed on basis of the Krauss model, and an application test scheme is designed. Simulation results show that the improved model solves the problem of acceleration changing abruptly, and improves the effectiveness and practicability of the V2X in-loop test.

Keywords: Cooperative Systems (V2X), Advanced Driver Assistance Systems, Cooperative ITS
Abstract: In this paper, a universal CACC is proposed to accommodate the fact that vehicle load changes from time to time. The same vehicle could weigh differently when hauling a different number of passengers or moving a different amount of cargo. To achieve this, a dynamic matrix control-based approach is designed. The proposed controller is formulated in space domain to take advantage of the historical information of the leader to improve control accuracy and stability. It was evaluated on a Carsim-Prescan integrated simulation platform. Sensitivity analysis was conducted in terms of speed and vehicle load. Results confirm that the same proposed controller is able to maintain string stability while handling varying speeds and vehicle loads.

Keywords: Advanced Driver Assistance Systems, Traffic Flow and Management, Intelligent Vehicle Software Infrastructure
Abstract: Recently, motorcycle accidents are increasing in developing countries. One of the main reasons for this is the increase in traffic volume due to an increased number of four-wheeled vehicles. This brings about a heterogeneous (mixed) traffic flow consisting of two-wheeled vehicles and four-wheeled vehicles, which can result in the occurrence of sideswipe collisions. We carried out a survey of two-wheeled vehicle driving in heterogeneous traffic flow by considering vehicle density, acceleration, and pore (lateral gap), among other factors. Based on the results of this survey, we aim to predict motorcycle riding that carries high risk of collision, and to prevent such accidents from occurring. In this paper, we describe a novel algorithm which is capable of predicting two-wheel driving using vehicle detection and pore consideration. The performance of the proposed algorithm is verified and its associated issues are described. In addition, an example of this prediction algorithm is preliminarily implemented as a smartphone application.

Keywords: Automated Vehicles, Self-Driving Vehicles, Deep Learning
Abstract: The paper proposes an enhanced observer design method for autonomous vehicles, with which the unmeasurable states in vehicle and chassis motion can be estimated. The novelty of the method is that the learning-based observer and the linear parameter varying (LPV) observer in a joint observer design structure are incorporated, which results in an improved performance level on the estimation error. Nevertheless, the proposed design method is able to guarantee the limitation of the estimation error, even if the error of the learning-based observer under all scenarios cannot be verified. Thus, the proposed method handles the main disadvantage of the learning-based approaches, i.e. the lack of performance guarantees, while their advantages, i.e. the improved observation performance in the operation of the observer are taken. The proposed method is applied on a lateral path following control problem, where the goal of the observer is to provide an accurate lateral velocity signal for the vehicle. The effectiveness of the method is illustrated through simulation examples on high-fidelity vehicle dynamic simulator CarSim.

Keywords: Vehicle Control, Automated Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: Dynamic/Kinematic model is of great significance in decision and control of intelligent vehicles. However, due to the singularity of dynamic models at low speed, kinematic models have been the only choice under such driving scenarios. Inspired by the concept of backward Euler method, this paper presents a discrete dynamic bicycle model feasible at any low speed. We further give a sufficient condition, based on which the numerical stability is proved. Simulation verifies that (1) the proposed model is numerically stable while the forward-Euler discretized dynamic model diverges; (2) the model reduces forecast error by up to 65% compared to the kinematic model. As far as we know, it is the first time that a dynamic bicycle model is qualified for urban driving scenarios involving stop-and-go tasks.

Keywords: Telematics, Legal Impacts, Automated Vehicles
Abstract: Due to the many remaining obstacles before reliability and safety can sufficiently be guaranteed for high-level automated vehicles (AVs), teleoperation or remote operation of partially automated vehicles by a human driver has become increasingly interesting to consider. However, remote operation, including remote driving, has so far only received little attention in legal scientific and transportation literature. This paper aims to establish some basic legal matters for remote driving by examining its regulatory development in three different jurisdictions. A combination of methods is employed including an examination of literature regarding AVs and their regulation. The main result is that current regulation in the examined jurisdictions intentionally addresses a future with high-level AVs, but to a large extent excludes regulatory details for remote operation. In conclusion, this paper argues that both present and coming regulation for automated vehicles ought to be more near future-oriented and address the concept of remote operation more explicitly. This, for regulation to be better in touch with current technology, for the benefit of a wider acceptance in society, for legal certainty, but also for innovation support and stability for investments in technology.

Keywords: Advanced Driver Assistance Systems, Automated Vehicles, Human-Machine Interface
Abstract: Autonomous cars can reduce road traffic accidents and provide a safer mode of transport. However, key technical challenges, such as safe navigation in complex urban environments, need to be addressed before deploying these vehicles on the market. Teleoperation can help smooth the transition from human operated to fully autonomous vehicles since it still has human in the loop providing the scope of fallback on driver. This paper presents an Active Safety System (ASS) approach for teleoperated driving. The proposed approach helps the operator ensure the safety of the vehicle in complex environments, that is, avoid collisions with static or dynamic obstacles. Our ASS relies on a model predictive control (MPC) formulation to control both the lateral and longitudinal dynamics of the vehicle. By exploiting the ability of the MPC framework to deal with constraints, our ASS restricts the controller's authority to intervene for lateral correction of the human operator's commands, avoiding counter-intuitive driving experience for the human operator. Further, we design a visual feedback to enhance the operator's trust over the ASS. In addition, we propose an MPC's prediction horizon data based novel predictive display to mitigate the effects of large latency in the teleoperation system. We tested the performance of the proposed approach on a high-fidelity vehicle simulator in the presence of dynamic obstacles and latency.

Keywords: Telematics, Image, Radar, Lidar Signal Processing, Automated Vehicles
Abstract: Vehicles with autonomous driving capabilities are present on public streets. However, edge cases remain that still require a human in-vehicle driver. Assuming the vehicle manages to come to a safe state in an automated fashion, teleoperated driving technology enables a human to resolve the situation remotely by a control interface connected via a mobile network. While this is a promising solution, it also introduces technical challenges, one of them being the necessity to transmit video data of multiple cameras from the vehicle to the human operator. In this paper, an adaptive video streaming framework specifically designed for teleoperated vehicles is proposed and demonstrated. The framework enables automatic reconfiguration of the video streams of the multi-camera system at runtime. Predictions of variable transmission service quality are taken into account. With the objective to improve visual quality, the framework uses so-called rate-quality models to dynamically allocate bitrates and select resolution scaling factors. Results from deploying the proposed framework on an actual teleoperated driving system are presented.

Keywords: Active and Passive Vehicle Safety, Collision Avoidance, Deep Learning
Abstract: As a core technology of the autonomous driving system, pedestrian trajectory prediction can significantly enhance the function of active vehicle safety and reduce road traffic injuries. In traffic scenes, when encountering with oncoming people, pedestrians may make sudden turns or stop immediately, which often leads to complicated trajectories. To predict such unpredictable trajectories, we can gain insights into the interaction between pedestrians. In this paper, we present a novel generative method named Spatial Interaction Transformer (SIT), which learns the spatio-temporal correlation of pedestrian trajectories through attention mechanisms. Furthermore, we introduce the conditional variational autoencoder (CVAE) framework to model the future latent motion states of pedestrians. In particular, the experiments based on large-scale traffic dataset nuScenes show that SIT has an outstanding performance than state-of-the-art (SOTA) methods. Experimental evaluation on the challenging ETH and UCY datasets confirms the robustness of our proposed model.

Keywords: Reinforcement Learning, Autonomous / Intelligent Robotic Vehicles, Collision Avoidance
Abstract: The simulation of real-world traffic is a challenging task that can be accelerated by imitation learning. Recent approaches based on neural network policies were able to present promising results in generating human-like driving behavior. However, one drawback is that certain behaviors, such as avoiding accidents, cannot be guaranteed with such policies. Therefore, we propose to combine recent imitation learning methods like GAIL with a rule-based safety framework to avoid collisions during training and testing. Our method is evaluated on highway driving scenes where all vehicles are controlled by our driving policies trained on the real-world driving dataset highD. In this setup, our method is compared to a standard neural network policy trained with GAIL. Agents using our method were able to match GAIL performance while additionally guaranteeing collision-free driving.

Keywords: Driver State and Intent Recognition, Situation Analysis and Planning, Advanced Driver Assistance Systems
Abstract: Understanding and predicting lane-changing intents on highways is fundamental for multi-lane cruise control systems and automated driving. Many studies have been carried out using the NGSIM data-set of trajectories on US highways with symmetric lane-changing behaviors. In this contribution, we present a statistical analysis of discretionary lane-changing maneuvers on German two-lane highways imposing overtaking to the left only (highD project). We aim to separately identify the underlying mechanisms that motivate drivers to overtake and to fold-down. The analysis is done using principal component analysis and logistic regressions based on speed-difference and distance variables with the four surrounding vehicles. The results show that two different mechanisms operate in case of overtaking and fold-down. Overtaking can be explained monotonically with only three variables: the distance and speed difference with the predecessor and the speed difference with the following vehicle on the adjacent lane. Fold-down is a more complex process involving more variables and relationships. The predictions based on the logistic regression are accurate for lane-keeping but limited for lane-changing maneuver, especially for fold-down. The limitations are due to non-linear behaviors of the fold-down maneuver for which the logistic regression is insensitive.

Keywords: Automated Vehicles, Situation Analysis and Planning, Legal Impacts
Abstract: With Regulation UNECE R157 on Automated Lane-Keeping Systems, the first framework for the introduction of passenger cars with Level 3 systems has become available in 2020. In accordance with recent research projects including academia and the automotive industry, the Regulation utilizes scenario based testing for the safety assessment. The complexity of safety validation of automated driving systems necessitates system-level simulations. The Regulation, however, is missing the required parameterization necessary for test case generation. To overcome this problem, we incorporate the exposure and consider the heterogeneous behavior of the traffic participants by extracting concrete scenarios according to Regulation's scenario definition from the established naturalistic highway dataset highD. We present a methodology to find the scenarios in real-world data, extract the parameters for modeling the scenarios and transfer them to simulation. In this process, more than 340 scenarios were extracted. OpenSCENARIO files were generated to enable an exemplary transfer of the scenarios to CARLA and esmini. We compare the trajectories to examine the similarity of the scenarios in the simulation to the recorded scenarios. In order to foster research, we publish the resulting dataset called ConScenD together with instructions for usage with both simulation tools. The dataset is available online at https://www.levelXdata.com/scenarios.

Keywords: Deep Learning, Self-Driving Vehicles
Abstract: Many machine learning applications can benefit from simulated data for systematic validation - in particular if real-life data is difficult to obtain or annotate. However, since simulations are prone to domain shift w.r.t. real-life data, it is crucial to verify the transferability of the obtained results. We propose a novel framework consisting of a generative label-to-image synthesis model together with different transferability measures to inspect to what extent we can transfer testing results of semantic segmentation models from synthetic data to equivalent real-life data. With slight modifications, our approach is extendable to, e.g., general multi-class classification tasks. Grounded on the transferability analysis, our approach additionally allows for extensive testing by incorporating controlled simulations. We validate our approach empirically on a semantic segmentation task on driving scenes. Transferability is tested using correlation analysis of IoU and a learned discriminator. Although the latter can distinguish between real-life and synthetic tests, in the former we observe surprisingly strong correlations of 0.7 for both cars and pedestrians.

Keywords: Autonomous / Intelligent Robotic Vehicles, Situation Analysis and Planning, Self-Driving Vehicles
Abstract: Autonomous vehicles have to take cautious decisions when driving in complex urban scenarios. Situation understanding is a key point towards safe navigation. High Definition maps with several layers of information are of great interest. They can supply different types of prior information such as the road network topology, the geometric description of the road, and also semantic information including traffic laws. Conjointly with the perception system, they provide representations of the static environment and allow to model interactions. In complex situations, it is mandatory for safety to get a reliable understanding of the vehicle situation to avoid inappropriate decisions. Confidence on the information supplied to decision-making must be tackled. This paper aims at proposing a spatial occupancy information representation at lane level with Lane Grid Maps (LGM). Based on areas considered as being of interest for the ego vehicle and sampled in the along-track direction, perception data is augmented to provide non-misleading information to the decision-making at a tactical level. An advantage of this representation is its ability to tackle information integrity with a good spatial sampling choice. This approach also needs to properly take into account the uncertainty of the ego vehicle localization, which has an impact on the estimated spatial occupancy of the perceived objects. This paper provides a method to set the proper sampling step in order to avoid oversampling and subsampling of the LGM for a given integrity risk level. The approach is evaluated under real driving scenarios in which several experimental vehicles were driven.

Keywords: Self-Driving Vehicles, Security
Abstract: Testing of Autonomous Driving Systems (ADSs) is of paramount importance. However, ADS testing raises several challenges specific to the domain. Typical testing (coverage criteria, test generation, and oracle definition) and debugging activities performed for software programs are not directly applicable to ADSs, because of the lack of proper test oracles, and the difficulty of specifying the desired, correct ADS behavior. We tackle these challenges by extending and combining existing approaches to the domain of testing ADS. The approach is demonstrated on an industrial path planner. The path planner decides which path to follow through a cost function that uses parameters to assign a cost to the driving characteristics (e.g., lateral acceleration or speed) that must be applied in the path. These parameters implicitly describe the behavior of the ADS. We exploit this idea for defining a coverage criterion, for automatically specifying an oracle, and for debugging the path planner.

Keywords: Automated Vehicles, Self-Driving Vehicles, Active and Passive Vehicle Safety
Abstract: The safety assessment of Automated Vehicles (AVs) is an important aspect of the development cycle of AVs. A scenario-based assessment approach is accepted by many players in the field as part of the complete safety assessment. A scenario is a representation of a situation on the road to which the AV needs to respond appropriately. One way to generate the required scenario-based test descriptions is to parameterize the scenarios and to draw these parameters from a probability density function (pdf). Because the shape of the pdf is unknown beforehand, assuming a functional form of the pdf and fitting the parameters to the data may lead to inaccurate fits. As an alternative, Kernel Density Estimation (KDE) is a promising candidate for estimating the underlying pdf, because it is flexible with the underlying distribution of the parameters. Drawing random samples from a pdf estimated with KDE is possible without the need of evaluating the actual pdf, which makes it suitable for drawing random samples for, e.g., Monte Carlo methods. Sampling from a KDE while the samples satisfy a linear equality constraint, however, has not been described in the literature, as far as the authors know.

In this paper, we propose a method to sample from a pdf estimated using KDE, such that the samples satisfy a linear equality constraint. We also present an algorithm of our method in pseudo-code. The method can be used to generating scenarios that have, e.g., a predetermined starting speed or to generate different types of scenarios. This paper also shows that the method for sampling scenarios can be used in case a Singular Value Decomposition (SVD) is used to reduce the dimension of the parameter vectors.

Keywords: Vehicle Environment Perception, Automated Vehicles, Active and Passive Vehicle Safety
Abstract: Scenario-based virtual validation of automated driving functions is a promising method to reduce testing effort in real traffic. In this work, a method for deriving scenario design criteria from a sensor modeling point of view is proposed. Using basic sensor technology specific equations as rough but effective boundary conditions, the accessible information for the system under test are determined. Subsequently, initial conditions such as initial poses of dynamic objects are calculated using the derived boundary conditions for designing logical scenarios. Further interest is given on triggers starting movements of objects during scenarios that are not time but object dependent. The approach is demonstrated on the example of the radar equation and first exemplary results by identifying relevance regions are shown.

Keywords: Unsupervised Learning, Convolutional Neural Networks, Deep Learning
Abstract: In this paper, we present a novel architecture and training methodology for learning monocular depth prediction, camera pose estimation, optical flow, and moving object seg- mentation using a common encoder in an unsupervised fashion. We demonstrate that the geometrical relationships between these tasks not only support joint unsupervised learning as shown in previous works but also allow them to share common features. We also show the advantage of using a two-stage learning approach to improve the performance of the base network.

Keywords: Deep Learning, Self-Driving Vehicles, Vehicle Environment Perception
Abstract: In this paper we present the Oxford Road Boundaries Dataset, designed for training and testing machine-learning-based road-boundary detection and inference approaches. We have hand-annotated two of the 10 km-long forays from the Oxford Robotcar Dataset and generated from other forays several thousand further examples with semi-annotated road-boundary masks. To boost the number of training samples in this way, we used a vision-based localiser to project labels from the annotated datasets to other traversals at different times and weather conditions. As a result, we release 62605 labelled samples, of which 47639 samples are curated. Each of these samples contain both raw and classified masks for left and right lenses. Our data contains images from a diverse set of scenarios such as straight roads, parked cars, junctions, etc. Files for download and tools for manipulating the labelled data are available at: oxford-robotics-institute.github.io/road-boundaries-dataset

Keywords: Deep Learning, Lidar Sensing and Perception, Reinforcement Learning
Abstract: Deep neural networks provide high accuracy for perception. However they require high computational power. In particular, LiDAR-based object detection delivers good accuracy and real-time performance, but demands high computation due to expensive feature-extraction from point cloud data in the encoder and backbone networks. We investigate the model complexity versus accuracy trade-off using reinforcement learning based pruning for PointPillars, a recent LiDAR-based 3D object detection network. We evaluate the model on the validation dataset of KITTI (80/20-splits) according to the mean average precision (mAP) for the car class. We prune the original PointPillars model (mAP 89.84) and achieve 65.8% reduction in floating point operations (FLOPs) for a marginal accuracy loss. The compression corresponds to 31.7% reduction in inference time and 35% reduction in GPU memory on GTX 1080 Ti.

Keywords: Lidar Sensing and Perception, Deep Learning, Self-Driving Vehicles
Abstract: In this paper we investigate the use of deep neural networks for 3D object detection in uncommon, unstructured environments such as in an open-pit mine. While neural nets are frequently used for object detection in regular autonomous driving applications, more unusual driving scenarios aside street traffic pose additional challenges. For one, the collection of appropriate data sets to train the networks is an issue. For another, testing the performance of trained networks often requires tailored integration with the particular domain as well. While there exist different solutions for these problems in regular autonomous driving, there are only very few approaches that work for special domains just as well. We address both the challenges above in this work. First, we discuss two possible ways of acquiring data for training and evaluation. That is, we evaluate a semi-automated annotation of recorded LIDAR data and we examine synthetic data generation. Using these datasets we train and test different deep neural network for the task of object detection. Second, we propose a possible integration of a ROS2 detector module for an autonomous driving platform. Finally, we present the performance of three state-of-the-artdeep neural networks in the domain of 3D object detection on a synthetic dataset and a smaller one containing a characteristic object from an open-pit mine.

Keywords: Sensor and Data Fusion, Radar Sensing and Perception, Automated Vehicles
Abstract: 3D multi-object tracking is a crucial component in the perception system of autonomous driving vehicles. Tracking all dynamic objects around the vehicle is essential for tasks such as obstacle avoidance and path planning. Autonomous vehicles are usually equipped with different sensor modalities to improve accuracy and reliability. While sensor fusion has been widely used in object detection networks in recent years, most existing multi-object tracking algorithms either rely on a single input modality, or do not fully exploit the information provided by multiple sensing modalities. In this work, we propose an end- to-end network for joint object detection and tracking based on radar and camera sensor fusion. Our proposed method uses a center-based radar-camera fusion algorithm for object detection and utilizes a greedy algorithm for object association. The proposed greedy algorithm uses the depth, velocity and 2D displacement of the detected objects to associate them through time. This makes our tracking algorithm very robust to occluded and overlapping objects, as the depth and velocity information can help the network in distinguishing them. We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark, as well as the baseline LiDAR-based method. Our method is online with a runtime of 35ms per image, making it very suitable for autonomous driving applications.

Keywords: Unsupervised Learning, Deep Learning, Convolutional Neural Networks
Abstract: Although Content Based Image Retrieval (CBIR) is an active research field, application to images simultaneously containing multiple objects has received limited research interest. For such complex images, it is difficult to precisely convey the query intention, to encode all the image aspects into one compact global feature representation and to unambiguously define label similarity or dissimilarity. Motivated by the recent success on many visual benchmark tasks, we propose a self-supervised method to train a feature representation learning model. We propose usage of multiple query images, and use an attention based architecture to extract features from diverse image aspects that benefits from this. The method shows promising performance on road scene datasets, and, consistently improves when multiple query images are used instead of a single query image.

Keywords: Self-Driving Vehicles, Deep Learning
Abstract: There have been many recent advancements in imitation and reinforcement learning for autonomous driving, but existing metrics generally lack the means to capture a wide range of driving behaviors and compare the severity of different failure cases. To address this shortcoming, we introduce Quantitative Evaluation for Driving (QED), which assesses different aspects of driving behavior including the ability to stay in the center of the lane, avoid weaving and erratic behavior, follow the speed limit, and avoid collisions. We compare scores generated by QED against scores assigned by human evaluators on 30 different drivers and 6 different towns in the CARLA driving simulator. In ``easy'' evaluation scenarios where better drivers are easily distinguished from worse drivers, QED attains 0.96 Pearson correlation and 0.97 Spearman correlation with human evaluators, similar to the baseline inter-human-evaluator 0.96 Pearson correlation and 0.95 Spearman correlation. In ``hard'' evaluation scenarios where ranking drivers is more ambiguous, QED attains 0.84 Pearson correlation and 0.74 Spearman correlation with human evaluators, slighter higher than the baseline inter-human-evaluator 0.78 Pearson correlation and 0.7 Spearman correlation. While QED may not capture every characteristic that defines good driving, we consider it an important foundation for reproducibility and standardization in the community.

Keywords: Situation Analysis and Planning, Automated Vehicles
Abstract: In traffic, there are often situations with more than one possible future development. One of these is entering a roundabout: If there is another vehicle in the roundabout, it may stay, preventing an unhindered entrance--or it may take the exit beforehand, leaving the roundabout empty. When facing this scenario with an automated vehicle, one possibility is to assume the worst case and act defensively. However, this neglects the fact that early observations give hints towards one or the other. The desired behavior would be wait-and-see, keeping the option for both, entering and braking, open.

We model the scenario as a Partially Observable Markov Decision Process (POMDP), a general framework for decision making under uncertainty. For solving, we use the POMCP algorithm. Evaluated in simulation, we can show that the POMDP reduces discomfort compared to the pessimistic approach and a baseline reactive method.

Keywords: Situation Analysis and Planning, Vulnerable Road-User Safety, Autonomous / Intelligent Robotic Vehicles
Abstract: This article examines probabilistic trajectory forecasting methods of vulnerable road users (VRU) for the motion planning of autonomous vehicles. The future trajectories of a cyclist are predicted by Quantile Surface Neural Networks (QSN) and Mixture Density Neural Networks (MDN), both modeling confidence regions around the cyclist’s expected locations. Confidence regions are approximated by different methods with varying degrees of complexity to bridge the gap between forecasting and planning. Model-Predictive Planning (MPP) based on these regions is used for the autonomous vehicle. The approach is evaluated using a case study regarding safe trajectory planning for overtaking cyclists. The experiments show the effectiveness of the approach. Different considerations on the use of combined probabilistic trajectory prediction and vehicle trajectory planning are included.

Keywords: Human-Machine Interface, Deep Learning, Convolutional Neural Networks
Abstract: The lack of sleep (typically <6 hours a night) or driving for a long time are the reasons of drowsiness drivering and caused serious traffic accidents. With pandemic of the COVID-19, drivers are wearing masks to prevent infection from it, which makes visual-based drowsiness detection methods difficult. In this paper, electroencephalogram (EEG) signals are used to detect the vehicle driver drowsiness based on deep learning and attention mechanism. First of all, an 8-channels EEG collection hat is used to acquire the EEG signals in the simulation scenario of drowsiness driving and normal driving. Then the EEG signals are pre-processed by using the linear filter and wavelet threshold denoising. Secondly, the neural network based on attention mechanism and deep residual network (ResNet) is trained to classify the EEG signals. Finally, an early warning module is designed to sound an alarm if the driver is judged as drowsy. The system was tested under simulated driving environment and the drowsiness detection accuracy of the test set was 93.35%. Drowsiness warning simulation also verified the effectiveness of proposed early warning module.

Keywords: Deep Learning, Sensor and Data Fusion, Information Fusion
Abstract: Video captioning aims to generate textual descriptions according to the video contents. The risk assessment of autonomous driving vehicles has become essential for an insurance company for providing adequate insurance coverage, in particular, for emerging MaaS business. The insurers need to assess the risk of autonomous driving business plans with a fixed route by analyzing a large number of driving data, including videos recorded by dash cameras and sensor signals. To make the process more efficient, generating captions for driving videos can provide insurers concise information to understand the video contents quickly. A natural problem with driving video captioning is, since the absence of ego-vehicles in these egocentric videos, descriptions of latent driving behaviors are difficult to be grounded in specific visual cues. To address this issue, we focus on generating driving video captions with accurate behavior descriptions, and propose to incorporate in-vehicle sensors which encapsulate the driving behavior information to assist the caption generation. We evaluate our method on the Japanese driving video captioning dataset called City Traffic, where the results demonstrate the effectiveness of in-vehicle sensors on improving the overall performance of generated captions, especially on generating more accurate descriptions for the driving behaviors.

Keywords: Sensor and Data Fusion, Radar Sensing and Perception, Vehicle Environment Perception
Abstract: Extended object tracking methods are often based on the assumption that the measurements are uniformly distributed on the target object. However, this assumption is often invalid for applications using automotive radar or lidar data. Instead, there is a bias towards the side of the object that is visible to the sensor. To handle this challenge, we employ a Gaussian Mixture (GM) density to model a more detailed measurement distribution across the surface and extend a recent Kalman filter based elliptic object tracker called MEM-EKF* to get a closed-form solution for the measurement update. An evaluation of the proposed approach compared with classic elliptic trackers and a recent truncation-based approach is conducted on simulated data.

Keywords: Vehicle Environment Perception, Self-Driving Vehicles, Convolutional Neural Networks
Abstract: Data fusion plays a significant role in autonomous driving domain. Using an efficient combination of sensors like LiDAR, radar, and cameras could determine how quickly and accurately a vehicle makes all kinds of decisions related to road safety. In this article, we propose two approaches to improve object distance estimation by combining camera and LiDAR sensors. This work is inspired by the work presented in [2]. We propose to use instance segmentation and hierarchical clustering algorithms to resolve estimation errors generated when two or several bounding boxes (bbox) of detected objects overlap with each other. KITTI and Waymo databases were used to evaluate the accuracy of the proposed approaches. Finally, we compare the accuracy of our approaches with the accuracy proposed in [2] for some specific scenarios.

Keywords: Vehicle Control, Self-Driving Vehicles, Deep Learning
Abstract: Identifying tire and vehicle parameters is an essential step in designing control and planning algorithms for autonomous vehicles. This paper proposes using a new method: Simulation-Based Inference (SBI), a modern interpretation of Approximate Bayesian Computation methods (ABC) for parameter identification. Simulation-based inference is an emerging method in the machine learning literature and has proven to yield accurate results for many parameter sets in complex problems. We demonstrate in this paper that it can handle the identification of highly nonlinear vehicle dynamics parameters and gives accurate estimates of the parameters for the governing equations.

Keywords: Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles, Driver State and Intent Recognition
Abstract: There are few open source autonomous driving planners that are general enough to be used directly, or which could be easily customized to suit a particular application. OpenPlanner 1.0 was introduced back in 2017 to fill this gap. It was developed and integrated with the open source autonomous driving framework Autoware. Since then, many improvements have been introduced, following the original design goals. In this paper, the basic design will be re-introduced along with the latest developed technologies. The new planner is called OpenPlanner 2.0 and includes several new techniques such as multiple HD road network map formats support, trajectory and behavior estimation, planning based HMI support, path generation using kinematics based motion simulation and lane change behavior. OpenPlanner 2.0 is already attracting attention from the autonomous driving research and development communities; universities and companies. Several projects are using and contributing to its code base. Some of these applications will be discussed in this paper as well. A comparison between OpenPlanner and other open source planners showing the aspects where it is superior is also presented.

Keywords: Mapping and Localization, Automated Vehicles, Information Fusion
Abstract: In this study, we propose Eagleye, an open-source software, that performs lane level localization in an urban environment. A low-cost GNSS receiver, IMU, and velocity sensor are used for position estimation. The feature of this method is that it is optimized to take full advantage of the averaging effect using time series data longer than a few tens of seconds. This optimization improves the estimation performance by reducing the GNSS multipath in urban areas. In order to verify the effectiveness of the system, we conducted accuracy evaluation of the proposed method and performance comparison tests with expensive position estimation systems. As a result of the test, we confirmed that the proposed method can estimate the relative position results with an accuracy of 0.5 m per 100m and the absolute position performance with an accuracy of 1.5 m. In addition, it was confirmed that the performance of the proposed method was equivalent to that of an expensive system. Therefore, we believe that the proposed method can effectively estimate the location in an urban environment.

Keywords: Lidar Sensing and Perception, Mapping and Localization, Vehicle Environment Perception
Abstract: In this work, we present a detailed comparison of ten different 3D LiDAR sensors for the tasks of mapping and vehicle localization, using as common reference the Normal Distributions Transform (NDT) algorithm implemented in the self-driving open source platform Autoware. LiDAR data used in this study is a subset of our LiDAR Benchmarking and Reference (LIBRE) dataset, captured independently from each sensor, from a vehicle driven on public urban roads multiple times, at different times of the day. In this study, we analyze the performance and characteristics of each LiDAR for the tasks of (1) 3D mapping including an assessment map quality based on mean map entropy, and (2) 6-DOF localization using a ground truth reference map.

Keywords: Situation Analysis and Planning, Societal Impacts, Smart Infrastructure
Abstract: We describe an experiment of optimizing parking of privately owned connected autonomous vehicles (CAV) involving a municipal limitation of the empty cruising distances. Assuming that a considerable fraction of connected autonomous vehicles will be in private ownership, the problem of parking emerges, calling for a novel and regulated solution. To this end, we propose an auction-based parking lot assignment that optimizes the parking costs and penalizes the empty cruising. To grasp a hypothetical situation, we assess the properties of the proposed mechanism by simulations. A highly abstracted circular city model is presented as a simulation environment to the parking problem, which also considers the regulations limiting the extent of empty cruising. This hypothetical model includes two kinds of parking facilities (curb-sided parking lots and parking houses), various human activity models, pricing, and distance calculations calibrated by values measured in contemporary Budapest. Simulations indicate that using an auction mechanism does not necessarily increase parking charges significantly. The results also call attention to proper empty cruising regulations as parking lot occupancy and vehicle kilometer traveled strongly depend on them.

Keywords: Automated Vehicles, Situation Analysis and Planning, Collision Avoidance
Abstract: Trajectory planning is one of the most important modules of the Autonomous Driving Systems (ADSs), which aims to achieve a safe and comfortable interaction between the ADSs and obstacles. Currently, it remains a challenging issue to simultaneously ensure the comfort and safety of the planned trajectory, especially in dynamic traffic scenarios. In this paper, a trajectory planning method is proposed for autonomous vehicles to drive in dynamic traffic scenarios considering both comfort and safety. First, trajectory candidates are generated through the separation of path generation and velocity generation, and then some cost functions are constructed to evaluate each trajectory candidate to obtain the final trajectory. The proposed trajectory generation method guarantees the continuity of generated trajectory in both curvature and jerk, and the cost functions are proposed with the Trajectory Comfort Evaluation Model (TCEM) and Trajectory Safety Evaluation Model (TSEM), which balance the comfort and safety of a trajectory. Experiments prove the effectiveness of the proposed trajectory planner and its robustness in dynamic traffic scenarios.

Keywords: Autonomous / Intelligent Robotic Vehicles, Lidar Sensing and Perception, Unsupervised Learning
Abstract: Scalable systems for automated driving have to reliably cope with an open-world setting. This means, the perception systems are exposed to drastic domain shifts, like changes in weather conditions, time-dependent aspects, or geographic regions. Covering all domains with annotated data is impossible because of the endless variations of domains and the time-consuming and expensive annotation process. Furthermore, fast development cycles of the system additionally introduce hardware changes, such as sensor types and vehicle setups, and the required knowledge transfer from simulation. To enable scalable automated driving, it is therefore crucial to address these domain shifts in a robust and efficient manner. Over the last years, a vast amount of different domain adaptation techniques evolved. There already exists a number of survey papers for domain adaptation on camera images, however, a survey for LiDAR perception is absent. Nevertheless, LiDAR is a vital sensor for automated driving that provides detailed 3D scans of the vehicle’s surroundings. To stimulate future research, this paper presents a comprehensive review of recent progress in domain adaptation methods and formulates interesting research questions specifically targeted towards LiDAR perception.

Keywords: Reinforcement Learning, Self-Driving Vehicles, Deep Learning
Abstract: Applications of reinforcement learning (RL) are popular in autonomous driving tasks. That being said, tuning the performance of an RL agent and guaranteeing the generalization performance across variety of different driving scenarios is still largely an open problem. In particular, getting good performance on complex road and weather conditions require exhaustive tuning and computation time. Curriculum RL, which focuses on solving simpler automation tasks in order to transfer knowledge to complex tasks, is attracting attention in RL community. The main contribution of this paper is a systematic study for investigating the value of curriculum reinforcement learning in autonomous driving applications. For this purpose, we setup several different driving scenarios in a realistic driving simulator, with varying road complexity and weather conditions. Next, we train and evaluate performance of RL agents on different sequences of task combinations and curricula. Results show that curriculum RL can yield significant gains in complex driving tasks, both in terms of driving performance and sample complexity. Results also demonstrate that different curricula might enable different benefits, which hints future research directions for automated curriculum training.

Keywords: Unsupervised Learning, Automated Vehicles, Vision Sensing and Perception
Abstract: This work presents a two-staged, unsupervised domain adaptation process for semantic segmentation models by combining a self-training and self-supervision strategy. Self-training (i. e., training a model on self-inferred pseudo-labels) yields competitive results for domain adaptation in recent research. However, self-training depends on high-quality pseudo-labels. On the other hand, self-supervision trains the model on a surrogate task and improves its performance on the target domain without further prerequisites. Therefore, our approach improves the model's performance on the target domain with a novel surrogate task. To that, we continuously determine class centroids of the feature representations in the network's pre-logit layer on the source domain. Our surrogate task clusters the pre-logit feature representations on the target domain regarding these class centroids during both training stages. After the first stage, the resulting model delivers improved pseudo-labels for the additional self-training in the second stage. We evaluate our method on two different domain adaptions, a real-world domain change from Cityscapes to the Berkeley Deep Drive dataset and a synthetic to real-world domain change from GTA5 to the Cityscapes dataset. For the real-world domain change, the evaluation shows a significant improvement of the model from 46% mIoU to 54% mIoU on the target domain. For the synthetic to real-world domain change, we achieve an improvement from 38.8% to 46.42% on the real-world target domain.

Keywords: Vision Sensing and Perception, Convolutional Neural Networks, Sensor and Data Fusion
Abstract: Reliable object detection is critical for autonomous vehicles (AV). An AV must be safely guided towards its destination under different illumination conditions and avoid obstacles. Thermal infrared (TIR) camera sensors can provide robust image quality under any illumination. Past object detection work using TIR sensors focused on detecting only pedestrians by filtering thermal values. Other approaches leveraged the advantages of a pre-trained RGB-based model. However, the thermal threshold-based filtering can increase false positives depending on the TIR camera capability. Moreover, a large and new TIR training dataset is needed to improve the accuracy of the RGB-based object detection networks. The time and effort to annotate new data are significantly high. In this paper, we propose efficient and practical approaches to provide robust object detection from TIR images. %We first focus on a detailed analysis of the thermal representation of TIR images and demonstrate how the thermal values can be used effectively. We first reduce the cost of training with new data by using an automated process. To increase the final object detection accuracy, we next propose fusion methods that combine results from dual TIR camera sensors. Finally, we substantiate the practical feasibility of our approach and evaluate the substantial improvement in object detection accuracy. We use various detection networks and datasets on discrete Nvidia GPUs and an Nvidia Xavier embedded platform, commonly used by automotive OEMs.