Keywords: Automated Vehicles, Reinforcement Learning, Recurrent Networks
Abstract: In this paper, we present a safe deep reinforcement learning system for automated driving. The proposed framework leverages merits of both rule-based and learning-based approaches for safety assurance. Our safety system consists of two modules namely handcrafted safety and dynamically-learned safety. The handcrafted safety module is a heuristic safety rule based on common driving practice that ensure a minimum relative gap to a traffic vehicle. On the other hand, the dynamically-learned safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, the dynamically-leaned safety module incorporates a model lookahead beyond the immediate reward of reinforcement learning to predict safety longer into the future. If one of the future states leads to a near-miss or collision, then a negative reward will be assigned to the reward function to avoid collision and accelerate the learning process. We demonstrate the capability of the proposed framework in a simulation environment with varying traffic density. Our results show the superior capabilities of the policy enhanced with dynamically-learned safety module.

Keywords: Reinforcement Learning, Deep Learning, Vehicle Control
Abstract: In recent years, deep reinforcement learning (DRL) algorithms have been widely studied and utilized in the area of Intelligent Transportation Systems (ITS). DRL agents are mostly trained with transition pairs and interaction trajectories generated from simulation, and they can achieve satisfying or near optimal performances under familiar input states. However, for relative rare visited or even unvisited regions in the state space, there is no guarantee that the agent could perform well. Unfortunately, novel conditions are inevitable in real-world problems and there is always a gap between the real data and simulated data. Therefore, to implement DRL algorithms in real-world transportation systems, we should not only train the agent learn a policy that maps states to actions, but also the model uncertainty associated with each action. In this study, we adapt the method of Bayesian ensemble to train a group of agents with imposed diversity for an energy management system of a delivery vehicle. The agents in the ensemble agree well on familiar states but show diverse results on unfamiliar or novel states. This uncertainty estimation facilitates the implementation of interpretable postprocessing modules which can ensure robust and safe operations under high uncertainty conditions.

Keywords: Reinforcement Learning, Self-Driving Vehicles, Automated Vehicles
Abstract: Reinforcement learning (RL) can be used to create a tactical decision-making agent for autonomous driving. However, previous approaches only output decisions and do not provide information about the agent's confidence in the recommended actions. This paper investigates how a Bayesian RL technique, based on an ensemble of neural networks with additional randomized prior functions (RPF), can be used to estimate the uncertainty of decisions in autonomous driving. A method for classifying whether or not an action should be considered safe is also introduced. The performance of the ensemble RPF method is evaluated by training an agent on a highway driving scenario. It is shown that the trained agent can estimate the uncertainty of its decisions and indicate an unacceptable level when the agent faces a situation that is far from the training distribution. Furthermore, within the training distribution, the ensemble RPF agent outperforms a standard Deep Q-Network agent. In this study, the estimated uncertainty is used to choose safe actions in unknown situations. However, the uncertainty information could also be used to identify situations that should be added to the training process.

Keywords: Reinforcement Learning, Self-Driving Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: Deep Reinforcement Learning has proved to be able to solve many control tasks in different fields, but the behavior of these systems is not always as expected when deployed in real-world scenarios. This is mainly due to the lack of domain adaptation between simulated and real-world data and that of a train-test distinction that increase the risk of overfitting on the training scenarios. In this work, we investigate these problems in the autonomous driving field, especially for a maneuver planning module for roundabout insertions. In particular, we develop a system based on multiple environments in which agents are trained simultaneously, evaluating the behavior of the model in different scenarios. Finally, we develop techniques in order to reduce the gap between simulated and real-world data showing that this increases the generalization capabilities of the system both on unseen and real-world scenarios.

Keywords: Self-Driving Vehicles, Deep Learning, Reinforcement Learning

Keywords: Reinforcement Learning, Self-Driving Vehicles, Deep Learning
Abstract: Navigating through unsignalized intersections is one of the most challenging problems in urban environments for autonomous vehicles. Existing methods need to train specific policy models to deal with different tasks including going straight, turning left and turning right. In this paper we formulate intersection navigation as a multi-task reinforcement learning problem and propose a unified learning framework for all three navigation tasks at the intersections. We propose to represent multiple tasks with a unified four-dimensional vector, which elements mean common sub-task and three specified target sub-tasks respectively. Meanwhile, we design a vectorized reward function combining with deep Q-networks (DQN) to learn to handle multiple intersection navigation tasks concurrently. We train the agent to navigate through intersections by adjusting the speed of the ego vehicle under given route. Experimental results in both simulation and real-world vehicle test demonstrate that the proposed multi-task DQN algorithm outperforms baselines for all three navigation tasks in several different intersection scenarios.

Keywords: Reinforcement Learning, Automated Vehicles, Deep Learning
Abstract: Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network input to have a pre-defined input-size -- in semantic environments assuming a maximum number of vehicles. Additionally, this vectorial representation is not invariant to the order and number of vehicles. To mitigate the above-stated disadvantages, we propose combining graph neural networks with Actor-Critic reinforcement learning. % third point As graph neural networks apply the same network to every vehicle and aggregate incoming edge information, they are invariant to the number and order of vehicles. This makes them ideal candidates to be used as networks in semantic environments -- environments consisting of objects lists. Graph neural networks use relational information contained in the graph and do not have to implicitly infer these. Moreover, as graph neural networks have similar characteristics to traditional convolutional layers, they are not only able to propagate first-order, but also higher-degree information. We demonstrate our approach using a highway lane-change scenario and compare the performance of graph neural networks to traditional ones. Moreover, we show that graph neural networks are capable of handling a varying number and order of vehicles during training and application. To demonstrate the convolutional properties, we conduct ablation studies by varying the position of a single-vehicle.

Keywords: Reinforcement Learning, Self-Driving Vehicles, Deep Learning
Abstract: Several key scenarios, such as intersection navigation, lane changing, and ramp merging, are active areas of research in autonomous driving. In order to properly navigate these scenarios, autonomous vehicles must implicitly negotiate with human drivers. Prior work in driving behaviors presents reinforcement learning as a promising technique, as it can leverage data as well as the underlying decision-making structure of driving with interaction. We apply a hierarchical approach to decision-making, where we train a high-level policy using reinforcement learning, and execute the policy’s output on a low-level controller. This hierarchical structure helps increase the policy’s overall safety, and allows the learning component to be agnostic to the low-level control scheme. We validate our approach on a simulation using real-world highway data and find improved results compared to prior work in ramp merging.

Keywords: Unsupervised Learning, Sensor and Data Fusion, Lidar Sensing and Perception
Abstract: LiDAR sensors are crucial in automotive perception for accurate object detection. However, LiDAR data is hard to interpret for humans and consequently time-consuming to label. Whereas camera data is easy interpretable and thus comparably simpler to label. Within this work we present a transductive transfer learning approach to transfer the knowledge for the object detection task from images to point cloud data. We propose a multi-modal adversarial Auto Encoder architecture which disentangles uni-modal features into two groups: common (transferable) features, and complementary (modality-specific) features. This disentanglement is based on the hypothesis that a set of common features exist. An important point of our framework is that the disentanglement is learned in an unsupervised manner. Furthermore, the results show that only a small amount of multi-modal data is needed to learn the disentanglement, and thus to transfer the knowledge between modalities. As a result we our experiments show that training with 75% less data of the KITTI objects, the classification accuracy achieved is of 71.75%, only 3.12% less than when using the full data set. The implications of these findings can have great impact in perception pipelines based on LIDAR data.

Keywords: Smart Infrastructure, Unsupervised Learning, Sensor and Data Fusion
Abstract: The paper presents a method for audio-based vehicle counting (VC) in low-to-moderate traffic using one-channel sound. We formulate VC as a regression problem, i.e., we predict the distance between a vehicle and the microphone. Minima of the proposed distance function correspond to vehicles passing by the microphone. VC is carried out via local minima detection in the predicted distance. We propose to set the minima detection threshold at a point where the probabilities of false positives and false negatives coincide so they statistically cancel each other in total vehicle number. The method is trained and tested on a traffic-monitoring dataset comprising 422 short, 20-second one-channel sound files with a total of 1421 vehicles passing by the microphone. Relative VC error in a traffic location not used in the training is below 2% within a wide range of detection threshold values. Experimental results show that the regression accuracy in noisy environments is improved by introducing a novel high-frequency power feature.

Keywords: Sensor and Data Fusion, Advanced Driver Assistance Systems
Abstract: This work presents a Virtual Sensor for vehicle planar velocities and tire forces. The Virtual Sensor is based on an Extended Kalman Filter utilizing a vehicle model and onboard sensors. The Virtual Sensor is evaluated at different stages, starting from a model with lower complexity, i.e. a 3 degrees of freedom bicycle model, and evolving to higher complexity, i.e. a 10 degrees of freedom vehicle model with wheel suspension. The estimation results from the different stages are compared qualitatively as well as with experimental data. The evaluation brings forth the gains, limitations and drawbacks when increasing model complexity accompanied by sensor additions. This gives insight when a trade-off is required between accuracy and model complexity, e.g. in Advanced Driver Assistance Systems. The conducted analysis also clarifies performance dependencies on the sensors as well as sensor redundancies, which is of added value, e.g. for robustifying against sensor failures.

Keywords: Sensor and Data Fusion, Deep Learning, Image, Radar, Lidar Signal Processing
Abstract: Fusing the raw data from different automotive sensors for real-world environment perception is still challenging due to their different representations and data formats. In this work, we propose a novel method termed High Dimensional Frustum PointNet for 3D object detection in the context of autonomous driving.

Motivated by the goals data diversity and lossless processing of the data, our deep learning approach directly and jointly uses the raw data from camera, LiDAR, and radar.

In more detail, given 2D region proposals and classification from camera images, a high dimensional convolution operator captures local features from a point cloud enhanced with color and temporal information.

Radars are used as adaptive plug-in sensors to refine object detection performance.

As shown by an extensive evaluation on the nuScenes 3D detection benchmark, our network outperforms most of the previous methods.

Keywords: Sensor and Data Fusion, Automated Vehicles, Lidar Sensing and Perception
Abstract: Occupancy grids are widely used in robotics and autonomous systems to create a representation of the environment. Originally designed to map a static environment, recently also dynamic occupancy grids are emerging for the handling of dynamic scenes. However, a major drawback of (dynamic) occupancy grids is the high computational cost (memory and compute) to store and process the information. This is due to the fact, that occupancy grids usually divide the environment in cells of the same size (i.e. uniform grids). As a result, the computational cost increases quadratically with decreasing cell size. Therefore, for many use cases, a trade-off between accuracy (high resolution grid) and distance covered by the grid is required to keep the computational cost in an acceptable range. To overcome this issue we propose in this paper a novel approach for dynamic occupancy grids using non-uniform cell sizes. Our results show, that these non-uniform occupancy grids reduce the numbers of required cells and therefore the computational cost significantly without compromising on the quality of the results.

Keywords: Sensor and Data Fusion, Vehicle Environment Perception, Information Fusion
Abstract: The information of the road grade plays an important role in improving the ride comfort and fuel consumption. This paper proposes a Quaternion Unscented Kalman Filter (QUKF) to estimate the road grade accurately, which needs only measurements from low-cost Inertial Measurement Unit (IMU). The model is built based on the data from accelerometer and gyroscope. The quaternion, which represents orientations and rotations, is chosen to be the state variables, while the three-axle acceleration is set as measurement vector. The proposed observer is tested and verified using the simulation software CarSim and MATLAB Simulink under several scenarios. To compare the performance of the algorithm, the Kalman filter and complementary filter are also implemented under the same simulation conditions. The results illustrate that the presented observer improves the accuracy and stability. Finally, the results of experiments are delivered and the performance of the filter is assessed against the output of a complete GPS/INS available in the same real-world dataset.

Keywords: Sensor and Data Fusion, V2X Communication, Vision Sensing and Perception
Abstract: Cooperative perception with many sensors involved greatly improves the performance of perceptual systems in autonomous vehicles. However, the increasing amount of sensor data leads to a bottleneck due to limited capacity of vehicle-to-X (V2X) communication channels. We leverage lossy learned image compression by means of an autoencoder with adversarial loss function to reduce the overall bitrate. Our key contribution is to focus image compression on regions of interest (ROIs) governed by a binary mask. A transmitter-sided semantic segmentation network extracts semantically important classes being the basis for the generation of a ROI. A second key contribution is that the mask is not transmitted as side information, only the quantized bottleneck data is transmitted. To train the network, we use a loss function operating only on the pixels in the ROI. We report peak-signal-to-noise ratio (PSNR) both in the entire image and only in the ROI, evaluating various fusion architectures and fusion operations involving input image and mask. Showing the high generalizability of our approach, we achieve consistent improvements in the ROI in all experiments on the Cityscapes dataset.

Keywords: Sensor and Data Fusion, Advanced Driver Assistance Systems, Vehicle Environment Perception
Abstract: Lane detection plays a crucial role in any autonomous driving system. Currently commercialized vehicles offer lane keep assist and lane departure warning via integrated smart cameras, deployed for road markings detection. These sensors alone, however, do not generally ensure adequate performance for higher autonomy levels. In this paper, a multi-sensor tracking approach for generic lane boundaries is proposed. This solution is based on well-established filtering techniques and supports a flexible clothoid spline representation. It relies on fine-tuned measurement models, tailored on collected data from both off-the-shelf and prototype smart sensors. The implementation takes into account real-time constraints and ADAS ECUs scarcity of resources. The result is finally validated against lane-level ground truth and experimental data acquisitions.

Keywords: Self-Driving Vehicles, Convolutional Neural Networks, Vulnerable Road-User Safety
Abstract: Following detection and tracking of traffic actors, prediction of their future motion is the next critical component of a self-driving vehicle (SDV) technology, allowing the SDV to operate safely and efficiently in its environment. This is particularly important when it comes to vulnerable road users (VRUs), such as pedestrians and bicyclists. These actors need to be handled with special care due to an increased risk of injury, as well as the fact that their behavior is less predictable than that of motorized actors. To address this issue, in the current study we present a deep learning-based method for predicting VRU movement, where we rasterize high-definition maps and actor's surroundings into a bird's-eye view image used as an input to deep convolutional networks. In addition, we propose a fast architecture suitable for real-time inference, and perform an ablation study of various rasterization approaches to find the optimal choice for accurate prediction. The results strongly indicate benefits of using the proposed approach for motion prediction of VRUs, both in terms of accuracy and latency.

Keywords: Vulnerable Road-User Safety, Sensor and Data Fusion, Intelligent Ground, Air and Space Vehicles
Abstract: This paper summarizes the progress in developing a rugged, low-cost, automated ground cone robot network capable of traffic delineation at lane-level precision. A holonomic omnidirectional base with a traffic delineator was developed to allow flexibility in initialization. RTK GPS was utilized to reduce minimum position error to 2 centimeters. Due to recent developments, the cost of the platform is now less than 1,600. To minimize the effects of GPS-denied environments, wheel encoders and an Extended Kalman Filter were implemented to maintain lane-level accuracy during operation and a maximum error of 1.97 meters through 50 meters with little to no GPS signal. Future work includes increasing the operational speed of the platforms, incorporating lanelet information for path planning, and cross-platform estimation.

Keywords: Vulnerable Road-User Safety
Abstract: The breakthrough of intelligent vehicles will also be determined by the safety gain they provide. In order to perform accident avoiding reactions at the earliest point in time possible, predictions about the future behavior of other traffic participants are needed. The most exposed share of traffic participants regarding this issue are single-track two-wheelers (1T2W): they share the road with cars and trucks but are not as agile due to their kinematics. Furthermore, they are faster than pedestrians but comparably vulnerable as those.

In order to guarantee their safety, we make use of their movement restricting kinematics. We simulate three typical classes of 1T2W under conservative assumptions about their agility in order to generate a spatial region in which they have to be due to physics after a fixed prediction horizon of up to 1.5 seconds. The proposed approach was verified in experiments with real high-dynamic driving maneuvers.

Keywords: Vision Sensing and Perception, Deep Learning, Automated Vehicles
Abstract: Big data leads to a great success of deep learning in computer vision. Unfortunately, big datasets are often not balanced in all dimensions and rare cases are often under-represented. On-board data collection by a moving vehicle can capture thousands of normal pedestrians and vehicles, but what about special persons like police officers, road workers, and school guards? Not only that those types of classes are hard to get, they are crucial to be recognized and classified as such for the task of automated driving. Future self-driving cars need to interact with their environment and need to also understand and follow the signals and instructions of those special persons. In this paper, we show how to classify special person types using Convolutional Neural Networks. The big data imbalance is handled by data augmentation using Generative Models, showing a clear advantage over classical data augmentation. The classification performance of special persons can be significantly improved using our Generative Model based Data Augmentation.

Keywords: Automated Vehicles, Vulnerable Road-User Safety, Recurrent Networks
Abstract: Traffic violation and the flexible and changeable nature of pedestrians make it more difficult to predict pedestrian behavior or intention, which might be a potential safety hazard on the road. Pedestrian motion state (such as walking and standing) directly affects or reflects its intention. In combination with pedestrian motion state and other influencing factors, pedestrian intention can be predicted to avoid unnecessary accidents. In this paper, pedestrian is treated as non-rigid object, which can be represented by a set of two-dimensional key points, and the movement of key point relative to the torso is introduced as micro motion. Static and dynamic micro motion features, such as position, angle and distance, and their differential calculations in time domain, are used to describe its motion pattern. Gated recurrent neural network based seq2seq model is used to learn the dependence of motion state transition on previous information, finally the pedestrian motion state is estimated via a softmax classifier. The proposed method only needs the previous hidden state of GRU and current feature to evaluate the probability of current motion state, and it is computation efficient to deploy on vehicles. This paper verifies the proposed algorithm on the JAAD public dataset, and the accuracy is improved by 11.6% compared with the existing method.

Keywords: Vulnerable Road-User Safety, Recurrent Networks, Automated Vehicles
Abstract: Driving in urban traffic requires making quick and safe decisions while interacting with multiple pedestrians and other road users. Early anticipation of others' intentions is especially important in predicting their future behavior.

In this work, we explore the human ability to estimate intentions of pedestrians in typical urban traffic conditions. Towards this goal, we analyze the results of our large-scale experiment that involved over 700 subjects to establish a human reference point for the task of pedestrian intention estimation. We determine what visual features correlate with human decisions and the relative difficulty of scenarios and validate our conclusion using a linear logistic model. Furthermore, we propose two models to demonstrate the benefits of using intention for pedestrian trajectory and future crossing action prediction. Our experiments show that an improvement of up to 5% can be achieved on both tasks.

Keywords: Convolutional Neural Networks, Deep Learning, Sensor and Data Fusion
Abstract: 3D localization of persons from a single image is a challenging problem, where advances are largely data-driven. In this paper, we enhance the recently released EuroCity Persons detection dataset, a large and diverse automotive dataset covering pedestrians and riders. Previously, only 2D annotations and image data were provided. We introduce an automatic 3D lifting procedure by using additional LiDAR distance measurements, to augment a large part of the reasonable subset of 2D box annotations with their corresponding 3D point positions (136K persons in 46K frames of day- and night-time). The resulting dataset (coined ECP2.5D), now including LiDAR data as well as the generated annotations, is made publicly available for (non-commercial) benchmarking of camera-based and/or LiDAR 3D object detection methods. We provide baseline results for 3D localization from single images by extending the YOLOv3 2D object detector with a distance regression including uncertainty estimation.

Keywords: Vision Sensing and Perception, Vulnerable Road-User Safety, Reinforcement Learning
Abstract: Achieving excellent generalization on perceiving real traffic scenarios with diversity is the long-term goal for building robust autonomous driving systems. In this paper, we propose to discover potential shortness of certain perception module by analyzing its worst-scenario performance. However, with the benchmark datasets growing huge and tremendous, exhaustive searching for the worst perception scenario (WPS) seems to be time consuming and unnecessary. To address, we present an automatic searching scheme empowered by reinforcement learning. In this case, worst scenario mining is formulated as a discrete search problem. A single layer recurrent neural network with LSTM neurons is employed to predict WPS according to the searching reward, which is optimized by a vanilla policy gradient method. Moreover, to deal with the imbalanced distribution of real traffic scenarios, a KNN-like retrieval is utilized for searching closest scenario samples. Effective yet efficient, the proposed method has been validated by finding the most challenging scenarios for various vehicle detectors on KITTI, BDD100k and our own benchmark set EVB. Further experiments reveal that detection networks with structural similarity share the similar WPS.

Keywords: Unsupervised Learning, Deep Learning, Image, Radar, Lidar Signal Processing
Abstract: In the past few years, the performance of road defect detection has been remarkably improved thanks to advancements in various studies on computer vision and deep learning. Although large-scale and well-annotated datasets enhance the performance of detecting road defects to some extent, it is still challengeable to derive a model which can perform reliably for various road conditions in practice, because it is intractable to construct a dataset considering diverse road conditions and defect patterns. To end this, we propose an unsupervised approach to detect road defects, using Adversarial Image-to-Frequency Transform (AIFT). AIFT adopts the unsupervised manner and adversarial learning in deriving the defect detection model, so AIFT does not require annotations for road defects. We evaluate the efficiency of AIFT using GAPs384 dataset, Cracktree200 dataset, CRACK500 dataset, and CFD dataset. The experimental results demonstrate that the proposed approach detects various road detects, and it outperforms existing state-of-the-art approaches.

Keywords: Advanced Driver Assistance Systems, Deep Learning, Recurrent Networks
Abstract: For the path planning of autonomous vehicles, it is important to predict the future trajectory of the surrounding vehicles. However, predicting future trajectory is difficult because it needs to consider the invisible interaction between the vehicles in a dynamic driving environment. In this paper, a new approach, which considers the interaction between surrounding vehicles, is proposed for accurate prediction of the future trajectory. The proposed method provides continuous predicted trajectories over time in the longitudinal and lateral directions, respectively. The deep ensemble technique is also used to predict the uncertainty of the estimated trajectory. This paper performs the training and verification of the algorithm using NGSIM dataset, which is the vehicle driving data obtained through actual vehicle driving.

Keywords: Autonomous / Intelligent Robotic Vehicles, Deep Learning, Self-Driving Vehicles
Abstract: This paper presents online-capable deep learning model for probabilistic vehicle trajectory prediction. We propose a simple encoder-decoder architecture based on multi-head attention. The proposed model generates the distribution of the predicted trajectories for multiple vehicles in parallel. Our approach to model the interactions can learn to attend to a few influential vehicles in an unsupervised manner, which can improve the interpretability of the network. The experiments using naturalistic trajectories at highway show the clear improvement in terms of positional error on both longitudinal and lateral direction.

Keywords: Deep Learning, Cooperative ITS, Automated Vehicles
Abstract: Efficient driving in urban traffic scenarios requires foresight. The observation of other traffic participants and the inference of their possible next actions depending on the own action is considered cooperative prediction and planning. Humans are well equipped with the capability to predict the actions of multiple interacting traffic participants and plan accordingly, without the need to directly communicate with others. Prior work has shown that it is possible to achieve effective cooperative planning without the need for explicit communication. However, the search space for cooperative plans is so large that most of the computational budget is spent on exploring the search space in unpromising regions that are far away from the solution. To accelerate the planning process, we combined learned heuristics with a cooperative planning method to guide the search towards regions with promising actions, yielding better solutions at lower computational costs.

Keywords: Self-Driving Vehicles, Advanced Driver Assistance Systems, Deep Learning
Abstract: Traffic light detection by camera is a challenging task for autonomous driving mainly due to the small size of traffic lights in the road scene especially for early detection. The limited resolution in the corresponding area of traffic lights reduces their contrast to the background, as well as the effectiveness of the visual cues from the traffic light itself. We believe understanding the scene semantics between traffic lights and their surroundings can play a vital role in tackling this challenge. Towards this goal, we build a generative adversarial network (GAN) model to predict the existence of traffic lights from the road scene image where existing traffic lights are removed with image inpainting. %and anchor center sampling. Using Cityscape dataset, we verify that the proposed GAN model indeed captures the desired semantics by showing effective predictions of existence of traffic lights that are consistent with real images. Moreover, we leverage this model to augment the training data where traffic lights are inserted to the road scene images based on the prediction of the GAN model. While the augmented images may not be realistic looking, results show that such data augmentation can improve the traffic light detector performance that is comparable to using additional real data collection, and better than other data augmentation with various randomization schemes. These results verify the importance of semantic consistency for data augmentation to improve the traffic light detection.

Keywords: Autonomous / Intelligent Robotic Vehicles, Vehicle Control, Intelligent Ground, Air and Space Vehicles
Abstract: In this work we evaluate Iterative Linear Quadratic Regulator(ILQR) for trajectory tracking of two different kinds of wheeled mobile robots namely Warthog , an off-road holonomic robot with skid-steering and Polaris GEM e6, a non-holonomic six seater vehicle. We use multilayer neural network to learn the discrete dynamic model of these robots which is used in ILQR controller to compute the control law. We use model predictive control (MPC) to deal with model imperfections and perform extensive experiments to evaluate the performance of the controller on human driven reference trajectories with vehicle speeds of 3m/s- 4m/s for warthog and 7m/s-10m/s for the Polaris GEM

Keywords: Advanced Driver Assistance Systems, Reinforcement Learning, Automated Vehicles
Abstract: Lane-change maneuvers are commonly executed by drivers to follow a certain routing plan, overtake a slower vehicle, adapt to a merging lane ahead, etc. However, improper lane change behaviors can be a major cause of traffic flow disruptions and even crashes. While many rule-based methods have been proposed to solve lane change problems for autonomous driving, they tend to exhibit limited performance due to the uncertainty and complexity of the driving environment. Machine learning-based methods offer an alternative approach, as Deep reinforcement learning (DRL) has shown promising success in many application domains including robotic manipulation, navigation, and playing video games. However, applying DRL for autonomous driving still faces many practical challenges in terms of slow learning rates, sample inefficiency, and non-stationary trajectories. In this study, we propose an automated lane change strategy using proximal policy optimization-based deep reinforcement learning, which shows great advantage in learning efficiency while maintaining stable performance. The trained agent is able to learn a smooth, safe, and efficient driving policy to determine lane-change decisions (i.e. when and how) even in dense traffic scenarios. The effectiveness of the proposed policy is validated using task success rate and collision rate, which demonstrates the lane change maneuvers can be efficiently learned and executed in a safe, smooth and efficient manner.

Keywords: Deep Learning, Sensor and Data Fusion
Abstract: Recent work has shown that it is possible to learn neural networks with provable guarantees on the output of the model when subject to input perturbations, however these works have focused primarily on defending against adversarial examples for image classifiers. In this paper, we study how these provable guarantees can be naturally applied to other real world settings, namely getting performance specifications for robust virtual sensors measuring fuel injection quantities within an engine. We first demonstrate that, in this setting, even simple neural network models are highly susceptible to reasonable levels of adversarial sensor noise, which are capable of increasing the mean relative error of a standard neural network from 6.6% to 43.8%. We then leverage methods for learning provably robust networks and verifying robustness properties, resulting in a robust model which we can provably guarantee has at most 16.5% mean relative error under any sensor noise. Additionally, we show how specific intervals of fuel injection quantities can be targeted to maximize robustness for certain ranges, allowing us to train a virtual sensor for fuel injection which is provably guaranteed to have at most 10.69% relative error under noise while maintaining 3% relative error on non-adversarial data within normalized fuel injection ranges of 0.6 to 1.0.

Keywords: Advanced Driver Assistance Systems, Vehicle Control, Information Fusion
Abstract: Automated driving is one of the main drivers in the automotive industry. On the way to full automation current Advanced Driver Assistant Systems (ADAS) and Automated Driving Systems (ADS) backed by new and enhanced sensor systems take over more and more driving tasks. Developers and engineers are challenged with the increasing Operational Design Domain (ODD) of their systems. The application of these systems has become a daunting task, as a manifold of new situations has to be covered. The number of application parameters has skyrocketed and their scopes are intertwined and not always visible to the vehicle’s driver. Virtual, simulation based approaches are on the rise to give developers another tool for the application of their systems. In order to retrieve valid evaluations from the simulation, current research focuses on objectifying the subjective passenger assessments that so far were only conceivable in real world driving tests. Furthermore, with objective and comparable simulation results on statistically significant data samples, application parameters whose effects are not clearly visible in real world driving tests can now be applied systematically and justified. We propose a process reference model for the virtual application of predictive control features with a clear focus on the quality and representativity of the virtual application.

Keywords: Lidar Sensing and Perception, Vision Sensing and Perception, Sensor and Data Fusion
Abstract: This work presents an extrinsic parameter estimation algorithm between a 3D LIDAR and a Projective Camera using a marker-less planar target, by exploiting Planar Surface Point to Plane and Planar Edge Point to back-projected Plane geometric constraints. The proposed method uses the data collected by placing the planar board at different poses in the common field of view of the LIDAR and the Camera. The steps include, detection of the target and the edges of the target in LIDAR and Camera frames, matching the detected planes and lines across both the sensing modalities and finally solving a cost function formed by the aforementioned geometric constraints that link the features detected in both the LIDAR and the Camera using non-linear least squares. We have extensively validated our algorithm using two Basler Cameras, Velodyne VLP-32 and Ouster OS1 LIDARs.

Keywords: Information Fusion, Vehicle Control, Deep Learning
Abstract: This paper proposes a road friction coefficient estimation method based on the fusion of machine vision and vehicle dynamics. A vehicle-mounted camera is used to obtain the front image. Based on the deep learning method, the road surface in the image is segmented and identified to obtain the road type. Besides, a road friction estimator with the tire longitudinal force estimation is designed. With the visual estimation results, a fusion estimation method is designed based on the structural parameter optimization. The results of simulation experiments show that the fusion estimator has the characteristics of fast convergence and high accuracy.

Keywords: Autonomous / Intelligent Robotic Vehicles, Unsupervised Learning, Information Fusion
Abstract: Road friction estimation concerns inference of the coefficient between the tire and road surface to facilitate active safety features. Current state-of-the-art methods lack generalization capability to cope with different tire character- istics and models are restricted when using Bayesian inference in estimation while recent supervised learning methods lack uncertainty prediction on estimates. This paper introduces variational inference to approximate intractable posterior of friction estimates and learns an amortized variational inference model from tire measurement data to facilitate probabilistic estimation while sustaining the flexibility of tire models. As a by-product, a probabilistic tire model can be learned jointly with friction estimator model. Experiments on simulated and field test data show that the learned friction estimator provides accurate estimates with robust uncertainty measures in a wide range of tire excitation levels. Meanwhile, the learned tire model reflects well-studied tire characteristics from field test data.

Keywords: Information Fusion, Lidar Sensing and Perception, Deep Learning
Abstract: In object detection for autonomous driving and robotic applications, conventional RGB cameras often fail to sense objects under extreme illumination conditions and on texture-less surfaces, while LIDAR sensors often fail to sense small or thin objects located far from the sensor. For these reasons, an intuitive and obvious choice for perception system designers is to install multiple sensors of different modalities to increase (in theory) the detection robustness. In this paper we focus on the analysis of an object detector that performs early fusion of RGB images and LIDAR 3D points. Our goal is to go beyond the intuition of simply adding more sensor modalities to improve performance, and instead analyze, quantify, and understand the performance differences, strengths and weaknesses of the object detector under three different modalities: 1) RGBonly, 2) LIDAR-only, and 3) Early fusion (RGB and LIDAR), and under two key scene variables: 1) Distance of objects from the sensor (density), and 2) Illumination (Darkness). We also propose methodologies to generate 2D weak semantic training masks, and a methodology to evaluate the object detection performance separately at different distance ranges, which provides a more reliable detection performance measure and correlates well with object LIDAR point density.

Keywords: Autonomous / Intelligent Robotic Vehicles, Deep Learning, Information Fusion
Abstract: Forecasting the motion of surrounding obstacles (vehicles, bicycles, pedestrians and etc.) benefits the on-road motion planning for intelligent and autonomous vehicles. Complex scenes always yield great challenges in modeling the patterns of surrounding traffic. For example, one main challenge comes from the intractable interaction effects in a complex traffic system. In this paper, we propose a multi-layer architecture Interaction-aware Kalman Neural Networks (IaKNN) which involves an interaction layer for resolving high-dimensional traffic environmental observations as interaction-aware accelerations, a motion layer for transforming the accelerations to interaction-aware trajectories, and a filter layer for estimating future trajectories with a Kalman filter network. Attributed to the multiple traffic data sources, our end-to-end trainable approach technically fuses dynamic and interaction-aware trajectories boosting the prediction performance. Experiments on the NGSIM dataset demonstrate that IaKNN outperforms the state-of-the-art methods in terms of effectiveness for traffic trajectory prediction.

Keywords: Vulnerable Road-User Safety, Recurrent Networks, Convolutional Neural Networks
Abstract: Pedestrian crossing action prediction is a crucial task for autonomous driving. Numerous studies show that an early estimation of the pedestrian's intention can decrease or even avoid a high percentage of accidents. In this paper, different variations of a deep learning system are proposed to attempt to solve this problem. The proposed models are composed of two parts: a CNN-based feature extractor and an RNN module. All the models were trained and tested on the JAAD dataset. The results obtained indicate that the choice of the features extraction method, the inclusion of contextual variables and the chosen RNN type have a great impact on the final performance

Keywords: Vulnerable Road-User Safety, Advanced Driver Assistance Systems, Collision Avoidance
Abstract: Vehicle-pedestrian interaction (VPI) is one of the most challenging tasks for automated driving systems. The design of driving strategies for such systems usually starts with verifying VPI in simulation. This work proposed an improved framework for the study of VPI in uncontrolled pedestrian crossing scenarios. The framework admits the mutual effect between the pedestrian and the vehicle. A multi-state social force based pedestrian motion model was designed to describe the microscopic motion of the pedestrian crossing behavior. The pedestrian model considers major interaction factors such as the accepted gap of the pedestrian's decision on when to start crossing, the desired speed of the pedestrian, and the effect of the vehicle on the pedestrian while the pedestrian is crossing the road. Vehicle driving strategies focus on the longitudinal motion control, for which the feedback obstacle avoidance control and the model predictive control were tested and compared in the framework. The simulation results verified that the proposed framework can generate a variety of VPI scenarios, consisting of either the pedestrian yielding to the vehicle or the vehicle yielding to the pedestrian. The framework can be easily extended to apply different approaches to the VPI problems.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception
Abstract: This paper presents an experimental study on 3D person localization (i.e. pedestrians, cyclists) in traffic scenes, using monocular vision and LiDAR data. We first analyze the detection performance of two top-ranking methods (PointPillars and AVOD) on the KITTI benchmark, with respect to varying Intersection over Union (IoU) settings and the underlying parameters of 3D bounding box location, extent and orientation. Given that the KITTI dataset contains relatively few 3D person instances, we also consider the new EuroCity Persons 2.5D (ECP2.5D) dataset, which is one order of magnitude larger. We perform domain transfer experiments between the KITTI and ECP2.5D datasets, to examine how these datasets generalize with respect to each other.

Keywords: Human-Machine Interface, Novel Interfaces and Displays, Self-Driving Vehicles
Abstract: Autonomous Vehicles (AV) will transform transportation, but also the interaction between vehicles and pedestrians. In the absence of a driver, it is not clear how an AV can communicate its intention to pedestrians. One option is to use visual signals. To advance their design, we conduct four human-participant experiments and evaluate six representative AV visual signals for visibility, intuitiveness, persuasiveness, and usability at pedestrian crossings. Based on the results, we distill twelve practical design recommendations for AV visual signals, with focus on signal pattern design and placement. Moreover, the paper advances the methodology for experimental evaluation of visual signals, including lab, closed-course, and public road tests using an autonomous vehicle. In addition, the paper also reports insights on pedestrian crosswalk behaviours and the impacts of pedestrian trust towards AVs on the behaviors. We hope that this work will constitute valuable input to the ongoing development of international standards for AV lamps, and thus help mature automated driving in general.

Keywords: Human-Machine Interface, Advanced Driver Assistance Systems, Cooperative ITS
Abstract: As vulnerable road users, cyclists were often killed or injured because they were not perceived by drivers on rural roads on time. Nowadays, many sensor technology and assistance systems are used to improve traffic safety and efficiency. However, few studies focus on the interaction between drivers and cyclists, especially enhancing drivers’ perception of cyclists. Following a cooperative support framework between drivers and vehicles, we designed an interface to improve drivers’ perception of cyclists using Augmented Reality (AR) and then evaluated it in the driving simulator. The results show that driving behavior regarding the distance to the cyclist shows a significant change while the driver is supported by the developed AR interface. Consequently, the results implicate that the safety of vulnerable traffic participants can be increased, using a cooperation strategy between the driver and the vehicle. The behaviour while driving manually can be persuaded by the system without any distraction of the driver.

Keywords: Vision Sensing and Perception, Human-Machine Interface, Vulnerable Road-User Safety
Abstract: Gestures from traffic police give the authorized information, especially in some urgent situation. Thus, understanding of traffic police instruction accurately and promptly is particularly crucial for the automated driving system. However, this task is a great challenge not only because of the dynamic and diversity characteristics of the human gesture, but also the high requirement for real-time performance in each frame. We propose an online activity recognition method based on pose estimation and Graph Convolutional Networks (GCN) to recognize the traffic police gesture in frame level. The main contribution in this work is the development of an online framework based on graph convolutional networks for traffic police recognition. Our approach obtained the state-of-the-art results on Traffic Police Gesture Recognition (TPGR) dataset.

Keywords: Vehicle Environment Perception, Vulnerable Road-User Safety, Automated Vehicles
Abstract: Risk mitigation is an important element to consider in risk evaluation. It is known that safety features have helped to decrease the death ratio over the years. However, to date, each driver assistance system works on a single domain of operation. The problem still remains in how to use perception to contextualize the scene in order to fully minimize the severity of a collision in a complex emergency scenario. Up to now, works on cost maps have consider simple contextualized object in mitigation scenarios. For instance, the use of binary allowed/forbidden zones or, a fixed weight to each type of object in the scene. In this work, the risk of injury issued by accidentology is employed to each class of object present in the scene. Each class of object presents then a probability of injury with respect to the impact speed and ethical/economical/political factors. The method generates a cost map containing the probability of collision associated to the risk of injury. It dynamically contextualizes the objects, since the risk of injury depends on the characteristics of the scene. Simulation and dataset results validate that changing the referred parameters alters the context and evaluation of the scene. Then, the proposed methodology allows a better assessment of the surroundings by creating a dynamic navigation cost map for complex scenarios.

Keywords: Vulnerable Road-User Safety, Mapping and Localization, Sensor and Data Fusion
Abstract: Over the last decade, the number of accidents involving Vulnerable Road Users (VRU), i.e. pedestrians, cyclists and motorbike drivers, has not decreased in the same way as accidents between passenger cars have. Cooperative systems based on Vehicle-to-X (V2X) communication make it possible to directly exchange information between VRUs and vehicles and to increase the overall situational awareness beyond the capabilities of on-board ranging sensors. To detect and avoid collisions, vehicles require up-to-date and precise information on the location and trajectory of VRUs. In this paper, we propose a VRU localization system based on Global Navigation Satellite System (GNSS), inertial sensors and ultra-wideband (UWB) round-trip-delay ranging technology. We present an exhaustive measurement campaign comprising pedestrians, cyclists and vehicles performed in an urban setting and show first results on the localization performance for a pedestrian crossing an intersection. In the experiments, the pedestrian inertial system supported by GNSS and UWB ranges is able to achieve 0.65m 1-sigma-position accuracy.

Keywords: Mapping and Localization, Sensor and Data Fusion, Self-Driving Vehicles
Abstract: Augmented reality visualizes map content or sensor observations by projections that fit naturally into the surroundings, thereby providing relevant information in a convenient and safe way for the customer. A stable projection of map data into the driver’s field of view requires precise knowledge of the vehicle’s pose in six degrees of freedom (dof) with respect to a map. We developed an efficient 6dof localization based on odometry, camera recognized landmarks and a high definition (HD) map using an extended Kalman filter (EKF). As a result, we compared the accuracy of two fusion methods on simulated driving scenarios based on real HD map data: (i) fusing 2D landmarks from the image plane directly, and (ii) fusing 3D landmarks assuming prior 3D reconstruction. Using five types of traffic signs and pole-like landmarks, we found that the accuracy difference between the approaches reduces with increasing number of landmarks per cycle. In presence of five or more landmarks per cycle, the 2D measurement approach reaches the accuracy of the 3D measurement approach. Thus, with many landmark types, the 2D approach becomes a beneficial alternative: While achieving comparable accuracy, it circumvents 3D reconstruction, thereby reducing the computational effort, which is highly desirable in autonomous driving.

Keywords: Mapping and Localization, Lidar Sensing and Perception, Vehicle Environment Perception
Abstract: This paper presents a LiDAR-based 3D Monte Carlo localization (MCL) with an efficient distance field (DF) representation method. To implement 3D MCL, high computing capacity is required because the likelihood of many pose candidates, i.e., particles, must be calculated in real time by comparing sensor measurements and a map. Additionally, a large-scale map is needed for allocation to embedded computers since autonomous vehicles are required to navigate wide areas. These make it difficult for 3D MCL implementation. This paper first presents an efficient DF representation method while considering the 3D LiDAR-based localization characteristics. Because each DF voxel has the closest distance from occupied voxels, swift comparison of the sensor measurements and map can be achieved. Consequently, 3D MCL using the likelihood field model (LFM) can be executed in real time. Furthermore, this paper presents a method for improving the localization robustness to environmental changes without increasing memory and computational cost from that of the LFM-based MCL. Through experiments using the SemanticKITTI dataset, we show that the presented method can efficiently and robustly work in dynamic environments.

Keywords: Mapping and Localization, Self-Driving Vehicles, Lidar Sensing and Perception
Abstract: In this paper, we present SROM, a novel real-time Simultaneous Localization and Mapping (SLAM) system for autonomous vehicles. The keynote of the paper showcases SROM's ability to maintain localization at low sampling rates or at high linear or angular velocities where most popular LiDAR based localization approaches get degraded fast. We also demonstrate SROM to be computationally efficient and capable of handling high-speed maneuvers. It also achieves low drifts without the need for any other sensors like IMU and/or GPS. Our method has a two-layer structure wherein first, an approximate estimate of the rotation angle and translation parameters are calculated using a Phase Only Correlation (POC) method. Next, we use this estimate as an initialization for a point-to-plane ICP algorithm to obtain fine matching and registration. Another key feature of the proposed algorithm is the removal of dynamic objects before matching the scans. This improves the performance of our system as the dynamic objects can corrupt the matching scheme and derail localization. Our SLAM system can build reliable maps at the same time generating high-quality odometry. We exhaustively evaluated the proposed method in many challenging highways/country/urban sequences from the KITTI dataset and the results demonstrate better accuracy in comparisons to other state-of-the-art methods with reduced computational expense aiding in real-time realizations. We have also integrated our SROM system with our in-house autonomous vehicle and compared it with the state-of-the-art methods like LOAM and LeGO-LOAM.

Keywords: Self-Driving Vehicles, Lidar Sensing and Perception, Image, Radar, Lidar Signal Processing
Abstract: In the paper a fast and consistent method to manage uncertainties on detected traffic agents providing reliable results is presented. The information provided by a LiDAR- based object detector is combined with a high-definition map to identify the drivable space of the carriageway. Because the use of a HD map requires the use of a localization system, the uncertainty of the estimated pose shall be handled carefully. A novel approach taking into account the localization uncertainty in the perception task by direct propagation of it onto the LiDAR points is proposed. It is compared with a classical propagation that relies on linearized approximation. The good performances of this approach in terms of integrity are demonstrated by the use of real data acquired at the entrance of a roundabout being a particularly complex situation.

Keywords: Lidar Sensing and Perception, Self-Driving Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: LIDAR sensors are bound to become one of the core sensors in achieving full autonomy for self driving cars. LIDARs are able to produce rich, dense and precise spatial data, which can tremendously help in localizing and tracking a moving vehicle. In this paper, we review the latest finding in 3D LIDAR localization for autonomous driving cars, and analyze the results obtained by each method, in an effort to guide the research community towards the path that seems to be the most promising.

Keywords: Automated Vehicles, Vehicle Environment Perception, Deep Learning
Abstract: Visual localization in vehicle navigation remains a crucial image retrieval task to determine the best matched image. Developing an efficient algorithm to address the localization issues of vehicle is highly difficult, for severe appearance variations with vehicles moving around can bring about significant challenges and big obstacles. In this paper, we propose the CFVL framework which takes panoramas into use in the localizer and the system processes from coarse to fine, in order to attain more robust and stable descriptors. NetVALD descriptors based on explicit panorama construction, which are regarded robust to appearance changes, are extracted in the coarse stage, while Geodesc keypoint descriptors, which are believed to detect more detailed information, are utilized in the fine stage, so as to perceive the accurate localization. A comprehensive set of experiments is carried on several datasets with different appearances across seasonal cycling, illumination variations, diverse traversals, and so on, to verify the effectiveness of the coarse stage and fine stage in our system. Brute Force (BF) matching and Fundamental Matrix mapping are utilized to match and locate correct locations after coarse stage and after fine stage. The accuracy of the coarse matching and fine matching are verified separately. Our system is demonstrated to be with high location recall, generalization capacity across different environments.

Keywords: Lidar Sensing and Perception, Deep Learning, Vehicle Environment Perception
Abstract: In this paper, we consider the transformation of laser range measurements into a top-view grid map representation to approach the task of LiDAR-only semantic segmentation. Since the recent publication of the SemanticKITTI data set, researchers are now able to study semantic segmentation of urban LiDAR sequences based on a reasonable amount of data. While other approaches propose to directly learn on the 3D point clouds, we are exploiting a grid map framework to extract relevant information and represent them by using multi-layer grid maps. This representation allows us to use well-studied deep learning architectures from the image domain to predict a dense semantic grid map using only the sparse input data of a single LiDAR scan. We compare single-layer and multi-layer approaches and demonstrate the benefit of a multi-layer grid map input. Since the grid map representation allows us to predict a dense, 360° semantic environment representation, we further develop a method to combine the semantic information from multiple scans and create dense ground truth grids. This method allows us to evaluate and compare the performance of our models not only based on grid cells with a detection, but on the full visible measurement range.

Keywords: Automated Vehicles, Situation Analysis and Planning, Active and Passive Vehicle Safety
Abstract: Vehicles with Automated Driving Systems (ADS) operate in a high-dimensional continuous system with multi-agent interactions. This continuous system features various types of traffic agents (non-homogeneous) governed by continuous-motion ordinary differential equations (differential-drive). Each agent makes decisions independently that may lead to conflicts with the subject vehicle (SV), as well as other participants (non-cooperative). A typical vehicle safety evaluation procedure that uses various safety-critical scenarios and observes resultant collisions (or near collisions), is not sufficient enough to evaluate the performance of the ADS in terms of operational safety status maintenance. In this paper, we introduce a emph{Model Predictive Instantaneous Safety Metric} (MPrISM), which determines the safety status of the SV, considering the worst-case safety scenario for a given traffic snapshot. The method then analyzes the SV's closeness to a potential collision within a certain evaluation time period. The described metric induces theoretical guarantees of safety in terms of the time to collision under standard assumptions. Through formulating the solution as a series of minimax quadratic optimization problems of a specific structure, the method is tractable for real-time safety evaluation applications. Its capabilities are demonstrated with synthesized examples and cases derived from real-world tests.

Keywords: Self-Driving Vehicles, Situation Analysis and Planning, Recurrent Networks
Abstract: Failures in autonomous driving caused by complex traffic situations or model inaccuracies remain inevitable in the near future. While much research is focused on how to prevent such failures, comparatively little research has been done on predicting them. An early failure prediction would allow for more time to take actions to resolve challenging situations. In this work, we propose an introspective approach to predict future disengagements of the car by learning from previous disengagement sequences. Our method is designed to detect failures as early as possible by using sensor data from up to ten seconds before each disengagement. The car itself is treated as a black box, with only its state data and the number of detected objects being required. Since no model-specific knowledge is needed, our method is applicable to any self-driving system. Currently, no public data of real-life disengagements is available. To test our approach, we therefore use autonomous driving data provided by BMW that was collected with BMW research vehicles over multiple months. We show that an LSTM classifier trained with sequences of state data can predict failures up to seven seconds in advance with an accuracy of more than 80%. This is two seconds earlier than comparable approaches from the literature.

Keywords: Situation Analysis and Planning, Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles
Abstract: Mixed-Integer Quadratic Programming (MIQP) has been identified as a suitable approach for finding an optimal solution to the behavior planning problem with low runtimes. Logical constraints and continuous equations are optimized alongside. However, it has only been formulated for a straight road, omitting common situations such as taking turns at intersections. This has prevented the model from being used in reality so far. Based on a triple integrator model formulation, we compute the orientation of the vehicle and model it in a disjunctive manner. That allows us to formulate linear constraints to account for the non-holonomy and collision avoidance. These constraints are approximations, for which we introduce the theory. We show the applicability in two benchmark scenarios and prove the feasibility by solving the same models using nonlinear optimization. This new model will allow researchers to leverage the benefits of MIQP, such as logical constraints, or global optimality.

Keywords: Automated Vehicles, Situation Analysis and Planning, Active and Passive Vehicle Safety
Abstract: On the path towards mass deployment of automated vehicles (AVs) several challenges have still to be resolved. One of these is the development of approaches that allow the safe operation of an AV within uncertain environments, while not imposing excessive safety margins. Existing proposals, such as the Responsibility Sensitive Safety (RSS) approach from Intel/Mobileye, are a good first step towards this goal. At the same time, these approaches are often too conservative, and hence further improvements are required. However, this behavior is not due to their underlying semantic rules, rather it is a problem of proper situation assessment within the models. Therefore, we propose to consider the risk of a driving situation inside RSS. The result is a novel risk-aware RSS approach, which allows to significantly reduce the safety margins (i.e. increased traffic density) in a situation-dependent manner, while risk limits are maintained.

Keywords: Automated Vehicles, Deep Learning, Situation Analysis and Planning
Abstract: Automated vehicles rely heavily on data-driven methods, especially for complex urban environments. Large datasets of real world measurement data in the form of road user trajectories are crucial for several tasks like road user prediction models or scenario-based safety validation. So far, though, this demand is unmet as no public dataset of urban road user trajectories is available in an appropriate size, quality and variety. By contrast, the highway drone dataset (highD) has recently shown that drones are an efficient method for acquiring naturalistic road user trajectories. Compared to driving studies or ground-level infrastructure sensors, one major advantage of using a drone is the possibility to record naturalistic behavior, as road users do not notice measurements taking place. Due to the ideal viewing angle, an entire intersection scenario can be measured with significantly less occlusion than with sensors at ground level. Therefore, we created a comprehensive, large-scale urban intersection dataset with naturalistic road user behavior using camera-equipped drones as successor of the highD dataset. The resulting dataset contains more than 13 500 road users including vehicles, bicyclists and pedestrians at intersections in Germany and is called inD. The dataset consists of 10 hours of measurement data from four intersections and is available online for non-commercial research at: https://www.inD-dataset.com

Keywords: Vehicle Environment Perception, Situation Analysis and Planning, Automated Vehicles
Abstract: Reliable representation of multi-vehicle interactions in urban traffic is pivotal but challenging for autonomous vehicles due to the volatility of the traffic environment, such as roundabouts and intersections. This paper describes a semi-stochastic potential field approach to represent multi-vehicle interactions by integrating a deterministic field approach with a stochastic one. First, we conduct a comprehensive evaluation of potential fields for representing multi-agent intersections from the deterministic and stochastic perspectives. For the former, the estimates at each location in the region of interest (ROI) are deterministic, which is usually built using a family of parameterized exponential functions directly. For the latter, the estimates are stochastic and specified by a random variable, which is usually built based on stochastic processes such as the Gaussian process. Our proposed semi-stochastic potential field, combining the best of both, is validated based on the INTERACTION dataset collected in complicated real-world urban settings, including intersections and roundabout. Results demonstrate that our approach can capture more valuable information than either the deterministic or stochastic ones alone. This work sheds light on the development of algorithms in decision-making, path/motion planning, and navigation for autonomous vehicles in the cluttered urban settings.

Keywords: Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles, Situation Analysis and Planning
Abstract: This paper characterizes safe following distances for on-road driving when vehicles can avoid collisions by either braking or by swerving into an adjacent lane. In particular, we focus on safety as defined in the Responsibility-Sensitive Safety (RSS) framework. We extend RSS by introducing swerve maneuvers as a valid response in addition to the already present brake maneuver. These swerve maneuvers use the more realistic kinematic bicycle model rather than the double integrator model of RSS. We show that these swerve maneuvers allow a vehicle to safely follow a lead vehicle more closely than the RSS braking maneuvers do. The use of the kinematic bicycle model is then validated by comparing these swerve maneuvers to swerves of a dynamic single-track model. The analysis in this paper can be used to inform both offline safety validation as well as safe control and planning.

Keywords: Autonomous / Intelligent Robotic Vehicles, Advanced Driver Assistance Systems, Situation Analysis and Planning
Abstract: Autonomously following a leading vehicle is a major step towards fully autonomous vehicles. The contribution of this work consists in the development, implementation, and validation of two following modes: 1) Exact Following: accurate compliance with the reference path. 2) Flexible Following: tolerate deviation from the reference path in order to avoid obstacles. The proposed method can easily be integrated into existing frameworks for autonomous vehicles. Therefore our approach is flexible enough to be applied to a large variety of different vehicles. To demonstrate the feasibility of our approach an experimental validation is carried out on two autonomous vehicles with major differences in kinematics, weight, and size: A cross-country wheelchair and an off-road truck. Both exact and flexible following have been successfully demonstrated in unstructured outdoor environments.

Keywords: Automated Vehicles, Intelligent Vehicle Software Infrastructure
Abstract: Autonomous driving (AD) is getting closer to our life, but the severe traffic accidents of autonomous vehicle (AV) happened in the past several years warn us that the safety of AVs is still a big challenge for the AD industry. Before volume production, the automotive industry and regulators must ensure the AV can deal with dangerous scenarios. Although road test is the most common method to test the performance and safety of an AV, it has some manifest disadvantages, e.g., highly risky and unrepeatable, low efficiency and lack of useful critical scenarios. Critical-scenario-based simulation can effectively address these problems and become an important complement to road test. In this paper, we present a novel approach to extract critical scenarios from real traffic accident videos and re-generate them in a simulator. We also introduce our integrated toolkit for scenario extraction and scenario test. With the toolkit, we can build a critical scenario library quickly and use it as a benchmark for AV safety assessment, among other purposes. On top of this, we further introduce our safety assessment criteria and scoring method.

Keywords: Smart Infrastructure, Vulnerable Road-User Safety
Abstract: High quality reference data is crucial for the development of autonomous driving applications. Unfortunately, datasets including fixed, reproducible static environments that contain manifold interactions between traffic participants are not widely available.

In this paper we propose a camera based trajectory estimation framework that enables the generation of reference trajectory data in stationary roadside infrastructure. We develop a Simple Online Realtime Tracking (SORT) algorithm that tracks objects in image space utilizing the tracking-by-detection paradigm with a deep neural network detector. By projecting tracks to a ground model, we are able to gather cartesian and georeferenced trajectories for manually driven and autonomous vehicles in the field. We evaluate the framework in stationary roadside infrastructure in the Test Area Autonomous Driving Baden-Württemberg, Germany. A vehicle equipped with inertial measurement unit and differential GPS is used to generate ground truth positions that are compared with our framework.

Keywords: Reinforcement Learning, Self-Driving Vehicles, Collision Avoidance
Abstract: Automated lane changing is a critical feature for advanced autonomous driving systems. In recent years, reinforcement learning (RL) algorithms trained on traffic simulators yielded successful results in computing lane changing policies that strike a balance between safety, agility and compensating for traffic uncertainty. However, many RL algorithms exhibit simulator bias and policies trained on simple simulators do not generalize well to realistic traffic scenarios. In this work, we develop a data driven traffic simulator by training a generative adverserial network (GAN) on real life trajectory data. The simulator generates randomized trajectories that resembles real life traffic interactions between vehicles, which enables training the RL agent on much richer and realistic scenarios. We demonstrate through simulations that RL agents that are trained on GAN-based traffic simulator has stronger generalization capabilities compared to RL agents trained on simple rule-driven simulators.

Keywords: Advanced Driver Assistance Systems, Intelligent Vehicle Software Infrastructure
Abstract: Context: We consider the structure of service-oriented architectures in vehicular software.

Aim: We aim at evaluating the structure and grouping of service architectures.

Method: We propose and discuss architectural metrics tailored towards automotive service-oriented architectures. We apply the metrics on an adaptive cruise control case example extracted from the AUTOSAR standard.

Results: The application of the proposed metrics to two different service groupings for ACC points clearly to the same service grouping that we consider, after a thorough analysis, to be better with respect to coupling and cohesion attributes.

Conclusion: We demonstrate the usefulness of proposed service group metrics in early design phases of the development process and validate the metrics on the case example of an adaptive cruise control function.

Keywords: V2X Communication, Cooperative Systems (V2X)
Abstract: Wireless congestion control and resource allocation for 802.11p based V2X safety communication have been widely investigated for a single Cooperative Awareness service, considering homogeneous resource requirement per vehicle. Future cooperative connected vehicles, will have heterogeneous capabilities and communication needs, which existing congestion control mechanisms have not fully addressed.

In this paper, we analyze issues with the channel congestion control protocol standardized in Europe by ETSI, regarding distributed resource allocation for heterogeneous number of services and message types per vehicle. We present a cooperative congestion control mechanism to orchestrate channel resource among a mixed distribution of vehicles with diverse resource requirements under channel congestion. Simulation based evaluation using standardized safety messages show the application performance improvement rendered by our proposed mechanism, compared to the standardized protocol.

Keywords: Automated Vehicles, Electric and Hybrid Technologies, Eco-driving and Energy-efficient Vehicles
Abstract: In recent years, all major car manufacturers have started to introduce predictive functionalities based on an electronic horizon for the autonomous on-highway operation of their vehicles. Using the Advanced Driver Assistance Systems (ADAS) for anticipatory driving is a fundamental approach to significantly reduce the fuel consumption and pollutant emissions of the internal combustion engines. Today’s Adaptive Cruise Control (ACC) systems try to maintain a constant speed selected by the driver without regarding the energy consumption of the vehicle. There is, however, a degree of freedom to apply cruise speed limits without direct driver involvement in order to save propulsion energy in an Autonomous Vehicle (AV). This work presents a novel velocity and energy optimization method for AV hybrid electric vehicles (HEVs) by using a stochastic optimization technique. By applying Particle Filters (PFs) in a routine of Stochastic Dynamic Programming (SDP) to solve the power split efficiently. The sweet-point operation of the powertrain is calculated by probability hypothesis densities along the distance-based prediction horizon. The optimization approach shows a cost trade-off between horizon resolution, length, iterative combinatorial optimality, and computational efficiency. Finally, the approach is applied to a PHEV vehicle model in a real-time ECU in the Worldwide harmonized Light Duty Test Cycle (WLTC) and in a hilly RDE Aachen-Eifel driving cycle.

Keywords: Situation Analysis and Planning, Telematics, Smart Infrastructure
Abstract: Battery electric vehicles (BEVs) are becoming more prevalent as improvements in battery technology and energy management continue to be made. As the number of electric vehicles grows, the demand for fast-charging stations is expected to increase dramatically. Thus, building new charging station infrastructure efficiently will be key to reducing upfront costs while meeting consumer demands. In this work, we propose a method for choosing a set of charging station locations that are optimized based on a set of given common vehicle demand points. As part of this solution, we also offer a novel abstraction of the road network on which energy-efficient paths that account for charge-time delays may be found. The current algorithm chooses the optimal charging locations for a single agent which has a route objective specified using temporal logic. To demonstrate the proposed method, the running example shows how a charging station could be chosen for an electric delivery vehicle. Simulations were run on sample road networks with a given set of demand points to service and potential charging station locations to compare. The method is shown to successfully rank potential charging stations in terms of their expected average charging time cost.

Keywords: Societal Impacts, Electric and Hybrid Technologies, Legal Impacts
Abstract: With rise in shared e-scooter deployment, there is growing interest in evaluating the proportion of recreational vs. non-recreational use so that public agencies can develop appropriate policies and infrastructure support. In the Indianapolis metropolitan area from September 2018 to May 2019, approximately 728,000 e-scooter trips covered 1,250,457 km (777,000 miles) during 162,000 hours of use. This paper focused on three regions with distinct land-use patterns that had 509,241 trips covering 879,209 km (546,317 miles) during 118,440 hours of use. A technique was proposed that compared the ratio of travelled distance to the linear distance between trip start and end points. Trips with ratio larger than 2.0 were assumed to be recreational. As agencies begin to adopt policies and infrastructure support, we believe the methodology described in this paper will be an important consideration for better understanding of the proportion of non-recreational “last-mile” users versus recreational users. The analysis will be helpful for agencies and decision makers in shaping better policies for the safe integration of e-scooters into urban ecosystems.

Keywords: Mapping and Localization, Information Fusion
Abstract: In this paper we propose an iterative Graph SLAM based approach to create HD maps from series production vehicles fleet data. Only high level sensor measurements provided by advanced driver assistance systems are used. This reduces the required bandwidth and makes this approach scalable to vehicle fleet data. Creating HD maps from fleet data enables up to date HD maps since no dedicated mapping vehicles are required. At first, the data is aligned based on odometry, GNSS and traffic sign measurements. Next, road boundary measurements are included, which results in an optimized lateral alignment. Finally, lane boundary association can be carried out. This results in an HD map containing high accuracy data of traffic signs, road and lane boundaries. The approach was evaluated using series production vehicle fleet data recorded on US highways and on a German autobahn covering a distance of 35.4km. The final HD map was evaluated with a groundtruth HD map and achieved an average error of 0.59m.

Keywords: Mapping and Localization, Vision Sensing and Perception, Active and Passive Vehicle Safety
Abstract: Calibration of relative poses between cameras is a challenging problem, known as extrinsic calibration, for non-overlapping cameras that do not share the field of view. We propose a method for calibrating non-overlapping in-vehicle cameras placed at front, back, left and right positions by using visual SLAM(vSLAM). Our proposal is to calibrate the cameras during the motion of 90-degree backing-up parking on the fly, without using any dedicated calibration equipment. With this motion, the adjacent cameras are able to have the close field of view at different moments. The relative poses can be computed if the maps computed with vSLAM on each camera are merged by using the common structures. Therefore, we propose an efficient calibration framework with this feature. The proposed method is divided into three steps: map reconstruction with vSLAM on each camera, map merging for all the cameras, and extrinsic calibration. Especially, we propose to separately utilize the frames for vSLAM and the ones for the calibration so that the accuracy of vSLAM can be maximized for the calibration. In the evaluation, the calibration was performed in a practical environment to investigate the performance in comparison with the ground truth acquired by using a calibration equipment.

Keywords: Mapping and Localization, Self-Driving Vehicles, Advanced Driver Assistance Systems
Abstract: This paper surveys a number of recent developments in modern Global Navigation Satellite Systems (GNSS) and investigates the possible impact on autonomous driving architectures. Modern GNSS now consist of four independent global satellite constellations delivering modernized signals at multiple civil frequencies. New ground monitoring infrastructure, mathematical models, and internet services correct for errors in the GNSS signals at continent scale. Mass-market automotive-grade receiver chipsets are available at low Cost, Size, Weight, and Power (CSWaP). The result is that GNSS in 2020 delivers better than lane-level accurate localization with 99.99999% integrity guarantees at over 95% availability. In autonomous driving, SAE Level 2 partially autonomous vehicles are now available to consumers, capable of autonomously following lanes and performing basic maneuvers under human supervision. Furthermore, the first pilot programs of SAE Level 4 driverless vehicles are being demonstrated on public roads.

However, autonomous driving is not a solved problem. GNSS can help. Specifically, incorporating high-integrity GNSS lane determination into vision-based architectures can unlock lane-level maneuvers and provide oversight to guarantee safety. Incorporating precision GNSS into LiDAR-based systems can unlock robustness and additional fallbacks for safety and utility. Lastly, GNSS provides interoperability through consistent timing and reference frames for future V2X scenarios.

Keywords: Automated Vehicles, Self-Driving Vehicles, Vision Sensing and Perception
Abstract: Autonomous vehicle navigation requires the desired trajectory and the current localization to be able to calculate the command that must be sent to the actuators. The localization of the vehicle (usually defined by a position vector and an orientation vector), can be provided by external systems. GPS localization is the most accurate solution but when it is no longer available or precise, there is a need for on-board localization estimation based on proprioceptive and exteroceptive sensors. Visual odometry is a well known approach to estimate the vehicle motion from a camera. Unfortunately, visual localization is subject to errors that increase over time (drift). In this paper, we provide a study of the impact of localization errors on the control of an autonomous vehicle. In order to validate visual odometry algorithms in simulation, a drift model is proposed. Real navigation experiments with errors on the localization are presented in order to characterize the drift model and the propagation of localization errors in the controller module and the associated command signal.

Keywords: Self-Driving Vehicles, Convolutional Neural Networks, Deep Learning
Abstract: Teaching autonomous vehicles to imitate human driving in complex, urban traffic scenarios is a difficult task. “End-to-end” autonomous driving systems, based on “imitation learning”, are a expecting approach. A model learns the relationships between sensing input and vehicle control signal outputs. These methods can successfully achieve driving in simple scenarios such as lane keeping. In contrast, the “mid-to-mid” autonomous driving methods now being proposed. In such framework, the model learns the relationships between pre-processed feature maps from the model-based system as input and the future position of the ego vehicle as the output. Mid-to-mid driving methods can direct vehicles more robustly than end-to-end driving methods in some complex driving environments. However, mid-to-mid driving methods use the results of the object detection module to create the feature map. If object detection fails, or detection performance is poor due to changes in the driving environment, prediction performance may also be degraded. Our proposed method uses a prediction module that outputs point grid maps directly, without the use of an object detection module, which are then incorporated into the feature map. Point grid maps represent the locations of surrounding vehicles and obstacles directly, based on LiDAR point cloud data. Since the results of object detection are not used by the prediction module, detection performance does not affect prediction performance. In this study we conduct two experiments, an off-line evaluation using a Lyft dataset, and an on-line evaluation using the CARLA simulator. The results show that our model can achieve the same level of ego-vehicle position prediction performance as a model using annotated object location information.

Keywords: Smart Infrastructure, Advanced Driver Assistance Systems, Intelligent Vehicle Software Infrastructure
Abstract: Self-driving vehicle systems have become mainstream as major international research and development companies compete for road detection and mapping projects such as localization. The latter plays an important role in self-driving systems because an accurate determination of the vehicle’s position improves trajectory and offset results. And with a good localization function, a high-precision map can be established. This paper proposes using pole maps to enhance vehicle localization while correcting previous vehicle localization trajectory offsets. We have established a pole feature map with pole-shaped characteristic point clouds along the road that are marked with spatial coordinate information. We use three-dimensional light detection and ranging (3D Lidar) as a sensor for environmental scanning, dynamic segment thresholds, and point cloud projections to the image to extract poles, which avoids identification errors typically caused by the scattering effect of 3D Lidar. The resulting pole map uses the occupancy grid map to filter out noise. In terms of localization, we continue to detect pole features while the vehicle is moving, and compare these with the map. When the results match, we update the current localized information. The localization point corrects the information of the trajectory of the undetected pole, which improves vehicle localization accuracy.

Keywords: Self-Driving Vehicles, Mapping and Localization, Sensor and Data Fusion
Abstract: This paper proposes a robust and reliable motion-based calibration algorithm of multiple LiDARs and INS (inertial navigation system). The conventional methods utilize appearance-based matching results or only motions of sensors. Therefore, this paper proposes a graph structure-based optimization method with three special constraints. The first constraint is matching-based constraint. This constraint provides reliable transformations between LiDARs. Second, motion-based constraint allows to calibrate between LiDARs and INS. Last one is rigid body constraint from an assumption of rigid body platform. This constraint significantly improves position errors from ambiguities of monotonous motions, and it makes LiDAR sensors possible to fully calibrate with 6-DoF (degree-of-freedom) motions of INS. The proposed method was analyzed with KITTI data and applied to real autonomous vehicle to show its superiority.

Keywords: Automated Vehicles, Situation Analysis and Planning, Advanced Driver Assistance Systems
Abstract: Virtual testing plays an important role in the validation and verification of automated vehicles. State-of-the-art approaches first generate a huge amount of test scenarios through simulations or test drives, which are later filtered to obtain relevant scenarios for a given set of specifications. However, only few works exist on synthesizing scenarios directly from specifications. In this work, we present an optimization-based approach to synthesize scenarios only from formal specifications and a given map. To concretize the specifications, we formulate predicates, which are subsequently converted to a mixed-integer quadratic optimization problem. We demonstrate how our method can generate scenarios for maps featuring merging lanes and intersections given a variety of specifications.

Keywords: Automated Vehicles, Situation Analysis and Planning, Collision Avoidance
Abstract: Navigation in congested environments is a challenge for autonomous vehicles and they should consider collision risk metric into their driving behavior. In this paper, we propose a novel two-fold indicator: On the one hand, single-shot risk works in space domain, considering geometries, locations and the velocities of the obstacles in the current scene. On the other hand, long-term risk considers the evolution of the current scene and provides risk values in time domain. The map information and different prediction models (e.g. reachable sets, probabilistic) are considered in the long-term risk, which can then be used in trajectory planning or decision making approaches. Our method can be applied to scenarios with arbitrary road topologies (intersections, roundabouts, highway, etc.) and it is suitable regardless of the scene prediction method. We formulate the single-shot (or short-term) risk with one single function fitted using Monte Carlo (MC) Simulations. The results are evaluated in real scenarios using HighD dataset and compared with other risk indicators such as THW and TTC. In addition, it is applied to a simple trajectory planner in order to demonstrate that the proposed approach imitates human driving style.

Keywords: Vision Sensing and Perception, Image, Radar, Lidar Signal Processing, Situation Analysis and Planning
Abstract: Due to their capability of acquiring aerial imagery, camera-equipped Unmanned Aerial Vehicles (UAVs) are very cost-effective tools for acquiring traffic information. However, not enough attention has been given to the validation of the accuracy of these systems. In this paper, an analysis of the most significant sources of error is done. This includes three key components. First, a vehicle state estimation by means of statistical filtering. Second, a quantification of the most significant sources of error. Third, a benchmark of the estimated state compared with state-of-the-art reference sensors. This work presents ways to minimize the errors of the most relevant sources. With these error reductions, camera-equipped UAVs are very attractive tools for traffic data acquisition. The test data and the source code are made publicly available.

Keywords: Situation Analysis and Planning, Vision Sensing and Perception, Traffic Flow and Management
Abstract: The availability of real-world data is a key element for novel developments in the fields of automotive and traffic research. Aerial imagery has the major advantage of recording multiple objects simultaneously and overcomes limitations such as occlusions. However, there are only few data sets available. This work describes a process to estimate a precise vehicle position from aerial imagery. A robust object detection is crucial for reliable results, hence the state-of-the-art deep neural network Mask-RCNN is applied for that purpose. Two training data sets are employed: The first one is optimized for detecting the test vehicle, while the second one consists of randomly selected images recorded on public roads. To reduce errors, several aspects are accounted for, such as the drone movement and the perspective projection from a photograph. The estimated position is comapared with a reference system installed in the test vehicle. It is shown, that a mean accuracy of 20 cm can be achieved with flight altitudes up to 100 m, Full-HD resolution and a frame-by-frame detection. A reliable position estimation is the basis for further data processing, such as obtaining additional vehicle state variables. The source code, training weights, labeled data and example videos are made publicly available. This supports researchers to create new traffic data sets with specific local conditions.

Keywords: Self-Driving Vehicles, Automated Vehicles, Situation Analysis and Planning
Abstract: The deployment of autonomous vehicles requires safety assurance and performance guarantees of the developed system. However, this is complex due to the number of scenario variations and uncertainty associated with the operating environment.To alleviate this challenge, we propose a model-based validation methodology that relies on a functional hierarchy for the breakdown and simplification of the system navigation functions, and the Backtracking Process Algorithm to identify, trace, and probabilistically quantify risk significant event sequences (scenarios) that lead to Top Events of interest (such as requirement violations). This methodology is demonstrated on a scenario with an occluded pedestrian crossing the road. We are able to identify risks associated with the actor classification problem and sudden changes in behavior of the pedestrian.

Keywords: Automated Vehicles, Situation Analysis and Planning
Abstract: In this paper, we propose an improved sampling-based hybrid A* (SBA*) algorithm for path planning of intelligent vehicles, which works efficiently in complex urban environments. Two main modifications are introduced into the traditional hybrid A* algorithm to improve its adaptivity in both structured and unstructured traffic scenarios. Firstly, a hybrid potential field (HPF) model considering both traffic regulation and obstacle configuration is proposed to represent the vehicle's workspace, which is utilized as a heuristic function. Secondly, a set of directional motion primitives is generated by taking the prior topological structure of the workspace into account. The path planner using SBA* not only obeys traffic regulations in structured scenarios but also is capable of exploring complex unstructured scenarios rapidly. Finally, a post-optimization step is adopted to increase the feasibility of the path. The efficacy of the proposed algorithm is extensively validated and tested with an autonomous vehicle in real traffic scenarios. The experimental results show that SBA* works well in complex urban environments.

Keywords: Situation Analysis and Planning, Deep Learning, Automated Vehicles
Abstract: Motion planning for autonomous vehicles is a challenging task, especially in dynamic environments. The motion of the vehicle itself needs to be considered while the vehicle needs to react to its surroundings at the same time. Sampling-based algorithms proved to be suitable to cope with these challenges. However, the performance of these algorithms is highly dependent on the sampling heuristics, which in turn are often hand crafted and thus need a large amount of tuning. Therefore, we developed two approaches based on deep learning to learn these heuristics for sampling-based motion planning in dynamic environments. The first approach predicts a discrete probability distribution for each point in time of the future trajectory, whereas the second approach directly predicts a variety of trajectories by using dropout sampling. Both approaches are based on an environment representation encoded as a grid-based tensor. The learned heuristics are integrated into an existing planning framework based on particle swarm optimization and are evaluated in several situations. This shows how to combine the strengths of machine learning based approaches and the traceability of rule- or model-based approaches. The evaluation demonstrates that, in total, we were able to improve on our current heuristics. However, none of the approaches performed consistently better in all scenarios evaluated.

Keywords: Traffic Flow and Management, Impact on Traffic Flows, Smart Infrastructure
Abstract: With the deployment of partially and highly automated vehicles, the automotive industry is greatly increasing its influence on road traffic. In order to ensure a positive influence of automated vehicles on traffic efficiency as well as traffic safety, simulations are broadly used for the development and testing of the required functions. Since these simulations are applied to evaluate the behavior of an automated driving function in the real world, an exact representation of the real world in the simulation is essential for the validity of the generated results. Therefore, there is a need for methods with which certain parameters of real-world situations may be quantified and applied to a simulation. In this work, we propose an approach to measure traffic flow and estimate the traffic state in a network based on extended floating car data. For this purpose, the data concerning the movement of the tracked vehicles is combined with the data regarding surrounding traffic gathered by the vehicles’ sensors. The aim of this combination is to achieve an accurate traffic observation on urban as well as rural roads with a minimal number of test vehicles gathering the data. The application of the method to simulated traffic results in an accurate estimation of the traffic volume. The functionality is also demonstrated based on a limited sample of real-world test data.

Keywords: Advanced Driver Assistance Systems, Deep Learning, Collision Avoidance
Abstract: Keeping the safe distance from the leading vehicle is crucial for transportation companies with a fleet of old cars. While modern Advanced Driver Assistant Systems (ADAS) might be able to estimate the distance from the front-leading vehicle, traditional ADAS do not usually offer this feature. An alternative solution is to monitor the distance using smartphones that are attached to a place such as a sun visor. The basic idea behind this approach is to detect the front-leading vehicle using the smartphone camera and estimate its distance from the car. Although SSD can achieve real-time performance on powerful GPUs, it remains challenging to run this model in real-time on mobile devices. In this paper, we propose a monocular distance estimator for forward-leading vehicles using a smartphone which is faster and more accurate than the state-of-the-art SSD detector. Specifically, we propose a layer-wise method to generate more efficient default boxes for the SSD and develop a lightweight method for estimating the distance accurately. Our experiments show that the proposed method reduces the number of default boxes by an average of 38.4% while it improves the detection rate and the processing speed compared to the original SSD. Moreover, our monocular distance estimator provides a proper safety buffer zone when the distance is greater than 20 meters.

Keywords: Automated Vehicles, Cooperative ITS
Abstract: Vehicle platooning is an innovative technology for intelligent transportation systems, where each vehicle in the platoon is required to autonomously follow its front vehicle's path unconditionally and accurately. For platoons without communication, accurate path and velocity estimation of the front vehicle is challenging and crucial. Instead of memorizing the original detection result of the front vehicle to generate a path, the path is estimated by fusing motion prediction and observation in this paper. Since there is a correlation between the two estimates, Split Covariance Intersection Filter is used to guarantee the fusion consistency. Besides, a motion model considering velocity is merged into the filter to achieve a precise estimate of the predecessor's velocity simultaneously. Moreover, a path generation approach is designed in a high-frequency loop independent of front vehicle detection to improve continuity of the path. Experimental validations in the real-world environment highlight the remarkable improvement in the accuracy of both path and velocity estimation. Meanwhile, the path generated by the proposed approach is more continuous compared with the commonly used method. Furthermore, complete vehicle platooning demonstrations in diverse environments prove the practicality and robustness of the proposed approach.

Keywords: V2X Communication, Intelligent Ground, Air and Space Vehicles
Abstract: The fleet management of mobile working machines with the help of connectivity can increase not only safety but also productivity. However, rare mobile working machines have taken advantage of V2X. Moreover, no one published the simulation results that are suitable for evaluating the performance of the ad-hoc network at a working site on the highway where is congested, with low mobility, and without building. In this paper, we suggested that IEEE 802.11p should be implemented for fleet management, at least for the first version. Furthermore, we proposed an analytical model for machines to estimate the ad-hoc network performance, i.e., the delay and the packet loss probability in real-time based on the simulation results we made in ns-3. The model of this paper can be further used for determining when shall ad-hoc or cellular network be used in the corresponding scenarios.

Keywords: Intelligent Vehicle Software Infrastructure, Self-Driving Vehicles, Eco-driving and Energy-efficient Vehicles
Abstract: With advances in sensing, computing and communication technologies, Connected and Automated Vehicles (CAVs) are becoming feasible. The advent of CAVs presents new opportunities to improve the energy efficiency of individual vehicles. However, testing and verifying energy-efficient autonomous driving systems are difficult due to safety considerations and repeatability. In this paper, we present a co-simulation platform to develop and test novel vehicle eco-autonomous driving technologies named InfoRich, which incorporates the information from on-board sensors, V2X communications, and map database. The co-simulation platform includes eco-autonomous driving software, vehicle dynamics and powertrain (VD&PT) model, and a traffic environment simulator. Also, we utilize synthetic drive cycles derived from real-world driving data to test the strategies under realistic driving scenarios. To build road networks from the real-world driving data, we develop an Automated Parser and Calculator for Map/Scenario named AutoPASCAL. Overall, the simulation platform provides a realistic vehicle model, powertrain model, sensor model, traffic model, and road-network model to enable the evaluation of the energy efficiency of eco-autonomous driving.

Keywords: Security, Autonomous / Intelligent Robotic Vehicles, Active and Passive Vehicle Safety
Abstract: Modern vehicles and in particular automated or autonomous vehicles utilize data gained from wheel speed sensors for various functionalities like determining current speed or electronic stability control. The wheel speed is derived from a multitude of hall sensors. By generating an electromagnetic field with a fitting frequency it is possible to change the wheel speed sensor’s interpretation of wheel speed. The braking system and electronic stability control would then base their actions on incorrect data possibly leading to a dangerous situation. In this research, we investigate practical feasibility and impact of such attacks. For this, an audio amplifier and a Helmholtz coil were used to emit electromagnetic fields with varying frequencies. We then analyse the effect of a single manipulated wheel speed sensor on the electronic stability control used in modern vehicles (using a hardware-in-the-loop setup). We finally describe consequences and possible countermeasure for the upcoming autonomous and highly automated vehicles.

Keywords: V2X Communication, Automated Vehicles, Situation Analysis and Planning
Abstract: In this paper we investigate the problem of a vehicle merging to a main road while another vehicle is approaching on that road. We utilize conflict analysis to help the decision making and control for vehicles of different automation levels. We demonstrate that using vehicle-to-everything (V2X) communication, e.g., basic safety message (BSM), we are able to prevent conflict between the two vehicles. We design a longitudinal controller for the merging vehicle and show that V2X communication is also beneficial in improving the time efficiency of the merge. The results are demonstrated by performing simulations based on real highway data.