Keywords: Convolutional Neural Networks, Lidar Sensing and Perception, Self-Driving Vehicles
Abstract: We present a robust real-time LiDAR 3D object detector that leverages heteroscedastic aleatoric uncertainties to significantly improve its detection performance. A multi-loss function is designed to incorporate uncertainty estimations predicted by auxiliary output layers. Using our proposed method, the network ignores to train from noisy samples, and focuses more on informative ones. We validate our method on the KITTI object detection benchmark. Our method surpasses the baseline method which does not explicitly estimate uncertainties by up to nearly 9% in terms of Average Precision (AP). It also produces state-of-the-art results compared to other methods, while running with an inference time of only 72ms. In addition, we conduct extensive experiments to understand how aleatoric uncertainties behave. Extracting aleatoric uncertainties brings almost no additional computation cost during the deployment, making our method highly desirable for autonomous driving applications.

Keywords: Convolutional Neural Networks, Lidar Sensing and Perception, Self-Driving Vehicles
Abstract: Autonomous vehicles require an accurate understanding of the underlying motion of their surroundings. Traditionally this understanding is acquired using optical flow algorithms on camera images, RADAR sensors which measure velocity directly or by object tracking through various sensors. We propose a novel method to estimate point-wise 3D motion vectors from LiDAR point clouds using fully convolutional networks trained and evaluated on the KITTI dataset. Besides, we show how this motion information can be used to efficiently estimate odometry. We demonstrate that our approach achieves significant speed ups over the current state of the art.

Keywords: Lidar Sensing and Perception, Vision Sensing and Perception, Mapping and Localization
Abstract: In this paper, we propose to calibrate a vehicle equipped with several 2D-LIDARs and cameras. Our goal is to offer a method which is easy to use with very little manual intervention and which doesn't need a special calibration target. Sensor data is recorded while the vehicle is driven along a trajectory in a man made environment with vertical walls. The first step is a classical autocalibration approach based on visual SLAM and bundle adjustment is used to recover the cameras internal and external parameters. The second step which is the main contribution of the paper, is a novel optimisation algorithm to compute the external parameters of the LIDAR. We present this method as whole because the user has to do a single data acquisition for camera and LIDAR calibration. Our approach is validated by real experimental results.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception, Unsupervised Learning
Abstract: In traditional LIDAR processing pipelines, a point-cloud is split into clusters, or objects, which are classified afterwards. This supposes that all the objects obtained by clustering belong to one of the classes that the classifier can recognize, which is hard to guarantee in practice. We thus propose an evidential end-to-end deep neural network to classify LIDAR objects. The system is capable of classifying ambiguous and incoherent objects as unknown, while only having been trained on vehicles and vulnerable road users. This is achieved thanks to an evidential reformulation of generalized logistic regression classifiers, and an online filtering strategy based on statistical assumptions. The training and testing were realized on LIDAR objects which were labelled in a semi-automatic fashion, and collected in different situations thanks to an autonomous driving and perception platform.

Keywords: Convolutional Neural Networks, Unsupervised Learning, Vision Sensing and Perception
Abstract: Perception in autonomous vehicles has progressed exponentially in the last years thanks to the advances of vision-based methods such as Convolutional Neural Networks (CNNs). Current deep networks are both efficient and reliable, at least in standard conditions, standing as a suitable solution for the perception tasks of autonomous vehicles. However, there is a large accuracy downgrade when these methods are taken to adverse conditions such as nighttime. In this paper, we study methods to alleviate this accuracy gap by using recent techniques such as Generative Adversarial Networks (GANs). We explore diverse options such as enlarging the dataset to cover these domains in unsupervised training or adapting the images on-the-fly during inference to a comfortable domain such as sunny daylight in a pre-processing step. The results show some interesting insights and demonstrate that both proposed approaches considerably reduce the domain gap, allowing IV perception systems to work reliably also at night.

Keywords: Convolutional Neural Networks, Vision Sensing and Perception, Self-Driving Vehicles
Abstract: The availability of a large quantity of labelled training data is crucial for the training of modern object detectors. Hand labelling training data is time consuming and expensive while automatic labelling methods inevitably add unwanted noise to the labels. We examine the effect of different types of label noise on the performance of an object detector. We then show how co-teaching, a method developed for handling noisy labels and previously demonstrated on a classification problem, can be improved to mitigate the effects of label noise in an object detection setting. We illustrate our results using simulated noise on the KITTI dataset and on a vehicle detection task using automatically labelled data.

Keywords: Deep Learning, Automated Vehicles, Self-Driving Vehicles
Abstract: Human drivers have complex and individual behavior characteristics which describe how they act in a specific situation. Accurate behavior models are essential for many applications in the field of autonomous driving, ranging from microscopic traffic simulation, intention estimation and trajectory prediction, to interactive and cooperative motion planning. Designing such models by hand is cumbersome and inaccurate, especially in urban environments, with their high variety of situations and the corresponding diversity in human behavior. Learning how humans act from recorded scenarios is a promising way to overcome these problems. However, predicting complete trajectories at once is challenging, as one needs to account for multiple hypotheses and long-term interactions between multiple agents. In contrast, we propose to learn Markovian action models with deep neural networks that are conditioned on a driver’s route intention (such as turning left or right) and the situational context. Step-wise forward simulation of these models for the different possible routes of all agents allows for multi-modal and interaction-aware scene predictions at arbitrary road layouts. Learning to predict only one time step ahead given a specific route reduces learning complexity, such that simpler and faster models are obtained. This enables the integration into particle-based algorithms such as Monte Carlo tree search or particle filtering. We evaluate the learned model both on its own and integrated into our previously presented dynamic Bayesian network for intention estimation and show that it outperforms our previous hand-tuned rule-based model.

Keywords: Deep Learning, Convolutional Neural Networks, Vision Sensing and Perception
Abstract: Training convolutional networks for semantic segmentation requires per-pixel ground truth labels, which are very time consuming and hence costly to obtain. Therefore, in this work, we research and develop a hierarchical deep network architecture and the corresponding loss for semantic segmentation that can be trained from weak supervision, such as bounding boxes or image level labels, as well as from strong per-pixel supervision. We demonstrate that the hierarchical structure and the simultaneous training on strong (per-pixel) and weak (bounding boxes) labels, even from separate datasets, constantly increases the performance against per-pixel only training. Moreover, we explore the more challenging case of adding weak image-level labels. We collect street scene images and weak labels from the immense Open Images dataset to generate the OpenScapes dataset, and we use this novel dataset to increase segmentation performance on two established per-pixel labeled datasets, Cityscapes and Vistas. We report performance gains up to +13.2% mIoU on crucial street scene classes, and inference speed of 20 fps on a Titan V GPU for Cityscapes at 512 x 1024 resolution. Our network and OpenScapes dataset are shared with the research community.

Keywords: Active and Passive Vehicle Safety, Automated Vehicles, Societal Impacts
Abstract: At Intel and Mobileye, saving lives drives us. But in the world of automated driving, we believe safety is not merely an impact of AD, but the bedrock on which we all build this industry. And so we proposed Responsibility-Sensitive Safety (RSS), a formal model to define safe driving and what rules an automated vehicle, independent of brand or policy, should abide to always keep its passengers safe. We intend this open, non-proprietary model to drive cross-industry discussion; let’s come together as an industry and use RSS as a starting point to clarify safety today, to enable the autonomous tomorrow.

Keywords: Convolutional Neural Networks, Lidar Sensing and Perception, Self-Driving Vehicles
Abstract: We present a robust real-time LiDAR 3D object detector that leverages heteroscedastic aleatoric uncertainties to significantly improve its detection performance. A multi-loss function is designed to incorporate uncertainty estimations predicted by auxiliary output layers. Using our proposed method, the network ignores to train from noisy samples, and focuses more on informative ones. We validate our method on the KITTI object detection benchmark. Our method surpasses the baseline method which does not explicitly estimate uncertainties by up to nearly 9% in terms of Average Precision (AP). It also produces state-of-the-art results compared to other methods, while running with an inference time of only 72ms. In addition, we conduct extensive experiments to understand how aleatoric uncertainties behave. Extracting aleatoric uncertainties brings almost no additional computation cost during the deployment, making our method highly desirable for autonomous driving applications.

Keywords: Convolutional Neural Networks, Lidar Sensing and Perception, Self-Driving Vehicles
Abstract: Autonomous vehicles require an accurate understanding of the underlying motion of their surroundings. Traditionally this understanding is acquired using optical flow algorithms on camera images, RADAR sensors which measure velocity directly or by object tracking through various sensors. We propose a novel method to estimate point-wise 3D motion vectors from LiDAR point clouds using fully convolutional networks trained and evaluated on the KITTI dataset. Besides, we show how this motion information can be used to efficiently estimate odometry. We demonstrate that our approach achieves significant speed ups over the current state of the art.

Keywords: Lidar Sensing and Perception, Vision Sensing and Perception, Mapping and Localization
Abstract: In this paper, we propose to calibrate a vehicle equipped with several 2D-LIDARs and cameras. Our goal is to offer a method which is easy to use with very little manual intervention and which doesn't need a special calibration target. Sensor data is recorded while the vehicle is driven along a trajectory in a man made environment with vertical walls. The first step is a classical autocalibration approach based on visual SLAM and bundle adjustment is used to recover the cameras internal and external parameters. The second step which is the main contribution of the paper, is a novel optimisation algorithm to compute the external parameters of the LIDAR. We present this method as whole because the user has to do a single data acquisition for camera and LIDAR calibration. Our approach is validated by real experimental results.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception, Unsupervised Learning
Abstract: In traditional LIDAR processing pipelines, a point-cloud is split into clusters, or objects, which are classified afterwards. This supposes that all the objects obtained by clustering belong to one of the classes that the classifier can recognize, which is hard to guarantee in practice. We thus propose an evidential end-to-end deep neural network to classify LIDAR objects. The system is capable of classifying ambiguous and incoherent objects as unknown, while only having been trained on vehicles and vulnerable road users. This is achieved thanks to an evidential reformulation of generalized logistic regression classifiers, and an online filtering strategy based on statistical assumptions. The training and testing were realized on LIDAR objects which were labelled in a semi-automatic fashion, and collected in different situations thanks to an autonomous driving and perception platform.

Keywords: Convolutional Neural Networks, Unsupervised Learning, Vision Sensing and Perception
Abstract: Perception in autonomous vehicles has progressed exponentially in the last years thanks to the advances of vision-based methods such as Convolutional Neural Networks (CNNs). Current deep networks are both efficient and reliable, at least in standard conditions, standing as a suitable solution for the perception tasks of autonomous vehicles. However, there is a large accuracy downgrade when these methods are taken to adverse conditions such as nighttime. In this paper, we study methods to alleviate this accuracy gap by using recent techniques such as Generative Adversarial Networks (GANs). We explore diverse options such as enlarging the dataset to cover these domains in unsupervised training or adapting the images on-the-fly during inference to a comfortable domain such as sunny daylight in a pre-processing step. The results show some interesting insights and demonstrate that both proposed approaches considerably reduce the domain gap, allowing IV perception systems to work reliably also at night.

Keywords: Convolutional Neural Networks, Vision Sensing and Perception, Self-Driving Vehicles
Abstract: The availability of a large quantity of labelled training data is crucial for the training of modern object detectors. Hand labelling training data is time consuming and expensive while automatic labelling methods inevitably add unwanted noise to the labels. We examine the effect of different types of label noise on the performance of an object detector. We then show how co-teaching, a method developed for handling noisy labels and previously demonstrated on a classification problem, can be improved to mitigate the effects of label noise in an object detection setting. We illustrate our results using simulated noise on the KITTI dataset and on a vehicle detection task using automatically labelled data.

Keywords: Deep Learning, Automated Vehicles, Self-Driving Vehicles
Abstract: Human drivers have complex and individual behavior characteristics which describe how they act in a specific situation. Accurate behavior models are essential for many applications in the field of autonomous driving, ranging from microscopic traffic simulation, intention estimation and trajectory prediction, to interactive and cooperative motion planning. Designing such models by hand is cumbersome and inaccurate, especially in urban environments, with their high variety of situations and the corresponding diversity in human behavior. Learning how humans act from recorded scenarios is a promising way to overcome these problems. However, predicting complete trajectories at once is challenging, as one needs to account for multiple hypotheses and long-term interactions between multiple agents. In contrast, we propose to learn Markovian action models with deep neural networks that are conditioned on a driver’s route intention (such as turning left or right) and the situational context. Step-wise forward simulation of these models for the different possible routes of all agents allows for multi-modal and interaction-aware scene predictions at arbitrary road layouts. Learning to predict only one time step ahead given a specific route reduces learning complexity, such that simpler and faster models are obtained. This enables the integration into particle-based algorithms such as Monte Carlo tree search or particle filtering. We evaluate the learned model both on its own and integrated into our previously presented dynamic Bayesian network for intention estimation and show that it outperforms our previous hand-tuned rule-based model.

Keywords: Deep Learning, Convolutional Neural Networks, Vision Sensing and Perception
Abstract: Training convolutional networks for semantic segmentation requires per-pixel ground truth labels, which are very time consuming and hence costly to obtain. Therefore, in this work, we research and develop a hierarchical deep network architecture and the corresponding loss for semantic segmentation that can be trained from weak supervision, such as bounding boxes or image level labels, as well as from strong per-pixel supervision. We demonstrate that the hierarchical structure and the simultaneous training on strong (per-pixel) and weak (bounding boxes) labels, even from separate datasets, constantly increases the performance against per-pixel only training. Moreover, we explore the more challenging case of adding weak image-level labels. We collect street scene images and weak labels from the immense Open Images dataset to generate the OpenScapes dataset, and we use this novel dataset to increase segmentation performance on two established per-pixel labeled datasets, Cityscapes and Vistas. We report performance gains up to +13.2% mIoU on crucial street scene classes, and inference speed of 20 fps on a Titan V GPU for Cityscapes at 512 x 1024 resolution. Our network and OpenScapes dataset are shared with the research community.

Keywords: Automated Vehicles, Deep Learning, Situation Analysis and Planning
Abstract: Neural network autonomous driving policies are widely explored. However, no matter using imitation learning or reinforcement learning, the network policies are generally hard to train, and the learned knowledge encoded in neural network policies are hard to transfer. We propose to modularize the complicated driving policies in terms of the driving attributes, and present the parallel attribute networks (PAN), which can learn to fullfill the requirements of the attributes in the driving tasks separately, and later assemble their knowledge together. Concretely, we first train a policy network that accomplish the base lane tracking attribute. The modules for the add-on attributes such as avoiding obstacles and obeying traffic rules are then trained to map the corresponding state to a satisfactory set of the vehicle action space. Finally the reference action given by the base policy is projected into the satisfactory sets so as to satisfy the requirements of all the attributes. Using the PAN, many complicated tasks that are hard to train from scratch can be easily trained; also unseen driving tasks can be solved in a zero-shot manner by assembling the pretrained attribute modules. We have validated the capability of our model on a class of autonomous driving problems with attributes of obstacle avoidance, traffic light and speed limit in simulation. Experimental results based on an obstacle avoidance task are also presented.

Keywords: Deep Learning, Unsupervised Learning, Image, Radar, Lidar Signal Processing
Abstract: Nowadays, a large amount of labeled data is crucial for deep neural network training. However, data labeling is still a time- and labor-consuming task, especially when labeling 3D point clouds. Meanwhile, object recognition has achieved great success on 2D images, even beyond the ability of humans. In this paper, we propose an effective framework to produce labeled data by using an image detector as a supervisor, and we train the network with a simple trick to eliminate noisy labels. For object-sparse scenes, this method is able to obtain good label data, while for object-dense scenes, we can use our training method to detect some of the corrupted labels. This is realized by building a cohesive camera and LiDAR system (named “Licam”) and performing target frustum region proposal on point clouds using the camera detection result. Efficient and effective vehicle detection is achieved based on this learning and training framework. We examine this method on the KITTI dataset [7] and our own road running data collected from a micro electro mechanical system (MEMS) LiDAR, demonstrating fast and accurate detection results. The results show that our automatic data labeling and training framework is effective and efficient. It provides the ability to obtain large-scale labeled data, and is easy to use for online learning.

Keywords: Deep Learning, Advanced Driver Assistance Systems, Unsupervised Learning
Abstract: Recent road marking recognition has achieved great success in the past few years along with the rapid development of deep learning. Although considerable advances have been made and many methods have been developed, they are often over-dependent on unrepresentative datasets and constrained conditions. In this paper, to overcome these drawbacks, we propose an alternative method that achieves higher accuracy and generates high-quality samples as data augmentation. With the following two major contributions: 1) The proposed deblurring network can successfully recover a clean road marking from a blurred one by adopting generative adversarial networks (GAN). 2) The proposed data augmentation method, based on mutual information, can preserve and learn semantic contexts from the given dataset. We construct and train a class-conditional GAN to increase the size of training set, which makes it suitable to recognize target. The experimental results have shown that the our proposed framework generates deblurred clean samples from blurry ones, and outperforms other methods even with unconstrained road marking datasets.

Keywords: Deep Learning, Self-Driving Vehicles, Vehicle Control
Abstract: Imbalanced datasets are deathful for model training. In the field of steering prediction, imbalanced training is the core reason that model is unable to predict sharp steering value well. This paper proposes a new loss framework to train the robust end-to-end model, which is named SteeringLoss. The imbalanced distribution of steering value for datasets is analyzed, which is similar with Gaussian distribution. With the feature of distribution, the gain factor is added to square loss function. This new SteeringLoss framework is able to improve the impact of sharp steering value while maintain the impact of small steering value. Experiment results show the SteeringLoss based model performs higher performance than traditional square loss based model with higher prediction precision and wider prediction range. Meanwhile, is able to control the time of training process, and can control the model performance, different distribution needs to choose different pair of parameters. What's more, the SteeringLoss framework is suitable for imbalanced training with similar distribution of dataset. The code can be found at: https: //github.com/weiy1991/SteeringLoss.

Keywords: Deep Learning, Sensor and Data Fusion, Self-Driving Vehicles
Abstract: We propose a data-driven approach to online multi-object tracking (MOT) that uses a convolutional neural network (CNN) for data association in a tracking-by-detection framework. The problem of multi-target tracking aims to assign noisy detections to a-priori unknown and time-varying number of tracked objects across a sequence of frames. A majority of the existing solutions focus on either tediously designing cost functions or formulating the task of data association as a complex optimization problem that can be solved effectively. Instead, we exploit the power of deep learning to formulate the data association problem as inference in a CNN. To this end, we propose to learn a similarity function that combines cues from both image and spatial features of objects. Our solution learns to perform global assignments in 3D purely from data, handles noisy detections and varying number of targets, and is easy to train. We evaluate our approach on the challenging KITTI dataset and show competitive results. Our code is available at https://git.uwaterloo.ca/wise-lab/fantrack

Keywords: Recurrent Networks, Deep Learning, Mapping and Localization
Abstract: The use of 2D laser scanners is attractive for the autonomous driving industry because of its accuracy, light-weight and low-cost. However, since only a 2D slice of the surrounding environment is detected at each scan, it is a challenge to execute important tasks such as the localization of the vehicle. In this paper we present a novel framework that explores the use of deep Recurrent Convolutional Neural Networks (RCNN) for odometry estimation using only 2D laser scanners. The application of RCNNs provides the tools to not only extract the features of the laser scanner data using Convolutional Neural Networks (CNNs), but in addition it models the possible connections among consecutive scans using the Long Short-Term Memory (LSTM) Recurrent Neural Network. Results on a real road dataset show that the method can run in real-time without using GPU acceleration and have competitive performance compared to other methods, being an interesting approach that could complement traditional localization systems.

Keywords: Recurrent Networks, Convolutional Neural Networks, Deep Learning
Abstract: Most of the semantic segmentation approaches have been developed for single image segmentation, and hence, video sequences are currently segmented by processing each frame of the video sequence separately. The disadvantage of this is that temporal image information is not considered, which improves the performance of the segmentation approach. One possibility to include temporal information is to use recurrent neural networks. However, there are only a few approaches using recurrent networks for video segmentation so far. These approaches extend the encoder-decoder network architecture of well-known segmentation approaches and place convolutional LSTM layers between encoder and decoder. However, in this paper it is shown that this position is not optimal, and that other positions in the network exhibit better performance. Nowadays, state-of-the-art segmentation approaches rarely use the classical encoder-decoder structure, but use multi-branch architectures. These architectures are more complex, and hence, it is more difficult to place the recurrent units at a proper position. In this work, the multi-branch architectures are extended by convolutional LSTM layers at different positions and evaluated on two different datasets in order to find the best one. It turned out that the proposed approach outperforms the pure CNN-based approach for up to 1.6 percent.

Keywords: Recurrent Networks, Sensor and Data Fusion
Abstract: State estimation is one of the essential tasks for autonomous systems, which is required in multiple applications, such as localization, object tracking, mapping, and many more. Various solution paradigms have been proposed by the scientific community to solve this well established problem. Some of the commonly used techniques are based on Bayesian inference, specifically the particle filter as it has the advantage of mod- eling arbitrary distributions without the unimodal assumption limitation. The particle filter has proven to be successful in many applications, however, a large set of particles is required in order to have robust estimates against noisy measurements and erroneous data association, which impedes the runtime performance of the filter. This study shows that a learning framework that trains a recurrent neural network with labeled data generated from the probabilistic estimation of a particle filter is capable of exploiting the expressive power of neural networks in order to capture the behavior of the filter and handle the noisy sensory information. The trained model is capable of performing the estimation with lower runtime complexity, making it applicable to numerous autonomous systems. This approach also proves to be helpful in situations when the ground truth data can not be accessed. We train and evaluate our strategy using raw GPS sensor measurements from the Oxford RobotCar dataset. The results show that the performance of the recurrent network closely matches that of the particle filter without exhaustive tuning and that the network is able to generalize effectively on test datasets as well.

Keywords: Automated Vehicles, Reinforcement Learning, Vehicle Control
Abstract: Lane change is a challenging task which requires delicate actions to ensure safety and comfort. Some recent studies have attempted to solve the lane-change control problem with Reinforcement Learning (RL), yet the action is confined to discrete action space. To overcome this limitation, we formulate the lane change behavior with continuous action in a model-free dynamic driving environment based on Deep Deterministic Policy Gradient (DDPG). The reward function, which is critical for learning the optimal policy, is defined by control values, position deviation status, and maneuvering time to provide the RL agent informative signals. The RL agent is trained from scratch without resorting to any prior knowledge of the environment and vehicle dynamics since they are not easy to obtain. Seven models under different hyperparameter settings are compared. A video showing the learning progress of the driving behavior is available. It demonstrates the RL vehicle agent initially runs out of road boundary frequently, but eventually has managed to smoothly and stably change to the target lane with a success rate of 100% under diverse driving situations in simulation.

Keywords: Autonomous / Intelligent Robotic Vehicles, Reinforcement Learning, Automated Vehicles
Abstract: In this study, we elaborate the procedure of designing a supervisory controller for the Autonomous Transit on Demand Vehicle (ATODV) system. Reinforcement learning is implemented to reduce the mean waiting time of the passengers, and a cost function is introduced to penalize the energy consumption of the electric vehicles. A stochastic simulation environment for an ATODV pilot project is coded in the Python environment to train the autonomous cart decision process as agents with artificial intelligence. Passenger group behavior, get-on and get-off times, destinations are modeled as random variables. A single Deep Q-Learning Network is trained subject to multi-agent settings. The ATODV system's independent decision making for the carts to reduce the passenger's waiting time while constraining the energy consumption and empty vehicle motion is evaluated.

Keywords: Reinforcement Learning, Situation Analysis and Planning, Automated Vehicles
Abstract: Navigating urban environments represents a complex task for automated vehicles. They must reach their goal safely and efficiently while considering a multitude of traffic participants. We propose a modular decision making algorithm to autonomously navigate intersections, addressing challenges of existing rule-based and reinforcement learning (RL) approaches. We first present a safe RL algorithm relying on a model-checker to ensure safety guarantees. To make the decision strategy robust to perception errors and occlusions, we introduce a belief update technique using a learning based approach. Finally, we use a scene decomposition approach to scale our algorithm to environments with multiple traffic participants. We empirically demonstrate that our algorithm outperforms rule-based methods and reinforcement learning techniques on a complex intersection scenario.

Keywords: Image, Radar, Lidar Signal Processing, Lidar Sensing and Perception, Sensor and Data Fusion
Abstract: This work focuses on a novel probabilistic approach for extended vehicle tracking, where multiple spatially distributed measurements can originate from the target, and kinematic state and geometry variables are estimated jointly. Prominent shape features extracted from raw measurement points contain spatial uncertainties due to noise in sensor measurements, the feature extraction process, approximation error of shape hypothesis, partial vision occlusion, to name a few. This work proposes a novel tracking paradigm that respects the variant spatial measurement model subject to changes in target pose and sensor viewpoint. This is achieved through probabilistic projection of the spatial measurement points to the predicted measurement sources on the visible side(s) of the target shape. The spatial uncertainties in the shape features are probabilistically modelled and incorporated in the unscented Kalman filter based estimation. The proposed approach is validated with field experiment results using cameras and a laser range scanner.

Keywords: Vehicle Environment Perception, Mapping and Localization, Self-Driving Vehicles
Abstract: Self-vehicle localization is one of the fundamental tasks for autonomous driving. Most of current techniques for global positioning are based on the use of GNSS (Global Navigation Satellite Systems). However, these solutions do not provide better than 2-3 m in open sky environments [1]. Alternatively, the use of maps has been widely investigated for localization since maps can be pre-built very accurately. State of the art approaches often use dense maps or feature maps for localization. In this paper, we propose a road sign perception system for vehicle localization within a third party map. This is challenging since third party maps are usually provided with sparse geometric features which make the localization task more difficult in comparison to dense maps. The proposed approach extends the work in [2] where a localization system based on lane markings has been developed. Experiments have been conducted on a Highway-like test track using GPS/INS with RTK correction as ground truth. Error evaluations are given as cross-track and along-track errors defined in the curvilinear coordinates [3] related to the map.

Keywords: Lidar Sensing and Perception
Abstract: This work studies a supervised learning method using 3D LiDAR data for autonomous driving applications. A system of semantic segmentation, including range image segmentation, sample generation, track-level annotation and supervised learning, is developed. The formation and content of a data sample is studied intensively to address the specialty of 3D LiDAR data, which can be represented at a Cartesian or a 2D polar coordinate system, and composed of a segment as the foreground and/or the neighborhood points as the background. A CNN-based classifier is trained to map a given sample to an object label. Qualitative and quantitative experiments show that the background information and multiple feature map fusion significantly improve the performance of the classifier.

Keywords: Lidar Sensing and Perception, Autonomous / Intelligent Robotic Vehicles, Image, Radar, Lidar Signal Processing
Abstract: Multiple lidars are prevalently used on mobile vehicles for rendering a broad view to enhance the performance of localization and perception systems. However, precise calibration of multiple lidars is challenging since the feature correspondences in scan points cannot always provide enough constraints. To address this problem, the existing methods require fixed calibration targets in scenes or rely exclusively on additional sensors. In this paper, we present a novel method that enables automatic lidar calibration without these restrictions. Three linearly independent planar surfaces appearing in surroundings is utilized to find correspondences. Two components are developed to ensure the extrinsic parameters to be found: a closed-form solver for initialization and an optimizer for refinement by minimizing a nonlinear cost function. Simulation and experimental results demonstrate the high accuracy of our calibration approach with the rotation and translation errors smaller than 0.05rad and 0.1m respectively.

Keywords: Lidar Sensing and Perception, Unsupervised Learning
Abstract: We propose a method for off-road drivable area extraction using 3D LiDAR data with the goal of autonomous driving application. A specific deep learning framework is designed to deal with the ambiguous area, which is one of the main challenges in the off-road environment. To reduce the considerable demand for human-annotated data for network training, we utilize the information from vast quantities of vehicle paths and auto-generated obstacle labels. Using these auto-generated annotations, the proposed network can be trained using weakly supervised or semi-supervised methods, which can achieve better performance with fewer human annotations. The experiments on our dataset illustrate the reasonability of our framework and the validity of our weakly and semi-supervised methods.

Keywords: Lidar Sensing and Perception, Convolutional Neural Networks, Self-Driving Vehicles
Abstract: This paper presents a novel CNN-based approach for synthesizing high-resolution LiDAR point cloud data. Our approach generates semantically and perceptually realistic results with guidance from specialized loss-functions. First, we utilize a modified per-point loss that addresses missing LiDAR point measurements. Second, we align the quality of our generated output with real-world sensor data by applying a perceptual loss. In large-scale experiments on real-world datasets, we evaluate both the geometric accuracy and semantic segmentation performance using our generated data vs. ground truth. In a mean opinion score testing we further assess the perceptual quality of our generated point clouds. Our results demonstrate a significant quantitative and qualitative improvement in both geometry and semantics over traditional non CNN-based up-sampling methods.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception, Convolutional Neural Networks
Abstract: We propose a deep convolutional object detector for automated driving applications that also estimates classification, pose and shape uncertainty of each detected object. The input consists of a multi-layer grid map which is well-suited for sensor fusion, free-space estimation and machine learning. Based on the estimated pose and shape uncertainty we approximate object hulls with bounded collision probability which we find helpful for subsequent trajectory planning tasks. We train our models based on the KITTI object detection data set. In a quantitative and qualitative evaluation some models show a similar performance and superior robustness compared to previously developed object detectors. However, our evaluation also points to undesired data set properties which should be addressed when training data-driven models or creating new data sets.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception, Self-Driving Vehicles
Abstract: Lidar sensors are often used in mobile robots and autonomous vehicles to complement camera, radar and ultrasonic sensors for environment perception. Typically, perception algorithms are trained to only detect moving and static objects as well as ground estimation, but intentionally ignore weather effects to reduce false detections. In this work, we present an in-depth analysis of automotive lidar performance under harsh weather conditions, i.e. heavy rain and dense fog. An extensive data set has been recorded for various fog and rain conditions, which is the basis for the conducted in-depth analysis of the point cloud under changing environmental conditions. In addition, we introduce a novel approach to detect and classify rain or fog with lidar sensors only and achieve an mean union over intersection of 97.14 % for a data set in controlled environments. The analysis of weather influences on the performance of lidar sensors and the weather detection are important steps towards improving safety levels for autonomous driving in adverse weather conditions by providing reliable information to adapt vehicle behavior.

Keywords: Lidar Sensing and Perception, Convolutional Neural Networks, Self-Driving Vehicles
Abstract: A considerable amount of annotated training data is necessary to achieve state of the art performance in perception tasks using point clouds. Unlike RGB-images, LiDAR point clouds captured with different sensors or varied mounting positions exhibit a significant shift in their input data distribution. This can impede transfer of trained feature extractors between datasets as it degrades performance vastly. We analyze the transferability of point cloud features be- tween two different LiDAR sensor set-ups (32 and 64 vertical scanning planes with different geometry). We propose a super- vised training methodology to learn transferable features in a pre-training step on LiDAR datasets that are heterogeneous in their data and label domains. In extensive experiments on object detection and semantic segmentation in a multi-task setup we analyze the performance of our network architecture under the impact of a change in the input data domain. We show that our pre-training approach effectively increases performance for both target tasks at once without having an actual multi-task dataset available for pre-training.

Keywords: Lidar Sensing and Perception, Image, Radar, Lidar Signal Processing, Vehicle Environment Perception
Abstract: Reliable and precise object tracking is an essential requirement for automated driving. The majority of LiDAR-based tracking algorithms resort to raw range measurements only. In contrast, we propose a novel method to extract compact and salient features from LiDAR intensities. Using the example of an evasive steering maneuver of a leading vehicle, we show that leveraging these intensity features allows for a more accurate estimation of object states. The resulting early detection of target object rotation allows an automated driving system additional time for deriving an appropriate driving policy.

Keywords: Lidar Sensing and Perception, Mapping and Localization, Image, Radar, Lidar Signal Processing
Abstract: Most LiDAR odometry algorithms estimate the transformation between two consecutive frames by estimating the rotation and translation in an intervening fashion. In this paper, we propose our Decoupled LiDAR Odometry (DeLiO), which -- for the first time -- decouples the rotation estimation completely from the translation estimation. In particular, the rotation is estimated by extracting the surface normals from the input point clouds and tracking their characteristic pattern on a unit sphere. Using this rotation the point clouds are unrotated so that the underlying transformation is pure translation, which can be easily estimated using a line cloud approach. An evaluation is performed on the KITTI dataset and the results are compared against state-of-the-art algorithms.

Keywords: Lidar Sensing and Perception, Deep Learning, Automated Vehicles
Abstract: We present a learning-based method to estimate the object bounding box from its 2D bird’s-eye view (BEV) LiDAR points. Our method, entitled BoxNet, exploits a simple deep neural network that can efficiently handle unordered points. The method takes as input the 2D coordinates of all the points and the output is a vector consisting of both the box pose (position and orientation in LiDAR coordinate system) and its size (width and length). In order to deal with the angle discontinuity problem, we propose to estimate the double-angle sinusoidal values rather than the angle itself. We also predict the center relative to the point cloud mean to boost the performance of estimating the location of the box. The proposed method does not rely on the ordering of points as in many existing approaches, and can accurately predict the actual size of the bounding box based on the prior information that is obtained from the training data. BoxNet is validated using the KITTI 3D object dataset, with significant improvement compared with the state-of-the-art non-learning based methods

Keywords: Automated Vehicles, Intelligent Vehicle Software Infrastructure, Situation Analysis and Planning
Abstract: As represented by automated driving systems, the complexity of in-vehicle systems is rapidly increasing. The process related to the design, implementation, and validation of the system has become complicated. To reduce this complexity, a function description method suitable for the development of an automated driving system and an interpreter for directly executing this function description in software are proposed. To describe the function logically and strictly, the description sentence is based on an ontology model for traffic context. The description is also based on one-time predicate logic to enable verification by a theorem prover. By using this method, the behaviors of the automated driving system are directory linked with the description of the function definition.

Keywords: Self-Driving Vehicles, Situation Analysis and Planning
Abstract: In this paper, we introduce a practical two-layer planning framework for autonomous vehicles operating in unknown off-road environment. The first layer refers to a global path planning layer, searching a shortest global path from road network according to given task points. We build up road network through pre-processing on Google earth and refactoring network in terms of vehicle real historical trajectory on real-time. The second layer refers to a local planning layer, solving a real-time planning problem to generate a collision-free and kinematic-feasible local path by a hybrid trajectory planning method. Our method has been verified in real off-road environment. Experimental results show that the proposed planning method performs well in off-road environment.

Keywords: Self-Driving Vehicles, Convolutional Neural Networks, Vision Sensing and Perception
Abstract: Self-driving decides an appropriate control considering the surrounding environment. To this end, self-driving control methods by using a convolutional neural network (CNN) have been studied, which directly input the vehicle-mounted camera image to a network and output a steering directory. However, if we need to control not only steering but also throttle, it is necessary to grasp the state of the car itself in addition to the surrounding environment. Moreover, in order to use CNNs for critical applications such as self-driving, it is important to analyze where the network focuses on the image and to understand the decision making. In this work, we propose a method to solve these problems. First, to control both steering and throttle simultaneously, we propose using the current vehicle speed as the state of the car itself. Second, we introduce an attention branch network (ABN) architecture to a self-driving model, which enables visually analyzing the reason of the self-driving decision making by using an attention map. Experimental results with a driving simulator demonstrate that our method controls a car stably, and we can analyze the decision making by using the attention map.

Keywords: Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles, Security
Abstract: In this paper, we address the problem of secure pose estimation of an autonomous vehicle (AV) under cyber attacks. An extended Kalman filter (EKF) is used to fuse measurements from multiple sensors including GPS, LIDAR, and IMU. To deal with the possible sensor attacks, we design a cumulative sum (CUSUM) detector to monitor the inconsistency between the predicted pose via mathematical model and the sensor measurement. An EKF reconfiguration scheme is proposed to mitigate the influence of sensor attacks once the compromised sensor is identified. The feasibility and effectiveness of the proposed secure pose estimation method are validated using a simulation platform built on Autoware and Gazebo.

Keywords: Automated Vehicles, Self-Driving Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: One remaining challenge for Automated Driving (AD) that remains unclear to this day is its assessment for market release. The application of previous strategies derived from the V-model is infeasible due to the vast amount of required real-road testing to prove safety with an acceptable significance. A full set of requirements covering all possible traffic scenarios for testing and AD system can still not be derived to this day. Several approaches address this issue by either improving the set of test cases or by including other virtual test domains in the assessment process. However, all rely on simulations that can not be validated as a whole and therefore not be used for proving safety. This work addresses this issue and exhibits a method to verify the use of simulation in a scenario-based assessment process. By introducing a pipeline for reprocessing real-world scenarios as test cases we demonstrate where errors emerge and how these can be isolated. We unveil an issue in simulation which may cause behavior changes of the AD function in resimulation and thus makes the straight forward use of simulation in the assessment process impossible. A solution promising to minimize reprocessing errors and to avoid this behavior change is presented. Finally, this enables the local variation of real-world driving tests in a solely simulative context yielding verified and usable results.

Keywords: Assistive Mobility Systems, Societal Impacts, Automated Vehicles
Abstract: Various manufacturers have presented concepts for autonomous vehicles in recent years. However, none of these concepts were designed specifically for the needs of a multi-generation family. A vehicle that meets these requirements can be used independently even by those who are dependent on an accompanying family member when using a conventional car. Furthermore, an autonomous family vehicle, as discussed in this paper, is intended for private use. The basis of the requirements of such a vehicle are the physical and mental abilities of its users. This paper pursues the approach of deriving initial possible requirements for an autonomous family vehicle at the example of considerations regarding two different of accompanied rides in a conventional car. Additionally, safety aspects of such a vehicle are mentioned and potential solutions for a prototypical implementation of an autonomous family vehicle are suggested.

Keywords: Self-Driving Vehicles, Situation Analysis and Planning, Vehicle Control
Abstract: As autonomous vehicles (AVs) inch closer to re- ality, a central requirement for acceptance will be earning the trust of humans in everyday driving situations. In particular, the interaction between AVs and pedestrians is of high importance, as every human is a pedestrian at some point of the day. This paper considers the interaction of a pedestrian and an autonomous vehicle at a mid-block, unsignalized intersection where there is ambiguity over when the pedestrian should cross and when and how the vehicle should yield. By modeling pedestrian behavior through the concept of gap acceptance, the authors show that a hybrid controller with just four distinct modes allows an autonomous vehicle to successfully interact with a pedestrian across a continuous spectrum of possible crosswalk entry behaviors. The controller is validated through extensive simulation and compared to an alternate POMDP solution and experimental results are provided on a research vehicle for a virtual pedestrian.

Keywords: Intelligent Vehicle Software Infrastructure, Self-Driving Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: Although many research vehicle platforms for autonomous driving have been built in the past, hardware design, source code and lessons learned have not been made available for the next generation of demonstrators. This raises the efforts for the research community to contribute results based on real-world evaluations as engineering knowledge of building and maintaining a research vehicle is lost. In this paper, we deliver an analysis of our approach to transferring an open source driving stack to a research vehicle.

We put the hardware and software setup in context to other demonstrators and explain the criteria that led to our chosen hardware and software design. Specifically, we discuss the mapping of the Apollo driving stack to the system layout of our research vehicle, fortuna, including communication with the actuators by a controller running on a real-time hardware platform and the integration of the sensor setup. With our collection of the lessons learned, we encourage a faster setup of such systems by other research groups in the future.

Keywords: Autonomous / Intelligent Robotic Vehicles, Vehicle Control, Collision Avoidance
Abstract: This paper investigates a method for pose control of multiple nonholonomic vehicles using time-optimal control in a Model Predictive Control (MPC) framework. The vehicles are driving on a limited space considering field boundaries to build-up a formation. Distributed Model Predictive Control (DMPC) in a priority-based manner reduces computation time and avoids vehicle collisions. Priorities are automatically set by evaluating the target poses of the vehicles. We demonstrate our method in simulations. It produces near-optimal trajectories and reduces the compuation time in comparison to centralized MPC.

Keywords: Deep Learning, Vision Sensing and Perception, Self-Driving Vehicles
Abstract: The present work demonstrates how an autonomous navigation system of 'End-to-End' deep learning principles is directly improved in its response process, depending on the information obtained by different input images configurations. For this, a methodology was developed to allow working with RGB and depth images, which were obtained through a Microsoft Kinect V2 sensor device. Three cameras were used for this experiment. The images of the different cameras were concatenated or grouped, generating new and different input configurations from the vision system. To develop the presented methodology, two support and validation systems were implemented. Through the process of computer simulation, it was able to test the first approaches and define the most important ones. In order to validate the proposed methodology and solutions in real world situations, a 1/4 scale automotive vehicle was prototyped. Finally, the experiments shows the importance of the use of multi-cameras systems for a better performance of autonomous navigation systems based on End-to-End learning approach, heaving an average error of 2.41 degrees in the best configuration tested, with three RGB cameras.

Keywords: Automated Vehicles, Cooperative Systems (V2X), Cooperative ITS
Abstract: This paper proposes the application of market-based mechanisms to establish cooperative behavior within traffic scenarios involving autonomous vehicles. The aim is to understand the suitability of commonly used auction rules as a potential mechanism for tactical-level collective decision-making in platoon applications considering near-reliable communications. As a proof of feasibility, two auction clearing rules are compared, namely the first-price and the second-price sealed bid auctions. Both rules are tested according to their impact on coalition welfare and communication. To realistically assess the system performance, we designed an integrated simulation platform composed of an agent-based platform, a microscopic traffic model, and a vehicular network simulator. Results show the viability of two rules to maintain high satisfaction among platoon members, that can lead to stable formations and consequently better traffic conditions. From a communication perspective, a non-negligible delay is present, and should be taken into account when implementing an auctioning mechanism for real-world deployment.

Keywords: Automated Vehicles, Cooperative Systems (V2X), Vehicle Control
Abstract: The problem of designing a decentralized Cooperative Adaptive Cruise Control (CACC) resilient to Denial-of-Service (DoS) attacks while satisfying performance requirements is studied. A decentralized proportional-derivative hybrid controller is employed for string stability of a platoon of vehicles. A timer triggering the arrival of new measurements from preceding vehicles augments the closed-loop system. A computationally affordable algorithm based on matrix inequalities is devised to design a CACC controller able to meet performance specifications and to guarantee string stability up to an estimated number of consecutive packet dropouts. Finally, the effectiveness of the proposed approach is shown in a numerical example.

Keywords: Automated Vehicles, Self-Driving Vehicles, Situation Analysis and Planning
Abstract: Autonomous driving requires general methods to generalize unpredictable situations and reason in complex scenarios where safety is critical and the vehicle must react in a reliable manner. In this sense, digital maps are a crucial component for relating the location of the vehicle and identifying the different road features. In this work, we present a decision-making architecture which does not require detailed prior maps. Instead, OSM is used to plan a global route and an automatically generate driving corridors, which are adapted using a proposed vision-based algorithm. Moreover, a grid-based approach is also applied to consider the localization uncertainty. Those self-generated driving corridors are used by the local planner to plan the trajectories the vehicle will follow. Our approach integrates global, local and HMI components to provide the required functionalities for autonomous driving in a general manner.

Keywords: Automated Vehicles, Self-Driving Vehicles, Vehicle Control
Abstract: Overtaking is a challenging task in the field of autonomous driving, especially on roads with an opposite lane and oncoming vehicles. Since trajectory planning is repeated cyclic it is highly important to trigger the maneuver only if it is guaranteed that collision-free trajectories that satisfy kinematic constraints exist at each planning step. The goal of this paper is to present an algorithm for planning overtaking trajectories on large temporal horizons in real-time. The main idea is as follows: once overtaking is desired by the behavior module an initial trajectory is simulated using a path tracking control algorithm for lane changing combined with a classical PI-controller for approaching the target speed. The controllers are parametrized in a way that the simulated trajectory will satisfy kinematic constraints. If no collisions are detected a corridor containing the simulated trajectory is created to state constraints for a subsequent optimal control problem to relax the trajectory and smooth it to be comfortable to the vehicle passengers.

Keywords: Self-Driving Vehicles, Active and Passive Vehicle Safety, Vulnerable Road-User Safety
Abstract: This paper presents our research platform SafeVRU for the interaction of self-driving vehicles with Vulnerable Road Users (VRUs, i.e., pedestrians and cyclists). The paper details the design (implemented with a modular structure within ROS) of the full stack of vehicle localization, environment perception, motion planning, and control, with emphasis on the environment perception and planning modules. The environment perception detects the VRUs using a stereo camera and predicts their paths with Dynamic Bayesian Networks (DBNs), which can account for switching dynamics. The motion planner is based on model predictive contouring control (MPCC) and takes into account vehicle dynamics, control objectives (e.g., desired speed), and perceived environment (i.e., the predicted VRU paths with behavioral uncertainties) over a certain time horizon. We present simulation and real-world results to illustrate the ability of our vehicle to plan and execute collision-free trajectories in the presence of VRUs.

Keywords: Advanced Driver Assistance Systems, Human Factors and Human Machine Interaction, Automated Vehicles
Abstract: The situation awareness of drivers during takeover from autonomous to manual mode is important for avoidance of accidents. Previous studies have revealed that takeover time and general performance vary strongly in different situations. The studies also revealed that the variation is due to surrounding traffic conditions, complexity of the driving scenario, secondary tasks, speed of ego vehicle, and takeover request experience. The aim of this study is to further explore the scope and dependencies of the aforementioned variables to better equip driver assistance and supervision systems with the necessary framework to suitably assist drivers during takeovers. In other words, the intention is to define a formal set of rules to enable the automated driving system determine a suitable takeover request time for different scenarios. First, this contribution discusses the design of takeover variables such that the effects of the variables are systematically varied to generate different driving situations. Afterwards, experimental results under different variable combinations are discussed. The results include a comparison of objective measures and subjective measures. An initial set of rules are established to model the interaction to improve intelligent driver assistance systems.

Keywords: Human-Machine Interface, Sensor and Data Fusion, Convolutional Neural Networks
Abstract: Assessing the driver's attention and detecting various hazardous and non-hazardous events during a drive are critical for driver's safety. Attention monitoring in driving scenarios has mostly been carried out using vision (camera-based) modality by tracking the driver's gaze and facial expressions. It is only recently that bio-sensing modalities such as Electroencephalogram (EEG) are being explored. But, there is another open problem which has not been explored sufficiently yet in this paradigm. This is the detection of specific events, hazardous and non-hazardous, during driving that affects the driver's mental and physiological states. The other challenge in evaluating multi-modal sensory applications is the absence of very large scale EEG data because of the various limitations of using EEG in the real world. In this paper, we use both of the above sensor modalities and compare them against the two tasks of assessing the driver's attention and detecting hazardous vs. non-hazardous driving events. We collect user data on twelve subjects and show how in the absence of very large-scale datasets, we can still use pre-trained deep learning convolution networks to extract meaningful features from both of the above modalities. We used the publicly available KITTI dataset for evaluating our platform and to compare it with previous studies. Finally, we show that the results presented in this paper surpass the previous benchmark set up in the above driver awareness-related applications.

Keywords: Human-Machine Interface, Driver Recognition, Novel Interfaces and Displays
Abstract: Automotive manufactures are rapidly developing more advanced in-vehicle systems that seek to provide a driver with more active safety and information in real-time, in particular human machine interfaces (HMIs) using mixed or augmented reality (AR) graphical elements. However, it is difficult to properly test novel AR interfaces in the same way as traditional HMIs via on-road testing. Instead, simulation could likely offer a safer and more financially viable alternative for testing AR HMIs, inconsistent simulation quality may confound HMI research depending on the visual fidelity of each simulation environment. We investigated how visual fidelity in a virtual environment impacts the quality of resulting driver behavior, visual attention, and situational awareness when using the system. We designed two large-scale immersive virtual environments; a “low” graphic fidelity driving simulation representing most current research simulation testbeds and a “high” graphic fidelity environment created in Unreal Engine that represents state of the art graphical presentation. We conducted a user study with 24 participants who navigated a route in a virtual urban environment via direction of AR graphical cues while also monitoring the road scene for pedestrian hazards, and recorded their driving performance, gaze patterns, and subjective feedback via situational awareness survey (SART). Our results show drivers change both their driving and visual behavior depending upon the visual fidelity presented in the virtual scene. We further demonstrate the value of using multi-tiered analysis techniques to more finely examine driver performance and visual attention.

Keywords: Novel Interfaces and Displays, Human-Machine Interface
Abstract: In the near future, automated vehicles will be integrated in traffic. This will lead to interactions between automated vehicles and drivers of manually driven cars. Due to the fact that with a distracted passenger, communication is only possible to a limited extent, this paper researches the potential of external human-machine interfaces in a road bottleneck scenario in urban traffic. To investigate this capability, a driving simulator study with 46 participants was conducted. The experiment varied the message of the automated vehicle as well as the moment of displaying it. The two concepts consisted of a display, which was mounted at the front of the automated vehicle and a laser projection. The research question this paper addresses is, whether displaying the right of way can support the oncoming human driver in passing through the narrow section of the road. The results show that the passing time of the participants reduces significantly when communicating via an external human-machine interface compared to the condition without an interface.

Keywords: Novel Interfaces and Displays, Image, Radar, Lidar Signal Processing, Deep Learning
Abstract: Generating annotations which can be used to train new models has become an independent field of research within machine learning. Its goal is producing highly accurate annotations as cost efficient as possible. 3D point clouds are the common sensor output when recording 3D data from a mobile platform. The latest ways of annotating 3D point clouds include their visualization on a 2D screen. This method contradicts the goal of time-efficient annotating since it is unintuitive and therefore unnecessarily time consuming. We present a novel labeling technique in Virtual Reality. Using our tool, we accelerate the process of data annotation significantly compared to existing approaches. Furthermore, we will give the machine learning community access to our tool and create a new community-labeled dataset for autonomous driving. Furthermore we plan to set up an annotation benchmark in which primarily commercial annotation companies but also researchers active in annotation can take part in. We present results from an experimental plattform based on Oculus Rift indicating a huge potential for VR annotations.

Keywords: Novel Interfaces and Displays, Hand-off/Take-Over, Automated Vehicles
Abstract: Drivers' ability to engage in non-driving related tasks (NDRTs) is a promise of automated driving and office work an important use-case therein. We claim that potentially negative effects on road safety need to be compensated by adaptive in-vehicle interfaces that support NDRTs by design. In this paper, we present the conception and evaluation of a novel dual-task interface that is based on findings from both automotive research and office ergonomics. The steering wheel prototype aims at enabling productivity while retaining or even improving safety in Take-Over situations. In a driving simulator study with N=22 participants, we tested the prototype in two variations, haptic vs. touchscreen keyboard design, against the baseline ``notebook on the lap''. Results show significant improvements regarding gaze reaction, typing performance, and subjective ratings of the haptic as compared to the touch keyboard. Most promisingly, Take-Over reaction time decreased by 40 percent when using the haptic prototype instead of the conventional notebook. Based on our findings, we recommend the use of adaptive input devices to assist the driver and prevent mode confusion. We further suggest avoiding the use of tablets in L3 driving - even when integrated into the steering wheel - in order to meet safety requirements.

Keywords: Human Factors and Human Machine Interaction, Situation Analysis and Planning, Passive Safety
Abstract: In shared space, pedestrians are often found walking in groups and behaving differently than individual pedestrians. However, automatically detecting pedestrian groups with high accuracy is not trivial given the dynamic environment and interactions in mixed traffic. Instead of tedious manual work and in order to cope with large scales of data, we propose a time-sequence DBSCAN (Density-Based Spatial Clustering of Applications with Noise) for pedestrian group detection. It is based on pedestrian coexisting time and Euclidean distance between them. Our approach outputs reliable results with high IoU (Interaction over Union) values. It can be easily adapted to other groups, e.g., cyclists and animals. In addition to individual behavior, the output data with the differentiation of group behavior can be used in further studies in intent detection and motion prediction.

Keywords: Intelligent Vehicle Software Infrastructure, Hand-off/Take-Over, Advanced Driver Assistance Systems
Abstract: Self-driving cars rely on a plethora of algorithms in order to perform safe driving manoeuvres. Training those models is expensive (e.g. hardware cost, storage, energy) and requires continuous updates. This paper proposes a cloud-based framework for continuous training of self-driving AI models. In addition to training standalone models, the framework is capable of leveraging pre-trained models in expediting the training on environment changes (e.g. new driver or new car model). As use-case, this paper focuses on a driver's behaviour while the vehicle's control is being transferred between the driver and the self-driving AI.

A human driver can hand over the control of a vehicle's driving tasks to an automated system, when that system's confidence level is high enough. Reciprocally, there are situations where that control has to be handed back to the human driver. This paper proposes a novel real-time system for Driving Not-Driving (DND) detection, which is able to capture the ability of the driver to re-take control of a vehicle when the automated driving system transitions from a higher to a lower level of automation (e.g. L3 to L2 vehicle automation). We are using a computer vision-based Driver Monitoring System (DMS) that captures in real-time head and eye movements. These are captured in the car and transferred to the cloud where a DND model is trained for a specific driver. The DND classification model is deployed in the vehicle and predicts if the driver is ready or not to resume control at a given time.

The cloud-based framework proposed in this paper shows an end-to-end cycle of collecting, training and deploying self-driving AI technology, with the additional features of continuous and transfer learning.

Keywords: Reinforcement Learning, Sensor and Data Fusion, Image, Radar, Lidar Signal Processing
Abstract: We formulate camera and LiDAR fusion tracking as a sequential decision-making process. With our deep reinforcement learning framework, we try to optimize the tracking trajectory to be as accurate, smooth, and long as possible. In contrast to traditional fusion algorithms involving complex feature and strategy design and hyperparameters tuned for different scenarios, our fusion agent can learn the confidence of each input by tracking the results from raw observation in a data-driven fashion. Given the input states of different sensors, our approach chooses one input with a higher expected cumulative reward as the observation of a Kalman filter to iteratively predict the target position. The expected cumulative reward is estimated with a convolutional neural network, trained with a modified DQN algorithm, which takes inputs from both LiDAR and a camera. Through case studies and quantitative result evaluation on our dataset from the 4th Ring Road in Beijing, our algorithm is validated to achieve more accurate and robust tracking performance.

Keywords: Image, Radar, Lidar Signal Processing, Sensor and Data Fusion
Abstract: This work deals with the efficient fusion of multiple disparate sensors, namely THz radar, stereo camera and lidar for autonomous vehicles. In particular, we develop a target tracking algorithm that is object agnostic i.e. we seek to detect any potential object in the scene and track it while also preserving extended target characteristics such as length and width. To this end, we first use conventional clustering and labelling methods in order to generate consistent features from each sensor independently. The features from each sensor are then transformed into a set of bounding boxes located in both range and cross range. The bounding box parameters are then fed into the proposed efficient multi-sensor target tracking algorithm. This is achieved by modifying the Gaussian mixture-PHD filter (GM-PHD) by incorporating a set of class labels that associate a state to a set of sensors. The performance of the proposed method target tracking method is verified using both synthetic and real world data.

Keywords: Information Fusion, Cooperative Systems (V2X), Smart Infrastructure
Abstract: Data from infrastructure sensors can significantly improve the field of view for intelligent vehicles (IV), both in terms of range and completeness. In the MEC-View project, we investigate how automated driving (AD) can benefit from incorporating such data in the perception processing chain. On the infrastructure side, a central computational node, called MEC-Server, is connected to a base station and receives objects from multiple roadside sensors. Those are used to create a fused environmental model, which is distributed to vehicles close by via a managed cellular network.

To use tracks received from the MEC-Server in IV perception, we propose a hybrid vehicular perception system that is able to fuse both local onboard sensor data as well as estimations by the MEC-Server. For this, we discuss multiple approaches to track-level fusion and data association, including their application in our perception system. Using careful interface design, we are able to avoid many non-linearities and are able to minimize the amount of approximations involved.

For evaluation, we present a experimental setup for track-level fusion schemes that is based on virtually augmented real-world measurements and facilitates targeted adaptation of influencing variables while ensuring real-world applicability. A comparison of different fusion schemes provides insights into their relative performance and shows directions for real-world applicability.

Keywords: Information Fusion, Vehicle Environment Perception, Mapping and Localization
Abstract: In this paper we propose a method for accurate ego-lane localization using camera images, on-board sensors and lanes number information from OpenStreetMap (OSM). The novelty relies in the probabilistic framework developed, as we introduce a modular Bayesian Network (BN) to infer the ego-lane position from multiple inaccurate information sources. The flexibility of the BN is proven, by first, using only information from surrounding lane-marking detections and second, by adding adjacent vehicles detection information. Afterward, we design a Hidden Markov Model (HMM) to temporary filter the outcome of the BN using the lane change information. The effectiveness of the algorithm is first verified on recorded images of national highway in the region of Clermont-Ferrand. Then, the performances are validated on more challenging scenarios and compared to an existing method, whose authors made their datasets public. Consequently, the results achieved highlight the modularity of the BN. In addition, our proposed algorithm outperforms the existing method, since it provides more accurate ego-lane localization: 85:35% compared to 77%.

Keywords: Advanced Driver Assistance Systems, Sensor and Data Fusion, Active and Passive Vehicle Safety
Abstract: External sensing for automative applications are key tools for the development of Advanced Driver Assistance Systems (ADAS), since they can sense and analyse the environment around the vehicle by providing pictures of the scene behind the vehicle. Parking assistance systems are already available in the market. However, most of these applications are based on ultrasonic sensors, wide-angle image cameras, RADAR, etc, which present some drawbacks such as dependency to light conditions or high maintenance cost, among others. This paper proposes an approach for assisting drivers to park through the processing of data derived by a 3D Time-of-Flight (ToF) camera and the reconstruction of the objects identified around the vehicle. The proposed technique is focused on fusion of two parallel processing technologies, a visual one through the intensity image and a spatial one through the point cloud. Both of them are centered on the detection of the vehicle's plate to estimate its position and determine free spots in the parking. This novel methodology improves the detection of surrounding elements, since it helps to solve two main problems with this kind of devices: 1) the degraded performance under bright ambient light problem (occurring mainly in outdoor parkings), that causes shadows and brightness in the images, hindering its process to detect objects; and 2) the limited detection of low reflection objects such as dark cars. Moreover, this fusion allows to link each pixel of the image with a 3D position, and vice versa, giving the point cloud a visual reference. The system is evaluated through a Renault Twizy platform in real conditions.

Keywords: Sensor and Data Fusion, Radar Sensing and Perception, Vehicle Environment Perception
Abstract: Occupancy grids (OG) are widely used for low-level fusion of radar data in various automotive applications. At the core of OG generation, usually, there is an inverse sensor model (ISM), which is a conditional cell occupancy probability model. Traditional ISM's lack mechanisms decreasing occupancy likelihoods along directions that produce no detections; thus, false detections tend to perpetuate on the OG. In this paper, we propose a novel Inverse Sensor Model including a "positive" component describing occupancy probabilities induced by radar detections and a "negative" component handling lack of detections in a given direction. This dual model proves especially useful in multi-sensor/multi-frame context since false detections by different radars and/or at different moments are uncorrelated and thus can be efficiently mitigated.

Keywords: Sensor and Data Fusion, Image, Radar, Lidar Signal Processing, Vehicle Environment Perception
Abstract: Early and accurate detection of crossing pedestrians is crucial in automated driving to execute emergency manoeuvres in time. This is a challenging task in urban scenarios however, where people are often occluded (not visible) behind objects, e.g. other parked vehicles. In this paper, an occlusion aware multi-modal sensor fusion system is proposed to address scenarios with crossing pedestrians behind parked vehicles. Our proposed method adjusts the detection rate in different areas based on sensor visibility. We argue that using this occlusion information can help to evaluate the measurements. Our experiments on real world data show that fusing radar and camera for such tasks is beneficial, and that including occlusion into the model helps to detect pedestrians earlier and more accurately.

Keywords: Sensor and Data Fusion, Automated Vehicles
Abstract: In the field of autonomous driving, multi-beam light detection and ranging (3D LIDAR) system and global navigation satellite system/integrated inertial navigation system(GNSS/INS) are widely used in high-definition map construction, localization and obstacle detection. As 3D LIDAR system and INS have their own coordinate systems, the calibration of the two mentioned systems is required. In this paper, a new algorithm for calibrating the coordinate system of 3D LIDAR and INS is proposed, which consists of three parts. The first procedure is to project two point clouds to the world coordinate system based on the initial transform matrix between 3D LIDAR and INS with the real-time data from INS. Then optimal point-to-point correspondences can be found between two frames of point cloud data through registration method. Finally, the loss function is constructed with the sum of the Euclidean distances of the corresponding points and optimized by using perturbation model of Lie algebras, so as to obtain the optimal transform matrix.With different given initial calibration parameters, test results of both simulation and real experiments validate the proposed algorithm and quantify its accuracy and robustness.

Keywords: Sensor and Data Fusion, Autonomous / Intelligent Robotic Vehicles
Abstract: Fusing multiple sensors is vital for the precise on-line localization of mobile robots. The Extended Kalman filter is well suited for this purpose and allows to directly integrate multiple absolute measurements in the filter. However, multiple relative state measurements cannot be fused directly into the filter because they measure differences between a past state and the current state, which introduces correlations. These interdependencies can be modeled by stochastic cloning, which introduces a state augmentation by cloning the respective state estimates connected by a relative measurement. This paper investigates the impact of multiple mixed relative and absolute updates on the estimated state. An approach to feed back information from absolute updates to the cloned state by explicit cloning is presented and compared to existing implicit methods.

Keywords: Lidar Sensing and Perception, Mapping and Localization, Sensor and Data Fusion
Abstract: Localization is an essential technique for many robotic tasks such as mapping and navigation. Scan matching has been fused with other sensors to solve the problem at GPS restricted areas, where an accurate error model describing matching precision at various scenes is indispensable. We proposed an end-to-end method to learn a scene adaptive error model of LiDAR scan matching. A CNN (Convolutional Neural Network) is learnt to map from a LiDAR scan to an information matrix of the matching result, and a localization framework is proposed to fuse the results of LiDAR scan matching based on its error model. Experiments are conducted using both simulated and real world data, where the former is to validate the proposed method of its adaptability at various simple but typical scenes, while the later is to examine the method's practicability at real world environments. We demonstrate the performance of learning covariance error model, and examine the localization accuracy by comparing with other traditional methods. Efficiency of the proposed method is demonstrated.

Keywords: Sensor and Data Fusion
Abstract: A fast lidar-camera fusion method is proposed to detect road in autonomous vehicles. The height data of lidar is transformed to spherical coordinate system to increase the data density. The RGB data of camera is also transformed to spherical coordinate system to match with the lidar data. The amount of data is greatly reduced by spherical coordinate transformation, which results in the fast running time. The transformed images of height, red, green, and blue are input to the CNN. A dialed convolution structure is newly proposed to improve the learning accuracy by expanding the receptive field of CNN. The experimental results using the KITTI data set are finally presented to show the usefulness of the proposed method.

Keywords: Sensor and Data Fusion, Image, Radar, Lidar Signal Processing, Vehicle Environment Perception
Abstract: For the purpose of extended object tracking in multiple hypothesis tracking algorithms such as the Gaussian mixture probability hypothesis density filter (GMPHD), we develop an approach for the combination of different contour estimates. The developed approach works for tracking algorithms that represent target shapes using contour functions to describe the target shape as the distance of the contour to a reference point over the angle. In a heterogeneous multiple sensor setup, the individual sensors' measurements lead to different extent estimates due to their individual measurement principles. Thus a straight forward use of the extended object state in the traditional merging algorithm either results in unexpected shapes, or tracks cannot be merged due to the differing shape of the objects. Our merging procedure explicitly takes the extent estimates into account by using a merging function. The choice of the merging function provides the means to reach objectives such as a conservative or a generous extent estimate. We evaluate the approach using simulated multi-sensor data in a GMPHD filter. Compared to the traditional merging method, our approach results in better shape estimates.

Keywords: Sensor and Data Fusion, Vehicle Environment Perception, Intelligent Ground, Air and Space Vehicles
Abstract: This article presents a methodology to estimate with a single and integrated observer (virtual sensor) the main components of the suspension system: suspension state, load transfer and road profile. The proposed observer (an embedded algorithm) provides the system state, the road profile, the normal tire-ground forces and the load transfer without requiring expensive sensors. The sensors required are standard in commercial vehicles with actuated or semi-actuated suspension. The observer structure is based on Kalman filter technique due to its implementation simplicity. The proposal is validated at different operation points using a high fidelity automotive simulator, OKTAL-SCANeR TM studio-Callas® and experimental data.

Keywords: Self-Driving Vehicles, Information Fusion, Sensor and Data Fusion
Abstract: In this paper, we focus on obtaining 2D and 3D labels, as well as track IDs for objects on the road with the help of a novel 3D Bounding Box Annotation Toolbox (3D BAT). Our open source, web-based 3D BAT incorporates several smart features to improve usability and efficiency. For instance, this annotation toolbox supports semi-automatic labeling of tracks using interpolation, which is vital for downstream tasks like tracking, motion planning and motion prediction. Moreover, annotations for all camera images are automatically obtained by projecting annotations from 3D space into the image domain. In addition to the raw image and point cloud feeds, a Masterview consisting of the top view (bird's-eye-view), side view and front views is made available to observe objects of interest from different perspectives. Comparisons of our method with other publicly available annotation tools reveal that 3D annotations can be obtained faster and more efficiently by using our toolbox.

Keywords: Automated Vehicles, Cooperative Systems (V2X)
Abstract: This talk with discuss the state-of-the-art and a potential evolution of self-driving cars. It will outline approaches and challenges for achieving full autonomy for self-driving cars and elaborate the potential of cooperativity for automated driving. The talk will look at market introduction issues and how these are approached by different stakeholders. We will look at many examples, including the DARPA and GCDC Challenges. The talk will draw on lessons learned and challenges to be overcome from the perspective of the German Research Priority Program "Cooperative Interactive Automobiles".

Keywords: Collision Avoidance, Vehicle Control, Autonomous / Intelligent Robotic Vehicles
Abstract: Emergency scenarios may require trajectory planning at actuator and friction limits within tight obstacle constraints. This paper presents an approach to handle such scenarios using Linear Time Varying Model Predictive Control, in the presence of both static and dynamic obstacles. With the proposed approach, the controller transitions between a kinematic vehicle model, which is stable at low velocities, and a dynamic vehicle model, which is more accurate at high velocities, based on a threshold velocity. This ensures the functionality of the controller in emergency braking scenarios and enables manoeuvre initialization with a full braking trajectory. An adjustable safety distance concept is introduced that considers the criticality of the scenario and tightens or loosens the obstacle avoidance and friction constraints accordingly. Simulation results show that the controller is capable of handling various critical situations, including those when the vehicle comes to a stop at the end of a high speed evasive manoeuvre. Furthermore, it is more effective at keeping a safety distance from obstacles in such manoeuvres than by using fixed safety distance concept.

Keywords: Collision Avoidance, Advanced Driver Assistance Systems, Human-Machine Interface
Abstract: Automated emergency braking (AEB) systems—which alert a driver to approaching hazards and automatically brake—are currently available in some vehicles and will soon be widespread. Due to the uncertainties inherent in any environment and difficulties in processing sensor data, these systems are prone to both false alarms and system misses of hazards. A pressing design concern is whether to bias these systems toward a higher likelihood of false alarms versus system misses for non-fatal events. We investigated how drivers form mental models of the AEB system and how that influences their reliance on the system during a critical, potentially fatal, failure. In a full vehicle driving simulator, participants experienced nine interactions that reflected the system’s level of bias toward false alarms or toward misses for non-fatal events or that demonstrated perfect performance. When a potentially fatal event occurred, participants trained to expect misses were better able to avoid a pedestrian in the road after a detection failure than those using a system with perfect performance or false alarms. These findings suggest that systems biased toward misses in non-fatal events encourage driver vigilance and preparedness for potentially fatal events.

Keywords: Vehicle Control, Electric and Hybrid Technologies, Eco-driving and Energy-efficient Vehicles
Abstract: Rule-based (RB) energy management strategies are widely used in hybrid-electric vehicles because they are easy to implement and can be used without prior knowledge about future trips. In the literature, parameters used in RB methods are tuned and designed using known driving cycles. Although promising results have been demonstrated, it is difficult to apply such cycle-specific methods on real trips of last-mile delivery vehicles that have significant trip-to-trip differences in distance and energy intensity. In this paper, a reinforcement learning method and a RB strategy is used to improve the fuel economy of an in-use extended range electric vehicle (EREV) used in a last-mile package delivery application. An intelligent agent is trained on historical trips of a single delivery vehicle to tune a parameter in the engine-generator control logic during the trip using real-time information. The method is demonstrated on actual historical delivery trips in a simulation environment. An average of 19.5% in fuel efficiency improvement in miles per gallon gasoline equivalent is achieved on 44 test trips with a distance range of 31 miles to 54 miles not used for training, demonstrating promise to generalize the method. The presented framework is extendable to other RB methods and EREV applications like transit buses and commuter vehicles where similar trips are frequently repeated day-to-day.

Keywords: Vehicle Control, Automated Vehicles, Advanced Driver Assistance Systems
Abstract: In this work, a comparative study is carried out with two different predictive controllers that consider the longitudinal and lateral jerks as additional parameters, so that comfort constraints can be included. Furthermore, the approaches are designed so that the effect of longitudinal and lateral motion control coupling can be analyzed. This way, the first controller is a longitudinal and lateral coupled MPC approach based on a kinematic model of the vehicle, while the second is a decoupled strategy based on a triple integrator model based on MPC for the longitudinal control and a double proportional curvature control for the lateral motion control. The control architecture and motion planning are exhaustively explained. The comparative study is carried out using a test vehicle, whose dynamics and low-level controllers have been simulated using the realistic simulation environment Dynacar. The performed tests demonstrate the effectiveness of both approaches in speeds higher than 30 km/h, and demonstrate that the coupled strategy provides better performance than the decoupled one. The relevance of this work relies in the contribution of vehicle motion controllers considering the comfort and its advantage over decoupled alternatives for future implementation in real vehicles.

Keywords: Cooperative Systems (V2X), Sensor and Data Fusion, Vehicle Environment Perception
Abstract: A fundamental for an automated driving car is the awareness of all its surrounding road participants. Current approach to gather this awareness is to sense the environment by on-board sensors, like camera or radar. In the future Vehicle-to-X (V2X) might be able to improve the awareness, due to V2X's communication range superiority compared to the on-board sensors' range. Due to a limited amount of communication partners sharing their own ego states, current research focuses particularly on cooperative perception. This means sharing objects perceived by local on-board sensors of different partners via V2X. In this paper, temporal and spatial alignment of the shared objects is reviewed at track-level. State-of-the-art during the alignment procedure is the compensation of the sender motion in the object state and to perform the coordinate transformation of the object state using the predicted sender state. We propose to use the non-predicted sender state for the transformation and therefore to neglect the sender motion compensation. Finally, both approaches are evaluated to figure out the best choice.

Keywords: V2X Communication, Vehicle Environment Perception, Advanced Driver Assistance Systems
Abstract: We propose a prediction model to forecast the coverage of road networks in vehicle-to-vehicle or vehicle-to-infrastructure (V2X) networks for cloud-based automotive applications. The model is derived from fleet tests in the City of Munich (Germany). It considers the fleet and the road network characteristics by splitting the network into sub-networks and using the fleet’s relative mileage on the sub-networks. The correlation of the spatial coverage and the fleet’s mileage is analyzed for each sub-network showing that the expected degressive correlation exists. The derived regression model also shows a comparable fit for a data series taking the driving direction into account. Finally, we validated the model’s ability to predict the temporal coverage by reducing the considered time intervals and taking the number of observations into account. The results show that the model can be used to predict the availability, the up-to-dateness and the accuracy of extended floating car data (XFCD).

Keywords: V2X Communication, Telematics, Smart Infrastructure
Abstract: Low latency is required for connected and intelligent vehicle applications. For instance, safety applications have strict latency requirements. Mobile edge computing (MEC) improves the latency by bringing the computational resources closer to the data source. However, despite the improvements in latency by MEC, the latency will vary based on traffic load and signal conditions. We seek to answer the question: is the current latency environment acceptable for our application? This paper presents our Reliable Latency Decision (ReLaDec) algorithm. ReLaDec uses prior information and the last latency samples to decide whether the latency will be acceptable to our application. The decisions are done with predictable confidence and with ReLaDec the false positive decision rate never exceeds the set maximum value. We demonstrate the performance of the algorithm using 860000 latency samples that include stationary and driving data over multiple USA states.

Keywords: Collision Avoidance, Vehicle Control, Autonomous / Intelligent Robotic Vehicles
Abstract: Emergency scenarios may require trajectory planning at actuator and friction limits within tight obstacle constraints. This paper presents an approach to handle such scenarios using Linear Time Varying Model Predictive Control, in the presence of both static and dynamic obstacles. With the proposed approach, the controller transitions between a kinematic vehicle model, which is stable at low velocities, and a dynamic vehicle model, which is more accurate at high velocities, based on a threshold velocity. This ensures the functionality of the controller in emergency braking scenarios and enables manoeuvre initialization with a full braking trajectory. An adjustable safety distance concept is introduced that considers the criticality of the scenario and tightens or loosens the obstacle avoidance and friction constraints accordingly. Simulation results show that the controller is capable of handling various critical situations, including those when the vehicle comes to a stop at the end of a high speed evasive manoeuvre. Furthermore, it is more effective at keeping a safety distance from obstacles in such manoeuvres than by using fixed safety distance concept.

Keywords: Collision Avoidance, Advanced Driver Assistance Systems, Human-Machine Interface
Abstract: Automated emergency braking (AEB) systems—which alert a driver to approaching hazards and automatically brake—are currently available in some vehicles and will soon be widespread. Due to the uncertainties inherent in any environment and difficulties in processing sensor data, these systems are prone to both false alarms and system misses of hazards. A pressing design concern is whether to bias these systems toward a higher likelihood of false alarms versus system misses for non-fatal events. We investigated how drivers form mental models of the AEB system and how that influences their reliance on the system during a critical, potentially fatal, failure. In a full vehicle driving simulator, participants experienced nine interactions that reflected the system’s level of bias toward false alarms or toward misses for non-fatal events or that demonstrated perfect performance. When a potentially fatal event occurred, participants trained to expect misses were better able to avoid a pedestrian in the road after a detection failure than those using a system with perfect performance or false alarms. These findings suggest that systems biased toward misses in non-fatal events encourage driver vigilance and preparedness for potentially fatal events.

Keywords: Vehicle Control, Electric and Hybrid Technologies, Eco-driving and Energy-efficient Vehicles
Abstract: Rule-based (RB) energy management strategies are widely used in hybrid-electric vehicles because they are easy to implement and can be used without prior knowledge about future trips. In the literature, parameters used in RB methods are tuned and designed using known driving cycles. Although promising results have been demonstrated, it is difficult to apply such cycle-specific methods on real trips of last-mile delivery vehicles that have significant trip-to-trip differences in distance and energy intensity. In this paper, a reinforcement learning method and a RB strategy is used to improve the fuel economy of an in-use extended range electric vehicle (EREV) used in a last-mile package delivery application. An intelligent agent is trained on historical trips of a single delivery vehicle to tune a parameter in the engine-generator control logic during the trip using real-time information. The method is demonstrated on actual historical delivery trips in a simulation environment. An average of 19.5% in fuel efficiency improvement in miles per gallon gasoline equivalent is achieved on 44 test trips with a distance range of 31 miles to 54 miles not used for training, demonstrating promise to generalize the method. The presented framework is extendable to other RB methods and EREV applications like transit buses and commuter vehicles where similar trips are frequently repeated day-to-day.

Keywords: Vehicle Control, Automated Vehicles, Advanced Driver Assistance Systems
Abstract: In this work, a comparative study is carried out with two different predictive controllers that consider the longitudinal and lateral jerks as additional parameters, so that comfort constraints can be included. Furthermore, the approaches are designed so that the effect of longitudinal and lateral motion control coupling can be analyzed. This way, the first controller is a longitudinal and lateral coupled MPC approach based on a kinematic model of the vehicle, while the second is a decoupled strategy based on a triple integrator model based on MPC for the longitudinal control and a double proportional curvature control for the lateral motion control. The control architecture and motion planning are exhaustively explained. The comparative study is carried out using a test vehicle, whose dynamics and low-level controllers have been simulated using the realistic simulation environment Dynacar. The performed tests demonstrate the effectiveness of both approaches in speeds higher than 30 km/h, and demonstrate that the coupled strategy provides better performance than the decoupled one. The relevance of this work relies in the contribution of vehicle motion controllers considering the comfort and its advantage over decoupled alternatives for future implementation in real vehicles.

Keywords: Cooperative Systems (V2X), Sensor and Data Fusion, Vehicle Environment Perception
Abstract: A fundamental for an automated driving car is the awareness of all its surrounding road participants. Current approach to gather this awareness is to sense the environment by on-board sensors, like camera or radar. In the future Vehicle-to-X (V2X) might be able to improve the awareness, due to V2X's communication range superiority compared to the on-board sensors' range. Due to a limited amount of communication partners sharing their own ego states, current research focuses particularly on cooperative perception. This means sharing objects perceived by local on-board sensors of different partners via V2X. In this paper, temporal and spatial alignment of the shared objects is reviewed at track-level. State-of-the-art during the alignment procedure is the compensation of the sender motion in the object state and to perform the coordinate transformation of the object state using the predicted sender state. We propose to use the non-predicted sender state for the transformation and therefore to neglect the sender motion compensation. Finally, both approaches are evaluated to figure out the best choice.

Keywords: V2X Communication, Vehicle Environment Perception, Advanced Driver Assistance Systems
Abstract: We propose a prediction model to forecast the coverage of road networks in vehicle-to-vehicle or vehicle-to-infrastructure (V2X) networks for cloud-based automotive applications. The model is derived from fleet tests in the City of Munich (Germany). It considers the fleet and the road network characteristics by splitting the network into sub-networks and using the fleet’s relative mileage on the sub-networks. The correlation of the spatial coverage and the fleet’s mileage is analyzed for each sub-network showing that the expected degressive correlation exists. The derived regression model also shows a comparable fit for a data series taking the driving direction into account. Finally, we validated the model’s ability to predict the temporal coverage by reducing the considered time intervals and taking the number of observations into account. The results show that the model can be used to predict the availability, the up-to-dateness and the accuracy of extended floating car data (XFCD).

Keywords: V2X Communication, Telematics, Smart Infrastructure
Abstract: Low latency is required for connected and intelligent vehicle applications. For instance, safety applications have strict latency requirements. Mobile edge computing (MEC) improves the latency by bringing the computational resources closer to the data source. However, despite the improvements in latency by MEC, the latency will vary based on traffic load and signal conditions. We seek to answer the question: is the current latency environment acceptable for our application? This paper presents our Reliable Latency Decision (ReLaDec) algorithm. ReLaDec uses prior information and the last latency samples to decide whether the latency will be acceptable to our application. The decisions are done with predictable confidence and with ReLaDec the false positive decision rate never exceeds the set maximum value. We demonstrate the performance of the algorithm using 860000 latency samples that include stationary and driving data over multiple USA states.

Keywords: Security
Abstract: The transformation in the automotive industry continues and topics such as intelligent software applications and over-the-air connectivity push future vehicle innovations, which will lead to a further increase in the number of connected vehicles in the upcoming years. Many wireless connections between vehicles to the infrastructure or mobile devices arise. In order to protect this communication against security attacks, various protection mechanisms have to be integrated into the vehicle to ensure information security. One of these measures is the implementation of a distributed access control to protect different communication channels against unauthorized access attempts. This requires a well-defined assignment of access permissions for each communication node combined with certain environmental conditions such as location, time or vehicle state. However, current automotive systems do not have extensive access controls. Only for the execution of safety-critical diagnostic services an extended authorization level is available. In this publication, we present an attribute-based access control, which is specifically designed for an automotive E/E architecture with several domain controllers. Furthermore, we evaluate our approach with a proof-of-concept for an exemplary diagnostic service request.

Keywords: Cooperative ITS, Automated Vehicles, Cooperative Systems (V2X)
Abstract: This research presents a novel framework for automated fault detection in cyber-physical systems, with specific focus on large-scale vehicle networks. Agents in a network develop system identification models of themselves which are sent to a local or global authority. The authority excites the system models and generates a fixed-size vector for each one using an echo state network coupled with an autoencoder. The resultant vectors are grouped using standard clustering algorithms, with each group representing similar system model responses. A human expert labels each group once, so that any new group members can be can be associated with the group label. The largest group is assumed to be operating nominally, with all other groups representing a fault or off-nominal operation. We apply our framework to a detailed vehicle cooling system model to demonstrate its efficacy.

Keywords: Cooperative ITS, Autonomous / Intelligent Robotic Vehicles, Collision Avoidance
Abstract: In this paper, a multi-agent probabilistic optimization algorithm is applied to the problem of multi-vehicle coordination. The algorithm is known as ``Probability Collectives'' (PC) and has roots in Game Theory and Optimization theory. It is traditionally used for finding optimal solutions of NP-hard problems such as the traveler salesman problem. On the other end, the version of the PC presented in this paper focuses on a minimal complexity implementation for solving the coordination problem in a very short time. Besides time constraints, the emphasis in the design is put on ensuring that the algorithm always comes up with a feasible solution. Simulations show that both objectives are reached while having a decentralized and polyvalent algorithm. Additional benefits of the PC algorithm include robustness to agent failure and the possibility to accommodate non-collaborative vehicles (market penetration of autonomous vehicles <100%). These characteristics will be investigated later.

Keywords: Cooperative ITS, V2X Communication, Cooperative Systems (V2X)
Abstract: Interaction among vehicles based on wireless communication technologies is an efficient way to improve traffic conditions and safety. In this context, The Grand Cooperative Driving Challenge (GCDC) is an event that aims at developing cooperative systems for intelligent vehicles. Although the two editions of the event (2011 and 2016) have provided conditions for a series of experiments such as platoon stability and lane-merging, the traffic flow in lane-merging scenarios didn't receive so much attention. This paper presents a preliminary study on traffic flow using GCDC 2016 interaction protocol for lane-merging by varying the driver imperfection model and the number of vehicles. We also propose an extension of the standard protocol. We evaluated three lane-merging interaction protocols using the Simulation of Urban Mobility (SUMO). The first protocol is very similar to the GCDC 2016, the second is the default lane change model commonly used in SUMO and the third is a variation of the GCDC interaction protocol. The experiments were conducted using similar conditions and number of vehicles of the event. We analyzed the lane-merging total time for all vehicles to complete the maneuver, the maximum string platoon length and the average platoon speed. Therefore, we could observe that the interaction protocol extension proposed decreased the duration time to complete the lane-merging maneuver of all vehicles without drastically compromising maximum string length and average platoon speed. The flexibility of this extension is closer to real traffic scenario, in which vehicles can perform the merge without having a pre-defined order.

Keywords: Cooperative Systems (V2X), Information Fusion, Situation Analysis and Planning
Abstract: Use of cooperative information, distributed by road-side units, offers large potential for intelligent vehicles (IVs). As vehicle automation progresses and cooperative perception is used to fill the blind spots of onboard sensors, the question of reliability of the data becomes increasingly important in safety considerations (SOTIF, Safety of the Intended Functionality).

This paper addresses the problem to estimate the reliability of cooperative information for in-vehicle use. We propose a novel method to infer the reliability of received data based on the theory of Subjective Logic (SL). Using SL, we fuse multiple information sources, which individually only provide mild cues of the reliability, into a holistic estimate, which is statistically sound through an end-to-end modeling within the theory of SL.

Using the proposed scheme for probabilistic SL-based fusion, IVs are able to separate faulty from correct data samples with a large margin of safety. Real world experiments show the applicability and effectiveness of our approach.

Keywords: Cooperative Systems (V2X), Vehicle Environment Perception
Abstract: The growing penetration of on-board communication units is enabling intelligent vehicles to share their sensor data with cloud computing platforms as well as with other vehicles. Although this unlocks the possibility of a variety of emerging applications, the massive amount of data traffic in vehicular networks is expected to pose a big challenge in the long term. In this paper, we shed light on the potential of value-anticipating networking to tackle this issue. A vehicle sending a piece of information first anticipates the value of that information for potential receivers. When the network is congested, the sender may defer or even cancel transmissions of less valuable information, so that important information can be delivered to receivers more reliably. We investigate the applicability of this concept to cooperative perception, where vehicles exchange processed sensor data over vehicle-to-vehicle (V2V) networks to collaboratively improve coverage and accuracy of environmental perception. Through simulations based on realistic road traffic, we show that value-anticipating V2V communications can significantly improve the performance of cooperative perception under heavy network load.

Keywords: Situation Analysis and Planning, Self-Driving Vehicles, Cooperative Systems (V2X)
Abstract: Creating rational driving options and designing the decision process to select the best solution in a traffic situation with multiple participants present is a challenging problem. Other participants could be cooperating communication-enabled autonomous vehicles or vehicles controlled by human drivers with egoistic goals. This work introduces a novel approach to coordinate the behavior of multiple vehicles in generic traffic scenes. Our three-step method generates motion options neglecting vehicle interactions at first. Afterward, a mixed-integer linear optimization problem is solved to find the optimally coordinated motion patterns, followed by an online re-calibration based on the observed behaviors in reality. We demonstrate and evaluate the applicability in an evasive maneuver requiring vehicle interaction in detail and also present an intersection scenario. We further show that cooperative behavior, as well as egoistic driver intentions, can be handled safely and analyze the properties of the proposed solution.

Keywords: Cooperative Systems (V2X), Privacy, Cooperative ITS
Abstract: Connected driving is a hot topic in the automotive industry and a leverage to push new Mobility as a Service (MaaS) methodologies, making vehicles an essential part of the Internet of Things (IoT). However, these new technologies often lead to security risks and privacy concerns, especially due to the increasing number of datasets exchanged between vehicles, drivers, and local infrastructure. Furthermore, the possibilities for vehicles to access heterogeneous services offered by different service providers are often limited due to rigid system boundaries. In this paper we present a novel federated service management concept for increased interoperability across distinct services in the field of Smart Mobility and Smart Cities. Our approach provides secure authentication and authorization between cars, their drivers, and other information systems, while retraining the level of privacy according to the users' preferences. The scalability and dynamic configurability of the solution and the elaborated proof-of-concept will set it apart from application-centered gateways to an embedded generic platform by virtue of its modular software design.

Keywords: Cooperative Systems (V2X), V2X Communication, Vehicle Control
Abstract: This paper is concerned with coordinated formation assignment and control of multiple connected vehicles in highway scenario. A dynamical interlaced layered formation generation method is introduced to provide safe distance among vehicles and efficiency for coordinated lane changing and formation switching simultaneously in real time. To assign vehicles to the generated formation, the optimal problem is modeled by introducing the function of numbers of changed lanes for all the vehicles with respect to the kinematic constraints, and on-board local controllers for the vehicles are developed. Simulation result indicates that, comparing to existing methods for lane assignment in multiple traffic scenarios, the method provided by this paper could increase the traffic efficiency by utilizing maximum road capacity while decreasing travel time for all the vehicles.

Keywords: Cooperative Systems (V2X), V2X Communication, Cooperative ITS
Abstract: Cooperative driving has caught the interest of many research centers around the world as it introduced a solution to many problems in traffic and especially the reduction of accidents. One of the cooperative driving applications is vehicle platooning, which has proven its ability to increase road capacity and decrease fuel consumption. As the platoon has to have a rules of organization to form and split safely, this paper introduces a dynamic platoon model and protocols with different control techniques applied on different sub-systems in the model. Additionally, the basic platooning maneuvers are governed by applying certain protocols to organize the formation of platoon in a safe manner. Several tests on the platooning model and maneuvering controls have been performed from real-life scenarios and using real-vehicle data and a software in the loop to verify the results and evaluate the performance of the proposed approaches. The results prove an acceptable performance of the control techniques applied on the platooning model.

Keywords: Cooperative Systems (V2X), Intelligent Vehicle Software Infrastructure, Cooperative ITS
Abstract: The response time of emergency vehicles (EVs) determines the outcome of many emergencies, thus improving the traversal time of EVs is of paramount importance. Vehicular communication is a key enabler of such an improvement. This paper studies two EV traversal algorithms, focusing mainly on their communication aspects. Fast moving dense traffic is modeled, and the traversal algorithms are implemented on top of the dedicated short-range communication (DSRC) / wireless access in vehicular environments (WAVE) protocol stack, while accounting for channel impairments such as path loss and fading. Algorithms that highlight the required packet transfers for the traversal and for safe lane changes are presented, and simulated in the VEINS framework for different traffic conditions. Simulation results show that the suitability of the EV traversal algorithms defer depending on the speed distribution of the vehicles. Further insights drawn from the simulation are utilized to fine tune the EV traversal algorithms and to decrease the traversal time further.

Keywords: Vehicle Control, Self-Driving Vehicles, Intelligent Ground, Air and Space Vehicles
Abstract: This paper proposes a novel control strategy integrated path-following with yaw motion control for autonomous distributed drive electric vehicles with differential steering (DS) technology. First, the path-following and vehicle dynamics model, and DS system are introduced and analyzed. Then, the control framework is proposed, where the model predictive control (MPC) is adopted for path-following and yaw motion control. Given the optimized command by MPC, the quadratic programming (QP) algorithm is applied for in-wheel motors’ torque allocation optimization. Series of simulation validations are carried out, proving that the proposed strategy can effectively achieve superior path-following effect, guarantee the vehicle yaw stability, and implement the steering control in DS system, simultaneously.

Keywords: Vehicle Control, Advanced Driver Assistance Systems
Abstract: In this paper, we focused on the yaw motion dynamics for light electric vehicles. The objective is to track the yaw rate reference in order to enhance lateral stability. Two controllers were designed, a simple PID controller and a third generation CRONE controller. The design methodology considers the total mass of the vehicle, the longitudinal speed, and the road adhesion as uncertain parameters. Later, we analyze the effect of vertical dynamics, on a controlled full vehicle system, where the variation of the normal wheel-contact forces was taken into consideration. A double lane change maneuver is simulated, and the results show the robustness of the CRONE controller against the uncertain parameters and its effectiveness to lessens the influence of load transfer variations.

Keywords: Vehicle Control, Self-Driving Vehicles, Automated Vehicles
Abstract: This paper presents the application of Graphics Processing Unit (GPU) to improve the control performance of sampling based predictive control algorithms. As an example problem, obstacle avoidance situation with parked cars in a street is modeled as a non-linear model predictive control problem. The car dynamics and non-linear constraints are considered to achieve collision avoidance. The control input must be optimized in every control step in real-time considering the non-linear constraints. Sampling based approach is used to solve this problem and one of the major limitations to this approach is the computational cost involved. In this paper, the sampling-based optimization algorithm was adapted to utilize the parallel compute capabilities of GPU using CUDA. The generated input sequence and the computational speeds were compared with a CPU based program for the same case. Proposed method is implemented in simulation experiment with car dynamics simulator to verify its performance in terms of path tracking. Finally, a general relationship between sample size and GPU acceleration of its calculation speed is also discussed.

Keywords: Vehicle Control, Cooperative Systems (V2X), V2X Communication
Abstract: The coordinated on-ramp merging is one of the most common but critical vehicular applications that require complex data transmission and low-latency communication in the Connected and Automated Vehicles (CAVs) environment. An effective way to address on-ramp merging is to leverage the edge computing to optimize the coordination among vehicles to achieve overall minimum vehicle travel time and energy consumption. In this study, we propose an Bi-level Optimal Edge Computing (BOEC) model for on-ramp merging in the CAVs environment to optimize both merge time and vehicle trajectory. The simulation results show that the proposed BOEC model achieves great benefits in vehicle mobility, energy saving and air pollutant emission reduction by providing an energy-efficient trajectory following the optimal merge time without compromising safety.

Keywords: Autonomous / Intelligent Robotic Vehicles, Vehicle Control, Deep Learning
Abstract: A reliable controller is critical for execution of safe and smooth maneuvers of an autonomous vehicle. The controller must be robust to external disturbances, such as road surface, weather, wind conditions, and so on. It also needs to deal with internal variations of vehicle sub-systems, including powertrain inefficiency, measurement errors, time delay, etc. These factors introduce issues in controller performance. In this paper, a feed-forward compensator is designed via a data-driven method to model and optimize the controller’s performance. Principal Component Analysis (PCA) is applied for extracting influential features, after which a Time Delay Neural Network is adopted to predict control errors over a future time horizon. Based on the predicted error, a feedforward compensator is then designed to improve control performance. Simulation results in different scenarios show that, with the help of with the proposed feedforward compensator, the maximum path tracking error and the steering wheel angle oscillation are improved by 44.4% and 26.7%, respectively.

Keywords: Vehicle Control, Active and Passive Vehicle Safety, Advanced Driver Assistance Systems
Abstract: On vehicles there exists a close coupling between brake and suspension dynamics, making semi-active damper control a promising way for brake maneuver optimization, which has been widely researched in the automotive and motorcycle field. Experimental data of bicycle dynamics analyzed in this paper show substantial differences to classical vehicle dynamics. By first-principle control-oriented modeling and full vehicle nonlinear multibody simulation it is shown that this can be traced back to two phenomena: the dynamic rider response and fork bending. These are very general effects, but crucial for vehicle dynamics control on bicycles, one of the most widely used means of transportation.

Keywords: Vehicle Control, Reinforcement Learning, Self-Driving Vehicles
Abstract: We present a control approach for autonomous vehicles based on deep reinforcement learning. A neural network agent is trained to map its estimated state to acceleration and steering commands given the objective of reaching a specific target state while considering detected obstacles. Learning is performed using state-of-the-art proximal policy optimization in combination with a simulated environment. Training from scratch takes five to nine hours. The resulting agent is evaluated within simulation and subsequently applied to control a full-size research vehicle. For this, the autonomous exploration of a parking lot is considered, including turning maneuvers and obstacle avoidance. Altogether, this work is among the first examples to successfully apply deep reinforcement learning to a real vehicle.

Keywords: Vehicle Control, Active and Passive Vehicle Safety, Advanced Driver Assistance Systems
Abstract: Suspension systems on bicycles have a tendency to severe brake-induced dive-in, caused by the small wheelbase in combination with a high center of gravity. Semi-active dampers allow the implementation of anti-dive functionality, preventing this behavior. Experimental analysis has shown that this yields significant advantages during brake control on level surfaces. In the presence of additional road excitation, however, a strong conflict arises. A specific test case is a bump occurring while braking, when the damping is set to the hardest value in order to mitigate dive-in. A simulative analysis illustrates that especially the dynamic wheel load is affected, which during braking is safety critical. By simulation and experimental implementation it is shown that using a simple semi-active control rule a decent trade-off can be found. Finally, the influence of the actuator response time is evaluated.

Keywords: Intelligent Vehicle Software Infrastructure, Vehicle Control
Abstract: A fast algorithm for checking whether an autonomous vehicle can arrive at a position at a given arrival time and velocity is the key to Autonomous Intersection Management (AIM). This paper presents a complete set of closed form equations that fully describes the set of all reachable arrival configurations in longitudinal motion planning if the vehicle's controller is a double integrator with bounded acceleration. This result improves the running time of the algorithm for checking the reachability of an arrival configuration from logarithmic time to constant time. We also apply the result to check the reachability in a segmented road and discuss how the algorithm can be applied to real vehicles.

Keywords: Vehicle Control
Abstract: The number of gears in a transmission system has increased significantly in the last decades, in order to always keep the engine in the optimal operating point, whether the vehicle is accelerating, decelerating or traveling with constant velocity. Several transmission control algorithms have been developed for road vehicles equipped with Automated Manual Transmission (AMT) or Automatic Transmission (AT) in order to find out and serve the drivers will and choose gear according to the actual driving status. The aim of this work is to investigate different gear shifting strategies, and to compare them in various aspects in simulation environment and tests including a real vehicle. When selecting the appropriate gear in the transmission, a number of parameters can be taken into consideration, e.g. actual engine speed, actual engine torque, fuel consumption, acceleration pedal position and the speed of the position change (position derivative), etc. To consider all of these parameters four gear selection algorithms have been implemented and tested.

Keywords: Vehicle Control, Automated Vehicles, Advanced Driver Assistance Systems
Abstract: In the literature, researchers studying the lateral dynamics of tractor-trailer systems have each developed their own reduced-order lateral dynamic model, each with their own assumptions, state representations, and derivation methods. Little to no work has been performed to compare the accuracy of these models to each other, nor to validate their results against high-fidelity multi-body simulations or real truck data. The purpose of this paper is to identify several reduced-order lateral dynamic models for tractor-trailer systems present in the current literature, simulate their estimated states through common driving maneuvers, and then compare their estimates against reference data from a high-fidelity multi-body dynamic model. From this comparison, we identify the reduced-order model that maintains the least error from the reference data as the best representation of a real tractor-trailer system. The results of our comparison will be useful to researchers interested in using a reduced-order dynamic model from the literature as the basis for articulation angle estimation, model-predictive control, or adaptive control algorithms for autonomous tractor-semitrailer systems.

Keywords: Automated Vehicles, Vehicle Control
Abstract: Motion planning&control systems (i.e., motion algorithmization systems) used in intelligent/autonomous vehicles usually require mathematical models to predict a vehicle behaviour. The models should be flexible in use, reliable, and sufficiently compact to be tractable in the analysis and by the on-board (embedded) computational devices. A modular approach to modelling presented in this paper allows building compact nonholonomic kinematic models of multi-articulated buses comprising a car-like tractor and the arbitrary number of segments (wagons) interconnected with passive rotary joints, and with fixed or steerable wheels of the wagons. The modelling methodology is flexible in use by admitting various locations of a traction drive in a kinematic chain, and letting for various selections of a guiding point for a vehicle. The models are valid under assumption of pure rolling of all the vehicle wheels (no skid/slip motion) which is reasonable for low-speed maneuvering conditions. Derivations of kinematic models for two popular structures of urban buses (i.e., articulated and bi-articulated pushers) are provided.

Keywords: Eco-driving and Energy-efficient Vehicles, Cooperative Systems (V2X)
Abstract: The eco-approach and departure (EAD) application for signalized intersections has been proved to be environmentally efficient in a Connected and Automated Vehicles (CAVs) system. The traffic and signal phase and timing (SPaT) information transmitted from the roadside equipment unit, vehicle equipped sensors (e.g. radars) and other connected vehicles are the main inputs to the existing algorithms. However, due to the limitation of the communication and sensing range, it is too late to start eco-driving until preceding traffic is fully detected. Instead, the historical data, such as queue length distribution may be applied to developing a robust speed profile that enables eco-driving to start in an early stage. In this paper, a two-phase iterative approach is developed with the use of historical queue distribution. A graph-based model is created with nodes representing states of the host vehicle and traffic condition, and directed edges with weight representing expected energy consumption between two connected states. The shortest path is calculated that minimizes the total energy consumption for vehicles approaching a pre-timed signalized intersection. Numerical simulations have shown that the proposed method is robust and adaptive to vary traffic and queue conditions, and could achieve around 9% energy savings compared to other baseline methods.

Keywords: Eco-driving and Energy-efficient Vehicles, Impact on Traffic Flows, Traffic Flow and Management
Abstract: Traffic oscillations on freeways are one of the most important causes of low vehicle energy efficiency, and they result in a large amount of vehicular emissions and high risk of incidents. Most existing studies of freeway eco-driving either focused on one-lane roads or relies on high market penetration rates of connected vehicles, which are not realistic for the early-stage development of connected vehicles. This paper develops an advanced vehicle control system with connected vehicles to mitigate freeway traffic oscillations. This system is able to work for smoothing traffic oscillations on multi-lane freeways with a small number of connected vehicles. It applies the connected vehicles to capture freeway traffic oscillations based on their trajectory information, and a variable speed limit control system is implemented on the connected vehicles to adjust over-passing volumes so as to mitigate downstream traffic oscillations. A mathematical model is proposed to analyze the effect of the system on mitigating traffic oscillations under freeways with multiple lanes. In addition, the system is implemented in both macroscopic and microscopic simulations to understand its benefits on smoothing traffic oscillations along freeway segments and increasing energy efficiency.

Keywords: Eco-driving and Energy-efficient Vehicles, V2X Communication, Electric and Hybrid Technologies
Abstract: Activities of transit buses traveling along arterial roads and city streets consist of frequent stops and idling events at many predictable occasions, e.g., loading/unloading passengers at bus stops, approaching traffic signals or stop signs, and going through recurrent traffic congestion, etc. Besides designing transit buses with electric powertrain systems that can save a noticeable amount of energy thanks to regenerative breaking, this urban traffic environment also unfolds a number of opportunities to further improve their energy efficiency via vehicle connectivity and autonomy. Therefore, this paper proposes a complete and novel simulation framework of integrated vehicle/powertrain eco-operation system for electric buses (Eco-bus) by co-optimizing the vehicle dynamics and powertrain (VD&PT) controls. A comprehensive evaluation of the proposed system on mobility benefits and energy savings has been conducted over various traffic conditions. Simulation results are presented to showcase the superiority of the proposed simulation framework of the Eco-bus compared to the conventional bus, particularly in terms of mobility and energy efficiency aspects.

Keywords: Electric and Hybrid Technologies, Eco-driving and Energy-efficient Vehicles, Sensor and Data Fusion
Abstract: Deviations between system current measurements and reality can cause severe problems in the power train of electric vehicles (EVs). Among others, these are inaccurate performance coordination and unnecessary power limitations during driving or charging. In this work, we propose a fleet-based framework to detect such deviations. Our main assumption is that the real value is the mean of all identically constructed EVs’ measurements for the same input. Under this assumption, we train individual on-board models to predict the current of the electric machine (EM) and transmit the model parameters to a back-end. There, we compare individual deviations of the predicted current against the fleet in the same scenario. We use the results to classify three fault sources. As models we choose two different Machine Learning algorithms: State Models and Long Short-Term Memory Neural Networks (LSTMs). These are evaluated on an artificial fleet of 34 EVs derived from real drive data containing three different kinds of faults. Results show that our proposed approach correctly classifies major measurement faults. Additionally, both models offer similar classifications. LSTMs are more accurate, whereas state models are less computationally complex, and thus better suited for electronic control units (ECUs).

Keywords: Assistive Mobility Systems, Electric and Hybrid Technologies, Eco-driving and Energy-efficient Vehicles
Abstract: The ongoing electrification of powertrains requires innovative solutions that allow a broad application of battery electric vehicles (BEVs) with respect to different driving tasks. Especially long distance journeys for fully electric vehicles are a major obstacle due to range anxiety, the need to recharge and the lack of precise information on the driving and charging scenarios needed.

An optimal strategy that consists of velocity as well as charging suggestions enables a seamless use of electric vehicles on long distance journeys. Through a dynamic programming (DP) approach, a global time optimality of driving and charging tasks for two use cases is derived and presented. The existing control levers of vehicle speed and charger choice as well as the amount of charged energy are varied in their discretization. This is done under the aspects of overall travel time and a final state constraint. With regard to its computing time, the parameters' discretization is discussed. The applicability of the problem specific method is shown, optimal strategies are calculated. Also, it can be shown that the course of the state variable, i.e. the vehicles state of charge (SOC) dominates sensitivities in time and state deviation.

Keywords: Eco-driving and Energy-efficient Vehicles, Vehicle Control, Situation Analysis and Planning
Abstract: This paper presents an energy-efficient velocity control technique for a truck driving behind a slow moving truck. An observer is designed to estimate the acceleration capability of the leading vehicle, which is then used to predict the velocity and time trajectory of the leading vehicle. This information, together with information of road topography, is used by the ego vehicle to optimally plan its velocity alonga look-ahead horizon. The optimal planning is achieved by nonlinear model predictive control, where constraints are set to keep a safe distance to the leading vehicle and arrive at the destination within a given time. The designed controller is tested on various driving cycles. The proposed technique is also tested on a traffic light scenario, where information about the position of the traffic light and timing of its signals is considered to be known. The simulations results show that the proposed technique can save a significant amount of fuel. Keywords: Predictive Cruise Control, Leading vehicle, ModelPredictive Control, Observer, Nonlinear optimization

Keywords: Eco-driving and Energy-efficient Vehicles, Intelligent Vehicle Software Infrastructure, Advanced Driver Assistance Systems
Abstract: Prediction of ego-vehicle speed for powertrain control has drawn attention to a means to improve the energy efficiency of vehicles. In particular, the understanding of driving situations using Intelligent Transport System (ITS) information can help improve the prediction of accuracy. In this study, a velocity prediction algorithm based on Markov chain is proposed. However, when various ITS information is added, the size of the prediction algorithm rapidly increases, which increases a computational time and is difficult to implement to real-time systems. To solve the problem of the increased computational power, this paper proposes a velocity prediction algorithm based on a fuzzy Markov chain. This algorithm, designed for a vehicle driving on a specified route, combines a fuzzy Markov chain and a speed constraint model. The fuzzy Markov chain stochastically predicts an ego-vehicle’s speed within a constraint area and solves the problem of increasing algorithm size when adding various pieces of input data. The speed constraint model, which is estimated by empirical matrices, estimates the constraint area used in the fuzzy Markov chain. Through simulation, the algorithm is evaluated to reduce computational time by 85.5% whilst maintaining a prediction accuracy of 99.1%.

Keywords: Collision Avoidance, Automated Vehicles, Advanced Driver Assistance Systems
Abstract: Safety testing of advanced driver assistance systems (ADAS) and advanced driving functions (ADF) is a challenging task due to the impossibility of performing a sufficient road testing. In order to overcome this limitation, simulations are usually included in testing. In a previous work, the safety of and autonomous overtaking function -- proposed as a part of ADAS -- has been evaluated with respect to collision rate performance for a particular scenario. Notwithstanding the advantages of that approach, there are also limits, in particular when a reaction from other traffic participants can significantly alter the collision risk, e.g. when an overtaking autonomous vehicle is reached by another vehicle on the overtaking lane during the maneuver. According to when and how this vehicle will brake, the collision risk will strongly change independently from the ADAS reaction. Against this background, instead of modeling the human driver's response to a cut-in maneuver, we suggest using three key variables, namely Time-To-Collision, Time-Headway and inter-vehicle distance, which in some way capture the instantaneous behavior of the vehicle coming from the rear. These variables are then used as an alternative performance assessment metrics for the autonomous overtaking function.

Keywords: Active and Passive Vehicle Safety, Advanced Driver Assistance Systems
Abstract: Simulation studies are nowadays an invaluable tool for the design, as well as for the safety evaluation and verification of Advanced Driver Assistance Systems (ADAS) or Automated Driving Functions (ADF). In case ADAS/ADF are developed using simulation studies, they are (usually) designed to avoid accidents by means of suitable control actuations. However, the safety of those ADAS/ADF depends on the explicit and implicit assumptions made during the design process. If well designed, those assumptions will cover the vast majority of cases that might occur during real world driving. It is hardly possible though to account for all types of thinkable scenarios and accounting for highly improbable cases already during the design process might lead to tremendous additional costs in terms of performance. It is therefore proposed here to use a three layer safety philosophy for ADAS/ADF: The functionalities are designed based on realistic assumptions regarding traffic situations and scenarios. Subsequently, based on a catalog of test scenarios the crash boundary for the newly developed functionality will be identified, i.e. the test case parametrization will be searched for in which it is still possible to avoid a crash. In a last step, those cases for which a crash cannot be avoided will be analyzed with respect to the expected consequences for the people involved. This third step will be referred to as "beyond-design-basis safety assessment" (BDBSA) throughout this paper. A new, easy to apply methodology is proposed here for evaluating the consequences of such unavoidable crash scenarios in BDBSA.

Keywords: Automated Vehicles, Situation Analysis and Planning, Vehicle Control
Abstract: As basic modules of intelligent vehicle, path planning and its tracking have been developed rapidly. However, the trajectory generated by the traditional path-speed decoupled planning method is not feasible in the time domain. In this paper, a method based on improved RRT is proposed to achieve efficient planning and smoothing. In order to realize the matching of track points in space and time domain, closed-loop prediction is adopted to know the actually tracked path more accurately. Taking the nonlinear characteristics of lateral and longitudinal dynamics of the vehicle and the saturation of actuators into account, a unified conditional integral control law is designed to guarantee the global asymptotic stability of the tracking error and to avoid the degradation of actuators performance due to the divergence of the integral operation. Simulations and experiments prove that the proposed planning method is more efficient, the control algorithm can effectively track the planned trajectory, and the prediction method can accurately predict the actual driving trajectory, which is very important for collision detection.

Keywords: Situation Analysis and Planning, Automated Vehicles, Active and Passive Vehicle Safety
Abstract: Modern validation approaches of advanced automotive safety systems involve simulations of human driving behavior in safety-critical traffic events. Critical situations are often painstakingly enumerated and modeled, and it is difficult to establish confidence that the space of critical traffic events is adequately covered. This work presents an automated method for identifying and clustering critical situations that capture severity and frequency of occurrence, thereby allowing for risk-based safety validation. We demonstrate the ability of the new approach to accelerate the safety validation of an automotive safety system using importance sampling and efficiently optimize its parameters.

Keywords: Situation Analysis and Planning, Legal Impacts, Self-Driving Vehicles
Abstract: To realize a market entry of autonomous vehicles in the foreseeable future, the behavior planning system will need to abide by the same rules that humans follow.

Product liability cannot be enforced without a proper solution to the approval trap.

In this paper, we define a semantic abstraction of the continuous space and formalize traffic rules in linear temporal logic (LTL). Sequences in the semantic state space represent maneuvers a high-level planner could choose to execute. We check these maneuvers against the formalized traffic rules using runtime verification.

By using the standard model checker NuSMV, we demonstrate the effectiveness of our approach and provide runtime properties for the maneuver verification.

We show that high-level behavior can be verified in a semantic state space to fulfill a set of formalized rules, which could serve as a step towards safety of the intended functionality.

Keywords: Situation Analysis and Planning, Reinforcement Learning, Self-Driving Vehicles
Abstract: For highly automated driving above SAE level 3, behavior generation algorithms must reliably consider the inherent uncertainties of the traffic environment, e.g. arising from the variety of human driving styles. Such uncertainties can generate ambiguous decisions, requiring the algorithm to appropriately balance low-probability hazardous events, e.g. collisions, and high-probability beneficial events, e.g. quickly crossing the intersection. State-of-the-art behavior generation algorithms lack a distributional treatment of decision outcome. This impedes a proper risk evaluation in ambiguous situations, often encouraging either unsafe or conservative behavior. Thus, we propose a two-step approach for risk-sensitive behavior generation combining offline distribution learning with online risk assessment. Specifically, we first learn an optimal policy in an uncertain environment with Deep Distributional Reinforcement Learning. During execution, the optimal risk-sensitive action is selected by applying established risk criteria, such as the Conditional Value at Risk, to the learned state-action return distributions. In intersection crossing scenarios, we evaluate different risk criteria and demonstrate that our approach increases safety, while maintaining an active driving style. Our approach shall encourage further studies about the benefits of risk-sensitive approaches for self-driving vehicles.

Keywords: Situation Analysis and Planning, Deep Learning, Vehicle Environment Perception
Abstract: Highly automated driving systems are required to make robust decisions in many complex driving environments, such as urban intersections with high traffic. In order to make as informed and safe decisions as possible, it is necessary for the system to be able to predict the future maneuvers and positions of other traffic agents, as well as to provide information about the uncertainty in the prediction to the decision making module. While Bayesian approaches are a natural way of modeling uncertainty, recently deep learning-based methods have emerged to address this need as well. However, balancing the computational and system complexity, while also taking into account agent interactions and uncertainties, remains a difficult task. The work presented in this paper proposes a method of producing predictions of other traffic agents' trajectories in intersections with a singular Deep Learning module, while incorporating uncertainty and the interactions between traffic participants. The accuracy of the generated predictions is tested on a simulated intersection with a high level of interaction between agents, and different methods of incorporating uncertainty are compared. Preliminary results show that the CVAE-based method produces qualitatively and quantitatively better measurements of uncertainty and manage to more accurately assign probability to the future occupied space of traffic agents.

Keywords: Collision Avoidance, Situation Analysis and Planning, Self-Driving Vehicles
Abstract: Due to the current developments towards autonomous driving and vehicle active safety, there is an increasing necessity for algorithms that are able to perform complex criticality predictions in real-time. Being able to process multi-object traffic scenarios aids the implementation of a variety of automotive applications such as driver assistance systems for collision prevention and mitigation as well as fall-back systems for autonomous vehicles.

We present a fully model-based algorithm with a parallelizable architecture. The proposed algorithm can evaluate the criticality of complex, multi-modal (vehicles and pedestrians) traffic scenarios by simulating millions of trajectory combinations and detecting collisions between objects. The algorithm is able to estimate upcoming criticality at very early stages, demonstrating its potential for vehicle safety-systems and autonomous driving applications. An implementation on an embedded system in a test vehicle proves in a prototypical manner the compatibility of the algorithm with the hardware possibilities of modern cars. For a complex traffic scenario with 11 dynamic objects, more than 86 million pose combinations are evaluated in 21 ms on the GPU of a Drive PX~2.

Keywords: Situation Analysis and Planning, Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles
Abstract: Behavior planning in urban environments must consider the various existing uncertainties in an explicit way. This work proposes a behavior planner, based on a POMDP formulation, that explicitly considers possibly occluded vehicles. The future field of view of the autonomous car is predicted over the whole planning horizon. Both, occlusions which are generated by static as well as generated by dynamic objects are hereby considered. We use Monte Carlo sampling to generate possible future episodes that are used to derive an optimized policy. The sampled episodes consider the uncertain behavior of the known traffic participants as well as the existence probability of so-called phantom vehicles in occluded areas. By representing all possible, occluded vehicle configurations by its reachable set instead of single particles, a very efficient representation is found. Therefore, we ensure to consider all possible configurations which may drive out of the occluded area in our optimized policy. We propose a generic formulation of the POMDP problem that can be applied to various scenarios for urban driving. Its performance is demonstrated by using simulation scenarios at intersections including multiple vehicles and occlusions caused by static and dynamic objects. It is shown, that the autonomous vehicle approaches occluded areas by far less conservative than a baseline strategy which considers only the current field of view (fov). This is because various, future scenarios are already considered in the policy. In fact, we show that our planner is able to drive nearly the same trajectories as an omniscient planner would.

Keywords: Self-Driving Vehicles, Situation Analysis and Planning, Collision Avoidance
Abstract: Local motion planning plays an important role in an autonomous driving system. And applying mature local motion planning methods to real traffic scenarios with regular constraints is one of the keys to the applications of autonomous vehicles. In this paper, we present a local motion planning method combined with High-definition (HD) maps. Through the HD map defined by OpenStreetMap, the local motion planner can obtain the prior knowledge of traffic scenarios and achieve path planning and optimization accordingly. In order to improve the safety and comfort of the obstacle avoiding process, we also propose an inertia-like path selection algorithm based on this planning method. We evaluated the proposed method on our designed autonomous driving experimental platform 'Pioneer' and participated in the 2018 Intelligent Vehicles Future Challenge. In the competition, the 'Pioneer' successfully completed all the races and won the championship without any manual intervention.

Keywords: Situation Analysis and Planning, Active and Passive Vehicle Safety, Collision Avoidance
Abstract: This paper presents a novel collision mitigation method for traffic scenarios in which accidents are inevitable. When a predicted accident is imminent, the suggested software agent changes the trajectory of the ego vehicle to optimize the predicted crash configuration. The target function results from the degree of crash severity, based on the deformation of the safety cage in the occupant area. We have performed numerous crash simulations with detailed FEM models to determine the crash severity for possible crash configurations. Since inevitable crash scenarios are inherently time-critical, the focus is on fast optimization. The proposed heuristic search algorithm computes a collision trajectory from a given initial state by searching for the maneuver leading to the lowest crash severity in the reachable crash configuration set. We rely on a previously developed approach to trajectory modeling that guarantees a minimum execution time for trajectory generation and collision detection and respects the limits of vehicle dynamics. Simulations prove the applicability of the proposed algorithm.

Keywords: Advanced Driver Assistance Systems, Collision Avoidance, Vision Sensing and Perception
Abstract: Due to the lack of observation of rear moving objects by the passenger or driver of a vehicle, accidents happen frequently when they open the door of the vehicle. In this paper, we propose a novel vehicle open door safety system based on cyclist detection using fisheye camera and Improved Deep Convolutional Generative Adversarial Nets (IDCGANs). First of all, a fisheye camera is automatically turned on and captures the rear information of the vehicle when the vehicle is stopping. After that, a simple and effective method based on longitude coordinate is used to correct the distorted images. Second, Improved Deep Convolutional Generative Adversarial Nets is used to generate sample for data training. As the limited training datasets of cyclist and lots of annotation information which takes much time and efforts, we use Improved Deep Convolutional Generative Adversarial Nets to generate synthesized images of cyclist and use them as the training data for cyclist detectors. At last, Faster R-CNN detector is employed to train and detect the cyclist. The system was tested on realistic experiments and reached 87.2% precision rate and 95.3% recall rate. The feasibility of the proposed system for vehicle open door safety is demonstrated through simulation and test results.

Keywords: Advanced Driver Assistance Systems, Vehicle Control, Automated Vehicles
Abstract: Controlling the longitudinal movement of heavy-duty vehicles ased on optimal control can be a cost-efficient way of reducing their fuel consumption. Such controllers today mainly exist for vehicles in haulage applications, in which the velocity is allowed to deviate from a constant set-speed. For distribution vehicles, which is the focus of this paper, the desired and required velocity has large variations, which makes the situation more complex. This paper describes the implementation of an optimal controller in a real heavy-duty distribution vehicle. The optimal control problem is solved offline as a Mixed Integer Quadratic Program, which yields reference trajectories that are tracked online in the vehicle. Some important steps in the procedure of the implementation are, except for designing the controller: developing a positioning system for the test track where the experiments are performed, estimating the parameters of the resistive forces, and setting the velocity constraints. Simulations show a potential of 10% reduction in fuel consumption without increasing the trip time. Experiments are then performed in a Scania truck, with the optimal solution as reference for the existing cruise control functions in the vehicle. It is concluded that in order to verify the fuel savings experimentally, the low-level controllers in the vehicle must be modified such that the tracking error is decreased.

Keywords: Advanced Driver Assistance Systems, Collision Avoidance
Abstract: A method that can robustly estimate the distance to a foregoing vehicle with a monocular camera is in high demand for developing inexpensive Adaptive Cruise Control (ACC) systems. The accuracy of the estimated distances heavily depends on the road geometry and the true separation between the self-vehicle and the target vehicle. That motivates us to develop a distance estimation method that has two important properties: (a) it can deal with non-planar road surfaces from the first principle, and (b) it can estimate not only the distance but its variance of the error through mathematical analyses. Experiments on the KITTI dataset demonstrated that the proposed distance estimator is more robust and stable than the planar assumption model, and we verified by Monte Carlo simulations that our error model correctly predicts the variance of the error in the estimated distances.

Keywords: Advanced Driver Assistance Systems, Self-Driving Vehicles, Unsupervised Learning
Abstract: As development of Automated Driving Systems (ADS) advances, new methods for validation and verification (V&V) are needed. A promising approach for V&V is scenario- based testing. Recorded real-world-driving-data constitutes a useful source for the extraction of scenarios as they ensure a high level of realism. Since recorded data is typically unlabeled, the benefits drawn from the large amounts of available data are limited, e.g. the data interpretation with respect to the inherent driving scenarios is challenging. Manual data inspection or rule- based approaches are hardly scalable to neither big datasets nor numerous different scenario types. Hence, there is a need of automated data analysis tools, e.g. for labeling on a semantic level. Many current approaches try to accomplish that based on neural networks. This arises the need for a consistent, valid and machine-readable representation of a driving scenario. In this paper, a representation of a scenario is defined as a top-view grid, comprising the dynamic objects and the static environment, thereby allowing a consistent interpretation of all relevant aspects of a driving scenario. Furthermore, temporal scenario aspects have to be covered as well. Therefore, a neural network architecture for the extraction of both spatial and temporal features is described. Using the proposed feature extractor, the recorded driving data gets transformed to a reduced abstract feature space. With the autoencoder, a method for efficient training of the feature extractor is described. The combined approach enables the development of scalable and efficient methods for data analytics in large quantities of real driving data, e.g. automated labeling of big datasets or the scanning for rarely happening corner cases.

Keywords: Advanced Driver Assistance Systems, Automated Vehicles, Human-Machine Interface
Abstract: In order to benefit from the potential of automated driving, the systems have to be used appropriately by drivers. Usage is in turn dependent on trust, which is, among other things, influenced by system knowledge and experience. One way of conveying system knowledge is by presenting feedback of limitations and constraints. The present study investigates the influence of specific as opposed to non-specific feedback on the evolution of trust with an intelligent parking assistant (IPA). The research focused on the question how the trust development can be described in a level 2 automation system and how it differs between feedback specificity levels. Therefore, 22 participants took part in an on-road longitudinal study, in which they performed a total of forty parking maneuvers while using the IPA. During the first twenty parking maneuvers no specific feedback was presented in case of a fault due to system limitations, whereas during the second twenty parking maneuvers specific feedback corresponding to situational factors and system constraints was given. The results show that non-specific feedback does not lead to an increase of trust at all. The errors that occur due to system and sensor limitations remain inexplicable and thus unpredictable. On the other hand, specific feedback that corresponds to the situation leads to the formation and stabilization of trust in accordance with the power law of learning. Therefore, it is advisable that users ideally receive specific feedback in order to gain knowledge of a system’s limitations. This promotes the development of trust and ensures appropriate system use.

Keywords: Advanced Driver Assistance Systems, Recurrent Networks, Passive Safety
Abstract: The ability to robustly detect abnormal driving behavior has the potential to limit traffic accidents and save many lives. Abnormal driving behavior that threatens road safety includes aggressive, anxious, nervous, and unstable driving. Any of these can lead to dangerous situations in traffic. Therefore, we aim to provide a robust mechanism to detect such abnormal driving behavior. In this paper, we present our work in this regard which focuses on neural networks-based anomaly detection approaches. We consider autoencoder replicator neural networks and long short-term memory networks; comparing them to a previously employed Isolation Forest. We show that introducing a post-processing approach, that takes into account the recent history of a vehicle, reliable anomaly detection for driving behavior can be achieved based on the recurrent neural network. Its performance is well suited for application in a large scale detection system for driver assistance or autonomous vehicles.

Keywords: Advanced Driver Assistance Systems, Collision Avoidance, Recurrent Networks
Abstract: The success of Intelligent Driver Assistance (IDA) depends on the system's ability to accurately model the state of traffic surrounding the ego vehicle and predict driving behavior of the surrounding vehicles in order to help the ego driver make the best informed decisions in real-time. The ability to predict acceleration behavior is crucial as a first step towards modeling traffic patterns. In this paper, we show that Long Short-Term Memory (LSTM) neural networks are capable of producing acceleration distributions from which accurate future acceleration values can be sampled. Furthermore, state values calculated from these acceleration predictions are used as input for future predictions, showing that these networks are capable of generating realistic simulated vehicle trajectories over short prediction horizons.

Keywords: Advanced Driver Assistance Systems, Collision Avoidance
Abstract: The purpose of this paper is to analyze naturalistic driving data and crash data in the United States of America concerning the multiple risk-inducing factors which exist in real traffic. The derived method allows to identify neutral characteristics occurring in many situations and extract riskinducing attributes from real data by conducting the Successive Odds Ratio Analysis (SORA). The SORA algorithm uses two different types of data, e.g., baseline and crash data, calculates the criticality of each attribute, and evaluates combinations whereby the total criticality is affected positively or negatively. This paper focuses on the exemplary environment-related variables which are provided by the considered databases. Based on identified risk-inducing attributes, their associated characteristics will be investigated by using three measures, i.e., Support, Confidence, and Lift. The method has the potential to generate a scenario catalog consisting of critical test cases for the development of advanced driver assistance systems.

Keywords: Advanced Driver Assistance Systems, Deep Learning, Automated Vehicles
Abstract: When driving at night, a good illumination of the road ahead is crucial. With autonomous driving at close temporal proximity, this not only concerns human drivers but also autonomous systems capable of controlling the car. To achieve fully autonomous driving, a variety of sensors are integrated into the vehicles. Cameras act as one of the major sensors. However, due to their passivity, cameras cannot see well in the dark. To mitigate this shortcoming, modern cars are equipped with powerful headlights that provide proper illumination of the road ahead while avoiding the dazzling of other traffic participants. To use the headlights' full potential and to also provide advanced light functionality like glare-free high beam, they need to be properly adjusted. After the initial calibration during production, this setting is prone to undesirable degradation, primarily due to mechanical reasons. We present a completely new application of computer vision and machine learning to automatically detect wrongly adjusted headlights by estimating their pitch angle from the images of a vehicle-attached camera for advanced driving assistance systems (ADAS). We show that we can achieve high performance in terms of accuracy and robustness by training a deep neural network in an end-to-end fashion. To demonstrate the benefits of our proposed approach, an additional handcrafted baseline method is implemented.

Keywords: Advanced Driver Assistance Systems, Collision Avoidance, Human-Machine Interface
Abstract: A driver-vehicle-road (DVR) model based on kinematic vehicle model is proposed in this paper. In this DVR model, the kinematics vehicle-road model is adopted, and the driver model considering the human driver's characteristics is also included. Thus the behaviors of human driver's preview and neuromuscular delay can be considered in design of path planner and controller by using this DVR model. The repulsive force field based on the artificial potential field (APF) and the circle decomposition of vehicle shape are used to describe the constraints of obstacle avoidance and the road departure avoidance. Based on the proposed DVR model, a trajectory planer using model predictive control (MPC) is designed with consideration of collision and lane-departure avoidance, driver's intention, and vehicle occupant comfort. Simulation results show that with the proposed planner, the vehicle can successfully avoid static/moving obstacles and return to the original lane without lane departure. Simulation results indicate that the proposed kinematic vehicle model based DVR model can be used to design the path planner in normal driving and some typical driving scenarios. And the proposed path planner can provide the vehicle driven by different human drivers with individually safe trajectories in typical scenarios of obstacle avoidance.

Keywords: Automated Vehicles, Collision Avoidance, Active and Passive Vehicle Safety
Abstract: The need for safety assurances in Automated Driving (AD) is becoming increasingly critical with the accelerating deployment of this technology. Beyond functional safety, industry must guarantee the operational safety of automated vehicles. Towards that end, Mobileye introduced the Responsibility Sensitive Safety (RSS), a model-based approach to Safety. In this paper we expand upon this work introducing the C++ Library for Responsibility Sensitive Safety, an open source executable implementation of RSS. We provide architectural details to integrate the C++ Library for Responsibility Sensitive Safety with AD Software pipelines as safety module overseeing decision making of driving policies. We illustrate this application with an example integration with the Baidu Apollo AD stack and simulator, that provides safety validation of the planning module. Furthermore, we show how the C++ Library for Responsibility Sensitive Safety can be used to explore the usefulness of the RSS model through parameter exploration and analysis on minimum safe longitudinal distance, considering different weather conditions. We also compare these results with half-of-speed rule followed in some parts of the world. We expect that the C++ Library for Responsibility Sensitive Safety becomes a critical component of future tools for formal verification, testing and validation of AD safety and that it helps bootstrap the AD research efforts towards standardization of safety assurances.

Keywords: Active and Passive Vehicle Safety, Lidar Sensing and Perception, Vehicle Environment Perception
Abstract: The stated goal of the automotive industry is the development of autonomous driving. In addition to comfort, the improvement of vehicle safety is one of the main reasons. One of the new opportunities is that information from the environmental sensors can be used to detect a crash and trigger actuators of passive safety systems before the impact occurs. However, the collision partner as well as the crash parameters must be known in detail to avoid a false activation. While the crash velocity can be precisely determined by radar, LiDAR sensors in forward-looking sensor systems provide accurate information about the geometry of an object. Hence, this paper presents a methodology to estimate the contour of oncoming vehicles in the pre-crash phase using LiDAR. An optimized and fast algorithm derives the main vehicle vertices from the LiDAR point cloud. Afterwards, the relevant vehicle contour and the longitudinal axis is determined. The result is a significantly more detailed vehicle contour compared to conventional bounding boxes. Tests on real cars showed high accuracy and robustness of the estimation on static as well as on dynamic measurements. Next research steps will be included more difficult scenarios and higher velocities.

Keywords: Deep Learning, Automated Vehicles, Active and Passive Vehicle Safety
Abstract: Generative Adversarial Networks (GANs) have achieved outstanding results in generation of realistic data, particularly for image data. Autonomous driving systems are equipped with a large suite of sensors to obtain robustness and redundancy. Ultrasonic sensors are commonly used because of their low-cost and reliability of near-field distance estimation. However, machine learning algorithms are not commonly used for ultrasonic data, as it requires extensive datasets whose creation is time-consuming, expensive and inflexible to hardware and environmental changes. On the other hand, there exists no method to simulate these signals deterministically. Thus, we present a novel approach for synthetic ultrasonic signal simulation using conditional GANs (cGANs). To the best of our knowledge, we present the first realistic data augmentation for automotive ultrasonics sensors. The performance of cGANs allows us to bring the environment simulation to a high quality close to realistic data. By using our setup and environmental parameters as condition, the proposed approach is flexible to external influences. Due to its low complexity and smaller time effort needed for data generation, the proposed method outperforms other simulation algorithms such as finite element method. We verify the outstanding accuracy and realism of our method by applying a detailed statistical analysis and comparing the generated data to an extensive amount of measured signals.

Keywords: Active and Passive Vehicle Safety, Automated Vehicles, Collision Avoidance
Abstract: As vehicular safety functions that intervene in dangerous driving situations use sensor measurements for interpreting the driving situation, they are typically very vulnerable to sensor imperfections and measurement errors have a negative impact on both the safety and the satisfaction of the customer. Therefore, a new methodology for the robust design of an automatic emergency braking (AEB) system is proposed, which considers sensor measurement errors, selects the best decision rule used by the function of the AEB system for triggering an emergency brake intervention and covers several scenarios in which the designed AEB system is supposed to work. The robust function and sensor design for the AEB system is formulated as optimization problems based on a stochastic model. Numerical examples illustrating the elaborated theoretical results show how the new design methodology provides the designer with design spaces from which the optimal parameter values are chosen, with a ranking of the decision rules based on which the best decision rule is selected and with the worst cases from the set of considered scenarios. Moreover, the proposed design methodology generalizes and can be applied to design functions and sensors of other vehicular safety systems as well.

Keywords: Automated Vehicles, Self-Driving Vehicles, Situation Analysis and Planning
Abstract: The aim to integrate automated vehicles in manual traffic motivates the investigation of the communication of road users. Especially in situations of high traffic density, an automated lane change without cooperation of interacting traffic participants cannot be executed. Therefore, an automated vehicle needs a distinct interpretable strategy to inform interacting road users of its intention. In this presented driving study, different variations to announce a lane change to interacting traffic are performed on a test track and are evaluated (with 40 participants) in a within-subjects design. To gain standardized situations, all lane change maneuvers are executed automatically. During the study, different factors to announce a lane change, like the time to set the indicator, a weak or strong deceleration to the target gap or a lateral offset in advance of the lane change are investigated. Moreover, the influence of the lane change direction as well as the velocity of the target gap are analysed. The study illustrates that not only an indicator is crucial to announce a lane change. Also a strong deceleration to the target gap influences the processing of the information and cooperation of interacting traffic participants. Other factors, such as a lateral offset in advance of a lane change is evaluated as less important. In addition, the study implies higher cooperation and an influence of the lane change direction at slow velocities.

Keywords: Automated Vehicles, Situation Analysis and Planning, Collision Avoidance
Abstract: State of the art emergency brake assistant systems solely based on sensor measurements reduced the number of traffic accidents drastically in recent years. In order to be able to react on road users who elude the vehicle’s field of view because of sensor limits or occlusions, this paper presents an approach to anticipate potential hidden traffic participants in occluded areas in the decision making process of an autonomous vehicle. A Partially Observable Markov Decision Process is used to determine the vehicle’s longitudinal motion in a probabilistic planning approach. Observations are made using the vehicle’s field of view. Therefore the field of view is calculated with a generic sensor setup in dependence of the current and predicted state of the environment, making the vehicle anticipate future developments including hidden traffic participants. Hence, first regions of interest are determined to make assumptions about vehicles that may be located in hidden areas. An arbitrary number of hidden traffic participants can be included in the planning process. We demonstrate the approach in two scenarios, one where the vehicle has to move cautiously into the intersection with a minimum number of actions and in a typical scenario for urban traffic. Evaluation shows, that the approach is able to anticipate hidden road users correctly and act accordingly.

Keywords: Autonomous / Intelligent Robotic Vehicles, Self-Driving Vehicles, Vehicle Control
Abstract: In this paper, we propose a framework for generating motion primitives for lattice-based motion planners automatically. Given a family of systems, the user only needs to specify which principle types of motions, which are here denoted maneuvers, that are relevant for the considered system family. Based on the selected maneuver types and a selected system instance, the algorithm not only automatically optimizes the motions connecting pre-defined boundary conditions, but also simultaneously optimizes the end-point boundary conditions as well. This significantly reduces the time consuming part of manually specifying all boundary value problems that should be solved, and no exhaustive search to generate feasible motions is required. In addition to handling static a priori known system parameters, the framework also allows for fast automatic re-optimization of motion primitives if the system parameters change while the system is in use, e.g, if the load significantly changes or a trailer with a new geometry is picked up by an autonomous truck. We also show in several numerical examples that the framework can enhance the performance of the motion planner in terms of total cost for the produced solution.

Keywords: Self-Driving Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: The number of people living in urban environments is continuing to grow, which leads to a rising demand for intelligent transportation solutions. By flexibly supplementing public transport, driverless shuttles are widely regarded as a big building block for seamless mobility in the future, e.g. as first and last mile delivery systems. Therefore, we would like to introduce the Continental Urban Mobility Experience (CUbE), our research platform for driverless shuttles in urban environments. In contrast to the evolutionary automation of traditional cars, the CUbE helps to investigate new challenges and opportunities for fully autonomous driving in cities with a different kind of vehicle. Herein, we describe its system setup, early algorithmic approaches and share some of our real-world experiences, which we gained by extensive driving in an urban-like scenario.

Keywords: Autonomous / Intelligent Robotic Vehicles, Deep Learning, Sensor and Data Fusion
Abstract: Only a few existing works exploit multiple modalities of data for road detection task in the context of autonomous driving. In this work, a deep learning based approach is developed to fuse LIDAR point cloud and camera image features over a bird’s eye view representation. A two-stream fully-convolutional network is designed as encoder to extract general features of two types of data. Instead of limiting the fusion processing at a single stage or to a predefined extent, we propose a multi-stage residual fusion strategy to merge the feature maps in a residual learning fashion, and integrate the information at different network depth. Experiments conduct on KITTI road benchmark show that our proposed method has a significant improvement over single modality methods and other fusion approaches. And it is also among the top-performing algorithms.

Keywords: Self-Driving Vehicles, Sensor and Data Fusion, Vehicle Environment Perception
Abstract: For autonomous driving, knowledge about the current environment and especially the driveable lanes is of ut- most importance. Currently this information is often extracted from meticulously (hand-)crafted offline high-definition maps, restricting the operation of autonomous vehicles to few well- mapped areas and making it vulnerable to temporary or per- manent environment changes. This paper addresses the issues of map-based road models by building the road model solely from online sensor measurements. Based on Dempster-Shafer theory and a novel frame of discernment, sensor measurements, such as lane markings, semantic segmentation of drivable and non- drivable areas and the trajectories of other observed traffic participants are fused into semantic grids. Geometrical lane information is extracted from these grids via an iterative path- planning method. The proposed approach is evaluated on real measurement data from German highways and urban areas.

Keywords: Self-Driving Vehicles, Advanced Driver Assistance Systems, Vision Sensing and Perception
Abstract: Road-users are a critical part of decision-making for both self-driving cars and driver assistance systems. Some road-users, however, are more important for decisionmaking than others because of their respective intentions, ego-vehicle’s intention and their effects on each other. In this paper, we propose a novel architecture for road-user importance estimation which takes advantage of the local and global context of the scene. For local context, the model exploits the appearance of the road users (which captures orientation, intention, etc.) and their location relative to ego-vehicle. The global context in our model is defined based on the feature map of the last convolutional layer of the module which predicts the future path of the ego-vehicle and contains rich global information of the scene (e.g., infrastructure, road lanes, etc.), as well as the ego-vehicle’s intention information. Moreover, this paper introduces a new data set of real-world driving, concentrated around intersections and includes annotations of important road users. Systematic evaluations of our proposed method against several baselines show promising results.

Keywords: Self-Driving Vehicles, Collision Avoidance, Advanced Driver Assistance Systems
Abstract: Guaranteeing the safety of an autonomous vehicle (AV) is a challenging task, especially if the perceived environment is highly uncertain and other road users deviate from their expected trajectories. In this paper, we propose a probabilistic overall strategy for risk assessment and management of AV in highway through a Sequential Level Bayesian Decision Network (SLBDN) and an appropriate analytical formalization of criteria for anomaly detection based on a Dynamic Predicted Inter-Distance Profile (DPIDP) between vehicles. Accordingly, the proposed system is designed to take the suitable maneuver decision, have a safety retrospection and verification over the current maneuver risk and take appropriate evasive action autonomously from moving obstacles. Moreover, this probabilistic framework accounts for measurements uncertainty through an Extended Kalman Filter (EKF) and for vehicles' maximum capacities. Since the proposed strategy has a short response time, integrating safety verification in the decision-making process makes real time evasive decisions possible. Several simulation results show the good performance of the overall proposed control architecture, mainly in terms of efficiency to handle probabilistic decision-making even for risky scenarios.

Keywords: Automated Vehicles, Collision Avoidance, Vehicle Control
Abstract: Virtual testing of automated vehicles using simulations is essential during their development. When it comes to the testing of motion planning algorithms, one is mainly interested in challenging, critical scenarios for which it is hard to find a feasible solution. However, these situations are rare under usual traffic conditions, demanding an automatic generation of critical test scenarios. We present an approach that automatically generates critical scenarios based on a minimization of the solution space of the vehicle under test. By formulating a scenario parametrization and automatic determination of relevant parameter intervals, we are able to optimize the criticality of complex scenarios. We use evolutionary algorithms to tackle the resulting highly nonlinear optimization problem. Compared to our previous approach, we are now able to handle complex situations, in particular those involving intersections. Finally, we demonstrate our approach by generating critical scenarios from initially uncritical scenarios.