Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: Automated Vehicles

Keywords: V2X Communication, Advanced Driver Assistance Systems, Vehicle Environment Perception
Abstract: One defect of on-road V2X (Vehicle to Everything) related test is bad repeatability. The outside electromagnetic environment is un-controllable and test output performance is very un-stable. An obvious advantage of in-chamber test is the controllability of test environment parameters, which could mend the defect of on-road field test. In this paper, an in-chamber test scheme is designed to fulfil V2X related test. Corresponding operation conditions are analysis, which test cases are defined in detail. Moreover, the test parameter setting method is given. A typical application, FCW, is test to verify the effectiveness of proposed scheme. Test results show that output of in-chamber test is stable, which illustrate a good repeatability.

Keywords: Advanced Driver Assistance Systems, Vehicle Control, Cooperative ITS
Abstract: Cooperative Adaptive Cruise Control (CACC) in previous researches typically refers to the linear controller with a gap policy. The system could not be designed to fulfill multiple objectives. This inspires the concept of optimal control based CACC in this paper. The basic procedure of the proposed controller is to gather the information collected by each vehicle to the computation unit first, then plan the trajectory of all the followers by solving an optimal control problem, and dispatch the optimal motion command to each vehicle at last. This paper models CACC under optimal control framework. A numerical approach inspired by dynamic programming is adopted to solve the control problem. The stability of the proposed controller is thoroughly investigated in terms of both local stability and string stability. To verify the concept of controller, solution, and the analysis about stability, simulation is carried out. The simulation verifies that the numerical method is effective with respect to computation time. Both theoretical analysis and simulation proved that the proposed optimal control based CACC is both local stable and string stable. The low computation burden, local stability, and string stability together guarantee the future implementation of the proposed controller.

Keywords: Security, Automated Vehicles, Advanced Driver Assistance Systems
Abstract: In this paper, a novel model for the cyber-security analysis of Level 3 (L3) Automated Driving (AD) systems is proposed by integrating aspects of functional safety. The model is built based on the state-of-the-art framework for cyber security analysis, known as Threat Analysis and Risk Assessment (TARA), which quantifies the likelihood and the impact of attack and combines them in order to derive an attack risk value. The novelty lies in the bespoke integration of the impact calculation, which incorporates the notion of controllability of an attack by the AD system and/or by the driver. The proposed model is applied for the Urban Chauffeur and the Highway Chauffeur AD system functions, providing insights into the security risk in a wide area of distinct operational design domains as defined by SAE J3016. Remote attack surfaces (e.g., modifications of road infrastructure) are also taken into account in the analysis.

Keywords: Automated Vehicles, Cooperative Systems (V2X), Cooperative ITS
Abstract: Connected and autonomous/automated vehicle (CAV) technologies are shaping the design and the new developments in the automotive industry and, in a wider perspective, in the mobility sector as well. Despite the recent advances and on-going developments, and the enthusiasm around autonomous mobility systems, real-world testing of CAVs is a crucial element to allow the next generation of intelligent vehicles to come to our daily-life. The importance of realistic testing is recognised by academia, industry, public sector and stakeholders, and is reflected in all projects involving pilots and advanced prototyping. AUTOCITS is one of the projects where CAVs and interoperability tests have been conducted. This paper concentrates on the assessment and performance evaluation of tests carried out during the AUTOCITS’s Lisbon Pilot, in real-world conditions, involving CAVs and C-ITS technologies. New specific quantitative indicators (key performance indicators - KPIs) are proposed to back the assessment and evaluation criteria presented in this work. The KPIs’ expressions are provided, which demonstrated to be very difficult to find in the literature. Results are reported and discussed according to the scenarios and field-data recorded during the Pilot.

Keywords: Cooperative Systems (V2X), Automated Vehicles, Cooperative ITS
Abstract: Connected and automated vehicles can exploit V2X communications to coordinate their maneuvers and improve the traffic safety and efficiency. To support such coordination, ETSI is currently defining the Maneuver Coordination Service (MCS). The current approach is based on a distributed solution where vehicles coordinate their maneuvers using V2V (Vehicle-to- Vehicle) communications. This paper proposes to extend this concept by adding the possibility for the infrastructure to support cooperative maneuvers using V2I (Vehicle-to- Infrastructure) communications. To this aim, we propose a Maneuver Coordination Message (MCM) that can be used in cooperative maneuvers with or without road infrastructure support. First results show the gains that cooperative maneuvers can achieve thanks to the infrastructure support. This paper also analyses and discusses the need to define MCM generation rules that decide when MCM messages should be exchanged. These rules have an impact on the effectiveness of cooperative maneuvers and on the operation and scalability of the V2X network.

Keywords: Automated Vehicles, Self-Driving Vehicles, Security
Abstract: The testing of cyber-physical systems such as automated vehicles (AV) is difficult as engineers face challenges from both cybersecurity and safety domains that start to converge. For cybersecurity, conducting vulnerability testing even before mitigation designs are fixed requires the predication and modeling of adversaries’ malicious behaviors. For safety, complete testing at system-level is time-consuming and also infeasible due to the large combination of operational domains. To help engineers design cost-effective mitigation solutions, this paper presents a framework for constructing testing scenarios driven by cyber threats that can be evaluated early in the design process. The testing results can inform the design of mitigation strategies and help engineers in constructing security requirements such that the large solution space will converge more quickly on effective designs. We also illustrate how to build visualization tools to support this process.

Keywords: Deep Learning, Automated Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: Deep Learning methods are widely applied in Robotics and Automated Driving scenarios. The task of perception for Automated Driving in the real world is particularly challenging and requires a sufficient amount of high quality labeled training data for the algorithms to perform well. However, the means of obtaining real world datasets are limited. It is common practice to have human labelers involved at least to some extent. Regardless of whether the process is partially automated or not, these labels never represent perfectly accurate ground-truth. By investigating the recognition performance of a state-of-the-art object detector as a function of the quality of a labeled real world training set, we study the effect of labeling errors of various types and severity. To this end, the given labels are treated as a reference to which synthetic errors are added systematically in order to determine the performance of the object detector if trained on the erroneous dataset.

Keywords: Automated Vehicles, Active and Passive Vehicle Safety
Abstract: Automated vehicles come with promises for higher comfort and safety compared to standard human-driven vehicles. Various demonstrator vehicles with fully automated driving capabilities have been already presented with success. Yet, there is a large number of technical challenges to be solved until the safety levels comply with those required from safety standards, and most importantly with those for public acceptance. In this paper, we introduce the technical challenges resulting from the need for fault-tolerant capabilities of automated vehicles with no fallback-ready drivers. We then propose a concrete solution to these challenges. This includes a fault-tolerant architecture for automated driving systems. Furthermore we introduce, the safety co-pilot, that is a safety mechanism that ensures the coordinated operation of two or more redundant automated driving systems, by means of novel application-specific verification methods. We conclude our work with experimental proof of concept results of the proposed solution.

Keywords: Automated Vehicles, Self-Driving Vehicles
Abstract: Automated driving road vehicles are to operate in an unstructured, public real-world environment. The openness of the operational design domain, the serious safety risk, the complexity of the system itself, as well as the regulatory situation pose a large challenge to the automotive industry. Thus, a strategy is necessary to ascertain the validity of such systems. An extensive formalisation of the problem and solution proposal are provided in the authors' work [1]. This paper describes a simplified version of the formalisation. The question of validating open context systems is dissected into the interdependent aspects of purpose, context and realisation. This allows us to establish why undesirable gaps between the required, the specified and the eventually implemented behaviour can occur. These gaps refer to qualitatively different deviations of the system and are addressed by our novel algorithmic approach. Furthermore, the contributions and aspects left uncovered by normative regulations, i.e. ISO 26262 and ISO PAS 21448, are established.

Keywords: Situation Analysis and Planning, Automated Vehicles
Abstract: This paper presents a probabilistic framework for collision probability estimation. The framework uses information about objects' velocity and acceleration gathered from a larger real traffic data set in order to create a discrete Markov Chain model. This model is then used to predict other traffic participants motion in a given scenario and through this calculate the probability of a future collision. The framework is then analyzed with respect to potential errors that are created in the discretization process. Especially the errors related to the discrete velocity regions are investigated in more detail. The analysis is performed on a selection of critical scenarios from a larger data set in order to set scenario-based requirements of the state discretization resolution. In the end, there is a discussion about the implications for the collision probability estimate, as well as, suggested next steps in order to get a complete view of the precision of the estimate.

Keywords: Self-Driving Vehicles, Collision Avoidance
Abstract: We develop a set of contracts for autonomous control software that ensures that if all traffic participants follow the contracts, the overall traffic system will be collision-free. We express our contracts in Signal Temporal Logic (STL), a lightweight specification language that enables V&V methodologies. We demonstrate how the specification can be used for evaluation of the performance of autonomy software, and We provide preliminary evidence that our contracts are not excessively conservative, i.e., they are not more restrictive than existing guidelines for safe driving by humans.

Keywords: Autonomous / Intelligent Robotic Vehicles
Abstract: Navigating in complex and highly dynamic environments such as crowds is still a major challenge for autonomous vehicle such as autonomous wheelchairs or even autonomous cars. This article presents a new way of navigating in crowds by using behavioral clustering for the surrounding agents and representing the crowd as a set of moving polygons. Once the environment has been modelled in this way and the robot has all the information it needs, we then propose a navigation algorithm that is able to guide the vehicle through the scene. The key-points of this algorithm are that (1) it can avoid densely-populated areas in order to minimize the risk of being on a collision course with any of the surrounding dynamic obstacles, (2) it generates socially compliant trajectories.

Keywords: Autonomous / Intelligent Robotic Vehicles, Vulnerable Road-User Safety, Human-Machine Interface
Abstract: As part of a large UK-funded autonomous vehicle project (UK Autodrive), we examined pedestrian attitudes and road-crossing intentions using a real autonomous vehicle (AV) in an indoor arena. Two conceptual external human-machine interfaces (HMIs) were presented to display the vehicle’s manoeuvring intentions. Participants experienced a simulated road-crossing task to assess their interactions with the AV. Although neither HMI concept was entirely free of criticism, there were objective performance differences for a projection-based HMI concept, as well as critical subjective opinions in pedestrian responses to specific manoeuvring contexts. These provided insight into pedestrians’ safety concerns towards a vehicle where bi-directional communication with a driver is no longer possible, with suggestions for future vehicle HMI concepts.

Keywords: Driver State and Intent Recognition, Driver Recognition, Human-Machine Interface
Abstract: To study naturalistic in-cabin emotion we developed SKYNIVI, a modified open source driving simulator, with scenarios designed to elicit startle and frustration. We target generating these emotions because we believe that by detecting these it will be possible for autonomous vehicles to learn to drive better. We show how to use SKYNIVI to develop datasets that capture naturalistic emotions in drivers and passengers for algorithmic development. We recruited 51 participants as dyads and single drivers to participate in two different scenarios. We show that we were able to evoke hundreds of instances of our target emotions in this cohort and present an analysis of factors we found to impact emotional expression including: scenario design (p<0.01), demographic factors, personality and baseline affect (p<0.05). We find that having a second person in the vehicle impacts observed expressions of emotion (p<0.05, p<0.01) even when no difference in baseline affect is reported.

Keywords: Automated Vehicles, Driver State and Intent Recognition, Vehicle Control
Abstract: In future mixed traffic Highly Automated Vehicles (HAV) will have to resolve interactions with human operated traffic. A particular problem for HAVs is detection of human states influencing safety critical decisions and driving behavior of humans. We demonstrate the value proposition of neuro-physiological sensors and driver models for optimizing performance of HAVs under safety constraints in mixed traffic applications.

Keywords: Self-Driving Vehicles, Human-Machine Interface, Vulnerable Road-User Safety
Abstract: Trust plays a decisive role in the public’s acceptance of the new self-driving car technology. In order to better understand how to promote confidence in vehicle automation safety among the public, we studied pedestrian behavior shortly before and while crossing a marked crosswalk. Such information is also essential for setting parameters for automated vehicles to act accordingly during interactions with pedestrians. Through the analysis of the recorded videos and subjective qualitative data, we identified factors that potentially influence the perception of a road situation as safe in an environment in which vehicles operate with full driving automation (level 5) in a public space. A variety of responses were observed that exhibit several levels of trust, uncertainty and a certain degree of fear. It became clear, however, that the longer the people interacted with the vehicles, the more confident and trusting they became in automation capabilities. The existence of a communication system to interact with driverless vehicles was also evaluated as positive.

Keywords: Automated Vehicles, Human-Machine Interface, Vehicle Control
Abstract: Understanding how best to present information inside a semi-automated vehicle is a prevalent challenge in HMI design. There is an understanding that a driver’s trust and previous driving experience can affect the information they require inside a semi-automated vehicle. However, to date little is known about how these predispositions specifically affect the types of information that should be presented and importantly, how this changes with increased exposure to an automated system. In this paper, seventeen participants experienced twenty-six minutes of an automated driving simulation once every day for a week. The information to display was carefully chosen in accordance with the Skills, Rules, Knowledge model. The information was synchronized to the driving simulation and presented on a tablet in the driving simulator. Eye tracking was used to measure the information looked at. The results showed that trust increased significantly with increased exposure, but this had no correlation to any specific piece of information viewed. Drivers who were more prone to making lapses or errors (as measured by the Driver Behavior Questionnaire) tended towards using information that was less cognitively demanding. Finally, a driver’s propensity to making lapses was found to be a potential early predictor of trust, but this became less accurate with increased exposure to the semi-automated vehicle.

Keywords: Driver State and Intent Recognition, Advanced Driver Assistance Systems, Human-Machine Interface
Abstract: A lot of research has been done to model and predict a driver’s behaviors to improve driving safety. Inferring lane change maneuver can be a critical one among them. However, the lane change prediction problem is generally treated as a classification task in which the labels represent the probability of whether the driver will make a lane change in the upcoming few seconds. In our work, we formulate this problem as a regression task. The process of lane change behavior is analyzed to build a Gated Recurrent Units (GRU) network for predicting two time points during lane change behavior: a) When the driver will shift lane. b) When the lane change will be completed. We make a comparison of Long Short Term Memory (LSTM) network and Support Vector Machine (SVM) regression performance to show that our method can give a more precise prediction time. This work can be used to improve the safety performance of driver assistance systems and help other traffic participants having a safer environment.

Keywords: Lidar Sensing and Perception, Deep Learning, Automated Vehicles
Abstract: 3D semantic scene labeling is a fundamental task for Autonomous Driving. Recent work shows the capability of Deep Neural Networks in labeling 3D point sets provided by sensors like LiDAR, and Radar. Imbalanced distribution of classes in the dataset is one of the challenges that face 3D semantic scene labeling task. This leads to misclassifying for the non-dominant classes which suffer from two main problems: a) rare appearance in the dataset, and b) few sensor points reflected from one object of these classes. This paper proposes a Weighted Self-Incremental Transfer Learning as a generalized methodology that solves the imbalanced training dataset problems. It re-weights the components of the loss function computed from individual classes based on their frequencies in the training dataset, and applies Self-Incremental Transfer Learning by running the Neural Network model on non-dominant classes first, then dominant classes one-by-one are added. The experimental results introduce a new 3D point cloud semantic segmentation benchmark for KITTI dataset.

Keywords: Deep Learning, Lidar Sensing and Perception, Autonomous / Intelligent Robotic Vehicles
Abstract: The detection and localization of objects on point cloud provided by LiDAR sensors have various applications, especially for autonomous driving. As methods are being more efficient in terms of performance and execution speed, they operate in an end-to-end fashion, not allowing in case of errors the evaluation of which areas could cause failures. We propose in this paper a simple method that delivers from a simple statistical voxel encoding an intermediate top view heatmap that illustrates the global state of the scene seen by the network. Completely detected vehicles are represented on this map as elliptic blobs. Difficult cases such as occluded cars may not illustrate a complete spot, however their mark may still indicate that a possible hazard, allowing planification algorithms to decide a reduction of the velocity as something may suddenly appear. Furthermore, thanks to the alternative representation of objects, detections in terms of bounding boxes can be extracted even from the intermediate map. In addition to its implementation simplicity, the proposed architecture reaches high level of performance on the KITTI detection benchmark.

Keywords: Automated Vehicles, Cooperative Systems (V2X)

Keywords: Automated Vehicles, Cooperative Systems (V2X)
Abstract: Self-driving cars have experienced a booming development in the latest years, having achieved a certain degree of maturity. It is well known in the scientific community that the reliability and perception horizon of self-driving cars will be further enhanced by means of cooperation with other vehicles. Thus, Cooperative Automated Driving is expected to become the future of vehicle autonomy. The benefits of Cooperative Perception will be immense in mixed traffic environments, even for non-automated V2V-equipped and manually driven vehicles. This talk will analyze some of the practical issues affecting cooperative perception for collaborative automated driving. For such purpose, a bird-eye view of cooperative driving scenarios and technological constraints will be presented, followed by the discussion of experimental findings obtained in a series of real tests carried out with several V2V-equipped vehicles.

Keywords: Vulnerable Road-User Safety, Vision Sensing and Perception, Self-Driving Vehicles
Abstract: Recent accidents on roads involving cyclists and autonomous vehicles have raised an alarm in the industry to focus more on cyclist safety. Although there are plenty of datasets publicly available in the industry, they don't include enough instances of cyclists in different road conditions to run comprehensive tests. Therefore, in this work, we present a new publicly available Specialized Cyclist Dataset, which focuses solely on cyclist detection. Our dataset was recorded using a monocular RGB camera in various scenarios experienced by cyclists on roads in Autumn and Winter (with snow) for enabling researchers to run rigorous tests in various conditions. There are 62297 total images, about 18200 cyclists instances, and 30 different cyclists. Additionally in the dataset, we present the Specialized cyclist jersey with a diamond pattern designed specifically for improving detection accuracy compared to street clothes. For convenience, we utilized the popular KITTI labeling format and resolution in addition to Full HD resolution.

Keywords: Self-Driving Vehicles, Deep Learning, Sensor and Data Fusion
Abstract: While the detection and classification of simple objects encountered during autonomous driving sessions has been widely researched, the detection of complex objects and situations based on the combinations of objects in a scene remains relatively overlooked. This is especially difficult due to the cost of gathering labels for each complex scenario of interest before training a specialized model. To address this bottleneck of training data, we explore the applicability of weak supervision, or relying on higher level, noisier forms of supervision to label training data. Specifically, we use data programming, a paradigm that can learn the accuracy and dependency structure of these sources without using any ground truth labels and assign training labels accordingly. We focus on an example task of cyclist detection by comparing weak supervision, which relies on a set of user-defined rules over the outputs of detectors that identify people and bikes separately, to CyDet [1], which detects the cyclist as a complete object. We find that the weak supervision method can achieve a performance of 96.8 F1 points, 4.6 F1 higher than CyDet, without relying on any ground truth labels on the newly released Specialized Cyclist Dataset. We then discuss how heuristics can detect complex objects such as cyclists and by extension, situations, based on the output of existing object detection algorithms.

Keywords: Cooperative Systems (V2X), Vehicle Environment Perception, Vision Sensing and Perception
Abstract: Autonomous vehicles need a complete and robust perception of their environment to correctly understand the surrounding traffic scene and come to the right decisions. Making use of vehicle-to-vehicle (V2V) communication can improve the perception capabilities of autonomous vehicles by extending the range of their own local sensors.

For the development of robust cooperative perception systems it is necessary to include varying environmental conditions to the scenarios used for validation. In this paper we present a new approach to investigate a cooperative perception pipeline within simulation under varying rain conditions. We demonstrate our approach on the example of a complete vision-based cooperative perception pipeline. Scenarios with a varying number of cooperative vehicles under different synthetically generated rain variations are used to show the influence of rain on local and cooperative perception.

Keywords: Automated Vehicles, Cooperative Systems (V2X), Cooperative ITS
Abstract: Collective Perception (CP) or cooperative sensing enables vehicles and infrastructure nodes to exchange sensor information to improve their perception of the driving environment. CP enables vehicles to detect objects (e.g. non-connected vehicles, pedestrians, obstacles, etc.) beyond their local sensing capabilities. ETSI is currently developing the European standards for collective perception or cooperative sensing. This includes defining which information should be exchanged about the detected objects, and how often it should be exchanged. To this aim, different CP generation rules for collective perception are currently under analysis, and this paper presents an in-depth analysis of their performance and efficiency. The conducted analysis highlights the existing trade-offs between performance (capacity to detect surrounding objects) and efficiency (redundant detection and transmission of the same detected objects). It also demonstrates the need to design advanced policies that dynamically control the redundancy on the wireless channel while ensuring the capacity to reliably detect the driving environment.

Keywords: Automated Vehicles, Cooperative Systems (V2X)
Abstract: Tactical maneuver planning of multiple, communicating vehicles provides the opportunity to increase passenger safety and comfort. We propose a unifying method to orchestrate the motion of cooperative vehicles based on the negotiation of conflicting road areas, which are determined by reachable set computation. As a result, each vehicle receives an individual driving corridor for trajectory planning. The presented conflict resolution scheme has polynomial runtime complexity and is guaranteed to find the optimal allocation of road areas for each negotiation round. Our method is not tailored to specific traffic situations but is applicable to general traffic scenes with manually driven and automated vehicles. We demonstrate the universal usability of our approach in numerical experiments.

Keywords: Automated Vehicles, Self-Driving Vehicles, Security
Abstract: For a successful introduction of fully automated vehicles, they must behave both provably safe but also convenient, i.e. comfortable and not overcautious. Given the limited sensing capabilities, especially in urban scenarios where buildings and parking vehicles impose occlusions, this is a challenging task. While recent approaches gave first ideas for boundary conditions of safe behavior, an approach for convenient motion planning that fulfills these constraints is still remaining. Therefore, we utilize and enhance given safety approaches for occlusion handling in order to facilitate comfortable and safe motion planning. In order to do so, we consider worst case assumptions, arising from potential objects at critical sensing field edges, along with their probability. With this information, we can ensure to not act overcautiously while still moving provably safe. The potential of our approach is shown in a modified CommonROAD scenario.

Keywords: Situation Analysis and Planning, Self-Driving Vehicles, Vehicle Control
Abstract: This paper considers the problem of planning reference trajectories for cooperating autonomous vehicles. Our previous work [1] relied on mixed-integer quadratic programming (MIQP) for planning, while controllable sets of hybrid automata were used to assess feasibility. In this paper, we supersede MIQP in the online part and obtain plans by directly making use of controllable sets and adapting ideas from approximate multi-parametric programming. The run-time of the algorithm is not only lower, but also more predictable than that of MIQP solvers, which is crucial for real-time operation. Even though it returns suboptimal solutions, the algorithm also enables the use of smaller sampling times or longer planning horizons.

Keywords: Intelligent Vehicle Software Infrastructure, Cooperative ITS, Cooperative Systems (V2X)
Abstract: Mapping the logical software architecture of a vehicle to a technical solution is not a straightforward task. A particular challenge is communication: software components developed by different teams and deployed across the E/E architecture need to be able to exchange data. Middleware solutions have been developed to enable low coupling of distributed logical software components. Building a distributed architecture on a middleware solution is mostly accomplished by encapsulating logical components into middleware wrappers. This is not only time-consuming, but also requires platform-specific understanding, and results in a multitude of architectural variants tailored for particular set-ups. For instance, lengthy validation processes ensuring functional correctness and safety require simulations of intelligent vehicle systems in different simulators, environments, and on different abstraction levels. This leads to the necessity of individual integration schemes for both simulation and deployment. We propose a component-based modeling approach separating platform-agnostic logical models from middleware aspects. Therefore, the model compiler is instrumented with middleware tags related to the elements of the logical model. Generating the required middleware code automatically, we aim at better component re-usability minimizing the need for hand-crafted glue-code for interprocess and simulator integration.

Keywords: Automated Vehicles, Convolutional Neural Networks, Vision Sensing and Perception
Abstract: In autonomous driving research, one of the bot- tlenecks is the shortage of a well-annotated dataset to train deep neural networks for object detection. Specifically, a dataset focusing on harsh weather conditions is insufficient. The purpose of this research is to explore the power of utilizing synthetic data for training object detection deep neural networks under harsh weather conditions. We introduce a state-of-the-art automated pipeline to collect synthetic images from a high realism video game and generate training data which can be used for training an autonomous driving object detection neural network. We use our synthetic dataset, KITTI, and Cityscapes to train three separate object detection neural networks and employ the PASCAL object detection criteria to evaluate each neural networks’ performance. The results from the experiment indicate that the neural network trained by our synthetic dataset outperforms its counterparts and achieves higher average precision (AP) in detecting images under harsh weather conditions. The result sheds a light on employing synthetic data to resolve the challenges in the real world.

Keywords: Driver State and Intent Recognition, Sensor and Data Fusion, Recurrent Networks
Abstract: Modern vehicles produce big data with a wide variety of formats due to missing open standards. Thus, abstractions of such data in the form of descriptive labels are desired to facilitate the development of applications in the automotive domain. We propose an approach to reduce vehicle sensor data into semantic outcomes of dangerous driving events based on aggressive maneuvers. The supervised time- series classification is implemented with Random Forest and Recurrent Neural Network separately. Our approach works with signals of a real vehicle obtained through a back-end solution, with the challenge of low and variable sampling rates. We introduce the idea of having a dangerous driving classifier as the first discriminant of relevant instances for further enrichment (e.g., type of maneuver). Additionally, we suggest a method to increment the number of driving samples for training machine learning models by weighting the window instances based on the portion of the labeled event they include. We show that a dangerous driving classifier can be used as a first discriminant to enable data integration and that transitions in driving events are relevant to consider when the dataset is limited, and sensor data has a low and unreliable frequency.

Keywords: Recurrent Networks, Vision Sensing and Perception, Automated Vehicles
Abstract: We present an experimental study comparing various Recurrent Neural Network architectures for the task of Vulnerable Road User (VRU) motion trajectory prediction in the intelligent vehicle domain. Making use of temporal motion cues and visual appearance features, we design multi-cue RNN-based architectures with dedicated optimization process to predict future moving trajectories from historical consecutive frames. Experiments are performed on image sequences recorded from on-board a moving vehicle and public tracking datasets. In particular, the Tsinghua-Daimler Cyclist Benchmark (TDCB) has been augmented with additional annotations (various VRU types) to support the evaluation of object tracking approaches and trajectory prediction methods. This newly introduced dataset is termed TDCB-Track. We demonstrate the effectiveness of the proposed RNN architectures on the public MOT16 dataset and the TDCB-Track dataset. We show that the proposed approaches outperform simpler baseline methods and stay ahead with the state-of-the-art.

Keywords: Unsupervised Learning, Smart Infrastructure, Intelligent Vehicle Software Infrastructure
Abstract: Diagnostic trouble codes (DTCs) are steadily produced by a vehicle's control units to support the diagnosis process when the vehicle is maintained or to initiate predictive maintenance. Although, DTCs carry a lot of information, possibly including environmental data such as the engine temperature, the velocity, etc., they are of little help to an automotive engineer if seen without a context. In fact, a concrete problem can mostly be diagnosed if an already known pattern of DTCs is present. However, detecting new patterns in masses of vehicle data gathered each day from thousands of vehicles and recognizing known patterns accurately cannot be performed manually by automotive engineers. We propose an unsupervised DTC pattern learning framework supporting the daily field data analysis of original equipment manufacturers (OEMs).

Keywords: Unsupervised Learning, Autonomous / Intelligent Robotic Vehicles, Deep Learning
Abstract: This paper proposes a method to evaluate Signal Temporal Logic (STL) robustness formulas using computation graphs. This method results in efficient computations and enables the use of backpropagation for optimizing over STL parameters. Inferring STL formulas from behavior traces can provide powerful insights into complex systems, such as long-term behaviors in time-series data. It can also be used to augment existing prediction and planning architectures by ensuring specifications are met. However, learning STL formulas from data is challenging from a theoretical and numerical standpoint. By evaluating and learning STL formulas using computation graphs, we can leverage the computational efficiency and utility of modern machine learning libraries. The proposed approach is particularly effective for solving parameteric STL (pSTL) problems, the problem of parameter fitting for a given signal. We provide a relaxation technique that makes this method tractable when solving general pSTL formulas. Through a traffic-weaving case-study, we show how the proposed approach is effective in learning pSTL parameters, and how it can be applied for scenario-based testing for autonomous driving and other complex robotic systems.

Keywords: Advanced Driver Assistance Systems, Automated Vehicles, Driver Recognition
Abstract: Because of the high accuracy and low cost, learning-based methods have been widely used to model driver behaviors in various scenarios. However, the performance of learning-based methods depend heavily on the quantity and coverage of the driving data.  When the new driver with insufficient data is considered, the accuracy of these methods cannot be guaranteed any more. To solve this problem, the balanced distribution adaptation (BDA) is used to build the new driver’s decision making model in the lane change (LC) scenario. Meanwhile, a transfer learning (TL) based regression model, modified BDA (MBDA) is proposed to predict the driver’s steering behavior during the LC maneuver. Cross validation (CV) based model selection (MS) method is developed to obtain the optimal parameters in model training process. A series of experiments are carried out based on the simulated and naturalistic driving data to verify the TL based classification and regression models. The experimental results indicate that the BDA and MBDA have an outstanding ability in knowledge transfer. Compared with support vector machine (SVM) and Gaussian mixture regression (GMR), the proposed methods show a better performance in the decision making of lane keep/change and the prediction of the driver’s steering operation.

Keywords: Automated Vehicles, Self-Driving Vehicles, Situation Analysis and Planning
Abstract: A main difficulty in autonomous driving is the assurance of maneuver acceptability by other traffic participants. Thus, knowledge about social interaction needs to be incorporated into the motion planning process. In this paper we present a model based framework to verify the acceptance of considered maneuvers and to plan social compliant motions. Therefore, we fuse two powerful approaches, one for decision-making and one for planning and show how the methods benefit from each other. Our method adheres to the classical structure of decision-making with subsequent trajectory planning and is consistent in the sense that both components are subject on the same, identical parametrized driver model. The overall method is real-time capable and the resulting trajectories adhere to kinematic constraints. Thus, the approach is applicable in real-world systems.

Keywords: Self-Driving Vehicles, Automated Vehicles, Autonomous / Intelligent Robotic Vehicles
Abstract: Autonomous cars have to navigate in dynamic environment which can be full of uncertainties. The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area. To safely and efficiently drive in the presence of these uncertainties, the decision-making and planning modules of autonomous cars should intelligently utilize all available information and appropriately tackle the uncertainties so that proper driving strategies can be generated. In this paper, we propose a social perception scheme which treats all road participants as distributed sensors in a sensor network. By observing the individual behaviors as well as the group behaviors, uncertainties of the three types can be updated uniformly in a belief space. The updated beliefs from the social perception are then explicitly incorporated into a probabilistic planning framework based on Model Predictive Control (MPC). The cost function of the MPC is learned via inverse reinforcement learning (IRL). Such an integrated probabilistic planning module with socially enhanced perception enables the autonomous vehicles to generate behaviors which are defensive but not overly conservative, and socially compatible. The effectiveness of the proposed framework is verified in simulation on an representative scenario with sensor occlusions.

Keywords: Driver Recognition, Driver State and Intent Recognition, Active and Passive Vehicle Safety
Abstract: Understanding human driving style and classifying driver’s risk pattern is the basis of traffic risk management. The recent rapid increase of the availability of taxi trajectory data, combined with the popular analysis techniques for big data, gives the chance of thorough analysis of taxi drivers’ driving style and risk pattern. In this paper, the driving characteristics of 10674 taxies (at Qiangsheng Taxi Corporation) in a month are extracted from trajectory data. The trajectory data includes time, position, motion, as well as operating status. The method adopted in this paper is entropy weight-analytic hierarchy process (Entropy-AHP) with speed, over speed behavior, driving stability, mileage and time, and fatigue driving as first-grade indexes. The weights of indexes and risk value are calculated, then all taxi drivers are grouped into five risk grades. The risk pattern recognized from the data could be particularly helpful for insurance companies to formulate differentiated pricing strategy.

Keywords: Lidar Sensing and Perception, Vehicle Environment Perception
Abstract: Autonomous vehicles require clear road markings and a high-level quality of infrastructure. This paper addresses road c ourse detection problem in non cooperative environments (i.e. absence or poor quality of road-marking, working zones, etc.). To cope with visual lane detection challenges in these difficult scenarios, we propose a virtual lane generation system to provide a comfortable and safe ride. Based on Lidar sensor, the dual-level virtual lane system consists of the combination of two blocks: the first constructs virtual lanes based on independent road-borders detection, while the second level uses dynamic objects detection and their trajectories in order to estimate the lane parameters. Furthermore, the system is able to anticipate road infrastructures thanks to the independent detection of road borders. Thus we are able to manage difficult use cases such as bifurcations and exit lanes without cartography. The performance is tested through extensive experiments with Cruise4U Valeo self-driving cars on highway and beltway roads. Experimental results demonstrate the accurate and robust performance of the proposed system.

Keywords: Advanced Driver Assistance Systems, Deep Learning, Vision Sensing and Perception
Abstract: Reliable perception and detection of objects are one of the fundamental aspects of vehicle autonomy. Although model-based approaches perform well in the area of planning and control, they often fail when applied to perception due to the open-world nature of problems for autonomous vehicles. Therefore, data-driven approaches to object detection and location are likely to be used in both self-driving cars and advanced driver assistance systems. In particular, the deep neural networks proved to be excellent in detection and classification of objects from images, often achieving super-human performance. However, neural networks applied in intelligent vehicles need to be explainable, providing rationales for their decisions. In this paper, we demonstrate how such an interpretation can be provided for a deep learning system that detects specific objects (charging posts) for driver assistance in an electric bus. The interpretation, achieved by visualization of attention heat maps, has twofold use: it allows us to augment the dataset used for training, improving the results, but it also may be used as a tool when fielding the system with the given bus operator. Explaining which parts of the images triggered the decision helps to eliminate misdetections.

Keywords: Intelligent Ground, Air and Space Vehicles, Advanced Driver Assistance Systems, Vehicle Control
Abstract: The paper presents a motion algorithmization subsystem used as a part of an advanced driver assistance system (ADAS) which is intended to help a human operator perform difficult maneuvers with intelligent urban buses in a more effective way. The algorithmization task, being a tight combination of motion planning and feedback control, is addressed in the presence of various constraints imposed on the nonholonomic vehicle motion, like limited motion curvature and curvature rate, obstacle collision avoidance, and maneuvering without a change of a longitudinal velocity sign (monotonic maneuvers). Description of the system is followed by exemplary results obtained on a laboratory testbed emulating a driver workplace with ADAS.

Keywords: Autonomous / Intelligent Robotic Vehicles, Advanced Driver Assistance Systems, Collision Avoidance
Abstract: In this work, a novel interval-based/data-driven safety verification technique is introduced for Intelligent/Autonomous Vehicles (I/AV). The interval arithmetic is adopted to enhance the reliability of the analytical models used for the autonomous navigation. Furthermore, a data-driven technique, which monitors the correlation relating variables of the modeled system, is adopted to ameliorate the uncertainty assessment. In such a manner, tight bounds of safety margins are obtained. To provide reliable safety verification, the proposed risk management approach has been integrated on an Adaptive Cruise Control (ACC) system. It permits to detect erroneous uncertainty estimation of an Extended Kalman Filter (EKF). Simulation results prove the overall risk management efficiency and its ability to handle uncertainties.

Keywords: Self-Driving Vehicles, Advanced Driver Assistance Systems, Collision Avoidance
Abstract: In this paper, a design of a multi-controller architecture (MCA) is presented. It effectively links model-based approaches and Artificial Intelligence (AI) developments for intelligent vehicles navigation in a highway. In this MCA, the model-based approach appears in the path planning (based on analytical target set-points definition) and the control law (based on a Lyapunov stability analysis). The AI-based approach appears in the proposed Two-Sequential Level Bayesian Decision Network (TSLBDN) for handling lane change maneuvers in uncertain environment and changing dynamic/behaviors of the surrounding vehicles. In addition, a combination of both trajectory prediction (based on dynamic target set-points and elliptic limit-cycles) and maneuver recognition based on Dynamic Bayesian Network (DBN) is proposed to infers surrounding vehicles actions. Several simulation results show the efficiency of the model-driven/data driven overall proposed control architecture.

Keywords: Self-Driving Vehicles, Automated Vehicles, Vehicle Environment Perception
Abstract: Automotive systems must undergo a strict process of validation before their release on commercial vehicles. With the increased use of probabilistic approaches in autonomous systems, standard validation methods are not applicable to this end. Furthermore, real life validation, when even possible, implies costs which can be obstructive. New methods for validation and testing are thus necessary. In this paper, we propose a generic method to evaluate complex probabilistic frameworks for autonomous driving. The method is based on Statistical Model Checking (SMC), using specifically defined Key Performance Indicators (KPIs), as temporal properties depending on a set of identified metrics. By studying the behavior of these metrics during a large number of simulations via our statistical model checker, we finally evaluate the probability for the system to meet the KPIs. We show how this method can be applied to two different subsystems of an autonomous vehicle: a perception system and a decision-making approach. An overview of these two systems is given to understand related validation challenges. Extensive validation results are then provided for the decision-making case.