Validating Complexity of Automated Vehicles.
Updated: Apr 10, 2022
Porsche Engineering says that closely linking digital and real tests creates the basis for an efficient and reliable validation process. This is extremely important, especially as highly automated driving functions enter series production.
Automated systems could let the driver Switch on autopilot in stressful traffic situations, allowing them to sit back and relax in the driver’s seat or letting the assistance system take care of the tiresome task of finding a parking space and parking – highly automated driving functions will play a major role in further increasing comfort and safety in road traffic in the future. Such systems are highly complex, and nothing must be left to chance in their development.
“Validation methods that rely exclusively on real tests inevitably reach the limits of their potential in the area of assistance functions,” says Frank Sayer, Senior Manager of the Virtual Vehicle Development discipline at Porsche Engineering. In purely mathematical terms, the test vehicles would have to travel hundreds of millions of kilometres on the road to prove the reliability of the automated functions. Today’s standard test vehicle fleets would need several decades to do this – an impossible undertaking.
Together with colleagues from across Porsche Engineering's global team, Sayer and his team have therefore developed a new validation methodology that can be applied with great flexibility. The concept is based on the intermeshing of virtual tests on the simulator and real checks on the road. Developers from the Porsche Engineering Virtual ADAS Testing Center (PEVATeC) platform and colleagues from other areas work together in teams – from validation to function release. “We apply the same methodology worldwide and can ensure a high validation standard,” says Sayer. To make this possible, a central server structure was established through which the international project teams can exchange their results and determine the next processing steps.
Validating a new driving function usually begins with an intensive exchange of information.
“In these discussions, we work out which traffic situations and scenarios are particularly relevant for testing a new driving function,” says Tille Karoline Rupp, development engineer at Porsche Engineering. “With a parking assistant, it’s a question of maneuvering, whereas automated driving on the highway focuses more on safe longitudinal and lateral steering in different speed ranges.”
Based on this information, she creates the digital route model in the computer on which the test simulation will be executed – a task that often takes several days. “The level of detail and thus the effort required for this ‘digital twin’ are highly dependent on the driving function that is to be tested, as well as on the respective phase in the development process,” explains Rupp. “Highly realistic models of streetscapes with dense street networks in inner cities are very time-consuming. In early phases of function development, sometimes simple scenes with a few objects are enough, scenes we can create as a generic model from the toolbox.” To limit the effort required for future testing, all route models are catalogued and stored on the server so that the experts can reuse them later. The degree of automation in the creation process is also constantly on the rise, for example, creating scenes using configurable algorithms or combining different map data.
Experts for tests in the simulator: Clara Marina Martínez (left) and Tille Karoline Rupp create digital track and vehicle models for validation. In close coordination with one another, the development engineers manage simulation and methodology projects for virtual ADAS development at Porsche Engineering and within the VW Group. They also share responsibility for advancing the Porsche Engineering Virtual ADAS Testing Center (PEVATeC). Clara Marina Martínez has a doctorate in Intelligent Hybrid Electric Vehicles and an M.Sc. in Automotive Mechatronics from Cranfield University. Tille Karoline Rupp studied mathematics and physics at the University of Stuttgart and has a B.Sc. in electrical engineering from Baden-Württemberg Cooperative State University.
Development engineer Clara Marina Martínez uses these datasets to test the new driving function using virtual vehicles via the PEVATeC platform. She runs various test scenarios – also defined in advance with the development experts – on the route models. For example, the simulations can be used to test traffic situations that cannot easily be replicated in real road traffic for safety reasons. However, simulation can also generate and reproduce weather and lighting phenomena more easily. Recurring patterns include, for example, vehicles at intersections crossing in different sequences and with variable speeds, cars overtaking or cutting in, or pedestrians running into the street. The results are then evaluated together with the specialist department.
The next step is usually driving tests. In this stage, the updated software datasets from the simulation computer are transferred to the test vehicles' control units to verify the calculation results under real driving conditions. The driving tests for validating automated driving functions usually take place on a closed test track such as the Nardò Technical Center (NTC); for the validation of automated driving functions, test drives on public roads are the exception for safety reasons.
Engineers at the NTC are working on using driving robots in more and more situations during driving tests. “This enables great precision with regard to driving manoeuvres, which can also be fully reproduced,” explains Pierpaolo Positano, Senior Manager of Engineering at the NTC. Even today, up to six automated vehicles can be used in test scenarios, with mechanical actuators taking over the operation of the accelerator, brake, and steering wheel. The robotic cars are controlled by local computers that create a synchronized multi-vehicle scenario. A digital copy of the entire test track in Nardò was created for this purpose: a detailed computer-based replica of the circuit that reflects all the properties of the real system is created as a digital twin.
“With the digital twin, the line between simulation and reality is becoming increasingly blurred,” says Positano. “Very realistic simulations can be performed on the computer because, in addition to the course of the road, we take 100 percent of the properties such as characteristics of the individual road sections into account. We can have real cars on the track controlled by robots with this basis. They follow the simulation results and repeat the tests to obtain additional measurements compared to the simulation data for final validation. With this continuous chain of simulation and testing, we have developed a procedure in which both parts validate each other. This significantly increases the robustness of the results,” says Positano.
New questions often arise from the driving tests at the NTC or on other test tracks, such as the effects of rain and snow on the driving function. The simulations are then adjusted and rerun. Their results are then transferred back to the test vehicle. “We work with the same software platform for simulations as for the test vehicles, so our results can be transferred 1:1 to reality and the vehicle systems can be brought up to our software level quickly and easily,” says Martínez. In an iterative collaboration between the simulation and testing departments, the team works through all the validation points until the new highly automated driving function has been validated optimally.
Virtual Curve NTC - ITALY
Actual Curve NTC - ITALY
The new validation methodology will also be introduced at Porsche Engineering in China shortly. Since the subsidiary was founded in 2014, the Shanghai location has been the interface to local companies and has been Porsche AG’s strategic partner for the Chinese market. Porsche Engineering is currently massively expanding its capacities in the area of highly automated driving in China, also to take local conditions into account in optimal fashion during development. for example, the multi-level sections on which traffic is routed above rather than alongside each other for space reasons.
“If the driving function is not designed with this in mind, ADAS malfunctions may occur because there is no elevation model and no sufficient simulation test,” says Zhengjun Xu, Senior Manager, Software Development (HAD and ADAS) at Porsche Engineering China.
The driving style on China’s roads also differs in some respects from Europe. For example, frequent lane changes and lane changes with minimal gaps make the transition between a normal situation and a safety-critical situation much more sudden. A different tuning of the vehicle sensor system is therefore required. “The importance of highly automated driving has been growing in China in recent years,” says Zhengjun Xu. “That’s because many drivers value convenience features like automated highway driving and automated valet parking. In addition, automated and autonomous driving are key technologies for the future automotive industry, so the Chinese government has issued numerous policies and regulations to guide and accelerate the development of this technology.”
Test area for connected vehicles
An area of around 30 square kilometers has been designated on the outskirts of Shanghai for tests under real conditions in public road traffic. A modern 5G mobile network will enable new approaches to data exchange between vehicles and infrastructure to be developed and tested there. “All of these China-specific requirements call for extensive local research and development expertise. The simulation must also be carried out in China,” says Zhengjun Xu. “It’s a big challenge to simulate so many complex scenarios that are specific to China. We believe that the PEVATeC platform will bring great benefits to the efficiency and quality of China’s ADAS and highly automated driving development in the future.”
Porsche Engineering is currently setting up a modern private 5G mobile network in Nardò, enabling real-time data transmission between the vehicles and stationary computers, for example. “This will create the basis for future test concepts in which we want to perform tests, including adjusting test parameters, in a fully automated manner,” Positano explains. “The robot-controlled vehicle sends all measurement data to a stationary computer, where it is evaluated and processed. If the results show that parameter changes make sense, they can be transferred to the car in real time so that the effects can then be analyzed while the test is still ongoing.”
Bietigheim-Bissingen - Germany
The developers of the Porsche Engineering Virtual ADAS Testing Center (PEVATeC) platform and colleagues from other areas work together in teams. Their cooperation begins with the validation and ends with the release of the function. The results of the simulations are evaluated together with the specialist departments.
Nardò - Italy
he Nardò Technical Center (NTC) is where the real driving tests take place. Robots are increasingly used to control the vehicle to make the results reproducible. In addition, there is a digital twin of the test track that faithfully reflects all the properties of the real system. The simulation and testing thus mutually validate each other.
Shanghai - China
Porsche Engineering is currently massively expanding its capacities in the field of highly automated driving in China in order to take local conditions optimally into account during development. In addition to real tests, virtual methods will also play an important role there in the future.