Massively Parallel Virtual Testing of Safety-Relevant Driving Systems

SPICETECH GMBH (End User and ISV) is a German company with a focus on data-centric IT solutions, machine learning and data science. XLAB d.o.o. (HPC Expert), is a Slovenian company with a strong research background in the fields of distributed systems, cloud computing, information visualization (medical, 2D, 3D geographical information services (GIS), sensors) and image processing (3D reconstruction, segmentation, GIS data fusion). HLRS (HPC Provider) is the HPC centre of the University of Stuttgart, Germany, and was the HPC provider in this experiment.

Fortissimo_SuccessStory_908_Spicetech.pdf

The Challenge

Various branches of the high-technology industry are experiencing a strong increase in the complexity of their products. In particular, for products and systems, which act autonomously in safety-critical environments, the cost for validation and certification is rising sharply. Mastering validation in complex environments, e.g. for autonomous cars in urban traffic scenarios, is one of the key challenges to advance technologies in realising affordable end-user prices. Virtual validation will be a key methodology to face this challenge, due to the fact that enormous amounts of test cases must be considered covering a huge testing and validation space.

The Solution

Test and validation aspects will have to be solved in multi-dimensional parameter spaces describing complex scenarios. The proposed approach allows the user to freely choose dimensions and parameter-spaces, which describe objects and conditions in the test scenarios. For example, for a car arriving at a road intersection, where multiple pedestrians and cars are interacting, a potential validation task would be to ensure a correct classification of all objects at the intersection. Just by positioning the cars and pedestrians and adjusting walking and driving speeds, this can easily lead to 10-15 dimensions of test space. The VALICY framework, developed in the course of this experiment, uses High-Performance Computing (HPC) and Artificial Intelligence (AI) in order to quickly scan these dimensions for object classification errors.

Business Impact

The VALICY framework offers the opportunity to integrate a highly complex virtual test and validation task into the product development process. VALICY is designed in such a way that it can be integrated into classical milestone-based development but, more interestingly, into iterative and incremental processes. Hence, high opportunities for fast test and validation feedback arise in fields, where software and hardware development move forward together.

The proposed approach samples high-dimensional test spaces efficiently, due to distributed parallel computations. Furthermore, the Machine Learning (ML) component ensures that after a short period of adjustments to identify critical cases, only the scenarios that ML expects to be of high relevance are tested and validated.

The VALICY framework is valuable for industry sectors that struggle, not only with high system complexity but also with new norms and regulations. Appropriately designed test and validation spaces can be created to support early and rapid knowledge of the certification process over a broad range of variations.

Benefits

Typical gains for end-users of the VALICY framework include:

  • Parameter variation runs can be provided for at least 10-15 testing dimensions.
  • Thousands of validation runs can be provided within one hour by using thousands of processors.
  • VALICY allows validation within a budget of a few thousands, instead of hundreds of thousands of euros.

Organizations Involved

End User & ISV: Spicetech GmbH
HPC Expert: XLAB d.o.o.
HPC Provider: HLRS