05 Next-Gen Automotive Testing Methodologies

Over the decades, the methods and tools of testing cars have evolved just as dramatically as the vehicles themselves. From the real-world testing of the early automobile era to the sophisticated simulation methodologies of today, the journey of automotive testing is marked by innovation and continuous improvement.

Real-World Testing: The Expensive Cornerstone of Automotive Development

In the automotive industry, a critical component of validating embedded software is through Real World Testing, where the System Under Test is the Complete Vehicle. ​The primary advantage of this approach is the very accurate test results it yields, closely mirroring the vehicle’s behavior in the hands of consumers. ​This methodology is conducting tests in actual driving conditions to accurately evaluate the vehicle’s performance and the embedded systems’ responses to real-time stimuli. ​

However, this accuracy comes at a high cost. Real World Testing is very expensive due to the logistical complexities of orchestrating such extensive testing procedures, including the costs associated with vehicle operation, test personnel, and the deployment of necessary equipment. ​

Moreover, the inherent variability of real-world conditions poses a significant challenge, making it difficult to reproduce specific scenarios consistently, which is essential for thorough testing and validation. Despite these challenges, the insights gained from this method are invaluable for ensuring the safety and reliability of the vehicle’s system in actual operating environments.​

Enter the New Era: Innovative Testing Methodologies

To address the limitations of real-world testing and enhance the efficiency of automotive development, engineers have pioneered innovative testing methodologies that leverage simulation, emulation, and virtualization technologies. These methodologies offer a more controlled and cost-effective approach to testing, enabling engineers to validate vehicle performance and functionality in a simulated environment.

1/ Vehicle in the loop testing (ViL)

Following Real World Testing, the automotive industry often employs Vehicle-in-the-Loop (ViL) (the first testing methodology in the XiL testing methodology) testing as an alternative methodology for evaluating vehicle systems/components. ​

In ViL testing, the System under Test remains the Complete Vehicle; however, the test conditions are meticulously simulated within a controlled laboratory environment and the vehicle run in the equipment called Roller chassis dyno. This method strikes a balance between authenticity and controllability by creating realistic driving scenarios using advanced simulation technologies. ​

ViL testing is less expensive than its real-world counterpart, reducing the financial burden on the testing process.​ While the test results are accurate, they typically do not reach the fidelity of data collected from real-world environments, primarily due to the challenges in simulating the unpredictability of on-road conditions.​

A significant advantage of this methodology is its better reproducibility, as laboratory conditions are far easier to standardize and replicate. This consistent reproducibility makes ViL an essential part of the testing regimen, allowing for repeated validation under uniform test conditions, which is crucial for identifying and addressing potential issues within the vehicle’s systems.​

2/ Hardware in the loop testing (HiL)

Hardware-in-the-Loop (HiL) testing serves as a more focused and systemic approach in the automotive industry. The System under Test in HiL is the Electronic Control Unit (ECU), isolated from the vehicle to stand as the sole component under scrutiny. By removing other variables, engineers can simulate the ECU’s surrounding environment with precision, directing their attention exclusively to the unit’s functionality and response patterns.​

Although setting up a HiL system is an expensive endeavor, it represents a long-term investment. The hardware and software used for HiL testing are capable of being repurposed for multiple ECUs and various test scenarios. The accuracy of HiL test results is highly dependent on the fidelity of the test bench and the sophistication of the simulation models employed. These models are critical in replicating the diverse conditions the ECU may encounter within a vehicle, determining the relevance and reliability of the test outcomes.​

This reusability, coupled with the high level of control over testing variables, results in good reproducibility. Consistency in test conditions means that the HiL testing process can reliably reproduce test results, which is essential for validating the ECU’s performance and ensuring it meets stringent industry standards. The HiL methodology, by concentrating on the ECU and its direct interfaces, provides a detailed and reusable testing framework, crucial for the iterative development and verification cycles in automotive software engineering.​

4/ Processor in the loop (PiL)

Processor-in-the-Loop (PiL) testing emerges as a streamlined and efficient strategy for embedded software verification in the automotive sector. In PiL testing, the System under Test comprises the microcontroller (µC) coupled with its software (SW), presenting a focused examination of the processor’s performance as it runs the intended software.​

One of the defining attributes of PiL testing is its cost-effectiveness. The test setup is affordable, primarily because it does not require the complete vehicle or even the full ECU hardware, thus minimizing the resources needed for conducting comprehensive testing. The testing apparatus used for PiL can be reused across different development projects, which enhances the return on investment over time and supports a sustainable testing lifecycle.​

The accuracy of PiL testing outcomes is contingent upon the quality and realism of the simulation models. These models are crucial in emulating the operational parameters and external influences that the microcontroller will encounter within the automotive environment. The more sophisticated and accurate the models, the more reliable the test results will be.​

4/ Software in the loop (SiL)

Software-in-the-Loop (SiL) testing is a critical step in the automotive software development lifecycle, focusing exclusively on the software components. Whether assessing complete or partial software systems, SiL testing scrutinizes the code’s functionality without the need for physical hardware components. Instead, virtual models simulate the hardware environment of the Electronic Control Units (ECUs) that the software will interact with in operation.​

While the initial setup for SiL testing infrastructure can be quite high—owing to the development of detailed, high-fidelity models and simulation environments—the marginal cost of conducting subsequent tests is significantly lower. Once established, the simulation environment can be run with minimal additional costs, making SiL an economical choice for ongoing software testing and development.​

The precision of the test results in SiL testing is heavily reliant on the accuracy of these virtual models and the simulated environment. The models must accurately reflect the physical characteristics and constraints of the ECUs to ensure that the software will perform as expected in real-world conditions.​

Additionally, SiL testing offers perfect reproducibility. Because it is digital, no noise and The virtual nature of the test environment allows for an exact replication of conditions and scenarios, enabling developers to consistently reproduce findings and iteratively refine the software. This level of reproducibility is indispensable for debugging and verifying that modifications to the software yield the expected outcomes. Consequently, SiL testing becomes an invaluable asset in the software verification process, providing cost-effective, highly accurate, and perfectly reproducible results that drive the development of reliable automotive software systems.​

Moreover, PiL testing boasts excellent reproducibility. The controlled simulation environment allows for tests to be repeated with great precision, ensuring that any discovered issues can be consistently replicated and addressed. This level of reproducibility is critical for iterative development processes, enabling engineers to refine and optimize the software in lockstep with the microcontroller’s capabilities. Consequently, PiL testing is an indispensable tool in the automotive industry, providing an accessible, reusable, and reliable means of ensuring that the microcontroller and software operate flawlessly together.​

5/ Model in the loop (MiL)

Model-in-the-Loop (MiL) testing stands as an initial and essential phase in the validation process for automotive software development. In this approach, the System under Test is not the software itself but its model. These models represent abstracted versions of the intended software, allowing developers to validate logic, algorithms, and system behavior before actual code implementation.​

The accuracy of MiL testing is inherently dependent on the fidelity of the software models and the code generator used to convert these models into executable code. The models need to encapsulate all the necessary details of the intended software behavior to ensure that the subsequent stages of development are based on a solid foundation. The code generator must also faithfully translate the model into code that can be executed for further testing phases, such as SiL or PiL.​

MiL testing is highly cost-effective. Since testing is conducted on models within a simulation environment, there are no hardware requirements or physical prototypes involved, which significantly reduces the expenses. The tools and software used for modeling and simulation are generally inexpensive compared to the costs associated with later stages of testing that require physical components or vehicles.​

One of the most significant advantages of MiL testing is its perfect reproducibility. Since the tests are run on digital models within a simulation, the same test scenarios can be executed as many times as needed, with no variation in the environmental conditions or test setup. This allows for an extremely high degree of confidence in the consistency of the test results. Reproducibility is crucial at this stage, as it ensures that any changes to the model will yield predictable and repeatable outcomes, thereby facilitating a smooth transition to more advanced stages of testing and development. The perfect reproducibility of MiL testing makes it an invaluable tool in the early stages of automotive software development, providing a reliable and economical means to refine system models before committing to the complexity of working with actual software and hardware.​

—

Vietsol’s cutting-edge Testing and Validation offers full coverage: Component, sub-system, system, vehicle level across all domains: Infotainment, body, Chassis, e-Powertrain and charging, ADAS, Network & Diagnostic, functional safety, Cybersecurity. Backed by esteemed partners like Vector Informatik and dSpace, our technical proficiency ensures accelerated, secure and cost-effective outcomes. Collaborate with our team for unparalleled results!