2. Integrated Design and Test#
The Reactis tool suite provides automated validation and testing capabilities that are able to find run-time errors, assertion violations and unreachable targets in models and code, even with short Reactis Tester runs. Using these capabilities, we consider an integrated approach to design and test that enables CI/CD to be highly effective at detecting issues early in the development process.
Instead of introducing Reactis at the test stage in a development process, the automated validation and testing capabilities of Reactis can be used via CI/CD earlier in the development process, before the test stage,
at the model design stage (for Simulink) and
at the code production stage (for C).
From a practical perspective, this involves creating a model (or code) harness with Reactis for Simulink (or Reactis for C) at the time the model (or code) is produced, and ensuring the harness inputs have ranges set. This set up can also be automated, if the required information is available.
The key advantages of this are as follows:
The effectiveness of CI/CD at the model design and code production stages is significantly increased because it can detect issues that static checks typically would not, such as:
inputs that cause a run-time error or assertion failure;
unreachable targets, often indicative of a logical flaw.
These types of issue are not usually detected until the test stage of a process but can be found with a run of Reactis Tester that is short enough to be performed frequently, on any change. The effectiveness of CI/CD can be further increased by introducing Validator objectives at the design stage, to ensure that models do not violate certain properties.
At all times during the model design and code production stages, the models and code, respectively, are ready for testing activities with Reactis, enabling:
longer automated testing to be scheduled at any time;
manual testing activities to be performed.
When design or coding stages overlap with testing, an evolving design may lead to conflicts with manually created tests. Reactis’ automation can be used to reduce the scope for conflicts, by reducing the number of manually created tests required: the objective of the test team is not to manually create tests with full coverage but to create the minimal partial tests that enable Reactis to achieve full coverage automatically. These partial tests would drive the system under test into states that Reactis is not able to reach automatically. In practical terms, these partial tests would be imported by Reactis Simulator (so that they do not need to mention all inputs and outputs) to produce a test suite that is then preloaded by Reactis Tester.
Furthermore, these partial tests can be picked up by the CI/CD used in the design stage, further increasing its effectiveness, potentially achieving full coverage testing at the design stage.
This flexibility to perform design and test concurrently is especially advantageous for development processes following agile methodologies or aiming for continuous testing.
There are other automated checks that a CI/CD pipeline using Reactis could perform at the design stage. For example:
Re-run regression test suites to check for required behavior.
In summary, the automated testing and validation capabilities of Reactis enable CI/CD automation that provides immediate testing feedback to designers, helping to identify defects early, and allows testing and design activities to be interleaved.