Chapter 13 Maximizing Coverage
Reactis Tester and Validator work hard to generate tests that maximize covered targets. Nevertheless, you may encounter models that present challenges to Reactis: generated test suites may not provide adequate coverage. This chapter discusses strategies for overcoming such situations.
When attempting to improve coverage, the following general strategy is often useful:
13.1 Constraining Input Values
In many cases coverage can be improved if tighter bounds are imposed on input values by inport constraints in the .rsi file. First, any range information for inputs should be entered. For example, for vehicle speed, rather than allow any double precision floating point value, we narrow the set of acceptable value to between 0 and 200. This greatly reduces the set of values for this input that Reactis must consider when generating tests. In some cases the allowable set of inputs can be reduced even further to a set of enumerated values. For example, a flag may be restricted to the set including only 0 and 1. In general, the goal is to reduce the number of possible values Reactis must consider when generating tests.
Probability weights may also be used to affect the likelihood with which different conditions become true. For example, in some models, large portions of the model become unreachable if an input flag has a certain value (e.g. this input might indicate the system being modeled should shut down). Assigning low probabilities to those values can increase coverage. Chapter 5 describes how to use the Reactis Info File Editor to set constraints for top-level inports.
When modifying the .rsi file in this fashion, it is good practice to save the modified .rsi file as a new file, so that the original settings are not overwritten. This can be done using the File -> Save Info File As... menu item in the top-level Reactis menu, as described in Chapter 5. Note that this operation automatically associates the new .rsi file with the model. The link to the previous .rsi file can be restored using the File -> Select Info File... menu item.
13.2 Modifying Test-Generation Parameters
Another approach for improving coverage is to change the parameters used by Tester and Validator. These parameters may be set in the launch screens of the respective tool and may be categorized as follows:
Certain model constructs that we term interruptible timers present a particular challenge to Reactis. These timers/counters require the application to hold some condition true for a given length of time or number of simulation steps. If the condition becomes false before the threshold is reached the timer/counter resets.
Reactis includes a mechanism called Timer Targets that enables you to identify those elements of your model which are timers or counters. When they are tagged, Reactis can often generate a test automatically to cause a timer/counter to tick up to its threshold. Please see Chapter 9 for details of how to instrument your model with a timer target directive.
The general procedure for using timer targets is as follows:
13.4 Interactively Tuning Test Suites
If the previously described techniques still do not result in a test suite with a satisfactory level of coverage, then some hand-tuning of the suite may be necessary. Reactis Simulator and Tester / Validator contain several features designed to facilitate this process, including test-suite pre-loading (Tester), and test-suite execution, breakpoints and test-suite editing (Simulator).
The general strategy to be pursued may be summarized as follows:
The idea is to help Tester cover things it finds difficult while still letting Tester do as much work as possible.
Coverage problems often arise because of difficult-to-satisfy conditions on Stateflow transition segments or within Simulink. Conditions might be difficult to satisfy for several reasons:
The strategy for coping with these problems can be summarized as follows:
The specific steps needed to implement this strategy are as follows (It is assumed that Simulator is already running and that the earlier test suite has been loaded):
The newly saved test suite may now be pre-loaded into Tester using the Preload Files selection box, and Tester rerun. This strategy may be repeated until an acceptable test suite is constructed. Be sure to save the resulting test suite so that it will not accidentally be overwritten by new runs of Tester.
If the steps described above still result in unsatisfactory coverage, consider testing subsystems of your model independently. Reactis makes this possible through a facility to extract subsystems from a model. The idea is to extract a subsystem, along with portions of the model involved in triggering the subsystem, and store the results as a new model in a separate .slx file.
The subsystem extraction utility is invoked by loading a model into Reactis, then right-clicking on the Simulink subsystem or Stateflow chart to be extracted, and selecting the Extract Subsystem entry in the pop-up menu. In the resulting dialog a filename under which to save the extracted subsystem may be specified. Reactis will then extract the subsystem, save it under its new name, open a new Reactis window, and load the extracted model. Note that the extracted subsystem is saved as a standard .slx file. That makes it easy to edit the extracted model using Simulink if changes to the triggering mechanism or inputs or outputs of the model are necessary.
If the extracted subsystem is a triggered subsystem or located within a triggered subsystem or contains triggered subsystems whose triggers are connected to something outside the extracted subsystem, Reactis will keep all blocks and charts that are part of the triggering mechanism. This works recursively, i.e. if the triggering block or chart is triggered itself, Reactis will trace back that trigger, too.
To determine which subsystems might be candidates for independent testing, you can use the following strategy: