Automated Testing and Validation with Reactis®

 
 Reactis User's Guide   Contents  |  Index
 Chapters:  1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | A | B | C

Chapter 17  Reactis for EML Plugin

The V2015 release of Reactis included a new product the Reactis for EML Plugin, which integrates with Reactis to offer white-box testing of the Embedded MATLAB (EML) portions of a model. When using the plugin, Reactis Tester will generate tests which attempt to exercise targets in EML code and Reactis Simulator will track and display the coverage status of EML code. The V2016 release of Reactis extends Simulator functionality to include basic debugging of EML code, including breakpoints, single-stepping, and the ability to view variable values.

Note that if not using the Reactis for EML Plugin, you can still test models with EML code, but the EML portions of a model will be treated as block boxes. They will be executed but coverage within EML code not be tracked or targeted by generated tests, and EML code will not be visible within Reactis Simulator.

EML can be introduced into a model in several ways:
 Simulink
   •  MATLAB Function block
   •  Truth Table block
 Stateflow
   •  MATLAB functions
   •  Truth Table functions with MATLAB language option
Note that a subset of the full Embedded MATLAB language is supported. To use models containing EML in Reactis, your model must only use the language subset supported by Reactis. Note that the same language restrictions apply no matter which of the four model constructs listed above are used to incorporate EML into your model. To avoid confusion we use the following abbreviations in the following discussion:

Embedded MATLAB (EML)
the subset of MATLAB supported by MathWorks for code generation.
Reactis Embedded MATLAB (REML)
the subset of EML supported by Reactis.

REML is under active development toward the ultimate goal of supporting a very large subset of EML. Please send requests to Reactive Systems (help@reactive-systems.com) if there are unsupported EML features you would like use.

Whether in white-box or black-box mode for EML, Reactis supports the same subset of EML. This subset is defined in Section 17.3.

17.1  Enabling the Reactis for EML Plugin

Enable the plugin by selecting File -> Global Settings..., selecting the Reactis for EML tab, and then in the resulting dialog (Figure 17.1) select Enable white-box analysis of Embedded MATLAB code (requires license for EML Plugin).


images/emlSettings_web.png
Figure 17.1: The Settings dialog with tab Reactis for EML selected.

The other controls in this tab let you toggle the display of line numbers in the main panel, set the background color of the line number bar, and toggle and set the color for drawing a grid (alternating the background color of each line) in the main panel.

As shown in Figure 17.2, after enabling the EML Plugin, the EML code contained in a model element will be displayed in the Reactis main panel when the element is selected in the hierarchy panel. Since Simulink stores the EML code within the model’s .slx file, no special configuration is required within Reactis. Reactis automatically extracts the EML code from the .slx file.


images/cruise_eml_web.png
Figure 17.2: In the hierarchy panel, the names of MATLAB Function blocks are prefixed with the red ML symbol and when such a node in the hierarchy is selected the block’s EML code is displayed in the main panel.

17.2  EML Coverage Metrics

Reactis uses a number of different coverage metrics to measure how thoroughly a test or set of tests exercises a model. In general, coverage metrics record how many of a given class of coverage targets (model features such as Stateflow states, EML program statements, etc.) have been executed at least once. Coverage metrics may be visualized using Simulator and are central to test generation and model validation using Tester and Validator.

Chapter 6 describes the coverage metrics that are tracked within the Simulink and Stateflow portions of models. Those metrics include some which are specific to Simulink, some which appear only in Stateflow, and others which are generic and can appear in either Simulink or Stateflow. Three of the generic metrics are also tracked in the EML code portions of models by the Reactis for EML Plugin. These are decision coverage, condition coverage, and modified condition / decision coverage (MC/DC).

Decision coverage tracks whether each decision in a program has evaluated to both true and false. The program elements that the Reactis for EML Plugin identifies as decisions are the conditional expressions in if statements and while loops.

Condition coverage tracks whether each condition (atomic Boolean expression) in a decision has evaluated to both true and false.

For the definition of MC/DC coverage, please see Chapter 6.

Finally, in addition to these generic coverage metrics, the Reactis for EML Plugin also tracks statement coverage – whether or not each EML statement has been executed at least once.

17.2.1  Tracking Coverage in EML Code

In Reactis Simulator, when Coverage -> Show Details is selected, unexercised coverage targets in EML code are reported visually as shown in Figure 17.3. Any unexecuted EML statement is rendered in red. If a decision has not evaluated to true it has a thin red overline. If a decision has not evaluated to false it has a thin red underline. If a condition has not evaluated to true it has a thick red overline. If a condition has not evaluated to false it has a thick red underline. If a decision has not met the MC/DC criteria, then the text of the decision is displayed in red. The MC/DC-related coverage details associated with a decision (Figure 17.4) may be displayed by right clicking on the decision and selecting View Coverage Details. For a description of this dialog, please see Chapter 6.

Lines containing unexercised targets are distinguished by a a thin red bar which is drawn just to the right of the line number of that line.


images/emlpCvgHighlight_web.png
Figure 17.3: Coverage highlighting in EML code.


images/emlpCvgDetails_web.png
Figure 17.4: View the MC/DC-related coverage details associated with a decision by right-clicking on the decision and selecting View Coverage Details.

17.2.2  Debugging EML Code


images/emlDebug_web.png
Figure 17.5: Debugging EML Code in Simulator.

When the EML Plugin is enabled, Simulator will step seamlessly into EML code, as shown in Figure 17.5. The numbered items in Figure 17.5 are the primary debugging operations supported by Simulator. These operations are:

  1. Breakpoints. A breakpoint may be toggled on lines within EML code which contain an executable statement by double-clicking on the line number, or by right-clicking on the line number and selecting Toggle Breakpoint.
  2. Single Stepping. EML code may be stepped through one statement at a time by clicking on the mini-step (  images/miniStepBtn_web.png  ) button or by selecting Simulate -> Step Into.
  3. Hovering. When execution is paused within EML code, hovering on a variable causes the value of the variable to be displayed. If execution has not yet passed the point where the variable was initialized, no value is displayed.

    When execution is stopped between simulation steps, hovering on a coverage target in EML code causes the coverage status to be displayed.

  4. Watched Variables. When execution is paused within EML code and a variable is active, the variable may be added to the watched variable panel by right clicking on the name of the variable and selecting Add to Watched.

17.3  Subset of Embedded MATLAB Supported by Reactis

This section describes the subset of EML that constitutes REML. REML supports the following aspects of EML. Note that in some cases not all aspects of a feature are supported. For example, a function might not support all numbers and types of input arguments.

17.3.1  Syntax

While REML generally supports the rich EML notation for defining matrices, REML does impose some syntax restrictions. The most prominent restriction is that if any element of a row is a non-trivial expression, then the row must be delimited with commas instead of spaces. For example, if a = 5 and b = 4 then [ a + b -4 -3 ] is a valid EML matrix which evaluates to [ 9 -4 -3 ]. The REML syntax does not allow this expression. Instead it requires it to be written [ a + b, -4, -3 ].

17.3.2  Types

  • double, single, uint8, int8, uint16, int16, uint32, int32, logical, string 1, enumerations 2, structures 3

17.3.3  Control flow

  • if statements
  • while loops
  • for loops
  • switch statement
  • break statement
  • continue statement
  • return statement

17.3.4  User-defined functions and subfunctions

  • local and persistent variables

17.3.5  Array indexing

  • scalar indexing, e.g. x(4)
  • vector indexing, e.g. x(2:5) or x(2:end) or x(:,3)

17.3.6  Logical operators

  • &, |, &&, ||, ~
  • and, or, xor, not, any, all

17.3.7  Relational operators

  • <, <=, ==, ~=, >=, >

17.3.8  Math operators

  • +, -, *, /, ^

17.3.9  Math functions

  • sin, cos, tan
  • asin, acos, atan, atan2
  • exp, log, log10
  • sinh, cosh, tanh
  • sqrt
  • fix, floor, ceil, round
  • mod, rem
  • sign, abs

17.3.10  Statistical functions

These functions support vector and matrix inputs, but not higher dimensional arrays.

  • mean, var, std

17.3.11  Matrix operations

  • transpose ( ' )
  • addition, subtraction
  • multiplication, division
  • inv
  • zeros
  • ones
  • eye
  • diag

17.3.12  Element-wise array operations

  • .*, ./, .\, .^

17.3.13  Vector operations

  • sum, prod, dot, size, min, max, length

17.3.14  Bit operation

  • bitset, bitget, bitand, bitor, bitxor, bitcmp, bitshift

17.3.15  Cast operations

  • double, single, uint8, int8, uint16, int16, uint32, int32, logical

17.3.16  String operations

  • strcmp

17.3.17  Miscellaneous functions

  • class
  • true, false
  • intmin, intmax
  • isempty, isequal, isfinite, isinf, isnan
  • struct

1
Support for strings is currently very limited.
2
defined by Simulink.IntEnumType classes
3
Simulink.Bus inputs or outputs