11. The Reactis Coverage Report Browser#

The Coverage-Report Browser lets you view detailed coverage information related to testing and can also export reports in HTML format for viewing and printing within a web browser. Figure 11.1 contains an annotated screen shot of the Coverage-Report Browser. The title bar includes information describing the test suite, tests, and test steps that were exercised before computing the coverage information and assertion violations. In Figure 11.1, the report describes the coverage attained after executing steps 1 through 25 of test 4 in test suite cruise.rst. When a report reflects the coverage attained by executing an entire test suite, only the test suite name appears in the title bar.

_images/cvgReportA.png

Fig. 11.1 The Coverage Report Browser.#

11.1. Labeled Window Items#

  1. The model hierarchy panel works much as the hierarchy panel does in the top-level Reactis window. In particular, it displays the subsystems that make up the model and how they are related. Clicking on a name causes coverage information for the subsystem to be displayed in the panels to the right. Double-clicking on a name causes the subsystem diagram to be displayed in the Simulator main panel.

  2. This panel displays summary coverage statistics for the subsystem currently selected in the hierarchy panel.

  3. This panel displays summary assertion violation statistics for the subsystem currently selected in the hierarchy panel.

  4. This panel shows coverage information for each child subsystem of the subsystem currently selected in the hierarchy panel. The information is presented as a matrix and includes a row for each child. The first column includes the names of the children, while each subsequent column includes statistics for one of the coverage measures tracked by Reactis. Figure 11.1, the entry “(6) 33%” indicates that child subsystem DesiredSpeed has 6 branches, 33% of were covered by steps 1 through 25 of test 4. Double-clicking on a row causes the child to become the current subsystem displayed in the Coverage-Report Browser.

    The rows of the table may be sorted based on the values of any column. To cause the values in a particular column to order the rows, simply click on the header of the column. Clicking once sorts in increasing order of the column entries, while clicking a second time sorts in decreasing order.

  5. This panel displays which specific targets within the currently selected subsystem are covered, uncovered or unreachable. The way this information is conveyed differs between coverage metrics:

    Decision, Condition, MC/DC

    For targets in the MC/DC-related metrics (decision, condition, MC/DC) the list includes one entry of type MC/DC for each decision. The status column shows covered when all decision, condition and MC/DC targets associated with the decision are covered. Right clicking on such an entry will activate a pop-up menu that includes the menu item Show Details. When this menu item is selected a dialog conveying coverage information similar to that shown in Figure 8.16 will appear. See the Coverage Information in the Main Panel section for a description of this dialog.

    MCC

    The list contains one entry of type “MCC” for each decision in the subsystem. The status column of that entry shows the percentage of MCC targets covered for that decision. Right clicking on such an entry will activate a pop-up menu that includes the menu item Show Details. When this menu item is selected a dialog conveying coverage information similar to that shown in Figure 8.17 will appear. See the Main Panel](section-coverage-info-main-panel) section for a description of this dialog.

    Lookup Table

    The list contains one entry of type “Lookup” for each lookup table in the subsystem. The status column of that entry shows the percentage of targets covered within that table. Right clicking on such an entry will activate a pop-up menu that includes the menu item Show Details. When this menu item is selected a dialog conveying coverage information similar to that shown in Figure 6.10 or Figure 6.11 will appear. See the Lookup Table Coverage section for a description of this dialog. For all other targets in the subsystem, this panel displays “covered”, “uncovered” or “unreachable” depending on the coverage status of the target.

    The rows of the table may be sorted based on the values of any column. To cause the values in a particular column to order the rows, simply click on the header of the column. Clicking once sorts in increasing order of the column entries, while clicking a second time sorts in decreasing order.

11.3. Exporting Coverage Reports#

The Coverage-Report Browser includes a facility that enables you to export coverage reports in HTML format. The resulting files may be viewed and printed using facilities in a web browser. The dialog shown in Figure 11.2 is invoked by selecting the Report > Export… menu item from the Coverage-Report Browser menu bar. Coverage details in the exported reports are organized so that each subsystem of the model has a section in the report; a subsystem’s section includes coverage information for the targets in that subsystem.

_images/exportCvgRptA.png

Fig. 11.2 The dialog for exporting coverage reports in HTML format.#

The labeled items in the dialog are used as follows.

  1. The radio button selected from this group specifies the portion of the model which will be included in the exported report.

  2. When this check box is selected, a table of contents containing a link to each section will be included at the top the exported report.

  3. When this check box is selected, the date the report was generated is included at the top of the report.

  4. When this check box is selected, the full operating system paths to the relevant .slx file and .rst file are included at the top of the report. The .slx file contains the model from which the report was generated. If the report conveys statistics after tests from a test suite have been executed, then the .rst file named is the file in which these tests reside.

  5. When this check box is selected, the list of excluded targets is included in the report.

  6. The Coverage Export Options panel can be used to select which coverage metrics are included in the coverage report. There are three choices for each metric:

    • Summary & Details. Targets of the metric will appear in both the coverage summary and coverage details sections of the report.

    • Summary Only. Targets of the metric will appear in the coverage summary only.

    • None. Targets of the metric will be omitted from the report entirely.

    Note that due to dependencies between metrics, some combinations are not allowed. For example, Summary & Details cannot be selected for Condition targets unless Summary & Details is also selected for decision targets.

  7. Click this button to display help for the coverage export.

  8. Click this button to preview the report to be exported.

  9. Click this button to export the report. A file selection dialog will appear to allow you to name the file where the report should be written.

  10. Click this button to abort report generation.

11.4. Coverage Report Contents#

Based on the selections in the Export to HTML dialog, a report will be generated with different elements. An example report is shown in Figure 11.3 that was generated using the export options shown in Figure 11.2. For each subsystem of the model, the report will include a summary section (coverage statistics for the subsystem) and details section (detailed information about the coverage of each target and assertion violations in the subsystem). The Export to HTML dialog lets you specify which coverage metrics should be included in the summary and detail sections.

_images/exampleHTMLRep.png

Fig. 11.3 Example exported HTML report#

When using the Reactis for C Plugin or the Reactis for EML Plugin, Reactis tracks coverage targets within the C or EML code incorporated into a model. In this case, HTML coverage reports will include source code annotated to convey coverage information as shown in Figure 11.4. Lines with uncovered statements are printed in red. A yellow status column to the left of the code and to the right of the line numbers displays additional coverage information.

The left portion of the status column gives the coverage status of each statement in the line. Each statement in the line will have an indicator that is either a red S (meaning uncovered statement) or test/step giving the test and step in which the statement was first exercised. If a line contains more than one statement, then a list of indicators will be separated by commas. A blue indicator S:excluded means the statement has been excluded from coverage tracking.

The right portion of the status column gives the coverage status for the decision-related targets in the line. The aggregate indicator takes the form D:xxx. The D symbol is used to convey that the information relates to a decision. The first position after the colon represents decision coverage, the second condition coverage, and the third MC/DC. Each x can be:

−

One or more target is not covered.

+

All targets are covered.

e

One or more targets are excluded and all non-excluded targets are covered.

Each aggregate decision indicator is a hyperlink that, when clicked, takes you to the details table for the decision.

_images/codeexampleHTMLCov.png

Fig. 11.4 Code coverage symbols in HTML report#

The indicators in Figure 11.4 can be interpreted as follows:

Line 20

D:--- means at least one decision, one condition, and one MC/DC target remain uncovered.

Line 16

D:+++ means all decision, condition and MC/DC targets have been covered.

Line 18

D:+-- means both decision targets have been covered, but at least one condition and one MC/DC target remain uncovered.