Chapter 10 The Reactis Coverage-Report Browser
The Coverage-Report Browser enables you to view detailed coverage
information related to testing and to export reports in HTML
format for viewing and printing within a web browser.
Figure 10.1 contains an annotated screen shot of the
Coverage-Report Browser. The title bar includes information describing
the test suite, tests, and test steps that were exercised before
computing the coverage information. Here, the report describes
coverage attained after executing steps 1 through 80 of test 4 in
test suite cruise.rst
. When a report reflects the coverage
attained by executing an entire test suite, only the test suite name
appears in the title bar.
Figure 10.1: The Coverage-Report Browser. |
10.1 Labeled Window Items
-
The model hierarchy panel works much as the hierarchy panel
does in the top-level Reactis window. Namely, it displays the
subsystems that make up the model and how they are related.
Clicking on a name causes coverage
information for the subsystem to be displayed in the panels
to the right. Double-clicking on a name causes the subsystem
diagram to be displayed in the Simulator main panel.
- This panel displays summary coverage statistics for the
subsystem currently selected in the hierarchy panel.
- This panel shows coverage information for each
child subsystem of the subsystem currently selected in the
hierarchy panel. The information is presented as a matrix and
includes a row for each child. The first column includes the
names of the children, while each subsequent column includes
statistics for one of the coverage measures tracked by
Reactis. In Figure 10.1, the entry “(6)
50%” indicates that child subsystem DesiredSpeed
has 6 branches, 50% of were covered by steps 1 through 80 of test 4.
Double-clicking on a row causes the child to become the current
subsystem displayed in the Coverage-Report Browser.
The rows of the table may be sorted based on the values of any
column. To cause the values in a particular column to order
the rows, simply click on the header of the column.
Clicking once sorts in increasing order of the column entries,
while clicking a second time sorts in decreasing order.
- This panel displays which specific targets within the currently
selected subsystem are covered, uncovered or unreachable.
The way this information is conveyed differs between coverage
metrics:
-
Decision, Condition, MC/DC
-
For targets in the MC/DC-related metrics (decision, condition, MC/DC)
the list includes one entry of type MC/DC for each decision.
The status column shows covered when all decision, condition and MC/DC targets
associated with the decision are covered. Right clicking on such an entry
will activate a pop-up menu that includes the menu item Show Details.
When this menu item is selected a dialog conveying coverage information similar
to that shown in Figure 7.16 will appear. See Section
7.6.2 for a description of this dialog.
- MCC
-
The list contains one entry of type “MCC” for each decision in the subsystem.
The status column of that entry shows the percentage of MCC targets covered
for that decision. Right clicking on such an entry
will activate a pop-up menu that includes the menu item Show Details.
When this menu item is selected a dialog conveying coverage information similar
to that shown in Figure 7.17 will appear. See Section
7.6.2 for a description of this dialog.
- Lookup Table
-
The list contains one entry of type “Lookup” for each lookup table in the subsystem.
The status column of that entry shows the percentage of targets covered
within that table. Right clicking on such an entry
will activate a pop-up menu that includes the menu item Show Details.
When this menu item is selected a dialog conveying coverage information similar
to that shown in Figure 6.10 or Figure 6.11
will appear. See Section 6.1.3 for a description of this dialog.
-
For all other targets in the subsystem, this panel displays
“covered”, “uncovered” or “unreachable” depending on the coverage
status of the target.
The rows of the table may be sorted based on the values of any
column. To cause the values in a particular column to order
the rows, simply click on the header of the column.
Clicking once sorts in increasing order of the column entries,
while clicking a second time sorts in decreasing order.
-
Report menu.
- The Report menu contains the following entries.
- Export...
- Invoke the dialog shown in
Figure 10.2 that enables the user to export the
coverage report in HTML format. See Section 10.3
for details.
- Exit.
- Exit the Coverage-Report Browser.
- Help menu.
- The Help menu contains the following entries.
-
Contents.
- Go to the table of contents in the
documentation.
- Index.
- Go to the index in the documentation.
- Coverage-Report Browser.
- Display Coverage-Report Browser help.
10.3 Exporting Coverage Reports
The Coverage-Report Browser includes a facility that enables you to export
coverage reports in HTML format. The resulting files may be viewed
and printed using facilities in a web browser. The dialog shown in
Figure 10.2 is invoked by selecting the
Report > Export... menu item from the Coverage-Report Browser
menu bar. Coverage details in the exported reports are organized so
that each subsystem of the model has a section in the report; a subsystem’s
section includes coverage information for the targets in that subsystem.
Figure 10.2: The dialog for exporting coverage reports in
HTML format. |
The labeled items in the dialog are used as follows.
-
The radio button selected from this group specifies the portion
of the model which will be included in the exported report.
- When this check box is selected, a table of contents containing a link
to each section will be included at the top the exported report.
- When this check box is selected, the date the report was generated is
included at the top of the report.
- When this check box is selected, the full operating system paths to the
relevant .slx file and .rst file are included at the top of the report.
The .slx file contains the model from which the report was generated.
If the report conveys statistics after tests from a test suite have
been executed, then the .rst file named is the file in which these
tests reside.
- When this check box is selected, the list of excluded targets is included in the report.
- The Coverage Export Options panel can be
used to select which coverage metrics are
included in the coverage report.
There are three choices for each metric:
-
Summary & Details.
Targets of the metric will appear in both the coverage summary and coverage details
sections of the report.
- Summary Only.
Targets of the metric will appear in the coverage summary only.
- None.
Targets of the metric will be omitted from the report entirely.
Note that due to dependencies
between metrics, some combinations are not allowed. For example,
Summary & Details cannot be selected for Condition targets
unless Summary & Details is also selected for decision targets.
- Click this button to display help for the coverage export.
- Click this button to preview the report to be exported.
- Click this button to export the report. A file selection dialog
will appear to allow you to name the file where the report should
be written.
- Click this button to abort report generation.
10.4 Coverage Report Contents
Based on the selections in the Export to HTML dialog, a report
will be generated with different elements. An example report is shown in
Figure 10.3 that was generated using the export options
shown in Figure 10.2. For each subsystem of the model,
the report will include a summary section (coverage statistics for the
subsystem) and details section (detailed information about the coverage of
each target in the subsystem). The Export to HTML dialog lets
you specify which coverage metrics should be included in the summary and
detail sections.
Figure 10.3: Example exported HTML report |
When using the Reactis for C Plugin or the Reactis for EML Plugin, Reactis tracks coverage targets
within the C or EML code incorporated into a model. In this case,
HTML coverage reports will include source code annotated to convey
coverage information as shown in Figure 10.4.
Lines with uncovered statements are printed in red. A yellow status
column to the left of the code and to the right of the line numbers
displays additional coverage information.
The left portion of the status column gives the coverage status of each
statement in the line. Each statement in the line will have an indicator
that is either a red S
(meaning uncovered statement) or
test/step
giving the test and step in which the statement was first
exercised. If a line contains more than one statement, then a list of
indicators will be separated by commas. A blue indicator S:excluded
means the statement has been excluded from coverage tracking.
The right portion of the status column gives the coverage status for the
decision-related targets in the line. The aggregate indicator takes the form
D:xxx
. The D
symbol is used to convey that the information
relates to a decision. The first position after the colon represents
decision coverage, the second condition coverage, and the third MC/DC.
Each x
can be:
-
-
- One or more target is not covered.
- +
- All targets are covered.
- e
- One or more targets are excluded and all non-excluded targets are covered.
Each aggregate decision indicator is a hyperlink that, when clicked, takes you
to the details table for the decision.
Figure 10.4: Code coverage symbols in HTML report |
The indicators in Figure 10.4 can be interpreted as follows:
-
Line 20
-
D:---
means at least one decision, one condition, and one MC/DC
target remain uncovered.
- Line 16
-
D:+++
means all decision, condition and MC/DC targets have been covered.
- Line 18
-
D:+--
means both decision targets have been covered, but at least
one condition and one MC/DC target remain uncovered.