4. Configuring a GitLab Pipeline#
A GitLab pipeline defines the jobs for CI/CD, and dependencies between the jobs.
A pipeline is configured by a file (typically) called .gitlab-ci.yml and
located at the root of the repository containing the project under test. Pipeline
configuration offers considerable flexibility - jobs involving Reactis could
be integrated in many ways, largely determined by the development process of a
project. We consider how Reactis can be used with pipeline features.
4.1. Pipeline Types#
There are various types of pipeline, triggered by different events. Pipelines that are triggered by changes to the repository, e.g. a Branch pipeline that is triggered by a commit, would typically use Reactis to perform quick checks. Of the checks described in the previous section, at the very least we suggest consistency checks be performed on every commit. Additionally:
static analysis can often quickly flag unreachable coverage targets,
short Tester runs can flag runtime errors or assertion violations.
These quick checks can provide significant benefits during development because subtle errors can be detected early.
Scheduled pipelines are triggered at particular times. These can be used to perform longer runs of Reactis Tester when resources are not otherwise in use, e.g. at night and weekends, allowing higher coverage of targets to be achieved. As models and code become more mature, high levels of coverage give higher assurance of the absence of runtime errors and assertion violations.
4.2. Pipeline Stages and Jobs#
A GitLab pipeline can be divided into a sequence of stages. Each stage can contain one or more jobs and every job in a stage must complete before the next stage begins. Effective use of pipeline stages requires an understanding of the intermediate files Reactis creates and what tools are used at each stage.
To run Simulator or Tester on a model, Reactis for Simulink requires a model
harness to be defined in a .rsi file. The harness identifies a subsystem in
the model (which may be the top-level system) and its inputs and outputs. A
.rsi file may contain multiple harnesses but only one is selected, which is
the active harness.
When Simulator or Tester is run on a model, the following steps are performed first:
Model translation
Reactis translates the model hierarchy under the active harness to an intermediate representation. The result is cached in a
.mwifile (if enabled in the.rsifile). If an up-to-date.mwifile already exists, this step does not need to be performed. This translation step temporarily invokes a separate instance of MATLAB. After translation, the separate MATLAB instance is not required and is closed.Static analysis
Reactis performs reachability analysis of certain types of coverage target using a solver to find unreachable targets, i.e. targets that are impossible to cover. The result is cached in a
.mwibfile (if enabled in the.rsifile). If an up-to-date.mwibfile already exists, this step does not need to be performed. When performing reachability analysis, Reactis Simulator has a 30 second timeout and Reactis Tester has a 120 second timeout, but these timeouts are not usually reached, except for very large models.
For CI/CD, it is most effective to run each step for all impacted
models before running subsequent steps,
to ensure runners do not duplicate this effort in different jobs.
This requires the .mwi files and .mwib files to be listed as artifacts
in the GitLab jobs for steps 1 and 2 above, respectively.
Furthermore, it is preferable to perform all model translation as a single job. This way, only one runner needs to invoke a separate instance of MATLAB, and is able to reuse that instance to translate all models. Normally Reactis closes the separate instance of MATLAB after a model has been translated, and invokes a new instance if another model needs to be translated. However, the time taken to invoke MATLAB and start Simulink is much greater than the time taken to load a model in Simulink and translate it (except for very large models) so the most effective use of resources is to do all translation reusing the same separate MATLAB instance. To do this, the script should set:
the method Reactis uses to invoke MATLAB to either Full MATLAB window, or Full MATLAB window (hidden);
the idle timeout to a non-zero number of seconds to ensure the separate MATLAB instance remains open for re-use.
The timeout should be sufficient to keep MATLAB open longer than the time
between generating one .mwi file and starting the next.
For this, 20 seconds is probably enough. These settings can be
configured from the Reactis GUI (File > Global Settings > MATLAB) or
with the following calls to the Reactis API.
oldMatlabEngineType = rsGetParameterValue('MatlabEngineType');
oldIdleTimeout = rsGetParameterValue('MatlabIdleTimeout');
rsSetParameterValue('MatlabEngineType', 'socket');
rsSetParameterValue('MatlabIdleTimeout', '20.0');
...
rsSetParameterValue('MatlabEngineType', oldMatlabEngineType);
rsSetParameterValue('MatlabIdleTimeout', oldIdleTimeout);
4.2.1. Job Artifacts#
Job artifacts are files that are considered outputs of a job, and are available after the job has finished. Files marked as job artifacts are available to other jobs at later stages and available to download, e.g. via a URL in the GitLab interface.
The (.mwi and .mwib) cache files generated by Reactis should be
listed as job artifacts so that jobs at later stages do not need
to repeat model translation and static analysis.
The (.rst) test suite files generated by Reactis Tester should be
listed as job artifacts so that, in the event of a failure,
the test suite can be downloaded and re-run locally to reproduce the failure.
Also, reports generated by running Simulator should be listed
as job artifacts, so that they can be downloaded.