MUT4SLX: Mutation Testing for Simulink Models

Several experience reports illustrate that mutation testing can support a “shift-left” testing strategy for software systems coded in textual programming languages like C++. However, for graphical modeling languages like Simulink, the relevant tools are missing.

MUT4SLX bridges this gap. The tool features 15 mutation operators, which are modeled after realistic faults (mined from an industrial bug database) and are fast to inject (because we only replace parameter values within blocks).

MUT4SLX is a toolbox for mutation testing of Simulink Models.

MUT4SLX comprises two main components: (i) mutant generation and (ii) mutant execution. The mutant generation component is responsible for reading relevant parameters from the configuration and loading the model, as well as identifying the possible mutants. Users can use the CSV output generated by this component to preview the number of mutants, ignore certain mutants, or prioritize specific mutants before mutant execution takes place. The mutant execution component executes the given test cases on each of the mutants to count the number of killed and live mutants. Currently, two test engines are supported: Signal Builder and Simulink Test.

MUT4SLX was validated on a Helicopter Control System, a representative example built following the guidelines set out in ARP4754A, DO-178C, and DO-331 certifications. The validation shows that MUT4SLX is capable of injecting 70 mutants in less than a second, resulting in a total analysis time of 8.14 hours.

The tool accesses the Simulink API via Python and uses a series of public domain Python libraries for generating the test coverage reports.

MATLAB version R2021b Python version 3.9.16 Libraries: matlab.engine

Test engineers working with automated build pipelines. Test engineers working on safety-critical systems are especially relevant.

Type of protection: GNU GENERAL PUBLIC LICENSE (Version 3, 29 June 2007) https://github.com/haliliceylan/MUT4SLX/

The tool is released under a GNU PUBLIC LICENSE. It guarantees freedom to share and change all versions of a program provided that you grant the same license.