Automated Test Data Generation for Model Transformation Testing

Abstract

Models and their automated transformations play a critical role in Model Driven Engineering (MDE). A significant challenge in testing model transformations is the automated generation of input test models. This involves generating meta-model instances that satisfy constraints defined on the meta-model which includes the constraints on metaelements and the multiplicity constraints. The problem becomes more challenging when the goal is to generate test models that cover specific paths of the transformation code - a common task in structural testing. The thesis proposes a novel search-based test model generation approach for structural testing of model transformations. The approach generates test models to achieve the desired structural coverage of the transformation code. The proposed test model generation strategy considers the constraints specified at the meta-model level and the multiplicity cardinalities of relationships between meta-elements to guide the generation of valid instances of the meta-model. The proposed strategy relies on a fitness function that utilizes the approach level and branch distance to generate instances that can cover the target branch of the transformation code. The approach proposes a number of heuristics as branch distance functions that solve model transformation predicates. A tool Model Transformation Testing Environment (Motter) is developed that automates the proposed approach. Motter takes the source and input meta-models as input and generates instances of test model that provide the required code coverage, for example, branch coverage of the model transformation code. The current implementation of the tool supports two widely used transformation languages, Atlas Transformation Language (ATL) and MOFScript. The thesis empirically evaluates the proposed approach on two transformations case studies, which are implemented in ATL and MOFScript. The case study in ATL is the popular benchmark Class2RDBMS model-to-model transformation case study, and the case study in MOFScript is a model-to-text industrial scale Real-Time Embedded Systems Test Simulation (RTES) code generator. For the empirical evaluation, four different widely search heuristics: Genetic Algorithm (GA), (1+1) Evolutionary Strategy/Algorithm (EA), Alternative Variable Method (AVM), and Random Search (RS) are tested in the comparative study. The result of the empirical evaluation shows that the proposed approach is successful in achieving the desired branch coverage for the selected transformation case studies and that the AVM significantly outperforms other algorithms. AVM has shown promising results in studies focusing on constraints solving, however it has not been used before for the generation of test cases to provide structural testing of model transformations. The result achieved by the AVM in the experiments are aligned with its previously reported performance as it successfully generates test cases and outperforms other algorithms in terms of the number of branches it can cover for both the case studies.