- Stefan Fischer
- Rudolf Ramler
- Claus Klammer
- Rick Rabiser
This is the online appendix for an industry study of testing highly configurable cyber-physical systems. The appendix includes the questionnaire that was answered by 19 practitioners in our study, and the interview guide, which was used in semi-structured interviews with 13 practitioners.
Cyber-physical systems, i.e., systems that seamlessly integrate computation and physical components, are commonly realized as highly-configurable systems. Testing such systems is particularly challenging because they typically comprise a large number of heterogeneous components that can be configured and combined in different ways. Despite a plethora of work investigating software product line testing in particular and software testing in general, variability in tests and how industry does actually manage testing highly configurable cyber-physical systems is not well understood. In this paper, we report the results of a multiple case study we conducted with three companies developing and maintaining highly-configurable cyber-physical systems focusing on their testing practices, with a particular focus on how they manage variability in tests. We conclude that experienced-based selection of configurations for testing is currently predominant. Variability modeling techniques are not utilized and the dependencies between configuration options are only partially modeled at best. However, the companies are aware of the situation and have the need and desire to cover more configuration combinations by automated tests. This in turn raises many questions, which might also be of interest to the scientific community and motivate future research.