Tackling combinatorial explosion: a study of industrial needs and practices for analyzing highly configurable systems

Abstract

Hundreds of dedicated analysis techniques for highly configurable systems have been conceived, many of them able to analyze properties for all possible system configurations. Unfortunately, it is largely unknown whether these techniques are adopted in practice, whether they address actual needs, or which strategies practitioners apply. We present a study [MNM+18] of analysis practices and needs in industry based on surveys and interviews. We confirm that properties considered in the literature (e.g., reliability) are relevant and that consistency between variability models and artifacts is critical, but that the majority of analyses for specifications of configuration options (a.k.a., variability model analysis) is not perceived as needed. We identified pragmatic analysis strategies, including practices to avoid the need for analysis. We discuss analyses that are missing and synthesize our insights into suggestions for future research.