Today's software systems are increasingly built from flexible combinations of components that can be configured in a multitude of different ways. For example, the popular Firefox web browser has more than a thousand configuration options. While a high degree of configurability has many benefits, it also makes testing and reasoning about configurable software a major challenge. The goals of this proposed work are 1) to develop new techniques to efficiently discover the structure of software systems' configuration spaces, and 2) to exploit that information to improve common software engineering tasks, specifically testing and program understanding. In particular, this work will allow software developers to quickly and accurately answer questions such as, What are the "right" configurations to test a system under? How can we avoid testing a system under "unnecessary" configurations? How does software configuration change as systems evolve? We expect that, as much of the software used today is configurable, the advances made in this project in understanding and testing configurable systems will have widespread benefits in the reliability and trustworthiness of critical software.
The proposed technical approach will pursue four main directions. First, the PIs will develop a representation of "effective" configuration spaces---the configurations needed to achieve a specific goal---that is both usable by software engineering tools and understandable to developers. Second, the PIs will develop novel techniques that compute the effective configuration space of a system. Third, the PIs will explore a number of software engineering applications that will make use of effective configuration space information, including configuration-aware test case selection, configuration-aware regression testing, and several program understanding tasks for configurable systems. Finally, the PIs will conduct a wide range of fundamental empirical studies that will test the research hypotheses and evaluate the proposed approach on large scale subject systems. Among others outcomes, these studies will examine the effective configurations of a range of systems; investigate how those configurations evolve over time; and determine how failures relate to system configurations. The results will impact both practice and education.
