The perceived quality of a software product depends strongly on field failures viz., defects experienced by users after the software is released to the field. Software managers work to constantly improve quality control processes, seeking to reduce the number of field failures. Static analysis is a powerful and elegant technique that finds defects without running code, by reasoning about what the program does when executed. It has been incubating in academia and is now emerging in industry. This research asks this question: How can the performance and practical use of static analysis tools be improved ? The goal of the research is to find ways to improve the performance of static analysis tools, as well as the quality-control processes that use them. This will help commercial and open-source organizations make more effective use static analysis tools, and substantially reduce field failures.
Using historical data from several open-source and commercial exemplars, the research will retrospectively evaluate the association of field failures with static analysis warnings. The research will evaluate the impact of factors such as experience of the developer, the complexity of the code, and the type of static analysis warning on failure properties such criticality, and defect latency (time until a defect becomes a failure). A wide variety of projects will be studied, including both commercial and open-source. The resulting data will be analyzed using statistical modeling to determine the factors that influence the success of static analysis tools in preventing field failures. Some field failures may have no associated static analysis warnings. This research will identify and characterize these failures, paving the way for new static analysis research. An integrated educational initiative in this proposal is the training of undergraduates by using bug fixes as pedagogical material; undergraduates will also help annotate the corpus of field failures with information relevant to our analysis. An important byproduct of this research, is a large, diverse, annotated corpus of field failures of use to other educators and researchers in empirical software engineering, testing, and static analysis.
