The objective of this EAGER proposal is to develop Adaptive Regression Testing (ART) Strategies: strategies that can guide software test engineers in selecting appropriate regression testing techniques to use on new versions of their software systems as those systems evolve. As an initial approach to creating ART strategies, the research will investigate Analytical Hierarchy Process (AHP) methods while implementing a prototype tool to support the process. Controlled experiment to evaluate the use of their ART strategies will be conducted. The work is advancing knowledge and understanding by providing new strategies for regression testing software systems over their lifetimes, empirical data about those strategies, and empirical approaches that can be used by other researchers to make further progress in this area. For further information see the project web site at URL http://cs.ndsu.edu/~hdo/projects/art-strategy.html
Regression testing is an important and necessary activity that can maintain the quality of modified software systems. Such regression testing plays an integral role in maintaining the quality of subsequent releases of software, but it is also expensive, accounting for a large proportion of the costs of software. Thus, to improve the cost-effectiveness of regression testing, many regression testing techniques have been proposed. While this research has made considerable progress in regression testing areas, one important problem has been overlooked. As systems evolve, the types of maintenance activities that are applied to them change. Differences between versions can involve different amounts and types of code modifications, and these changes can affect the costs and benefits of regression testing techniques in different ways. Thus, there may be no single regression testing technique that is the most cost-effective technique to use on every version. To address this problem, we developed Adaptive Regression Testing (ART) strategies and evaluated them for different context factors -- characteristics of the environments and engineering processes within which specific regression testing techniques are employed -- through rigorous empirical approaches. Under support of this grant, we engaged in several activities directed at the grant's objectives and produced the following outcomes. o Intellectual Merit * Developed an adaptive regression testing strategy that can select the most cost-effective alternative among regression testing techniques for a specific version considering software companies' situations and feedback from prior regression testing sessions. * Using the strategy, we performed empirical studies, and the empirical studies showed the following results: - The prioritization techniques selected by our strategy across the entire system lifetime can be more cost-effective than those used by the control approaches with the exception of some cases. - Adopting different types of test case prioritization techniques considering factors related to software artifacts is potentially a practical approach for organizations who have time pressure with the product release, due to the constraint budgetary problem and competitive software market. - Our strategy can assist researchers and practitioners in choosing cost-effective techniques across system lifetime. * To our knowledge, our studies are the first attempt to investigate the effectiveness of adaptive regression testing strategy. Our proposed strategy produced promising results, and we believe that our empirical methodology and findings from our study provide insights into how such investigation can be performed and what types of multiple criteria decision making approaches and evaluation criteria can be considered. o Broader Impacts * Contributions to human resource development: Primary contributions involved training of graduate students Md. Junaid Arafeen and Daniel Aceituna, in topic areas related to software testing and empirical studies. Md. Junaid Arafeen, a graduate student, was supported by this grant, and completed his M.S. degree in July, 2012. He is now working at one of the software development companies. * We produced several research papers including IEEE international conferences and Master's Thesis (Md. Junaid Arafeen). * We produced the following software tools. - Analytical Hierarchy Process tool - Clustering/requirements-based test case prioritization tool
