Every aspect of modern society, ranging from health care to national defense, depends on the availability of affordable, reliable software. The availability of this software, in turns, depends on helping software engineers to find high-quality components that they can assemble into finished software. Web sites currently provide access to millions of components that perform vital operations related to networking, graphics, data processing, and thousands of other functions. However, software engineers lack a validated, reliable method for selecting high-quality components that they can reuse in new software applications. Lacking such a method, software engineers sometimes use components that turn out, in retrospect, to be extremely difficult to reuse. This difficulty, in turn, increases the time required to create software, the cost of that software, and the potential for subtle bugs.
This research project is expected to provide a validated, reliable model for quickly assessing the reusability of components. This method will make use of "low-ceremony evidence" (LCE): information that characterizes different aspects of component quality yet is incremental, often informal, potentially context-dependent, and frequently contradictory. Examples of LCE include reviews, bug reports, and download counts of components. While each piece of LCE provides only an incomplete perspective into a component's quality, preliminary work suggests that the synthesis of LCE can be highly informative about component quality. This new research project (1) will use factor analysis to determine which pieces of LCE are mutually consistent, yielding scales for assessing one or more aspects of quality such as reusability, and (2) will statistically test how well these scales are correlated with the actual empirical difficulty that software engineers report with reusing those components. The resulting validated scales are expected to be useful for automatically assessing the quality of components in online websites. This would make it possible in future work to develop enhanced search engines enabling software engineers to quickly find high-quality components that they can use to create the software that society needs.
Programmers frequently must make decisions about the quality of components, which are combined to create new software. The long-term goal of this research is to help programmers make accurate quality assessments even when specifications and source code are not available for existing components. In pursuit of our long-term goal, above, the research objective of this EAGER proposal was to evaluate the effectiveness of our approach on one test problem: predicting the reusability of components. We analyzed online ratings, reviews and other data for nearly 1200 online components, computed statistics for each component based on these data, and used factor analysis to identify three groups of statistics (factors) that were each internally correlated. We then interviewed software engineers about the reusability of 36 other components and used linear regression to test how well the 3 factors actually corresponded to component reusability. We found that 2 of the 3 factors were indeed related to reusability. Specifically, the reusability of components could be predicted on the basis of component authors' prior work and the documentation provided about components. We have disseminated the specification of factor scales as part of a conference paper that is available online (via the ACM Portal website). As in other studies, this research revealed that the vast majority of code posted by programmers to repositories is never reused by anybody: but now we have factors that explain the reasons why reuse is so low and that can even predict which code will be reused. Furthermore, generalizing these results to other quality attributes other than reusability (e.g., performance or portability) would transform research in software engineering by introducing a new paradigm for quality assessment. In particular, generalizing this work would show that reliable decisions can be made from pieces of information that are individually unreliable. The factors uncovered in our research could be used in future work to develop enhanced search engines that highlight components which are potentially reusable and perhaps worthy of more time-consuming evaluation such as by applying formal methods. Additionally, our results reveal opportunities to improve online repositories through specific simplifications as well as enhancements. Integrating our results into infrastructure, in this manner, will impact practice by helping programmers to find and reuse high-quality components, thereby leading to more effective decision-making in situations when existing techniques for assessing reusability are of little practical use. Ultimately, our line of work will yield a scientifically sound approach for reasoning about quality attributes when specifications and code are unavailable. By making better-reasoned decisions, programmers will be better able to choose and use components and other code that provides the quality attributes needed for a particular situation, yielding improved software whose benefits accrue to users and society in general.
