The traditional measures of scientific productivity have been publications and citations. A major question in the area of science of science and innovation policy is whether it is possible to expand outcome measures beyond such metrics and directly measure the amount of technical advance or actual value generated by a science investment more quickly and descriptively. Another major question is whether it is possible to measure the incremental advances that characterize many projects, in addition to the traditional focus on major findings.
This research examines the potential contribution of a metrics system, based on expert coding of current research findings, that is both fine-grained and broad in scope. The general principles used to characterize the metrics are differences by treatment area and by stage in the process, which are the major sources of variations in health research problem. It assesses whether research findings can be measured in real time and be both linked back to specific projects and forward to the "pay-off" for society. The project applies a system of 20 metrics of health outcomes and 16 metrics of economic benefit that has been developed to all research projects funded in 2007 by the National Institutes of Health. It is used to code technical progress in six areas of morbidity as stated in research reports after two years of research.
The metrics system is implemented at the project level in order to permit an examination of whether the degree of diversity and the amount of inter-organizational involvement in research teams is either accelerating or decelerating results.Connecting advances in real-time to the composition of the research team can also help inform the management of "team science" and advance understanding of its impact on innovation.
The metrics system, if successful, could be used to add to the scientific basis for the evaluation of projects awarded in 2009 by NIH under the American Recovery and Reinvestment Act. The demonstration study also provides a benchmark for what can be reasonably expected in that short time period, given differences by treatment area and by stage in the process.
The approach could potentially be applied to other major policy objectives, such as alternative energies. Connecting advances in real-time to the composition of the research team also contributes findings about the management of ?team research? and its impact on innovation.
