This project develops powerful new measures and models that assess and predict the impact of academic research on science, technology, and the public. The evaluation of research has historically relied on quantity as a proxy for quality: progress is inferred from the amount of research produced and attention garnered, the number of articles is tallied, and academic citations and media mentions are summed. These quantities are cheaply measured, but cannot capture how fundamental a contribution is or identify the nature of its impact. Such nuanced evaluation of research quality demands that new claims be considered in the context of previous work. Building on progress in the measurement of scientific and technological novelty, as well as phylogenetic models of evolution, advances in computational language understanding, and the increasing electronic availability of historical and contemporary scientific texts, this project creates computational tools to identify scientific claims, embed them in their historical, conceptual, and geographical context, and thereby provide a multidimensional evaluation of the nature and scope of their impact. Claims extracted from text are complemented by article citations and unpublished expert opinions about paths of scientific influence, elicited through newly developed interactive online games. Taken together, these data provide the input for probabilistic models that exploit patterns in the structure of scientific language, the graph of citations, and expert opinion to assess the evolutionary and ecological importance of particular concepts and claims, articles and journals, scientists and institutions within the unfolding network of scientific and technical innovation. These rich models provide the nuanced assessment of research quality and impact needed by scientists and policy makers at all levels, from individual scientists choosing a new research focus to departments making tenure decisions or federal agencies setting funding priorities.
The methods and models developed by this project are relevant to scientists and policy makers in many areas. The initial focus, however, is on the broad field of chemistry and its diverse, overlapping subfields, from analytical, physical and organic chemistry to environmental and agricultural chemistry, pharmaceuticals and toxicology.
Broader Impacts
This project enables the creation of high resolution, dynamic maps of scientific influence in several broadly related fields. In education, such maps can be made interactive and serve as a teaching tool to help students understand the contributions and forces that have shaped their field of science. These maps also facilitate the social analysis of scientific production with a new level of precision, while orienting researchers and policy makers to the forces that shape scientific importance and technological promise. Most importantly, these influence maps provide the foundation for a collection of sharp measures that can identify different types of importance and the benefits and risks associated with each so that they can be assessed and balanced by researchers and policy-makers. By clarifying what is published and where, and by tracing the conceptual careers of scientists and inventors, this research offers insight into the factors that guide and deflect scientific attention and how these factors can be harnessed to achieve the greatest impact from public investments in science and technology.
