This EAGER project will test a novel method for determining the relative value of promoting interdisciplinarity of research projects. Specifically, the research team will investigate the effectiveness of the Human and Social Dynamics (HSD) priority area awards. Using a novel method to test how well the HSD program has been at stimulating research that crosses disciplinary, organizational, institutional, and geographic boundaries, this project will examine research teaming and research community networking characteristics, as well as measure the interdisciplinarity of publications by HSD researchers. This EAGER project will support exploratory and potentially transformative, work to study interdisciplinary perspectives. This project will perform non-traditional program evaluation in refining measures of the interdisciplinarity of a subset of NSF research. Using the HSD program as a proof of concept, the PI will test his novel methodologies. The project also explores alternative ways to visualize (map) such research, how it fits within the map of science, and social network dynamics. It also explores novel data sources for such assessments in seeking to ascertain the characteristics of especially fruitful research and the transfer of that knowledge across other research areas and beyond.
Broader impacts of this project are expected to be strong. If the proof of concept is successful, this project could lead to a very exciting method for assessing the value of funding of interdisciplinary research. The findings of this research should point to ways to facilitate interdisciplinary research knowledge interchange. This assessment should help point to what research approaches best enable such groundbreaking work and offer insights to enrich graduate training programs to enhance young researchers' networking and interdisciplinary skills.
Intellectual Merit: Research that integrates the social and natural sciences is vital to address many societal challenges, yet is difficult to arrange, conduct, and disseminate. This project analyzes the cross-disciplinary character of the research supported by a unique U.S. National Science Foundation program on Human and Social Dynamics ("HSD"). Figure 1 maps publications deriving from HSD support. The Web of Science [http://thomsonreuters.com/web-of-science] is a leading database that indexes articles published in some 12,000 leading science and social science journals. This widely used database groups those journals in 224 Web of Science Categories (WoSCs). The base map locates those 224 categories based on how frequently their articles cite each other. The WoSCs appear as the endpoints of the black lines showing strength of citation links for the Year 2010 for Web of Science. We then overlay the Year 2004 HSD project publications upon that base map – larger nodes indicating more journal publications. The map shows exceptional diversity -- research publications deriving from this support chiefly pertain to the Social & Behavioral Sciences [lower part of the map], but extend widely into the Bio & Medical Sciences [upper left], Environmental Sciences [upper right], and Physical Sciences & Engineering [lower right]. Also identified in the map are the five leading WoSCs in which HSD-supported research was published. We compare the HSD project publications to those from comparison projects (also funded by NSF); those are less diverse. We also map the papers that cite the HSD publications and, separately, map those that cite the comparison papers. Importantly, again, the HSD map shows strong engagement by the sciences. This offers evidence that HSD research exerts influence beyond the social and behavioral sciences. The HSD-citing papers overlay map is similar in appearance to the HSD publications map shown here [Garner et al., 2013]. In addition to visualizing these data, we apply several measures to help understand how interdisciplinary the research is. Integration scores, based on the diversity of references cited, indicate that the HSD-derived publications draw upon more diverse knowledge sources than do those of comparable programs. Diffusion scores, together with science overlay maps, show that uptake of the HSD publications extends into the natural, as well as social, sciences. Research networking analyses, together with a new composite mapping approach, point toward successful catalysis of a new research community. Our research team was particularly interested in the challenge in tracking the transfer of research knowledge. We experimented with a variety of measures, leading to the advent of two simple, but novel, metrics that we believe offer special potential for research assessment [Garner et al., to appear]. One measure – "citation velocity" – calculates how quickly published articles are cited by other journal papers. We found that the 2004 and 2005 HSD-derived papers were generally cited with similar lag times as the comparison group papers. A second measure – "citation distance" – gauges how far away the citing paper’s journal is from that in which the paper was published. This is based on the WoSCs of each, with distance scaled as in the science overlay mapping (such as Figure 1). We explored five research questions using these two new measures [Garner et al., to appear]. To investigate, we focused on 63 heavily cited HSD papers and 63 heavily cited comparison group papers. Those small numbers of papers receive a lot of citations – 4431 for the HSD papers and 5230 for the comparison group in about seven years following publication. Most importantly, we find that HSD publications are cited, on average, by more distant disciplines than are a set of comparison group publications. We also obtained evidence of different citation velocities of papers in different disciplines. Also, papers published in high impact (highly cited) journals tend to get cited faster and in more closely related journals (nearby disciplines). Broader Impact: These maps and citation distance & velocity measures enrich analyses of research knowledge diffusion patterns. They can be applied to facilitate social studies of science. They also support research assessment to inform research management at NSF and elsewhere. For one, assessment of the interdisciplinarity and extent of research influence of the NSF Human and Social Dynamics Program could inform future NSF programmatic decisions concerning such interdisciplinary funding initiatives. References: Garner, J., Porter, A.L., Borrego, M., Tran, E., and Teutonico, R. (2013). Facilitating Social and Natural Science Cross-Disciplinarity: Assessing the Human and Social Dynamics Program. Research Evaluation. 22 (2), 134-144. doi: 10.1093/reseval/rvt001. Garner, J., Porter, A.L., and Newman, N.C (to appear). Distance and Velocity Measures: Using Citations to Determine Breadth and Speed of Research Impact. Scientometrics.
