Science policy and other decision makers need qualitatively novel tools to help them identify the publications, patents, technology claims, grant proposals, or other developments that are most important or potentially profitable and deserve their full attention. They also need a tool that lets them analyze and mine terabytes of relevant data and presents aggregated results in an easy to understand way, enabling them to increase the quality of their decisions. However, more than just a tool is needed. The design of tools that truly support science policy making requires a detailed characterization and prioritization of the information needs of science policy makers, along with a conceptual framework that links the information needs of the policy maker to the available data, analysis and indicator types. Once these are in place, tool development that truly addresses the needs of today's science policy makers given the constraints of existing datasets as well as theoretical and algorithmic approaches can proceed. This study addresses three challenges. First, a detailed analysis is conducted of the information needs of a representative set of science policy makers including existing data, approaches and tools. Second, a theoretic conceptualization is developed of tasks relevant to science policymaking that maps the needs of policy makers to theoretically grounded and practically valuable processing pipelines that transform data into actionable information. This conceptualization is then made available online in a wiki-like format for community review and consensus building. Third, a prototypical tool--a macroscope--is designed to visualize structure, patterns, trends, and outliers in science and technology data sets that are too large and complex to be comprehensible to the researchers through direct observation--just like microscopes and telescopes make it possible to see things that are too small or too far away. The prototypical macroscope supports a well defined set of information needs, e.g., identification of emerging research frontiers or correlation of funding with publications and patents in an area of research and exploration of results using graphs and geospatial and science maps. Macroscope tool development will benefit from the NSF funded Scholarly Database (SDB)) that provides access to more than 20 million scholarly records, and the Cyberinfrastructure Shell (CIShell), which supports the easy plug and play of datasets and algorithms and the design of stand-alone tools. Usability studies are conducted to evaluate and optimize the macroscope. The macroscope as well as its support of advanced science mapping techniques will be introduced to a broader audience by means of the Places and Spaces: Mapping Science exhibit. Researchers and science policymakers in Japan will participate in this project, generating a bi-directional flow of best theory and practice. The scientific study and management of science and technology requires a shared terminology and conceptualization of how science works. It requires shared data repositories that can be used to run benchmarks and comparisons of algorithms, to thoroughly validate the combination of algorithms, indicators, and evaluation procedures. The theoretical component of this study, as well as the extension of the SDB and the prototypical design of a macroscope, will create a basic shared understanding and a freely available cyberinfrastructure and tool for the science of science (policy) community. The same capabilities that could make the macroscope a tool of choice for science policy makers could also make it attractive to other potential users. For instance, students or members of the public will be able to see the large scale structures of science, and find potential areas of interest for further study; researchers will be able to monitor and access research results, relevant funding opportunities, or find potential collaborators within and without their fields of inquiry. Program managers and reviewers will use it to meet their individual needs.
This project funded the development of the Science of Science (Sci2) Tool (http://sci2.cns.iu.edu) that supports the temporal, geospatial, topical, and network analysis and visualization of scholarly datasets at the micro (individual), meso (local), and macro (global) levels. Today, the tool is actively used by the National Science Foundation, the National Institutes of Health, the US Department of Agriculture, and the National Oceanic and Atmospheric Administration as well as researchers, industry representatives, and government officials from 67 countries. In addition, it co-funded the Places & Spaces: Mapping Science exhibit (http://scimaps.org) that was created to inspire cross-disciplinary discussion on how to best track and communicate human activity and scientific progress on a global scale. The exhibit has two components: the physical part supports the close inspection of high quality reproductions of maps for display at conferences and education centers; the online counterpart provides links to a selected series of maps and their makers along with detailed explanations of how these maps work. The exhibit is a 10-year effort. Each year, 10 new maps are added resulting in 100 maps total in 2014. Over the duration of the project, the exhibit has been on display at more than 100 public venues see http://scimaps.org/exhibitions. Among others, project results are captured in Börner, Katy. (2010). Atlas of Science: Visualizing What We Know. MIT Press (http://scimaps.org/atlas) that won the American Society for Information Science & Technology (ASIS&T) best Information Science book published in 2010 award.
