Social historians and researchers in social sciences rely to a great extent on text data for their research. Currently these data sources are increasingly available in electronic form. The Digging into Social Unrest project aims to develop software tools for automated analysis of news archives. The project will help researchers discover information and obtain answers to questions that now take large, if not unrealistic amounts of human effort to answer. The aim is to develop text mining tools which will provide social historians and scientists the means to detect, associate, and visualize the events and the underlying trends, people, and organizations related to social unrest.
This work is based on the notion that conditions leading to social unrest can be identified and extracted from news archives, which may include reports about events of varying degrees of organization and violence --ranging from social movement meetings, to protests, strikes, demonstrations, and riots. Careful study of these sources can be used to extract indicators analysts can use to predict societal instability trends and events.
The Digging into Social Unrest project will develop a text-mining based search system that improves entity and event extraction by building upon tools and services previously developed by members of this team. The tool, called ISHER (for Integrated Social History Environment for Research) will be a fully open source framework and will be designed with the intent of working on any digital text repository. Specifically, the tool will allow rich semantic metadata extraction for collection indexing, clustering, and classification.
The primary broader impact of this project is that it will develop a tool that can be used by a wide array of disciplines in the social sciences and humanities. In addition, the research will add to the body of knowledge on social unrest and is highly likely to spur research on comparable data archives by researchers in other fields.
This grant was made as part of the Digging Into Data Challenge, an international competition designed to foster research collaboration across countries and to encourage innovative approaches to analyzing large data sets in the social sciences and humanities. In addition to the US research team, this project includes researchers from the Netherlands and the United Kingdom.
