Remotely sensed imagery is a critical input to a wide range of research activities in environmental science, environmental management, and related domains. For example, analysts use multi-spectral imagery to detect and monitor forest disturbances, analyze habitat loss and fragmentation, and assess species diversity. A large proportion of recent research in remote sensing has been directed to automation of image analysis. The role of the human analysts is equally important to consider, however, because no fully automatic image analysis system currently exists. Through application of a visual analytics approach that couples human expertise with computer processing speed and consistency, it may be possible to improve accuracy, precision, and task-relevance of image-derived information. This coupling requires a more comprehensive understanding of the human analysts' perceptual and reasoning processes when they use the imagery. This doctoral dissertation research project will investigate cognitive tasks and fundamental visual stimuli used in the interpretation of aerial imagery within the application domain of forest management. To create an awareness of both high-level thought processes as well as low-level visual cues that are used by forest analysts, the doctoral student will use two cognitive methods. First, she will use applied cognitive task analysis, a method based on semi-structured interviews and diagramming activities, to uncover the knowledge structures and cognitive skills analysts use during the image analysis task. Second, she will conduct controlled cognitive experiments to identify visual cues deemed most important for visual interpretation of remotely sensed imagery. Using the knowledge gained during these two phases of research, a set of visual analytics tools will be developed to support semi-automated analysis of remotely sensed images for forest disturbances.

The current trend in remote sensing research focuses on the development and improvement of automated processes for addressing the increasing volumes of imagery data. These research efforts often fail to consider the importance of human operators in the process, and they do not consider the benefits that human-guided analytic processes can provide. This project will illuminate the cognitive processes that underlie image analysis, including both high-level thought processes as well as the low-level visual perception of imagery. Considering both facets of the image-interpretation process will provide an opportunity to clarify the links between these two aspects of image analysis and will provide direct input to development of visual analytic methods that connect human expertise with computational methods in productive ways. The project therefore will be useful for a range of activities, including forest science and management practices, geospatial intelligence analysis, and image analyst training. As a Doctoral Dissertation Research Improvement award, this award also will provide support to enable a promising student to establish a strong independent research career.

Project Report

Remote sensing images collected using sensors mounted on satellites, aircraft and other platforms, provides invaluable information to address many societal and scientific challenges. From transportation planning to complex environmental science research, these images of places at scales from the local to whole earth views can provide rich value-added geospatial products for understanding situations and processes, making decisions about current activities, and planning for the future. Raw images from remote sensing devices are often not photographs, they are digital records of reflected or emitted radiation. These raw measurements must be interpreted to be useful. Automated image interpretation processes typically do a good job at initial processing of the raw measurements. But, thus far, current computation methods cannot match the flexibility and creativity behind human vision and cognition for interpreting the results. For example, human experts in forestry can apply their expertise to identify changes to forest structure or health and generate hypotheses about the causes of those changes that cannot be extracted automatically. The inability to precisely translate human cognitive functions into computational software means that complex cases of image interpretation are still reliant on experts. The primary goal of this research was to identify the reasoning skills, perceptual cues, and knowledge used by experts during the interpretation of remote sensing imagery with an emphasis on imagery used to monitor forests. The motivation for doing so is to provide a base of knowledge needed to train image analysts and to improve computational methods that facilitate the image interpretation process. To address the goal, an in-depth work domain analysis was conducted in which interviews, diagramming activities, and workplace observation were carried out with a set of expert image analysts. Expert knowledge elicited from these activities was used to develop a theoretical model of the image interpretation process used by the experts to interpret forest disturbance as represented in standard imagery. Evidence suggests that individuals who had more experience in the interpretation of remote sensing imagery came to final interpretations more quickly than those with less experience. Image analysts with more experience exhibit more flexible and consistently faster analysis processes than less experienced analysts; this is often due to use the context of analysis to determine analytical shortcuts that save time but do not decrease accuracy of results. The results also showed that despite these differences, the level of agreement among analysts was high; final judgments about the forest disturbance type were identical. This work benefits image interpreters by providing them with an explicit description of the types of skills that they use and how those skills apply, skills that analysts may not have been aware that have. This can help analysts identify both more efficient methods and places in the interpretation process where special attention is needed to avoid errors. The results are also directly relevant to design of training materials for novice image analysts and to design of visual-computational methods that integrate computer processing capabilities with human expertise. In relation to novice image analysts, the study suggests that less experienced image interpreters may be able to achieve interpretation accuracy that matches experienced analysts, but that a potential limitation for doing so is a lack of cognitive flexibility. This limitation has implications for recent efforts to crowd source image analysis tasks to citizen scientists. An open question is whether citizen scientists can be successful in execution of image analysis tasks more complicated than search. Finally, this work benefits the greater remote sensing discipline in its re-focus on the human interpretation process. As many researchers move forward with automation-driven research agendas, this work takes a thoughtful look back at the human factors of analysis that remain important today.

Agency
National Science Foundation (NSF)
Institute
Division of Behavioral and Cognitive Sciences (BCS)
Type
Standard Grant (Standard)
Application #
1233769
Program Officer
sunil narumalani
Project Start
Project End
Budget Start
2012-09-01
Budget End
2014-08-31
Support Year
Fiscal Year
2012
Total Cost
$11,699
Indirect Cost
Name
Pennsylvania State University
Department
Type
DUNS #
City
University Park
State
PA
Country
United States
Zip Code
16802