This project uses the capabilities of wearable sensors for two inquiries into creativity. The first inquiry investigates the potential for analysis and visualization tools to help users generate novel mental models from wearable sensor data and explore the implications of such models on their lifestyle and wellbeing. The ability to monitor internal state and relate it to behavior and environment can be transformational, because it allows users to develop insights and provides them with hard data with which to monitor their own progress. By focusing on minimally-invasive and inexpensive sensors the developments will have broad appeal for the general public. Prior research in wearable sensors has mainly focused on predicting psychological state (e.g., affect) from physiological signals, and characterizing the users? environment (e.g., from accelerometers, audiovisual sensors). However, relatively little research has been devoted to exploring the relationship between the internal (i.e. physiological) state of users and their environments; unfortunately, one cannot be understood without the other. Study of this relationship is an area where we believe visual workspaces can have a significant impact.
The second inquiry seeks to explore how wearable sensors may support research in creativity outside of controlled laboratory settings. Experimental methods for creative cognition in laboratory settings have been very successful in identifying a number of cognitive processes and general principles of creativity that apply across a number of domains, from engineering design to the visual arts. However, these studies do not inform us about how creative processes take place in the real world, when users must deal with the demands of their lives and distractions in their environments. Wearable sensors provide an opportunity for the researcher (and the user) to develop an understanding of how physiological variables and real-world environments affect the creative processes. Studies of creative cognition in natural settings, correlating cognitive and behavioral metrics with data from wearable sensors, can validate and greatly extend our scientific understanding of creative thinking in the real world. Whereas retrospective reports of one?s creative ideas are limited by participants? memories and by their subjective introspection, probing people in real-world settings, as proposed in our experiments, requires neither introspection nor retrospection. Thus, validated metrics of creative ideation can be applied in natural contexts without the reactivity that results from laboratory and field experiments.
This project explored the design of infrastructure for performing long-term real-world studies involving the collection of data via wearable sensors and smartphones. The project developed software and hardware to enable the collection, management and analysis of such data. This data is characterized by being heterogeneous, meaning the data measures different features and is encoded in a variety of representations, yet synchronized. These characteristics pose challenges for prior data collection software in that each data type was recorded individually and without awareness of it being synchronized with other data streams. Additionally, much prior work on wearable sensors for collecting such data assumed short term use in a controlled setting. This project addressed both of these issues impeding the capture of data. Regarding the management and analysis of such data, the project developed an environment called PerCon for ingesting, browsing, searching, visualizing, and analyzing the synchronized heterogeneous data streams. The PerCon environment was evaluated in a controlled study examining the effects of including a visual workspace for examining and visually annotating data objects and the effects of having the environment recommend additional data objects based on the user's prior data interactions. The evaluation showed a strong positive effect from visual workspaces as users are able to rapidly examine and group data based on their evolving goals. The results showed a smaller positive effect for recommendations. Surprisingly, the value of the recommendations was largely tied to the availability of the workspace as in this condition users did not need to remove any existing information they were examining when acting on a recommendation. The intellectual and conceptual results of this project provide insight into the requirements for and trade-offs among designs for the next generation of long-term studies involving wearable sensors and smartphones. Additionally, this project has provided strong evidence of the value of interpretive visual workspaces for heterogeneous data analysis and uncovered an unexpected interaction between the value of system-generated recommendations and the availability of such a workspace. Broader outcomes of this project are the development of methods for use across a wide range of disciplines, the development of software that can be used to enhance STEM education, and that the PerCon software environment is being put to use in additional application domains with the potential for packaging for more general use.
