Project Proposed: This project, developing an instrument that serves as an interactive personal care and human activity monitoring center, aims to keep a person with high quality life and safe at home as long as possible. The instrument enables privacy-preserving and secure data sharing through wireless connection with remote users in an assistive living environment. Providing mental and emotional support, the zooscopion (zScope) can connect devices, humans, objects, and the environment. It can connect to other assistive living projects, making them interoperable and can deliver a Digital Library of sanitized research data and cases with high educational and training value. zScope combines and correlates many types of data and extracts events of interest that indicate changes, risks, etc. It can analyze facial expressions to detect pain, environmental data, house data (such as door opening, telephone sounds, vacuum cleaner, etc.), human performance metrics (e.g., hand strength), both in continuous and discrete format. Data are modeled and assembled in meaningful ways to predict and prevent physical and digital problems (e.g., respectively, falls and intrusions). Privacy and security are being made part of the data modeling at the design phase. The instrument will take sensor data, human body measurements, camera data when requested, known pattern of behavior from other cases, brain scans, and clinical information, aiming to provide high resolution displays of longitudinal as well as episodic events. It outputs a visual interactive display of patterns and significant human behavioral 'events' valuable in assistive environments, setting where to use non-invasive monitoring technologies, helping recognize 'behavioral biomarkers' that will be connected to other types of health indicators that may come from brain imaging, genetic analysis, clinical results, or psychological evaluations. It will work with the next generation of data that include behavioral, clinical, body motion, etc., and have low latency tracking. Broader Impacts: This work enables human-centric type of experiments and provides novel new ways of interaction, visualization, and secure collaboration. Developing 'smarter' living environments for the aged opens new ways to education with immersive compelling projects that provide a better understanding of the role of science and engineering when combining health data (genomic information) to behavior, predict trends, and provide indicators of how medication and clinical assessments connect to longitudinal behavior. Long term goals include behavioral markers for assessing the confluence of environment, drugs, and human psychology. The instrument is also expected to respond to queries regarding emerging needs for new analysis of collected information. It includes training and educational modules with search and browsing tools and a recommender facility to support decision making and use stored strategies. Moreover, utilizing existing outreach programs, the project will support local minority students and high school students A new generation of scientists that can work together across domain silos towards human centered goals might be in the making!
zScope is a one stop human monitoring instrument that collects and analyzes multisensing human activity data to support independent living of the elderly in assistive pervasive environments. The zScope instrument provides multiple views into the life of an elderly person that include eating, sleeping, walking, and other activities. An important focus is to understand and track the needs of a human, provide guidance to his/her caregivers, ensure the safety of the elderly and provide smart services that improve the quality of their lives. The research carried out has produced a big body of publications in many different related scientific areas and has brought great visibility to the challenges involved in applying computer technology to provide in place aging. The project developed a number of novel methods and software and hardware for efficient data collection, analysis, interpretation, implementation and data management. As a fist step of the project, methods were developed and experiments were done to optimally place sensors in order to ensure that key areas of daily life activities were covered and monitored. The experiments included the use of both static and mobile sensors and took into consideration reliability concerns, such as sensor failures, in order to improve robustness. A wide range of methods and tools were developed that range from supporting indoor localization, fall detection, medication tracking and sleep monitoring, to limb and finger motion tracking to enable at home rehabilitation and gesture recognition. As part of zScope development, tools were designed and implemented to detect events of interest in the monitored environment and to provide alerts and recommendations to the interested parties, such as caregivers, doctors or family members. To facilitate interaction with computerized systems, especially for people with disabilities, intelligent interfaces were developed. One type of interface supports robust audio-visual speech recognition and dialog management, while another interface supports as eye-tracking and point-of-gaze detection in 3D space. Both types of interfaces are important assistive technologies for zScope users. In addition to passive data collection and analysis, there was development of active, assistive robotic actuators to solve certain needs. For example, tools were developed to support human interaction with human-like robotic companions, which were found to have therapeutic effects when interacting with special populations. Besides the human-like robots, the project also developed robotic platforms, such as smart wheelchairs, that can be used by people with severe disabilities, utilizing alternative inputs such as eye tracking and voice. The vast amount of data generated and collected though zScope, require advanced storage, management and querying techniques. Methods were thus developed using state-of-the-art data warehousing, retrieval and analysis. The work published as a result of this project includes methods for multi-modal data fusion, feature selection and dimensionality reduction, searching and matching on time-series data, and robust classification. Subsets of the data collected during the life of the project have also been made available for public use. The results and findings of this project constitute a valuable asset on the way towards the commercialization and wider adoption of pervasive technologies related to assisted living. Further work needs to be done in the area of experimentation to identify best ways for adopting the outcomes to real life, using actual human subjects in realistic settings.
