This research project will develop methods and algorithms to assist people with the procedures required to operate their home medical devices without errors. The goal is for the system to be trained on sample recordings of the person operating the device correctly, so that later it will be able to detect deviations from the correct operation sequence in the currently performed procedure, and automatically provide corrective feedback to the user, including segments of the video portions that show the appropriate steps. The project provides a new paradigm for learning by observation that does not require complete understanding of detailed activities in arbitrary visual and sensor sequences, but merely aligns a given new sequence in known context with previously established training data to detect significant deviations. The approach has four components: (1) defining the key states in an operational procedure and the sensors required to best detect and later communicate the proper operation of a portable home medical device; (2) training the system by observing multiple correct operations; (3) observing a new instance of the operation sequence and recognizing that this operation deviates from the training data in a significant way; and (4) providing corrective feedback to the user in the form of audio and video prompts. The research aims to understand the common types of steps required in the operations of home medical devices, map how the critical indicators of these steps can be detected through appropriate sensors, train a system to recognize these steps in the context of a specific human operator, establish a range of required training repetitions for different operational step types and corresponding sensors, and provide a set of suitable interventions to the end user when errors occur. The experiments will establish the range of training data set sizes for the automated classification of device operations. The research expects to yield a taxonomy of typical operational steps from an observational perspective for a set of devices such as infusion pumps, and establish the most effective sensors or sensor combinations to detect the successful completion of each type of step. In addition, the work will help find suitable passages in the video portion of the training observations to use as corrective feedback, together with other interactive dialog interventions that may be appropriate for the particular user.
The long-term goal of this research is to develop a cognitive assistance system to learn and represent sequences of steps in the operation of home medical devices through multi-sensor observation and interaction with a human operator. Examples of some of the targeted home healthcare devices are respirators and nebulizers (to help breathing), dialysis machines, infusion pumps, home monitoring devices for blood pulse oxygen, EEG, and ECG. The project will develop a means for home users of these devices to ensure that the correct procedures are followed and accurate operations result. The system provides ongoing feedback to assist users in their device operation by ?watching? the process via different sensing technologies and providing appropriate guidance when required. The target population immediately benefiting from this work would be patients with mild cognitive impairments who would be supported with the automated coach in their use of home medical devices. The growing user base includes elderly people living at home, but requiring support from home medical devices. These medical devices can allow a patient to live independently with minimal assistance, as long as the home medical devices provide the required health support. The end result may be a reduction in errors and in the number of calls for assistance in the operation and maintenance of home medical devices. This will allow people to live independently at home for an average longer period than at present and thereby reduce health care system costs. The research is valuable for medical device companies with respect to device design, verification, and validation processes, offering insights into what sensors and communication devices could be most beneficial for integration into the device itself.
This project has been developing methods and systems to assist people with the procedures required to operate their home medical devices without errors. The goal is for the systems to be trained on sample recordings of the person operating the device correctly, and be able to detect deviations from the correct operation sequence in the currently performed procedure, and provide corrective feedback to the user, including segments of the video portions that show the appropriate steps. The project consists of four components: Defining the key states in an operational procedure and the sensors required to best detect the proper operation of the device Training the system by observing multiple correct operations, Observing a new instance of the operation sequence and recognizing that this operation deviates from the training data in a significant way and Providing corrective feedback to the user in the form of audio and video prompts. The project aims to understand the common types of steps required in the operations of home medical devices, map how the critical indicators of these steps can be detected through appropriate sensors, train a system to recognize these steps in the context of a specific operator, establish a range of required training repetitions for different operational step types and corresponding sensors, and provide a set of suitable interventions to the end user when errors occur. The target population benefiting from this work are elderly people living at home, but requiring support from home medical devices such as glucose monitors or dialysis machines. These medical devices can allow a patient to live independently with minimal assistance, as long as the home medical devices provide the required health support. The a key outcome has been in validating the feasibility of automatically detecting errors in the operation of medical devices. The work provides a new paradigm for learning by observation that does not require complete understanding of detailed activities in arbitrary visual and sensor sequences, but merely aligns a given new sequence in known context with previously established training data to detect significant deviations. Our work also has developed principled mechanisms for corrective feedback, that may be appropriate for the device coach. The potential impact of this work is not only that it will reduce errors in the operation of medical devices, but it will allow people to live independently at home for a longer period than before, thus saving the health care system billions of dollars resulting from earlier admissions to nursing homes or morbidity from incorrect operation of in-home medical devices. Ultimately, this paradigm can also extend to the repair and maintenance of numerous other devices by non-specialized personnel.
