The aim of this proposal is to conduct research on the foundational models and algorithms in computer vision and machine learning for an egocentric vision based active learning co-robot wheelchair system to improve the quality of life of elders and disabled who have limited hand functionality or no hand functionality at all, and rely on wheelchairs for mobility. In this co-robt system, the wheelchair users wear a pair of egocentric camera glasses, i.e., the camera is capturing the users' field-of-the-views. This project help reduce the patients' reliance on care-givers. It fits NINR's mission in addressing key issues raised by the Nation's aging population and shortages of healthcare workforces, and in supporting patient-focused research that encourage and enable individuals to become guardians of their own well-beings. The egocentric camera serves two purposes. On one hand, from vision based motion sensing, the system can capture unique head motion patterns of the users to control the robot wheelchair in a noninvasive way. Secondly, it serves as a unique environment aware vision sensor for the co-robot system as the user will naturally respond to the surroundings by turning their focus of attention, either consciously or subconsciously. Based on the inputs from the egocentric vision sensor and other on-board robotic sensors, an online learning reservoir computing network is exploited, which not only enables the robotic wheelchair system to actively solicit controls from the users when uncertainty is too high for autonomous operation, but also facilitates the robotic wheelchair system to learn from the solicited user controls. This way, the closed- loop co-robot wheelchair system will evolve and be more capable of handling more complicated environment overtime.
The aims ofthe project include: 1) develop an method to harness egocentric computer vision-based sensing of head movements as an alternative method for wheelchair control; 2) develop a method leveraging visual motion from the egocentric camera for category independent moving obstacle detection; and 3) close the loop of the active learning co-robot wheelchair system through uncertainty based active online learning.

Public Health Relevance

The project will improve the quality of life of those elders and disabled who rely on wheelchairs for mobility, and reduce their reliance on care-givers. It builds an intelligent wheelchair robot system that can adapt itself to the personalized behavior of the user. The project addresses the need of approximately 1 % of our world's population, including millions of Americans, who rely on wheelchair for mobility.

Agency
National Institute of Health (NIH)
Institute
National Institute of Nursing Research (NINR)
Type
Research Project (R01)
Project #
5R01NR015371-02
Application #
8914675
Study Section
Special Emphasis Panel (ZEB1)
Program Officer
Diana, Augusto
Project Start
2014-09-01
Project End
2018-08-31
Budget Start
2015-09-01
Budget End
2017-08-31
Support Year
2
Fiscal Year
2015
Total Cost
Indirect Cost
Name
Stevens Institute of Technology
Department
Biostatistics & Other Math Sci
Type
Biomed Engr/Col Engr/Engr Sta
DUNS #
064271570
City
Hoboken
State
NJ
Country
United States
Zip Code
Long, Chengjiang; Hua, Gang; Kapoor, Ashish (2016) A Joint Gaussian Process Model for Active Visual Recognition with Expertise Estimation in Crowdsourcing. Int J Comput Vis 116:136-160