Veterans who use powered mobility devices including those with high-level spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS), and multiple sclerosis (MS) often experience serious upper extremity impairments. In addition, many older Veterans and Veterans with hemiplegia may also suffer from functional loss in their upper extremities. Management and care of upper extremity impairments often involve a range of assistive solutions. However, product availability and technological advancement for manipulation assistance fall far behind those for mobility. Many of these individuals, despite their independent mobility, cannot reach for a glass of water, make a simple meal, and pick up a tooth brush. They still require assistance from a personal caregiver for essential activities of daily living (ADLs) involving reaching and object handling/manipulation. With the rapid advancement of robotics technology, assistive robotic manipulators (ARMs) emerge as a viable solution for assisting Veterans with upper extremity impairments to complete daily tasks involving reaching, object handling, and manipulation. ARMs are often equipped with many degrees of freedom (DOF), but users cannot control all of the DOFs at the same time with a conventional joystick, and need to switch modes quite often to complete even simple manipulation tasks, especially when an ARM gets close to the target and need to be aligned appropriately for manipulation. Thus existing ARMs suffer from the lack of efficiency and effectiveness especially in an unstructured environment. The goal of this project is to combine vision-guided shared (VGS) control with two types of environment modifications to address the effectiveness and efficiency of ARMs for real-world use. The two types of environment modifications include using commercial or custom adaptive tools (e.g., a holder that can hold a bottle or jar so an ARM can open it), and adding fiducial markers (similar to QR codes) to objects or adaptive tools to make vision-based tracking robust and reliable for real-world applications. Built upon the environment modifications, the VGS control will allow a user to initiate any task by moving an ARM close to a tagged object, and the ARM to take over fine manipulation upon detecting the target.
The specific aims are to (1) to develop, implement, and bench-test the VGS control with fiducial markers and adaptive tools; (2) to evaluate the new control among powered wheelchair users who will use a wheelchair-mounted ARM to complete a set of everyday manipulation tasks; (3) to explore the potential benefits and limitations of ARMs enhanced by the VGS control with fiducial markers and adaptive tools in a one-month pilot home trial. We expect to improve manipulation functions of Veterans with upper limb impairments through a more practical and usable implementation of vision-based robotic control and human-robot interaction technologies. The mission of the VA Prosthetic & Sensory Aids Service (PSAS) is to provide comprehensive support to optimize the health and independence of the Veteran. The proposed work clearly supports this mission by maximizing the potential of ARMs in assisting Veterans with everyday manipulation tasks and supporting the adoption of this advanced assistive technology.
Veterans with upper limb impairments due to neuromuscular conditions (e.g., high-level spinal cord injury and amyotrophic lateral sclerosis) or other physically disabling conditions often have difficulties in performing activities of daily living that require object handling and manipulation. Assistive robotic manipulators (ARMs) have emerged as a potential solution to mitigate the difficulties, frustration, and lost independence experienced by these individuals. However, achieving effective and efficient control of an ARM has been a challenge due to the less intuitive control. The goal of this project is to develop and evaluate a vision-based shared control approach for ARMs along with environment modifications to address the effectiveness and efficiency of ARMs for real-world use. The enhanced ARMs could potentially lead to improved independence and ability to integrate into community of these veterans, as well as attendant care savings.
