In the early days of computing user and device were distant, often located in separate rooms. Then personal computers "moved in" with users, and eventually mobile devices moved computing into users' pockets. More recently, wearable devices have brought computing into constant physical contact with the user's skin, which allows these devices to sense more of the user and act in a more personal manner. The main question that drives this project is: what is the next interface paradigm that supersedes wearable devices? To study the next generation of human-computer interaction, this research will create new devices that stimulate a wide range of bodily senses, such as smell, temperature, and skin sensations, to inform the engineering of the technology we will use in the future to communicate with each other and with data. While some posit that the next step will be to implant the interface within our bodies, much like what is already being done with pacemakers and insulin pumps, this project instead focuses on how new devices will integrate with the user's biological senses and actuators. This body-device integration approach will allow the engineering of interactive devices that intentionally "borrow" parts of the body for input and output, rather than adding more technology to the body (for example, interactive systems based on electrical muscle stimulation that are able to move the user's muscles using computer-controlled electrical impulses and thereby to achieve the functionality of haptic devices such as exoskeletons without the bulky motors). In particular, the work will develop new hardware for the senses of smell, temperature and touch, and will test applications in accessibility (for smell-impaired individuals) and skill-acquisition. The work will leverage collaborations with neuroscience to ensure that project outcomes ultimately have broad impact by enabling the creation of more realistic and natural interactions, which are especially important for the success of virtual and augmented reality.
To these ends, three classes of challenges must be tackled. (1) Hardware layer: devices that output to the body will require new types of actuators smaller than those found in wearables, and the project will realize this by leveraging the user's senses and actuators, respectively, for input and output. (2) User layer: devices that influence the user's biology provide benefits (for instance, they can accelerate the user's reaction time) but also reduce the user's sense of agency, so the project will develop interaction techniques that prioritize the user's sense of agency while retaining the benefits of using the interactive device. (3) Application layer: for any new interface paradigm to succeed requires determining in which application areas it performs well and why. Special attention will be paid to investigating the effects of the new technology on accessibility.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
