The project develops computer vision and pattern recognition technologies for visual sentiment understanding and visual sentiment editing. The interdisciplinary research team investigates the problem of understanding how images and video convey emotion. The project develops methods to infer, edit, and synthesize visual sentimental content in image/videos, addition to their semantic contents. The project applies developed technologies to reduce violence from multimedia materials for children, and negative psychological impacts from social media for posttraumatic stress disorder (PTSD) patients. The project integrates research and education by creating new interdisciplinary courses and training graduate students. The project builds connection with the veteran academic resource center on the campus to help PTSD patients to recover from mental health problems. The research team also shares collected data with research communities.
This research develops visual sentiment understanding algorithms through joint extraction of sentiments and semantics, in order to advance the understanding of how semantic entities substantiate and carry sentiments at a fine-grained object or pixel level. Computer vision algorithms and psychometric assessment techniques are combined to automatically analyze visual and recognize sentiments and emotions from multimedia materials and social media contents posted and shared by veterans. The research also explores methods of visual sentiment editing to reduce violence from multimedia materials and social media contents. The research can help (1) to protect children from accessing violent multimedia materials, and (2) to provide appropriate social media contents for applications of automatically detecting violent contents from veteran-shared multimedia.
