This project explores a new direction in computer vision, which is to model the context dependent visual semantics associated with images in social multimedia. The context dependent visual semantics, e.g., the intended and perceived sentiment of an image in social multimedia, are dynamically formed based on the various contextual information associated with it. This is different from the static visual semantics that conventional computer vision research focused on studying, such as the object category presented in the image.
The project develops a set of new networked and context aware probabilistic latent semantic models, which integrate situated contextual information into visual content analysis for modeling context dependent visual semantics. The research team is verifying two hypotheses: 1) the context dependent semantics needs to be holistically modeled and jointly inferred from a collection of related images; and 2) related context dependent visual semantics, such as intended and perceived meaning of an image, also needs to be jointly modeled for more robust recognition.
The project is integrated with education through training graduate and undergraduate students. The outcome of the research can be applied to many domains, such as targeted online advertisements; open source information analysis and social event prediction; and social multimedia security.
