This exploratory project studies how different groups of people tend to create and describe (tag) videos of the same common event differently. Since each of these affinity groups share particular backgrounds, languages, interests, and locations, they also tag and browse these videos differently as well. For example, an international health crisis is portrayed rather differently by the news media of China compared to that of the U.S. Chinese coverage tends to include historical clips, talks about countries, and has less focus on individual participants. In contrast, U.S. coverage tends to be more contemporary, uses countries mainly to identify the background of individuals, and prefers to highlight the actions and reactions of people who are fully named. The aim of this project is to develop new ways of making the videos of other groups to be more accessible and more understandable to different groups by developing a browser that graphically illustrates the visual differences across such videos, and that translates the preferred tags of one group into the preferred tags of the other. The resulting new browser would make a broad range of videos, especially those in a different language, easier to find, scan, and compare.
This exploratory project has three major components: (1) Development of a novel, shareable catalog of statistically significant cross-group differences. To do this, it will first map visual and textual features, from many videos about a single topic, into a joint latent space. Then, by using reliable shared visual cues, it will determine tag relationships using variants of canonical correlation analysis, sort them using measures such as G2, and validate them against users who are members of both groups. (2) Exploration of how well these non-linear methods can be extended to the videos of multiple pairs of affinity groups with more subtle differences, such as comparing the U.S. to Canada. In particular, it will use a novel variant of the PageRank algorithm to track the influence and persistence of visual memes across groups, validating this by ground truth. (3) Implementing and evaluating a prototype browser that visualizes on parallel timelines the perspectives of different groups, both statically and dynamically as they evolve over time. The project will measure the usual browser attributes (time, accuracy, satisfaction, ease of use), but also some unusual and exploratory ones (appropriateness of retrieval, accuracy of tag translations, increases in user engagement, impact on journalists).
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
