This project investigates social multimedia for geographic discovery. Specifically, community-contributed ground-level images and videos are used to map what-is-where on the surface of the Earth in much the same way that overhead images taken from air- or space-borne platforms have been used for decades in the traditional field of remote sensing. The overarching premise is that georeferenced social multimedia data can be considered a form of volunteered geographic information. Further, it can enable geographic discovery not possible through traditional means. The framework, termed proximate sensing, is applied to two challenging geographic discovery problems: 1) land-use classification, and 2) mapping public sentiment such as how scenic a particular geographic location is. Land-use classification is an important problem but is often not possible using overhead images since remote sensing does not record activities. This project instead applies and extends state-of-the-art techniques in image understanding to ground-level images and videos to map land-use. The motivation is similar for mapping public sentiment.
The broader impacts include developing K-12 spatial literacy curricula through an Engineering Projects in Community Service team whose client is the Merced County Office of Education. University of California-Merced was recently classified as an Hispanic Serving Institution, making it one of only a handful of research universities with this designation nationwide. Undergraduate students from underrepresented groups will be involved in developing a GeographUSA project whereby volunteers can upload geographically representative photos of the United States. Results, datasets, and other project artifacts will be made available through the project website (http://vision.ucmerced.edu/projects/socialmultimedia/).
