Digital audio-visual content can nowadays be modified and simulated with unprecedented realism. In response to the challenge of malicious media manipulation, we propose to develop new theories, methods, and a comprehensive suite of tools that can be used to verify content integrity in both audio and visual modalities at multiple levels under diverse contexts. The results will have significant impacts in scientific areas as well as many practical applications of national interest, such as trustworthy news reporting, surveillance security, intelligence gathering, criminal investigation, financial transactions, and medical information management. Our efforts will be based on an important paradigm, called blind passive media forensics, which fundamentally differs from conventional approaches using cryptographic signatures and watermarking. The proposed methods extract latent, unique signatures from the signals and sensing devices to detect tampering anomalies, so that verification requires only the media content at hand without any additional information or preparation. Specifically, we propose a systematic framework for device signature consistency checking and a joint multi-modal approach for verifying media integrity. We will also analyze various attack scenarios, construct rigorous benchmark datasets and performance metrics, and take proactive steps in disseminating results and promoting awareness of this critical area through public testing of online systems.
