The proliferation of powerful smart-computing devices (e.g., smartphones, surveillance systems) capable of production, editing, analysis, and sharing of multimedia files and associated technological advances have affected almost every aspect of our lives. The use of digital multimedia (images, audio, and video) as evidence is rapidly growing in multiple applications, including legal proceedings and law enforcement. However, forensic audio examiners are facing a new challenge of analyzing evidence containing audio from social networking websites, because audio editing and manipulation tools are both sophisticated and easy to use, increasing the risk of audio forgery. The goal of this project is to develop a framework and methods to support audio forensics examiners in detecting and localizing tampering in audio files, including developing novel algorithms to associate files to specific recording devices; creating methods to detect and estimate the risks of attempts to evade existing forgery detectors; evaluating speaker recognition systems in the presence of audio replay attacks; and collecting a large and diverse dataset of recordings that can be used for benchmarking of existing and future audio forensic analysis tools and techniques. The project also has a significant educational component, consisting of a set of hands-on activities involving media generation, manipulation, and analysis aimed at outreach and broadening participation in science, technology, engineering and mathematics (STEM) disciplines including forensic science, digital signal processing, and statistical data analysis and digital forensics.
The project is has four main research thrusts. The first will involve designing effective microphone fingerprint modeling and extraction algorithms tailored for audio forensic applications. The team will leverage microphone calibration methods, statistical signal processing techniques for blind microphone fingerprint estimation, and system identification methods for linking an audio recording to a specific recording device. The second thrust aims to investigate the impact of anti-forensic attacks on existing forgery detectors and replay attacks on speaker recognition systems. The research team will design attack models to perturb the underlying forgery detection feature space and analyze performance of existing and new algorithms under these anti-forensics attacks. The third research effort will be focused on designing new audio forensic analysis algorithms robust to these and other emerging anti-forensic attacks. The team will use manipulation methods for anti-forensic attacks and a game-theory-based framework for attack-aware tamper detection and design new forensic methods based on findings of these activities. The fourth research thrust will aim at developing a first-of-its-kind research commons for audio forensics consisting of benchmarking datasets, algorithms, and tools. The team will collect audio from both controlled settings and crowdsourcing in the wild, and use known audio manipulation, editing, and anti-forensic techniques to generate tampered datasets. The team will design and deploy the benchmarking testbed, ForensicExaminer, using a micro services architecture.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
