The broader impact/commercial potential of this I-Corps project is significant in the area of light field messaging. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. Information is embedded within images on a display so that the messages are machine-readable but invisible to humans. The electronic display, such as a billboard or kiosk, retains its purpose to convey visual information such as maps, advertisements and schedules. Simultaneously, the display conveys a time-varying message to a camera-equipped computational system, such as a smartphone or robot. The near-term target application is interactive televisions, computer displays, and electronic billboards using existing cameras on smartphones. With light field messaging, a phone-based mobile app could: retrieve product information from a television ad, obtain nearby traffic conditions from a roadside billboard, or obtain walking directions within a large airport from a kiosk. This approach is expected to provide significant new avenues of interactive marketing and media. Additionally, the technology will provide novels methods of indoor localization, where GPS is not precise. Light field messaging for sending navigation cues to robotic systems and self-driving cars is a promising future application.
This I-Corps project uses an innovative deep learning architecture to embed a new message in each frame of video. The goal of this technology is invisible communication to a machine, and not subliminal communication to users who may observe the image. The approach is comprised of software-based algorithms with two main components: 1) Embedding method that embeds/codes hidden messages into photos and videos together and 2) Recovery method on the camera side that retrieves the message. Application software will run on display systems for embedding and on smartphones or other camera systems for obtaining messages from embedded imagery on electronic displays. Unlike older methods such as digital steganography, this approach uses modern neural networks to learn a robust coding method that overcomes the distortion of the light field transmission channel. This messaging paradigm also improves on prior QR-codes because the code does not distract from the visible image and the code is dynamic, allowing significantly more information to be transmitted. The message is fully contained within a single frame of video so that issues of time synchronization are avoided.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
