Although high-cost, data-intensive multi-camera systems have been widely used for mobile human tracking and recognition, the pyroelectric infrared (PIR) sensor has a variety of advantages including dramatically low costs, chemical stability, high sensitivity to human body thermal variation, and extremely low sensory data throughput.
This project implements an Intelligent Compressive Multi-Walker Recognition and Tracking (iSMART) testbed based on PIR Sensor Networks (PSN). The novelties of iSMART include three aspects: (1) Context-aware region-of-interest (RoI) exploration to achieve an inherent tradeoff between area of sensor coverage and degree of information acquisition resolution. This research uses strict mathematical models to measure RoI context. (2) Decentralized inference / learning for in-network intelligence. This project develops a belief-propagation-based distributed inference scheme with data-to-object association for continuous tracking and recognition of multiple walkers. It uses orthogonal-projection-based distributed learning for sensor calibration and feature model training. (3) Networked, compressive sampling structures and sensing protocols. This project extends the latest progress in compressive and multiplex sensing theories to guide the design of novel networked sensor receiver pattern geometries and decentralized sensing protocols.
The above research efforts will lead to a novel low-cost, high fidelity wireless distributed sensing system for multiple walker recognition and tracking. As an alternative to video camera systems, iSMART systems can be widely deployed to automatically monitor airports, customs / harbors, and other critical national infrastructures. This project will also generate interesting hands-on labs on intelligent sensor / sensor networks and class projects for both undergraduate and graduate students.
Project Outcomes 1. Summary of Project Outcomes The goal of this project is to implement an Intelligent Compressive Multi-WalkerRecognition and Tracking (iSMART) testbed based on Pyroelectric Sensor Networks (PSN). It has achieved the following 3 major outcomes: (1) Built a multi-sensor system for low-cost, low-complexity human activity detection system: In this work, we utilize multi-modality low-data-throughput sensors (thermal, acoustic, pressure, photon/laser) for human activity measurement. (2) Invented a compressive tracking system based on binary sensors: We have invented a compressive tracking framework using distributed binary sensors. We can achieve the minimum data throughput for an accurate multi-target tracking system through novel spatial sampling schemes. (3) Region of Interest (RoI) based Mobile Targets Scenario Recognition: We have invented a PSN (pyroelectric sensor network)-based mobile targets recognition system, which aims to achieve multi-target, complex scenario recognition. 2. Intellectual Merits Our significant findings through this research include the following three aspects: Finding 1: It is entirely feasible to build an accurate human behavior detection system via low-cost, low-complexity sensors: In our work, we have used multi-modality low-data-throughput sensors (thermal, acoustic, pressure, photon/laser) for human activity measurement. Among these sensors, thermal and acoustic sensors are passive; pressure and photon sensors are active. The thermal sensing modality are based on pyroelectric infrared sensors, which can detect the variation of thermal energy caused by human movements. The acoustic sensing modality are based on MEMS acoustic sensors, which can detect change of acoustic energies caused by speeches or other human activities. The pressure sensing modality are based on optical fibers embedded within floor mats. (2) to interface with FPGA boards we developed, (3) to change communication topology, and (4) to perform power management. Besides, high-resolution sensors such as Kinect and active 3D scanners are used for sensor calibration. The whole system setup is shown in Figure 1. Our system (Figure 2) is a distributed cognitive sensing platform, based on low-cost distributed sensors, which 1) can acquire human activity information with high efficiency and fidelity; 2) can achieve situation-aware system operation through agent-based cognitive intelligence. Finding 2: We can just use binary (0, 1) sensor network to achieve real-world multi-target tracking: We have designed a simple, binary sensor network for the compressive tracking. The sampling geometry design is equivalent to encoding the whole observation space into a set of codes. We choose the LDPC matrix as the compressive measurement matrix due to its well-known null-space property (NSP), which can preserve the structure of k-sparse source vectors. The decoding procedure of binary measurements is based on linear programming and Bayesian estimation. A binary sensor generates one bit of information to represent its detection result. Conventional geometries of a binary sensor’s FOV include a disk, a lobe, and a line, as shown in Figure 3. Figure 4 shows the satisfactory tracking performance with different sampling geometries and different compression ratios. Finding 3: We can significantly enhance the crowded scene recognition performance via context-aware situation awareness: Context is any information that can be used to characterize the situation of an entity. An entity could be a person, or an object relevant to the interaction between a user and a system. In our work, we use formal scenarios information and the signal intensity information as the context of latter scenarios. It should be noted that scenarios (like one human subject walking, two human subject walking, human A is walking, etc.) themselves are contexts to human tracking and identification in terms of data-target association and gait features extraction. We have designed a context-enhanced sensing scheme to overcome the drawbacks of conventional object-based sensing schemes. The snapshots of experiments are shown in Figure 5. By using sensor fusion techniques, our framework successfully identifies multi-target scenarios with a high recognition rate. 3. Broader Impacts (1) Publications: Our results have been published in a few high-quality journal papers, which disseminate our outcomes to the community. Here we list a few of our publications: Qingquan Sun, Fei Hu, Qi Hao, "Human Activity Modelling and Situation Perception Based on Fiber-Optic Sensing System," (to appear) IEEE Transactions on Systems, Man and Cybernetics: Systems. 2014. Qingquan Sun, Fei Hu, Qi Hao, "Mobile Targets Scenario Recognition via Low-cost Pyroelectric Sensing System: Towards an Accurate Context Enhancement" is accepted by IEEE Transactions on Systems, Man and Cybernetics: Systems. Vol 44, Issue: 3, pages 375-384, 2014. Qi Hao, Fei Hu, Yang Xiao, "Multiple Human Tracking and Identification with Wireless Distributed Pyroelectric Sensors," IEEE Systems Journal (special issue on Biometrics), vol. 3, no. 4, pp. 428-439, Dec. 2009. (2) Education: We have developed and offered a new course called "ECE 493/593 – Mobile Device Programming" based on this research. We have also guided over 20 senior students on their Capstone designs. They got familiar with the sensor hardware design as well as sensor-based human behavior detection.
