Recent advances in miniature camera technology have lead new generation endoscopic devices called capsule cameras [1~5] which use a swallowable pill-size miniature wireless video sensor to acquire and transmit video image sequences while traveling along the gastrointestinal(GI) tract. Although this revolutionary technology offers patients with painless examination experience, it however only provides two dimensional (2D) sequential video images that contains no three dimensional (3D) information of observed targets. Recognition and evaluation of pathological structures and the estimation of their spatial dimension can only be achieved by experience, making a diagnosis decision very subjective. For daily clinical usages, the unprecedented huge data set (more than 57,600 raw 2D images per exam) requires in average 45 minutes of physician's time to review. Locating the spatial position of a specific target with respect to patient's body is fairly difficult task due to lack of 3D information. The primary objective of this SBIR program is to develop a 3D image processing computational software platform, dubbed as the """"""""Video-to-3DTM"""""""", which is able to (automatically) produce, for the first time, an integrated, patient-specific, and quantitatively measurable gastrointestinal (GI) tract 3D model based upon the 2D video sequence acquired by a capsule camera during an exam. The ability to perform quantitatively 3D measurement of pathological structures with calibrated color texture would make endoscopic diagnosis more objective and reproducible. Furthermore, an integrated 3D model of patient-specific GI tract and 3D fly-through visualization software capability would assist diagnosis and intervention planning, save physicians tremendous time in video reviewing, and provide accurate 3D intra-body localization of """"""""target of interest"""""""" with respect to patient's body. Since a capsule camera moves freely inside the GI tract, its 6-degree-of-freedom motion is neither controllable nor accurately measurable. We therefore need to develop sophisticated computational efficient algorithms to obtain a very robust estimation of the camera motion from the uncalibrated image sequence, and then reconstruct 3D structure using the known camera motion and other 3D image processing techniques. The """"""""Video-to-3DTM"""""""" Software Platform is designed to carry out the following tasks: (1) 3D Modeling: Inter-correlate over 57,600 images acquired by any capsule camera to reconstruct a high resolution patient-specific 3D model of GI tract; (2) 3D Visualization: Provide texture super-resolution and 3D fly-through capability for the 3D GI tract model to help physicians to visualize and diagnose quickly, interactively, accurately, and efficiently; (3) 3D Measurement: Perform quantitative 3D measurement of interested pathological structures; (4) 3D Localization: Determine accurate 3D intra-body location of targets within patient's body. (5) Computer-aided diagnosis: The ultimate goal of the proposed Video-to3D system is to be able to assist doctors to detect, classify, and identify certain targeted diseases. The proposed """"""""Video-to-3DTM"""""""" adds one more dimension to the existing 2D capsule camera technology, literally and figuratively.
Specific aims of Phase I effort to build the proposed Video-to-3DTM software include:
Aim 1 : Design software architecture for the Video-to-3D computational platform;
Aim 2 : Develop and optimize algorithms and software components;
Aim 3 : Perform extensive tests using simulation platform and images acquired by in-vivo capsule cameras;
Aim 4 : Assess Phase 1 system performance and prepare for Phase II work plan.
We propose to develop a 3D image processing computational software platform, dubbed as the """"""""Video-to-3DTM"""""""", capable of producing (automatically) an integrated, patient-specific, and quantitatively measurable gastrointestinal (GI) tract 3D model based upon the 2D video sequence acquired by a capsule camera during an exam. The proposed Video-to-3DTM software platform blazes an entirely new trail in the in- vivo capsule camera technology by providing, for the first time, the unprecedented capability of 3D GI tract modeling, 3D fly-through visualization, 3D measurement of pathological structures and 3D intra-body localization of target area with respect to patient's body for diagnosis and intervention planning. The proposed """"""""Video-to-3DTM"""""""" adds one more dimension to the existing capsule camera technology, literally and figuratively. Given the expanding needs in biomedical imaging research, the proposed software platform technology is ultimately generalizable, scalable, extensible, and interoperable. In addition to produce leapfrog performance advances for the revolutionary capsule camera technology and many other existing medical diagnosis instruments (endoscopes, gastroscope, cystoscope, colonscope, etc.), potential applications of 3D model reconstruction technology developed herein span a broad spectrum of many fields of applications, such as pipe inspection, turbine engine diagnosis, smart transportation systems, security and surveillance, vehicle navigation, mobile robotics, teleconferencing, virtual reality products, on-line distribution of live images, event and location viewing, urban modeling, entertainment, sports webcasts, and many others. ? ? ?
