This project will develop novel photoacoustic ultrasound imaging techniques to enable accurate, reliable, and efficient fusion of intraoperative ultrasound (IOUS) images with preoperative images, real time video, and other data to assist surgeons in minimally invasive surgery and other interventional procedures. Ultrasound provides many advantages for intraoperative use, and interest in using IOUS in both open and laparoscopic surgery, in providing guidance for biopsies, tumor ablation, brachytherapy, and robotic surgery is growing. One significant factor limiting the use of IOUS is the necessity for the surgeon to relate the information in the ultrasound images to preoperative information such as CT or MRI images and to what he or she is seeing in a laparoscopic video monitor or in direct visualization. Computer-based information fusion and visualization systems have significant potential to overcome this limitation, provided that they can 1) reliably register IOUS images to the surgical scene as observed, e.g., by laparoscopic video;2) continuously register IOUS and video to preoperative data;and 3) track surgical tools in IOUS images. Current systems typically relying on optical or electromagnetic (EM) navigational trackers have several limitations, including 1) system complexity;2) accuracy limitations;3) line-of-sight restrictionsor the presence of extra wires in the surgical field.
Our specific aims for this R21 feasibility projct are as follows:
Aim 1 : Real-time fusion of IOUS and video images: Develop and evaluate innovative photoacoustic methods for real-time, continuous registration of intraoperative US to video camera coordinate systems.
Aim 2 : Real-time tracking of registered preoperative data: Demonstrate and evaluate real-time, continuous tracking and display of registered preoperative models in IOUS and video camera coordinates.
Aim 3 : Real-time identification and tracking of interstitial tools in IOUS: Develop illumination and processing methods for generating PA signals on interstitial tools. Demonstrate and evaluate the use of these methods for locating the tiny tines of a typical probe used in RF ablation of tumors in solid organs. Our basic hypothesis is that many of these limitations can be overcome by the novel application of intraoperative photoacoustic (PA) imaging methods to permit direct, real-time localization of fiducial points in ultrasound images. In PA imaging, short pulses of laser light are converted into acoustic signals that can be detected in ultrasound images. If multiple laser spots are projected onto an organ surface, they may be located both in video and in ultrasound, thus permitting direct registration of IOUS-to-camera coordinates. Similarly, PA imaging may be used to locate small, embedded markers inside tissue to permit real-time tracking of registered preoperative images and models relative to IOUS. And it may be used to locate points on interstitial surgical tools and probes, thus permitting direct registration and tracking relative to IOUS. We will evaluate the technology developed in artificial &ex vivo phantoms and in a small in vivo pilot study.

Public Health Relevance

This project will develop novel ultrasound imaging techniques for minimally invasive surgery and other interventions such as biopsies, tumor ablation, brachytherapy, and robotic surgery. These techniques will enable accurate, reliable, and efficient fusion of intraoperative ultrasound (IOUS) images with real time video from endoscopic cameras, preoperative CT or MRI images, and other data to assist surgeons in performing surgery more accurately, quickly, and safely. Although this project uses resection of kidney and liver tumors and minimally invasive ablation of liver tumors to provide immediate focus, the techniques developed will be broadly useful across a wide range of clinical applications.

National Institute of Health (NIH)
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Exploratory/Developmental Grants (R21)
Project #
Application #
Study Section
Biomedical Imaging Technology Study Section (BMIT)
Program Officer
Krosnick, Steven
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Johns Hopkins University
Engineering (All Types)
Schools of Engineering
United States
Zip Code