This cross disciplinary project for instrument development combines the fields of computer vision (and computer science overall) with radiation therapy and behavioral sciences. This project is a joint venture among the Medical School, the Psychology Department, and the Institute of Technology. The goal is to develop a minimally invasive, full-body, patient tracking system to be used with the helical tomotherapy system currently in use at the institution. The purpose of this effort is to detect when a patient becomes misaligned during radiation treatment. Adding visual feedback to the helical tomotherapy treatment should improve the safety of the patient, reduce the considerable treatment time, and improve the cure rate. It should be noted that the proposed instrument has a wide variety of implications for other forms of radiation therapy and medical systems that require a specific patient pose with respect to the medical device. In order for these devices to revolutionize the treatment field, the interaction of the patient with the device should be studied from a behavioral sciences point of view. In this project, behavioral sciences will play an important role in capturing the behavioral patterns exhibited when the patient is in treatment, and in devising new patient placement protocols with profound impacts on cancer treatment.
During the early stages of this development, the proposed instrument will not be used in human trials, focusing instead on the following tasks:
- Development of efficient computer vision algorithms (structured light) for detecting the patient's movements in a non-intrusive way that simultaneously offers the possibility for creating a closed-loop helical tomography instrument, - Selection of features, so cumbersome reflective markers are no longer needed, - Creation of the probabilistic framework necessary to utilize the features found from multiple calibrated cameras to determine the probability distribution of likely body positions, - Incorporation of behavioral science research that utilizes cyber-enabled principles to create patient-friendly medical devices and treatments, and - Experimental validation of the proposed instrument and improvement of the instrument with cyber-enabled behavioral science input.
Broader Impacts:
This project introduces computer vision methodologies/hardware and cyber-enabled behavioral science research to radiation therapy with the objective of having safer and more effective radiation treatment. It may cause fundamental changes in the way that humans and medical devices interact. Educational programs will enable training to the instrument/methodologies. Some of these methodologies will be included in the curriculum. A website will be developed for dissemination of findings. Users from around the world will be given access. Outreach programs involving demonstrations to pertinent groups will be created.
This project sudied many problems in the area of using computer vision to detect the movement of patients in radiation therapy machines. The first problem that was studied is the one of measuring the quality of camera and projector placement for structured light systems. Placement of sensors is important for all detection tasks as clever algorithms can be defeated by poor sensor placement, but good placement can lead to acceptable results even from subpar algorithms. Camera placement for observation tasks in multiple camera systems is an area that has been explored. However, the interaction between projectors and cameras in structured light systems is significantly different than the interactions in multiple camera systems, leading to different placement criteria. The quality of differing camera and projector placements was determined in this project by examining the physics of the problem. The spread of light from the projector into the scene and the uncertainty in the camera's detection of this light was modeled and accounted for. Taking this into consideration, judging the placement of cameras and projectors in such systems could be performed mathematically. One of the main areas where improved placement is applicable is tracking a patient's body position during radiation therapy, where small body movements direct radiation where it is not intended. To detect such movements, the cameras and projectors in a structured light system must be placed such that the body can be reconstructed with the desired precision. Another problem that was studied is the problem of calibrating the projector-camera system. Structured light systems operate on similar principles to stereoscopic camera systems, except that at least one camera is replaced by a projector. The 3D scene reconstructions generated can be considered as good as that constructed by a stereoscopic camera system. One central difficulty in using a stereoscopic camera set to perform scene reconstruction is that features in one camera's image have to be precisely matched to features in the other cameras' images. This can be difficult, and it is known as the correspondence problem. Structured light systems are often used instead of stereoscopic camera systems because by illuminating known patterns with a projector, the correspondence problem is greatly simplified. This project also developed a Euclidean calibration method for camera and projector systems in general scenes as the ones found in radiation therapy machines. This method is optimal in terms of the reprojection error of scene points. In addition, this method does not require manual human intervention to move specific occlusions so they intersect calibration patterns just right, or painstaking hand measurement of illuminated points. A set of simulations was run which characterizes the reprojection error of the method with respect to the level of corruption in the camera images. It was shown that the error in the camera and projector method is, on average, a little higher than the error for calibration-pattern based stereo camera calibration, but that its variance increases with increased image noise. Poor placement of the cameras and projector was shown to be an aggravating factor in these situations. The real-world efficacy of the method was demonstrated by calibrating a camera and projector system on complex and cluttered scenes commonly found in the helical tomotherapy environment.
