Detection and analysis of branching structures and/or texture is very challenging; it arises in many areas of science and engineering (e.g., medical images, chemical compounds, etc). The objective of this proposal is to develop novel approaches to model, detect, and analyze branching structures obtained from multimodality data. Such representation and analysis tools are expected to make many complex problems more tractable. Examples include identifying and recognizing a large number of structure classes; discovering new relationships among structure, texture, and function or pathology; evaluating hypotheses; developing modeling tools; assisting with surgical design; and managing medical image data efficiently. Specifically, the investigators plan to explore three research topics under this project: (1) To develop descriptors of branching structures and texture, and knowledge discovery tools that will enable hypotheses generation and evaluation and improve modeling of branching structures; (2) To design automated algorithms and a flexible framework to detect branching structures. The investigators are especially interested in addressing challenges of occlusion and topology change; (3) To demonstrate the applicability of the proposed tools to breast imaging by building a prototype database of images from various modalities and associated clinical data that will provide advanced analysis and visualization capabilities. Though the investigators use breast imaging as the driving application, the proposed project is expected to provide software and data resources that can assist clinical tasks and scientific discoveries in general. Developing automated tools to effectively characterize, detect, and classify tree-like structures in images would provide great insight into the relationship between the branching topology and function or pathology. The investigators plan to further contribute to the medical/scientific community by disseminating the related software and annotated data sets. The educational goals include incorporating research findings to graduate courses at Temple (data mining course and medical image analysis seminar) and at the University of Pennsylvania (medical image analysis course).
Detection and analysis of branching structures and/or texture is very challenging and arises in many areas of science and engineering (e.g., medical images, chemical compounds, etc). In this project, novel approaches have been developed towards modeling, detecting, and analyzing branching structures obtained from multimodality data. The project has produced research outcomes mainly from three aspects (generating over 30 scientific publications): (1) Novel and effective tools have been investigated and developed for exploring branching structures. In particular, descriptors have been invented for describing branching nodes (hybrid local image feature, texton-like representation, etc.), branching texture (bag-of-visual words representation, multi-fractal spectrum texture descriptor, icosahedron histogram of oriented gradient, etc), and branching topology (extended Prufer coding). Learning-based tools have been integrated for further exploration of these descriptors, such as the use of histogram intersection kernel for texture analysis. (2) Effective algorithms have been designed for automatic branching structure detection. Such algorithms include the ensemble learning-based branching ROI (region-of-interest) detection and branching node detection, conditional random field for branching pattern segmentation, and random forest-based vessel segmentation. (3) The developed technologies have been applied to real world medical image analysis tasks. In particular, mammographic and galactographic images from a breast cancer study have been analyzed, lung CT images have been analyzed to discover disease and normal patterns, and dental radiograph and CBCT images have been investigated by applying trabecular structure analysis. Among the many results of the project is also a dataset of synthetic (but realistic looking) mammographic images that has been generated. These synthetic images can be used to test various algorithms prior to starting clinical trials as a cost and time effective alternative. The availability of representation and analysis tools such as those developed in this project is expected to make many difficult problems more tractable. Examples include recognizing a large number of structure classes, discovering new relationships among structure, texture and function or pathology, evaluating hypotheses, developing modeling tools, helping in surgical design, and enabling efficient management of medical image data. In particular in the breast or lung imaging domain, problems such as understanding anatomy and physiology, estimating cancer risk, assisting diagnosis, improving computer simulation, will become easier to attack. The work performed in the project facilitates the process of medical decision making by providing tools for automatic extraction of the most discriminative features of regions of interest in medical images of various modalities and efficient retrieval of similar regions in large collections of such images. It enables researchers to integrate, manipulate and analyze large volumes of image data conducting large-scale epidemiological trials of many afflictions. Analysis of image-based clinical trials using these tools facilitates advances in both diagnosis, and treatment and provides new insight into the relation of anatomy and function. I addition to the contributions already presented, the techniques and tools developed in this project also generate impact to other related fields. The multi-fractal texture algorithm has been used for general texture analysis involving non-branching patterns. The learning techniques used in the texture analysis have been applied in computer vision tasks including visual tracking and category classification. Relevant results can be found in the publications generated from the project. The project has also supported several educational activities of its team members. Two regular courses (CIS 8543: Computer Vision and CIS9664 Knowledge Discovery and Data Mining) involving medical imaging analysis topics and one graduate seminar on Management and Analysis of Biomedical Data have been taught at Temple. The project involved in total one postdoc, six PhD students, one MS student, and four undergraduate students (including two female students and one under-represented minority student, three of which were supported by REU). In addition, the Co-PI has served as a STEM Faculty Mentor for the ``Scientists as Teachers; Teachers as Scientists" program to supervise one PhD student and one high school teacher for the joint college-high school education activity. This project has contributed in the development of a unified multidisciplinary curriculum in data mining and knowledge discovery with a focus on improving diagnosis and medical decision making as well as on discovering new medical knowledge attracting students from multiple fields and promoting interdisciplinary learning.
