This research project employs a knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs. Selected objects of interest in a medical image (e.g. x-ray, MR image) are segmented, and contours are generated from these objects. Features (e.g., shape, size) and content (e.g., spatial relationships among objects) are extracted and stored in a feature and content database. Knowledge about image features can be expressed as a hierarchical structure called a Type Abstraction Hierarchy (TAH). A knowledge-based semantic image model is developed which consists of four layers (raw data layer, feature and content layer, schema layer, and knowledge layer) to represent the various aspects of an image objects' characteristics. The model provides a mechanism for accessing and processing spatial, evolutionary, and temporal queries. A knowledge-based spatial temporal query language (KSTL) is formed which supports operators for approximate matching via feature and content, conceptual terms, and temporal logic predicates. Further, a visual query language is also developed that accepts point-click-and-drag visual iconic input on the screen that maps into KSTL. The results from this research are applicable to a variety of multimedia information systems.
