This proposed project will develop a framework for organizing images that allows the specific types of relationships between those images to be represented, manipulated, highlighted, enhanced, and studied. The technical challenges involve building new representational and algorithmic systems to capture the major "longitudinal categories" that relate heterogeneous images to each other within collections. The problem of relationship between images is normally posed through registration, which is most often highly contextualized. This work will capture the steps necessary to specify registration as a metadata construction that enables a range of granularities in mapping images to each other, and heterogeneous relationship across organizational categories such as time (diachronic), multi-modal, and instances related by a semantic object. The work is highly interdisciplinary and results can be generalized to other problem sets.
The Project An image of a rare book or ancient plant specimen can be interesting; with more than one image, we can compare and learn even more. This project developed techniques for comparing large libraries of images. Any image might be interesting to different researchers for different reasons: What can we learn about the ink on this medieval manuscript? What can we learn about the language of the text in the manuscript? Can we read badly faded text? Can we identify the genus and species of a preserved botanical specimen? Can we trace the history of comments written on the page where the specimen is mounted? Each different way of looking at an image is an axis. This project developed techniques for studying images according to the many axes that different kinds of scholars, addressing different kinds of problems, might want to investigate. Medieval manuscripts are often in terrible condition—wrinkled, faded, damaged by water and insects, faded with time. An important axis of study is change over time. Another important axis is the image of a text on a manuscript and a transcription of that text. For images of physical objects, researchers may need to make measurements, to determine for example the genus and species of a 300-year-old plant specimen; if the object or the image of the object is distorted, accurate measurement is impossible. And the most important axis, as the number of images increases by orders of magnitude, is the axis of organization, citation, and organization: if we have 11,000 images of a single ancient book, how can we easily create, maintain, and share links among them all, knowing which image we are looking at, but also knowing that there are 24 other images of the same page, and that the page is one of hundreds, and each of those has many images, too. Each project is different, but our goal was to create a framework of techniques, workflows, and methods of citation that allows any image-based project to organize, align, and retrieve information along whatever axes of comparison are interesting for that project. Intellectual Merit We captured over 20,000 images of ancient manuscripts—a Latin edition of the New Testament from the 8th century, and a Greek edition of the HomericIliad from the 11th century—photographed in black-and-white under different wavelenghts of light. We developed software to combine those B&W images into full color images, not only to show a natural view of the pages, but also to highlight information in "false color" that is otherwise invisible. Vellum manuscripts wrinkle with time, and this distortion can prevent accurate analysis and measurement. We developed techniques for capturing a 3-dimensional model of pages, aligning that with 2-dimensional images, and thus "digitally flattening" the pages. To take perfect digital photographs of rare books and objects is extremely difficult, requireing elaborate setup, equipment, and lights; this is often impossible. It is often possible to correct less-than-perfect images in software, but this is impractical with a collection of many thousands of images, each of which has its own unique pattern of distortion. We developed techniques for using contextual clues in the image to correct for inadequate camera placement, automatically, across a large image library. Scholarship depends on citation, and a good citation is concise and independent of technology We have exposed our many thousands of images through a networked service that allows discovery, identification, and retrieval of images and specific parts of images through concise citations that do not need to change as technology changes. This includes tools that make it easy to create citations to particular parts of an image, "regions-of-interest". Broader Impact This was a computer science project, but also a collaboration with scholars of ancient language, ancient manscripts, history, and botany. Any discipline—science, art, humanties—works with images, needs to cite images, and wants to take advantage of the power of modern technology to see more deeply and broadly. Links The St. Chad Gospels. Background, multispectral image-alignment. Lichfield Biblical Manuscripts. Semantic alignment (image-data with textual content). Mark Catesby’s Natural History. Cross-domain alignment fed into a publishing workflow, aligning herbarium specimens with page-images from an 18th century book of illustrations. Digital Library of Botanical Images. Generated based on canonical citation, with links to the "image-citation tool" for capturing region-of-interest data as URNs.
