The world contains not only objects and features, but also relations holding between them. When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the visual appearance of the individual objects (e.g., a red apple, a wooden bowl), but also the relations "containment" (in) and "support" (on). A surprising number of everyday tasks depend on these and other relational representations, such as assembling furniture, packing a suitcase, reading a medical chart, or navigating a scene. How does the mind represent visual relations themselves, beyond the objects participating in them? The work proposed here explores an exciting hypothesis about how the mind extracts relations from images ?? namely, that relations are properly *perceived*, in a fast and automatic manner akin to the perception of more familiar visual properties such as size, color, or shape. Across multiple case studies ?? including the perception of fit, balance, containment, support, adhesion, enclosure, and more ?? this work will take a psychophysical approach to the perception of relations, asking whether relational perception proceeds rapidly, automatically, reflexively, and in ways that interact with other perceptual processes.
The work proposed here has three primary aims: (1) To characterize the kinds of relations we perceive; (2) To understand how such relations are extracted by the mind; and (3) To elucidate their function in the mind at large. First, what kinds of relations can we see? Previous work has focused mostly on basic geometric and spatial relations (such as being beside, above, or behind); but objects in the world are related to each other in far richer ways. Here, the investigator will catalog the kinds of relations that appear in perception, with a special focus on ?force-dynamic? relations, including combining, containing, supporting, balancing, covering, tying, connecting, hanging, and other relations in which objects exert physical forces on one another. Second, do we only consider, judge, or infer the relations between objects? Or can we also see them directly? The next aim of this proposal is to investigate the nature of relational perception, by asking whether the mind extracts visual relations in ways that show signatures of genuinely perceptual processing, such as speed, automaticity, reflexiveness (or cognitive impenetrability), and interaction with other perceptual processes. Third, why do we perceive relations at all? Are they just curious quirks of the mind, or do they support other kinds of knowledge? For example, once we see that two objects are connected, do our minds automatically predict that tugging on one will bring the other along? Once we see that two object-parts can combine into a whole, is it easier to remember or count such objects? The final aim of this project is to explore how relational perception supports other sophisticated kinds of mental processing, including automatic prediction of physical contingencies between objects.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
