There is a tension in the cognitive sciences between the view that different cognitive systems have qualitatively different structure and are specialized in their nature (e.g. Fodor,1983), and the view that these systems normally interact in order to produce higher-level knowledge. Consider the case of language and vision. Although research suggests that these systems encode different types of information, it is also true that vision & language interact allowing us, e.g., to talk about what we see (Jackendoff, 1987,1996). Furthermore, there is empirical evidence showing that language and vision interact (e.g. in developmental research, e.g. Larissa & Smith, 2005; Lowenstein & Centner, 2004; in visual memory research, Stefurak & Boynton, 1986; Alvarez & Cavanagh, 2004). But, what is the mechanism by which language and vision interact? Specifically, how does the presence or absence of linguistic labels (Samuelson & Smith, 2005; Lowenstein &Gentner, 2004), or denying access to language (e.g. Stefurak et al., 1986) visual processes? Previous work (Dessalegn & Landau, 2005), has shown that directional linguistic labels (e.g. """"""""the red is on the left"""""""") help children form a durable representation of a visual target (including the binding of color and location), reducing the rate of confusion between a target and its mirror image.
The first aim of the proposed project is to test a specific hypothesis about the mechanism by which language enables children to encode and maintain an accurate representation of a visual target. Dessalegn & Landau have also shown that there is a developmental aspect to the interaction between language and vision. While 4-year olds do not automatically use language to form a durable representation of a visual target (although they are able to use language if the experimenter provides appropriate linguistic labels), adults cannot help but use language. Thus, the second aim of the proposed project is to study the development of the automatic language-vision interaction observed in adults by testing 3-year olds, 6-year olds, 9-year olds and adults. Understanding how language and vision interact, and the development of this interaction, is crucial in solving many educational and health-related concerns. As an example, consider abnormalities that affect an individual's visual abilities while leaving language abilities intact (e.g. people with the genetic disorder Williams Syndrome, Mervis et al., 2000). Understanding how language and vision interact will shed light on ways we can use language to completely or partially bypass visual difficulties in this group. ? ? ? ?
| Dessalegn, Banchiamlack; Landau, Barbara; Rapp, Brenda (2013) Consequences of severe visual-spatial deficits for reading acquisition: evidence from Williams syndrome. Neurocase 19:328-47 |
| Dessalegn, Banchiamlack; Landau, Barbara (2013) Interaction between language and vision: it's momentary, abstract, and it develops. Cognition 127:331-44 |
