(List organised by order of presentation)
The interplay of language and speech-accompanying gesture
Sotaro Kita, University of Warwick
This talk concerns how speech-accompanying “iconic gestures” are shaped by language. Iconic gestures depict motion, action, and shape with hand movements. They represent referents based on similarity between gestural form and referents. One school of thoughts assumed that these gestures are generated from visuo-spatial imagery in the speaker’s mind, without any linguistic influences (e.g., Krauss, Che) I will provide evidence against the view that iconic gestures are generated from the interface between visuo-spatial thinking and linguistic thinking. In particular, I will show evidence that content of gestures are shaped by linguistic details in concurrent speech in two studies. First, I will present evidence that gestural contents are influenced by what lexical resources and syntactic structures are readily available in a given language (Kita & Özyürek, 2003; Mol & Kita, 2012). One key piece of evidence concerns how manner and path of motion events are represented in speech and gesture. Second, I will present evidence that gestural contents are influenced by how language expresses the relationship between events with different types of clause linkage (e.g., “X happened, and then Y happened” vs. “Once X happened, Y happened) (Debreslioska & Kita, in preparation). These results indicate that gestures do not reflect imagery free of language; they reflect imagery that are shaped for the purpose of speaking, that is, imagery shaped by thinking-for-speaking (Slobin, 1987).
Fundamental multimodality: The visual mode of language
Mandana Seyfeddinipur, Endangered Languages Documentation Programme, SOAS
The study of language began with the study of ancient written text. This practice has shaped linguistics and is still shaping our practices in linguistics, language documentation and in psycholinguistics. The way we conceptualise language and language use has implications for our research practices. We still focus on what we think can be written down and often disregard what we think cannot be easily written down.
But, typically, when we speak, we cannot only hear each other but also see each other. The evanescent nature of language use is implicitly seen as a limitation to our object of study. However, modern technology in video recording allows us to overcome these perceived limitation. It allows us to collect and analyse linguistic data which can answer important questions about for example language evolution, linguistic diversity and universals and social grounding of multilingualism.
Language is grounded in face-to-face interaction and speaking is a joint activity (Clark 1996). Language acquisition is a process that takes place in face-to-face contexts and our cognitive system automatically integrates both, what we hear and what we see (McGurk & McDonald 1976). When we speak, we use our hands to gesture and the information provided in this visual, gestural modality is also integrated automatically in our mind. The gestures we use contribute crucially to our understanding of what speakers are communicating (Kendon 2004). Communities have developed alternate sign languages used in e.g. mourning practices (Kendon). Deaf people develop fully fledged sign languages in the manual modality (Meir et al. 2012).
However, despite this basic multimodal nature of language use we often still do not document language use to its full extent due to restricting our recordings to audio or restricting video recording to a few genres like story telling. In this talk I will exemplify the multimodal nature of language use, focusing on manual gesture in its various forms and functions from indexing to semantic specification, and discourse structure marking.
I will discuss its implications for research practices involving linguistic data. The role of video recording and the way language use needs to be video recorded to provide useable material for linguistic and ethnographic documentation and analysis will be highlighted. A methodology for training the much needed video recording will be suggested which embeds the technical training of video technology and recording within a theoretically grounded understanding of language use.