This paper examines the interactive embodiment and formation of ideas among group members during brainstorming sessions. Drawing from the findings of micro-analytic studies of interaction, this study investigates the roles of talk and gestures in the process of proposing ideas, accepting or declining them, elaborating upon or modifying them, and integrating them in order to achieve agreement concerning the decisions the members arrive at.
A group meeting among three college students working to create a short movie for a class project is examined. During their meeting, they brainstorm the ideas for the project and compile them together to design each scene of their movie. The goal of their interaction is to reach agreement on the way to set up their project and film each scene. The three main components that they discuss are the movements of the characters, the angles and positions of the camera, and the background scenes and settings. By switching roles between the cameraman and the characters as well as describing the background settings, they imaginatively create the movie scenes that they plan to film and bring the proposed ideas into shape. Interestingly, the group members do not draw a picture or use visual representations to describe what they imagine. Instead, their ideas, which exist in their head, are made visually available for others to comment on, elaborate upon, or (dis)agree with through gestures in combination with talk and other semiotic means. The present study attempts to explicate what such a process reveals about gestures in a collaborative activity.
1.1. Gesture and depiction in interaction
A growing number of studies on gestures deployed during conversations have revealed their importance in organizing interaction. Studies focusing on interaction have demonstrated that gesture is not a mere externalization of one's psychological state or by-product of speaking. Instead, the synchronization of gesture and speech makes an ongoing activity a publicly observable practice and furnishes the opportunity for others to organize their coordinated entry into the activity (e.g., Goodwin, 1995, Goodwin, 2003a, Goodwin, 2003b, Goodwin, 2004, Goodwin and Goodwin, 1986, Hayashi, 2005, Heath, 1986, Heath, 2002, LeBaron and Streeck, 2000, Mondada, 2007, Murphy, 2005, Schegloff, 1984, Streeck, 1993, Streeck, 1994, Streeck, 2009a, Streeck, 2009b and Streeck and Hartge, 1992). For example, Goodwin and Goodwin (1986) show that vocal and visual displays of one's inability to find a word (such as pauses or a “thinking face”) and related gestures that indicate the word's meaning not only demonstrate the kind of activity in which the speaker is currently engaged (a word-search in this case), but also elicit others’ co-participation in the activity.
It is also important to note that a gesture can be perceived and processed even when produced in overlap with others’ speech. Streeck and Hartge (1992) and Mondada (2007) illustrate gestures’ function of projection. Speakers, by employing gestures, can claim their turn to speak and/or pre-indicate the type of an upcoming activity before others finish speaking so that other participants can coordinate their next actions accordingly. Gesture, through its feature of making an ongoing or upcoming action visible, is a social practice that enables coordination among participants.
Among the various modes of gestural activity, the most relevant to the present study is its function of depiction or representation. According to research on gesture taking a psychological perspective, “iconic gestures” (McNeill, 1992), which provide images to the semantic content of co-occurring speech, reflect the speaker's viewpoint (McNeill, 1992 and McNeill, 2005). It is pointed out that, unlike standardized gestures such as “emblems” (Ekman and Friesen, 1969), which are used independent of speech, iconic gestures are interpretable only in combination with speech. While this psychological approach focuses on the mental processing of a single speaker's gesture without looking at the kind of actions that gestures accomplish, researchers focusing on interaction demonstrate how “depictive gestures” (Streeck, 2009a) can become a resource for organizing interaction (LeBaron and Streeck, 2000 and Streeck et al., 2011). They reveal that depictive gestures embody a real or imaginary object or person, knowledge, thought, emotion, pain, or imagination, and also make them “handle-able” (Streeck, 2009a:149). In other words, they serve as a local resource to enable co-orientation to and/or co-participation in an ongoing activity; they are a social practice rather than a mere representation of one's mind (e.g., Heath, 2002, Koschmann and LeBaron, 2002, LeBaron and Koschmann, 2003, LeBaron and Streeck, 2000, Murphy, 2005, Streeck, 2009a and Streeck, 2009b). For example, Murphy (2005) shows how three architects use talk, gesture, and material objects to collaboratively imagine a scientific laboratory building for a university. Since the employment of gestures and other communicative resources makes one's imagination visible in the shared space, the architects can add to or comment on one another's imagination and co-construct an imaginary building.
Additionally, unlike a psychological approach, which analyzes the meanings of a single speaker's gestures in relation to speech, research focusing on interaction demonstrates that a depictive gesture alone does not provide its meaning; rather, talk, gesture, other semiotic resources, surrounding environment, and ongoing activity in interaction contextualize each other to constitute an action. It is thus crucial to understand what a gesture does through a close examination of its coordination with other resources. It is also important to note that a gesture is understood in relation to the environment (“material world”) in which it is used, as well as the activity in which it is embedded (Streeck et al., 2011). The shared knowledge among participants in a given environment during a specific activity makes the use of gesture and talk a meaningful action. Examining an instructional situation in which a teacher explains the use of tools, LeBaron and Streeck (2000) show how the meaning of a gesture is understood based on the shared experiential knowledge constituted through the previous interaction. The hand movements of a teacher showing how to use a tool while actually handling it become shared knowledge between the teacher and the students, enabling them to understand what the hand movements represent even when the teacher is not holding the tool. Note that depicting by gestures is not simply “copying” (Streeck, 2009a), but “characterizing”; it is how the teacher's hands actually handle the tool, namely, his experience with the tool that his hand movements depict, rather than what the tool looks like.
Taking this latter approach to gesture, I investigate how talk, other semiotic resources, surrounding environment, and the type of ongoing activity contextualize gestures in interaction. It is useful to note that the participants in the present data build shared knowledge as they describe their imagination by coordinating gesture and talk, as well as use the shared knowledge as they understand the meaningful gestural movements in relation to talk. By looking at the reuse of prior gestures, I thus illustrate both processes: how the employment of gestures reveals the emergence of shared knowledge and how the shared knowledge contextualizes gestures in interaction.
1.2. Repetition of gestures
In the present data, similar gestures or gestural forms are deployed repeatedly by different speakers. This study thus specifically focuses on how gesture repetition contributes to the processes of collaborative brainstorming for a group idea. McNeill (1992), from a psycholinguistic perspective, discusses that repetition of the same gesture can signal continuity – a cohesive connection of themes among temporally separated segments in the discourse. Recurrence of two or more gestural features of shape, space, orientation, movement, and so on across a stretch of discourse forms a larger discourse unit termed catchment (McNeill et al., 2001). A catchment represents both semantic and visual information and offers cohesive linkages of meaning across different discourse segments. That is, a catchment reveals what parts of the discourse a speaker regards as similar and groups together. Even without verbally explicating it, therefore, a gesturer can indicate the connection of a meaning across different parts of a discourse through the repetition of the same gesture.
While a catchment applies to the reproduction of gestural features only by a single speaker, research focusing on interaction has investigated the repetition of gestures across multiple speakers in conversation. Koschmann and LeBaron (2002) examine how speakers use gestures to describe what they know, and demonstrate that reuse of the gestures previously produced by others can indicate a link back to the previous sequence. A speaker can thus connect the current discussion to a prior one by reproducing a previous gesture. Murphy's (2005) study on architects’ collaborative imagining, mentioned earlier, also demonstrates that the participants reproduce previous gestures to refer to earlier parts of interaction. By repeating previous gestures, they can indicate the connection between the new suggestion and the imagination already constructed. These studies indicate that repetition of the same gestures by multiple speakers not only indicates semantic cohesion across turns at talk but also ties together different actions in different sequential positions in an unfolding interaction.
The present study attempts to extend the previous findings regarding the functions of gesture repetition by investigating different patterns of repetition and their relationship to the ongoing activity. I first investigate what gesture repetition accomplishes during joint idea construction. In addition, I analyze how different forms or patterns of repetition occur. In the present data, while the participants often repeat each other's gestures, they do not always reproduce a previous gesture in exactly the same way; sometimes, gestures are repeated in a modified form. This study, therefore, investigates the relevance of gesture repetition to the joint idea-shaping processes, as well as different ways in which gestures are repeated (full, partial, and modified repeat).
The group members in the present data employ multiple communicative resources, mainly talk and depictive gestures, during joint brainstorming in which they describe their ideas concerning the details of movie settings, character actions, and camerawork. What the participants do with talk and gestures is a depiction of movie scenes, which they intend to create. Talk and gesture not only make something yet to exist accessible to the participants but also embody the process of idea construction – the repeated process of proposing ideas, accepting or declining them, solving a problem in them, elaborating on or modifying a part of them, and integrating them into a larger whole. Coordinated employment of talk and gesture makes the activity of constructing a group idea an interactive accomplishment.
This study has examined how gesture repetition is employed as an interactional resource during a joint activity. LeBaron and Streeck (2000) demonstrate that gesture repetition marks what is represented by the gesture as having become shared knowledge within an unfolding activity. In the present data, as the group members jointly form imagined movie scenes, a gesture produced with a proposal becomes shared knowledge among the participants and is used as a local tool to refer to (an aspect of) a previous proposal. Gesture repetition thus allows a speaker to point to a previous context in which the gesture has been produced. Note that a gesture's meaning does not always remain the same throughout the interaction. A gestural meaning is contextualized through talk, ongoing activity, and shared knowledge, and can be developed throughout the successive reuse of the gesture. For example, Godzilla's “hitting” gesture originally produced by Brad in Excerpt (2) means “Godzilla hitting a restroom door” when produced after Doug depicts a restroom door, and “Godzilla hitting the restroom door of a Mexican restaurant” after Hugo's suggestion of using a picture of a Mexican restaurant.
By referring to a previous context through repetition of a prior gesture, one can create connections among different sequential pieces of talk, as Koschmann and LeBaron (2002) discuss. Speakers in the present data reproduce a prior gesture when proposing something new to point back to a previous proposal and to signal that an unfolding proposal is based on the previous one. Therefore, repetition of a prior gesture across turns can visually display the connection among successive proposals during the course of group idea construction; gesture repetition shows how aspects of multiple proposals are put together by multiple speakers as well as how a part of a prior proposal is modified or elaborated through interaction. One essential role of gesture repetition during collaborative idea construction is thus to create coherence across multiple proposals.
More importantly, the analyses also reveal that differences in how a gesture is reproduced lead to different actions. On the one hand, participants can indicate acceptance of the previous speaker's idea, a cooperative move, by fully repeating the gesture produced when it is first proposed. In Excerpt (1), the participants construct a group idea by repeating entire gestures from one another as they accept each other's suggestions and add new ideas. A cooperative process of fully accepting each other's suggestions is made visible through the full repetition of prior gestures. On the other hand, a partial or modified repetition of a previous gesture signals something different from the simple acceptance of a previous proposal: while the repeated feature of a previous gesture represents what part of a previous proposal is accepted, modification of the gesture reveals what exactly the speaker does not accept as well as what he suggests alternatively. Since most parts are the same, the differences can easily be detected. In Excerpt (2), for example, the second proposer produces the same gestural format as the first proposer but changes one of its features. He does so before the first proposer finishes his turn, starting to move his hands and producing a different right-hand movement, overlapping with the first proposer's gestural movement. In doing so, he indicates that his proposal is largely based on the previous one, but at the same time, highlights the part he proposes independently from the previous proposal. In Excerpt (3), the participants extract a feature of the first proposer's gesture and reproduce it, demonstrating that they accept a part of the first proposal. However, at the same time, they verbally suggest an alternative idea as they modify the first proposer's gesture in a way that can fit the newly suggested idea. A modified or partial repeat of a gesture, therefore, allows them to perform both cooperative and competitive moves simultaneously. Hence, while displaying attentiveness to a prior proposal and showing what they accept, the participants can also implicitly indicate non-acceptance as well as their new suggestion. During brainstorming, suggestion of an idea makes accepting or declining a relevant next action. While acceptance is a preferred response (Pomerantz, 1984), repetition of others’ gestures allows a speaker to perform a dispreferred action (non-acceptance in this case) in an implicit way; non-acceptance can be embedded in a form of partial repetition of a prior gesture. Thus, gesture repetition during an activity of collaborative idea construction can both manage social sensitivity when performing a dispreferred action and contribute to group construction of ideas.
The present study thus extends the previous findings on the functions of gesture repetition during a collaborative activity among multiple participants; gesture repetition is more than a local tool for referring to a prior segment of talk. In collaborative idea construction, wherein group members aim at cooperating with each other and reaching a mutual agreement, there is also competition regarding whose idea will be accepted. The ways gestures are repeated can thus reveal both cooperation and competition among the participants.
