The kineikonic mode is a term used to denote the moving image as a multimodal form. It is derived from the Greek words kinein, to move (also part of the origin of ‘cinema’), and eikon, image. It was coined by Burn and Parker (2001) to avoid the problems of the usual words used within cinema and film, both of which privilege these specific cultural forms over, for instance, television. It was also intended to explicate a multimodal theory of the moving image which combined the emphasis on the semiotic ‘grammar’ of film established by theorists such as Christian Metz with an attention to the signifying features of contributory modes such as speech, dramatic gesture, music, space, lighting, costume. These modes are seen as orchestrated by the framing and ordering functions of filming and editing. Properly speaking, then, the kineikonic mode is a multimodal ensemble: it contains both the modes themselves as well as the interplay of those modes as they move through time and space in a moving image. Using a term that places the modes into the single mode of kineikonic emphasizes the governing role of what Metz called the ‘cinematic code’ of filming and editing that has existed over the past century. In addition, it highlights how the integration of modes, such as written words, visual images, and transitions, are salient to both the production and interpretation of moving images (Curwood & Gibbons, 2009; Gibbons, 2010). The kineikonic mode unifies what is culturally understood as a form of the moving image.
An image is a re-production of something that sustains features of likeness. It is a term that is used to refer to many different things: photographs, drawings, impressionist paintings, film, three dimensional representations, and, beyond these, images in a mirror, dreams, memories, even the ‘mental images’ prompted by verbal descriptions. Multimodality attends to images that are material entities, such as photographs, monuments, film, and so on. It asks how the image has been made, what it is a representation of, what ideas and attitudes it communicates and how this is achieved, as well as investigating how social relations are constructed (i.e. how the ‘viewer’ is encouraged to relate to the image), and, overall, what the image is being used to do, such as to inform, explain, persuade, warn, entertain, and so on. A number of analytical ‘tools’ have been developed in order to carry out such analysis. For example, Kress and van Leeuwen (1996) developed the idea of visual modality in order to evaluate the truth claims of an image. In an advertisement, aspects of a person’s appearance might be exaggerated or diminished through colour saturation in order to create a slightly idealized representation of this individual. Image can also construct the social position of the ‘viewer’ through ‘offer’ or ‘demand’ (ibid), which is created through the proximity (close-up or distant), orientation (front on or sideways) and gaze (averted or direct) of represented people. These features provide clues to how the ‘viewer’ is being encouraged to evaluate the person represented in the image, and hence what the image is being used to do. A multimodal approach also asks how an image relates to other modes, such as writing alongside a photograph in a newspaper (Knox, 2007) or the sound in a film or animation (van Leeuwen, 2005; Burn, 2003), as well as the actions and interactions (e.g. action, gaze, speech, gesture) entailed in the process of producing it.
Metaphor is a term with a long history, notably within literary traditions where it is a literary ‘device’ whereby one object or process is described in terms of another for rhetorical purposes. For instance, metaphor may be used to give a heightened effect or to provide a particular perspective on a process. Lakoff and Johnson claim that metaphors play a crucial role in systematically structuring concepts, not just language.
Charles Forceville is a cognitivist linguist who has made a significant contribution to theorizing the area of multimodal metaphor that is looking at how metaphors structure concepts across a range of modes. Forceville (2009) defines multimodal metaphors as follows: ‘Multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes’. He distinguishes modes that are written language, spoken language, visuals, sound, music, gesture (as well as olfaction, taste, and touch).
Most work on multimodal metaphor within Cognitive linguistics pertains to the interaction between visuals and written language or gestures and spoken language. There is also some work on the contributions of sound and music to metaphor (see Forceville and Urios-Aparisi, 2009). Within the Cognitive Linguistics paradigm, there are two strands that pioneer multimodal metaphor/multimodal discourse analysis: Forceville’s work on pictorial/visual and multimodal metaphor; and work that focuses on the interrelation between gestures and spoken language notably work by Cornelia Mueller, Alan Cienki, and Irene Mittelberg who focus on conceptual metaphor as manifest in non-verbal modalities.
Discourse is a contested term rooted in different disciplines and used in a variety of ways. In a narrow sense, discourse can be understood as language in use – everyday ways of talking – what James Gee in his book Social linguistics and literacies: Ideology in discourses (1990) refers to as ‘little d’ discourse. In a broader sense it can be used to refer to a system of language use and other meaning-making practices (e.g. behavior, dress, and customary practices/habits) that form ways of talking about social reality what Gee refers to as ‘big D’ Discourse. For example the Discourse of traffic regulation, commercial Discourse, medical Discourse, or legal Discourse. In socio-linguistics, discourse tends to be used to refer to extended stretches of speech or writing and to draw attention to the uses and organization of language in its social context. In sociology and philosophy, the writings of Foucault have been particularly important; bringing into focus not only the social origins but also the social effects of power that discourse has on social practices. Critical Discourse Analysis (CDA) is usually associated with the work Chouliaraki and Fairclough’s work that provides a method of analyzing texts to complement Foucault’s theories and concepts as well as offers examples for the study of media discourse, policy discourse, and interactional discourse.
Discourse is an important term for multimodality and many working in this area are concerned with understanding the use and effects of Discourse through the uses of modes and their arrangement in modal ensembles. The assumption is that all multimodal texts, artefacts and communicative events are always discursively shaped; and that all modes, in different ways, offer means for the expression of discourses. From this perspective, different discourses may be brought into play modally and, therefore, the choice of modes may itself be used analytically to indicate the presence of different discourses in specific texts.
As increasingly media are multi-media forms that occupy multimodal spaces it is perhaps not surprising that CDA has strong links with multimodality, notably in the work of Chouliaraki. In her edited collection Self-Mediation: New Media, Citizenship and Civil Selves (2012), she looks into the multi-modality of new media discourses, such as convergence journalism and social networking sites, so as to explore how these discourses blur the boundaries between private and public selves and change the ways in which we understand and enact practices of citizenship.