Abstract
Abstract: The quickly growing field of multimodality has hitherto primarily found its theoretical roots in linguistics, both systemic functional linguistics (SFL) and, more recently, also cognitive linguistics. Although the best work in these paradigms helps visual studies and multimodality come into their own as bona fide scholarly disciplines, it is undesirable to rely exclusively on linguistic theories to model forms of communication that are only partially verbal, or not verbal at all.
It is better to analyse what we can broadly define as a message (a command, a piece of information, a poem, an advertisement, a political cartoon, a film) in terms of “content” that a more or less specific communicator wants to convey to a more or less specific audience using a more or less specific medium. This medium can draw on one or more “modes” (language, visuals, sound, music, bodily behaviour) to try and convey this message. Importantly, this message exists arguably on the level of cognition before it is presented in a specific medium – although it is of course only its material form that allows the message to be communicated in the first place.
What we need, therefore, is an inclusive model of communication that takes into account the identities of the communicator, the audience, as well as their relation, and that does not privilege specific media and/or modes over others. The contours of such a model actually already exist in Sperber & Wilson’s Relevance Theory: Communication and Cognition (Blackwell 1995 [1986]). Relevance theory(RT)’s central claim is that each act of communication comes with the presumption of its own optimal relevance to its envisaged audience. Although this claim is not dependent on a specific medium, hitherto RT scholars (typically: linguists) have almost exclusively analysed face-to-face exchanges between two people who stand next to each other. In order to fulfil RT’s potential to develop into an inclusive theory of communication, it is necessary to explore how it can be adapted and refined to account for (1) messages in other modes than (only) the verbal mode; and (2) mass-communication. In Visual and Multimodal Communication: Applying the Relevance Principle (Oxford UP 2020) I take a first step toward this goal by proposing how RT works for mass-communicative messages that are either standalone visuals or combine visuals with short verbal written texts, resulting in multimodal texts.
Assuming familiarity with “classic” RT, I will present my view of how RT can account for the interpretation of a picture or a multimodal message of the picture-plus-verbal-text variety. Technically formulated, certain (elements of) visuals are coded, and hence have a precise, fully explicit meaning for the message’s envisaged audience, whereas other (elements of) visuals have a meaning that is (strongly or weakly) implicit, that must therefore be inferred on the basis of contextual and situational clues – the latter potentially resulting in small or not so small differences of interpretation by different addressees.
In the talk I will discuss this issue drawing on examples from different genres, including logos & pictograms, advertisements, and cartoons.
Optional preparation for the talk: Forceville, Charles (2014). “Relevance Theory as model for analysing multimodal communication.” In: David Machin (ed.), Visual Communication (51-70). Berlin: Mouton de Gruyter. For a summary, see https://link.growkudos.com/1kgvnwj0idc
Biographical information: Charles Forceville is associate professor in the Film/Media Studies department of Universiteit van Amsterdam, NL (http://www.uva.nl/en/profile/f/o/c.j.forceville/c.j.forceville.html). The key theme in his research is the question how visuals, alone or in combination with other modes, convey meaning. Committed to cognitivist and relevance-theoretic approaches, he writes on multimodality in various genres and media (documentary, animation, fiction film, advertising, comics, cartoons, pictograms & traffic signs, pictures in children’s books). He published Pictorial Metaphor in Advertising (Routledge, 1996). With Eduardo Urios-Aparisi he edited Multimodal Metaphor (Mouton de Gruyter, 2009); with Tony Veale and Kurt Feyaerts Creativity and the Agile Mind (Mouton de Gruyter, 2013); and with Assimakis Tseronis Multimodal Argumentation and Rhetoric in Media Genres (Benjamins, 2017). His monograph Visual and Multimodal Communication: Applying the Relevance Principle appeared in 2020 (Oxford University Press).
Under his guidance, students of HKU Utrecht made a series of five short animation film on narratology (and one on the JOURNEY metaphor), available on YouTube (2014-2019). Forceville has profiles on Researchgate.net and Academia.edu.
Relevance Researchers' Network 