DOI: 10.24308/iass-2014-079

University of Iasi, Romania

oanaculache@gmail.com

Abstract

The aim of this theoretical paper is to propose a new perspective on the brand communication process, by using the social semiotic methodology in order to refer to the following two elements: the channel and the code that are used to convey a message. We build our argument on the diagram of signification based on Jakobson’s work (1960) and we briefly address its main elements: sender, receiver, message, code, contact, and context. Next, we use in our study a qualitative research method, namely multimodality, which has its origins in social semiotics (Kress 2010). In our paper, we treat the two elements of the communication model: the channel (contact) and the code. There are studies in the literature that analyze these two elements separately in the conceptualization of the communication process, as distinct parts that exert an influence on each other, together with other four elements indicated by Jakobson. Nonetheless, we identify in social semiotics a more recent perspective, multimodality, and we argue that this implies a particular relationship between these two elements – channel and code, that merge and define a more complex entity – the semiotic mode. Multimodality, whose object is represented by modes, brings into the frame an issue that has been overlooked, namely the fact that the mode refers both to the nature of the physical support and to the socially developed codes, which means that the interaction between channels and codes should be considered from a higher-ranked perspective – the semiotic multimodality. Drawing from Pauwels’ definition of multimodality – the case when, within a message, “… at least two input (senses) or output (medium/device) modes (or sub-modes) are involved” (2012: 250), we make a distinction between three sets of terms: modes/sub-modes, input/output modes, multimodality/multimedia. According to these distinctions, we define the way modes’ interaction generates meaning in the process of brand communication. Then, we briefly discuss the multimodal and multisemiotic mechanisms (O’Halloran 2005) involved in brand communication – intrasemiosis and intersemiosis (with an emphasis on transduction) and finally, we discuss limitations of our study and directions for future research. The contribution of our paper is to shift the focus from the distinction code – channel (medium) to modes, in order to reconceptualize brand communication as a multimodal situation of communication.

1. Introduction

Recent studies in communication focused on developing models meant to help practitioners improve brand communication effectiveness. For specialists interested in the way brands communicate, many communication models are available in fields, such as: advertising, public relations, marketing communication or brand management (Duncan and Moriarty, 1998; Fuertes-Olivera, Velasco-Sacristán, Arribas-Baño, and Samaniego-Fernández 2001; Duncan 2005; Kliatchko 2008; Kapferer 2012) and rely on previous semiotic studies that aimed to define the elements of a genuine communication act, irrespective of the agents it involved.

According to the main communication models, the communication process implies at least 3 elements: (1) an agent who creates meaningful signs and whose intention is to convey a particular idea, (2) a set of signs that bear the intended meaning, and (3) an agent who receives the message. The first known communication model belongs to Aristotle (in Narula 2006: 136), who created it to illustrate the situation of a speech held in front of an audience. In most of the communication models that followed it, scholars (Wiener 1948; Lasswell 1948; Shannon and Weaver 1949; Schramm 1955; Berlo 1960; DeFleur [1970] 1975) changed the names of elements in order to better serve the specific area of interest, for example: the speaker became a communicator, sender, transmitter or encoder; the auditor or recipient is also known as a decoder, interpreter, destination; the speech became message, utterance or signal.

Apart from this terminological adaptation, we consider that the most interesting changes operated on the communication models are represented by the new elements scholars proposed, such as: context, code, channel, noise, and feedback. Researchers understood that the communication process is not influenced exclusively by the communicators and the signs they operate with, but also by other elements that interfere. For example, a receiver can make a distinction between a denotative and a connotative meaning of a sign by taking into consideration the context of communication or some ideas are better expressed in words than images.

2. Starting point - Jakobson’s communication diagram

One of the scholars that noticed this was the Russian linguist Roman Jakobson (1960), who argued that a communication act would imply 6 elements: addresser, message, context, contact, code, addressee, to whom he attached 6 functions: emotive/expressive, conative, referential, phatic, metalingual, poetic. Jakobson’s communication elements (1960) are often represented in the literature as a diagram or model.

   Fig. 1: Jakobson’s diagram of signification

Although there were several critics against this linear, one-way act of communication, Jakobson’s diagram of signification proves the complexity of meaning-making, a process one could not reduce to signs, because it also comprises other components that interact and influence the final meaning. The communication between the addresser and the addressee implies not only the agents and the message. The message is a meaningful set of signs that elicits the receiver’s response in a context indicating whether the intended meaning is a denotative or connotative one. Also, the addresser needs to convey specific ideas, in consequence, the medium he chooses may vary based on the features of communication. In order to assure communication, the addresser uses a code limited by its own meaning potential and presumably shared by both interacting agents in the communication process.

Jakobson’s work inspired researchers to propose communication models for various areas of interest, including areas related to brand communication. Fuertes-Olivera and his co-authors (2001) applied Jakobson’s diagram of signification in advertising and proposed the following structure: (1) advertiser as the addresser; (2) product or service as reality (context); (3) advertisements as messages; (4) a written, spoken or digital channel as contact; (5) advertising language, images, sound etc. as code; and (6) target audience as addressee.

The diversity of communication models developed for different fields, such as: advertising, public relations, and marketing communications can be used separately for each act of brand communication. Nonetheless, we should consider that brand communication is a complex process, where all the instruments and techniques (public relations, advertising, personal selling, direct marketing, Internet marketing) are integrated into a singular strategy of branding. When planning strategic brand communication, specialists should take into consideration the way their messages get to the stakeholders. That is why a singular communication model from a semiotic perspective might be more appropriate, especially because in branding, any contact between brands (by their representatives) and stakeholders generates a message about the brand and contributes to brand’s public image.

Irrespective of the communication techniques selected to convey messages, the communication specialist should always consider that any communication act highly depends on the channels and codes he uses. In our paper, we address the channel and the code by using Jakobson’s diagram of signification, a model where both elements are included. However, any other model that comprises codes and channels might be adapted in consequence, as we believe that they exert a powerful influence on the way brands succeed to communicate an idea to their stakeholders. There are studies in the literature (Jakobson 1960; Berlo 1960) that treat these two elements separately when defining or analyzing the communication process, like they were distinct parts that exerted an influence on each other and performed specific functions, together with the other four elements indicated in Jakobson’s diagram of signification. Nonetheless, we identify in social semiotics a more recent perspective, multimodality, and we argue that it implies a particular relationship between channels and codes, that merge and define a more complex entity – the semiotic mode.

3. Towards a social semiotic perspective – the multimodal method

In the last decades scholars tried to better understand the communication process and proposed new theories in semiotics. Halliday’s (1978) studies in linguistics inspired contemporary semioticians to define a new concept – multimodality – that could explain the way meanings are created. Multimodality is a semiotic method that set the foundations of a relatively new area of study, as it has been developed in the twentieth century by scholars in an attempt to extend and deepen research in the field of communication (Kress 2010; Bateman 2008; Kress and van Leeuwen 2006; O’Halloran 2004; Lim 2004; Lemke 1998; Schriver 1997). Multimodality refers to the cultural technologies of representation (Kress 2010: 30–31) in the communication process and implies various sets of signs, also referred to as modes, that co-work within a message in order to create an integrated meaning.

Modes are defined by their social-cultural nature and meaning potential. Kress (2010: 79) emphasized that modes are “... socially shaped and culturally given semiotic resources”. The nature of the relationship between modes and the social-cultural factors can be understood as a relationship between codes and context. As for modes’ meaning potential, it represents their capacity of expressing and representing an object, a characteristic referred to as modes’ “affordances” (Gibson 1997). The extent to which a mode can express and represent its object is limited, and implies that addressers get to select and combine a repertory of modes, or semiotic resources, in order to create a “modal ensemble” (Kress 2010; Lόpez Rodríguez, Prieto Velasco, and Tercedor Sánchez 2013) consistent with their communication interests. Developing on Kress’ ideas (2010), we consider that any act of brand communication is a dynamic and complex process of signs creation in which the encoder shapes an idea that materializes through a plurality of modes of communication, so that the message could be compelling to the receiver.

4. The relationship between channels, codes, and modes

As scholars have already agreed (Kress, Jewitt, Ogborn, and Tsatsarelis 2001: 2), every communication is multimodal and every mode has its own characteristics. As a result, in order to reach the communication goals, brands should find the right combination of modes that can convey the intended ideas to the stakeholders. In our paper, we argue that the multimodal method presents modes as entities that refer both to the nature of the physical support of the message and to the socially shaped and culturally developed codes, that is two elements of the communication process. This perspective is better identified if we consider Pauwels’ approach, that defines multimodality as the situation when, within a message, “… at least two input (senses) or output (medium/device) modes (or sub-modes) are involved” (2012: 250).

4.1. The dichotomy “modes” and “sub-modes”

Pauwels’ definition of multimodality generates a better understanding of the way modes refer both to channels and codes. According to Pauwels, there is a distinction that can be made between modes: a higher-ranked mode, which we will refer to as “mode” or “supermode” (2012) and a lower class of modes, which we will refer to as “sub-modes”. This approach reveals modalities as more general concepts, with a direct link to the five perceptual senses: visual, auditory, tactile, gustatory, and olfactory. Any communication process that implies two supermodes, for example the auditory and the visual mode, is multimodal. In Pauwels’ terms, supermodes give more details concerning the medium of communication, represented by the very channel that appears in the communication process.

As we have emphasized, the five perceptual senses are considered to be supermodes, whereas the literature often refers to modalities as sets of codes: image, writing, layout, music, gesture, speech, moving image, soundtrack, colours, and 3D objects (Kress 2010: 28). Codes are sets of semiotic resources that permit encoding messages by using signs whose meanings are shared both by the sender and the receiver. In Pauwels’ terms, sub-modes represent a lower-ranked type of modes that give more details concerning the codes used in the communication process.

4.2. The dichotomy “input modes” and “output modes”

After we have clarified that the distinction between modes and sub-modes, in Pauwels’ terms, refers to channels and codes, we can proceed to the second relevant distinction, between input and output modes. The input modes, corresponding to the five senses, represent the set of resources the recipient uses in order to receive the message. The input modes refer to the psychological processes of perception that take place on the part of the receiver when he gets the message he has to decode. In this respect, multimodality implies that whenever a receiver uses at least two senses in order to get and decode a message, it is the case of a multimodal message.

The output modes, represented by the device or medium of communication, refer to the set of resources necessary to assure the expression and representation of the object. Output modes, used by the sender of the message during the encoding stage, are presumably known by both communicators, so that meaningful outputs to become understandable inputs. As a result, the distinction input modes – output modes is meant to separate the resources used by the agents in order to assure communication and it refers on the one hand to the supermodes linked to the transmission channel, and on the other hand, to the sub-modes linked to the codes.

4.3. The dichotomy “multimodal” and “multimedia”

As we stated before, multimodality implies a repertory of modes. At this point, in order to avoid confusion, it is important to mention that the multimodality is not particularly the case of multimedia. As Kress (2010: 30–31) pointed out, there is a difference between these two concepts. Multimodality implies the use of multiple semiotic resources that permit the representation of objects, whereas multimedia represents the use of technologies for disseminating messages (television, radio, World Wide Web, outdoor media, press). In fact, the concept of “multimedia” is considered to have a more restricted meaning, as it refers exclusively to technical characteristics of devices, without any connection to the human communicative or perceptive act (Pauwels 2012).

5. Proposed communication model based on Jakobson’s diagram

The arguments we have presented above regarding the alternative way of using channels and codes in the communication process lead us to modify Jakobson’s communication model. As depicted in the figure below, the proposed model in this paper implies a transformation: we suggest replacing the channel and code with a higher-ranked element: the modality that covers both the channel and the codes, as we have already proved.

Fig. 2: Proposed communication model based on Jakobson’s diagram.

In what follows, we will discuss the implications of the revised model in the situation of brand communication, by presenting arguments for generating meaning via multimodality.

5.1. Generating meaning by interacting modes

From a brand communication perspective, after analyzing the message support types, we can notice that all the channels require at least two perceptive modes, for example: advertising usually implies the visual and auditory mode, personal selling may imply auditory, visual, tactile, gustatory and olfactory mode, public relations mostly imply auditory and visual mode, Internet marketing implies visual and auditory mode. As we can see, although it is a very popular brand communication tool, advertising is the least permissive in matter of multimodality: television only permits auditory and visual mode, radio only permits auditory mode, the press only permits visual mode, outdoor media only permits visual and auditory modes, even digital technologies are restricted to two modes: visual and auditory. The lack of flexibility in advertising, from a multimodal perspective, represents a great disadvantage, because multimodality can offer brands the opportunity to create and convey multisensory messages, capable of having a greater impact on consumers.

When selecting a modality, for example an image, we can easily think of the possible channels that permit using images to convey a message: television, websites, outdoor media, or press. In fact, there is a more ellaborate connection we can build between supermodes, sub-modes and the technical support of the message. There is a causal link between these, as one channel may or may not permit a supermode and, furthermore, a sub-mode. In brand communication, the selected channel (or channels mix) should serve the communication goals, that is to permit conveying the intended meaning by using the most appropriate modes.

From another point of view, sub-modes also indicate the type of codes used in the message. If we think about a specific sub-mode, we automatically identify the subsequent codes, considered as sets of semiotic resources that bear a meaning established in a cultural context. For example, image as mode includes colors and shapes, gestures include nonverbal language as intrinsic code, and layouts include distances.

5.2. Multimodal and multisemiotic mechanisms

The problem of mode must be analyzed also by taking into consideration two multisemiotic processes that take place: intrasemiosis and intersemiosis. Intrasemiosis concerns the meaning within different semiotic resources (O’Halloran 2004), whereas intersemiosis refers to the interaction of the various semiotic resources, in order to engender a new meaning or set of meanings in the context of a connectedness and interweaving between modes (Lim 2004: 239; Kress et al. 2001:25). The multisemiotic mechanisms, implying the use of multiple semiotic resources, are strongly connected to multimodality, that uses various modes to convey a message. In the multimodal frame, there are two processes linked to intrasemiosis and intersemiosis: transformation and transduction. These two refer to the translation of modes within a mode (transformation) or between modes (transduction), as Kress (2010: 43) pointed out.

In branding, advertisers usually use several channels of communication – for example, an advertising campaign may imply television and radio commercials, outdoor media and press advertorials – adapting the unitary message to each channel, as multimodality permits translation of modes. Translation can occur by shifting from one mode to another – writing into speaking (the case of transduction) or by shifting elements whithin a mode – formal to colloquial tone, hard-rock to classical soundtrack (the case of transformation). As a result, transduction permits the translation of meanings originally conveyed through a particular mode into meanings conveyed through a different mode. Transformation only implies an internal change, without changing the modes of expression. Both transduction and transformation encourage creativity in branding, by searching alternatives to convey the intended meanings to stakeholders. Modes’ translation, and especially transduction, has been a subject of interest for semioticians in the field of multimodality, who identified it as an intersemiotic process determined by shifts between semiotic codes, also known as “resemioticization” (Iedema 2003) or “semantic reconstruals” (O'Halloran 1999a, 1999b). Multimodality provides the means to create a diversity of coherent messages by using transduction to build a cluster of meanings that improve the information assimilation process by using various modes that rearticulate each other and generate ratification (Kress, 2010: 169). We should also consider that, by ratification, a mode’s role is to confirm another mode’s meaning, so that to help the receiver to have a more accurate understanding of the intended meaning.

6. Conclusion and limitations of our study

According to the arguments we have presented in this paper, we consider there is a strong connection between modes, codes, and channels. The multimodal messages imply using output modes and input modes in the communication process. In order to better represent an object or an entity, the various supermodes and their subsequent sub-modes could be used by semioticians and be strategically combined in a “semiotic harmony” (Kress 2010: 157).

In our opinion, when planning brand communication, for example an advertising campaign, instead of creating a media plan and then the advertisements, we could proceed differently: considering the message a brand wishes to convey to its stakeholders, the meaning-making process should start with the best combination of modalities that could represent the ideas. According to Kress’ (2010: 169) idea of ratification, the modes do not act as duplicates – on the contrary, they complement each other. The concept of “intersemiosis” designates the unitary coordination of semiosis created across multiple sign systems (Ravelli 2008: 30) so that they are not perceived to be isolated, but as having a unitary meaning. As Thibault (2000: 321) and Baldry (2004: 87) noted, the meaning of a multimodal message is a composite product/process involving the way in which different resources are co-deployed. Consequently, it should not be divided into a number of separate “channels” or “codes”. Also, Lemke (1998) introduces the concept of “multiplication of meaning”, as an orchestrated combination of meanings, different from redundancy, which brings something new while selecting codes to adapt messages to each channel.

We are arguing that multimodality can help brand specialists to plan the communication strategy, so that it would better serve the communication goals. Considering multimodality as a method that combines semiotic resources to express the intended message in the most loyal way, the use of modes could be more effective than the traditional way.

In this context, we consider that multimodality might prove itself to be a useful tool in the field of brand communication, and that this integrative perspective would help brand specialists improve their communication techniques and instruments. It is important to mention that, in addition to the other elements that compose the communication process, this alternative way of planning a communication strategy should definitely take into consideration not only the best modes that would guide brand specialists’ choices regarding the communication channels and codes, but also the social-cultural aspects, such as the most effective channels to reach the stakeholders and the possible connotations of codes.

The contribution of our paper is to shift the focus from the distinction code – channel (and medium) to modes, in order to reconceptualize brand communication as a multimodal situation of communication, drawing from recent studies in social semiotics. Despite its originality, this proposition also has limitations. Our study focused on the structure of Jakobson’s model and we did not analyze the impact of the structural changes on the functions attached to the diagram of signification. Also, by replacing two structural elements of Jakobson’s model, future research should also take into consideration studying any possible implication in terms of functions attached to the model, especially concerning the phatic and metalingual functions linked to channels and codes.

Note: This paper is supported by the Sectoral Operational Programme Human Resources Development (SOP HRD), financed from the European Social Fund and by the Romanian Government under the contract number POSDRU/159/1.5/S/133675.

References

BALDRY, Anthony P. 2004. Phase and transition, type and instance: patterns in media texts as seen through a multimodal concordancer. In Kay. L. O’Halloran (ed.),Multimodal Discourse Analysis: Systemic-Functional Perspectives, 83–108. London & New York: Continuum.

BATEMAN, John A. 2008. Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. New York: Palgrave Macmillan.

BERLO, David K. 1960. The Process of Communication. New York: Rinehart and Winston.

DEFLEUR, Melvin L. & Sandra BALL-ROKEACH. 1975 [1970]. Theories of Mass Communication, 5th edn. New York: David McKay.

DUNCAN, Tom. 2005. Principles of Advertising & IMC, 2nd  edn. Boston, MA: McGraw Hill-Irwin.

DUNCAN, Tom & Sandra E. MORIARTY. 1998. A communication-based marketing model for managing relationships. Journal of Marketing 62(2), 1–13.

FUERTES-OLIVERA, Pedro A., Marisol VELASCO-SACRISTÁN, Ascensión ARRIBAS-BAÑO & Eva SAMANIEGO-FERNÁNDEZ. 2001. Persuasion and advertising English: Metadiscourse in slogans and headlines. Journal of Pragmatics 33(8), 1291–1307.

GIBSON, James J. 1977. The theory of affordances. In Robert Shaw & John Bransford (eds.), Perceiving, Acting and Knowing: Toward an Ecological Psychology, 67–82. Hillsdale, NJ: Lawrence Erlbaum Associates.

HALLIDAY, Michael A. K. 1978. Language as a Social Semiotic. London: Edward Arnold.

IEDEMA, Rick. 2003. Multimodality, resemioticization: extending the analysis of discourse as a multi-semiotic practice. Visual Communication 2(1), 29–57.

JAKOBSON, Roman. 1960. Closing Statement: Linguistics and poetics. In Thomas Sebeok (ed.), Style in Language, 350–377. Cambridge, MA: MIT Press.

KAPFERER, Jean. N. 2012. The New Strategic Brand Management. Advanced Insights and Strategic Thinking, 5th edn. London, Philadelphia & New Delhi: Kogan Page.

KLIATCHKO, Jerry. 2008. Revisiting the IMC construct. A revised definition and four pillars. International Journal of Advertising 27(1), 133–160.

KRESS, Gunther. 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. New York: Routledge.

KRESS, Gunther, Carey JEWITT, Jon OGBORN & Charalampos TSATSARELIS. 2001. Multimodal Teaching and Learning: The Rhetorics of the Science Classroom. London & New York: Continuum.

KRESS, Gunther & Theo VAN LEEUWEN. 2006. Reading Images: The Grammar of Visual Design, 2nd ed. London & New York: Routledge.

LASSWELL, Harold D. 1948. The structure and function of communication in society. In Lyman Bryson (ed.), The Communication of Ideas, 37–51. New York: Harper and Row.

LEMKE, Jay. 1998. Multiplying meaning: visual and verbal semiotics in scientific text. In J. R. Martin and Robert Veel (eds.), Reading science: critical and functional perspectives on discourses of science, 87–113. London: Routledge.

LIM, Fei V. 2004. Developing an integrative multi-semiotic model. In Kay. L. O’Halloran (ed.), Multimodal Discourse Analysis: Systemic-Functional Perspectives, 220–246. London & New York: Continuum.

LΌPEZ RODRÍGUEZ, Clara I., Juan A. Prieto VELASCO & Maribel Tercedor SÁNCHEZ. 2013. Multimodal representation of specialized knowledge in ontology-based terminological databases: the case of EcoLexicon. The Journal of Specialized Translation 20, 49–67.

NARULA, Uma. 2006. Dynamics of Mass Communication Theory and Practice. New Delhi: Atlantic Publishers and Distributors.

O’HALLORAN, Kay L. 2005. Mathematical Discourse: Language, Symbolism and Visual Images. London & New York: Continuum.

O’HALLORAN, Kay L. (ed.). 2004. Multimodal Discourse Analysis: Systemic-Functional Perspectives. London & New York: Continuum.

O’HALLORAN, Kay L. 1999a. Interdependence, interaction and metaphor in multisemiotic texts. Social Semiotics 9(3), 317–354.

O’HALLORAN, Kay L. 1999b. Towards a systemic-functional analysis of multisemiotic mathematics texts. Semiotica 124(1/2), 1–29.

PAUWELS, Luc. (2012). A multimodal framework for analysing websites as cultural expressions. Journal of Computer-Mediated Communication 17, 247–265.

RAVELLI, Louise J. (2008). Analysing space: adapting and extending multimodal frameworks. In Len Unsworth (ed.), Multimodal Semiotics: Functional Analysis in Contexts of Education, 17–33. London & New York: Continuum International Publishing Group.

SCHRAMM, Wilbur. 1955. How communication works. In Wilbur Schramm (ed.), Process and Effects of Mass Communication, 3–26. Urbana, Illinois: University of Illinois Press.

SCHRIVER, Karen A. 1997. Dynamics in Document Design: Creating Texts for Readers. New York: John Wiley and Sons.

SHANNON, Claude E. & Warren WEAVER. 1949. The Mathematical Theory of Communication. Urbana, Illinois: University of Illinois Press.

THIBAULT, Paul J. 2000. The multimodal transcription of a television advertisement: theory and practice. In Anthony P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age, 311–385. Campobasso: Palladino Editore.

WIENER, Norbert. 1948. Cybernetics, or Control and Communication in the Animal and the Machine. Cambridge, MA: MIT Press.