DOI: 10.24308/iass-2014-140

University of Lapland, Rovaniemi, Finland

silja.nikula@ulapland.fi

Abstract

My aim is to define the concept of narrativity in the context of static images. I see the process of image making as visual storytelling, and use my own artwork as an example. I sketch a model of narrative layers, combining semiotic concepts and principles from the narrative theory of literary studies.

Graphic images are material, concrete pictures, and all images are based on some kind of resemblance (Mitchell 1987: 10). Within an image space, meanings arise from both the properties of the elements and their composition (Engelhardt 2007). A fundamental strength of images is that they are able to reproduce the appearances in reality. Compositional structures have become established as conventions in the course of history. The study pays special attention to the ways in which metaphoric expression, symbols and the illustration method give rise to connotations and thus increase the variety of meanings construed from a picture.

However, in the final interpretation, the role of audience is salient, and representative pictures have multiple meanings and stay ambiguous. I shed light on images as producers of meanings and as their own kind of language. Because my background is graphic designer´s, the discussion is in the domain of visual communication design.

Picture 1. Woodcut. Silja Nikula 2008.

1. Introduction

First I tell the background of my graphic image (picture 1). It is based on my travel experiences in Prague. My aim was to tell about the spirit of the place by visual means: using lines, forms and colours on a two-dimensional surface. Spirit of a place is a salient concept in humanistic geography, emphasizing the personal experience of a person staying in the place (Relph 1976). This graphic image I see as a narrative: the story is the content of expression, so my experience during travel, and narrative or discourse is the form of that expression. As a visual storyteller, I arranged my experiences and told about them. This was one of my travel pictures made during the years 2008–2011, and that practice-based research led me sketch this model of narrative layers.

I use the metaphor of layers to express, not only how meanings can be added step-by-step, but also that each layer is a part of a whole, and the layers work together in meaning making. Still, I don´t mean that the first layer is lower or subordinate to the others, but all the layers carry special visual resources to be used in different communication contexts. Sometimes only the first level is justified, but in many cases settling more layers makes the audience see multiple meanings.

The study of narrative structure has traditionally focused on verbal storytelling. In visual modes, it has been adapted to moving picture media as film; meanwhile storytelling practices within static images have been neglected. However, recently Michael Ranta (2013) has built the bridge between narratology and the theory of art, using paintings as examples, and presenting possible criteria for narrativity or tellability. I understand these terms broadly: visual representations can serve as descriptions and expositions; they can express the states of affairs, and offer aesthetic experiences. They can also tempt audiences to interpret ideas as an argument.

2. Theoretical commitments

When constructing my model, I combine ideas from narrative theory in literature with the concepts of semiotics.

In defining the concept of narrative, often three criteria are suggested: temporality, causation and human interest (Cortazzi 1993: 85). Within narrative theory, the distinction between the story and discourse is recognized (Chatman 1978: 19, 23; Ranta 2013: 5; Cortazzi 1993: 83). A story means the chain of events, or rendered content while discourse is the expression, how something is transmitted (Ranta 2013: 5). I express something that I believe is interesting to a reader, a story to share with my audience. Seymour Chatman (1978: 21, 33) calls this process “transforming”: a story or statement is composed by someone and mediated within the limits of each presentation medium.So it is possible to present a story in different media, but the definition of storytelling has to be modified according to the medium used (Rimmon-Kenan 1999: 15). Seymour Chatman (1978: 20) calls it transposability, when meaning that the story remains the same, but one expressing medium or language is transferred to another.

Visual language is different from a natural one, having its own strengths and weaknesses in transmitting ideas. The meanings come through the composition and properties of elements (Engelhardt 2007). In their visual design grammar, Günther Kress and Theo van Leeuwen (2006) already have concerned how narrative theory applies to composition. For example left and right sides in a composition space have different information values, and so do upper and lower parts. Properties of elements I understand in two different ways: first, a semantic level, how elements are named on a denotational level – second, what kind of lines, colours and shapes are used as a syntactic level. The illustrator is active in meaning making and transmitting ideas; the properties of elements are connected with the compositional choices.

In adapting narrative theory, my main concern is in temporality, considered as a necessary condition for a narrative. There cannot exist a story without different events happening in the time (Cortazzi 1993: 86). How far it is possible to use chronological order as a criterion of narrative? I discuss here, what kind of visual means may give the viewer the impression of movement.

Charles Peirce´s (1991) three sign classes as semiotic concepts are connected with narrative theory, as defining the properties of visual elements. To sharpen the referencing process in visual storytelling, also figurative tropes metonymy and metaphor are used.

3. The Model of Narrative Layers

3.1. First layer based on iconicity: Imitating reality

Picture 1 was born after my travel to Prague. If you have visited there, you perhaps recognize the bridge and the statue, and remember seeing high churches there. The elements refer to the scenery that I saw in my own eyes. Because your eye perception is very similar to mine, you can call back your memories through the visual appearance of elements. Here comes one salient positive property of images, iconicity. All images are based on some kind of resemblance, similarity between elements and their referents (Mitchell, 1987: 10). Through the combinations of lines, shapes and colours, we can imitate optical experiences in the real world (Messaris 1997: 3) – often also give access to emotional responses. Images are able to reproduce those aspects that the image-maker has selected. While in written language the signs are arbitrary, a visual representation has the potential to more closely signify how things appear (Messaris 1994: 117).

My illustration is simplified and does not imitate every detail. However, visual presenting has not to be a precise match between the picture and the reality. Small hints can be enough, because we realize that a picture as a representation only imitates the eye perception. In sketches, audiences can fill the gaps with their imagination, because they are active in the reconstruction of narratives (see Ranta 2013: 7). Also you may imagine this black-and-white drawing as coloured scenery as you have seen in travel magazines, your own eyes or imagination.

I also selected the format of this graphic image, arranging a meaningful area. However, it is broadening over its frames, when the viewers think the scenery continuing. This reference is based on metonymy, also a means of creating a realistic impression. When seeing one small part, the bigger whole can be traced, which is the kind of the most common metonym, called synechdoke (van Leeuwen 2005: 34; Mitchell 1987: 27). The viewers use their own subjective imagination to interpret what kind of idea is represented.

However, iconicity does not mean that the object really exists. Even unnatural objects can be presented in a very realistic way. Using linear perspective emphasizes reality. The hegemony of this perspective is in the way it lays claims to being a “natural” representation (Mitchell 1987: 37; Goodman 1976:10). In my woodcut, you assume that the bridge is nearer than the scenery, because it is shown bigger. Three-dimensionality is one criterion in creating credibility (Goodman 1976: 34–35), which also van Leeuwen (2005: 167) argues in his concept of modality. When colours are natural, depth is arranged, details included, we often count on the happening existing in real life. So, changing colours unnatural is one possibility to violate reality. Including details also gives the impression of a documentary representation, in spite those details were irrelevant to the basic content idea or message (Barthes 1993: 99–108).

3.2. Second layer: Living creatures make it an event

3.2.1. Expanding the properties of elements

The picture also extends beyond its frame in another way: an event before and after this arrested moment can be seen. Even on pure scenery, a chain of events may be seen, as a flow of seasons or different times of day. Still according to the narrative theory the living creatures are essential. Seymour Chatman (1978: 34), for example suggests that one cannot account for events without recognizing the existence of the things causing them, or being affected by them. So when I put a human character on my picture space, I made it possible to see more meanings. Also there is at least one other living creature: the bird flying in the sky. Now, there comes a reference to a chain of events and time passing by (see Ranta 2013: 7; Mikkonen 2005: 21). Something is happening before and after this arrested moment.

Not only actions are interpreted through people represented. Gestures also serve as the hints of an emotional state. As Russian psychologist Alfred L. Yarbus has proved, our eyes fix first on those features that can help identify the person, and the fixations tend to cluster around the eyes and the mouth (Cairo 2013: 103). Eyes are not only physical features; they often indicate intention and give rise to connotations. In this way, iconicity can give access to a broad spectrum of emotional responses, because viewers can, in their mind´s eye, combine the visual representation with their real-life experiments (Messaris 1997: xiv–xvii). In my picture, the person´s mouth is open, messy movement and shouting going on.

3.2.2. Making meanings with composition

The idea of movement is supported by the means of arranging visual elements. The choices in composition are resources in telling and making meanings. Visual structuring creates a meaningful proposition by means of visual syntax (Kress and van Leeuwen 2006: 47). In their grammar of visual design Kress and van Leeuwen (2006) define narrative images having vectors, which are lines to show, what kind of action there is going on. Vectors show the directions of movements and can lead our eyes, even indicate the order in reading the elements. However, there are no certain rules for that. In my example picture, there is a vector between the flying character and the statue. The horizontal shape of the bridge emphasizes the same direction.

The information values of left and right are conventions to produce meanings and can be seen as narrative resources. In western countries, the left side is the old one, while right has something new. (Kress & van Leeuwen 2006: 179–185.) I put more speed by fading speed lines that are conventions people interpret as moving (see Hietaharju 2006, 91). Still here I made a mistake: Because the carving was made in a very spontaneous way, I did not think that the final picture would be turned as its mirror image when printed on paper. Also the direction of movement was changed, and according to the information values, I am now returning home. The purpose was vice versa; the idea was to face the challenges of the future.

Kress and van Leeuwen (2006: 4, 179–193) also concern top and bottom in their visual grammar: whereas the bottom is seen as a real world, the top is imaginative. Fictive elements are often settled on the upper side of the composition area. In my image, the bridge at the lower part is dark and seems to be heavy. There is clear contrast between steady ground and the light sky. Flying naturally takes place in the air in real life, so the bird and the clouds are settled on the upper part of the picture. However, there is also something irrational, because one of the flying creatures is a human being. I tell the reason later.

When trying to express movement in a static image, causes problems. Still if we think about an entire static image broadly, different stages of happenings can be put on the same surface. Placing legs in different positions on a picture surface, for example, makes us interpret them as moving. The movement is going forward if the direction is from left to right – within most western countries. Also when the same characters are settled on the same surface several times, viewers perhaps think that there is an action going on, and time is passing by. Also a path, river or a road, can serve as an analogue of time-flow, for example starting the story from front part and going back part (Mikkonen 2005: 171, 203). However, we can’t be sure how the composition is interpreted. In my example picture, “speed lines” can be seen, because the direction of carving is horizontal like the movement. It is also a natural and easiest way to make it, because the wooden block directs the carving. Also the fading style in colouring is a hint to see this lady flying. These are conventions that viewers might interpret as a movement. Added to that, many small different carved elements that follow the flying person, arrange chaos, and might even strenthen the idea of uncontrollable movements (see Gombrich 1982, 55).

3.3. Layer of comment: Iconicity based on metaphor

Is my presentation outside reality? Well, in Prague we were walking along the streets and bridges, then there came a rain and strong wind, and my umbrella was turned around. However, people don´t normally fly, and nor did I in Prague. In that sense, this is fiction. But it is true in a metaphorical sense. Often the meanings given to a travel are more important than the place itself. We actually live our stories and organize our experiences and knowledge about the world (Bruner 1987). That happened in my picture. I put a layer of comment as Nelson Goodman presents (1976: 8), meaning that the attitude and opinion are strongly seen in the picture. In visual representations commenting can be made through metaphor, often presenting an abstract concept by using concrete things or a physical situation (Messaris 1997: 9).

In my first layer, I concerned with iconicity based on eye-perception. Another type of iconicity is metaphoric, where similarity is based on analogy. In my Prague picture, I presented myself flying over the bridge, towards an old statue of the medieval town. Contrast between the flying character and the statue can be seen: first one is messy, the other one staying static and steady. This image became a metaphor for my research that I had just started. This was one of the first pictures connected my dissertation. The research was practice based and my own background was not traditionally academic. That made the start challenging.

Mitchell (1987: 10, 13) classifies metaphors as verbal images, defining them as short descriptions, “sometimes suggesting nothing more than a recurrent abstract idea”. Metaphors are based on associations in mental imagery (Fiske 1994: 127–131). As a multimodal concept metaphor can be applied to images. In presenting, some aspects are highlighted, others neglected. (van Leeuwen 2005: 30, 34.) The resemblances are the grounds for the metaphor, but still metaphoric images stay ambient and suggestive. Based on certain hints, the audience thinks that metaphorical interpretation is preferable than a literal one. For example, combining items in different scales, putting usual and unusual elements together or manipulating the colours unnatural can emphasize the feeling of another reality. Still, you perhaps fulfil my one-coloured drawing in your imagination and think in colour. I used only one colour as a purpose to to create coherence between wide varieties of elements.

For visual communication designers, creating new kinds of reality through metaphor is interesting and challenging, because the representations have to be somehow understood, still remain surprising and fresh (Messaris 1997: 9–10). They have special power in persuasive use, because they are suggestive (Messaris 1997: 219–225). However, to serve as arguments, they need an interpreter to complete the argument by drawing the conclusion.

3.4. Symbols compressing meanings

The sign categories, icon and symbol, can work together in one element and the final character is defined by the context and the interest of the viewer. Here the bird and the bridge might be interpreted as symbols, but if they are interpreted so, we don´t actually know. Symbols are signs when used and understood as such (Fiske 1994: 121; Kress & van Leeuwen 2006: 8). Without defining the using context and audience we can´t be sure. As illustrations in cemeteries for example, birds may be symbols of soul or spirit, flying away from body.

Symbols are in many cases useful within communication purposes, because they compress meanings. With using one small element, a great number of ideas can be told. Based on subjectivity and ambiguity, symbols give rise to connotations. On the other hand, they can limit the space of possible meanings. Within one cultural area, meanings are common because they are based on a habit, convention or agreement. Symbols designed for information at the airports reach for one exact meaning, while the bird increases tellability in my picture, because it has multiple meanings, based also on individual interpretations.

3.5. Illustration method giving rise to connotations

Some connotations surely derive from the contents, but also the formal properties of elements affect interpretations. An image works in a system that is loaded at both a syntactic and semantic level (Goodman 1976). For example the shape, thickness and roughness of a line, or colour saturation carry meanings on a syntactic level. Here I also concern with the third sign class, indexicality. Indexical marks can be seen on the surface of my graphic image, referring to the process of constructing the image. If in a photograph indexicality is an unseen frame, not having content of its own (Laakso 2003: 132), in my drawing it is a part of the content, carrying an expressive charge and intensifying the message (Nikula 2014: 76).

In my final artwork, can the material surface, carving and colouring style be seen. The clouds are soft and have lighter shade, as the flying person is expressed in sharp shapes. Brushstrokes, carving or other traces of making often enchance the attractiveness of visual representations and perhaps viewers’ attention is held for a longer time. Through expression styles, the artists´emotions can be transfered.

When artwork is made by drawing – not a detailed photograph, it contains only meaningful elements selected by the illustrator (see Barthes 1986: 82). In visual communication context that means paying attention to the most important parts of the message. Distilling the essential features of a complex object is informative.

4. Final interpretation: viewer filling the gaps

I see a layer as an appropriate metaphor and it has also been used earlier to describe the visual resources within communication use. Not only Nelson Goodman (1976: 8) uses it, but also Roland Barthes (1986: 71–92) describes the denotation and connotation as two layers of meaning. Whereas the layer of denotation is the layer of what or who, the layer of connotation says, what ideas or values are expressed through the way they are represented (van Leeuwen 2005: 37).

However, the role of audience is active in the process of interpreting a representative image, because it gives indirect suggestions for meaning making. The narratives include gaps and demand the active efforts of meaning making (Ranta 2013: 8). The meanings are individual and subjective, but culturally shared. Pictorial media lacks narrative precision compared with verbal ones (Ranta 2013: 7–8). Because perceiving is selective, it depends on the needs and interests of audiences, how different viewers notice different affordances. From the point of view of designer’s, the images offer potential for meaning making. Based on the perceivable properties of the objects, this potential may be recognized – or stay latent, if not noticed. (van Leeuwen 2005: 273.)

I see my model of narrative layers as resources for visual designers, to be taken use, varied and adapted to different communication situations. It was created for representational still images that are open to interpretation – not for diagrammatical or conventional images like traffic signs. I argue that certain kind of realism is needed for tellability, to establish the connection with the mental images of the viewers’. Also the border between design and narrative has to be studied. When stylised, the picture becomes a description of description and looses the referent in reality (Uspenski 1991; Nikula 2014: 77). Also Wendy Steiner (2004: 155–157) sees, that realism in narrating makes the audience see more meanings.

This model was based on my own artwork production during the years 2008–2011 and the final dissertation was published in the year 2012. Now I am testing how it could be adapted in real communication contexts and other illustration methods, as photographs.

Referances

BARTHES, Roland. 1993. Tekijän kuolema tekstin syntymä. Translated: Lea Rojola ja Pirjo Thorel. Tampere: Vastapaino. Le degré zéro de l'écriture [1953].

BARTHES, Roland. [1964] 1986. Kuvan retoriikkaa. In Lintunen Martti (ed.) Kuvista sanoin 3. Helsinki: Suomen valokuvataiteen museon säätiö, 71–92. Rhétorique de l’image [1964].

BRUNER, Jerome. 1986. Actual minds, possible worlds. Cambridge Mass: Harward University Press.

CAIRO, Alberto. 2013. The Functional Art. An Introduction to Information Graphics and visualization. New Riders.

CHATMAN, Seymour. 1978. Story and Discourse. Narrative Structure in Fiction and Film. Ithaca: Cornell University Press.

CORTAZZI, Martin. 1993. Narrative analysis. London & Washington, D.C.: The Falmer Press.

ENGELHARDT, Yuri. 2007. Syntactic Structures in Graphics. In Schirra Jörg R. J. (ed.) Image – Zeitschrift fur Interdiziplinäre Bildwisswnschaft  5/2007, 23–35.

GOMBRICH, E. H. 1982. The Image and The Eye. Further studies in the psychology of pictorial representation. Oxford: Phaidon Press Limited.

GOODMAN, Nelson. 1976. Languages of Art. An Approach to a Theory of Symbols. Indianapolis: Hackett Publishing Company, Inc. 2nd edition.

FISKE, John. 1994 [1990]. Merkkien kieli. Johdatus viestinnän tutkimiseen. 3th edition. [Introduction to Communication Studies] Edited and translated: Veikko Pietilä, Risto Suikkanen & Timo Uusitupa. Tampere: Vastapaino.

HIETAHARJU, Mikko. 2006. Valokuvan voi repiä. Valokuvan rakenne-elementit, käyttöympäristöt sekä valokuvatulkinnan syntyminen. [Tearing a photograph. Compositional elements, contexts and the brth of the interpretation.] University of Jyväskylä Dissertation.

KRESS, Günther & Theo VAN LEEUWEN. [1996] 2006. READING IMAGES. THE GRAMMAR OF VISUAL DESIGN. 2nd edition. London: Routledge.

LAAKSO, Harri. 2003. Valokuvan tapahtuma [The Event of a Photograph] Helsinki: Tutkijaliitto.

VAN LEEUWEN, Theo. 2005. Introducing Social Semiotics. London & New York: Routledge

MESSARIS, Paul. 1994. Visual literacy. Image, Mind & Reality. Colorado: Westview Press, Inc.

MESSARIS, Paul. 1997. Visual Persuasion: the Role of Images in Advertising. Thousand Oaks: SAGE Publications.

MIKKONEN, Kai. 2005. Kuva ja sana. Kuvan ja sanan vuorovaikutus kirjallisuudessa, kuvataiteissa ja ikonoteksteissä [Image and word. Interaction between Image and Word in Literature, Visual Art and Iconotexts] Helsinki: Gaudeamus.

MITCHELL, William John Thomas. [1986] 1987. Iconology. Image, Text, Ideology. Chicago: The University of Chicago Press.

NIKULA, Silja. 2012. Paikan henki. Matkailijan mielikuvasta graafiseksi kuvaksi [Spirit of a Place. From a tourist´s concept into a graphic image]. Rovaniemi: University of Lapland dissertation.

NIKULA, Silja. 2014. Narrative Potential in My Graphic Images. In Riitta Brusila & Hannu Vanhanen (ed.) Integrated Media in Change. Rovaniemi: Lapland University Press, 65–82.

PEIRCE, Charles Sanders. 1991. Peirce on Signs. Writings on Semiotic by Charles Sandes Peirce. Edited by James Hoopes. Chapel Hill: The University of North Carolina Press.

RANTA, Michael. 2013. (Re-)Creating Order. Narrativity and Implied World Views in Pictures. STORYWORLDS 5/2013. University of Nebraska Press, 1–30.

RELPH, Edward. 1976. Place and Placelessness. London: Pion Limited.

RIMMON-KENAN, Slomith. 1999. Kertomuksen poetiikka. 2nd edition. [Narrative Fiction: Contemporary poetics] Translated: Auli Viikari. Tietolipas 123. Helsinki: Suomalaisen Kirjallisuuden Seura.

STEINER, Wendy. 2004. Pictorial Narrativity. Ryan Marie-Laure (ed.) Narrative across Media. The Languages of Storytelling. Board of Regents of the University of Nebraska, 145–177.

USPENSKI, Boris. 1991 [1970]. Komposition poetiikka. Taideteoksen sommittelun periaatteet. [Poetics of a composition. The principles in composing art.] Helsinki: Kustannus Oy Orient Express.