DOI: 10.24308/iass-2014-110

Polyxeni Manoli

University of Thessaly, Volos, Greece

pegyma@hotmail.com

Maria Papadopoulou

University of Thessaly, Volos, Greece

mariapap@uth.gr

Abstract

The study aimed to explore whether young children, who did not have formal reading skills, could speculate the linguistic message depicted in contemporary multimodal texts based on information provided in various modes (such as, colour or typography). The contribution of each mode to the meaning-making process was investigated as well. A further aim of the study was to probe into students’ answers in relation to their age. The sample of the study consisted of 45 young children; 26 of the participants were five to six years old and 19 were four to five years old. The basic tool used in the research was a matching exercise derived from a picture book intended for young children. Semi-structured individual interviews were used for the data collection, which were tape-recorded and, later on, were transcribed and processed. The results of the study indicated that the majority of the children were able to use the typographic and colour cues along with separate letters they already knew to guess the linguistic content of short multimodal texts. It was also revealed that colour and typography played a critical role in helping students match the linguistic elements with the visual ones. Regarding the age variable, it was found that the group of the older children seemed to perform better than the younger ones but the difference was not found to be statistically significant. At the same time, the results highlighted the fact that children at this age tend to rely more on visual cues when approaching written texts. The study pointed out the need for educators to extend the limits of the semantic field of literacy beyond language texts in order to allow for multimodal texts, as nowadays, students, even from an early age, are exposed to an increasing dominance of multimodal texts -both print and digital texts- that involve a complex interplay of linguistic elements, visual images, graphics, and design elements.

1. Theoretical issues

Till recently literacy education in the Western societies has focused solely on language, ignoring all other communicational modes of meaning-making (Kress and Van Leeuwen 2006). However, in the last decades of the 20^th century, the technological development along with the wider changes in the socio-economic and cultural field imposed radical changes in the representational and communicational landscape that led to the need to redefine and broaden the limits of literacy in order to fulfill the current and future communication needs of all citizens (New London Group 1996). Contemporary multimodal texts require a new definition of literacy as new learning needs have arisen (Unsworth 2001). People, especially youths, are continually exposed to multimodal texts either print or digital, such as websites, video games, picture books, texts, magazines, advertisements, and graphic novels that include a complex interplay of written texts, visual images, graphics, and design elements (Kress et al. 2001; Kress and Leeuwen 2006; Unsworth 2001). As Fleckenstein (2002) asserts readers encounter more multimodal texts in their everyday life than print-based texts. In this context, meaning is derived from ways that are increasingly multimodal (Cope and Kalantzis 2000). According to Kalantzis and Cope (2012: 2), “we need to supplement traditional reading and writing skills with multimodal communications”. Multimodality refers to the active and dynamic interrelationship among the different semiotic modes of meaning which individuals can draw on during interaction with various texts to derive meaning, though one mode can prevail over the others (Baldry and Thibault 2006; New London Group 1996). In addition, research in the field of multimodality has highlighted new ways of conveying meaning from texts emphasizing the fact that becoming an efficient reader/viewer of multimodal texts is much more multitasking than just reading the words of a text. At the same time, multimodality, in particular the visual mode, can help young readers who have not developed formal reading skills access a text (Arizpe and Styles 2003; Pantaleo 2005; Papadopoulou 2001a; Yannicopoulou 2004). Images, colour or even typography can provide cues revealing the information in a text. Indeed, research has shown that children can even be more efficient readers of pictures than adults (Kiefer 1995; Meek 1988).

Among other texts, picture books for children have always relied on the visual mode to make meaning. Although there is quite a lot of research on how children read images in picture books (Arizpe and Styles 2003; Kiefer 1995; Styles and Arizpe 2001; Walsh 2003), only a small body of research regards picture books as multimodal texts and provides tools for the understanding of the contribution of the various modes, such as, typography or colour, to the meaning-making process (Papadopoulou, Manoli, andZifkou 2014; Serafini and Clauzen 2012; Walker and Linda 2003; Walsh 2003). In this context, acknowledging the contribution of the various semiotic modes to the meaning-making process, the purpose of the present study was to explore whether young children (4-6 year-old preschoolers) could speculate the linguistic message of multimodal texts included in a picture book based on information provided in various modes. Additionally, the study investigated the contribution of each mode to the meaning-making process as well as the relation between children’s answers and their age.

2. Method

2.1. Research questions

Allowing for the theoretical framework discussed above and the purpose of the present study, the following research questions were addressed:

• Could young children, who did not have formal reading skills, identify meaning in contemporary multimodal texts derived from picture books based on information provided in various semiotic modes (such as, colour or typography).

• Which of the semiotic modes contributed most to the meaning-making process?

• Was there an association between children’s correct answers and their age?

2.2. Participants

Forty-five (45) children aged four (4) to six (6) years old participated in the study. Twenty-six (26) of the participants were five to six years old and 19 were four to five years old. The sample of the study came from four state nursery schools in the city of Volos, in central Greece, while one nursery school located in the suburbs of the city of Volos. The choice of the sample relied on the following criteria: the children did not face learning difficulties or any other mental disorder; they had not developed formal reading skills. The two first criteria were judged based on their teachers’ perceptions, while the development of formal reading skills was assessed through a short test administered prior to the study. All participants had Greek as their first language (L1).

2.3. Research instrument

The basic tool used in the research was a constructed matching exercise drawn from a picture book, which was entitled Me + you = together. The art of being a grandmother (Marantidou 2010). The book was written in Greek but the specific extract was translated into English by the researchers. The specific extract was chosen for its variety of semiotic modes that provide visual salience by means of conventional or more expressive typography and colour. Initially, the page consisted of images and short phrases beneath each image explaining and extending what was symbolically depicted in each image. However, the page was modified for research purposes; the phrases were separated from the relevant images to construct a matching exercise. The first part of the phrases was written in normal black typefaces, while the second part was written using semiotic modes which diverged from the normal black typefaces.

The first image depicted a birthday cake with a candle, while the corresponding phrase was ‘the land of birthday’. The special element of the phrase was that the letter ‘i’(‘ι’ in Greek) of the word birthday (γενεθλίων in Greek) was written in a larger typeface than the rest letters and the accent was yellow bearing a resemblance to the flame of the candle.

‘the land of birthday’

The second image depicted pairs of feet dancing and hands moving around while the relevant phrase was ‘the land of dance’. The distinctive feature of the phrase was that the last part of the sentence was written three times on three different lines, which diverged from the regular line spacing and the horizontal dimension of the rest of the phrase to remind readers of the concept of dancing.

‘the land of dance’

The third image depicted a Christmas tree, while the corresponding phrase was ‘the land of Christmas’. What made the phrase diverge from the normal black typefont was the use of colour; namely, half of the letters of the word ‘Christmas’ (Χριστούγεννα in Greek) were in red, while the rest were in green to resemble the colours used to depict the Christmas tree in the image.

‘the land of Christmas’

The fourth image showed the sun, while the related phrase was ‘the land of light’. The distinctive feature of the phrase was the use of colour, which was gradually changing; namely, the word ‘light’ [φωτός in Greek] was written using the shades of yellow, such as red or orange, to help readers associate the phrase with the sun.

‘the land of light’

The last image depicted a piano, while the corresponding phrase was ‘the land of music’. The distinctive features of the phrase were that the letter ‘o’ of the Greek word‘μουσικής’ (music in English) was replaced by a note to remind readers of the concepts of the piano and music; moreover, the specific word diverged from the regular line spacing and the horizontal dimension of the rest phrase.

‘the land of music’

2.4. Data collection and coding

Semi-structured individual interviews were used for the data collection, which were carried out in Greek, the participants’ mother tongue. The interviews were conducted at the different nursery schools and lasted for 20 minutes approximately. The interviews were tape-recorded and, later on, were transcribed verbatim and processed to have objective record and reanalyze data after the interviews being conducted (Nunan 1992). The researchers coded the data independently until they had reached 90% agreement (inter-rater reliability) on the coding of the participants’ answers. In cases in which disagreement on the coding scheme occurred, the researchers met to discuss possible differences in order to arrive at a high level of consistency concerning the number of students’ correct answers and the type of semiotic modes identified by preschoolers as the reason which helped them answer correctly (Charmaz 2000; Patton 1990). The thorough data management and analytic procedures, such as the verbatim transcription and accurate records of the interviews, contributed to the validation of the research findings. The research belonged to a broader survey on preschoolers’ ability to derive meaning from multimodal texts, which was conducted in spring of 2011 and lasted for three months (Papadopoulou et al. 2014).

3. Results

The data of the study were analyzed using the Statistical Package for Social Sciences (SPSS) version 17.0. In accordance with the aims of this study, descriptive statistics as well as the statistical analysis of one-way Anova were computed. The level of significance was set at .05. To examine children’s correct answers, descriptive statistics, particularly frequencies, were computed. It was found that the majority of children were able to match the linguistic message with the correct image drawing on the semiotic modes used to convey meaning. Table 1 presents the frequencies and percentages of students’ correct guessing for each image:

Table 1. Frequency of students’ correct answers per image

In a next step, it was attempted to investigate the contribution of the various semiotic modes to the meaning-making process. Based on Table 2 below, it was shown that primarily colour and then typography played a critical role in helping students match the linguistic elements with the images, since most of the children seemed to have mainly relied on these two modes in order to justify their answers.

Table 2. Children’s reported justifications

Two third of the children (67%) achieved to match the image of the birthday cake to the corresponding phrase ‘the land of birthday’. Their choices were mainly driven by typography (“there are two candles”; “I saw the cake and I found the letter of the cake [i]”) and by colour (“there is a yellow accent”; “there is red here and red there”). Most children’s false matching (47%) was again based on colour associating the image to phrases abounded with it, such as ‘the land of Christmas’ or ‘the land of light’. A disparity of justifications was given by children for the second image which was correctly matched by 56% of the sample, a rather low percentage. While some children took notice of the fact that the last word (‘dance’) diverged from the usual linear alignment of sentences in print texts reminding them of the concept of dancing (“the letters are moving”: “[the letters] are going up and down”), other justifications such as colour, lettering and various answers (“it’s the last one”) were also equally mentioned. Most false answers were based on colour. Children were mainly attracted by the colour which, in this case, led to false answers. Mostly younger children failed to do the right matching. Relying on previous research (Papadopoulou 2001b), it is known that children find it difficult to accept the non linear alignment of a sentence, as most young children consider that writing has a horizontal layout). Letters and colour were the main modes that guided children’s attention to the matching of the third image. Twenty nine children made the right match and half of them justified their answers based on the knowledge of some letters appearing in the phrase (“there is an ‘X’”). An almost equal number of children referred to colour (“there is green here and green there”; “the letters are green and red”). False matching also indicated the ways children use to approach the visual mode. Some children mistakenly matched the image of the Christmas tree with the phrase ‘the land of music’ referring to the note over the letter ‘o’ associating Christmas with charades and songs. The image of the sun was easily associated with the ‘land of light’ by most children who used the mode of colour to justify their matching (77%). Few children made false matching also referring to colour but relating the ‘sun’ to other colorful phrases, such as ‘the land of birthday’. Last but not least, most children, who correctly related the image of the piano to the phrase ‘the land of music’, noticed the transformation of the letter ‘o’ to a music note.

Additionally, in order to investigate whether students’ correct answers associated with their age, one-way anova analysis was performed. The total number of children’s correct answers was used as the dependent measure. The results revealed that, though children aged five seemed to report more elements (M = 3.5, SD = 1.5) than younger children aged four (M = 3.05, SD = 1.5), this difference was not found to be statistically significant, F (1, 44) = .97, p > .05.

4. Discussion

The main aim of the study was to explore whether preschoolers (four and five year old) who had not developed formal reading skills could derive meaning from contemporary multimodal texts based on information provided in various semiotic modes (such as, colour, typography or other modes); another aim was to investigate which of the semiotic modes contributed most to the meaning-making process. The age variable was also examined in this study.

According to the findings of the present study, it seemed that preschool children were able to detect the information provided by various modes and use it as a cue to reveal the linguistic content of the phrases. More than two thirds of the children were able to answer the matching activity correctly based on information provided in various semiotic modes; in addition, they were able to provide a proper justification in each case. In particular, the results indicated that the mode of colour, followed by typography were mainly identified by children as the modes that contributed to the visual salience of the phrases and helped most of them answer correctly. Regarding the age variable, though it was found that the group of older children seemed to perform better than the younger group, this difference was not statistically significant requiring further research.

Overall, it was shown that the visual elements which deliberately blurred the boundaries between image and letter were identified by most children in the attempt to match the images with the correct phrases. In fact, the presence of colour and the rather ‘awkward’ typographic features of the words have attracted children’s attention in conjunction with their pre-existing familiarity with some letters of the alphabet (usually those appearing first in a word) due to the activities conducted in the nursery schools. The latter strategy is often used by pre-schoolers, who have not developed formal reading skills yet, in the attempt to have an early access to literacy (Papadopoulou & Poimenidou 2004). Few references were made to other reasons, such as ‘the only card left’ or ‘it is the smallest or the biggest card’.

There is no doubt that any deviation from the conventional typographic form of text makes the text impressive, funny, and familiar to pre-schoolers, as texts intended for this age are multimodal, in which the visual mode, especially colour, typography and images prevail. In fact, preschoolers, who have not developed full literacy skills yet, tend to draw on the visual mode when approaching written texts;  this  tendency should be cultivated and enhanced by educators, as nowadays  children, even from an early age are exposed to a variety of multimodal texts where the various modes of communication interact to produce meaning (Baldry and Thibault 2006; Kress and van Leeuwen 2001). Yet, in most cases, school curricula and educators draw more on language ignoring the contribution of other modes to the meaning-making process. In this context, there should be a shift of interest from language to the development of a multimodal literacy. As Martens et al. (2012: 293) advocated: “helping children read picture books multimodally and expanding their understandings of text not only enriches their reading, but also moves them toward being multimodal readers and creators of meaningful texts in our world”.

Taking everything into account, there seems to be an urgent need for educators and those who plan educational policy to familiarize students with multimodal texts enabling them to analyze, produce and critically approach information conveyed in various modes. Students have to be taught the processes involved in constructing meaning from multimodal texts since they are increasingly prevalent in everyday life. In this way, educators should teach students to take advantage of the various semiotic modes of communication in order to help them have better access to literacy and meet the communicational demands of the era (Kress and van Leeuwen 2001).

References

ARIZPE, Eveline & Morag STYLES. 2003. Children reading pictures. Interpreting visual texts. London and New York: Routledge Falmer. 

BALDRY, Anthony & Paul John THIBAULT. 2006. Multimodal transcription and text analysis. London & Oakville: Equinox.

CHARMAZ, Kathy. 2000. Grounded theory: Objectivist and constructivist methods. In Norman DENZIN & Yvonna LINCOLN (eds.), Handbook of qualitative research, 769-802. California: SAGE Publications.

COPE, Bill & Mary KALANTZIS. 2000. Introduction: Multiliteracies: The beginnings of an idea. In Bill COPE & Mary KALANTZIS (eds.), Multiliteracies: Literacy learning and the design of social futures, 3-8. New York: Routledge.

FLECKENSTEIN, Kristie. 2002. Inviting imagery into our classrooms. In Kristie FLECKENSTEIN, Lina CALENDRILLO & Demetrice WORLEY (eds.), Language and images in the reading-writing classroom, 3-26. Mahwah, NJ: Erlbaum.

KALANTZIS, Mary & Bill COPE. 2012. Literacies. Australia: Cambridge University Press.

KIEFER, Barbara. 1995. The potential of picturebooks: From visual literacy to aesthetic understanding. Englewood Cliffs, NJ: Prentice Hall.

KRESS, Gunther, Carey JEWITT, Jon OGBORN & Charalambos TSATSARELIS. 2001. Multimodal teaching and learning: The rhetorics of the science classroom. London: Continuum.

KRESS, Gunther & Theo VAN LEEUWEN. 2001. Multimodal discourse: The modes and media of contemporary communication. London: Arnold.

KRESS, Gunther & Theo VAN LEEUWEN. 2006. Reading images: The grammar of visual design. London: Routledge

MARANTIDOU, Xara. 2010. Εγώ + Εσύ = Μαζί. Η Τέχνη του να Είσαι Γιαγιά [Me + you = together. The art of being a grandmother]. Athens: Papyros.

MARTENS, Prisca, Ray MARTENS, Michelle Hassay DOYLE, Jenna LOOMIS & Stacy AGHALAROV. 2012. Learning from picture books: Reading and writing multimodally in first grade. The Reading Teacher 66(4). 285–294.

MEEK, Margaret. 1988. How texts teach what readers learn. Stroud, England: Thimble Press.

NEW LONDON GROUP. 1996. A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review 66(1). 60-92.

NUNAN, David. 1992. Research methods in language learning. United States of America: Cambridge University Press.

PANTALEO, Sylivia. 2005. Reading young children’s visual texts. Early childhood research and practice, 7(1). http://ecrp.uiuc.edu/v7n1/pantaleo.html  (accessed 10 January 2011).

PAPADOPOULOU, Maria. 2001a. Multimodality as an access to writing for preschool children. International Journal of Learning 8. 1-15.

PAPADOPOULOU, Maria. 2001b. Αντιλήψεις και Αναπαραστάσεις των Παιδιών Προσχολικής Ηλικίας για τη Γραφή [Ideas and representations of young children about writing]. In P. PAPOULIA-TZELEPI (ed.), Emergent literacy. Research and practices, 97-120. Athens: Kastaniotis.

PAPADOPOULOU, Maria & Maria POIMENIDOU. 2004. Διαβάζουμε τα γράμματα ή τα μετράμε; Οι στρατηγικές των παιδιών προσχολικής ηλικίας για να προσεγγίσουν ένα γραπτό κείμενο [Reading or counting letters: Preschool children's strategies while approaching a text]. In N. ZOUKIS & Th. MITAKOS (eds). Proceedings of the 3rd Congress of the Pedagogical Society of Greece. Retrieved from http://www.pee.gr/wpcontent/uploads/praktika_synedrion_files/e_athena/sin_ath.htm

PAPADOPOULOU, Maria, Polyxeni MANOLI & Elisavet ZIFKOU. 2014. Typography: How noticeable is it? Preschoolers detecting typographic elements in illustrated books.The International Journal of Early Childhood Learning 20(2). 23-36.

PATTON, Michael Quinn. 1990. Qualitative evaluation and research methods. Newbury Park: SAGE Publications.

SERAFINI, Frank & Jennifer CLAUSEN. 2012. Typography as semiotic resource. Journal of Visual Literacy 31(2).1-16.

STYLES, Morag & Evelyn ARIZPE. (2001). A gorilla with “grandpa’s eyes”: How children interpret visual texts-a case study of Anthony Browne’s Zoo. Children’s Literature in Education 32(4). 261-281

UNSWORTH, Len. 2001. Teaching multiliteracies across the curriculum: Changing contexts of text and image in classroom practice. Great Britain: Open University Press.

WALKER, Sue & Linda REYNOLDS. 2003. Serifs, sans serifs and infant characters in children’s reading books. Information Design Journal 11(3). 106-122.

WALSH, Maureen. 2003. Reading pictures: What do they reveal? Young children’s reading of visual texts. Reading 37(3). 123-130.

YANNICOPOULOU, Angela. 2004. Visual aspects of written texts: Preschoolers view comics. L1- Educational Studies in Language and Literature 4. 169-181.