Abstract

Visual narratives, such as found in comics and picture books, are often seen as easy to understand as their visual language is familiar to many from a young age, however, recent neuroscience research suggests it is complex. This paper discusses the main challenges in the visual design of the award-winning ‘Comics for Inclusive English Language Learning’ (CIELL) mobile application (app) that supports the needs of learners with dyslexia in the additional language class. The innovative teaching approach embodied in the app promotes literacy through webtoons (mobile comics) and has been designed to be dyslexia-friendly and inclusive. Around 10% UK population have dyslexia and other specific learning differences, however these students are greatly underrepresented in higher education and this is a worldwide issue of social justice. Language qualifications such as IELTS are key to accessing English-medium universities worldwide and so can be a major barrier. In addition, there is increasing pressure on language teachers to provide digital support for students. A narrative account from the visual designer explores the two principal challenges of the visual design, illustrated with webtoons from the app. First, some forms of essay writing are not typical stories and lack characters and key structural elements such as a climax, so characters had to be invented. Secondly, the visuals need to reinforce the memorisation processes of learners with dyslexia, so metaphors were used. However, this is not simply a process of translation or literal illustration as the visuals add another layer of meaning to the stories and therefore needs to be taken into account within the overall design. This educational approach shows how visual narratives in webtoons can create an inclusive, dyslexia-friendly mobile app for teaching and learning of English as an additional language. While this is only one example, it contributes insight into the theoretical and practical challenges of the visual design and may inspire further research in this under-researched area.

Keywords: comics; webtoon; visual metaphor; dyslexia; inclusion; mobile learning

Part of the Special Issue Visual literacies and visual technologies for teaching, learning and inclusion

1. Introduction

Visual narratives are a common form of storytelling that are well established in many cultures and may take on many forms to convey news, information, or provide entertainment, for example, comics, graphic novels, photojournalism, videos and documentary films (Cohn, 2013). Visual narratives are so familiar to us from an early age that they are often considered easy to understand (Cohn, 2020). However, research on comprehension of visual narratives and cognitive neuroscience that measures brain activity suggests that the cognitive processing involves complex visual languages (Cohn, 2019). This apparent ease of understanding has meant that comics feature in many educational interventions: to assist with language difficulties (Millogo, Gaufreteau & Millogo, 2021), to support language learning (Källvant, 2015), and to promote literacy, critical thinking and 21^st century skills for the workplace (Krusemark, 2017). The multimodal nature of comics, or mixture of text and sequential art, can be seen as more authentic than purely textual literature, especially online and in social media where communications are rarely about words alone and more often requires multimodal literacy practices to interpret a combination of pictures and words (Mills & Unsworth, 2017).

Comics are widely used in teaching to promote literacy, critical thinking and discussion about a wide range of issues such as citizenship (Brand EU, 2018), regional development (Biancarelli et al., 2014), and social and environmental issues (Comics Uniting Nations, 2015). Comics are increasingly being used in technology-enhanced learning, for example, the COMMIX (2017) project created training and guidance for teachers on how to use freely available tools so that learners can produce their own comic strips. Shiang’s (2018) research suggests why this may be beneficial. Shiang (2018) compared two groups of English language learners who either produced a comic from a text or translated the text. Those who created comics scored higher on a reading comprehension test suggesting that synthesis of the text and the construction of the multimodal comics developed their understanding.

Research into the use of comics in learning an additional language has shown that they can motivate and inspire reluctant and struggling readers (see reviews by Källvant 2015; Themelis & Sime, 2020). They can positively affect attitudes towards reading (Cimermanová, 2015), increase motivation (Jones, 2010), reduce test anxiety (Popa & Tarabuzan, 2015) and develop a passion for storytelling (Versaci, 2008). Positive benefits can be achieved by strengthening emerging skills and making learning more fun (Zimmerman, 2010). Learners with dyslexia may have trouble with retaining information in memory (Kormos, 2017) therefore they may need more time and repetition to memorise information. This may make it more difficult to maintain focus on a task (Kormos & Kontra, 2008) so they are in danger of losing motivation (Kormos & Csizér, 2010). Adopting a multimodal approach, such as using comics in language teaching, can be helpful as there are fewer words per page and the sequential nature of the comic strip helps the reader follow the plot more easily as a result this may reduce stress and make reading more appealing.

Learners with dyslexia may feel that they are not being adequately supported by their higher education institution (Griffin & Pollack, 2009). Due to lack of confidence and low self-esteem, learners with dyslexia may need support for negative emotions (Richardson, 2021) and Kormos (2020) suggests that this can be achieved by adopting an inclusive approach. Providing an inclusive approach in the language learning class may also help overcome the challenges that teachers experience in identifying and supporting learners with dyslexia. An inclusive approach to teaching may be designed to support those with dyslexia but can also benefit learners without a dyslexia diagnosis (Richardson, 2021). However, it is worth remembering that no solution will suit all learners, and even the diagnosis of dyslexia is a broad categorisation where no two learners are identical in their cognitive abilities, emotional experiences, or coping strategies.

From the teacher’s perspective, recognizing learners with dyslexia may be a challenge in the language class, as not all learners with dyslexia in their first language have problems reading in a second language, or vice versa (Kormos, Košak-Babuder & Pižorn, 2019). This is compounded by the possibility that teachers may be over-reporting their ability to recognize learners with dyslexia in their language classes (Kormos & Kontra, 2008). Providing an inclusive teaching approach reduces the need for identification of learners with dyslexia and individual accommodations as the same approach is used for those learners with and without dyslexia.

In response to these issues, the Comics for Inclusive English Language Learning (CIELL) project consortium designed and implemented the CIELL mobile application (app) to assist language teachers in implementing inclusive teaching practices. The CIELL app provides a free, digital resource for language teachers that helps support the needs of learners with dyslexia in a socially inclusive manner (Sime & Themelis, 2021). Specifically, the learning objective of the app is for learners to understand how to plan, or structure, an essay, such as an opinion essay, a review, or an article. This is important for language learners studying for the writing component of English language qualifications such as British Council examination (IELTS) and Cambridge University English proficiency at the Common European Framework of Reference for Languages Level of B2+. These English language qualifications are essential for students to access English-medium universities.

In this paper, we take a detailed look at the visual design, and discuss the main challenges in designing an inclusive, dyslexia-friendly, mobile app that uses comic art, in the form of webtoons, to reinforce the learning process and support multimodal meaning-making. A webtoon is a comic that is designed to be read on a smart phone through a mobile app. We propose that when designed and implemented well, a mobile app could be a valuable device to improve the student learning experience but there is a lack of theoretical understanding and practical guidance on how to design an app to support learners with dyslexia. So, the question is “What are the challenges in visual design of a dyslexia-friendly, multimodal mobile app based on webtoons?” The answer to this question may be of interest to learning technologists, app designers, language teachers and researchers interested in this novel technology-enhanced learning approach.

The next two sections discuss dyslexia and inclusion in more detail, as well as introducing the CIELL app which embodies this innovative approach to supporting language learners with dyslexia in an inclusive manner. Sections 4 to 8 contains the visual designer’s narrative (second author) about the visual design of the webtoons, in particular, the two main challenges and how they were overcome.

While the app has been very well received by language teachers and gained an award for teaching innovation from the British Council (BC ELTons, 2021), evaluation of the app as a teaching resource is considered to be outside the scope of this paper which focuses on the visual design. For an evaluation of the app see Joannidou and Sime (2021). So, why do we need to focus on supporting learners with dyslexia?

2. Dyslexia and access to higher education

In the UK, there is a serious under-representation of students with dyslexia and other specific learning differences in Higher Education and this is an issue of social justice. Dyslexia is a specific learning difficulty where learners may struggle with processing words, spelling, handwriting, reading, memory and attention (Kormos & Smith, 2012). The British Dyslexia Association says that 10% of the UK population have specific learning difficulties including dyslexia, dyscalculia and dyspraxia (BDA, 2019). This issue is not restricted to the UK and is a problem worldwide (Maunsell, 2022), for example, Shaywitz and Shaywitz (2020) claim that 20% of the USA population have dyslexia although the methods of diagnosis do vary and may contribute to this difference. Within higher education, a comparative study by MacCullagh, Bosanquet & Badcock (2017) found that only 0.48% of UK and 0.16% of Greek students have dyslexia. This falls far short of the 10% seen in the UK population and shows how opportunities within this group are severely limited. So, the need for teachers to support this learning difficulty is strong and widespread.

While there are increasing numbers of students with dyslexia entering higher education many struggle with the label “dyslexic”, and with negative experiences of education (Pollack, 2005). There can be a large variation between individuals with dyslexia in the challenges that they experience and in the compensatory strategies that they develop over time (Pino & Mortari, 2014; Richardson, 2021). Common challenges are difficulties in note taking in lectures, problems with organisation and time management, and memorisation (BDA, 2022). As a result, there are a wide range of coping strategies used by learners with dyslexia (Richardson, 2021) hence, it can be difficult to understand how best to provide support. However, if learners with dyslexia are not supported effectively in our education systems, these difficulties can result in learners that are demotivated and marginalized (Pino & Mortari, 2014; Richardson, 2021). One of the main difficulties for students with dyslexia is transforming thoughts and ideas into written essays and many adopt coping strategies that include visual formats and visual techniques, such as mind mapping concepts, using essay plan diagrams to assist with structuring writing, or highlighting text with colour to assist in memorisation (Pino & Mortari, 2014).

These challenges also exist in the English as an additional language class, however coping strategies used in the first language do not necessarily transfer to the second language learning setting so dyslexia may become more apparent (Schwarz & Terrill, 2000). For English language learners, lack of English language qualifications can be a barrier to accessing English-medium higher education institutions worldwide. So, there is a great need for teachers to provide support for learners with dyslexia in the second language class.

3. The CIELL app and inclusive design

There is some debate in the literature about how we should view disability. There are three perspectives on learning disabilities, the medical model, the minority group paradigm and social construction theory (Brown & Broido, 2015). The medical model is based on a scientific comparison with a norm, or average person, stressing the biological limitations of an individual and not taking into account social and environmental factors. The minority group paradigm assumes that there is diversity in ability and sees people with disabilities as a minority group where societal factors such as power and prejudice are major obstacles. A third perspective is based on social construction theory and embraces people with and without disabilities as part of human diversity. This view recognizes societal influences and sees disability as a socially constructed phenomenon and not simply an individual characteristic. This is the theoretical framework that we have adopted in designing a socially inclusive app.

By social inclusion, we mean that the app is designed with an inclusive pedagogical approach that rejects identifying learners as different and aims to make learning available to all students while accepting that variety is normal:

Inclusive pedagogy is an approach to teaching and learning that supports teachers to respond to individual differences between learners but avoids the marginalisation that can occur when some students are treated differently. (Florian & Spratt 2013, p. 119)

A systematic scoping review of research into inclusive pedagogies in higher education revealed that there are two different theoretical approaches based on underlying values with contrasting positions (Stentiford & Koutsouris, 2020). While all students are expected to have common and individual needs, the emphasis on difference or commonality varies; one approach emphasises individual differences while the other prioritises commonality. So, if an individual differences approach is taken then the emphasis is on providing support that meets the needs of each particular individual, however, many students do not disclose their diagnosis or may be undiagnosed and therefore not receive appropriate support. An example of prioritising commonality is seen in the approach of Universal Design for Learning (CAST, 2018) which aims to meet the needs of students without highlighting differences that may exacerbate marginalisation. Universal Design for Learning (UDL) assumes that any design that does not cater for human variation is disabling to learners (Richardson, 2021). The CIELL app had been designed with the UDL generic design guidelines in mind, and consequently uses multimodality and visual representations of knowledge. The app can be downloaded for free from Google Play, the Apple Store, or viewed in a browser (http://ciell.eu/app).

As designers we recognize that dyslexia manifests itself in different ways in different individuals so while the app does not guarantee that it is suitable for all learners with dyslexia, it has been designed to be inclusive and as ‘dyslexia-friendly’ as possible to reduce the need for teachers to modify their teaching resources in the language classroom. Adopting an inclusive approach like this means that the app can be used by learners with dyslexia and those who do not have dyslexia.

The app design supports learners with dyslexia and other specific learning difficulties through its multimodal presentation of attractive webtoons which aim to enhance motivation and engagement and also by supporting memorisation through repeated practice and a visual representation. The multimodal approach means that learners can read the text, look at the visual images or listen to the text being read to them. The essays, written by teachers of English as an additional language, are based on topics related to the 17 United Nations Sustainable Development Goals for 2030 (UN SDG, 2022). This means that the topics have global relevance and so should appeal to many learners as they address social, economic and environmental issues such as food poverty, inequality, climate change, environmental degradation, peace and justice, homelessness, and water pollution, see Figure 1. The topics are not sequenced so the learner is free to choose any webtoon that appeals to them thereby increasing the likelihood of engagement. Memorisation is promoted through: the colourful webtoons that illustrate the essays; a visual representation in the form of an infographic which provides guidance on how to structure a basic essay, and the opportunity for repeated practice as there are 78 assessment tasks and 17 webtoons (for more information, see Sime & Themelis, 2021).

The next section provides a narrative account of the visual design of the webtoons from the visual designer, the second author, that explains the reasoning behind the design decisions and the solutions to two main challenges.

4. Visual design and the importance of metaphor

The main consideration of language teachers involved in the development of the project, was to ensure that the visual content accompanying the texts, would facilitate learners’ comprehension of abstract concepts, especially for those with learning difficulties such as dyslexia.

The first issue that has been raised was whether images could constitute an accurate translation of verbal content. Translation as a procedure where an original text, often called as the ‘source text’ is replaced by a substitute one, the ‘target text’ (House, 2018), the concept of ‘text’ used in a semiotic sense, is a well discussed topic among semiotics scholars and theoreticians of communication. Victoria Welby, more than a century ago, approached translation as a method of research and discovery in which “everything suggests or reminds us of something else” (Welby, 1903, p. 34). Russian linguist, Roman Jakobson, described it as ‘interpretation’ while Umberto Eco (2001, 2003) argues that “translation is based on a process of negotiation where the loss of something is accepted in order to gain something else” (Kourdis, 2015, p. 308). A negotiation that occurs not just between words but between cultures.

All of these views suggest the impossibility of achieving an ‘accurate translation’ between two different verbal semiotic systems. This is due to the cognitive and psychological state, the cultural and ideological positions, of the translator who first has to decode the source text in order to encode it again in another sign system, as well as the specific inherent characteristics of each code which might resist translation. An impossibility that becomes even more evident in attempts of “intersemiotic translation or transmutation, an interpretation of verbal signs by means of signs of nonverbal sign systems” (Jakobson, 1959).

So, treating images as translations in the strict sense of the term was rather impossible, images would rather function as visual explanations by employing analogies and metaphors, devices frequently used by educators and designers in order to explain new concepts. As Lakoff and Johnson (2003) state:

metaphor that is typically viewed as characteristic of language alone … is pervasive in everyday life, not just in language but in thought and action. Our ordinary conceptual system, in terms of which we both think and act, is fundamentally metaphorical in nature. (Lakoff & Johnson, 2003, p. 3)

To give an example of the use of visual metaphor as a hermeneutic tool, see Figure 2. “The Great Green Wall” webtoon describes the efforts being made by people in eleven African countries against desertification (https://www.greatgreenwall.org). While the text mentions that they are “combatting” land degradation, the image presents an African woman marching in a pose typical of a soldier. She is looking ahead (to the future) in a determined gaze holding something resembling a spear, whose top is cropped out of the image. It is only revealed as an agriculture tool in the next panel where the same character is depicted leading a team of women holding similar tools. They are all walking in desert up a hill (against difficulties). The character is inspired by Thikene Ndiaye – The President of the Women's Committee Fatou Seye in Senegal. She is deliberately illustrated with the same colours that represent earth (brown) and acacia (yellow) as used throughout the webtoon.

This paper also discusses some other methods adopted in order to enhance textual comprehension such as the creation of parallel, character-driven storytelling, and other design choices regarding the aesthetics of the final product.

5. Why comics?

Firstly, a distinction has to be made between comics, and certain other types of visual representations that might be considered as stories. More precisely illustrations, photos or infographics might employ narrativity, however, comics use images in a very different way compared to all that media. Images in certain illustrated books, can be read isolated or out of their given order. In contrast, comics tell stories through the use of sequential images, that is to say panels that are read and perceived by audiences one after the other, so they depend on and affect one another.

Comic panels, indeed, present in a visual format all the mandatory structural elements of a typical narrative, namely interrelated events in time and space, a beginning, a middle and an end. Stories in their turn, can function as memorisation tools, exactly because they can present information in a coherent and connected way instead of isolated and disconnected facts and concepts (Banister & Ryan, 2001). Visual narratives can support even better memorisation, especially for learners with lower level reading skills, since the “human brain processes images 60,000 times faster than text, and 90 percent of information transmitted to the brain” (Eisenberg, 2014).

Thus, when complex content is visualised, learners with a range of learning difficulties like dyslexia, can grasp its meaning easier. With better understanding, learners with dyslexia can get further motivated and engaged in the learning process, since stress that interferes with memory and attention is diminished. As Cole puts it “emotions can open and close the gates of learning” (Cole, 2014).

6. Two main challenges

In a game there are rules:

Without the basic rules or disciplines, however, there is no motivation, test of skill, or ultimate reward — in short, no game. The rules are the means to the end, the conditions the player must understand thoroughly and work with, in order to participate. (Rand, 1965, p. 4)

This section discusses the two main challenges in the visual design with illustrations from the app.

6.1 Essay versus stories

The first main challenge is related to the difference between essays and stories. Essays are informative pieces of writing and while their tone may vary among different types, their intention is to make a specific point whether by informing, persuading, describing or explaining certain ideas to readers. Most of them are composed with all the mandatory structural elements of a typical narrative, yet they are not typical narratives, and certainly they are not scenarios. Scenarios are texts that are written in order to be visualised, e.g. to take the form of a film, or a comic book.

Instead of presenting data, abstract ideas, opinions or arguments, most story plots focus on characters and their actions. Characters in their turn, may not be necessary requirements of a narrative as Aristotle suggested in Poetics. Aristotle prioritised plot and actions over characters, however they make stories more interesting for audiences. Neuroscientists Yuan, Major-Girardin and Brown (2018) found that with all forms of storytelling, the brain was activated and affected by the character’s thoughts, emotions and deeds.

Another important element essays lack compared to narratives, is a certain ‘peak’: an element that in most intriguing stories, falls in the middle or closer to the end and occurs when the subject faces a conflict, reaches a revelation, or changes their behaviour. In contrast, in most CIELL essays each paragraph introduced new ideas thus it seemed to have its own climax and resolution. Without the help of the above story conventions, comics created for the CIELL app would risk becoming disconnected depictions of certain concepts rather than stories. In order to extract a coherent story out of each essay’s information, certain characters were introduced even if the texts were not explicitly about them. These ‘agents of information’, at times, were inspired by public figures known for their contribution in different domains, such as: Ray Charles, an American musician; and Ladi Kwali, a Nigerian potter.

Ray Charles, the American musician who became blind during childhood due to glaucoma, was the inspiration behind the “Reduce Inequality” webtoon dealing with issues of accessibility in education and public services for people with disabilities. While the essay was talking of people with disabilities in general, the images presented a particular case of a blind boy and his mother, both figures being adaptations of the actors who played these roles in the American biographical film “Ray” directed by Taylor Hackford (2004), see Figure 3.

Ladi Kwali, the Nigerian potter, whose ceramics became art objects that have been exhibited and appreciated worldwide, served as the model behind the “Decent Work” webtoon that discusses labour rights and entrepreneurship. A fictionalised version of a young Ladi Kwali, who appears to be working alone under unsafe conditions, is compared with the story of a young woman, working at her own enterprise in much better conditions in Sifnos, a Greek island also known for its long tradition in pottery making, see Figure 4.

By the end of the essay the ‘older self’ of Ladi Kwali seems to teach her younger self the art of pottery making, see Figure 5. The conclusion of the essay, which argues that educated and trained individuals can lead a country to success, is visualised by a stylised picture of the older Ladi Kwali as it actually appears at the back of the Nigerian 20 Naira banknote. Both Ray Charles and Ladi Kwali represent achievement, as though they may have faced poverty and discrimination, they excelled at their craft and became pop culture icons.

Most types of essay do not contain characters, the sole exception is narrative essays which include characters, may use dialogue, and all incidents revolve around a single motif. Such essays may use expressive means or tools frequently used in cinema such as flashbacks and flash-forwards, and rely on concrete, sensory details to convey their point (EssayInfo, 2018). The “Agatha Christie” webtoon is such an example, as it is an essay that already contains a character at the centre of its narration.

While the author is presenting Agatha as the “queen of crime stories”, her representation brings connotations of certain depictions of Queen Elizabeth. In the same webtoon, Agatha’s primary school teacher’s facial features are borrowed from Emmeline Pankhurst, the organiser of the UK suffragette movement, see Figure 6. Thus, Agatha’s character is enriched with several connotations, those of a very important person as well as those of a very determined and active figure.

Hence, inserted characters functioned as agents of information and brought additional connotations, while at the same time they ensured a unity of action and established a visual continuity between the panels. While designing, I tried to devote as many panels as I could to their actions to give the audience more information about these figures, knowing that a character is not just someone who acts, but a person with motives that explain his or her behaviour.

Eventually, even if each paragraph of the essay seemed to contain its own climax, the interest of the spectator was guided towards the last paragraph, the conclusion. Like Freytag proposes “such elaborate climaxes, these must be united by means of shorter scenes, like joints, in such a manner that the stronger interest will always rest on the later elaborate scene” (Freytag, 1984/1896, p. 213).

6.2 Literal versus metaphorical

The second main challenge in the visual design is related to the difference between literal translation and the use of metaphor. In bimodal forms of expression such as film, images do not repeat what the dialog is saying, rather, they complete or extend it. The same goes for comics, pictures are understood in context and in combination with verbal information, and it is precisely this interaction that produces meaning. Images may have a parallel relation with the text’s meaning, they may add information, or even evoke an ironic contrast with the verbal part. However, that was not the case with the CIELL app whose educational nature was on the one hand, one of its basic aspects, and on the other hand a certain constraint. In order to facilitate the learning of learners with dyslexia, images had to be explanatory, thus visually representing the text they accompanied.

From a semiotic perspective all images are seen as signs. However, others are iconic (in Greek: εικόνα, from the verb εἴκω = resemble), indexical (i.e. dependent on context) or symbolic. Physical objects are visible entities, thus in design terms they can be ‘literally’ represented, so a pictorial representation of a chair, an icon of a chair, has a physical resemblance to the signified, i.e. the thing being represented, the physical chair. But what shapes or colours should a designer choose for the concept of development, thinking or dreaming? It becomes clear that abstract concepts have to be treated differently.

In creating the visual content designs of the CIELL webtoons I decided that, even in cases where the option of literal translation was possible, the result would probably bore readers who might perceive the app as an illustrated dictionary rather than a collection of visualised stories. So, in order to describe the text without visually repeating it, or literally representing what it was saying, I used visual metaphors, one of the most important tools of filmmakers, “which is the ability of images to convey a meaning in addition to their straightforward reality” (Brown, 2011). While designing, I tried to follow the meaning(s) of the text, panel by panel, however, at the same time I was writing my own scenario that would eventually take the form of images. Still, whatever story was invented, it could never contradict the text nor antagonise its meaning. The depicted actions of characters, or any other visual representations had to metaphorically reflect the text's meaning. 

The next section focuses on two applied examples of this design approach where the visuals appear to have two levels of reading: the first is the literal, it is my scenario, the action and describes what is actually going on in the panels; the second is metaphorical and reflects the text’s meaning.

7. Employing metaphor: Images as visual explanations

This section discusses the visual design through two examples: the first is taken from the “Agatha Christie” webtoon which is an essay focusing on an issue related to the United Nations Sustainable Development (UN SD) Goal 4 of quality education; and the second is “The Invisible Girls” webtoon which is an essay based on a research report into gender equality and is related to the United Nations Sustainable Development Goal 5 (UN SDG, 2022).

7.1 Agatha Christie

The author of this particular narrative essay (Chryssa Themelis) was reasoning why Agatha was her favourite author, by describing many different facts about her life which are depicted as “frame stories” – shorter stories within the general one. This section focuses on the visualisation of the first and the third paragraphs of the essay.

7.1.1 Frame story 1: Agatha at home, writing

Text: “Despite her learning difficulties (dysgraphia) she was a prolific writer and is known for her: 66 detective novels; 14 short stories; the world's longest-running murder mystery play called ‘The Mousetrap’; and six romance novels.”

Image, level 1 (My scenario: literal - action): Through the point-of-view of the narrator we see an everyday scene from Agatha’s life, in her living room, see Figure 7. We see a close up of her hand writing a sentence on a piece of paper. She is sitting at her secretaire, an old writing desk filled with books. Eventually, she interrupts her usual writing routine, she gets up, crossing the dark living room where several torn handwritten pages are lying in the floor. She is heading to the corridor where she puts on her coat while a mouse is spying on her. Then she puts her hat on. At the next panel, she gets her award from the Queen.

Image, level 2 (metaphor - meaning): Distorted writing serves as an indexical sign of bad handwriting as well as a metaphor for Agatha’s dysgraphia, while the number of books that surround her provides evidence that she was a “prolific” writer. The torn pages on the floor might be a metaphor for the concept of “short stories” compared to the books on her desk a characteristic format of “detective novels”. Other types of stories like “mystery plays” or “romance novels” are implied by the general mood and lighting of the images as well as by her actions. While Agatha is putting on her hat, she is depicted in a ‘romantic mood’ by the way she is wearing her hat, the soft lighting of the image, or elements such as the flowers reflect the meaning of the words “Romance Novels”. Finally, her routine, the fact that she is always writing until the last moment, even before a very important ceremony, is a visual metaphor, also an aetiology, for her persistence as the text mentions: “The Queen awarded her the title: Dame Commander of the Order of the British Empire for her contribution to literature. Her talent, persistence and hard work, despite her learning difficulties, make her exceptional.”

Thus, some concepts mentioned previously or in the following paragraphs might be implied in these part’s visuals as connectors, elements that facilitate the flow and preserve the unity and the coherence of the narrative. The last panel where Agatha holds the Queen’s hand, suggesting her receiving the award, consists of an ellipse: many actions that took place in the diegetic world before this event are left out, “still, the reader can easily interpret the whole sequence on the basis of its similarities to known sequences in daily life” (Lefèvre, 2000).

7.1.2 Frame story 3: Agatha on the London Underground

Text: “As a famous novelist, playwright, and short story writer Agatha Christie is called the queen of crime mysteries. As a matter of fact, she continued to write, experimenting with different types of thriller and murder mystery stories for all her life. Her crime novels have produced some of the most recognizable characters in British literature such as Miss Marple and Hercule Poirot. The New Yorker Journal states that detective stories were invented by Edgar Allan Poe, but Agatha Christie revolutionised murder mysteries and they have become firmly rooted in British culture. Agatha Christie could be a role model for all of those who struggle with writing. Miss Marple and Hercule Poirot are widely recognized characters and their stories are read in many languages across the world”.

Image, level 1 (My scenario: literal- action): Agatha is sitting on a bench, in the London underground of the beginning of the 20th century, see Figure 8. Behind her, stands a man (Edgar) who is looking at her. As the train arrives, the doors are opening and Agatha is entering the train, leaving him behind her. Inside the carriage, where there is a dead body, passengers are wearing gabardine coats, hats and are smoking pipes. Agatha observes them, smiling, as if she knows the identity of the murderer. A man, sitting somewhere in the carriage, is hiding his face with a book. Zoom in on the man’s bloodied fingers, and the book’s cover, Agatha Christie's “Murder on the Orient Express” in a Chinese translation.

Image, level 2 (metaphor - meaning): The establisher of this sequence, a dark tunnel of the London underground is a visual metaphor for the concept ‘detective story’, then the fact that Edgar is standing behind Agatha, resembles the visual arrangement of elements in a typical timeline thus implies that he might be the man who arrived first, and consists a metaphor for the word ‘inventor’. This impression might be reinforced by the fact that, probably, Agatha is leaving this station and Edgar behind her - station / σταθμός in Greek also means milestone or turning point. Eventually, the way Agatha opens the doors of the train is a visual metaphor for the verb “revolutionised”, and a connotation of expressions like ‘opening doors to the future’.

Finally, the conventional dress of all other passengers in the train, is a metaphor for the popularity of detective stories in Britain. The proudly standing figure of Agatha visualises the concept of ‘role model’, while the stylized close up of the Hercules Poirot (drawn with less detail) persona, is a metaphor for the concept of a ‘widely recognisable character’. The last panel of the sequence zooms out revealing “The Murder on the Orient Express” cover in a Chinese translation, which is a visual metaphor for ‘foreign languages’ or other distant cultures, reminiscent of the expression ‘it is all Chinese to me’!

7.2: The Invisible Girls

The essay for this webtoon compares the lives and educational opportunities of young boys and girls in developing countries around the world. Based on a report on girls’ rights and realities (Plan, 2016), it provides examples from three different countries Zimbabwe, Nicaragua, and Pakistan to which three distinct colours were assigned, and served as locations or “settings” for the visual narrative, see Figure 9. For every new idea or example introduced, one short sub-narrative was created, located in one of the three locations. Eventually, all three interconnected sequences form one narrative which comes together in the last scene.

Text: “Due to poverty and societal factors, they [girls] often get married young, focus on household chores, and have limited free time to study. For example, where finances are limited parents may choose to educate their son rather than their daughter. As a result, they depend on their spouse for survival and have difficulty realising their dreams.”

Image, level 1 (My scenario: literal - action): A little boy plays football around garbage, while his mother (or elder sister) is washing the dishes, see Figure 10. Later on, the boy is reading a book, while his little sister prepares food. Then, he goes to school while his mother and sister both stay at home, in the darkness.

Image, level 2 (metaphor - meaning): While the text mentions that word “poverty” the reader of the comic sees the garbage and a stray dog in premier plan. Both the scene setting and the stray dog function as visual metaphors for the word “poverty” recalling expressions like ‘a person dying like a dog’. Then, literally the boy is going to school, but metaphorically ‘he is realising his dreams’ as the text is saying. The dark room, and the fact that both girls stay still while the boy is moving on are visual metaphors for ‘the inability that girls have to realise their dreams’. Finally, the last panel represents the effect of ‘having no dreams’, as mentioned in the definition of a narrative: “a chain of events of cause and effect relationship occurring in time and space” (Thompson & Bordwell, 2008). Thus, in narrative terms the effect coincides with the girls looking from the ‘inside’ to the ‘outside’ like prisoners do.

8. Style as additive information and technical constraints

After the initial planning, the visual design process became more technical. However, the representation process adds information in a way that affects the overall interpretation of the concept depicted. Thus, the general idea of a chair in the platonic sense, might become this or that specific chair, a Louis XV chair, a Bauhaus chair, a broken chair, a distorted or a typical one. Similarly, stylistic choices, such as the degree of stylisation, the naturalistic or expressive colour, or the use of a certain typography, all contribute to the production of meaning (Kourdis, 2015). Graphic choices, by completing or competing an image’s meaning(s), one thing is certain, they alter it. As Lefèvre (2000) argues “whatever graphic style an artist uses, he is always not only depicting something, but also expressing at the same time a visual interpretation of the (fictive) world. Every drawing style implies an ontology of the representable or visualizable”. Hence, the formal characteristics of a visual can only be separated from its content, for the sake of studying its inner mechanisms and production methods.

Secondly, the digital format of the comics, played a decisive role in their creation, in fact it affected not only the overall design strategies of the visual material of the app, but its content as well. The publication format and the medium’s nature guided the whole production team from the very beginning of the process: sentences had to be shorter, images had to adapt in different screen sizes, the whole material of the app could not exceed a certain number of megabytes in size. Printed comics have this quality, where readers through seeing a page, can instantly understand it as a whole whose parts operate as units of an interconnected whole. In digital formats such as webtoons, panels exist in a continuum, read with a scroll down without the intervention of separate pages, and in this particular case they were all designed in square format, so that they could be easily rearranged depending the different screen sizes of user’s tablets and mobile phones, see Figure 11.

Finally, since the app size could not exceed certain megabytes, webtoons had to be visualised with a limited number of panels, thus imposing an even more fragmented narrative style on top of the ‘elliptical nature’ of the medium (Lefèvre, 2000). Due to this technical constraint each essay’s coherence had to be established in three to seven panels, thus resembling the format of comic strips.

9. Conclusion

There is a serious under-representation of learners with dyslexia in higher education. While technology-enhanced learning could be used to help address this issue there is a lack of theoretical understanding and practical guidance on how to design resources to support learners with dyslexia. Focusing on language learners seeking English qualifications that provide access to English medium universities, we reviewed the literature to discover the needs of learners with dyslexia, how comics are being used for learning, and identified some generic design principles for inclusive design based on universal design for learning (CAST, 2018). However, the guidance was not specific enough, especially in relation to the visual design, so we asked: “What are the challenges in the visual design of a dyslexia-friendly, multimodal mobile app based on webtoons?”

Design is all about problem solving, where constraints are seen as requirements, not as restrictive factors. A basic aspect of the CIELL app was its educational and inclusive nature. A key challenge in the visual design was that essays lacked basic elements of typical stories such as descriptions of characters, or structural elements such as a climax. Characters had to be invented where they were not explicitly stated thus ensuring a visual continuity between the panels, while at the same time guiding the readers’ interest towards the last scene.

Secondly, the focus on the needs of learners with dyslexia required visuals to be not only communicational, but also assist in memorisation as well. Images were treated as visual explanations: visual metaphors were used in order to reflect the texts’ meanings, even where it was possible to literally represent them. By developing a scenario in which the character’s depicted actions would metaphorically reflect the meanings of the text, the essay was ‘translated’ in visual terms. 

However, this role as translator was not transparent. By selecting a specific metaphor, each time, the designer of the visual content was equally contributing, with the author of the essays and with the reader of the webtoons, to the production of meaning. As words were polysemic the same would happen with their supposed visual equivalents and their formal aspects. The visual part would pick up and expand on the meanings expressed by the verbal part. Representative images always carry connotations, yet abstract shapes and colours add information too, due to their inherent characteristics. Then, style, as a series of choices, does the same: it suggests worlds.

Hopefully, while analysing and interpreting words, choosing, organising and synthesising shapes, the ethical concerns of the texts are preserved, and add meaning in a sense that the author of the essays would approve so that they are suitable for the learning purpose and the learning audience for whom they are designed.

The contribution of this research is that it provides insight into the challenges of visual design of an inclusive approach to technology-enhanced learning in the language class based on webtoons. It illustrates how visual narratives in webtoons can be used to create a dyslexia-friendly app to teach essay planning for language learners with and without dyslexia. As such it provides practical guidance and explains the theoretical basis of the socially inclusive app in the hope that this might assist future researchers and app developers. However, this is just one approach to designing technology-enhanced learning resources for language learners with dyslexia and we hope that this will inspire researchers to investigate the area further.

Acknowledgements

The CIELL project (https://www.ciell.eu) has been co-funded by the European Union under the ERASMUS+ programme, with Centre for Technology-Enhanced Learning, Educational Research Department, Lancaster University (UK); Language Centre, University of Cyprus (Cyprus); Institute for Learning Innovation, Friedrich-Alexander University (Germany); and AKTO Art and Design College (Greece). All figures in this paper were developed by Eleni Tsampra.

About the authors

Julie-Ann Sime, Department of Educational Research, Lancaster University, Lancaster, United Kingdom.

Julie-Ann Sime’s research interests are in innovation in technology-enhanced learning in post-compulsory education. Julie-Ann is particularly interested in online learning in groups, teams and communities and on design for learning practices, educational design research methodology, and social inclusion of students with dyslexia (and other specific learning differences), ADHD and autism.

Email: [email protected]

ORCID: 0000-0002-3964-0684

Twitter: @julieannsime

Eleni Tsampra, Department of Graphic Design and Visual Communication University of West Attica, Athens, Greece.

Eleni Tsampra is a designer whose research and scholarly interests range widely from narratology and semiotics to pop culture and information design.

Email: [email protected] or [email protected]

Behance: @elenitsampra

Vimeo: @elenitsampra

Instagram: @eleni_tsampra

Article Information

Article type: Full paper, double-blind peer review.

Publication history: Received: 01 December 2020. Revised: 29 May 2022. Accepted: 01 June 2022. Published: 11 July 2022.

Cover image: Eleni Tsampra

References

Banister, F. & Ryan, C. (2001). Developing science concepts through story-telling. School Science Review, 83(302) 75-83.

BC ELTons (2021, November 16). British Council announces ELTons 2021 winners. https://www.britishcouncil.org/about/press/british-council-announces-eltons-2021-winners

BDA. (2019). British Dyslexia Association. https://www.bdadyslexia.org.uk/dyslexic

BDA. (2022). Am I dyslexic?: Signs of dyslexia (adult). https://www.bdadyslexia.org.uk/advice/adults/am-i-dyslexic/signs-of-dyslexia

Biancarelli, F., Grenson, O., Raffaele, S., Speltens, O., Teng P., You, & Miel, R. (2014). Partners. https://ec.europa.eu/regional_policy/en/information/publications/youth/2014/comic-partners

Brand EU (2018). DESANTIX launches Weekly Web Animated Comic Strips of Captain Euro®. http://www.brandeu.eu/2018/10/desantix-launches-weekly-web-animated-comic-strips-of-captain-euro/

Brown, B. (2011). Cinematography: Theory and Practice (2nd ed.) Focal Press. https://www.oreilly.com/library/view/cinematography-theory-and/9780240812090/10_Chapter5.html

Brown, K.R. & Broido, E. (2015). Engaging students with disabilities. In S. Harper & S. Quaye (Eds.). Student engagement in higher education: Theoretical perspectives and practical approaches for diverse populations. Routledge.

CAST (2018). Universal design for learning guidelines, version 2.2. http: //udlguidelines.cast.org

Cimermanová, I. (2015). Using comics with novice EFL readers to develop reading literacy. Procedia - Social and Behavioral Sciences, 174, 2452–2459. https://doi.org/10.1016/j.sbspro.2015.01.916

Cohn, N. (2013). The visual language of comics: Introduction to the structure and cognition of sequential images. Bloomsbury.

Cohn, N. (2019). Visual narratives and the mind: Comprehension, cognition, and learning. In K. D. Federmeier & D. M. Beck (Eds.), The psychology of learning and motivation: Knowledge and vision (vol. 70, pp. 97-127). https://doi.org/10.1016/bs.plm.2019.02.002

Cohn, N. (2020). Who understands comics? Questioning the universality of visual language comprehension. Bloomsbury.

Cole, D. (2014, March 13). A message from your brain: I’m not good at remembering what I hear. National Geographic. https://news.nationalgeographic.com/news/2014/03/140312-auditory-memory-visual- learning-brain-research-science/

Comics Uniting Nations (2015). Comics. http://www.comicsunitingnations.org/comics/

COMMIX (2017). COMMIX methodology or how to apply comics in education: Or how to make fun when teaching and learning. http://www.commix-project.eu/resources-pdf/COMMIX-methodology.pdf

Eco, U. (2001). Experiences in translation. University of Toronto Press.

Eco, U. (2003). Mouse or Rat?: Translation as Negotiation. Weidenfeld & Nicolson.

Eisenberg, H. (2014, September 15). Humans process visual data better. Thermopylae Sciences + Technology. http://www.t-sciences.com/news/humans-process-visual-data-better

EssayInfo (2018). Narrative essay. EssayInfo writing guides. https://essayinfo.com/essays/narrative-essay/

Florian, L. & Spratt, J. (2013). Enacting inclusion: A framework for interrogating inclusive practice. European Journal of Special Needs Education, 28(2), 119-135. https://doi.org/10.1080/08856257.2013.778111

Freytag, G. (1896). Freytag’s Technique of the drama, an exposition of dramatic composition and art. S.C. Griggs & Company. https://www.ebooksread.com/authors-eng/gustav-freytag/freytags-technique-of-the-drama-an-exposition-of-dramatic-composition-and-art-yer.shtml

Griffin, E., & Pollak, D. (2009). Student experiences of neurodiversity in higher education: Insights from the BRAINHE project. Dyslexia, 15, 23-41. http://doi.org/10.1002/dys.383

Hackford, T. (2004). Ray. IMDb. https://www.imdb.com/title/tt0350258/

House, J. (2018). Translation: The basics. Routledge.

Jakobson, R. (1959). On linguistic aspects of translation. Harvard University Press.

Joannidou, S. & Sime, J.A. (2021). Comics for inclusive English language learning: The CIELL app, supporting dyslexic English language learners. In N. Zoghlami, C. Brudermann, C. Sarré, M. Grosbois, L. Bradley, & S. Thouësny (Eds), CALL and Professionalisation - Short papers from EUROCALL 2021 (pp. 161-166). https://doi.org/10.14705/rpnet.2021.54.1326

Jones, E. (2010). The use of comic books style reading material in an EFL extensive reading program: A look at the changes in attitude and motivation to read in English in a Japanese university. Language Education in Asia, 1(1).

Källvant, L. (2015). Reading comics in the language classroom: A literature review. Göteborgs Universitet. https://gupea.ub.gu.se/bitstream/handle/2077/41223/gupea_2077_41223_1.pdf

Kormos, J. (2017). The second language learning processes of students with specific learning difficulties. Routledge.

Kormos, J. (2020). Specific Learning Difficulties in ELT: Part of the Cambridge Papers in ELT Series. Cambridge University Press. https://www.cambridge.org/gb/files/2415/7856/0079/CambridgePapersInELT_SpLDs_2020_ONLINE.pdf

Kormos, J. & Csizér, K. (2010). A comparison of the foreign language learning motivation of Hungarian dyslexic and non-dyslexic students. International Journal of Applied Linguistics, 20, 232–250. https://doi.org/10.1111/j.1473-4192.2009.00247.x

Kormos, J. & Kontra, E. H. (2008). Hungarian teachers’ perceptions of dyslexic language learners. In J. Kormos & E. H. Kontra (Eds.) Language Learners with Special Needs (pp. 189-213). http://dx.doi.org/10.21832/9781847690913-010

Kormos, J., Košak-Babuder, M. & Pižorn, K. (2019). The role of low-level first language skills in second language reading, reading-while-listening and listening performance: A study of young dyslexic and non-dyslexic language learners. Applied Linguistics, 40(5), 834–858. https://doi.org/10.1093/applin/amy028

Kormos, J. & Smith, A. M. (2012). Teaching languages to students with Specific Learning Differences. Multilingual Matters.

Kourdis, E. (2015). Semiotics of Translation: An Interdisciplinary Approach to Translation. In P. P. Trifonas (Ed.), International Handbook of Semiotics (303-320). Springer. https://doi.org/10.1007/978-94-017-9404-6_13

Krusemark, R. (2017). Comic books in the American college classroom: A study of student critical thinking, Journal of Graphic Novels and Comics, 8(1), 59-78, https://doi.org/10.1080/21504857.2016.1233895

Lakoff, G. & Johnson, M. (2003). Metaphors we live by (2nd ed.). The University of Chicago Press.

Lefèvre, P. (2000). Narration in Comics. Image [&] Narrative. Online Magazine of the Visual Narrative, 1. http://www.imageandnarrative.be/inarchive/narratology/pascallefevre.htm

MacCullagh, L., Bosanquet, A. & Badcock, N. A. (2017). University students with dyslexia: A qualitative exploratory study of learning practices, challenges and strategies. Dyslexia, 23, 3-23. http://doi.org/10.1002/dys.1544

Maunsell, M. (2022). Dyslexia in a Global Context: A Cross-Linguistic, Cross-Cultural Perspective. Latin American Journal of Content & Language Integrated Learning, 13(1), 92-113. https://doi.org/10.5294/laclil.2020.13.1.6

Millogo, V., Gaufreteau, A., & Millogo, D. C. (2021). Narrative-based interventions in first grade student with language difficulties: An exploratory study. Academia Letters, Article 4398. http://www.doi.org/10.20935/al4398

Mills, K. A. & Unsworth, L. (2017). Multimodal literacy. Oxford Research Encyclopedia of Education. https://doi.org/10.1093/acrefore/9780190264093.013.232

Pino, M. & Mortari, L. (2014). The inclusion of students with dyslexia in higher education: A systematic review using narrative synthesis. Dyslexia, 20, 346-369.

Plan (2016). Counting the invisible: Girls’ rights and realities. Plan International. https://reliefweb.int/sites/reliefweb.int/files/resources/countingtheinvisble_zimbabwe.pdf

Pollak, D. E. (2005). Dyslexia, the self and higher education: Learning life histories of students identified as dyslexic. Trentham.

Popa, N. L., & Tarabuzan, E. O. (2015). Using comic strips in teaching and learning French as foreign language: Changes in motivational beliefs. Review of Artistic Education, 9/10, 273–278.

Rand, P. (1965). Design and the play instinct. Education of Vision. https://www.csus.edu/indiv/e/estiokom/design_play_instinct.pdf

Richardson, G. (2021). Dyslexia in higher education. Educational Research and Reviews, 16(4), 125-135. https://doi.org/10.5897/ERR2021.4128

Schwarz, R., & Terrill, L. (2000). ESL Instruction and adults with learning disabilities. Center for Adult English Language Acquisition. http://www.cal.org/caela/esl_resources/digests/LD2.html

Shaywitz, S. & Shaywitz, J. (2020). Overcoming dyslexia: The essential program for reading problems at any level (2nd ed.). Sheldon Press.

Shiang, R. F. (2018). Embodied EFL reading activity: Let’s produce comics. Reading in a Foreign Language, 30(1), 108-129.

Sime, J.A. & Themelis, C. (2021). Comics for inclusive, technology-enhanced language learning Compendium: A guide for teachers on using the CIELL App. CIELL Learning Resources. https://ciell.eu/wp-content/uploads/2021/04/CIELL-Compendium-v3.pdf

Stentiford, L. & Koutsouris, G. (2020). What are inclusive pedagogies in higher education? A systematic scoping review. Studies in Higher Education, 46(11) 2245-2261. https://doi.org/10.1080/03075079.2020.1716322

Themelis, C, & Sime, J.A. (2020). Comics for inclusive technology-enhanced language learning. In S. Mavridi & V. Saumell (Eds.), Digital Innovations and Research into Language Learning (pp. 93-114). IATEFL.

Thompson, K., & Bordwell, D. (2008). Film History: An Introduction (8th ed.), McGraw-Hill Higher Education.

UN SDG (2022). Do you know all 17 SDGs? United Nations. https://sdgs.un.org/goals

Versaci, R. (2008). "Literary literacy" and the role of the comic book: Or, "You teach a class on what?". In N. Frey, & D. Fisher (Eds.), Teaching visual literacy: Using comics, graphic novels, anime, cartoons, and more to develop comprehension and thinking skills (pp. 91-111). Corwin.

Welby, V. (1903). What is meaning? Studies in the development of significance. Macmillan. https://archive.org/details/whatismeaningst01welbgoog/page/n72/mode/2up

Yuan, Y., Major-Girardin, J., & Brown, S. (2018). Storytelling is intrinsically mentalistic: A functional magnetic resonance imaging study of narrative production across modalities. Journal of Cognitive Neuroscience, 30(9), 1298-1314. https://doi.org/10.1162/jocn_a_01294

Zimmerman, B. (2010). Using digital comics for language learning. ELearn, 2010(1). https://dl.acm.org/citation.cfm?id=1710033