The weekly column

Article 48, February 2001

Visuals & Language Learning: Is There A Connection?

By Christine Canning-Wilson

Center for Excellence in Applied Research and Training (CERT College),
Higher Colleges of Technology - Abu Dhabi

Abstract:

This paper will examine the current trends of using visual images in the foreign and second language classroom with EFL/ESL learners. It will review the current data base of literature, look at more modern theories & effects of visuals on learning as well as suggest practical applications for the classroom. Moreover, the paper will also examine how a study iniative at United Arab Emirates University revealed that visuals can help exploit and further the use of language by non-native speakers of English.

1.0 Literature Review

A picture may be worth a 1,000 words, but it can also lead to speechlessness. Over the centuries theorists have often wondered whether or not we can see before we think. Pablo Picasso was famous for stating "I paint what I think not what I see." Therefore, it only seems logical as language educators we ask the same questions to determine the effect of visuals on learning.

Let’s start with a general framework of how we learn and see our world. If you were to look at a glass of water, many would argue it was half-full whilst others would debate it was half-empty. Perception affects learning. How a learner views an object, text or symbol can affect written and oral communication. Let’s take another example, if you were asked to count the windows in your home, would you be more apt to take a mental picture of the room before counting? A wide variety of interesting questions can be asked in connection with the topic of imagery in learning. Thus far, research in human learning has provided answers to only a few. Still even fewer answers are available from research directed specifically towards the learning process. Perhaps, this is because humans have an ability to form a "mental" picture of the room before making a judgement. Even those blind from birth can facilitate learning and recall using imagery instructions. Thus, the mental image of what we have seen may at times make a greater impact than auditory input. Therefore, visual cues are important, since they either facilitate or distract from understanding.

In the area of foreign language and second language research empirical evidence showing relationships between the picture and acquiring the target language is scant. However, in the areas of cognitive psychology and visual/verbal behavior the literature is saturated with empirical data. Therefore, as researchers and classroom teachers, it is essential that we investigate how visuals and images can aid in the area of second language learning.

Canning (2000) states that there is considerable confidence placed in the value of audio-visual aids to enhance the learning of foreign languages, yet there is little empirical data and research to support the proposition that video or visuals facilitate in the learning of foreign languages. However, with the amount of time devoted to using video/visuals in the F/SL classroom, research is warranted to show how audio-visual aids enhance the language learning process. Currently, many constraints on the studies completed over the past two-decades, show that there are several limitations to be recognized when examining results. For example, many studies have been done with visual aids and not with actual English language videos. Secondly, the use of intact groups instead of random groups (who were studying only one foreign language) have been used in most visual studies in relationship to language learning.

Omaggio (1979) believes that researchers in order to find valid results would need to ask whether or not the same findings would hold true in different languages. With such unanswered questions, hypotheses and theories, yet to be proven with quantitative measures, practitioners must ask themselves is there sufficient evidence to support continued use of visual aids in the learning of foreign languages to a justify the allocation of resources for them in the F/SL classroom?

MacLeod (1980) believes there is a relationship between the process of comprehending a linguistic statement corresponding to a visual scene. Over the years this ideas has intrigued psychologists because it seems to be an important element in many kinds of mental activity of two forms of stimuli converting to internal representations. Macleod’s studies have involved participants verifying or rejecting simple linguistic statements with an equivalent description of a simple picture. His findings show that it takes longer to verify complex statements depending on its linguistic complexity.

Likewise, Cronin and Myers (1997) study shows that there was no significant difference between two treatment groups between the cognitive test scores and listening test gain scores for student using multimedia IMI with visuals versus multimedia IMI with no visuals. Rather their research implied that information presented visually and verbally is represented differently in memory. Instead, it suggests that connections be formed within the re-visual and verbal representations and referential connections.

According to more current research, the more sensory modes in which mental representation is stored, the more likely they will be remembered (Borsook. Higginbotham &Wheat, 1992) Bagget (1989) posits that images are stored in memory. These images contain more information because they have more cognitive pegs that can be used to make associative and referential connections between visual representations and information held in long term memory. This research suggests learners employ dual coding to construct a mental model of the learning experience. Therefore, it can be implied that visuals may evoke relevant knowledge for those that have it to draw on. It is then assumed that most learners will integrate available symbol systems (visual, audio and/or text) to construct or elaborate on a model of the situation as a strategy to recreate in their mind a picture image of an event.

Morra (1989) shows that children tend to use the strategy based on phonemic coding; whereas, adults used a "mental mode" strategy. A mental mode is an analog representation of the structure. Whether perceived or conceived the model(s) represent the item being described with descriptive language. (Example the knife is in front of the glass - now draw the depiction from what you hear) Images can be presumed to be symbolic and do not need to be in the form of quasi-pictorial images.

Kosslyn (1981) revolutionary article posits the computational theory of images, which states that visual mental images are transitory data structures that occur in an analogue of spatial medium. The theory is described in terms of detailed claims about the mental structures and processes invoked during imagery. When a stimulus is terminated suddenly, some of the information emanating from the stimulus continues to be available in the nervous system for about a second after termination.

It is also fair to state that research shows that people can construct a mental representation of the semantic meaning of a story from either audio or visual information alone. However, it appears that when presented together each source provides additional, complementary information that retains some of the characteristics of the symbol system of origin. For example, children recall sounds and expressive language from the audio track and visual details from the visual track. It also appears that the bushier nature of representations derived from the visual symbol systems are better for building mental models of the situation than are representations based on audio linguistic information (Kozma 1991).

As Rowe (1994) states Multimedia data can require significant examination time to find desired features such as content analysis. Researchers must examine what can be learned indirectly about the things depicted in a pictorial multimedia, visual images, signs and symbols. Many researchers are taking a critical look at the use of mnemonic associations. By focusing on verbal and imagery mnemonics, whereby a word, a phrase and/or a sentence serves as a form of visual imagery and as a mediator between what is known and what is to be learned. Cohen's (1987) research includes verbal and imagery encoding mnemonics, whereby verbal material and visual imagery serve as cognitive mediators. His work indicates that in the second stage of mnemonics involves what is known as keyboard mnemonics, where images are associated with words in a foreign language. This imagery link is an intended result is that an encounter with the foreign word will evoke the keyword, which in turn reveals the imagery link and finally the native language equivalent can then be retrieved from this interaction by forming a mental image linking the ideas. Cohen further suggests that the learner have to link the native-language word to the imagery and/or verbal material containing the key word. Research by Pressley, Levin and Delany (1982) show in their research that data indicates that there is a slight advantage in favor of imagery mnemonics over verbal mnemonics.

The use of representational visuals to supplement the orthographic symbols of text generally increases recall of illustrated text information. (Issing et all 1989; Kuntz eta l 1989; Livie et all 1982 Levin 1987; Peeck 1987; Pressley 1987; Stone 1981) Ehrilich and Johnson-Laird (1982) showed that adults interpret spatial descriptions by constructing mental models of them. An alternative strategy may be to encode descriptions verbatim and then try to represent them overtly, one sentence at a time.

Yet, how does the visual symbol integrate itself into the skill areas? Let’s start with the basic skill of reading. Letters are symbols. They stand for a sound and when put together with other "letters/symbols",they form meaning through recognizable words. Words become visual symbols and clues, which can aid in the written communication. Thus, reading is usually defined as extraction of meaning from text (Gibson and Levin 1975).

As Haber and Hershon (1980) claim: "vision is certainly a central one in the understanding of the reading process". More specifically, it is the process by which written or printed symbols are translated into a representation in which meaning is already accessible - a translation to a form of language from which a reader is able to derive meaning. This definition makes clear that the reading process is intimately tied to other language processing, especially the ability to extract meaning from speech. However, reading is more than printed speech. The visual components are so different from the auditory ones that reading and listening, while sharing the same language, make very different demands on the information such as the processing skills of the perceiver (Clark and Clark 1977).

Studies (Polyak 1980;1982; Smith 1983; Barlow 2000) suggest that reading requires the active search of a text. Meaning cannot be passively obtained. Eye movements are therefore an integral component of the reading process Sless 1983). As one of the researchers explains, 100 words of difficult text may have over 900 character spaces whereas 100 words of easy text might have only 600 characters. Reading speed is 1104 character spaces per minute and 320 syllables per minute as well as 200 words per minute. For college students, reading speeds can be a low as 50 words per minute for very difficult technical material, and as high as 500 to 1000 words per minute for easy novels being read for relaxation. Good readers do not move their eyes more rapidly than poor reader does do nor is movement speed different for easy as compared to difficult text.

Reduncdancy in the visual, such as print, increases the size of the effective field of view. For example, Johnson et al (1989) shows that when subjects are asked to scan a word in order to detect a pre-designated target letter, the strategy they adopt is quite reminiscent of a memory scan, rather than a visual scan, in that they appear to engage in an exhaustive search. The data were interpreted as literally inferring that:

1. Words are processed holistically
2. Such processing is not under the strategic control of the perceiver
3. Component -level letter information can obtain only by deriving it within memory from the word-level representation.
4. Finally it was suggested that usual difference in scan rate obtained between words and non-words may not reflect any direct influence of that factor on the scan rate itself, but rather it may be derivative effect that stems from a difference in the type of scan.

Pezdek, Simon, Soteckert and Kiely (1987) two experiments between reading comprehension and visual comprehension of similar materials shown through a television media showed that participants who watched the television had a higher visual and spatial ability as assessed by psychometric test performance and secondly, were more likely to utilize an imagery based strategy in the sentence picture verification task.

2.0 Fleming System of Visual Classification

The Fleming System dictates that all people systematically go through the same sequence of events in order to comprehend a visual image. First, a given learner recalls, recognizes, acknowledges and reproduces a mental picture of the visual input. Second, the learner goes through an analytic period. It is at this point that the learner attempts to separate, identify, and compare the components of the illustration. This includes assessing, judging, describing and/or identifying components of the desired image. The next stage finds the learner trying to combine components, formulate new relationships and generalize information based on the input. In the application stage, the learner attempts to "apply" the visual to a situation. Finally researchers hypothesize that the learner demonstrates a desired interest in the visual.

3.0 Defining A Visual

The concept of defining what constitutes a visual is by nature complex. A visual is any projected or non-projected image that can be classified into illustrations, visuals, pictures, perceptions, mental images, figures, impressions, likeness, replicas, reproductions or anything that would help a learner see an immediate meaning (Canning 1997; 2000). The visual is considered projected when it is planned for and executed with an intended meaning. In contrast, the non-projected visual is the result of an spontaneous occurance of an image that is usually unplanned and occurs in relationship as a result of a triggered catalyst.

4.0 Visual and Its Physical Relationship to the Body

It is a biological fact that the eye is not separate from the brain. The eye and brain are part of the same organ. Because human beings have evolved to have several distinct intelligence and no one general intelligence, interpretation of visuals and the use of visuals may affect learners in different ways. Studies such as those in the early 1970s at the University of California at Berkley show that the different hemispheres of the brain may affect human learning. Physical perception of what is scene may differ amongst different learners thus allowing a greater range of responses to any given visual. For example if "Learner X" is given a picture the student may be able to interpret how they see the picture and relate it to language in two distict ways. First they may use top down visuals to test ideas against facts or solve specific problems. Secondly the learner may use bottom up visuals to scan and organize information with the use of graphics (Canning 1997).

5.0 Visual Preference

Learners prefer visuals that are colored, contain a story, relate to previous experiences and that can be associated with places, objects, persons, events or animals of which they are familiar. Likewise, it has also shown that if a visual stimulus is suddenly terminated that it remains available in the conscious system only for about a second in detail. Moreover, learners prefer time when comprehending a visual. For example, when a given visual is flashed with time for comprehension and/or is masked with another visual that recognition for the learner can be impaired. When selecting a picture to enhance language learning, a visual should be chosen to illustrate the point. The illustrations should make a statement and be comparative. It is highly recommended that the picture be interpretive and to the point. For example, pictures used on the Acer & Peacock Test, SPEAK Test, Borgotta Test of Visual Response and most listening/aural tests show reasonable judgement in content and often enhance learning, sensory acuteness and the testing situation as a whole.

Canning-Wilson (2000) recently stated in a presentation on research and visual learning that visuals are ineffective when used incorrectly. She suggested that improper use of visuals in a lesson or text, include but are not limited to:

1. the use of violent scenes
2. too many distractors
3. too crowded or cause an overwhelming effect of information
4. too small or not clearly defined
5. stereotyped images
6. poor reproduction
7. not related to text
8. irrelevant captioning
9. offers too much information
10. unclear picture which doesn’t compliment the text
11. poorly scaled illustration
12. cluttered composition that is not esthetically meaningful

Careful selection of pictures to elicit answers from students should be chosen. Often pictures are chosen because they are meaningful to the teacher or tester. Pictures should have a universal appeal and be relevant to the learner. A learner should have be familiar and able to speak on the basics of the picture in his first language. For example, giving an Arab student a picture of a polar bear in an ice field (during a testing situation) would be an unfair practice. The student may be able to speak about the basic diet and habitat, but would not be able to fully expand on the topic without previously learning background related materials.

6.0 Advantages of Visuals on Learning

To use visuals in the context of language learning. One must subscribe to the following assumptions:

1. Imagery can facilitate learning.
2. Imagery (combined with the use of texts) make subjects more likely to think about the process of the language more fully.
3. Imagery may hinder language learning. For example, imagery has been known to interfere with performance in various concurrent visual tasks.

A learner’s sensitivity to language and their ability to create relations amongst words can be further enhanced by the use of visuals. Graphic images can bring out more detailed, knowledgeable, responsive, awareness to the object, situation or text being communicated. Moreover, the use of a picture may lead a learner to more abstract thought as well as the ability to distinguish ideas or to demonstrate greater precision counting organizing skills through the use of logical structure. Overall, an bring out a more complex sensitivity in the learner (Canning, 1998).

Pictures help individual learners predict information, infer information, deduce information, analyze today’s world so that it can be brought into today’s classroom and offer social settings which can immerse or expose the learner to new ideas or further promote an already created setting. If a visual is used in a testing or teaching situation it can enhance clarity and give meaning to the text or to the message being communicated. Visuals can serve to create a solid link between the material learned and the practical application of it on a test (Canning, 1998).

The nature of a graphic images serves as a catalyst and stimulus. Moreover, the visual can offer input, output and/or feedback on materials learned. Likewise, a learner using a visual may be more apt to give an added response to the traditionally expected form. The response may change according to the picture in collaboration with the task as it works as an interactive negotiator. Although some practitioners may argue that a picture can serve as an impetus, others would equally argue that the same picture has the potential to urge an opinion (Canning 1998). As teachers manipulate texts daily in lessons, teachers can also control and open a picture to a variety of language learning activities as well as a variety of interpretations.

Visuals allow for greater cognitive mapping and navigating in an environment. The use of visuals can either lead to sensory acuteness or to sensory depravation. In the process, perception becomes the ability to process the stimuli as meaningful to the viewer.

Visuals are a good and useful tool for examination purposes because they lead the learner into drawing out language from their own knowlede and personal experiences through exposeure to, immersion to the stimuli presented before them. Visuals permit strategies to orgazine knowledge into semantic or associative clusters. In testing and teaching situations pictures items can can be developed to test whether the students understands the syntax or structure of the target language. Visuals allow for options, responses alternatives patterns and ranges. Students can see immediate meaning in terms of vocuabulary recognition provided the item exists in the first language. Pictures can be develped into a test to see whether the learner understands the structure and the syntax (Canning 1998; 2000).

The advantages of pictures suggest that visuals can help make a task or situation more authentic. Pictures can help testers and teachers to identify or manipulate structures, vocabulary, functions, situations and skills. Visuals used as testing prompts can be used to measure semantic and associative clusters. Pictures allow students to focus on the whole item or a piece of an item. Pictures can give or not give a context depending on what needs to be tested. Pictures offer test takers options for more interpretive responses, patterns and ranges to answer posited exam questions.

Visual testing and teaching prompts can aid in measuring syntactical, phonological, lexical and cultural proficiency. Perhaps this is because a visual cue may also be accompanied by a written cue to focus a students’ attention on the specific lexical unit being furnished. Pictures can force full or partial student production. For example, Visuals may possibly be able to empower test takers to decode information using top-down or bottom-up processes. Top down encoding in the visual process would include the learner testing ideas against facts or solving specific problems by viewing concept maps to relate their ideas to facts or ideas found in a second language text or test. In contrast, bottom up encoding in the visual process would include using the visual image to help learners to scan, sort and/or organize information in a testing environment.

The use of visual stimuli enables a learner and/or test-taker the ability to predict information, to infer information as well as to deduce information from the picture or mental image. Another beneficial reason for using pictures on examinations is authenticity. Visuals can bring today’s world into the language learning environment by serving as an reflection of what exists outside of the foreign or second language classroom. This form of stimuli can allow the learner to analyze his/her own world. Visual stimuli can expose as well as immerse the test-taker into a familiar or unfamiliar environment, social setting or practical environment.

7.0 Initial Findings of Study

A preliminary study was undertaken in the Spring of 1997 at the University General Requirements Unit at United Arab Emirates University. 145 Female students on the Maqam Campus were asked to write an essay about a technical item called a "Blender". Group A was given a visual graphic of a blender in the upper right hand corner and the directions which read: Please write a paragraph about the function of a blender. Group B was given only the direction without any visual reference. All classes that semester were exposed to kitchen items in the classification unit and description unit of the curriculum. Spelling, grammar and mechanics had no influence on the results of the study. The study was interested in whether or not the visual aided in the writing prompt.

7.1 Results

The preliminary data of the study revealed that 93% of the candidates who were given the prompt with a picture were able to describe the blender. Only 52% of the participants who did not have a visual stimulus were able to write about the basic descriptions of the blender. (See figure 1a)

Figure 1a.

The answers of Group A were more indepth than Group B, who did not have a visual supplement to the writing prompt. As a result of the picture, the essays from Group A were able to describe individual parts and the whole body of the blender with more detail, accuracy and depth than their counterparts. (See figure 1b)

Figure 1b.

It is evident that the use of a visual aid increased the learners’ chances for improving and developing their language abilities. The picture allowed the student to interpret the use and physical appearance in great detail that might not otherwise be obtained with just a textual prompt. Therefore, in this instance, it can be concluded that visuals positively affected that learning strategies and writing capabilities of nonnative speakers of English.

8.0 Future Direction of Visual Study

It is recommended that more studies be completed in the Arab region as well as in the rest of the world to see how language is affected by visual input. More emphasis should be put on the possibility that visual images affect how learners learn and how teachers teach. The recipicol relationship between visuals and language learning needs to take further direction in order to offer teachers the best methods for exploiting visual realia in the classroom. These studies may take form, but are not limited to the relationships found in the areas of testing, linguistics, ESOL, ESP and the ESL/EFL realm of language learning.

References

• Barlow, L. and Canning, C., (1998). "Test writing for distant learning students", In C.Coombe (ed.)., Current Trends in English Language Testing, TESOL Arabia CTELT Conference Proceedings, 1:1, from the 1^st Annual Current Trends in English Language Teaching Conference, hosted by United Arab Emirates University, May 15, 1997.
• Canning, C. (1999) ‘Ideas for Transforming Your Classroom Into a More Visually Aesthetic Learning Center’, Visual Support in English Language Teaching (VSELT) Newsletter, 2:2, p. 1-2
• Canning, C. (1998) ‘Visual Perception and Language Learning’, TESOL Arabia News, 6:1, November 1998, pp. 8-9
• Canning, C. (1998) ‘The Role of Pictures in Academic Testing’, TESOL Arabia News, 5:5, June 1998. pp. 19-21
• Canning, C., "Implementing Visual Support in F/SL Materials", Visual Support in English Language Teaching (VSELT), 1:1, March 1998 p.1-2
• Canning, C., "Visual support and language teaching", TESOL Arabia News, Volume 5:4, March 1998.pp.8
• Canning, C., ‘Perception and Visual Reading: Do You See What I See?’, TESOL Arabia News, 6: 1, November, 1998 pp 11-12
• Canning, C. "Theoretical use of Visuals". In Canning, C. and Koester, J., Illustrated Visual Aids for Academic English, 1:1,MGVA Publication, UGRU Unit In-house Publication, Fall 1997. Pp.2-4
• Canning, C (2000) Visual Research, Invited presentation for the TESOL Arabia Special Interest Group, Hilton Ballroom, April 14, 2000 at the TESOL Arabia 2000 Conference "Bridging the Gap Between Learner and Teacher", April12-14, 2000
• Canning, C., Barlow, L. and C. Kawar (1998). "Computer Generated Visuals and Reading’. In C. Coombe (ed.) Current Trends in English Language Testing, v.2, October 1998, pp. 157-164 TESOL Arabia Conference Proceedings held at Al Ain University, Al Ain, United Arab Emirates, May 6, 1998.2:1, October 1998 pp.157-164
• Clark and Clark (1977) Psychology and Language : An Introduction to Psycholinguistics Hardcourt and Brace NY
• Coombe, C., Kinney, J. and C. Canning (1998). ‘Issues in the Evaluation of Academic Listening Tests’, in Caroline Chapman and Dianne Wall (eds.) Language Testing Update, v.24, Autumn 1998. Pp.32-45
• Cronin, M and Myers S (1997) The effects of visuals versus no visuals on learning outcomes from Interactive multimedia instruction, Journal of Computing in Higher Education Spring 1997, Vol. 8 (2), 46-71
• Ehrilich and Johnson-Laird (1982) Journal of Verbal Learning and verbal Behavior, 21, 296-306
• Gibson and Levin (1975) the Psychology of Reading MIT Press, Cambridge MA
• Haber, R and Hershenson, M. The Psychology of Visual Perception, Holt Reinheart and Winston publishing NY 1980
• Johnson, N, Pugh, K, and Blum. A, (1989) Journal of Memory of Language, 28, 155-163
• Morra, S (1989) Developmental Differences in the Use of Verbatim versus Spatial Representations in the Recall of Spatial Descriptions: A probabilistic Model and an Experimental Analysis, Journal of Memory and Language, 28, 37-55 (1989)
• McLeod, C (1980) Strategy Choice and Strategy Training in Sentence - Picture Verification, Journal of Verbal Learning and Verbal Behavior, 19, 531-548
• Omaggio, A "Pictures and Second Language Comprehension: Do they Help?" Foreign Language Annals 12 (1979): 107-16.
• Pezdek, k, Simon. S Stoeckert and J. Kiely, (2987)Indivdual differences in television comprehension, Memory and Cognition, 15 (5), 428-435
• Pressley, M, Levin, J.l & Delany, H (1982) The mnemonic keyboard method. Review of Educational Research, 52, 61-91
• Rowe, N (1994) Inferring Depictions in Natural-language Captions for Efficient Access to Picture Data, Information Processing and Management, Vol. 30, No. 2, p.p. 380-388

About the author

Christine Canning-Wilson is on the English Faculty at the Center of Excellence in Applied Research and Technology. Currently, she is serving as the Curriculum Coordinator for CERT College at the Higher College of Technology. She holds a Master’s Degree from West Virginia University, a Royal Society of Arts Diploma (DTEFLA) from Cambridge University and a Bachelor’s Degree in English from the University of Massachusetts. She holds 4 separate teaching licenses in the Commonwealth of Massachusetts. She has been an invited speaker around the world at many prestigious events and publishes in the field of Foreign and Second Language Research on a regular basis. She is the past supervisor in the UGRU English Department of UAE University and Chairperson of the Media Graphics Committee. She is the past Chairperson of the TESOL Arabia 2000 Conference and Executive Council. She is still currently the Chairperson of the TESOL Arabia Video Special Interest Group. She recently was the recipient of the Quality Teaching and Learning Grant for her upcoming book on teacher training ideas & resources for the Higher Colleges of Technology in the United Arab Emirates. Christine can be contacted at Christine.canning@hct.ac.ae .

Questions or comments about this week's article? Why not post them on our Discussion Forum