This chapter was contributed by Dr. Rebecca E. Burnett, former Director of Writing and Communication at Georgia Tech, and Dr. L. Andrew Cooper, former Assistant Director of Writing and Communication at Georgia Tech.
- What Is Communication?
- What Is Multimodal Synergy?
- What Are Examples of Multimodality?
- Why Focus So Much on Modes Other than Writing?
- How Does Multimodal Synergy Relate to Expectations for Students?
- How Does Multimodal Synergy Enhance Intellectual Development?
- Why Are Self-Reflection & Self-Assessment about Communication Processes & Artifacts Important?
Let’s get directly to the point: Can you be successful in academic, professional, personal, and community contexts without multimodal competence? No. Few questions about communication have black-and-white, either/or answers, but this one comes close. Academic courses, internships, and jobs are as dependent on your ability to give oral presentations, to work collaboratively, and to design effective visuals as on your ability to write reports. Personal, professional, and community relationships are as dependent on respectful nonverbal communication as on coherent conversation.
To be a successful multimodal communicator, you need to be able to understand the features and power of each mode separately and also be able to create, analyze, and interpret them synergistically. You need to be actively engaged as a communicator, not only working to master multimodal strategies that enable you to be effective but also understanding that in practice these strategies aren’t separable; they’re synergistic.
Consider the modes of WOVEN communication. Which are your strengths?
Which do you need to improve? Which do you prefer? Which do you avoid?
#Remember
#Understand
Synergy refers to objects or actions that work together. Simply put, synergy is what results when things interact and explains why the power of the whole is greater than the sum of its separate parts. You should think of effective communication as being synergistic—involving the interaction of written, oral, visual, and nonverbal modes. For example, written communication involves both a productive part (writing) and a receptive part (reading). Writing can be individual or collaborative and can be produced in a print or an electronic document. In the process of writing, writers talk both to themselves and to others, often sketch ideas in preparation for writing, use design elements to influence audience response (from headings and paragraphs to font choices and illustrations), and are affected by the environment in which they work. This short list of some of the synergistic aspects of writing shows that modes don’t exist in isolation. Your academic, professional, personal, and community success depends in large part on your competence with all the modes and your recognition that they function synergistically. As a capable communicator, your communication will be multimodal—that is, WOVEN.
As you advance academically at Georgia Tech and later assume professional roles, communication will become increasingly challenging. Professionals in all fields—ranging from architecture to medicine, engineering to management, environmental science to computer science, the military to public policy—are more successful if they are effective communicators who can express their arguments compellingly in multiple modes and media. Effective communicators have strong written, oral, and visual competencies. In addition, with rapidly expanding social networking technologies, multimodal competencies increasingly rely on electronic modes of communication—for example, cell phones, social media, the Internet, and digital video.
In this chapter, you will learn more about factors affecting the synergy of modes and media. The chapter defines communication and addresses issues related to rules and conventions affecting modes and media. To define communication, it considers two questions you might ask about a writing and communication class: Can’t I just learn the rules? and What do conventions have to do with rhetoric and multimodality? Once the definition of communication is established, the chapter then discusses multimodal synergy and describes a series of multimodal examples created by students at Georgia Tech. The chapter concludes with a discussion of four important questions:
- Why focus so much on modes other than writing?
- How does multimodal synergy relate to expectations for students?
- How does multimodal synergy enhance intellectual development?
- Why are self-reflection and self-assessment about communication processes and artifacts important?
What Is Communication?
If you ask 100 different people to define communication, you may get nearly 100 variations in their responses. This chapter establishes common ground for your work in English 1101 and English 1102, focusing on what communication is and isn’t.
Communication is an area of disciplinary study (in the sense of intellectual inquiry) as well as a dynamic, interpersonal practice (in the sense that communication enables us to interact with others, whether in classrooms, workplaces, homes, or communities). Some people think that this study and practice are about the transmission of information—a one-way process that involves a sender encoding information and a receiver decoding that information.
It’s not.

SIGN
NO to the Transmission Model of Communication
This sign is a reminder that the transmission model is not a sufficient explanation of the way communication works, whether in written, oral, visual, or nonverbal communication because, in part, context and interaction are omitted.
Courtesy of Georgia Tech’s Writing and Communication Program.
Communication is dynamic and synergistic, involving many factors that go far beyond simple transmission. Consider these basic features of communication:
- Communication is dynamic, involving an interactive process affected by context, intention, and interpretation.
- Communication involves rhetorical elements in interdependent (that is, synergistic) relationships; each rhetorical element (e.g., content, context, purpose, audience, argument, organization, evidence, medium, design, and various kinds of conventions) is affected by others.
- Communication involves writers, readers, speakers, listeners, designers, and viewers, all of whom interact in specific contexts and respond to specific situations.
- Communication has conventions for writing, speaking, designing, and nonverbal behaviors that are culturally determined and change with situation and time.
Communication isn’t just about the words, sounds, or images of a message; it’s about the entire communicative situation. For example, specific audiences have idiosyncratic intentions, abilities, expectations, contexts, and experiences, all of which affect interpretation.
What about just learning the rules?
Communication is not about learning a single, well-defined set of rules that tells how and how not to communicate in every situation. Learning mechanical rules by themselves, decontextualized and without nuance, doesn’t make you a good communicator. It just makes you a good rule follower. In fact, following rules while ignoring cultural differences in language and ignoring the rhetorical elements and their synergy usually results in inadequate, inappropriate, or oversimplified communication.
You can learn some rules, but you can’t learn the rules because a single set of rules governing all communication situations simply doesn’t exist. Academic and professional communication involves certain conventions; you’ve probably learned conventions as the rules governing grammar and mechanics. Those conventions constitute what we commonly refer to as Standard American English, which is used in both written and oral communication in most U.S. academic and professional contexts. Standard American English differs from other versions of written and spoken English in the United States. Standard American English also differs from English in Canada, England, Australia, Hong Kong, Singapore, South Africa, and India—to name just some of the places where English is a primary language. (See below for one small example of the way culture influences conventions.)
“Correctness” is cultural.
English language conventions in the United States and the United Kingdom differ. For example, people educated in the United States generally put commas and periods inside quotation marks. People educated in the United Kingdom generally put commas and periods inside only if the punctuation belongs to the quotation; otherwise, the punctuation is placed outside.
U.S. Example: Accessible means “easy to approach, reach, see, understand, or use.”
U.K. Example: Accessible means “easy to approach, reach, see, understand, or use”.
In the United States, you need to know the conventions of Standard American English because they help U.S. academic and professional audiences not only understand your ideas but also perceive you as someone who is capable of performing appropriately in U.S. academic and professional contexts. In other words, when you’re in the United States, following U.S. conventions contributes to your professional ethos, which gives you authority and credibility as a communicator. The authority and credibility associated with a strong professional ethos are powerful. You can use your versatility as a communicator to bring people’s attention to the issues that matter to you most; by demonstrating your mastery of language conventions, you can more easily persuade your audiences to understand—and adopt—your point of view. (See Chapter 8 for more on the conventions of Standard American English.)
Learning about common academic and professional conventions for communication can make you a competent and powerful communicator in many situations, but it won’t prepare you for all situations. For example, if you’re tweeting about your favorite TV show, you won’t write formal academic prose that follows all the conventions of Standard American English. The conventions that govern social spaces like Twitter differ from those that govern most academic and professional contexts. Good communicators know how to adapt to different contexts, recognizing and using the conventions that best enable them to share their ideas and build productive relationships. Although no single set of rules governs all communication situations, you can learn conventions associated with different contexts, cultures, and countries and use them strategically to represent yourself favorably in a variety of situations.
What do conventions have to do with rhetoric & multimodality?
Learning about rhetorical elements—such as context, purpose, and audience—helps you to analyze communication situations and choose the conventions and styles that fit them best. Understanding rhetorical principles is the key to adaptive strategies that define successful communicators. A strong rhetorical foundation allows you to adapt to different conventions within a mode. For example, understanding the needs of different audiences helps a scientist giving oral presentations to choose precise scientific terminology for an audience of professional experts or to choose a more general vocabulary for an audience of nonexperts.
A strong rhetorical foundation also allows you to adapt to the conventions of different modes and media. For example, an architect creating print plans for a new building might use shading and other conventions that suggest three-dimensionality in a static image. The same architect creating plans on a computer might use software that not only allows people to see a three-dimensional view by rotating a dynamic image but also enables viewers to step into a three-dimensional representation to see the building from the inside. The conventions of print and digital media allow for different representations of the same ideas. A competent communicator analyzes the context, purpose, and audience of the communication to determine which type of representation—and thus which modes and media—will be most effective.
With this background to help you think about modes and media in academic and professional communication, let’s return to the topic of multimodal synergy. To understand why multimodal synergy enhances communication, we first need to extend our definition of multimodality and then look at several examples.
What Is Multimodal Synergy?
According to researcher and theorist Gunther Kress, “[C]ommunication is always and inevitably multimodal” (“Gains and Losses” 5). Multimodality is so important, Kress explains, that “it is no longer possible to understand language and its uses without understanding the effect of all modes of communication that are co-present in any text” (“Multimodality” 337). Colleagues of Kress, including Carey Jewitt, explain that communication includes “all meaning-making systems” (Jewitt et al. 5).
Medium—technologies of dissemination, such as printed book, CD‑ROM, or computer application”
—C. Jewitt (184)
Mode—organized, regular means of representation and communication, such as still image, gesture, posture, and speech, music, writing, or new configurations of the elements of these”
—C. Jewitt (184)
We use the term multimodality to describe communication that occurs in different forms, or modes, and through different media. Understanding the differences is not always easy because the terms mode and medium are sometimes used in ways that overlap. For example, writing is a form in which you can represent your ideas, but it is also a means through which you can share your ideas with others. Writing is a mode that gives form to your ideas, while printed pages and Web pages are media through which you might choose to share ideas.
The overlap between modes and media is not simply a matter of fuzzy thinking or imprecise terminology. If a mode is a form and a medium is the means, the two quite naturally go hand in hand: writing as a mode (the combination of symbols, generally alphanumeric characters, and spaces to form units of meaning such as words and sentences) doesn’t exist until you put it into a medium (such as the paper or Web page on which the combined symbols appear). Similarly, a mode is rarely discrete. For the sake of analysis, we can distinguish written communication from visual communication by defining the former as the combination of alphanumeric symbols that make meaning for readers and the latter as the use of images that also make meaning by appealing to the eye.
This distinction between mode and medium allows us to think about a printed word as something with characteristics that differ from a primarily visual artifact, such as a photograph. Such thinking is useful because examining the ways in which printed words and photographs make meaning differently can help us to understand how to use words and photographs more effectively. In another sense, though, the distinction between a word and an image is artificial because a word is also an image. It makes meaning by appealing to the eye. The three examples of “word” that follow demonstrate ways in which a word is an image.

Courtesy of Georgia Tech’s Writing and Communication Program
The numerals and alphabetic representation of “word” are in the written mode; they are combinations of symbols that form units of meaning—the word “word”—but the differences in typographic style, size, and upper- and lowercase font emphasize various ways in which these words are also images. The first “word,” which appears in a common, professional-looking font (Times New Roman) and all capital letters, might convey the ordinariness and perhaps the authority of the word “word.” The second “word,” which appears in a script font intended to mimic handwriting (Mistral) in an italic style and in all lowercase letters, might convey the idea that a word can be informal and difficult to control. The third “word,” which appears in a fancier script font (Vivaldi) and in upper- and lowercase letters, might convey the idea that a word can be elegant and formal.
A word is both alphabetic and visual. Since one characteristic of a written word is that it corresponds to certain sounds, it can also be oral. A word can appear on a Web site, so it can also be electronic. Finally, the way a word looks might give you cues about gestures you could make while speaking it: for example, you might use a dramatic flourish of your hands as you speak a word written like version 1; you might bend over and whisper as you speak a word written like version 2; and you might put on airs and use a snooty accent while saying version 3. A word, therefore, can be completely WOVEN: written, oral, visual, electronic, and nonverbal.
Etymology of “synergy”
synergy: “1650s, ‘cooperation,’ from Modern Latin synergia, from Greek synergia ‘joint work, a working
together, cooperation; assistance, help,’ from synergos ‘working together,’ related to synergein ‘work
together, help another in work,’ from syn- ‘together’ (see syn-) + ergon ‘work’ (see organ). Meaning
‘combined activities of a group’ is from 1847; sense of ‘advanced effectiveness as a result of
cooperation’ is from 1957.”
SOURCE: Online Etymology Dictionary.
Since synergy happens when things work together productively, the term multimodal synergy describes the ways in which modes and media coexist and interact within any instance of communication. The word synergy was first used in the English language in the mid-17th century, and the meaning has remained relatively stable, as you can see in the textbox explaining the etymology of the word.
Whether you are analyzing or creating an artifact such as an advertisement that superimposes words on a photograph or a Web site that integrates text, sound, and images on a screen, you should consider the ways in which modes and media work synergistically to inform and persuade your target audiences.
Why Focus So Much on Modes Other than Writing?
By emphasizing multimodal synergy, Georgia Tech’s Writing and Communication Program is not trying to de-emphasize the importance of writing or to add more items to your list of things to study. Rather, you are being asked to study the elements that coexist and interact with writing. Multimodal synergy isn’t about learning more; it’s about learning better.
Why not just focus on writing?
Not all colleges and universities teach multimodal communication as part of their core requirements. In fact, Georgia Tech’s Writing and Communication Program was once called simply the Writing Program, and though classes sometimes included digital media and other forms of communication beyond alphabetic text on printed pages, writing still dominated the objectives and outcomes for students’ learning. Why? The answer is deceptively simple: historically, the ability to write well has long been considered the hallmark of an educated person. This idea is problematic for two reasons: first, the notion of writing “well” suggests absolute definitions of good and bad writing, and second, the notion of an “educated” person suggests that a person simply is or isn’t educated. This thinking involves binaries; that is, it reduces multiple possibilities into paired options such as good/bad and is/isn’t, and in doing so it oversimplifies both writing and education.
As other sections of this textbook demonstrate, writing is rhetorical, and thus it depends on variable elements such as audience and context. What’s appropriate for one audience in one context could be entirely inappropriate for another audience in another context, so communicators can’t be certain of writing well in every situation simply because they are successful in a college writing program.
Binary formulations about writing and education do more than oversimplify the rhetorical dimensions of communication: they imply hierarchies of value that are inextricable from economic class and other aspects of cultural identity. Judgments about appropriate and inappropriate writing (what some people might even call “good writing” and “bad writing”) are, in a sense, matters of taste; and, as sociologist Pierre Bourdieu argues in his seminal study Distinction, “taste classifies, and it classifies the classifier,” and thus it can “fulfill a social function of legitimating social differences” (6–7). In other words, when you judge something to be good or bad (or appropriate or inappropriate), you reveal something about your own education while you imply that a certain type of education (probably yours) is intrinsically better than others. Georgia Tech’s own president spoke with English 1101 students about the complexity of planning an education for the 21st century, one that moves students well beyond binary formulations.

PHOTOGRAPH OF FORMAL PRESENTATION
Georgia Tech’s President, Dr. G. P. “Bud” Peterson.
President Peterson is speaking to first-year students in an English 1101 class about the complexity of 21st-century education.
Credit: Photo © R. E. Burnett.
For example, if you judge a piece of writing that violates grammatical conventions to be bad, you reveal what you have learned about grammatical conventions. Opportunities for learning are not equal for everyone. If you learned about grammatical conventions in high school or college, then you had the privilege of attending high school or college instead of having to work full-time for your own or others’ sustenance. Knowledge about art, literature, communication, and other areas of study are a kind of cultural capital, associated with social status, and as Bourdieu claims, some forms of cultural capital “can only be acquired by means of a sort of withdrawal from economic necessity” (53–54). When someone judges a person as “educated” because she or he writes “well,” then the judge’s claim, intentional or not, reflects on that person’s economic background. The judgment classifies the person whose writing it judges, and in doing so, it classifies the judge as the sort of person who has enough cultural capital to make such a judgment. Thus, the judge’s taste differentiates people with economic access to education from people who lack such access.
How do the power & privilege of communication contribute to cultural capital?
Reading and writing alphabetic text have a particularly strong historical association with cultural capital. In an autobiographical account of his life as a slave in the United States in the 19th century, Frederick Douglass writes about how he learned that “it was unlawful, as well as unsafe, to teach a slave to read” from a slave owner who explained that a slave “should know nothing but to obey his master—to do as he is told to do.” As Douglass learned to read and write, he gained knowledge and power—and thus he defied the constraints of his social station. The connection Douglass makes between this sort of knowledge and political power marks alphabetic text as a—and perhaps the—privileged form of communication. To paraphrase an old adage, pens and keyboards, not swords and guns, have the might to make and break nations.
If reading and writing have such a privileged relationship to culture and power, students, parents, teachers, and legislators might have good reason to contend that writing is the most important mode and that it should therefore be the primary, if not the sole, focus of communication instruction in college. While in many situations writing might be the most powerful and appropriate form of communication, the question “Is writing the most important mode?” demands a binary yes/no answer, and the answer is rarely that simple. The importance of writing is, like everything else related to communication, highly contextual. Instead of trying to judge one mode as better or more important than the others, successful communicators will assess when, why, and how one mode might be the most powerful or appropriate while considering how that mode’s interactions with other modes will affect it. Georgia Tech emphasizes multimodality in part because a communicator cannot fully understand or effectively use writing or any other mode without understanding how that mode relates to others within a given rhetorical situation.
Scientific communication provides many instructive examples about the ways in which combinations of modes and media create historic successes and failures. One obvious example of a scientist whose success depends in part on his competence as a communicator is Stephen Hawking. Beyond his articles for peer scientists, Hawking writes books and creates films and television specials for general audiences (including children). Hawking has been able not only to help his ideas gain traction but also to gain enormous popularity—including being the subject of the 2014 film The Theory of Everything.
However, writing isn’t the only form of communication competence that successful science requires. One of the most notorious incidents in the history of scientific debates about the possibility of cold fusion does not stem from a piece of writing but from public speaking. When Stanley Pons and Martin Fleischmann announced at a press conference on March 23, 1989, that they had achieved cold fusion, they started more than two decades of public outrage and condemnation, and their work became a synonym for junk science. Simply put, accuracy and credibility are critical aspects of good communication. Recently, cold fusion is getting a second look by scientists, reenergizing the debate. Of course, a piece of writing is rarely purely written, and a press conference is never purely oral: you can find graphs and pictures in Hawking’s academic works, you can see the colorful images alongside the text in Hawking’s children’s books, you can see Pons and Fleischmann’s infamous press conference on YouTube, and you can see and hear a 2009 story about the recent investigations related to cold fusion on the television program 60 Minutes.
Writing certainly remains important, and thus in your communication classes at Georgia Tech you will be expected to write a great deal—and to write well. Nevertheless, most writing today occurs on computers, and like the scientific examples just cited, most professional communication today requires communicators to use writing in conjunction with other modes in order to share their ideas and perform successfully in their jobs. The emphasis on multimodality in English 1101 and 1102 affirms writing’s importance and seeks to enhance your cultural maneuverability by teaching you to communicate using the modes and media that define contemporary society.

PHOTO
Frederick Douglass, his second wife, Helen Pitts Douglass, (right), and her sister Eva Douglass (center). Douglass was an American abolitionist whose first book, Narrative of the Life of Frederick Douglass, an American Slave, was published in 1845.
Credit: National Park Service (FRDO 3912).
MODES INFLUENCE ARGUMENTATION
What do you agree with and disagree with in the following quotation? #Evaluate
“From the emergence of alphabetic writing as a portable, durable, and replicable means for the preservation of knowledge, to the development of typography and mass literacy, and to the early and mid-20th-century emergence of cinema, radio, and television, shifts in dominant modes of information at once tend to generate moral, intellectual, and institutional panic and critique, as well as powerful evangelical discourses about their capacity to revolutionize thinking, everyday life, and, of course, the practices of education.”
SOURCE: Carmen Luke, “Pedagogy, Connectivity, Multimodality, and Interdisciplinarity”
How Does Multimodal Synergy Relate to Expectations for Students?
The synergy of modes and media is similar to the synergy that occurs in learning and memory. Think about the sorts of cues that might help you to remember a film you saw a long time ago. If you saw the line “You’re gonna need a bigger boat” in print, you might not think of a particular film title. If you saw the line in print and heard someone read it aloud, you still might not get it. If you saw and heard the line and also viewed a drawing of the related animal, your chances of thinking of the film’s title are better.
If you saw and heard the line, saw a picture, and heard the repeating notes from John Williams’s famous soundtrack, you’d be very, very likely to think of the film Jaws (1975). Exposure to ideas expressed in different modes and media gives you multiple ways to recall and analyze those ideas.
How does multimodality help learning?
Learning and memory are more likely to be successful when multiple modes and media are involved, but combinations of modes and media won’t affect everyone the same way. Some people believe they learn better by reading, while others believe they learn better by hearing, by seeing, or by movement and tactile engagement. However, you should not conflate personal preferences and abilities with a singular learning style.
Cognitive psychologists suggest that such individual preferences and abilities don’t neatly add up to a learning style. Instead, virtually everyone can learn new information, whether the meaning is presented in a visual, auditory, or kinesthetic manner. While many people prefer learning through visual, auditory, or tactile means, their personal preferences and abilities don’t impede their learning but rather strengthen it.

ILLUSTRATION
Shark
Credit: Timothy Knepp, U.S. Fish and Wildlife Service.
Your chances of learning increase when you recognize that your abilities across modes (written, oral, visual, nonverbal/kinesthetic) can enhance your learning and your recall as well as your use of new ideas and processes. Your own multimodal communication allows you greater power as a learner. For example, encountering the same idea in different modes/forms is likely to improve your learning because each version (or iteration) of the idea can reinforce the others. When a professor lectures about the concepts you’ve read about in a textbook, the oral communication reinforces the written communication. When you draw a diagram that uses arrows to show how pictured objects relate to one another and then write a paragraph that explains what the arrows between the objects mean, your combination of visuals with alphabetic text increases your chances of recalling the relationships you represent.
Though research is unlikely to uncover a single best way to accommodate all approaches to learning, a survey of research about multimodality conducted by the Metiri Group for Cisco Systems indicates that “students engaged in learning that incorporates multimodal designs, on average, outperform students who learn using traditional approaches with single modes.” By increasing your competence in multiple modes and media, you increase your chances of understanding the unfamiliar and sometimes difficult concepts you encounter in college.
Conduct an online search for the Cisco report, “Multimodal Learning Through Media: What the Research Says.” Read about the myths and then learn about what is affecting your own learning. #Understand
The communication courses you take at Georgia Tech allow you to gain experience with different modes and media. Given time to reflect about these modes and media, you should be able to develop insight about ways to take advantage of them to increase your learning. This insight should help you to learn more and to learn faster, making you not only a better communicator but also a better student.
The communication courses you take at Georgia Tech allow you to gain experience with different modes and media. Given time to reflect about these modes and media, you should be able to develop insight about ways to take advantage of them to increase your learning. This insight should help you to learn more and to learn faster, making you not only a better communicator but also a better student.
How does multimodality encourage translation, transformation, & transference?
The notion that a single idea can be expressed in different modes and media requires some exploration. As media scholar Marshall McLuhan famously proclaimed, “the medium is the message.” You can’t separate form from content. If you return to the display of word in three different typefaces above on this page, where the appearance of the word changes what it expresses, the ideas are inseparable from the shapes they take. However, you can represent a single idea through different modes and media, using those modes and media strategically to increase your audiences’ understanding of the idea. The content of the sentence “A word is both written and visual” and the visual depiction of that idea exist simultaneously. The content and its visual depiction demonstrate that modes coexist and interact. To reinforce the coexistence and interaction (that is, the synergy), consider how your reaction to the content of this book would be affected if the visual representation were transformed—if fonts were dramatically different and if the color and texture of the paper on which this book is printed were markedly different. In general, considering transformation makes the synergy between written and visual modes clear. Learning more about concepts we call translation, transformation, and transference will help you understand more about multimodality.
Translation involves adapting information for a new audience. In the workplace, translation might involve adapting information in a medical journal to a mass-market newspaper. In English 1101 or English 1102, you might translate information you have collected in a face-to-face interview with a musician, artist, performer, scientist, or engineer by simplifying the language so it makes sense to the general reader.
Translation is rhetorical—whether you are translating text designed for English readers to one for an audience of Japanese readers or translating a visual designed for experts in aerospace engineering to one for nonexperts watching CNN. You consider the rhetorical situation and preferences of your new audience—contextual influences, conceptual sophistication, vocabulary, responsiveness to various kinds of evidence, familiarity with different kinds of visuals, preferences in design, appropriateness of mode and medium for the purpose, and so on. Imagine, for example, that you have interviewed a biomechanical engineer who has invented a new kind of prosthesis. For a short article in an annual report, you need to translate the engineer’s discussion of materials and mechanics into a description that an audience without an engineering background can understand.

PHOTO
Close-up of a domestic passport. Oversized reproduction of photo, address, and signature page that all black citizens in South Africa needed to carry with them during apartheid, now displayed as a nearly 7’ tall exhibition in the Kwa Muhle Museum, Durban, South Africa.
Credit: Photo © R. E. Burnett.

PHOTO
Signature page on domestic passport. Oversized reproduction of the kind of passport that all black citizens in South Africa were required to carry with them during apartheid, now displayed in the Kwa Muhle Museum, Durban, South Africa.
Credit: Photo © R. E. Burnett.
Transformation involves changing and reshaping ideas or information. These examples provide some sense of the nature of transformation:
- Changing genre (e.g., moving a print brochure to information on a Web site)
- Changing scale (e.g., from a postage stamp to a poster)
- Changing medium (e.g., from a live demonstration to a video)
- Changing scope (e.g., from a lengthy manual to a one-page tip sheet)
- Changing color palette (e.g., from a four-color to a sepia-tone photograph)
- Changing pacing (e.g., from a self-paced to an automated PowerPoint)
When you transform an idea from a representation in one mode or medium to a representation in another mode or medium, you consider how the transformation affects rhetorical elements. How does the transformation affect your audience? How does it reflect your purpose? How does it change the organization of your ideas?
Consider the challenge of helping people to understand concepts and practices foreign to their experience and culture. For example, how can a museum in South Africa help foreign visitors understand the horrors of apartheid—legally enforced racial segregation? One way is to reproduce the passes (shown in the accompanying photos of a museum exhibit) that black South Africans needed in order to go anywhere outside the black settlements.
In the exhibit about apartheid (legal in South Africa from 1948 to 1994), the museum transformed a pass from its original dimensions (about the size of a modern passport) to a dramatically large model (nearly 7′ tall). The transformation in scale reinforced the power differential—official government control that stripped black South Africans of their citizenship, eliminated voting rights, restricted employment, and forced people to relocate to segregated black settlements. A transformed pass was displayed to represent the tremendous control it wielded. The transformed pass is larger than the 6′ model of a man holding an actual pass.
Transference involves applying communication strategies from one context to another. The ability to transfer strategies is one of the most important skills you can learn in college; literally, you are transferring your learning from one situation to another. Gaining enough familiarity with a concept or strategy to help you in a single situation doesn’t do you much good in the long run (though it may help you pass a quiz the next day). What’s important is long-term learning—the ability to generalize from one situation to another.
The greater the similarity in the two situations, the closer the proximity in time, and the greater the relevance you see from one situation to another, the more likely you are to transfer your learning. So, when in week three of an English course you learn that evaluating sources involves assessing credibility, relevancy, currency, and reliability, you’re likely to use these criteria not only in the paper you write in week four but also in the history paper you write in week five. You may be less likely to use these criteria in an engineering report you write in week 14. Why? Even though all three disciplines (English, history, and engineering) require evaluating sources, you may see English and engineering as dissimilar, have forgotten what you temporarily learned 10 weeks ago, and consider the work you do in English to be irrelevant to the work you do in engineering. The problem might be compounded by the fact that your engineering professor knows that you were supposed to learn about evaluation in English classes, sees the similarity, proximity, and relevance to your engineering assignment, and expects you to transfer the learning without direct instruction. If you have actually learned about evaluating sources, however, you should be able to transfer the knowledge from one situation to another. For learning to be transferrable, you must be able to adapt your prior knowledge to new situations.
One productive way to increase transference is to increase your use of verbal and visual metaphors, which help you compare dissimilar or unfamiliar concepts, processes, or objects. By identifying similarities, you can develop new ways of thinking, transferring what you know to help you think about unfamiliar concepts, processes, or objects in new ways. Metaphors can help you carry over or transfer meaning from one situation to another.
Transference has additional practical value for you. In your study and practice of communication, both in communication classes and in other classes, jobs, or situations where you are expected to communicate effectively, you will be engaged in two broad kinds of learning, both of which involve transference:
- Learning to communicate involves developing effective ways to share ideas. You should expect to transfer what you learn in communication classes to all other communication you do in academic, workplace, personal, and community situations. You can see the value of transferring strategies such as making arguments, adapting to various audiences, selecting and organizing evidence, respecting different cultures, and designing effective pages and screens.
- Communicating to learn involves using communication in order to understand new and sometimes difficult concepts. Specifically, you can use communicating to learn strategies that include writing definitions and descriptions, creating comparison tables, listing arguments and counterarguments, diagramming key components, synthesizing opposing views, summarizing specifications, categorizing kinds of evidence, and graphing changes over time.
Etymology of “metaphor”
metaphor: “late 15c., from Middle French metaphore (Old French metafore, 13c.), and directly from Latin metaphora, from Greek metaphora ‘a transfer,’ especially of the sense of one word to a different word, literally ‘a carrying over,’ from metapherein ‘transfer, carry over; change, alter; to use a word in a strange sense,’ from meta- ‘over, across’ (see meta-) + pherein ‘to carry, bear’ (see infer).”
SOURCE: Online Etymology Dictionary.
How Does Multimodal Synergy Enhance Intellectual Development?
The ways you think about and use modes and media influence your intellectual development. This section focuses on cognition and representation, encouraging what is often called metacognition—that is, thinking about thinking, a kind of self-awareness about your own thinking processes that enables you to monitor and change these processes in productive ways. The metacognitive questions in the margins of this textbook are just one way it prompts reflective thinking and action regarding composing and multimodal synergy. Metacognition helps you consider ways to use modes and media to represent various ideas, processes, and objects.
How does multimodality affect cognition?
The term cognition refers to mental processes involved in thinking—which includes activities such as knowing, understanding, remembering, questioning, analyzing, evaluating, using, and solving problems. When we think, we respond to multimodal cues, but because we have individual approaches to learning, we respond to and often prefer one mode over another. For example, when you’re learning new information, do you prefer to read about it or watch a video? Do you prefer a hands-on activity that lets you try out the new idea or process? Do you prefer observing a face-to-face demonstration or listening to a podcast? Your preference for one mode or another doesn’t make you a better or worse thinker or learner (see the discussion of learning styles earlier in the chapter), but it influences the ways in which you may choose to learn and communicate. Unless you have specific learning disabilities, you can learn in all modes, so you need to be at least competent in creating and interpreting all modes. Personal and professional success is more likely to come if you are excellent rather than merely competent.
Try to think of a communication situation that only involves one mode—for example, only writing (with no elements of design) or only orality (with no nonverbal cues). Examples are difficult to think of because nearly all communication situations involve more than one mode. #Understand
Regardless of the mode, and no matter how good you are in engineering, math, and science, most theorists believe that thinking requires language. The classic example is based on a theory offered more than 50 years ago by linguist Benjamin Whorf, who speculated that language determines the way we think. Even though most linguists think Whorf’s position is extreme, they nonetheless agree that language lets us name, recall, investigate, and build on abstractions (e.g., feelings), phenomena (e.g., color), objects (e.g., the Hubble telescope), events (e.g., a soccer game), and processes (e.g., splicing a gene). Being good with language (written, oral, and nonverbal) is a strong influence on your ability as a thinker. The better you are with language, the more productive you will be as a thinker and doer.
Thoughtfully select and name three of the colors represented in the color chart. See if others can accurately match your color names with the actual colors. The names you selected aren’t arbitrary, so what makes the task difficult?#Understand #Apply
But language alone, even if you’re very good at it, isn’t enough for effective communication and thinking. Many people also think in images—that is, they engage in visual thinking. Consider a simple example: words help us differentiate “teal” and “sage,” but images do an even better job. Notice the bar that presents 12 colors on a continuum of color. While most of us have the physiological ability to perceive the differences in these colors, we do not have consistent language for naming these colors.

CHART
Color continuum chart for green.
SOURCE: “Full Greens,” MyPantone Web Site.
What is the relationship between epistemology & visual thinking? Epistemology is the study of the nature of knowledge, including its origin, nature, methods, and limits. Epistemology is especially concerned with ways of knowing and concentrates on questions related to thinking such as these:
- What is knowledge?
- How is knowledge acquired?
- Where does knowledge come from?
- What are the limits of knowledge?
- How does experience affect knowledge?
- How do we know what we know?
- Why do we know what we know?
- How can knowledge be tested?
Does visual thinking have epistemological functions or are visuals simply a way of defining and illustrating points already made in language? Visual thinking—defined to include visual imagination as well as perception of various kinds of images (drawings, videos, photographs, diagrams, symbols), and mental operations related to these images—is common in engineering, medicine, and the sciences and has been called “omnipresent in mathematics” (Giaquinto). If visual thinking is important to these disciplines, does it do more than provide a support function for language? Can visual thinking function (without language) as a means of knowing, understanding, remembering, questioning, analyzing, evaluating, using, and solving problems? Is visual thinking critical for creating multimodal artifacts?
Many researchers would answer yes to the above questions. For example, for more than 50 years, Rudolph Arnheim has claimed that visual thinking is central to our cognition. In a lengthy excerpt from an interview, he explained his position:
[S]ensory knowledge, upon which all our experience is based, creates the possibilities of language. Our only access to reality is sensory experience, that is, sight or hearing or touch. And sensory experience is always more than mere seeing or touching. It also includes mental images and knowledge based on experience. All of that makes up our view of the world. In my opinion, “visual thinking” means that visual perception consists above all in the development of forms, of “perceptual terms,” and thereby fulfills the conditions of the intellectual formation of concepts; it has the ability, by means of these forms, to give a valid interpretation of experience. Language, on the other hand, is in itself without form; one cannot think in words, since words cannot contain an object. Language is instructed by sensory perception. It codifies the given knowledge through sensory experience. This doesn’t mean that language isn’t tremendously significant for thought, for all of human development. Human existence is unimaginable without language. I am only stressing that language is an instrument of that which we have gained through perception, in that it confirms and preserves the concepts it forms. We encourage you to consider visual thinking as having enormous epistemological value.
(Grundmann and Arnheim)
The debate about whether language or visuals form the basis for thinking is less important than recognizing that both language and visuals influence the way you represent ideas, processes, and objects; both are critical to your ability to create, interpret, and use multimodal artifacts.
Reread Rudolf Arnheim’s statement about visual thinking and language. Do you agree? Disagree? What aspects of your experience influence your position? #Evaluate
How does multimodality affect representation?
The term representation refers to the way that you think about and instantiate an idea, process, or object. Several broad categories can influence your representation of a particular idea, process, or object:
- The nature of the idea, object, or process itself, information that is usually considered verifiable
- Your perceptions that influence your interpretations of verifiable information
- Sociocultural factors about context or situation in which the idea, process, or object you’re representing will be used, including beliefs, customs, or behaviors typical in a particular group or population
- Your selection of information to represent, based on your sense of appropriateness for the audience and purpose
- The nature of modes and media you choose to represent the idea, object, or process
- Your selection of modes and media as well as your competence in using them
These categories interact synergistically to result in what is often called a “mental model,” or your own internal representation, that is made into a multimodal artifact. As a successful student, you need to become competent in developing mental models—that is, visualizing the representation. Your mental model can be adjusted as you reflect on it to more accurately represent the idea, process, or object and to capture the details you consider appropriate for the audience and purpose.
Imagine you’re interested in maps as objects people can use more easily and conveniently than big, printed paper maps that can be folded and refolded multiple times but are inconvenient to use if you’re walking the streets in an unfamiliar city, trying to find a particular place. You can create a mental model of the kind of map you’d like to use. You can imagine the information this new kind of map would need to display, the symbols or icons that it would use (ones that are easy to see and interpret), the reactions people would have to seeing you use this new kind of map, and so on. Your mental model could also include ideas about the modes and media of the map: Is it entirely visual? Does it have words or symbols or both? Does it talk to you? Can you talk to it? Does it work wherever you are—regardless of location or weather? Does it create a path for you to follow? Does it require training to use, or is it intuitive? You can easily add or discard information in your mental model, responding to the context and sociocultural situation.
Your mental model can be represented multimodally. The success of your representation depends, in part, on your mastery of the “grammar” of the modes you select. As Gunther Kress and Theo van Leeuwen explain, “All modes have grammars” (1). Part of your responsibility is to learn those grammars—the concepts and the conventions—and use them productively. For example, to make your mental model of a map into an actual artifact, you may need to learn some of the concepts and vocabulary of maps. Some of the terms may be familiar (bearing, coordinates, or elevation) while other map terms may be less familiar (azimuth, gradicule, or plat). (For additional map terms, simply do an Internet search for <map terms> or <map glossary> or <cartography terms>.) All of them will help you consider ways to instantiate your mental model.
Your representation cannot be thought of simply as factual reporting, even if all of the features you describe can be verified as accurate. This is because the concept, process, or object you represent in various modes and media exists in a context, so, as psychologist Lev Vygotsky’s work makes clear, you need to make sure that your representation reflects the context. When you consider the concept, process, or object in context, you can develop alternative views that need to be examined. During the “talking to yourself” phase, you consider and discard, consider and reshape, and consider and formulate your representation. For example, in making your mental model into an actual, usable map, you need to consider a range of contextual factors such as these: How much time will you have to refer to a map? How will the light affect your ability to see the map? Will you be using the map while you’re standing still or walking?
Why Are Self-Reflection & Self-Assessment about Communication Processes & Artifacts Important?
You shouldn’t be surprised if your instructors ask you to reflect about your communication processes as well as about the artifacts you create. In general, reflection means deliberate consideration or careful analysis and contemplation of a particular subject. Historically, reflection has taken a number of forms, but the definition has remained relatively stable. Many people take time every day or every week to reflect on their activities; they consider what they’ve done, how they could have done it differently, and why they did it in a particular way.
In Chapter 5 you will read about the importance of showcasing your work in a carefully curated portfolio that includes reflective comments about your processes and products. In this section, your book discusses the merits of reflection itself as a common practice and as a part of the composing process. Reflection has great potential to help you improve your work, both the processes and the products.
The best reason for engaging in reflection is that it is a metacognitive function that enables and reinforces transference. Reflection is the best way to increase the likelihood that what you learn in one activity or assignment transfers to other activities and assignments. You can generate reflective comments—jotted in a journal, written on your laptop, or recorded on a digital recorder—during your work on a project.
Etymology of “reflection”
reflection: “late 14c., reflexion, in reference to surfaces throwing back light or heat, from Late Latin reflexionem (nominative reflexio) ‘a reflection,’ literally ‘a bending back,’ noun of action from past participle stem of Latin reflectere ‘to bend back, bend backwards, turn away,’ from re- ‘back’ (see re-) + flectere ‘to bend’ (see flexible). Of the mind, from 1670s. Meaning ‘remark made after turning back one’s thought on some subject’ is from 1640s. Spelling with -ct- recorded from late 14c., established 18c., by influence of the verb.”
SOURCE: Online Etymology Dictionary.
The questions below can help as you compose reflections. These questions are not intended to be exhaustive or mutually exclusive. They’re representative of the kinds of questions you can generate. The overlap among the questions demonstrates that although the composing process has identifiable parts, the process itself is multifaceted, synergistic, and recursive. You certainly should not feel compelled to answer all of the questions. Rather, pick and choose ones that you believe can help you increase your facility in all modes and media because your reflection is an opportunity for you to consider their synergy.
Reflective Questions
- What is my purpose in this reflection?
- Who is my audience for the reflection?
- What argument am I making?
- How can I organize my reflection?
- What kinds of evidence are appropriate and persuasive (quotations from my journal, quotations from my artifact)?
- How much attention do I need to give to language and design conventions?
Role
- What can you say about your work as a writer, speaker, or designer in completing this assignment?
- How have your own experiences and biases affected the way you engaged in the process and affected the way you selected the modes and media for the artifacts?
- How have your social circumstances (e.g., race, sexual orientation, socioeconomic class, gender, religious preference) affected your artifact?
Multimodality & Media
- What might happen to the artifact you’ve created if you changed the mode? If you changed the medium?
- In what ways are the modes and media in your own work synergistic?
- How did you creatively and productively use conventions in each mode?
Collaboration
- In a collaborative project, what was your role and how well did you carry it out?
- What would your teammates say about your reliability, contributions, attitude, cooperation, and quality of work?
- Based on your most recent collaborative project, what collaborative strategies can you improve? What leadership strategies can you improve? What support strategies can you improve?
Performance
- What’s your favorite aspect of the design of your visual artifact (Web site, brochure, poster, PowerPoint, etc.)? Why?
- What’s your favorite sentence in the entire paper or oral presentation? Why?
- What are three of the best aspects of the assignment you just completed? What makes them so good?
Development
- What parts of your project would you make sure not to repeat in the future? What would you absolutely want to do again?
- What would you have done differently if you had more time or greater access to software or other tools?
- How did your views of the work change or stay the same? To support your position, quote from the reflective comments you wrote during the project.
Learning
- What did you learn that wasn’t part of the official assignment?
- What did you learn in doing this assignment that surprised you?
- What have you learned that you can carry to the next communication task?
Consider how you want to present your reflection—as a journal largely or exclusively for yourself, as email to reviewers of your draft, as a memo largely for your instructor, as a prefatory or concluding commentary for the audience, as a blog entry commenting about your own processes or style, as an entry in a class discussion board about developing communication competencies, or in some other form (O’Neill).
What is assessment, & why is it important?
Assessment matters because it lets you know what works and what doesn’t in a particular process and for a particular artifact. In Chapter 5, you will learn more about the assessment practices and rubric used by the Writing and Communication Program. However, the best assessment you can get is the complete and conscientious assessment you learn to do for yourself. Experts in disciplinary and professional arenas are able to seek feedback from others, but they also conduct rigorous, thorough self-assessment. They are often their own toughest critics.
Who else can assess your multimodal work?
- Your instructor is an invaluable critic. Seek feedback. Listen carefully and take notes during conferences. Read your instructor’s written remarks. Ask questions about what you don’t understand. Follow up by checking in your textbook for further explanations.
- Your peers are also invaluable critics. Although they usually make very different remarks than your instructor, they are a good audience. When a peer identifies problems, for example, in logic, evidence, or coherence, you should pay attention.
- Your friends outside of class may be willing to review your work. Provide them with the assignment sheet and the assessment criteria so they can make useful remarks.
- If you are doing a service-learning or client-based project, your client may well be able to provide feedback. Be specific in indicating what you want the person to review. You usually shouldn’t ask a client to proofread, but a client can provide invaluable insight about aspects of your argument that are (or aren’t) persuasive.
What tools can you use to assess your own work and progress? Your instructor will give you assessment criteria (probably a list or a rubric) for each assignment. While the criteria separate various rhetorical elements and modes for purposes of assessment, you know that in practice these elements are inseparable. For example, you can discuss “audience” as a separate rhetorical element in a paper you write or a Web site you develop, but in practice audience cannot be separated from the organization you use, the argument you make, the examples you select, or the visuals you incorporate. While the separation is artificial, you can think of the separate elements as a way to gain greater understanding about what feeds the synergy of the final product.
References
Arnheim, Rudolph. Visual Thinking. U of California P, 1969.
Bourdieu, Pierre. Distinction: A Social Critique of the Judgement of Taste, translated by Richard Nice. Routledge, 1984 (1979).
Douglass, Frederick. Narrative of the Life of Frederick Douglass, An American Slave, Chapter 7. Project Gutenberg, 1845. 2009.
Giaquinto, Marcus. Visual Thinking in Mathematics: An Epistemological Study. Oxford UP, 2007.
Grundmann, Uta, and Rudolf Arnheim. “The Intelligence of Vision: An Interview with Rudolf Arnheim.” Cabinet, vol. 2, 2009, rev. 2015.
Jewitt, Carey. “Multimodality and New Communication Technologies.” Discourse and Technology: Multimodal Discourse Analysis, edited by Phillip Levine and Ron Scollon. Georgetown UP, 2004, pp. 184–95.
Jewitt, Carey, Gunther Kress, Jon Ogborn, and Charalampos Tsatsarelis. “Exploring Learning through Visual, Actional, and Linguistic Communication: The Multimodal Environment of a Science Classroom.” Educational Review, vol. 53, no. 1, 2001, pp. 5–18.
Kress, Gunther. “Gains and Losses: New Forms of Texts, Knowledge, and Learning.” Computers and Composition, vol. 22, 2015, pp. 5–22.
Kress, Gunther. “Multimodality: Challenges to Thinking About Language.” TESOL Quarterly, vol. 34, no. 2, 2000, pp. 337–40.
Kress, Gunther. “Rhetorics of the Science Classroom: A Multimodal Approach.” Multimodal Teaching and Learning: The Rhetorics of the Science Classroom. New York: Continuum, 2001.
Kress, Gunther, and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. Routledge, 2006.
Luke, Carmen. “Pedagogy, Connectivity, Multimodality, and Interdisciplinarity.” Reading Research Quarterly, vol. 38, no. 3, 2003, pp. 397–403.
“Metaphor.” Online Etymology Dictionary. 2001–2015.
Metiri Group. Multimodal Learning Through Media: What the Research Says. Cisco Systems. 2008.
O’Neill, Peggy. “Reflection and Self-Assessment: Resisting Ritualistic Discourse.” The Writing Instructor. 2002.
“Reflection.” Online Etymology Dictionary. 2001–2015.
“Synergy.” Online Etymology Dictionary. 2001–2015.
“Wiki.” Wikipedia. 2015.