Assembling Arguments: Multimodal Rhetoric and Scientific Discourse

By Jonathan Buehl

Scientific arguments—and indeed arguments in most disciplines—depend on visuals and other nontextual elements; however, most models of argumentation typically neglect these important resources. In Assembling Arguments, Jonathan Buehl offers a concentrated study of scientific argumentation that is sensitive to both the historical and theoretical possibilities of multimodal persuasion as it advances two related claims. First, rhetorical theory—when augmented with methods for reading nonverbal representations—can provide the analytical tools needed to understand and appreciate multimodal scientific arguments. Second, science—an inherently multimodal enterprise—offers ideal subjects for developing general theories of multimodal rhetoric applicable across fields.

In developing these claims, Buehl offers a comprehensive account of scientific persuasion as a multimodal process and develops a simple but productive framework for analyzing and teaching multimodal argumentation. Comprising five case studies, the book provides detailed treatments of argumentation in specific technological and historical contexts: argumentation before World War I, when images circulated by hand and by post; argumentation during the mid-twentieth century, when computers were beginning to bolster scientific inquiry but images remained hand-crafted products; and argumentation at the turn of the twenty-first century—an era of digital revolutions and digital fraud.

Each study examines the rhetorical problems and strategies of specific scientists to investigate key issues regarding visualization and argument: 1) establishing new instruments as reliable sources of visual evidence; 2) creating novel arguments from reliable visual evidence; 3) creating novel arguments with unreliable visual evidence; 4) preserving the credibility of visualization practices; and 5) creating multimodal artifacts before and in the era of digital circulation.

Given the growing enterprise of rhetorical studies and the field's contributions to communication practices in all disciplines, rhetoricians need a comprehensive rhetoric of science—one that accounts for the multimodal arguments that change our relation to reality. Assembling Arguments argues that such rhetoric should enable the interpretation of visual scientific arguments and improve science-writing instruction.

• Language: English
• Publisher: University of South Carolina Press
• Release date: Jan 20, 2016
• ISBN: 9781611175622

Jonathan Buehl

Jonathan Buehl is an associate professor of English and director of the Business and Technical Writing Program at Ohio State University. He has taught technical and scientific writing in academic and industrial contexts since 2004. Buehl’s essays have appeared in College Composition and Communication and Technical Communication Quarterly. .

Book preview

Part 1

Motives and Methods for a Multimodal Rhetoric of Science

Chapter 1

Scientific Visuals

Rhetorical Potential and Rhetorical Problems

Neither the bare hand nor the understanding left to itself are of much use. It is by instruments and other aids that the work gets done, and these are needed as much by the understanding as by the hand. And just as instruments improve or regulate the movement of our hands, so instruments of the mind provide suggestions or cautions to the understanding.

Francis Bacon, Novum Organon, Book I, Aphorism II

There are several ways to truth; the scientific way is one of these. There are several ways of perceiving a scientific truth, but the simplest is the visual one.

Blodwen Lloyd, Science in Films

In 2006 Geoffrey Chang, an award-winning researcher at the Scripps Research Institute, retracted five papers that described the structures of complex cellular proteins. In the first and most influential of the retracted papers, Chang and his colleagues described the arrangement of the atoms in MsbA, a protein in the bacterium Escherichia coli. MsbA is an ATP-binding cassette (ABC) transporter—a biochemical mechanism that moves molecules between the layers of a cell’s membrane. ABC transporters are proteins of interest because pathogens might use these mechanisms to eliminate antibiotic molecules. Thus, these proteins could be crucial elements in the development of antibiotic resistance in bacteria strains.

Chang’s breakthrough paper—with its convincing stereoscopic visualizations of MsbA’s structure—warranted publication in Science in 2001. (One set of his stereoviews appears in plate 1, following page 148.)* Chang then used the structure as the basis for other work, including determining the structures of MsbA for the pathogens Vibrio cholera (Chang 2003) and Salmonella typhimurium (Reyes and Chang 2005). Unfortunately, the molecular structure presented in 2001 was inaccurate, and hence the other structures were also flawed.

As Greg Miller reported in a news feature for Science, a blip in the code of an in-house data-analysis program flipped two columns of data, inverting the electron density-map from which [Chang’s] team had derived the final protein structure (1856). This error propagated through the visualization process, resulting in a faulty determination of the protein’s structure. Specifically, the flipped data changed the chirality or hand of portions of the visualized molecule. A molecule’s chirality can significantly affect how it reacts and functions; for example, the tuberculosis drug ethambutol is a left-handed molecule whose right-handed counterpart causes blindness (Li and Haynie 451). The difference between Chang’s representation of MsbA and a representation based on the correct chirality is demonstrated in plate 2 (following page 148).

Chang’s unintentional error had serious consequences for the scientific communities studying ABC transporters. MsbA was the first ABC transporter to be mapped successfully, and other crystallographers studying this class of proteins built on Chang’s work when conducting research on similar structures. In the light of Chang’s retraction, they had to reconsider their work (Greg Miller 1857). Moreover, Chang’s error had significant consequences even before it was revealed; specifically, researchers working with different protein-characterization methods faced skepticism when their findings did not corroborate Chang’s structure. As Greg Miller noted, the biochemist David Clarke had a hard time persuading journals to accept [his group’s] biochemical studies that contradicted Chang’s MsbA structure, and grant applications that did not agree with Chang’s structure were, in Clarke’s words, given a rough time (Greg Miller 1857). Chang’s erroneous structure had become a persistent but troublesome fact.

The errors in Chang’s MsbA structure were formally recognized when Kaspar Locher, a researcher working on a similar bacterial protein (Sav1866), compared his work to Chang’s structure. According to Miller, "After pulling up Sav1866 and Chang’s MsbA from S. typhimurium on a computer screen, Locher says he realized in minutes that the MsbA structure was inverted" (1856). Locher and his coauthor, Roger Dawson, included a stereo-view graphic (plate 2) to compare the two proteins in their Nature report documenting the structure of Sav1866. The MsbA structure was presented as a purple wire structure; their Sav1866 structure was presented in green.

The purpose of Dawson and Locher’s paper was to define the Sav1886 structure and not to contradict Chang’s MsbA structure specifically, so they included the visual in an online supplement and not in the body of the paper. Nevertheless, the visual comparison serves three purposes: (1) It demonstrates the incompatibilities between the structures as they were defined; (2) it preempts the rejection of the Sav1886 structure on the grounds that it does not agree with Chang’s structure; and (3) it shows that Chang’s structure can be compatible with Sav1886 if the underlying data of the MsbA structure is corrected. The article’s text presented a verbal refutation whose conclusion clearly stated the problem: The observed architectures of MsbA and Sav1866 remain incompatible, even when considering that the proteins may have been trapped in distinct states, and the differences—if real—would indicate a convergent evolution of the two proteins (182). This explicit refutation was clearly necessary, given the accrued authority of the Chang structure, and Dawson and Locher’s text is brief but thorough. They argue that their structure is consistent with similar proteins; hence, it is more accurate than the Chang structure, which is inconsistent with these other findings. Moreover, by changing the chirality of part of Chang’s structure in Part B of their figure, Dawson and Locher bring it in line with previous expectations for both it and analogous proteins. Thus, they do not dismiss the entirety of Chang’s work, just the faulty premise that led to a faulty structure.

Rhetoricians would identify Dawson and Locher’s rhetorical tactics as visual and verbal forms of arguments from incompatibility and analogy. The superimposed structures in Part A of their figure offer a state of affairs contradicting the aligned structures in Part B—and both states cannot be true. Although the true structures of Part B are not identical, they are visually like each other—as one would expect of homologous structures.

Through word and image, Dawson and Locher rhetorically restructured reality for their field, but their powerful argument is only part of what makes this case so interesting. Through Miller’s report, we get a sense of Locher’s multimodal composition process—the conception and assembly of the argument. Seeing Chang’s structure on the screen prompted Locher to realize its faults immediately; the protein-structure database provided the raw material Dawson and Locher needed to craft a refuting visual argument. Reactions of other scientists to the entire ABC-transporter episode reveal other rhetorical issues at the heart of modern science.

In a Nature letter to the editor titled Pretty Structures, But What About the Data?, the biochemist Chris Miller reacted to the Chang incident by elaborating on what he sees as systemic problems in the culture of protein-structure scientists:

The mistake so clearly illustrates two lessons that we aging baby boomer professors ram down the throats of our proteomically aroused graduate students: (i) that those lovely colored ribbons festooning the covers and pages of journals are just models, not data, and (ii) that you invite disaster if you don’t know what your software is actually doing down there in the computational trenches. Students have a hard time subsuming these dicta into their souls for two reasons: the tyranny of authority (the vanity journals occupying the vanguard) and the inherent beauty of the macromolecular models that emerge, as if by magic, from the user-friendly crystallographic software accumulated over decades through the generous labor of the field’s talented reciprocal space-cadets.*

Miller’s comments and the case at large illustrate concerns about agency and algorithmic premise-driven visualization (know what your software is actually doing), the emotional appeal of visual artifacts (the inherent beauty of the macromolecular models), disciplinary history and conventions ([visualization] software accumulated over decades), the ethos of specific publications (the tyrannical authority of prestige journals), and the representational and rhetorical status of images ([visual] models are not data). Such concerns are not limited to proteomics, nor are they unique to our technologically saturated era. Digital technologies have enabled new possibilities for representing data and circulating arguments; however, anxiety about the rhetorical and epistemological status of scientific images has always been part of science, beginning in the period often called The Scientific Revolution.

Communicating science has always been a messy multimodal enterprise in which instruments and representations of their output allow scientists to make contestable claims about reality. Daniel Freedberg’s The Eye of the Lynx describes how Galileo Galilei and his colleagues in the Lincean Society—the first scientific academy—capitalized on the power of images when documenting celestial and terrestrial phenomena. Visual representations of Galileo’s telescopic observations provided crucial support for his astronomical arguments. Similarly, Frederico Cesi, the founder of the Lincean Society, and other Linceans used microscopes to draw images of unprecedented detail; they also developed illustrated compendia to catalog and classify plants, animals, fossils, and fungi. Although these early-modern scientists found images useful in arguing about and documenting nature, the rhetorical power of these scientific images came with physical constraints, representational limitations, and epistemological risks.

Material constraints affected how images were reproduced and circulated. Cesi wanted to publish Galileo’s depictions of sunspots in folio form, but he had to settle for quarto-sized images (Freedberg 125). Woodcuts provided the means for reproducing images in the society’s compendia, such as the Tesoro Messicano, a descriptive tome of species from the New World that the Linceans copied from Spanish manuscripts. However, the woodcut medium created rather rudimentary images that could not capture many of the details critical for identifying a species (304). Moreover, although the Linceans embraced visualization, they were also wary of images. As Freedberg has explained, Cesi struggled with the relationship between detail and abstraction in creating usable images: From the beginning Cesi’s work was riven by a fundamental tension between the desire to picture everything and the desire for order. Pictures showed too much. They could convey texture and color and irregularity in meticulous detail; but it was precisely this that detracted from their ability to show what was essential and regular about a thing (349).

Cesi and the Linceans were also challenged by seemingly precise but impossible images. In compendia and taxonomic codices, detailed images of absurd anthropomorphic plants and fictional beasts could masquerade as visualized facts. In some cases, obviously false images were even propagated for political reasons. For example, the Lincean Johannes Faber knowingly used image and text to defend the existence of Dracunculus Barberinus, a fabricated species of winged reptile named after the powerful Barberini family. The Linceans’ patron Cardinal Francesco Barberini owned a supposed Dracunculus specimen, so they could not afford to ignore or openly debunk the false dragon (Freedberg 362–66).

Despite the four hundred years of scientific development and technological change between them, the contemporary protein crystallographers Dawson and Locher have much in common with the members of the Lincean Society. Both groups faced rhetorical tasks requiring combinations of images and text, and their rhetorical performances were both enabled and constrained by available technologies of observation and reproduction. For different reasons, each was challenged by the ability of scientific images to construct false but seemingly factual representations. Dawson and Locher needed an image to refute Chang’s accepted protein structure; Faber used an image to validate a ridiculous fact, the existence of the Dracunculus. As these cases and dozens of others demonstrate, visualization offers creators of scientific discourse immense rhetorical potential and introduces equally significant rhetorical problems; however, a comprehensive rhetorical account of multimodal scientific argumentation has yet to emerge. This neglect has been noticed by rhetoricians of science and scholars in other fields of science studies.

In the past ten years, rhetoricians have identified scientific visuals as a significant but understudied aspect of scientific persuasion. Jeanne Fahnestock observed in a 2005 state-of-the-field piece that practitioners of an improved rhetoric of science will have to continue and increase their engagement with the pervasive and persuasive role of visuals in scientific texts (Rhetoric of Science: Enriching the Discipline 283). More recently, Alan Gross has identified the same gap to establish exigency for his cognitivist approach to verbal/visual interaction. In characterizing the rhetoric of science through five foundational texts (Bazerman, Prelli, Moss, Condit, and Ceccarelli), Gross observes that none of these major works in the rhetoric of science analyze a single visual (Presence as a Consequence 266). Moreover, he explains, major texts that have engaged the visual when explicating texts—such as Gross, Harmon, and Reidy’s Communicating Science and Fahnestock’s Rhetorical Figures in Science—do not by any means treat [visuals] as semiotic equals (266).

Calls for a rhetorical approach to scientific visuals have also been heard in other sectors of the science-studies landscape. Peter Galison has observed that his field— History and Philosophy of Science (HPS)—needs a syncretic understanding of argumentation and visualization. In his 2008 essay Ten Problems in the History and Philosophy of Science, Galison identifies understanding technologies of argumentation as the third problem on his hit list of intellectual challenges:

When the focus is on scientific practices (rather than discipline-specific scientific results per se), what are the concepts, tools, and procedures needed at a given time to construct an acceptable scientific argument? We already have some good examples of steps toward a history and philosophy of practices: instrument making, probability, objectivity, observation, model building, and collecting. We are beginning to know something of the nature of thought experiments—but there is clearly much more to learn. The same could be said for scientific visualization, where, by now, we have a large number of empirical case studies but a relatively impoverished analytic scheme for understanding how visualization practices work. So, cutting across subdisciplines and even disciplines, what is the toolkit of argumentation and demonstration—and what is its historical trajectory? (116)

Understanding technologies of argumentation is important for philosophical and historiographical reasons, but it is also important for developing science-writing pedagogy. Technical communication scholars need to better understand the multimodal practices of scientific argumentation so that we can better prepare our students to create effective arguments in the twenty-first century.

Assembling Arguments: Multimodal Rhetoric and Scientific Discourse demonstrates that rhetoric—the millennia-old discipline of studying and producing persuasive discourse—can provide the analytical machinery needed to grapple with the multimodal means used to create scientific arguments. I demonstrate this capacity by applying rhetorical theory to six questions related to situations faced by scientists and scientific editors:

How do scientists produce persuasive visuals?

How are visualization practices established as scientifically credible?

How do scientists deploy visualized data to assemble new arguments?

How do scientists use verbal and visual means to transform problematic data into acceptable support for novel claims?

What are the practical and ethical implications of modifying visual artifacts for scientific arguments?

How have scientists and scientific editors perceived the rhetorical affordances and practical constraints of new technologies?

I approached these questions through case studies of multimodal scientific arguments; these cases then informed the model of multimodal rhetoric described in chapter 2.

My first study (part 2) takes a rhetorical approach to a classic case from the history of science—the development of X-ray diffraction crystallography. When German physicists bombarded a crystal sample with X-rays in 1912, scientists were still trying to define the recently discovered phenomena. Were X-rays particles? Were they waves? The German experiment resulted in compelling visual evidence that X-rays are indeed waves, but physicists disagreed over how to interpret the X-ray images. The multimodal arguments surrounding this debate demonstrate how arguments based on causality, analogy, and incompatibility can integrate images and text to create new visualization tools.

My second study (part 3) examines another classic case—the confirmation of sea-floor spreading as the mechanism of continental motion. Although the continental drift hypothesis had been circulating since Wegener proposed it in 1912, many scientists considered it mere speculation because no one could empirically verify how the continents had moved. Marine magnetism data collected in the 1960s supported a hypothesis that volcanic upwelling in midocean ridges was slowly forcing the continents apart. However, the data required rhetorical fashioning to become appropriate support for the argument. The logical resources offered by rhetorical figures were visually encoded to create paradigm-shifting arguments from causality and transitivity.

My third case (part 4) presents the rhetorical history of a 2007 article that made controversial claims with significant implications for climate studies. What the article’s authors described as twilight zones between clouds and aerosols are complex regions of the atmosphere around clouds whose properties were not accounted for appropriately in contemporary climate models. Initially rejected by local peers and peer reviewers, their paper required significant revisions before it was accepted. These scientists developed and refined an array of verbal and visual strategies to overcome resistance and to argue the twilight zone into existence. By coordinating close readings of multiple drafts with comments from the authors and an empirical reception analysis, I was able to track the production of their multimodal dissociation argument and its circulation as a viable knowledge claim.

My last two cases examine how digital media enrich and complicate multimodal argumentation in scientific contexts. First, I study the ethical problems and rhetorical possibilities of adjusting images with digital tools. High-profile instances of scientific fraud and more ambiguous cases of honest scientists modifying images inappropriately forced scientific editors to mark clearer boundaries between rhetorical presentation and dishonest fabrication in the Age of Photoshop. Disciplinary discussions of imaging practices—when read through the terms of Perelman and Olbrechts-Tyteca’s The New Rhetoric—reveal otherwise tacit assumptions about the rhetorical and epistemological functions of the modern scientific image.

My final case study provides a rhetorical history of moving images in formal scientific arguments. After describing how celluloid film and VHS tapes have supported written arguments in the past, I analyze a sample of contemporary arguments incorporating digital videos. Although scientists recognized the persuasive affordances of moving images long before the advent of the Internet, the electronic distribution of scientific articles created new opportunities to activate those affordances.

Although each case offers close readings of specific texts, contexts, and technologies, together they demonstrate the model of multimodal rhetoric described in the next chapter. But, before describing this model and the rhetorical tools I used when developing it, I need to address two conceptually large but syntactically simple questions that readers unfamiliar with rhetorical studies of science might be asking: Why rhetoric? Why science?

Why Rhetoric?

Until recently, much of the research on scientific visuals has come from scholars studying the history of science, philosophy of science, and sociology of science or from art historians interested in scientific images. Scholars working in these disciplines certainly offer important insights on the history, status, and functions of scientific visualizations; however, as Fahnestock, Gross, and Galison have separately argued, the primacy of visual communication in scientific knowledge-making warrants the development of a comprehensive rhetorical understanding of the intersection of scientific visualization and argumentation. But what might that entail? What are the differences between a rhetorical approach and other approaches?

First, as Alan Gross has argued, the rhetoric of science differs from the history, philosophy, and sociology of science in that it foregrounds the explication of communicative artifacts: Rhetoric ‘stars’ the texts, tables, and visuals, that is, it makes their hermeneutic unraveling central (Starring the Text ix). Second, rhetoricians star the text because we teach the text.

Rhetoricians who study scientific discourse often teach (and train others to teach) science writing, technical communication, and similar courses: Our research informs our teaching, and our teaching informs our research. The pedagogical motives for studying the rhetoric of science are summarized well in the introduction to Bazerman’s foundational work Shaping Written Knowledge: As a university teacher of writing I was charged with preparing students to write academic essays for their courses in all disciplines. Since academic assignments bear a loose relationship to the writing done by mature members of disciplines, a serious investigation of writing within the disciplines promised to turn up information useful to teaching undergraduates (3). Bazerman’s pedagogical needs led him to produce thorough interdisciplinary accounts of the experimental article as socially situated discourse. Ultimately, his work and the work of others have influenced how science writing is taught to university science students. For example, Penrose and Katz’s textbook Writing in the Sciences: Exploring the Conventions of Scientific Discourse marshaled the rhetoric-of-science corpus to help students navigate the rhetorical situations of the scientific life. Moreover, as Moskovitz and Kellogg (2005) and Zerbe (2007) have argued, rhetorical approaches to science can inform general composition courses, thereby improving students’ scientific and cultural literacies while introducing them to academic writing.

The research that evolved into Assembling Arguments was also motivated by pedagogical concerns. As an instructor of science writing in both university and industrial settings, I’ve encountered teaching moments involving visuals in which common prescriptions to be clear, be ethical, and add captions were insufficient. For example, when developing report-writing course materials for a biotechnology company, I incorporated examples from drafts of documents in the company’s archives. One visual argument had some seemingly unusual features that worked well in stimulating student conversations about visuals. In a side-by-side comparison of two chromatograms, the author had adjusted the y-axis scale of one graph; it was in units half as large as the scale of the other graph. However, the author made no mention of the scale differences in the figure caption or in the report text.

Some students were appalled by the undocumented scale adjustment; others did not care about the adjustment at all; still others thought the adjustment was appropriate and perhaps necessary for the report’s argument. For the latter group, the x-axis position was important, not the modified y axis; moreover, the scale adjustment highlighted information in the one graph that they might not have seen otherwise. The most cynical students chalked up the mistake as a by-product of composition practices; they assumed the author just plugged in whatever graphs came out of the system.

Different groups had different opinions, so I had no clear sense of what was correct in this institutional context. However, it was clear to me that these simple graphics, produced by a standard instrument, were enmeshed in a network of tacit rhetorical practices. To help these students, I needed to account for the rhetorical activities taking place between the instrument recording the data and the documents establishing scientific facts.

After encountering problems in the classroom and puzzling over historical and contemporary scientific texts, I sought tools to make sense of what were seemingly specialized rhetorical tasks of scientific work—inventing new visualizing instruments from uncertain premises, repurposing older data graphics to support new hypotheses, convincing skeptical peers with image and text, and policing norms of representation when new technologies disrupt conventional practices. At first glance, the tools that helped me make senses of these practices might seem oddly eclectic—the rhetorical figures of classical rhetoric, the argumentation theory of Chaïm Perelman and Lucie Olbrechts-Tyteca, the visual grammar of Kress and van Leeuwen, Burke’s terministic screen. But each member of this eclectic set exposed specific aspects of scientific rhetorical situations and the multimodal compositions of persuasive scientists. In thinking through how these tools worked—and how they worked together—I developed a rhetorical model meant to be a simple but productive framework. This model for understanding multimodal arguments repurposes existing theory to account for how people coordinate cognitive, material, and social resources to conceptualize, assemble, and circulate persuasive arguments. Although Assembling Arguments is not an explicitly pedagogical tract, I reflect on this model’s implications for teaching in my concluding chapter. Those reflections consider what a focused study of multimodal scientific discourse might mean for rhetorical education at large.

Why Science?

There are many satisfying reasons for studying science through humanist lenses. We study science because its powerful discourse is the dominant discourse of our time. We study science to grapple with the ethical and epistemological issues entailed by its practices and applications. We study science to enable our students—within and beyond the academy—to create knowledge with its discourses and to understand the knowledge created by others. These are all fine reasons for humanists to study science. Indeed, rhetoric can support each of these projects by providing tools for probing the discursive guts of science. By understanding scientific discourse, we can help scientists communicate more effectively and help citizens respond to science thoughtfully and critically. However, for rhetorical theorists, there is another, more pragmatic reason for approaching science with the tools of rhetoric: Theorists interested in visual communication—and multimodal communication more broadly conceived—should study science because science offers methodological advantages.

Scientific arguments are productive subjects for rhetoricians interested in multimodal persuasion for four reasons. First, scientific situations are good sources for theory-building cases. Science works because scientists argue, and they argue through multimodal means. As Lemke has shown, every page of a scientific article tends to include at least one visual element (87–113). Gross, Harmon, and Reidy have gone so far as to say that this interaction between the verbal and the visual is the rhetorical core of scientific communication (218). Although such visual-verbal interaction has traditionally been the rhetorical heart of the scientific paper, new technological developments in academic publishing (such as embedded multimedia files) and scientific representation (such as data sonification) have allowed and will allow scientific rhetors to include a wider range of multimodal components in their arguments.* Second, scientific discourse is highly conventional. Once identified, the conventions of a given discourse community can help a rhetorician establish the novelty or typicality of specific cases. Third, the audiences of science discourses are relatively easy to identify. Assessing the audience for a rhetorical artifact is a key task in rhetorical analysis, and rhetoricians can make reasonable assumptions about the audiences of a scientific artifact based on its genre (for example, research report, review article, or grant proposal) and the institutional channels of its circulation (for example, the journal of a published article or the grant-making organization). Finally, scientific artifacts typically possess built-in features for tracking the effects of scientific arguments. As scholars of both visual rhetoric (for example, Finnegan; Messaris) and the rhetoric of science (for example, Harris; Ceccarelli; Paul; Paul, Charney, and Kendall) have argued, empirical methods for studying reception are crucial for generating responsible rhetorical analyses. Although citations of a published article can never tell the whole story of rhetorical reception, examining citation histories is a consistent way of determining how ideas circulate or fail to circulate through discourse communities. Similarly, grant awards can provide tangible evidence of rhetorical success. In short, if you want to assemble a theory of how people argue with images (and sounds and other nondiscursive modalities), studying scientific cases can provide unique methodological affordances. Such affordances are needed to advance the so-called visual rhetoric project to its logical conclusion—a comprehensive multimodal theory of rhetoric.

*Stereoscopic visualizations (or stereoviews) are visualizations that create the illusion of three-dimensional objects through two-dimensional representations. Two versions of the same image are placed side by side but slightly offset. If you stare at a stereoview long enough, a three-dimensional image should appear to hover in between the doubled images.

*The phrase reciprocal space cadets alludes to the concept of reciprocal space, a crystallo-graphic concept developed by P. P. Ewald in the 1930s. Ewald played a significant role in the development of X-ray diffraction crystallography, the case discussed in part 2 of this book.

*Like data visualization, data sonification represents observable phenomena but with sound instead of visuals. For example, a team of Japanese researchers has sonified the movements of the nematode C. elegans, and empirical tests with a group of biologists revealed unique affordances of sound in research: There are some moments when the worms briefly stop their motion. With sounds, such moments are easily detected. But with video, the participants tend not to notice such moments, and [they] have the perception that the worms are moving smoothly without any interruption (Terasawa et al. 4).

Chapter 2

Toward a Multimodal Rhetoric of Science

It doesn’t matter how beautiful your guess is; it doesn’t matter how smart you are.… If it disagrees with experiment, it’s wrong.

Richard Feynman, The Character of Physical Law

In theory there is no difference between theory and practice. In practice there is.

Attributed to Yogi Berra, Albert Einstein, and Johannes van de Snepscheut

In his 2007 review of fifty years of visual rhetoric scholarship, Lester Olson observed, while we now have a wide range of conceptually-driven and historically-situated case studies [of visual rhetoric], we do not have a substantive treatise that might accurately be described as a theory of visual rhetoric (14). The shape and scope—indeed, the very possibility—of such a theory hinges on one’s definition of visual rhetoric. Is visual rhetoric a special class of rhetoric? Is it a subordinate subset of verbal rhetoric? Or is visual communication part and parcel of all rhetoric? For Cara Finnegan, if visual rhetoric is visual rhetoric, and verbal rhetoric is rhetoric, then the iconophobic dominance of text remains unquestioned, and visual rhetoric is forever subordinated to the traditional artifacts of public address (Visual Studies 244). She prefers to think of visual rhetoric as an approach to theory—a project of inquiry that considers the implications for rhetorical theory of sustained attention to visuality (244). Arguably, the ultimate consequence of sustained attention to the visual is a rhetoric that is not just iconophillic but truly multimodal—a rhetoric that accounts for the complete range of representational resources activated as means of persuasion.

It is hardly controversial to mark a rhetorical artifact as either perceivable or not perceivable through vision. A purely sonic radio address is not visual; neither are Braille documents and tactile pictures—haptic renderings of images used by people with vision impairments. However, most rhetorical forms have some visual component. Even when delivering a speech without the aid of visual support, a speaker also delivers his or her appearance—clothing, posture, gesture, mien, and so on. As Stephen Bernhardt has observed, we are always seeing the text when we read a text. Visually informative texts use visual affordances, such as Gestalt principles of proximity, contrast, and continuity, while other artifacts are only visual in that they inscribe words in sequence (94–104). Extreme examples of less visually informative (but nonetheless visual) documents are manuscripts from the scriptura continua tradition produced by ancient scribes, who did not even add spaces between words (see Saenger); however, even these documents were still seen by those who read them. At the other end of the visual-rhetorical spectrum are visual artifacts that communicate without any discursive means, such as the iconic photographs described by Hariman and Lucaites in No Caption Needed. Rhetors who use iconic images—such as the raising of the American flag at Iwo Jima or the Twin Towers of the World Trade Center before or during the events of September 11, 2001—rely on cultural memory to make meaning.

Although rhetorical artifacts might be coded and plotted based on levels of discursiveness and visuality, these are not the only categories of symbolic performance we can assess against calibrated scales. All rhetorical performances could be plotted on a sonic continuum, with soundless verbal texts at one end and exclusively tonal artifacts —such as fugues—at another.* The scripted speech—written language transmitted through spoken utterance—would fit somewhere in the middle of this continuum. One might even arrange rhetorical performances on scales according to haptic or olfactory (and presumably even gustatory) criteria, though the chemical modalities of smell and taste cannot encode language.† Of course, modalities without words are significantly limited; nevertheless, their rhetorical potential underscores the need for holistic multimodal theories of rhetoric—the logical conclusion of the visual rhetoric project. The largest challenge of a more capacious model of rhetoric is accounting for the interactions across modalities, a challenge that has been investigated from different theoretical perspectives and with diverse methodologies. This critical and methodological diversity is demonstrated in the range of approaches to the visual-verbal interaction problem.

Accounting for Multimodality: Learning from the Visual-Verbal Interaction Problem

How do words and images interact to create meaning? Many conceptual frameworks have been used to address this question—a question that has vexed and continues to vex semiotic theorists, cognitive theorists, and rhetoricians. Different approaches foreground different aspects of multimodal meaning-making. In this summary, I compare several approaches to frame the problem space that influenced the development of my own approach to multimodal rhetoric.

In his oft-cited essay Rhetoric of the Image, the semiotician Roland Barthes recognized two types of word-image interaction: anchorage and relay. Anchorage occurs when words fix the meaning of an otherwise polysemous image; for example, a newspaper caption explains the meaning of a photograph. Relay occurs when words and image complement each other to create meaning. In describing relay’s function in comic strips, Barthes explains that the words, in the same way as the images, are fragments of more general syntagm and the unity of the message is realized at a higher level, that of the story, the anecdote, the diegesis (157). In The Photographic Message, Barthes alludes to a third process—illustration, the visual-dominant counterpart to the text-dominant anchorage. Through illustration, the image helps to fix the meaning of the text. Barthes’s identification of the one-way and bidirectional processes in which image and text interact offered a useful initial taxonomy; however, other theorists have sought finer-grained and empirically grounded theories of image-text relations.

Alan Gross’s essays on verbal-visual interaction adapt the cognitive scientist Allan Paivio’s Dual-Coding Theory (DCT) as the foundation for a multimodal approach to scientific rhetoric. In the DCT framework, image and text are routed through different associative structures in the brain. Verbal codes are processed through sequential associations; visual codes are processed according to spatial associations. A third system of referential associations governs the cognitive interaction between these dual codes. For Gross, the fundamental differences between visual and verbal processing require a reconsideration of the text-based approaches so often applied to visual artifacts; that is, we need to stop expecting images to act like verbal texts because these visual phenomena are processed differently by our brains. Instead, rhetoricians need parallel sets of tools for accounting for images, text, and their interaction. When analyzing scientific arguments from Lavoisier and Wegener, Gross uses linguistics, logic, and narrative theory to account for the operations of the text, and he uses Gestalt