multimodal-discourse-analysis-as-the-confluence-of

Copyright©2004byGeorgetownUniversityPress,Washington,DC.Allrightsreserved.Unlessotherwiseindicated,allmaterialsinthisPDFFilearecopyrightedbyGeorgetownUniversityPress.Distribution,posting,orcopyingisstrictlyprohibitedwithoutwrittenpermissionofGeorgetownUniversityPress.MultimodalDiscourseAnalysisastheConfluenceofDiscourseandTechnologyRONSCOLLONANDPHILIPLEVINEGeorgetownUniversityTHATDISCOURSEANDTECHNOLOGYareintimatelyrelatedisnotanewperception.EventhephilosopherNietzschegotinawordonthesubject—“Ourwritingtoolsarealsoworkingonourthoughts”—accordingtoArthurKrystal(2002).Ourinterestinthisvolumeisnottotrytodemonstratethatdiscourseandtechnologyliveinasymbioticrelationship.OurinterestisinpresentingaselectedsetofpapersfromtheGeorgetownUniversityRoundTable2002(GURT2002),whichopenedupadiscus-sionamongdiscourseanalystsandothersinlinguisticsandinrelatedfieldsaboutthetwofoldimpactofnewcommunicationtechnologies:Theimpactonhowwecollect,transcribe,andanalyzediscoursedata,and,possiblymoreimportant,theimpactonsocialinteractionsanddiscoursesthemselvesthatthesetechnologiesarehaving.Discourseanalysisaswenowknowitisinmanywaystheproductoftechnolog-icalchange.Atthetimeoftheepoch-making1981GURT(Tannen1982),DeborahTannenchoseashertheme“Analyzingdiscourse:Textandtalk.”Discourseanalysiswasjustthenemergingasasubjectoflinguisticresearch.Thepapersinthatconfer-enceandinthatvolumewereaboutequallydividedbetweenstudiesoftext(dis-courseintheformofwrittenorprintedlanguage)andtalk(discourseintheformofspokenlanguagecapturedinsitubymeansofthetaperecorder).AsFrederickEricksonhasnoted,small,inexpensivecassettetaperecordersmadeitpossibletocapturelanguageinuseinawaythatwasprohibitivelydifficultbeforethe1960s.HewasoneoftheveryfewatGURT1981whowasalreadyusingsoundfilminhisresearch.Nowweareseeingtheproliferationofcommunicationtechnologiesfrompalm-sizeddigitalvideorecorderstocellphonesandchatroomsontheInternet.Journalsaregoingonline,andthesesarebeingsubmittedinmultime-diaformats.Theterm“multimodality”iscomingtobeusedacrossmanyfieldswithinwhichlinguistsworktoencompassthesemanynewtechnologicalchanges.Itwasourgoalinthisfifty-thirdannualconferenceatGeorgetownUniversity’sDe-partmentofLinguisticstobringtogetherscholarsworkinginavarietyoffieldsandinsubdisciplinesoflinguisticsbothtoassessthestateoftheartindifferentareasofresearchandtofacilitatecross-disciplinaryandcross-subfieldlinksinthedevelop-mentofresearchindiscourseandtechnologicalchange.MultimodalDiscourseAnalysisThesubthemeofGURT2002,multimodaldiscourseanalysis,wasintendedtohigh-lighttherecognitiondiscussedinmanyofthepapersinthisvolumethatalldiscourseismultimodal.Thatis,languageinuse,whetherthisisintheformofspoken1Copyright©2004byGeorgetownUniversityPress,Washington,DC.Allrightsreserved.Unlessotherwiseindicated,allmaterialsinthisPDFFilearecopyrightedbyGeorgetownUniversityPress.Distribution,posting,orcopyingisstrictlyprohibitedwithoutwrittenpermissionofGeorgetownUniversityPress.languageortext,isalwaysandinevitablyconstructedacrossmultiplemodesofcom-munication,includingspeechandgesturenotjustinspokenlanguagebutthroughsuch“contextual”phenomenaastheuseofthephysicalspacesinwhichwecarryoutourdiscursiveactionsorthedesign,papers,andtypographyofthedocumentswithinwhichourtextsarepresented.OneoftheproblemsofGURT2002thatwasatleastpartlyaddressedinthesepapersisthequestionofhowweshouldunderstandwordssuchasmultimodalityor,moresimply,modality.Forexample,inTheoVanLeeuwen’schapter,“modality”isderivedfromtheconceptofmodalityingrammaticalstudiesoflanguagewheretheprimarynotionscarriedbythe“modal”verbs(“might,”“could,”“should,”andsoforth)areextendedtomeananyofawidearrayofstancesthatmaybetakentotheexistentialstatusofarepresentation.Inhisthinking,“modality”inthistraditionalgrammaticalsenseneedstobekeptclearfromtheconceptofa“mode”ofcommuni-cation—anyofthemanywaysinwhichasemioticsystemwithaninternalgrammaticality,suchasspeech,color,taste,orthedesignofimages,maybedevel-oped.“Modality”inthegrammaticalsensemayberealizedwithinanyofthemany“modes”thatmaybeusedtocommunicate.Thus,“modality”ispolysemousinthatitmightmakereferenceeithertothegrammaticalsystemofexistentialstancesorsim-plytothepresenceoruseofmodesofcommunication.CareyJewitt’schaptertakesupasecondterminologicalproblemindiscussionsofmultimodality,theproblemofmediation.Shearguesformakingananalyticaldis-tinctionbetweenamodeofcommunicationandamediumofcommunication,though,ofcourse,therecanbenomodethatdoesnotexistinsomemedium.Thefor-merisasemioticsystemofcontrastsandoppositions,agrammaticalsystem,asVanLeeuwenhasnoted;thelatterisaphysicalmeansofinscriptionordistributionsuchasaprintedorhandwrittentext,makingthesoundsofspeech(inthephysicalsense),bodymovements,orlightimpulsesonacomputerscreen.Thenotionofmultimodaldiscourseanalysisinthepapersinthisvolumevariesquiteconsiderablyfrompapersthatfocusprimarilyontechnologicalmediatoonesthatfocusonwhatmightmoretraditionallyhavebeencallednonverbalcommunica-tion.Althoughwebelievethatthispolysemyandambiguitymayultimatelyneedtoberesolved,atleastforindivid