Conversational interaction for semantic access to

Revisedversionpublishedin:M.T.Maybury(Ed.),IntelligentMultimediaInformationRetrieval.MenloPark,CA:AAAI/TheMITPress,1997,pp.399-421.ConversationalInteractionforSemanticAccesstoMultimediaInformationAdelheitStein,JonAtleGulla,AdrianMüller,andUlrichThielGMD-IPSI,Dolivostraße15,D-64293Darmstadt,Germany{stein,gulla,amueller,thiel}@darmstadt.gmd.deAbstractEffectiveinteractionisessentialforeffectiveinformationretrieval.Theentireprocessofretrie-valinteractionshouldbeorganizedinawaythatallowstheusertoconcentrateonherprimarygoals,i.e.,clarifyingvagueinformationneeds,findingappropriatesearchstrategies,andasses-singtherelevanceofretrievedinformation.Accordingly,anintelligentretrievalsystemmustbecapableofinterpretingandnegotiatingambiguousinformationneedsinteractivelywiththeuser.TheMIRACLEsystemintroducedinthischapterofferssemanticaccesstoalarge,exper-imentalmultimediainformationbase.Boththeretrievalengineandthedialoguemanageremployabductivereasoningtogenerateplausibleinterpretationsofambiguoususerrequests/queriesandtocreatecoherentinteractionoptionsbasedonacomprehensivedynamicdialoguemodel.Asopposedtomoststate-of-the-artretrievalsystems,MIRACLEsupportstheuseractivelyonboththepragmaticandthecognitivelevel.1WhyConversationalInteraction?Currentmultimediainformationsystemsemployavarietyofadvanceddatarep-resentation,indexing,andretrievalmethods,butmostlyapplylesselaborate–oftensimplistic–modelsofhuman-computerinteraction.Usually,content-basedretrievalmethodsareemployedtoinducesimilarityrelationshipsamongtheavailableinformationobjects,whichcanthenbeinspectedbymeansofbrowsingoperators.However,astheinformationsystemsandthehuman-com-puterinteractionsgrowincreasinglycomplex,theentireprocessofinformationsearch,retrieval,andrelevanceassessmentneedstobeactivelysupportedbyeasy-to-usemultimodaluserinterfaces.Manyusersofinformationretrieval(IR)systemshaveinitiallyonlyvaguein-formationneedsandoftenencounterproblemsformulatingthemprecisely.2ConversationalInteractionforSemanticAccesstoMultimediaInformationVagueandambiguousinformationneedsaretypicalofmultimediaretrieval,butoccurintextretrievalaswell.However,whereasitseemstobeacceptablethatusersdiscardnon-relevanttextualitemsaslongashighlyrelevantdocumentsareshownatthetopoftherankinglists,therelevanceassessmentrequiredinmulti-mediaretrievalimposesanevenhighercognitiveloadontheuser.Somedatatypes,e.g.,audioandvideosequences,maybeaccessibleusingstandardretrievalmethods(basedontheirspeechcomponents,cf.Jonesetal.,thisvolume),buttheystilldifferfromtextualdocuments,becausetheycannoteasilybeskimmedliketextsorpictures.Inaddition,thetacitassumptionunderlyingmosttextre-trievalsystems–thatrelevancecanbereducedtotopicalsimilarity,which,inturn,isbasedonthe(multiple)occurrencesofindexterms–hasnotyetfoundaconvincingcounterpartfornon-textualmedia.Instead,thereisgrowingevidencethatretrieveddataitems(includingtexts)areassessedaccordingtomulti-dimen-sionalrelevancescaleswhichexpressthespecificneedsofusersintheircurrentsituation.Hence,weclaimthatclarificationofinformationneedsthroughinter-actionwiththesystemisasimportantataskastheemploymentofappropriateindexingandretrievalmechanisms.Asthistypeofinteractionincludesmeta-communicationaboutdialoguegoalsandstrategies,itcanbemodeledascon-versationalinteractionbetweenuserandIRsystem.Fromauser-centeredpointofview,intelligentmultimediaretrievalsystemsshouldbecapableofsupportingtheuserinthefollowingtasks:•Queriesmustbedisambiguatedtoassignappropriateretrievalmethods;•differentinterpretationsofthequerymustbemanaged,andtherelevantinformationobjectsmustbedeterminedforeachofthem;•theinformationobjectsmustbepresentedinsuchawaythattheusercanrelatethemtoherqueryandtoeachother;•unexpectedproblemsituationsanddialoguestrategiesshouldbenego-tiatedandclarifiedinteractivelywiththeuser.Thefirsttwoissuescanbetackledusingsemanticrepresentationsofinformationobjects,linkingthemtotheobjectsbycontent-basedretrievaloperators,andcombiningthemwithtechniquesforquerydisambiguation(e.g.,abduction).Whereasthethirdissuerequirespowerfulpresentationanduserguidancefacili-ties,thefourthaddressesthepragmaticaspectsofinformation-seekingdialogue.Inthischapterwewilldiscussourapproachtocombiningtheselevelsbyinter-twiningmultimediaIRanddialogueplanning.Stein,Gulla,Müller,andThiel3Thenextsectiongivesabriefaccountofrelatedresearchinvariousfieldsthatarerelevantforourwork.InSection3weintroducethesystemarchitectureofMIR-ACLE,whichintegratesalogic-basedretrievalengineanddialoguemanagerasitsmajorcomponents.Asbothemployabductivereasoning,wediscusstheuseofthisinferencetechniquefromtheretrievalpointofviewinSection4,andfromthedialogueplanningperspectiveinSection5,analyzingonetypicalexampletakenfromMIRACLE.Abriefsummaryandoutlineoffutureworkaregivenintheconclusionsofthechapter.2RelatedResearchAsourapproachtacklesaproblemontheborderlinebetweenthefieldsofin-formationretrievalandintelligentinterfaces/dialoguesystems,webrieflysketchresearchresultsfromtheseareaswhichhavebeeninfluentialforourwork.InIR,wehavetoregardtwolinesof