EClerk Office Assistant

EClerkOfficeAssistantbyJonathanEricWolfeSubmittedtotheDepartmentofElectricalEngineeringandComputerScienceinpartialfulfillmentoftherequirementsforthedegreeofMasterofEngineeringinComputerScienceandEngineeringattheMASSACHUSETTSINSTITUTEOFTECHNOLOGYSeptember2004cMassachusettsInstituteofTechnology2004.Allrightsreserved.Author...................................................................DepartmentofElectricalEngineeringandComputerScienceSeptember10,2004Certifiedby..............................................................SethTellerAssociateProfessorofComputerScienceandEngineeringThesisSupervisorAcceptedby.............................................................ArthurC.SmithChairman,DepartmentCommitteeonGraduateStudents2EClerkOfficeAssistantbyJonathanEricWolfeSubmittedtotheDepartmentofElectricalEngineeringandComputerScienceonSeptember10,2004,inpartialfulfillmentoftherequirementsforthedegreeofMasterofEngineeringinComputerScienceandEngineeringAbstractFordecades,peoplehavecontinuedtocollectaninordinateamountofpaperdocu-mentscontainingimportantinformationthatshouldbeeasilyaccessible.Thispaperclutterinhibitsindexingthisinformationandeasilysearchingthroughit.Thisthe-sispresentsthecodearchitectureanduserinterfacedesignoftheElectronicClerk,aproof-of-conceptelectronicofficeassistant.TheElectronicClerk(EClerk)isadevicetoassistinreducingpaperclutterintheofficeenvironment.Thedevicetakespaperandspeechasinput,performsdatabindingbetweeninputstreamsinordertoattachmetadatatoeachdocument,andstructuresthedatausingtheResourceDescriptionFramework(RDF)standard.ThehardwarestructureofEClerkconsistsofadedicatedcomputer,videocamera,scanner,touchscreen,andmicrophoneforcapturinginput.ThesoftwarestructureconsistsoftheGalaxyspeechrecognitionsystem,theHaystackinformationclientforretrievalandmodificationofthecollecteddata,opticalcharac-terrecognition,andagraphicaluserinterfacethatprovidescontinuousfeedbacktotheuser.Primarydesignprinciplesforthisdeviceincludeprovidingcontinuoususerfeed-backandrobustnesstoimperfectinputinordertoprovideatrulyusablesystem.ThesisSupervisor:SethTellerTitle:AssociateProfessorofComputerScienceandEngineering34AcknowledgmentsIwouldliketothankmythesisadvisor,SethTeller,forhiscreativeideasandenergythroughouttheentiredevelopmentofthisthesis.Hisvisionandsupporthasmadethisprojectpossible.IwouldalsoliketothankeveryonethatassistedintheintegrationofGalaxyandHaystack,includingDavidKarger,JimGlass,ScottCyphers,andVineetSinha.Specialthanksalsogotomyhardwarelabcolleagues,RoshanBaligaandKevinWang,forallthefuntimesandthesisreviews.Finally,manythankstofriendsandfamilyfortheunwaveringsupportthroughyearsofworkinghardandplayinghard.56Contents1TheEClerkVision151.1ThesisOverview..................................161.2TheNeedforReducedClutterintheWorkplace................171.3TheNeedforAUsableSystem..........................171.4DesignPrinciples.................................181.5UsageScenarios..................................182Background212.1RelatedWork....................................212.1.1Memex...................................212.1.2MemoryAssistant.............................222.1.3RemembranceAgent...........................232.2Pre-existingComponentsofEClerk.......................232.2.1GalaxyforSpeechRecognition.....................232.2.2HaystackforInformationStorageandRetrieval............252.2.3ScanningandOpticalCharacterRecognition.............263DesignPrinciples273.1ProvideContinuousFeedback..........................273.2DeferNon-CriticalOperationsWheneverPossible..............273.3BeRobusttoUserInput.............................283.4ProvideModularityandMobility........................294SystemArchitectureandComponents314.1EClerkDesignOverview.............................314.2HardwareComponents..............................324.3SoftwareComponents..............................334.3.1GalaxyIntegration............................334.3.2HaystackIntegration...........................344.3.3EClerkControlSoftware.........................364.3.4SymmetricMultiprocessinginLinux..................374.3.5Qt......................................3875SoftwareDesign–EClerkControlSoftware395.1EClerkHub.....................................395.1.1FacilitatingModuleCommunication..................395.1.2PerformingBindings...........................405.2EClerkModules..................................415.2.1SpeechModule..............................415.2.2VideoModule...............................435.2.3ScanModule................................455.2.4OCRModule................................465.2.5StorageModule..............................465.2.6Text-to-SpeechModule..........................475.2.7ScriptingModule.............................476SoftwareDesign–UserInterfaceDesign496.1EntryMode.....................................496.1.1DisplayingCurrentInput.........................496.1.2DisplayingCurrentDeviceStatus....................506.1.3PromptingtheUser............................516.1.4DisplayingBindingsas“Virtual”Documents...........