李开复-数字十年的自然用户界面

UserInterfaceintheDigitalDecadeKai-FuLeeCorporateVicePresidentMicrosoftCorporationGraphicalUserInterfaceUserInterfaceEvolutionCommandline1985PC1990GUIMultipleWindowsMenus1995InternetHyperlinksSearchEnginesGraphicalUserInterfaceWhatyouseeiswhatyougetGUIInteractionLowlearningcurve.Greatforfrequentactions.DifficultforrichdomainsTechnologyAdvancesBitmapdisplay.Mouse.HTTP/HTML.EnablesGraphicalUserInterfaceNaturalUserInterfaceUserInterfaceEvolutionCommandline1985PC1990GUIMultipleWindowsMenus1995InternetHyperlinksSearchEnginesDigitalDecadeXMLWebServicesSmartdevicesNaturalLanguageMultimodal(speech,ink…)PersonalAssistantNaturalUserInterfaceDowhatImeanNUIInteractionNatural.Scalable.Expressive.TechnologyAdvancesSmartdevices.XML.Webservices.Enables“Atransformingcapability[thatallows]youtospeaktoyourcomputeranditwillunderstandwhatyou'resayingincontext.”--GordonMoore,2002Super-Moore’sLawEnablehuman-levelspeechrecognition(SR):LeveragingMoore’sLaw+moredata+research.Predictable10%errorreductionormoreperyear.Human-levelperformancepossiblein10-20years.Alsoenablenaturallanguage,Bayesianlearning….TasksSRerrorrateHumanerrorrateSR-HumanGapFreestyletranscription30%4%19yearsAlphabetletters5%1%15yearsReadnewspapertranscription3%0.9%11yearsMicrosoft’sPracticalNUIPlanStartwithnaturalentrypoints:Speechfortelephony.Typingforsearchandhelp.Providesimple,evolutionaryUIforend-users.UserswillperceiveNUIas“bettersearch”.Search/helpneedstocomplementGUI.Buildon.NET:VisualStudiofordevelopment..NETframeworks,XML,webservices.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalBuy100sharesofMicrosoft.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalBuyMicrosoftstock.HighInternetTVPhonePDAEaseoftextinput(keyboard/pen)EaseofGUI(screen/Pointer)LowHighPCTabletPCScreenPhoneScreenPhonePDATabletPCCarCarInternetTVSmartDevicesNeedsNUI(speech)PhonePCScreenPhonePDATabletPCCarInternetTVARoadmapforSpeechEaseoftextinput(keyboard/pen)EaseofGUI(screen/Pointer)HighHighLowSpeech-OnlyTelephonyDictationMultimodalCommand/ControlSpeechDemonstrations.NETSpeechPlatformforTelephonySupporttelephonyspeechfromwebapplication.Reducescostofphonecallfrom$2to$0.10.Uses.NETFrameworkandVisualStudio.SupportsmultimodalUIusingSALT.BetainJune;RCinOctober.TabletPCSpeechSpeechdictationsisfasterthanhandwriting.SupportsmultimodalUI.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalFindemailfromJohnaboutthebudget.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalMyprinterisstuck.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalPrint10,000copiesinKinko’sBeijing.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalWhattimedoesBillGates’talkend?ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalSendflowerstomywifeonherbirthdayARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalHoldallcallsunlessit’surgent.ARoadmapTowardsNUIVisionTTSSRDialogTelephonyspeechSearchHelpTasksQ&ATextonlyDelegation&Pro-ActiveFederationAgentsPlanningMulti-ModalIwanttoplanavacationtoEurope.StructuredStorageneedsNUIFindemailfromJohnabouttheBudgetSELECTDocumentNameFROMLocalFileSystem,LocalMailSystem,LocalDocumentsWHEREAuthorcontains“John”andSubjectcontainsLSP_Expand(“Budget”)ORDERBYModifiedDate,CreateDateMyprinterisstuckSELECTDocumentNameFROMLocalFileSystem,LocalHardware,HardwareVendorSupportWHEREVendor=‘HP’andModel=‘DeskJet550”andBodycontainsLSP_Expand(“jam”)ORDEREDBYContext,LearnedBehavior.ShowmethenewDellnotebookSELECTDocumentNameFROMWebSearch,UserHeuristicsWHEREVendor=‘Dell’andCategory=‘laptops’andAge60MakethelettersonmyscreenbiggerSELECTTaskNamesFROMLocalTasks,OSTasks,USerHeuristicsWHERENamecontains‘VideoResolution’ORDERbyContext,LearnedBehaviorPlayBritneySELECTMediaNameFROMLocalFileSystem,WebMediaProvidersWHERENamecontains‘Britney’ORNamecontains‘Spears’ORDERBYCurrentDirectoryBias,LearnedBehavior,Name,ContextNaturallanguagedemonstrationsSmartsearch&help.Usersdon’tknowwheretogoorwhattotype.Companiesdon’tknowwhat’sonusers’minds.Wewanttoclosethatgap.Q&Afor“anyquestion”.Unstructuredstructuredtoolisneeded.Combinesstatisticalandnaturallanguage.ConclusionNaturalUIisneededfortheDigitalDecade:Scalable,naturalUIforsmartdevices.Retrievalofinformationfromstructuredstorage.UItoconnectedwebservices.NaturalUIwillarriveasanevolution:Telephonyspeech,sear