虚拟现实与人机交互-东软

虚拟现实中人机交互技术进展潘志庚杭州师范大学数字媒体与人机交互研究中心电话:[phone]电子邮件:[e-mail]QQ:“秀水清山”:443922077微信:图行天下(443922077)MSN:[e-mail]微博:图行天下(2370191992)教授难当还要当博客:关于自己貌不若潘安，而志存高远梦成华罗庚才可比张蔷，且明心静性不似柯以敏关于数字媒体与人机交互研究中心研究内容：图形图像声音动画几何信号都能处理媒体娱乐光照特征文化信息皆可计算中心定位：研究与应用并重，创新和转化优先一虚拟现实与人机交互生活虚拟化，谈天说地真假难分酷似现实生活交互人性化，指手划脚声情并茂尽在虚拟环境注解：真假难分（VR）、指手划脚声情并茂（HCI）城市规划(西游记乐园)培训和教育（ELVIS项目）游戏和娱乐虚拟商店和虚拟产品展示虚拟现实应用系统ZhigengPan,JiejieZhu,WeihuaHu,HungPakLun,XinZhou:InteractivelearningofCGinnetworkedvirtualenvironments.Computers&Graphics29(2):273-281(2005)JianfengLu,ZhigengPan,HaiLin,MingminZhang,JiaoyingShi:VirtuallearningenvironmentformedicaleducationbasedonVRMLandVTK.Computers&Graphics29(2):283-288(2005)MingminZhang,ZhigengPan,JiaoyingShi,PengWang,QingFengMi:Ahybridrenderingalgorithmfortextileobjects.ScienceinChina，52(3):490-499(2009)5虚拟体育仿真虚拟旅游和虚拟剧场数字文化遗产电子试衣虚拟现实的应用YongLiu,CongfuXu,ZhigengPan,YunhePan:Semanticmodelingforancientarchitectureofdigitalheritage.Computers&Graphics30(5):800-814(2006)NanXiang,HaiyingZhao,XiaojianZhou,etal.UEGM:UncertainEmotionGeneratorunderMulti-Stimulus[J].ComputerAnimationandVirtualWorlds,2011,22(3-4):141-149JingTong,JinZhou,LigangLiu,ZhigengPan,HaoYan.Scanning3DfullhumanbodiesusingKinects.IEEEVirtualReality,2012(IEEETransactionsonVisualizationandGraphics,18(4):643-650(2012))6典型案例1）数字奥运博物馆*数据采集*模型构造*动画生成*交互式漫游*寓教于乐ZhigengPan,WenzhiChen,MingminZhang,JianfengLiu,GangshanWu,VirtualRealityintheDigitalOlympicMuseum,IEEEComputerGraphicsandApplications,vol.29,no.5,pp.91-95,September./October.2009KaiYe,KunZhou,ZhigengPan,YiyingTong,BainingGuo:LowDistortionShellMapGeneration.IEEEVR2007:203-208(顶级会议)BingTang,ZhigengPan,LeZheng,MingminZhang:Interactivegenerationoffallingmotions.JournalofVisualizationandComputerAnimation17(3-4):271-279(2006)2）航海模拟系统获奖:国家科学技术进步二等奖(2008)完成人:金一丞,尹勇,潘志庚,任鸿翔等10人3）漫游京杭大运河运河概况：运河的形成和开凿历史，相关科学知识沿线风景：运河沿线的风土人情，历史典故，名胜古迹漫游体验：漕运模拟和体验；船闸系统模拟和体验；运河上的虚拟驾驶；全景图漫游体验。基于照片的动画演示WenzhiChen,MingminZhang,ZhigengPan,etal,Animations,Games,andVirtualRealityfortheJing-HangGrandCanal,IEEEComputerGraphicsandApplications,vol.30,no.3,May/June.20104）网上上海世博会获奖:上海市科学技术进步一等奖(2011)完成单位:上海世博局,浙江大学等单位二虚拟现实中的交互虚拟现实/增强现实/混合现实一、立项依据(2)头交互腿部交互表情交互手交互任天堂Wii2006年11月2010年6月微软Kinect2013年2月LeapMotion2013年初MYO臂环MITSpaceTop3D2012年10月体感交互(一)虚拟环境中的人机交互（1）交互的手段身体部件（人体，手[指点]，脚，头，眼球，表情/情感，姿势）耳朵[听觉，一般声音/空间声]，手交互设备三维鼠标，笔，触觉设备（2）交互的种类*二维/三维人机交互*可触摸式/非可触摸式交互*自然/非自然交互虚拟生物实验室虚拟生物实验室（我们的例子==》）基于标记的交互我们的实例系统*虚拟保龄球游艺机*虚拟乒乓球球*虚拟网球系统*虚拟网络马拉松ZhigengPan,WeiweiXu,JinHuang,MingminZhang,JiaoyingShi:Easybowling:asmallbowlingmachinebasedonvirtualsimulation.Computers&Graphics27(2):231-238(2003)JianfengLiu,ZhigengPan,XiangchengLi:Anaccelerometer-basedgesturerecognitionalgorithmanditsapplicationfor3Dinteraction.Comput.Sci.Inf.Syst.7(1):177-188(2010)ZhiquanFeng,MinmingZhang,ZhigengPan,BoYang,TaoXu,HaokuiTang,YiLi:3D-freehand-poseinitializationbasedonoperator'scognitivebehavioralmodels.TheVisualComputer26(6-8):607-617(2010)实例系统*虚拟网络马拉松[浙江大学+南京大学+国家体科所]MingliangXu,ZhigengPan,HongxingLu,YangdongYe,PeiLv,AbdennourRhalibi.Moving-TargetPursuitAlgorithmUsingImprovedTrackingStrategy.IEEETrans.ComputIntelligandAIinGames2(1):27-39(国际期刊，SCI)GengdaiLiu,MingliangXu,ZhigengPan,AbdennourRhalibi.Humanmotiongenerationwithmultifactormodels.JournalofVisualizationandComputerAnimation22(4):351-359(国际期刊，SCI)XiCheng,GengdaiLiu,ZhigengPan,BingTang.Fragment-basedresponsivecharactermotionforinteractivegames.TheVisualComputer25(5-7):479-485(国际期刊，SCI)(二)基于手势的自然交互2.1“点-线-面”交互模式2.2实时多线索手势跟踪互动清明上河图演示指尖位置手指方向手掌平面简单手势[环境配置]2.1“点-线-面”交互模式ZhigengPan,YangLi,etal.WeightedfeaturesandColorcuebasedmulti-cuehandtrackingalgorithm,IEEEVR2010:219-222(国际顶级会议)JunhaoShi,MingminZhang,ZhigengPan:Areal-timebimanual3Dinteractionmethodbasedonbare-handtracking.ACMMultimedia2011:1073-1076JiejieZhu,LiangWang,RuigangYang,JamesE.Davis,ZhigengPan:ReliabilityFusionofTime-of-FlightDepthandStereoGeometryforHighQualityDepthMaps.PAMI,33(7):1400-1414(2011)(国际)19Demo交互图例(1)3DWritingARChessRotationcontrolOrientation20•Application：游戏交互图例(2)•Application：Control3DPlane21演示demo22(三)智能人机交互（1）智能人机交互需求*自动识别用户的动作和位置*使用自然的交互方式*推断用户的交互意图（2）如何实现智能性*注册问题:无标记注册*识别人的常用动作*深度检测/位置感知(结合3D相机)*意图分析:对交互者的行为和意图进行理解(基于语义的场景建模)JiejieZhu,MiaoLiao,RuigangYangandZhigengPan.JointDepthandAlphaMatteOptimizationviaFusionofTime-of-FlightsensorandStereo.CVPR.2009ZhigengPan,QinggeJi,MingminZhang,YanfengLi,LinMei,Simulatingpatterndesignandlayoutforgroupcalisthenicsandeurhythmics,SimulationandGaming,2006,37(4):423-437（3）基于AVATAR的交互*虚拟化身（AVATAR）*AVATAR的个性化*AVATAR的表情和姿势*多个AVATAR的交互LingChen,GenCaiChen,HongChen,JackMarch,SteveBenford,ZhigengPan,AnHCImethodtoimprovethehumanperformancereducedbylocal-lagmechanism,InteractingwithComputers,2007,19(2):215-224(国际期刊，SCI)ZhigengPan,AdrianDavid,Cheok,HongweiYang,JiejieZhu,JiaoyingShi,Virtualrealityandmixedrealityforvirtuallearningenvironments,Computers&Graphics,2006,30(1):20-28(国际期刊，SCI)（4）情感交互*语音情感*表情的识别（计算机视觉/红外/其他传感）*行为(姿势)的识别和分析*用户和AVATAR的情感交互