Android多媒体框架概述

AndroidMultimediaFrameworkOverviewLiLi,SolutionandServiceWindRiver|©2010WindRiver.AllRightsReserved.2AgendaWhatisMultimediainamobiledevice–MPEGstandard–Fileformat–CodecAndroidMultimediaFramework–OpenCORE–OpenMAX–Whatwecandowiththis|©2010WindRiver.AllRightsReserved.3WhatisMultimediainmobiledeviceMultimediaisthemostimportantcomponentinmodernmobiledevicewithmodem–Multimediaframeworkisusedtoprocessvideo/audioinputandoutputtosatisfycertainfunctionalityincludingVideoandAudio–Storagevideoandaudioinmedia–Playbackvideoandaudio–RecordvideoandaudioAudio,Videodevices/inputPerfect!AudioVideo|©2010WindRiver.AllRightsReserved.4Whatistheirrelationship–Containerformatspecifyhowtowrapvariousmeta-data/stream,mostiscodeddatabycodec–Theoretically,acontainerformatcouldwrapanykindsofdata,mostcontainerformatsarespecializedforspecificdatarequirements–Containerdoesnotdescribehowthedatawarpedisencoded.Alwaysaprogrambeabletoidentifyandopenafile,butnotbeabletodecodecontaineddata.Youmaybeweretoldtodownloadrightdecoder.Codec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.5Codec/Decode–deviceorcomputerprogramcapableofencodingand/ordecodingadigitaldatastreamorsignal–Acodecencodesadatastreamorsignalfortransmission,storageorencryptionanddecodeitforplaybackorediting.codec=coder+decoder–Rawmultimediadataishuge,codeccompressthemtofacilitatestoreandtransferCodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.6FileFormat–Alsocallcontainerorwrapperformat–Specifyhowdifferentdataelementsandmetadatacoexistinacomputerfileorstream–Alwayscontaincodedvideo,codedaudio,subtitles,chapter-information,maybeadvertisementandsynchronizationinformationneededtoplaybackvariousstreamstogetherCodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.7Codec-Audio–AAC,MPEG-4AudioPart3subpart4–AC-3,DolbyDigitalcodec–AMR,AdaptiveMulti-RateAudiocodec–AMR-WB,AdaptiveMulti-RateWideband–MP2,MPEG1/2AudioLayerII–MP3,MPEG2AudioLayerIII–Vorbis,OGGaudio,opensourceproject–Wma,WindowsMediaAudio–RealAudio,RealNetwork–ALAC,AppleLosslessAudioCodecCodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.8Codec-Video–MPEG–1,MPEG–1Part2–MPEG–2/H.262,MPEG–2part2–MPEG–4ASP,MPEG–4Part2–MPEG–4AVC/H.264,MPEG–4Part10–VC–1TheinformalnameofSMTP421MvideocodecstandardInitiallydevelopedasaproprietyvideoformatbyMicrosoftbeforeitwasreleasedasaformalSMPTEstandardvideoformatonApril3,2006–VC–2Anopenandroyalty-freevideocompressionformat,Dirac.2010theSMPTEstandardizedDiracProasVC–2.–VC–3DNXHD,DigitalNonlinearExtensibleHighDefinitionalossyhigh-definitionvideopost-productioncodecengineeredformulti-generationcompositingwithreducedstorageandbandwithrequirementsTheDNXHDcodecwassubmittedtotheSMPTEorganizationastheframeworkfortheVC-3familyofstandard.Codec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.9StandardMPEG–MovingPictureExpertsGroup.–AworkinggroupofISO/IECinchargeofthedevelopmentofinternationalstandardsforcompression,decompression,processingandcodedrepresentationofmovingpicturesaudioandtheircombinationThestandardforstorageandretrievalofmovingpicturesandaudioonstoragemedia.ApprovedNov.1992–VCD–MP2,MPEG-1AudioLayerIIThestandardfordigitaltelevision.ApprovedNov.1994–DVD–MP3,MPEG-2AudioLayerIIICodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.10MPEG-4–Thestandardformultimediaapplication–adevelopingstandard–dividedintoanumberofpartsMostarelefttoindividualdeveloperstodecidedwhethertoimplementprobablynocompleteimplementationsoftheentrireMPEG-4setofstandardsTodealwiththis,thestandardincludesconceptof“profiles”and“levels”–allowingaspecificsetofcapabilitiestobedefinedinamannerappropriateforasubsetofapplicationsCodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.11–MPEG-4part2,AdvancedSimpleProfileusedbycodecssuchasDiVX,Xvid,NeroDigitaland3ivx,Quicktime6–MPEG-4part10,MPEG-4AVC(AdvancedVideoCoding)/H.264usedbyx264encoder,NeroDigitalAVC,Quicktime7,andhigh-definitionvideomedialikeBlu-rayDiscCodec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.12Video/Audiocontainer–MPEGprogramstreamstandardcontainerforMPEG-1andMPEG-2elementarystreamsonreasonablyreliablemediasuchasdisks;usedalsoonDVD-Videodiscs–MPEG-2programstream,MPEG-TSStandardcontainerfordigitalbroadcastingandfortransportationoverunreliablemedia;usedalsoonBlu-rayDiscVedio;typicallycontainsmultiplevideoandaudiostream,andanelectronicprogramguide–MP4StandardaudioandvideocontainerfortheMPEG-4multimediaportfolio,basedonMPEG-4Part12andJPEG2000Part12Codec/DecodeandFileFormat|©2010WindRiver.AllRightsReserved.13ContainerformatparsertorecognizeandunwrapfileCodectoencode/decodedata.SynchronizationamongvariousstreamMemory/BuffermanagementStreamtrackcontrol,playback,backwardsplay,forwardplayIntegratedintovideo/audiooutputsystemTakeadvantageofhardwareaccleration–Hardwarecodec–Hardwareoverlay–HardwareaudioflingerWhatneedtodowithMultimedia|©2010WindRiver.AllRightsReserved.14AndroidMultimediaFramework|©2010WindRiver.AllRightsReserved.15OpenCORE–InitialAndroidMultimediaFrameworkAndroidmultimediasubsystempro