+VMware大数据解决方案2014-架构

©2014VMwareInc.Allrightsreserved.VMwareBigDataSIGReferenceArchitectureChoicesforVirtualizingHadooponvSphereVMwareBigDataProductTeamMarch27,2014Agenda•Introductions•HadooponVMwareUpdate•ArchitectureConsiderationsforHadooponVMware•Q&ACONFIDENTIAL2HadooponVMwareUpdate•vSphereBigDataExtensionsv1.1–ReleasedJan2014–Multi-networksupport–User-definedpasswords–Elasticityenhancements(min/max,scheduledelasticity)–Upgradecapability•Partnernews–SupportforInteldistributioninBDE1.1–CertifiedwithHortonworksHDP2.0–HadoopStarterKitusingIsilon•Visitusat–EMCWorldinLasVegas–HadoopSummitinSanJoseCONFIDENTIAL3©2014VMwareInc.Allrightsreserved.ArchitectureConsiderationsforHadooponVMwareJustinMurray,TechnicalMarketingManager,VMwareAgendaCONFIDENTIAL5•AQuickLookatvSphereBigDataExtensions•ExampleArchitectures•VirtualizingHadoop:theStandardModel•TheCompute–DataSeparationModel•ALookatStorageArchitectures•Configuration-Summary•QuestionsandAnswersvSphereBigDataExtensionsReferenceArchitecture:32-ServerPerformanceTestCONFIDENTIAL7UptofourVMsperservervCPUsperVMfitwithinsocketsize(e.g.4VMsx4vCPUs,2X8)MemoryperVM-fitwithinNUMAnodesizeNativevs.Virtual,32hosts,16disksperhostSource:(usingTeraSortasanexample)75%ofDiskBandwidthJobMapTaskMapTaskMapTaskMapTaskReduceReduceHDFSDFSInputDataDFSOutputData12%ofBandwidth12%ofBandwidthSpills&Logsspill*.outSpillsMapOutputfile.outShuffleMap_*.outSortCombineIntermediate.outLargerArchitecturewithDataComputeSeparatedCONFIDENTIAL10VirtualizedHadoop–ChoicesforPlacement•C=computenode(TaskTracker)•D=DataNodePhysicalHostsVirtualMachineCombinedModel:aStandardDeploymentVirtualizationHostVMDKOSImage–VMDKHadoopVirtualNode1DataNodeExt4TaskTrackerExt4Ext4Ext4SharedstorageSAN/NASLocaldisksOSImage–VMDKVMDKVMDKVMDKVMDKVMDKVMDKVMDKHadoopVirtualNode2DatanodeExt4TaskTrackerExt4Ext4Ext4TheData-ComputeSeparationDeploymentModelVirtualizationHostOSImage–VMDKHadoopVirtualNode1TaskTrackerSharedstorageSAN/NASLocaldisksOSImage–VMDKVMDKVMDKVMDKVMDKVMDKVMDKVMDKHadoopVirtualNode2DataNodeExt4Ext4Ext4Ext4Ext4Ext4Ext4Ext4Ext4Ext4Ext4VMDKVMDKVMDKVMDKVMDKVMDKVMDKVMDKVMDK…DataPaths:CombinedvsData-ComputeSeparationVirtualizationHostVirtualizationHostHadoopVirtualNode1HadoopVirtualNode2TaskTrackerVirtualSwitchDataNodeHadoopVirtualNodeVirtualSwitchDataNodeTaskTrackerVMDKIsolationModelCompute-onlyClustersWithIsilonSharedstorageSAN/NASHadoopVirtualNode2NNNNNNNNNNNNdatanodeIsilonVirtualizationHostVMDKOSImage–VMDKOSImage–VMDKVMDKVMDKHadoopVirtualNode1Ext4JobTrackerExt4TempOSImage–VMDKExt4TaskTrackerExt4HadoopVirtualNode3Ext4TaskTrackerExt4HybridStorageModel-BestofBothWorlds–Masternodes:•NameNode,JobTracker,ZooKeeperetc.onsharedstorage•LeveragevSphereHAandFT–Workernodes•TaskTracker/DataNodemakeuseoflocalstorage•Lowercost,scalablebandwidth•TaskTrackerTempdataiswrittentolocalstorageforbestperformance(alternatesmaybeontoSSDorNFSstorage)LocalStorageSharedStorageWorkerDAS/SSD/NFSforTemp/ShuffleDataProvisionthevirtualmachinesattherightsize•Reserve6%ofphysicalmemoryontheESXiServerforvSphereusage•Avoidover-commitmentofmemoryintheGuestOSandatthehostlevel•ThevirtualmachinememorysizeandvCPUcountfitwithintheNUMAnode(andsocketsize/logicalprocessorcount)UseLargeMemoryPagesattheJVMandguestOSlevel(vSpheredoesthisathostlevelbydefault,exceptwhenphysicalmemoryislow)Sizethehardwaretocontaintheappropriatenumberofthevirtualmachinesabove.Hadoopisaworkloadthatvirtualizeswell.Thereareplentyofgoodreasonstodothis.vSphereGeneralGuidelines-Summary•VMwarevSphereBDEwebsite://•VirtualizedHadoopPerformancewithVMwarevSphere5.1•ABenchmarkingCaseStudyofVirtualizedHadoopPerformanceonvSphere5•ScalingtheDeploymentofMultipleHadoopWorkloadsonaVirtualizedInfrastructure(Intel-Dell-VMware)•ApacheHadoopHighAvailabilitySolutiononVMwarevSphere5.1•HadoopVirtualizationExtensions(HVE):@vmware.comKevinLeong,[e-mail],[e-mail]HadoopworkloadsworkverywellonVMwarevSphere•Variousperformancestudieshaveshownthatanydifferencebetweenvirtualizedperformanceandnativeperformanceisminimal•FollowthegeneralbestpracticeguidelinesthatVMwarehaspublishedvSphereBigDataExtensionsenhancesyourHadoopexperienceontheVMwarevirtualizationplatform•RapidprovisioningtoolfordeploymentofHadoopcomponentsinvirtualmachines•AlgorithmsforbestlayoutofyourHadoopdataandclustercomponentsarebuiltintotheBDEVHMandHVEcomponents•Designpatternssuchasdata-computeseparationcanbeusedtoprovideelastic