Google云计算部署介绍_0

CloudComputingPingYehgJune14,2008EvolutionofComputingwiththeNetworkNetworkNetworkComputingNetworkiscomputer(client-server)SeparationofFunctionalitiesClusterComputingSeparationofFunctionalitiesGridComputingUtilityComputingypgClusterandgridimagesarefromFermilabandCERN,respectively.EvolutionofComputingwiththeNetworkNetworkNetworkComputingNetworkiscomputer(client-server)Tightlycoupledcomputingresources:SeparationofFunctionalitiesClusterComputingTightlycoupledcomputingresources:CPU,storage,data,etcUsuallyconnectedwithinaLANManagedasasingleresourceSeparationofFunctionalitiesManagedasasingleresourceCommodity,OpenSourceGridComputingUtilityComputingypgClusterandgridimagesarefromFermilabandCERN,respectively.EvolutionofComputingwiththeNetworkNetworkNetworkComputingNetworkiscomputer(client-server)Tightlycoupledcomputingresources:SeparationofFunctionalitiesClusterComputingTightlycoupledcomputingresources:CPU,storage,data,etcUsuallyconnectedwithinaLANManagedasasingleresourceSeparationofFunctionalitiesManagedasasingleresourceCommodity,OpenSourceGridComputingResourcesharingacrossadministrativedomainsDecentralized,openUtilityComputingDecentralized,openstandards,non-trivialserviceGlobalResourceSharingypgClusterandgridimagesarefromFermilabandCERN,respectively.EvolutionofComputingwiththeNetworkNetworkNetworkComputingNetworkiscomputer(client-server)Tightlycoupledcomputingresources:SeparationofFunctionalitiesClusterComputingTightlycoupledcomputingresources:CPU,storage,data,etcUsuallyconnectedwithinaLANManagedasasingleresourceSeparationofFunctionalitiesManagedasasingleresourceCommodity,OpenSourceGridComputingResourcesharingacrossadministrativedomainsDecentralizedopenUtilityComputingDon'tbuycomputersleasecomputingDecentralized,openstandards,non-trivialserviceGlobalResourceSharingypgDon'tbuycomputers,leasecomputingpowerUpload,run,downloadOwnershipModelpClusterandgridimagesarefromFermilabandCERN,respectively.TheNextStep:CloudComputingServicesanddataareinthecloud,Servicesanddataareinthecloud,accessiblewithanydeviceconnectedtothecloudwithabrowserTheNextStep:CloudComputingServicesanddataareinthecloud,Servicesanddataareinthecloud,accessiblewithanydeviceconnectedtothecloudwithabrowserAkthilifdlAkeytechnicalissuefordevelopers:ScalabilitScalabilityApplicationsontheWebYourCoolestWebApplicationYouruserinternetsplatmap::::::::松下問童子松下問童子言師採藥去言師採藥去只在此山中只在此山中雲深不知處賈島《尋隱者不遇》IaskedthekidunderthepinetreeIaskedthekidunderthepinetree,Wheremightyourmasterbe?Heispickingherbsinthemountain,hesaid,thecloudistoodeeptoknowwherethecloudistoodeeptoknowwhere.JiaDao,Didn'tmeetthemaster,writtenaround800ADHowmanyusersdoyouwanttohave?YourCoolestWebApplicationApplicationTheCloudTheCloudHowmanyusersdoyouwanttohave?ThCldTheCloudYourCoolestWebApplicationGoogleGrowthN'9810000i25tNov.'98:10,000querieson25computersApr.'99:500,000querieson300computersSep.'99:3,000,000querieson2100computersSlbilitttScalabilitymattersCountingthenumbersClient/ServerPersonalComputerOne:ManyOne:OneCountingthenumbersClient/ServerPersonalComputerCloudComputingOne:ManyOne:OneMany:ManyDevelopertransitionDevelopertransitionWhatPowersCloudWhatPowersCloudComputing?CommodityInfrastructureSoftwarepgHardware/foo/barPfilhitchunk...chunk...chunk...chunk...Performance:singlemachinenotinterestingReliability:Distributedstorage:GoogleFileSystem(GFS)oMostreliablehardwarewillstillfail:fault-tolerantsoftwareneededoFault-tolerantsoftwareenablesuseDistributedsemi-structureddatasystem:BigTableofcommoditycomponentsStandardization:usestandardizedmachinessystem:BigTableDistributeddataprocessingtMRdtorunallkindsofapplicationssystem:MapReducegoogle.stanford.edu(circa1997)google.com(1999)“corkbdboardsGoogleDataCenter(circa2000)google.com(newdatacenter2001)google.com(3dayslater)CurrentDesignIhkdi•In-houserackdesign•PC-classmotherboards•Low-endstorageandLowendstorageandnetworkinghardware•Linux•+in-housesoftwareHowtodevelopawebapplicationthatscales?StorageDatabaseServingDataProcessingGoogle'sgsolution/replacementGoogleFileBigTableMapReduceGoogleFileSystemBigTableMapReduceAppEngineHowtodevelopawebapplicationthatscales?StorageDatabaseServingDataProcessingGoogle'shadoop:opensourcegsolution/replacementppimplementationGoogleFileBigTableMapReduceGoogleFileSystemBigTableMapReduceAppEnginePublishedOpenedon2008/5/28papersGoogleFileSystemFileFilenamespacechunk2ef7chunk...•Filesbrokenintochunks(typically64MB)•Chunkstriplicatedacrosschunk...chunk.../foo/bar•C