搜索
AnIntroductiontoHighPerformanceComputingonAWSScalable,Cost-EffectiveSolutionsforEngineering,Business,andScienceAugust2015AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page2of22©2015,AmazonWebServices,Inc.oritsaffiliates.Allrightsreserved.NoticesThisdocumentisprovidedforinformationalpurposesonly.ItrepresentsAWS’scurrentproductofferingsandpracticesasofthedateofissueofthisdocument,whicharesubjecttochangewithoutnotice.CustomersareresponsibleformakingtheirownindependentassessmentoftheinformationinthisdocumentandanyuseofAWS’sproductsorservices,eachofwhichisprovided“asis”withoutwarrantyofanykind,whetherexpressorimplied.Thisdocumentdoesnotcreateanywarranties,representations,contractualcommitments,conditionsorassurancesfromAWS,itsaffiliates,suppliersorlicensors.TheresponsibilitiesandliabilitiesofAWStoitscustomersarecontrolledbyAWSagreements,andthisdocumentisnotpartof,nordoesitmodify,anyagreementbetweenAWSanditscustomers.AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page3of22ContentsAbstract4 Introduction4 WhatIsHPC?5 GridsandClusters7 AWideSpectrumofHPCApplicationsintheCloud8 MappingHPCApplicationstoAWSFeatures10 LooselyCoupledGridComputing10 TightlyCoupledHPC10 Data-IntensiveComputing11 FactorsthatMakeAWSCompellingforHPC12 ScalabilityandAgility12 GlobalCollaborationandRemoteVisualization13 ReducingorEliminatingRelianceonJobQueues13 FasterProcurementandProvisioning14 SampleArchitectures15 GridComputingintheCloud15 ClusterComputingintheCloud16 RunningCommercialHPCApplicationsonAWS17 SecurityandGovernanceforHPC17 World-ClassProtection18 Built-InSecurityFeatures18 Conclusion20 Contributors20 FurtherReading21 Notes22 AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page4of22AbstractThispaperdescribesarangeofhighperformancecomputing(HPC)applicationsthatarerunningtodayonAmazonWebServices(AWS).Youwilllearnbestpracticesforclouddeployment,forclusterandjobmanagement,andforthemanagementofthird-partysoftware.ThiswhitepapercoversHPCusecasesthatincludehighlydistributed,highlyparallelgridcomputingapplications,aswellasmoretraditionalclustercomputingapplicationsthatrequireahighlevelofnode-to-nodecommunications.WealsodiscussHPCapplicationsthatrequireaccesstovarioustypesofhighperformancedatastorage.Thiswhitepapercoverscostoptimization.Inparticular,wedescribehowyoucanleverageAmazonElasticComputeCloud(EC2)SpotInstances1andstorageoptionssuchasAmazonSimpleStorageService(S3),AmazonElasticBlockStore(EBS),andAmazonGlacierforincreasedperformanceandsignificantcostsavingswhenmanaginglarge,scalableHPCworkloads.IntroductionAmazonWebServices(AWS)provideson-demandscalabilityandelasticityforawidevarietyofcomputationalanddata-intensiveworkloads,includingworkloadsthatrepresentmanyoftheworld’smostchallengingcomputingproblems:engineeringsimulations,financialriskanalyses,moleculardynamics,weatherprediction,andmanymore.UsingtheAWSCloudforhighperformancecomputingenablespublicandprivateorganizationstomakenewdiscoveries,createmorereliableandefficientproducts,andgainnewinsightsinanincreasinglydata-intensiveworld.OrganizationsofallsizesuseAWS.GlobalenterprisesuseAWStohelpmanageandscaletheirproductdevelopmentandmanufacturingefforts,toevaluatefinancialrisks,andtodevelopnewbusinessinsights.ResearchandacademicinstitutionsuseAWStoruncalculationsandsimulationsatscalesthatwerepreviouslyimpractical,acceleratingnewdiscoveries.InnovativestartupsuseAWStodeploytraditionalHPCapplicationsinnewandinnovativeways,especiallythoseapplicationsfoundinscienceandengineering.AWSalsoAmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page5of22providesuniquebenefitsforentirelynewcategoriesofapplicationsthattakeadvantageofthevirtuallylimitlessscalabilitythatcloudhastooffer.UsingAWS,youcanfocusondesign,simulation,anddiscovery,insteadofspendingtimebuildingandmaintainingcomplexITinfrastructures.AWSprovidesarangeofservices:fromvirtualserversandstoragethatyoucanaccesson-demand,tohigherlevelcomputinganddataservicessuchasmanageddatabasesandsoftwaredevelopmentanddeploymenttools.AWSalsoprovidesservicesthatenableclusterautomation,monitoring,andgovernance.WhatIsHPC?OnewaytothinkofHPCistocompareHPCrequirementstorequirementsforatypicalserver.HPCapplicationsrequiremoreprocessorcores–perhapsvastlymore–thanthecoresavailableinatypicalsingleserver,andHPCapplicationsalsorequirelargeramountsofmemoryorhigherstorageI/Othanisfoundinatypicalserver.MostHPCapplicationstodayneedparallelprocessing—eitherbydeployinggridsorclustersofstandardserversandcentralprocessingunits(CPUs)inascale-outmanner,orbycreatingspecializedserversandsystemswithunusuallyhighnumbersofcores,largeamountsoftotalmemory,orhighthroughputnetworkconnectivitybetweentheservers,andfromserverstohigh-performancestorage.Thesesystemsmightalsoincludenon-traditionalcomputeprocessing,forexampleusinggraphicalprocessingunits(GPUs)orotheracceleratorsattachedtotheservers.ThesespecializedHPCsystems,whendeployedatlargescale,aresometimesrefe