Perfsuite An Accessible, Open Source Performance A

PresentedatThe6thInternationalConferenceonLinuxClusters:TheHPCRevolution2005(LCI-05).ChapelHill,NC,April2005PerfSuite:AnAccessible,OpenSourcePerformanceAnalysisEnvironmentforLinuxRickKufrinNationalCenterforSuperco[e-mail],design,implementation,andcurrentstatusofanewsetofsoftwaretoolscalledPerfSuitethatistargetedtoperformanceanalysisofuserapplicationsonLinux-basedsystemsisdescribed.Thesetoolsemphasizeeaseofuse/deploymentandportability/reuseinimplementationdetailsaswellasdatarepresentationandformat.AfterayearofpublicbetaavailabilityandproductiondeploymentonLinuxclustersthatrankamongthelargest-scaleinthecountry,PerfSuiteisgainingacceptanceasauser-orientedandflexiblesoftwaretoolsetthatisasvaluableonthedesktopasitisonleading-edgeterascaleclusters.IntroductionIntheprefacetohisbook,TclandtheTkToolkit,JohnOusterhoutdescribeshismotivationsfordevelopingthewidelyusedTclscriptinglanguageasfollows[1]:Tclwasbornoffrustration.Intheearly1980smystudentsandIdevelopedanumberofinteractivetoolsattheUniversityofCaliforniaatBerkeley,mostlyforintegratedcircuitdesign,andwefoundourselvesspendingalotoftimebuildingbadcommandlanguages.Eachtoolneededtohaveacommandlanguageofsomesort,butourmaininterestwasinthetoolratherthanthecommandlanguage.Wespentaslittletimeaspossibleonthecommandlanguageandalwaysendedupwithalanguagethatwasweakandquirky.Furthermore,thecommandlanguageforonetoolwasneverquiterightforthenexttool,soweendedupbuildinganewbadcommandlanguageforeachtool.Thisbecameincreasinglyfrustrating.Inthefallof1987itoccurredtomethatthesolutionwastobuildareusablecommandlanguage.Ifageneral-purposescriptinglanguagecouldbebuiltasaClibrarypackage,thenperhapsitcouldbereusedformanydifferentpurposesinmanydifferentapplications…Inthespringof1988Idecidedtoimplementsuchalanguage,andtheresultwasTcl.AttheNationalCenterforSupercomputingApplications(NCSA),asatmanyotherresearchinstitutionsinboththeacademicandcorporatesectors,asimilarsituationarisesinthecontextofperformanceanalysisofsoftwareapplications.Agreatdealofmoneyisspentonacquiring,installing,andmaintainingstate-of-the-arthigh-performancecomputing(HPC)systemswiththeintentofachievingthegreatestpossibleperformanceformission-criticalapplications.Yetatthesametime,theprocessofsystematicallygathering,analyzing,storing,andsharingdatathatmeasurestherealizedperformanceoftheseapplicationscontinuestooccurlargelyinanadhocfashion,sometimeswithgreatdifficultyinevenacquiringtherelevantdata.Resultsofanalysesarefrequentlydiscardedorforgottenaftergeneratinga“scaling”or“optimization”plotortwoaftersufficientconfidenceisgainedthatperformancehasreachedasatisfactorylevelgiventheavailablehardware,supportinglibraries,andcompilertechnology.ToparaphraseOusterhout,applicationdevelopersandperformanceanalystsendupbuildinganewbadinfrastructureforacquisitionandreuseofapplicationperformancedata.Itisnotunusualtoseeeverythingbuiltfromscratchonanapplication-by-applicationbasis.PerfSuite:AnAccessible,OpenSourcePerformanceAnalysisEnvironmentforLinux2Afurtherunfortunateconsequenceofthishaphazardapproachtoapplicationperformanceanalysisisthatasubstantialproportionofdiscipline-specificcomputationalscientistsoftendonotknowtheactualperformancecharacteristicsoftheirproductionapplicationsindailyuse.Frequently,nothingmorethanbasictiminginformation(CPUorwall-clock)isusedtoassesswhetherornotanapplicationmakesefficientuseofcomputationalresources.Whiletiminginformationrepresentsthe“bottomline”intermsofabsoluteperformance,itofferslittleinsightatthemicroarchitecturallevelintoopportunitiesforperformanceimprovement.Oneoftenhearsofthegrowinggapbetweenpeakandachievedperformanceofreal-worldapplicationsonthehigh-performanceprocessorsoftoday—itmaybeequallytruethatthereisagrowinggapinawarenessoftheunderlyingcausesforperformancedeficienciesthatadverselyaffectoverallproductivity.ThisarticledescribesanefforttohelpaddressthissituationthathasresultedinthedesignandcreationofaninfrastructureforapplicationsoftwareperformanceanalysiscalledPerfSuite.Thisinfrastructurehasbenefitedgreatlyfromtheinnovationofothersaswellasfromadvancesmadeinthedevelopmentofunderlyingstandardsandportable,opensourcesoftware(OSS)foundationsthatcanreducethetimerequiredtoassemblethenecessarycomponentsintoaworkingsystemfromyearstomonths.Inturn,endusersofPerfSuiteinmanycasescanachievewithinminuteswhatmighthavepreviouslytakendays,weeks,ormoretoaccomplish.Fromtheoutset,PerfSuitehasbeendesignedwiththeenduseruppermostinmind:thethesishasbeenthatapplicationdevelopersandusersshouldnothavetobeburdenedwithbecomingexpertsinthedetailsandintricaciesoflow-levellibrariesthatenableaccesstoperformancedata.Ideally,developersshouldbeempoweredtoquicklyandeasilyreceiveaccurateanswerstofundamentalquestionssuchasthese:1.Howwellisanapplicationperforming?2.Howandwheremighteffortsbebestdirectedtoimproveanapplication’sperformance?PerfSuitehasalsobeendesignedwithareal-worldperspectiveonthechallengesofteninvolvedinsoftwaredeploymentonLinux