PAPI 3081 on Blue Gene L

PAPI3.0.8.1onBlueGeneLUsingnetworkperformancecounterstolayouttasksforimprovedperformancePresentationoverviewProjectobjectivesPAPIexplanationBlueGeneLexplanationCurrentstateofresearchProjectobjectivesUpgradePAPIonBG/LProvideinterfacefornetworkcountersAllowLawrenceLivermoreNationalLabuserstoalsohaveaccesstoPAPIUsingnetworkcounterstoplacetasksoptimallyonBG/LPAPI–IntroCourtesyof–IntroPAPIusefultoprofileyourownprograms.ManytoolsbasedonPAPIPapiEx–CommandlinemeasurementtoolPerfSuite–AggregatemeasurementandstatisticalprofilingpackageandAPIHPCToolkit–StatisticalprofilingpackageManymore!PAPI–SupportedplatformsIBM–POWER3,604,604e,POWER4CrayT3E,CrayX1AMD–Athlon,OpteronIntel–P1toP4,ItaniumIandIIUltraSparcI,II&IIIMIPSR10K,R12K,R14KAlphaPAPI–GenericInterfaceCallsequenceforgenericinterfacePAPI_library_init–InitializememoryforPAPI’sdatastructuresPAPI_create_eventset–CreateanemptylistofeventsPAPI_add_event–AddeventstobecountedPAPI_start–BegincountingalleventswithinthespecifiedeventsetPAPI_stop–StopallcountersandreadtheircurrentvaluesPAPI–Events:PresetsPresets–listofpredefinedeventsimplementedonallsystemswheretheycanbesupportedNotallpresetsavailableoneveryarchitecture(e.g.BG/LhasnocachelowerthanL3–thusL1cachehitpresetnotapplicable)NativeeventsformthebasicbuildingblocksforPAPIpresetsPAPI–Events:PresetsCourtesyof–Events:NativeInadditiontothepredefinedPAPIpresetevents,thePAPIlibraryalsoexposesamajorityoftheeventsnativetoeachplatformCanbeaddedtoeventsetsinthesamemanneraspresetsPAPI–Events:NativePAPI–InternalsArrayofeventsetsisthemainportionPAPI–OtherfeaturesMultiplexing–IftherearenotenoughhardwarecountersThreadsafe–ProfilingisthreadsafeOverflowdetection–HardwarecountershavelimitedspacePAPI–PAPI2vsPAPI3PAPI3significantlyreducedoverheadsforstarting,stoppingandreadingthecountersCourtesyof–PAPI2vsPAPI3BetternativeeventsupportinPAPI3BetterthreadsupportinPAPI3OverflowandProfilingenhancementsinPAPI3MyriadbugfixesandcodecleanupinPAPI3PAPI–PAPI2vsPAPI3OverlappingeventsetssupportedinPAPI2MinorchangesintheAPI–mostlydereferencingvariablesBlueGeneL–Intro65,536nodesconnectedin64x32x323DtorusNodesmadeupofPowerPC440embeddedprocessorsSmallerthanmostsupercomputersConsumeslesspowerBlueGeneLBlueGeneL-Networks3Dtorusnetwork(nodetonode)Treenetwork(broadcasts)BlueGeneL–HWcounters48universalperformancecounters4floatingpointunitcountersCounters32bit–mustusevirtualcounterstopreventoverflowBlueGeneL–HWcountersResearch–OverallgoalsNetworkhardwarecountersnewUsenetworkcounterstodeterminetrafficbetweentasksTrytooptimizeplacementoftaskstominimizecommunicationlatencyGivencountsanddistances:cost=counts*distance.MinimizeoverallnodesResearch–CountingFirstgoaltodeterminewhatisbeingcountedResearch–NetworksForeachMPIcall–determinewhichnetworkcountersarebeingusedTreeissupposedtobeforbroadcastsTorusissupposedtobeforpointtopointcommunicationAmbiguitiesinthespecificationResearch–FuturedecisionsHowtoprofileatargetapplicationManuallyinsertPAPIinstrumentation:alotofworkInstrumentbinarieswithcountingcodeWhatinformationtostoreAllcountsoneachnode:alotofdataSampleofallnodes:notasaccurate(whatifthetasksbehave/communicatedifferently?Research–FuturedecisionsHowtousecollectedinformationProfileanapplicationtoobtaincounterfeedbacktodetermineoptimizedstatictasklayoutDynamicallymigratetasksinresponsetocounters