High-Performance File IO in Java Existing Approach

High-PerformanceFileI/OinJava:ExistingApproachesandBulkI/OExtensionsDanBonacheaPhillipDickensyRajeevThakurzAbstractThereisagrowinginterestinusingJavaasthelanguagefordevelopinghigh-performancecomputingapplications.Tobesuccessfulinthehigh-performancecomputingdomain,however,Javamustnotonlybeabletoprovidehighcomputationalperformance,butalsohigh-performanceI/O.Inthispaper,werstexamineseveralapproachesthatattempttoprovidehigh-performanceI/OinJava|manyofwhicharenotobviousatrstglance|andevaluatetheirperformanceontwoparallelmachines,theIBMSPandtheSGIOrigin2000.WethenproposeextensionstotheJavaI/OlibrarythataddressthedecienciesintheJavaI/OAPIandimproveperformancedramatically.Theextensionsaddbulk(array)I/OoperationstoJava,therebyremovingmuchoftheoverheadcurrentlyassociatedwitharrayI/OinJava.Wehaveimplementedtheextensionsintwoways:inastandardJVMusingtheJavaNativeInterface(JNI)andinahigh-performanceparalleldialectofJavacalledTitanium.Wedescribethetwoimplementationsandpresentperformanceresultsthatdemonstratethebenetsoftheproposedextensions.1IntroductionThereisagrowinginterestinusingJavaforhigh-performancecomputingbecauseofthemanyadvantagesthatJavaoersasaprogramminglanguage.Tobeusefulasalanguageforhigh-performancecomputing,however,Javamustnotonlyhavegoodsupportforcomputationbutmustalsobeabletoprovidehigh-performanceleI/O,asmanyscienticapplicationshavesignicantI/Orequirements[6,22,34].Inthispaper,weinvestigatetheI/OcapabilitiesofJavaforhigh-performancecomputingandprovidesuggestionsforrelativelysimplechangestotheJavaI/Omodelthatcanimproveperformancesignicantly.Werstexamineseveralapproachesthatattempttoprovidehigh-performanceI/OinJava|manyofwhicharenotobviousatrstglance|andevaluatetheirperformance.Weperformexperimentsontwodierentparallelmachines,adistributed-memorysystem(IBMSP)andashared-memorysystem(SGIOrigin2000),bothofwhichemploymodernparallel/high-performancelesystems.WethenproposeextensionstotheJavaI/OlibrarythataddressthedecienciesintheJavaI/OAPIandimproveperformancedramatically.Theextensionsaddbulk(array)I/OoperationstoJava,therebyremovingmuchoftheoverheadcurrentlyassociatedwitharrayI/OinJava.Wehaveimplementedtheextensionsintwoways:inastandardJVMusingtheJavaNativeInterface(JNI)[24]andinahigh-performanceparalleldialectofJavadevelopedatU.C.BerkeleycalledTitanium[35,38].Wedescribethetwoimplementationsandpresentperformanceresultsthatdemonstratethebenetsoftheseextensions.EECSDepartment,UniversityofCaliforniaatBerkeley,Berkeley,[e-mail],IllinoisInstituteofTechnology,10West31stStreet,Chicago,[e-mail]erScienceDivision,ArgonneNationalLaboratory,Argonne,[e-mail]11.1I/OinHigh-PerformanceComputingManycomputationallyintensivescienticapplicationsalsoneedtoaccesslargeamountsofdata,andI/Oisoftenthebottleneckinsuchapplications[6,22,34].AcommonI/Orequirementisasfollows.Theapplicationhassomelargedatastructures,saymultidimensionalarrays,distributedamongprocessesinsomefashion.Thearraysmustbereadfromorwrittentoalecontainingtheglobalarray.Theprogrammaybeginbyreadinginaninputarrayandmaythenwritearraystolesseveraltimesduringthecourseofthecomputation.Thearraysintheseapplicationsarenotjustbytearrays,butratherconsistofintegers,oroating-pointnumbers,orsomeotherdatatype.Asweshallseeinthispaper,thefactthattheyarenotjustbytearraysisimportantinthecontextofusingJavaforI/Oinsuchapplications.Inaddition,thelesareusuallyrandom-accessles,andprocessesseektodierentlocationsinthelestoread/writedata.Inthispaper,wefocusontheproblemofconcurrentreadingorwritingofdatafrommultipleprocesses/threadstoacommonleinJava.Weassumethatalargeone-dimensionalarrayofintegersisblock-distributedamongprocessesandmustbereadfromorwrittentoacommonlecontainingtheglobalarray.Whilesimple,thisexampleissucienttodemonstratethestrengthsandweaknessesoftheJavaI/Omodelasapplicabletothebasicneedsofhigh-performancecomputingapplications.Ourexperimentsassume(andemploy)arandom-accesslethatisstripedacrossthedisksofaparallellesystem.MuchoftheresearchrelatedtoparallelI/OhasbeenperformedinthecontextofC,andCprovidesexcellentsupportforsuchoperations.Inparticular,Callowsthecastingofanarrayofanytypeintoanarrayofbytes,andmultidimensionalarrayscanbetreatedasone-dimensionalarraysofthesamesize.TheUnixI/Ofunctionssimplytakeapointertoaone-dimensionalarray,thenumberofbytestobereadorwritten,andtheosetintothele,andtheycarryouttherequestasasingleI/Ooperation.ItisalsoquitesimpletoperformparallelreadsandwritesinCwithouttheneedforsynchronization(onlesystemsthatsupportsuchaccess).Inparticular,eachprocesscanseektoanindependent(non-overlapping)regionofasharedrandom-accessleandthenperformitsreadsorwritestodisjointregionsoftheleinparallel.ThereareotheradvantagesofC/UnixbasedI/Oaswell.Oneadvantageisthatlocal(nonportable)hookstoaparallellesystemcanprovideexcellentperformanceenhancementsonsomemachines.Forexample,theODIRECToptionavailableontheXFSlesystemontheSGIOrigin2000allowstheapplica