ansys-fluent-nvidiagpu-userguide

DA-07311-001_v01|June2014ApplicationNoteACCELERATINGANSYSFLUENT15.0USINGNVIDIAGPUSAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|iiDOCUMENTCHANGEHISTORYDA-07311-001_v01VersionDateAuthorsDescriptionofChange01June16,2014VS/CCInitialreleaseAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|iiiTABLEOFCONTENTSAcceleratingAnsys®Fluent®UsingNVIDIAGPUs....................................51.Introduction.............................................................................................52.ActivatingtheGPUFeature..........................................................................63.ChangingAmgXConfiguration.......................................................................93.1AmgXVerbosity..................................................................................113.2ChoiceofSelectorAggregateSize............................................................123.3ChoiceofFGMRESMaximumIterations.....................................................133.4Choiceofgmres_n_restartsetting............................................................144.GPUMemoryRequirements.........................................................................155.EvaluatingGPUperformance.......................................................................18AcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|ivLISTOFFIGURESFigure1.FluentLauncherPanelinInteractiveModetoEnableandSpecifyGPUs...................6Figure2.SupportedCPU-GPUHardwareConfiguration...................................................7Figure3.UnsupportedCPU-GPUHardwareConfigurations...............................................8Figure4.AmgXAggregateSizeChoiceanditsEffectonMemoryRequirementsandPerformance.........................................................................................12Figure5.GPUMemoryEvaluationBasedontheExample...............................................16Figure6.No.ofTeslaK40GPUsRequiredBasedontheMemoryEvaluation........................17Figure7.SpeedupsinFluentbasedontheAMGPerformanceandLinearSolverFractions.......18AcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|5ACCELERATINGANSYSFLUENTUSINGNVIDIAGPUS1.INTRODUCTIONANSYS®Fluent®15.0userscannowspeeduptheircomputationalfluiddynamicssimulationsusingNVIDIA’sgeneralpurposegraphicsprocessingunits(GPGPUs)alongsideCPUs.ThepurposeofthisguideistohelpFluentUsersmakeinformeddecisionsabouthowto-ActivatetheGPUfeatureforFluentjobsChooseappropriatelinearsystemsolverconfigurationsettingsforthejobandtheirinfluenceonconvergence(residuals),performance(totaltime)andmemoryrequirementsontheGPUEvaluatememoryrequirementsandnumberofGPUsrequiredforthejobEvaluateGPUperformanceAcceleratingAnsysFluentUsingNVIDIAGPUsAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|62.ACTIVATINGTHEGPUFEATUREWhenrunningANSYSFluent15.0interactively,theParallelSettingstabintheFluentLauncherpanelasshowninFigure1allowsyoutospecifysettingsforrunningANSYSFluentinparallel.ThistabisonlyavailableifyouhaveselectedParallelunderProcessingOptions.Inthispanel,youcanspecifythenumberofCPUprocessesusingthe“Processes”fieldandspecifythenumberofGPUsusingthe“GPGPUsperMachine”field.ItisassumedthatnumberofGPUsonallmachines/nodesisthesame.Figure1.FluentLauncherPanelinInteractiveModetoEnableandSpecifyGPUsAcceleratingAnsysFluentUsingNVIDIAGPUsAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|7ForuserswhoarerunningANSYSFluent15.0inashellonaLinuxsystem,thefollowingcommandinvokesandspecifiesthenumberofGPUs:fluentversion-g-tnprocs-gpgpu=ngpgpus-ijournalfileoutputfilewhereversionmustbereplacedby2d,2ddp,3dor3ddpversionofANSYSFluentyouwanttorunnprocsspecifiesthetotalnumberofCPUprocessorsacrossallmachines/nodesngpgpusspecifiesthenumberofGPUspermachine/nodeavailableinparallelmode.Notethattthenumberofprocessespermachinemustbeequalonallmachinesandngpgpusmustbechosensuchthatthenumberofprocessespermachineisanintegermultipleofngpgpus.Thatis,fornprocssolverprocessesrunningonMmachinesusingngpgpusGPUspermachine,wemusthave:(nprocs)mod(M)=0(nprocs/M)mod(ngpgpus)=0ThesupportedCPU-GPUhardwareconfigurationisdescribedinFigure2.UnsupportedCPU-GPUconfigurationsaredescribedinFigure3.Figure2.SupportedCPU-GPUHardwareConfigurationAcceleratingAnsysFluentUsingNVIDIAGPUsAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|8Figure3.UnsupportedCPU-GPUHardwareConfigurationsAcceleratingAnsysFluentUsingNVIDIAGPUsAcceleratingANSYSFluent15.0UsingNVIDIAGPUsDA-07311-001_v01|93.CHANGINGAMGXCONFIGURATIONInANSYSFluent15.0,theAlgebraicMultigrid(AMG)linearsystemsolverusedontheCPUisdifferentfromthatusedontheGPU.Inthelattercase,theAmgXlibraryisusedtoperformthesolutionoflinearsystems.Itisastate-of-the-artlibrarythatcontainsimplementationofAMGforachievinghighperformanceontheGPUs.ThedefaultconfigurationinFluentisanouterFGMRESpreconditionedbyaninnerAMGsolver.WhenrunningFluent,onecouldoverwritethedefaultAmgXconfigurationsettingsviajournalfilecommandsbyspecifyingthe“rpsetvar”commandwiththeappropriatescopesetting.ThesampleFlue