Linux-DMA-from-User-Space-public

©Copyright2014Xilinx.LinuxDMAfromUserSpaceJohnLinnBasedonLinuxkernel3.14©Copyright2014Xilinx.PrerequisitesKnowledgeoftheLinuxkernelingeneralsuchasbuildingandconfiguringthekernelCharacterdevicedriverexperienceinLinuxExperiencewiththeCprogramminglanguageThissessionassumesyouhaveattendedprevioustrainingsessions–DeviceDriverFrameworks(TSC2014)–UserSpaceIODrivers(TSC2014)–LinuxDMAinDeviceDrivers(Part1ofDMAseries)©Copyright2014Xilinx.TheprimarycomponentsofDMAincludetheDMAdevicecontrol,memoryallocationandcachecontrolDMAinLinuxisdesignedtobeusedfromkernelspacebyahigherlayerdevicedriverTheDMAEngineinLinuxisaframeworkwhichallowsaccesstoDMAcontrollerdrivers(suchasAXIDMA)inaconsistentandmoreabstractmannerXilinxprovidesdevicedriverswhichplugintotheDMAEngineframework(AXIDMA,AXICDMA,andAXIVDMA)Memorycanbeallocatedusingkmalloc()forcachedmemoryordma_alloc_coherent()foruncachedmemoryDMAcachecontrolfunctionssuchasdma_map_single()anddma_unmap_single()areusedwithcachedmemorybuffersReviewFromLinuxDMAInDeviceDrivers©Copyright2014Xilinx.AchallengeinLinuxisdoingapplicationprocessinginuserspacewhilemovingdatatoandfromdevicesinthePLLinuxprovidesframeworksthatallowuserspacetointerfacewithkernelspaceformosttypesofdevices(exceptDMA)UserSpaceDMAisdefinedastheabilitytoaccessbuffersforDMAtransfersandcontrolDMAtransfersfromauserspaceapplication–Thisisnotanindustrystandardandthereareanumberofpossiblemethods–SimilarmethodshavebeenusedforyearswithdisplaysystemssuchasX11,astheyneededdirectaccesstovideoframebuffersXilinxSDIntegratormightbeaneasiersolutionforsomeapplicationsandshouldbeconsidered–ItusessimilarprincipleswithouttheuserimplementinganycodeIntroduction©Copyright2014Xilinx.AtypicalUserSpaceDMAapplicationcreatesdatawhichneedstobetransferredfromtheCPUmemoryto/fromacustomIPcoreExamples–FFTIPcoreprocessingablockofdata–CustomIPCoregeneratingblocksofdata–SeetheSpectrumAnalyzerTechTipApplicationsofUserSpaceDMAcontrol©Copyright2014Xilinx.ThesoftwaredesignismadeupofakernelspacedevicedriverandauserspaceapplicationTheXilinxAXIDMADeviceDriverandLinuxDMAEngineexistintheLinuxkernelTheDMAProxyDeviceDriverisacharacterdevicedriverthatusestheLinuxDMAEngineTheDMAProxyTestApplicationusestheDMAProxyDeviceDrivertocontrolDMAtransfersUserSpaceDMASoftwareExample(HighLevel)UserSpaceKernelSpace©Copyright2014Xilinx.CreationofacharacterdevicedriverthatextendsthefunctionalityoftheDMAkerneldriverfromtheLinuxDMAinDeviceDriverssessionCreationofauserspaceapplicationthatusesthecharacterdevicedrivertoperformDMAtransfersImplementationofioctl()inthedevicedriverandintheuserspaceapplicationtocausetheDMAEnginetoperformDMAtransfersImplementationofmmap()inthedevicedriverandintheuserspaceapplicationtomapkernelallocatedmemoryintouserspaceprocessaddressspaceTheseprinciplesshouldworkacrossanyDMAdevicethatissupportedbytheLinuxDMAEngineKeyLearningForTheSession©Copyright2014Xilinx.DMAProxySoftwareDetailedDesign©Copyright2014Xilinx.MovingdatabetweenuserspaceandkernelspaceistheprimarymethodforI/OsincetheapplicationisinuserspaceandthedevicedriversareinkernelspaceThecopy_to_user()functioncopiesabufferofbytesfromkernelspacetouserspaceThecopy_from_user()functioncopiesabufferofbytesfromuserspacetokernelspaceFunctionsalsoexistforcopyingasingledatumCopyingDataBetweenKernelandUserSpaceReview©Copyright2014Xilinx.ManysoftwaredesignscopydatafromuserspacetokernelspaceandfromkernelspacetouserspaceForlargerbufferscopyingdataisinefficientandinthecaseofDMAitdefeatsthepurposeofusingDMAtomovethedataAzerocopydesignavoidscopyingmemoryandisrequiredforuserspaceDMAapplicationsSomenetworkstacks(notLinux)provideazerocopydesignandachievehigherperformanceMappingakernelspaceallocatedmemorybufferintouserspaceremovestheneedtocopydataMappinguserspaceallocatedbuffersintokernelspacesothatadrivercanaccessthemisanothermethod–ThisismorecomplexandnotcoveredinthissessionZeroCopyBufferDesign©Copyright2014Xilinx.ThecharacterdeviceframeworkofLinuxprovidesfunctionalitysuchasopen(),read(),write()andclose()whichallowsadevicedrivertobeaccessedusingthefileI/OoperationsfromuserspaceItalsoprovidestheioctl()interfacewhichisusedtocontrolthedeviceinnonstandardwaysThefunctionprototypeinadriver:–int(*ioctl)(structinode*inode,structfile*filp,unsignedintcmd,unsignedlongarg);ThecmdandargargumentsarepassedfromuserspacetothedriverunchangedsuchthattheyareeasilyusedforcontrolTheioctl()functionofthedevicedrivercanperformanyfunctionalityincludingblockinguntilthefunctionalityiscompleteCharacterDeviceFrameworkReview©Copyright2014Xilinx.TheuserspaceapplicationneedstocontrolthekernelspacedrivertoallowDMAtransactionstobemanagedTheread()andwrite()fileoperationscouldeasilybeused–Thesedooffertheabilitytodoasynchronous(non-blocking)I/Ousingpoll()andselect()functionsTheioctl()fileoperationisdesignedfordevicecontrolandisusedtocontroltheDMAProxydevicedriverforsimplicityThemmap()fileoperationallowsmemoryofthedevicedrivertobemappedintotheaddressspaceofthecallerinauserspaceprocessTheUIOdriverframeworkprovidesanotheralternativeforthisdesignwhichissimplerbutlimited