Peer-to-peer systems

Resourcecontainers:AnewfacilityforresourcemanagementinserversystemsGauravBangaPeterDruschelJeffreyC.MogulDept.ofComputerScienceWesternResearchLaboratoryRiceUniversityCompaqComputerCorporationHouston,TX77005PaloAlto,CA94301gaurav,[e-mail]@pa.dec.comAbstractGeneral-purposeoperatingsystemsprovideinade-quatesupportforresourcemanagementinlarge-scaleservers.Applicationslacksufficientcontroloverschedulingandmanagementofmachineresources,whichmakesitdifficulttoenforceprioritypolicies,andtoproviderobustandcontrolledservice.Thereisafun-damentalmismatchbetweentheoriginaldesignassump-tionsunderlyingtheresourcemanagementmechanismsofcurrentgeneral-purposeoperatingsystems,andthebehaviorofmodernserverapplications.Inparticular,theoperatingsystem’snotionsofprotectiondomainandre-sourceprincipalcoincideintheprocessabstraction.Thiscoincidencepreventsaprocessthatmanageslargenum-bersofnetworkconnections,forexample,fromproperlyallocatingsystemresourcesamongthoseconnections.Weproposeandevaluateanewoperatingsystemab-stractioncalledaresourcecontainer,whichseparatesthenotionofaprotectiondomainfromthatofaresourceprincipal.Resourcecontainersenablefine-grainedre-sourcemanagementinserversystemsandallowthede-velopmentofrobustservers,withsimpleandfirmcontroloverprioritypolicies.1IntroductionNetworkedservershavebecomeoneofthemostim-portantapplicationsoflargecomputersystems.Formanyusers,theperceivedspeedofcomputingisgovernedbyserverperformance.WeareespeciallyinterestedintheperformanceofWebservers,sincethesemustoftenscaletothousandsormillionsofusers.Operatingsystemsresearchersandsystemvendorshavedevotedmuchattentiontoimprovingtheperfor-manceofWebservers.Improvementsinoperatingsys-temperformancehavecomefromreducingdatamove-mentcosts[2,35,43],developingbetterkernelalgo-rithmsforprotocolcontrolblock(PCB)lookup[26]andfiledescriptorallocation[6],improvingstabilityunderThispaperwillappearintheProceedingsofthe3rdUSENIXSym-posiumonOperatingSystemsDesignandImplementation(ODSI),NewOrleans,LA,Feb.1999.overload[15,30],andimprovingservercontrolmech-anisms[5,21].Applicationdesignershavealsoat-tackedperformanceproblemsbymakingmoreefficientuseofexistingoperatingsystems.Forexample,whileearlyWebserversusedaprocessperconnection,recentservers[41,49]useasingle-processmodel,whichre-ducescontext-switchingcosts.Whiletheworkcitedabovehasbeenfruitful,ithasgenerallytreatedtheoperatingsystem’sapplicationpro-gramminginterface(API),andthereforeitscoreabstrac-tions,asaconstant.Thishasfrustratedeffortstosolvethornierproblemsofserverscalingandeffectivecon-troloverresourceconsumption.Inparticular,serversmaystillbevulnerableto“denialofservice”attacks,inwhichamaliciousclientmanagestoconsumealloftheserver’sresources.Also,serviceproviderswanttoexertexplicitcontroloverresourceconsumptionpolicies,inordertoprovidedifferentiatedqualityofservice(QoS)toclients[1]ortocontrolresourceusagebyguestserversinaRent-A-Serverhost[45].ExistingAPIsdonotal-lowapplicationstodirectlycontrolresourceconsump-tionthroughoutthehostsystem.Therootofthisproblemisthemodelforresourcemanagementincurrentgeneral-purposeoperatingsys-tems.Inthesesystems,schedulingandresourceman-agementprimitivesdonotextendtotheexecutionofsig-nificantpartsofkernelcode.Anapplicationhasnocon-trolovertheconsumptionofmanysystemresourcesthatthekernelconsumesonbehalfoftheapplication.Theexplicitresourcemanagementmechanismsthatdoexistaretiedtotheassumptionthataprocessiswhatconsti-tutesanindependentactivity.Processesaretheresourceprincipals:thoseentitiesbetweenwhichtheresourcesofthesystemaretobeshared.Modernhigh-performanceservers,however,oftenuseasingleprocesstoperformmanyindependentactivities.Forexample,aWebservermaymanagehundredsorWeusethetermindependentactivitytodenoteaunitofcompu-tationforwhichtheapplicationwishestoperformseparateresourceallocationandaccounting;forexample,theprocessingassociatedwithasingleHTTPrequest.eventhousandsofsimultaneousnetworkconnections,allwithinthesameprocess.Muchoftheresourceconsump-tionassociatedwiththeseconnectionsoccursinkernelmode,makingitimpossiblefortheapplicationtocontrolwhichconnectionsaregivenpriority.Inthispaper,weaddressresourcemanagementinmonolithickernels.Whilemicrokernelsandothernovelsystemsofferinterestingalternativeapproachestothisproblem,monolithickernelsarestillcommerciallysig-nificant,especiallyforInternetserverapplications.Wedescribeanewmodelforfine-grainedresourcemanagementinmonolithickernels.Thismodelisbasedonanewoperatingsystemabstractioncalledaresourcecontainer.Aresourcecontainerencompassesallsystemresourcesthattheserverusestoperformaparticularin-dependentactivity,suchasservicingaparticularclientconnection.Alluserandkernellevelprocessingforanactivityischargedtotheappropriateresourcecontainer,andscheduledatthepriorityofthecontainer.Thismodelallowsfairlyarbitraryinterrelationshipsbetweenprotec-tiondomains,threadsandresourcecontainers,andcanthereforesupportawiderangeofresourcemanagementscenarios.Weevaluateaprototypeimplementationofthismodel,asamodificationofDigitalUNIX,andshow