Abstract An Experiment on Creating Scenario Profil

AnExperimentonCreatingScenarioProfilesforSoftwareChange1AnExperimentonCreatingScenarioProfilesforSoftwareChangePerOlofBengtsson&JanBoschDepartmentofSoftwareEngineeringandComputerScienceUniversityofKarlskorna/RonnebyS-37230Ronneby,[phone][PerOlof.Bengtsson|Jan.Bosch]@ipd.hk-r.seURL:[~pob|~bosch]AbstractScenarioprofilesareusedincreasinglyoftenfortheassessmentofqualityattributesduringthearchitecturaldesignofsoftwaresystems.However,thedefinitionofscenarioprofilesissubjectiveandnodataisavailableontheeffectsofindividualsonscenarioprofiles.Inthispaperwepresentthedesign,analysisandresultsofacontrolledexperimentontheeffectofindividualsonscenarioprofiles,sothatotherscanreplicatetheexperimentsonotherprojectsandpeople.Bothscenarioprofilescreatedbyindividualsandbygroupsarestudied.Thefindingsfromtheexperimentshowedthatgroupswithpreparedmembersprovedtobethebestmethodforcreatingscenarioprofiles.Unpreparedgroupsdidnotperformbetterthanindividualswhencreatingscenarioprofiles.Keywords:Experimentstudies,empiricalstudies,scenarios,scenarioprofile,softwarearchitecture,softwarequalityfactors,prediction1IntroductionDuringrecentyears,theimportanceofexplicitdesignofthearchitectureofsoftwaresystemsisrecognized[2,5,8,13].Thesoftwarearchitectureconstrainsthequalityattributesandthearchitectureshouldsupportthequalityattributessignificantforthesystem.Thisisimportantsincechangingthearchitectureofasystemafterithasbeendevelopedisgenerallyprohibitivelyexpensive,potentiallyresultinginasystemthatprovidesthecorrectfunctionality,buthasunacceptableperformanceorisveryhardtomaintain.Architectureassessmentisimportanttodecidethelevelatwhichthesoftwarearchitecturesupportsvariousqualityattributes.Theneedforevaluationandassessmentmethodshavebeenindicatedby[1,2,9,10,11].Architectureassessmentisnotjustimportanttothesoftwarearchitect,butisrelevantforallstakeholders,includingtheusers,thecustomer,projectmanagement,externalcertificationinstitutes,etc.AnExperimentonCreatingScenarioProfilesforSoftwareChange2Onecanidentifythreecategoriesofarchitectureassessmenttechniques,i.e.scenario-based,simulationandstaticmodel-basedassessment.However,thesetechniquesallmakeuseofscenarioprofiles,i.e.asetofscenarios.Forassessingmaintainability,forexample,amaintenanceprofileisused,containingasetofchangescenarios.Althoughsomescenariosprofilescanbedefinedas‘complete’,i.e.coveringallscenariosthatcanpossiblyoccur,mostscenarioprofilesare‘selected’.Selectedscenarioprofilescontainarepresentativesubsetofthepopulationofallpossiblescenarios.Tousetheaforementionedmaintenanceprofileasanexample,itis,formostsystems,impossibletodefineallpossiblechangescenarios,whichrequiresonetodefineaselectionthatshouldrepresentthecompletepopulationofchangescenarios.Scenarioprofilesaregenerallydefinedbythesoftwarearchitectaspartofarchitectureassessment.However,definingaselectedscenarioprofileissubjectiveandwehavenomeanstocompletelyverfiytherepresentativenessoftheprofile.Also,tothebestofourknowledge,nostudieshavebeenreportedabouttheeffectsofindividualsonthecreationofscenarioprofiles,i.e.whatisthedeviationbetweenprofilescreatedbydifferentindividuals.Thesameisthecaseforgroupsdefiningscenarioprofiles.Above,thegeneraljustificationoftheworkreportedinthispaperispresented.Asecondreasonforconductingthisstudyisthatin[5]weproposedamethodfor(re)engineeringsoftwarearchitecturesandarchitectureassessmentisakeyactivity.Aspartofthatmethod,wehavedevelopedatechniqueforscenario-basedassessmentofmaintainability[6].Animportantpartofthetechniqueisthedefinitionofamaintenancescenarioprofile.Sincetheaccuracyoftheassessmenttechniqueislargelydependentontherepresentativenessofthescenarioprofile,weconductedanexperimenttodeterminewhattheeffectofindividualsandgroupsisonthedefinitionofscenarioprofiles.Therefore,wewillprimarilyusemaintenancescenarioprofilesasexamples,eventhroughtheresultsarevalidforotherqualityattributesaswell.Theintentionoftheexperimentisthreefold:1.testingthreedifferentmethodsofsynthesizingscenarioprofiles.2.testthehypothesisthatthereisadifferencebetweenthemethods.3.findoutwhichoneofthethreemethodsthatarethebest.Toconducttheexperiment,weusedvolunteeringstudentsfromtheSoftwareEngineeringstudyprogramattheUniversityofKarlskrona/Ronneby,allcurrentlyontheirMastersyear.Theremainderofthepaperisorganizedasfollows.Inthenextsectionwedescribetheconceptofscenarioprofilesinmoredetail.Thedesignoftheexperimentispresentedinsection3,followedbytheanalysisandinterpretationoftheresultsinsection4.Relatedworkisdiscussedinsection5andthepaperisconcludedinsection6.AnExperimentonCreatingScenarioProfilesforSoftwareChange32ScenarioProfilesScenarioprofilesdescribethesemanticsofsoftwarequalityfactors,e.g.maintainabilityorsafety,foraparticularsystem.Thedescriptionisdoneinthetermsofasetofscenarios.Scenariosmaybeassignedanassociatedweightorprobabilityofoccurrencewithininacertaintime,butwedonotaddressthatinthispaper.Todescribe,forexample,themaintainabilityrequirementforasystem,welistanumberofscenariosthateachdescribeapossibleand,pre