A framework for the specification of test cases fo

AFrameworkfortheSpecicationofTestCasesforReal-TimeDistributedSystemsThomasWalteraandJensGrabowskibaComputerEngineeringandNetworksLaboratory(TIK),SwissFederalInstituteofTechnologyZurich,Gloriastrasse35,CH-8092Zurich,Switzerland,phone[phone],fax[phone],[e-mail],(ITM),MedicalUniversityofLubeck,RatzeburgerAllee160,D-23538Lubeck,Germany,phone[phone],fax[phone],[e-mail],(CTMF)isawellestablishedstandardwhichdenesandregulatestheconformancetestingprocedureforprotocolimplementations.Conformancetestingismeanttobefunctionalblack-boxtesting.Besidesconceptsandtermi-nology,CTMFstandardizestestingarchitecturesandtheTreeandTabularCombinedNotation(TTCN)asthetestspecicationlanguage.Sincemoreandmoredistributedsystemssuchasmultimedia,safety-criticalandreal-timesystemsrelyonthetimelyavailabilityofinformation,testingofreal-timerequirementsbecomesaseriousissue,too.Unfortunately,testingreal-timeandothernon-functionalrequirements(performanceandreliability)areoutsidethescopeofCTMF.InthispaperwepresentanextensionofCTMFwhichallowstospecifytestcasesfortestingreal-timerequirements.Theextensionincludesagenerictestingarchitectureandanotationfortestspecicationwhichiscalledreal-timeTTCN.Keywords:Softwaretesting,conformancetesting,real-time,real-timedistributedsystems,testingarchitectures,testnotations,real-timedistributedsystemstesting.1.IntroductionInthepast,distributedsystemsprovidedbest-eortcommunicationservices.Best-eortmeansthatanapplicationsendsdatawheneveritwantsto,inanyquantityandwithoutaskingortellingthenetworkrst,andthenetworkdeliversdataifitcanwithnoguaranteeofreliability,delaybound,throughput,jittercontroletc.Thiskindofserviceprovisionis,however,notsucientfornewclassesofreal-timedistributedsystemslikemultimediaandsafety-criticalsystems,e.g.,videoconferencingsystemsorprocesscontrolsystems.Forsuchsystemstoworkcorrectly,quality-of-serviceguaranteesareessential,andcommunicationtaskshavetobeperformedinreal-time.Thegrowingcomplexityofthesesystemsmakestheformalvericationofreal-timepropertiesimpossible(oratleastintractable).Insuchsituations,testingisafeasibleapproachtoget2condencethatadistributedsystembehavescorrectlyinallrespects,includingthereal-timebehaviourevenundervarioussituations.Testingmeansthatexpectedoutcomesasprescribedbythespecicationofthesoftwareprogramarecomparedwithobservedoutcomesasproducedbyrunsofthesoftwareprogram[7,20,37,45].Ifexpectedandobservedoutcomesdier,afaulthasbeendiscovered.Apairofinputsandexpectedoutputsistermedatestcase.Testspecicationistheprocessofidentifyingpairsofinputandexpectedoutput.AtestingmethodologyandframeworkforOSI-compliantdistributedsystemshavebeendevelopedinthepastbyISO(InternationalOrganizationforStandardization)andITU{T(InternationalTelecommunicationUnion|TelecommunicationStandardsSector)whichisknownastheconformancetestingmethodologyandframework(CTMF)[21].InthispaperwedevelopadownwardscompatibleextensionofCTMFfortestingreal-timedistributedsystems.Thecontributionsofthispapertoreal-timedistributedsystemstestingarethreefold:First,classicationsofdistributedsystemsandtypesoftestingarepresented.Theclassicationofdistributedsystemsconsidersthefollowingparameters:distribution(allowstodistinguishbetweenlocalanddistributedsystems),communication(distinguishesbetweensyn-chronousandasynchronouscommunicationaswellasbetweenunicast,multicastandbroadcastcommunication),andreal-time.Inourclassicationoftypesoftesting,weconsiderparam-eterssuchastestingobjectives(testingfunctionalandnon-functionalrequirements),testingapproaches(black-box,grey-boxandwhite-boxtesting),andtestingarchitectures(thecompo-nentsthatmakeupatestsystem).Second,basedontheseclassications,wedenetestingarchitecturessuitablefortestingadiversesetofdistributedsystemswithrespecttodierenttestingobjectives.Ourideaistohaveatoolboxofcomponentsthatcanbecombinedgener-icallyintoatestingarchitecturesuitablefortheapplicationorsystemtobetested.Third,atestnotationisproposedfortestingreal-timeaspectsofdistributedsystems.Essentially,ourapproachallowsthetimingofactionsrelativetotheoccurrenceofpreviousactions.Thepaperisstructuredasfollows:Section2givesaclassicationofdistributedsystemsanddistributedtesting.Weusethisclassicationfortheevaluationofexistingtestingframeworksandforidentifyingwhereourframeworktsin.Section3presentsthedetailsofourproposedtestingarchitectureandtestspecicationlanguage.Section4presentsanapplicationofourtestingmethodology.Theapplicationisinquality-of-servicetestingofavideoconferencingsystem.Beforesummarizingthemainissuesofthepaper,werelateourworktoresearchworkdocumentedintheliterature(Section5).2.Real-timedistributedsystemsandreal-timedistributedtestingReal-timedistributedsystemscoverawiderangeofsystemsfocusingonsolvingdierentcompu-tationalproblems.Intheliterature,denitionsofembeddedsystems,reactivesystems,real-timesystems,orhybridsys