MetricsandBenchmarkingPresentedtoWayneKellyPrincipalConsultantQualityAssurancePracticeApril27,20042of37Contents•IntroductiontoMetrics–WhatisMetrics?–WhyMetrics?–ProjectMetricsProgramObjectives–Metrics-CustomerNeed•MeasuresandMetrics–ImprovementCycle–Measures–DataElements–ToolsforMeasurement–MetricsCalculation•MetricsRoadmap•MetricsAnalysis-Examples3of37Itisnotsufficienttoproducesoftwaredefectmetrics-likehowmanyraised,howmanyoutstanding,fixedbyseverityandurgency.Wemustprovethattheactivitiesofthetestteamareacosteffectiveandefficientuseofstaff.TestMetricsWithoutTears-DavidVaughnQuote4of37IntroductiontoMetrics5of37WhatIsMetrics?•Metrics-Derivedfromtheword“metre”-Quantitativeindicatorofperformanceofas/wprocessorproduct.•Metricsisacannot-do-withoutprojectmanagementtool–Visibilityintoprojectexecution–Analysisofprocessstrengthsandweaknesses–youcannotmanageorimprovewhatyoucannotmeasure“Youcan’tcontrolwhatyoucan’tmeasure”-TomDeMarco6of37•Establishameasurementandcontinousimprovementcultureacrossthecompany.•Improvedqualityofsoftwareacceptedintoproduction•Helpmanagementandemployeesmakewell-informedanddecisions•Alertmanagementaboutriskstoprojectgoals,schedulesandsoftwarequality•ProvidesanobjectiveassessmentofthestateofthesoftwareWhyMetrics?7of37WhyMetrics?Continued•Metricsprovideinputsforfutureestimationsandplanning–Theactualperformanceofthepastinquantitativetermsbecomesastartingpointforfutureestimates–Helpsinsettinggoalsandtargets–Betterresourceplanning•Metricsidentifytheareasforimprovements–Forexample,metricscanidentifythephasewhichintroducesthemaximumnumberoferrors•Metricscanbeusedtoeliminateproblemareasandrootcauses8of37SeniorManagementProjectManagementProjectTeamsBenchmarkingSettingimprovementgoalsPerformanceacrossgroupsHowlargeisproject,ReducedefectsBetterestimatesattasklevelConsistency,predictabilityBetterestimation,projectcontrolEffort,defect,schedule,sizeCostofQuality,Effort&DefectDistribution,Quality,ProductivityEstimationAccuracy,Quality,Productivity,DevelopmentCost,ROIWhyisitMeasuredWhatisMeasuredMeasurementsdrivecontinuousimprovementacrosstheorganization9of37MetricsProgramObjectivesMustbeclearlydefinedandPOSITIVE•Drivenbycriticalbusinessobjectives•Measuresattheenterprise,program,projectandapplicationlevels•Providequantitativeinformationofkeydrivers(i.e.schedule,estimatingaccuracy,defects)thatimprovesdecisionmakingintimetoaffectthebusinessoutcomeandsuccessfulmanagementofaproject10of37CUSTOMERPERSPECTIVEWHATCUSTOMERNEEDSMUSTWESERVE?”LEARNINGPERSPECTIVETOACHIEVEOURGOALS,HOWMUSTOURORGANIZATIONLEARN,INNOVATEANDIMPROVE?”“WHATFINANCIALOBJECTIVESMUSTWEACCOMPLISH?WHATISTHECOSTPerFUNCTIONPOINT?”FINANCIALPERSPECTIVE“TOSATISFYOURCUSTOMERS,INWHICHINTERNALBUSINESSPROCESSESMUSTWEEXCEL?”INTERNALPERSPECTIVEMultipleperspectivesofaMetricsprogram11of37MeasuresandMetrics12of37MeasuresMeasuresarethefundamentalbuildingblocksofameasurementsystemMeasuresDefectsSizeEffortScheduleCost13of37MetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessInternalPerspective14of37MetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipInternalPerspectiveFinancialPerspective15of37ImprovementCycleMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipLearningPerspective16of37Measures-DataElements•Defects–#ofdefects–Defectdescription–Stageinjected&detected–Severity–Status•Effort–Plannedvs.actual–Activities(Task,review,rework)–Resourcegroups–Weekendingdate–PhasewisepersoneffortMeasuresMetricsDefectsSizeEffortScheduleCost•Schedule–Startdate(Planned,Actual)–Enddate(Planned,Actual)–Milestonedate(Planned,Actual)•Size–Programreference–Languageused–SourceLinesofCode(InitialandFinal)–FPoutofSLOC•Cost–PerPhase–PlanVsAct–Group17of37Measures–ToolsUsedtoAutomaticallyCapturetheDataElements•Defects–ExcelTemplate–ClearQuest–DefectLogs•Effort–ExcelTemplate–TimeTracking–MSProject–PlanViewMeasuresMetricsDefectsSizeEffortScheduleCost•Schedule–WBS–ExcelTemplate–MSProject–PlanView•Size–SLOCcountersdependingonthelanguage–LaborUnit•Cost–ProjectBudgets18of37Measures–SizeExampleoftoolsFP=FunctionPointProjectEMISHRModulesSLOC=SourceLinesofCodeSerialNo.ProgramReferenceLanguageFP/SLOCInitialSLOCInitialFPFinalSLOCFinalFPFinalFP-InitialFP1BillpaymentCOBRA2000.01005.05.02PurchaseorderHTML128300023.4400031.37.83PurchaserequisitionJAVA533005.7150028.322.64PurchaseadvanceC++5300.0400075.575.55ClaimsprocessingHTML12812349.6999978.168.56BenefitscalculatorCobol10700.06005.65.67MastermaintenanceHTML1284003.15003.90.88Claims1PL/SQL910.02002.22.29Claims2COBOL1070.04003.73.710Claims3COBOL1070.06786.36.311HTML1280.00.00.012ABAP/4160.00.00.013HTML1280.00.00.014HTML1280.00.00.015Cobol1070.00.00.016Cobol1070.00.00.0PROJECTSIZING19of37M