搜索
Ch6PerformanceAssessment06.10.02.doc106.10.02Chapter6.PerformanceAssessment1ByGretchenB.JordanandElizabethL.Malone2Developingcredibleandappropriatemeasuresofperformanceisamajorchallengefacingpubliclyfundedscienceorganizations.Currentassessmenttechniquesarenotsufficientforcurrentrequirements.Performanceinformationisneededforeffectivemanagementandfordemonstratingtherelevanceandvalueoftheorganization’sworktofundersandstakeholders.Inaddition,publiclyfundedscienceorganizationsintheUnitedStateshavealegislativemandatetodevelopperformancemeasuresandtoreportprogressonthosemeasureseachyearaspartofthebudgetingprocess.Theorganizationalliteratureisclearthatperformancemeasurementinfluencesorganizationalbehavior,forbetterorworse.Consequently,managersofpubliclyfundedsciencehavearesponsibilitytoparticipateinthedevelopmentanduseofbetterperformanceassessmenttechniquesandtousetheresultsoftheseimprovedassessmentstorefinetheirmanagementofthepublicsciencesystem.Inabroadsense,thefunctionofmanagersistomanageorganizationalperformance,thatis,tomanageallaspectsrelatedtothesettingandachievementoforganizationalgoalsandstrategy.Todothis,theyneedtoestablishexpectations,incollaborationwiththeiremployeesandstakeholders,agreeuponwaystomeasureperformancerelativetothoseexpectations,measureperformanceovertime,andusethosemeasurestoprovidefeedbackandtakeaction.Avarietyoftermsareusedtodescribethisprocessanditscomponents.Wewillusethetermperformanceassessment.Publicagenciesthatconductresearchareexaminingtheirassessmentinfrastructuretoensurethatitisadequatetodocumentandreportprogressandsuccess,demonstratetheirdeterminationtoimproveperformance,andprovideaccountability(Feller2000;Rip2000).However,numerouschallengesarebeingencountered.ThisreviewbeginsbyexaminingthedriversanddifficultiesinassessingR&Dperformanceingeneralandinpubliclyfundedscientificresearchinparticular.Itthendiscussesframeworksformeasurementandprovidesbriefdescriptionsofmajormethodsandspecificmeasuresthathavebeenfoundmostpertinenttotheassessmentofperformancebypubliclyfundedscienceorganizations.Twoexamplesareincludedtoillustratebothmeasuresandmethods,andtheuseofperformanceinformation.Thereviewconcludeswithalistofareasforactionandfurtherresearch.TheDriversofPerformanceAssessmentareLeadingtoanIncreasedEmphasisonPredictionThedemandforcomprehensiveperformanceassessmentofpublicresearchandtechnologydevelopment(R&D)programsispartofaworldwidedemandformoreaccountabilityinallpublicprograms.ThemostrecognizablemanifestationofthisdemandintheUnitedStatesistheGovernmentPerformanceandResultsActof1993(GPRA)(U.S.Congress1993),whichrequiresstrategicplans,performanceplans,andannualperformancereports.Whatisdrivingthisdemand?AnOrganizationforEconomicCooperationandDevelopmentstudy(OECD1997)1Relatedchaptersinclude:Strategy;ChangeManagement;andInnovation2ChrisCluettofBattelleMemorialInstituteassistedwiththebenchmarkingdiscussionCh6PerformanceAssessment06.10.02.doc06.10.022concludedthatthreeconcernsarenoticeableinallOECDperformancemanagement,althoughtodifferentdegrees:publicconcernthatgovernments(1)improveperformance,(2)clarifyresponsibilitiesandcontrol,and(3)realizecostsavings.IrwinFeller(2000),evaluatorandadvisortoR&Dagenciesandprogramscitesthreeotherreasonsforthefocusonperformancemanagementinresearchprograms:(1)aneedforaccountabilityintimesofflatbudgets;(2)concernthatpublicfundingisbeingprovidedinareasthatdonotobviouslycontributetonationalwell-being,suchasciviliantechnologies;and(3)erodingpoliticalacceptanceofclaimsbytheresearchcommunity,particularlybyacademics,tobeself-policing.Bothcommontypesofassessment--measurementandevaluation–arepartofthefabricofscienceprograms.Itisthescientificmethodtoreviewandimprove.However,asnotedinthefinalreportoftheEuropeanUnionNetworkonAdvancedScienceandTechnologyPolicyandPlanning(Rip2000):Themajorrationaleforevaluationshasshiftedandevolvedfromadesiretolegitimatepastactionsanddemonstrateaccountabilitytoaneedtoimproveunderstandingandinformfutureactions.Correspondingly,theissueofevaluationhasbroadenedawayfromanarrowfocusonquality,economy,efficiencyandeffectiveness,towardsamoreall-encompassingconcernwithperformanceimprovementandstrategydevelopment(Rip2000).GeorghiouandRoessner(2000)agreethatorganizationsareincreasinglycalledupontoprovidepredictiveassessmentspriortoorduringprogramimplementationtodefineorrefinestrategiesforaccomplishinggoals.Thisrepresentsashiftfromthepastwhenprogramevaluationsfocusedonretrospectiveor“summative”studiesafterprojectswerecompletedtodeterminewhetherornotobjectiveswereaccomplishedandtoclarifywhytheobservedoutcomeshadoccurred.Thesesummativestudieswereusedtodemonstrateeffectivenessandsuccesstofundingbodies.“Formative”evaluations,whichlookattheissuesofdesignandimplementationwhiletheprogramisstillunderway,aremoreorientedtoprovidingmanagerswiththeinformationtheyneedtoimproveon-goingprogramsanddevelopmoreeffectiveplansforthefuture.ThisincreasingemphasisonformativeevaluationsisalsoreflectedintheU.S.NationalScienceandTechnologyCouncilreportonAssessingFundamentalScie