2010GlobalCfConferenceAsiaPacificHongKongMeasurement & evaluation of communicationOldPblNThikiNMdiOld Problem, New Thinking, New MediaProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationThechequeredhistoryofmeasurementThe chequered history of measurement•Early 1990s•IPRA Gold Paper 1994•Late 1990s gathering momentum•Late 20thcentury rhetoric•21stchallenges & opportunities–Entrenched practices–Marketing dominance–Social mediaProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationSituationanalysisSituation analysis•Environment of accountability•MBO, KPIs, KRAs, Balanced scorecards•Public relations poorly measured•Press clippings (‘ink’), activities•Advertising metrics simplistic•Reach, frequency, ‘hits’, ‘eyeballs’, recall•Corporate and organisational communicationSlfti•Self‐reporting surveys•Similar to PRProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationWrongturns,sidetracksandcul‐de‐sacsWrong turns, side tracks and culdesacs•Simplistic, narrow and invalid measuresAdiiVlEil(AVE)–Advertising Value Equivalents (AVEs)•‘Black box’ measurement•Fttlkftttitt•Focus on outputs –lack of attention to outcomesProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationPyramidModelofCommunicationResearchPyramid Model of Communication Research(A)Behaviourprompted/changed(B)Attitudeformed/changed(C)AwarenessincreasedOUTCOMES(A)(B)(C)OUTPUTS-Publicitygained-Eventstaged-Webinfoposted-PublicationINPUTS-Storylist©forpublication-ProgramforaneventINPUTSg-Design&contentsforWebsite-DecisiononmediaoractivityProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationWhattomeasure–inputs/planninglevel?What to measure inputs/planning level?•Stakeholder awareness•Stakeholder understanding•Stakeholder perceptions•Stakeholder interests, concerns •Stakeholder media preferencesInformsstrategyandprovidesbenchmarksProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationWhattomeasure–outputslevel?What to measure outputs level?•Readership / viewership of publications, videos, etc•Event attendance•Web visitorsUnique visits; duration•Media coverageQualitatively as well as quantitativelyProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationWhattomeasure–outcomeslevel?What to measure outcomes level?•AwarenessE.g. your key messages, brand values•UnderstandingE.g. policies, vision, products•Perceptions (incl. top of mind)Reputation; competitor analysis•ResponseInquiries; comments; trialling•EngagementRevisits; joining; recommendations; retentionProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationWhattoMeasure:StepsofEvaluationWhat to Measure: Steps of Evaluation*BehaviouralChangeSales/ProfitsAttitudes/perceptionsReputationRelationshipsBehaviouralChangeAwareness/UnderstandingRetentionAttitudes/perceptionsPositioning(e.g.prominence,profile)ShareofVoiceCompetitorBenchmarkingOtthbliitblititWbithiDMtAudienceReachMessagePlacementOutputssuchaspublicity;publications;events;Websites;sponsorships,DM,etc*Makeaninquiry;buyaproductorservice;etcProfessor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationPyramidModelofCommunicationResearchPyramid Model of Communication Research(A)Behaviourprompted/changed(B)Attitudeformed/changed(C)AwarenessincreasedS(ttdtitti)CommonResearchMethodsOUTCOMES(A)(B)(C)Surveys(structuredquantitative)Interviews(selfadministered)Feedbackforms/minisurveys(e.g.atevents,inpublications)OUTPUTS-Publicitygained-Eventstaged-Webinfoposted-PublicationMediaContentAnalysis(incl.DIYsoftwareprograms)Webanalysisofvisits&infodownloadsINPUTS-Storylist©forpublication-ProgramforaneventResponsemechanisms(1800numbers;competitions,etc)Pre-testing(focusgroups,PDFs)INPUTSg-Design&contentsforWebsite-DecisiononmediaoractivityCasestudiesSecondarydata(existingresearch)Feedback(informal)Professor Jim Macnamara PhD, MA, BA, FPRIA, FAMI, CPM, FAMECProfessor of Public CommunicationPyramidModelofCommunicationResearchPyramid Model of Communication ResearchMeasurementMethodologies:(formal&informal)OUTCOMES(Functional&organisationalevaluation)Numberwho.change..…..………………………………………………..Quantitativesurveys(largescalestructured)behaviour…...……………………………………………..Sales;Votingresults;Adoptionrates;ObservationNumberwho…..changeattitudes……….……………………………………………Focusgroups;Surveys(targeted)(egCustomer,EmployeeShhldStifti)Rttitdi(formal&informal)OUTPUTSNumberwhounderstandmessages..………………………………Focusgroups;Interviews;Complaintdecline;ExperimentsNumberwhoretainmessages…….…………………………………Interviews;Focusgroups;Mini-surveys;ExperimentsNumberwhoconsidermessages……....…..…………………………Responsemechanisms(1800,coupons);InquiriesorShareholderSatisfaction);ReputationstudiesOUT-TAKES(Proposedbysomeasa4thstage)OUTPUTS(Process&programevaluation)Number&typeofme