A Critical Guide to Empirical Validation of Agent-

ACriticalGuidetoEmpiricalValidationofAgent-BasedEconomicsModels:Methodologies,Procedures,andOpenProblemsGiorgioFagiolo¤PaulWindrumyAlessioMonetazAbstractThispaperaddressesthemethodologicalproblemsofempiricalvalidationinagent-based(AB)economicsmodelsandhowthesearecurrentlybeingtackled.We¯rstidentifyasetofissuesthatarecommontoallmodellersengagedinempiricalvali-dation.Wethenproposeanoveltaxonomy,whichcapturestherelevantdimensionsalongwhichABeconomicsmodelsdi®er.Wearguethatthesedimensionsa®ectthewayinwhichempiricalvalidationiscarriedoutbyABmodelllersandwecriticallydiscussthemainalternativeapproachestoempiricalvalidationbeingdevelopedinABeconomics.Weconcludefocusingonasetof(asyet)unresolvedissuesforempiricalvalidationthatrequirefutureresearch.Keywords:Methodology,agent-basedcomputationaleconomics,simulationmod-els,empiricalvalidation,calibration,history-friendlymodelling.JELClassi¯cation:B41,B52,C63.¤Correspondingauthor:FacultyofEconomics,UniversityofVerona,Verona,ItalyandLabora-toryofEconomicsandManagement,Sant'AnnaSchoolofAdvancedStudies,Pisa,Italy,e-mail:[e-mail]:Sant'AnnaSchoolofAdvancedStudies,PiazzaMartiridellaLibertµa33,I-56127Pisa,Italy.Tel:[phone].Fax:[phone].yManchesterMetropolitanUniversityBusinessSchool,Manchester,UKandUNU-MERIT,UniversityofMaastricht,TheNetherlands,e-mail:[e-mail]conomics,Jena,Germany,email:[e-mail]11IntroductionTheaimofthispaperistoprovideacriticalguidetoalternativeapproachestoempiricalvalidationdevelopedinagent-based(AB)economicsinrecentyears.Morespeci¯cally,thepaperexploresasetoffundamentalmethodologicalproblemsfacedbyallresearchersengagedinempiricallyvalidatingeconomicsABmodelsanddiscussesalternativesolutionswithinthreedomains:(i)therelationshipbetweentheoryandempiricalresearch,(ii)therelationshipbetweenmodelsandthereal-worldsystemsbeingmodelled,and(iii)thewayinwhichavalidationproceduredealswith(i)and(ii).ThelasttwodecadeshaveseenarapidgrowthinABmodellingineconomics.Anexhaustivesurveyofthisvastliteratureisofcoursebeyondthescopeofthiswork1.However,beforeproceeding,itisusefultointroducethemainteningredientsthattendtocharacteriseeconomicsABmodels.1.Abottom-upperspective.Asatisfactoryaccountofadecentralisedeconomyistobeaddressedusingabottom-upperspectivebecauseaggregatepropertiesaretheoutcomeofmicro-dynamicsinvolvingbasicentities(agents)(Tesfatsion,2002).Thiscontrastswiththetop-downnatureoftraditionalneoclassicalmodels,wherethebottomleveltypicallycomprisesarepresentativeindividualandisconstrainedbystrongconsistencyrequirementsassociatedwithequilibriumandhyper-rationality.2.Heterogeneity.Agentsare(ormightbe)heterogeneousinalmostalltheircharac-teristics.Thesecanrangefrominitialendowmentsandotheragents'properties,allthewaythroughtobehaviouralrules,competencies,rationality,andcomputationalskills.3.Theevolvingcomplexsystem(ECS)approach.Agentsliveincomplexsystemsthatevolvethroughtime(Kirman,1997a,1997b).Therefore,aggregatepropertiesarethoughttoemergeoutofrepeatedinteractionsamongsimpleentities,ratherthanfromtheconsistencyrequirementsofrationalityandequilibriumimposedbythemodeller.Thisfocusoninteractionsgoeshand-in-handwithabottom-upapproachtosystemsmodelling.Higher-leveleconomicphenomenaandstructures,itisar-gued,aretheproductofsubtledi®erenceswithinaheterogeneousagentpopulationandthelocalinteractionsthatoccurbetweenindividualagents.Thehigherlevelphenomenathatemergewilldi®erasaconsequenceoftheinteractionsthatoccurbetweentheindividualmembers,andsubtledi®erencesthatexistwithinthehetero-geneouspopulation.Thishasimportantimplicationsformethodologicalpractice.Onecannot,itisargued,deducemacrobehaviourfromthebehaviourofan`aver-age'or`representative'individual(Kirman,1989,1992;ForniandLippi,1997).Forexample,onecannotunderstandthecollectiverationalityandoperationof¯rmsandconsumersthroughthestudyofanindividual¯rmorconsumer.Equally,oneisunabletoderivecollectiverationalityineconomicphenomena,suchasthefor-mationofstrategicalliances,throughtheconstructionofabstractrepresentative1Onthe(oftensubtle)di®erenceswhichcharacterisedi®erentresearchschoolsthathavebeenem-ployingABmodelstostudymarketandindustrydynamics(e.g.,evolutionaryeconomics,agent-basedcomputationaleconomics,neo-Schumpeterian,andhistory-friendlymodels),theinterestedreaderisre-ferredtoLane(1993a,b),DosiandNelson(1994),Nelson(1995),SilverbergandVerspagen(1995),Tesfatsion(1997,2002),Windrum(2004),Dawid(2006),andPykaandFagiolo(2005).AlsoseeGilbertandTroitzsch(1999),andWooldridgeandJennings(1995)foradiscussionofABtechniquesinothersocialsciences.2agents.Rather,macrophenomenaareaproductoftheinteractionsofheteroge-neous¯rmsandconsumers.Complexdynamicsofthemacrostructurearise,evenifeachindividualfollowsaverysimplebehaviouralrule.Hence,ABmodelshighlighttheheterogeneityof¯rmsandconsumers,therulesthatgoverntheirindividualbehaviour,andtherulesgoverningtheirinteractions(Lane,1993a,1993b).4.Non-linearity.TheinteractionsthatoccurinABmodelsareinherentlynon-linear(Silverbergeta