NoSQL.ppt

NoSQLByPerryHoekstraTechnicalConsultantPerficient,[e-mail]2Whythistopic?Client’sApplicationRoadmap–“Reductionofcycletimeforthedocumentintakeprocess.Currently,itcantakeanywherefromafewdaystoafewweeksfromthetimethedocumentsarereceivedtowhentheyareavailabletotheclient.”NewYorkTimesusedHadoop/MapReducetoconvertpre-1980articlesthatwereTIFFimagestoPDF.3AgendaSomehistoryWhatisNoSQLCAPTheoremWhatislostTypesofNoSQLDataModelFrameworksDemoWrapup4HistoryoftheWorld,Part1RelationalDatabases–mainstayofbusinessWeb-basedapplicationscausedspikes–Especiallytrueforpublic-facinge-CommercesitesDevelopersbegintofrontRDBMSwithmemcacheorintegrateothercachingmechanismswithintheapplication(ie.Ehcache)5ScalingUpIssueswithscalingupwhenthedatasetisjusttoobigRDBMSwerenotdesignedtobedistributedBegantolookatmulti-nodedatabasesolutionsKnownas‘scalingout’or‘horizontalscaling’Differentapproachesinclude:–Master-slave–Sharding6ScalingRDBMS–Master/SlaveMaster-Slave–Allwritesarewrittentothemaster.Allreadsperformedagainstthereplicatedslavedatabases–Criticalreadsmaybeincorrectaswritesmaynothavebeenpropagateddown–Largedatasetscanposeproblemsasmasterneedstoduplicatedatatoslaves7ScalingRDBMS-ShardingPartitionorsharding–Scaleswellforbothreadsandwrites–Nottransparent,applicationneedstobepartition-aware–Cannolongerhaverelationships/joinsacrosspartitions–Lossofreferentialintegrityacrossshards8OtherwaystoscaleRDBMSMulti-MasterreplicationINSERTonly,notUPDATES/DELETESNoJOINs,therebyreducingquerytime–Thisinvolvesde-normalizingdataIn-memorydatabases9WhatisNoSQL?StandsforNotOnlySQLClassofnon-relationaldatastoragesystemsUsuallydonotrequireafixedtableschemanordotheyusetheconceptofjoinsAllNoSQLofferingsrelaxoneormoreoftheACIDproperties(willtalkabouttheCAPtheorem)10WhyNoSQL?Fordatastorage,anRDBMScannotbethebe-all/end-allJustastherearedifferentprogramminglanguages,needtohaveotherdatastoragetoolsinthetoolboxANoSQLsolutionismoreacceptabletoaclientnowthanevenayearago–ThinkaboutproposingaRuby/RailsorGroovy/Grailssolutionnowversusacoupleofyearsago11Howdidwegethere?Explosionofsocialmediasites(Facebook,Twitter)withlargedataneedsRiseofcloud-basedsolutionssuchasAmazonS3(simplestoragesolution)Justasmovingtodynamically-typedlanguages(Ruby/Groovy),ashifttodynamically-typeddatawithfrequentschemachangesOpen-sourcecommunity12DynamoandBigTableThreemajorpapersweretheseedsoftheNoSQLmovement–BigTable(Google)–Dynamo(Amazon)•Gossipprotocol(discoveryanderrordetection)•Distributedkey-valuedatastore•Eventualconsistency–CAPTheorem(discussinasec..)13ThePerfectStormLargedatasets,acceptanceofalternatives,anddynamically-typeddatahascometogetherinaperfectstormNotabacklash/rebellionagainstRDBMSSQLisarichquerylanguagethatcannotberivaledbythecurrentlistofNoSQLofferings14CAPTheoremThreepropertiesofasystem:consistency,availabilityandpartitionsYoucanhaveatmosttwoofthesethreepropertiesforanyshared-datasystemToscaleout,youhavetopartition.Thatleaveseitherconsistencyoravailabilitytochoosefrom–Inalmostallcases,youwouldchooseavailabilityoverconsistency15AvailabilityTraditionally,thoughtofastheserver/processavailablefive9’s(99.999%).However,forlargenodesystem,atalmostanypointintimethere’sagoodchancethatanodeiseitherdownorthereisanetworkdisruptionamongthenodes.–Wantasystemthatisresilientinthefaceofnetworkdisruption16ConsistencyModelAconsistencymodeldeterminesrulesforvisibilityandapparentorderofupdates.Forexample:–RowXisreplicatedonnodesMandN–ClientAwritesrowXtonodeN–Someperiodoftimetelapses.–ClientBreadsrowXfromnodeM–DoesclientBseethewritefromclientA?–Consistencyisacontinuumwithtradeoffs–ForNoSQL,theanswerwouldbe:maybe–CAPTheoremstates:StrictConsistencycan'tbeachievedatthesametimeasavailabilityandpartition-tolerance.17EventualConsistencyWhennoupdatesoccurforalongperiodoftime,eventuallyallupdateswillpropagatethroughthesystemandallthenodeswillbeconsistentForagivenacceptedupdateandagivennode,eventuallyeithertheupdatereachesthenodeorthenodeisremovedfromserviceKnownasBASE(BasicallyAvailable,Softstate,Eventualconsistency),asopposedtoACID18WhatkindsofNoSQLNoSQLsolutionsfallintotwomajorareas:–Key/Valueor‘thebighashtable’.•AmazonS3(Dynamo)•Voldemort•Scalaris–Schema-lesswhichcomesinmultipleflavors,column-based,document-basedorgraph-based.•Cassandra(column-based)•CouchDB(document-based)•Neo4J(graph-based)•HBase(column-based)19Key/ValuePros:–veryfast–veryscalable–simplemodel–abletodistributehorizontallyCons:-manydatastructures(objects)can'tbeeasilymodeledaskeyvaluepairs20Schema-LessPros:-Schema-lessdatamodelisricherthankey/valuepairs-eventualconsistency-manyaredistributed-stillprovideexcellentperformanceandscalabilityCons:-typicallynoACIDtransactionsorjoins21CommonAdvantagesCheap,easytoimplement(opensource)Dataarereplicatedtomultiplenodes(thereforeidenticalandfault-tolerant)andcanbepartitioned–Downnodeseasilyreplaced–NosinglepointoffailureEasytodistributeDon'trequireaschemaCanscaleupanddownRelaxthedataconsistencyrequirement(CAP)22WhatamIgiv