搜索
TOPTENBigDataTRENDSFOR2017Top10BigDataTrendsfor20172016wasalandmarkyearforbigdatawithmoreorganizationsstoring,processing,andextractingvaluefromdataofallformsandsizes.In2017,systemsthatsupportlargevolumesofbothstructuredandunstructureddatawillcontinuetorise.Themarketwilldemandplatformsthathelpdatacustodiansgovernandsecurebigdatawhileempoweringenduserstoanalyzethatdata.ThesesystemswillmaturetooperatewellinsideofenterpriseITsystemsandstandards.EachyearatTableau,westartaconversationaboutwhat’shappeningintheindustry.Thediscussiondrivesourlistofthetopbig-datatrendsforthefollowingyear.Theseareourpredictionsfor2017.Bigdatabecomesfastandapproachable:OptionsexpandtospeedupHadoopSure,youcanperformmachinelearningandconductsentimentanalysisonHadoop,butthefirstquestionpeopleoftenaskis:HowfastistheinteractiveSQL?SQL,afterall,istheconduittobusinessuserswhowanttouseHadoopdataforfaster,morerepeatableKPIdashboardsaswellasexploratoryanalysis.ThisneedforspeedhasfueledtheadoptionoffasterdatabaseslikeExasolandMemSQL,Hadoop-basedstoreslikeKudu,andtechnologiesthatenablefasterqueries.UsingSQL-on-Hadoopengines(ApacheImpala,HiveLLAP,Presto,Phoenix,andDrill)andOLAP-on-Hadooptechnologies(AtScale,JethroData,andKyvosInsights),thesequeryacceleratorsarefurtherblurringthelinesbetweentraditionalwarehousesandtheworldofbigdata.FURTHERREADING:AtScaleBIonHadoopbenchmarkQ420161BigdatanolongerjustHadoop:Purpose-builttoolsforHadoopbecomeobsoleteInpreviousyears,wesawseveraltechnologiesrisewiththebig-datawavetofulfilltheneedforanalyticsonHadoop.Butenterpriseswithcomplex,heterogeneousenvironmentsnolongerwanttoadoptasiloedBIaccesspointjustforonedatasource(Hadoop).Answerstotheirquestionsareburiedinahostofsourcesrangingfromsystemsofrecordtocloudwarehouses,tostructuredandunstructureddatafrombothHadoopandnon-Hadoopsources.(Incidentally,evenrelationaldatabasesarebecomingbigdata-ready.SQLServer2016,forinstance,recentlyaddedJSONsupport.)In2017,customerswilldemandanalyticsonalldata.Platformsthataredata-andsource-agnosticwillthrivewhilethosethatarepurpose-builtforHadoopandfailtodeployacrossusecaseswillfallbythewayside.TheexitofPlatforaservesasanearlyindicatorofthistrend.FURTHERREADING:Uncommonsense:Thebigdatawarehouse2Organizationsleveragedatalakesfromtheget-gotodrivevalueAdatalakeislikeaman-madereservoir.Firstyoudamtheend(buildacluster),thenyouletitfillupwithwater(data).Onceyouestablishthelake,youstartusingthewater(data)forvariouspurposeslikegeneratingelectricity,drinking,andrecreating(predictiveanalytics,ML,cybersecurity,etc.).Upuntilnow,hydratingthelakehasbeenanendinitself.In2017,thatwillchangeasthebusinessjustificationforHadooptightens.Organizationswilldemandrepeatableandagileuseofthelakeforquickeranswers.They’llcarefullyconsiderbusinessoutcomesbeforeinvestinginpersonnel,data,andinfrastructure.ThiswillfosterastrongerpartnershipbetweenthebusinessandIT.Andself-serviceplatformswillgaindeeperrecognitionasthetoolforharnessingbig-dataassets.FURTHERREADING:Maximizingdatavaluewithadatalake3Architecturesmaturetorejectone-size-fitsallframeworksHadoopisnolongerjustabatch-processingplatformfordata-scienceusecases.Ithasbecomeamulti-purposeengineforadhocanalysis.It’sevenbeingusedforoperationalreportingonday-to-dayworkloads—thekindtraditionallyhandledbydatawarehouses.In2017,organizationswillrespondtothesehybridneedsbypursuingusecase-specificarchitecturedesign.They’llresearchahostoffactorsincludinguserpersonas,questions,volumes,frequencyofaccess,speedofdata,andlevelofaggregationbeforecommittingtoadatastrategy.Thesemodern-referencearchitectureswillbeneeds-driven.They’llcombinethebestself-servicedata-preptools,HadoopCore,andend-useranalyticsplatformsinwaysthatcanbereconfiguredasthoseneedsevolve.Theflexibilityofthesearchitectureswillultimatelydrivetechnologychoices.FURTHERREADING:Thecold/warm/hotframeworkandhowitappliestoyourHadoopstrategy4Variety,notvolumeorvelocity,drivesbig-datainvestmentsGartnerdefinesbigdataasthethreeVs:high-volume,high-velocity,high-varietyinformationassets.WhileallthreeVsaregrowing,varietyisbecomingthesinglebiggestdriverofbig-datainvestments,asseenintheresultsofarecentsurveybyNewVantagePartners.Thistrendwillcontinuetogrowasfirmsseektointegratemoresourcesandfocusonthe“longtail”ofbigdata.Fromschema-freeJSONtonestedtypesinotherdatabases(relationalandNoSQL),tonon-flatdata(Avro,Parquet,XML),dataformatsaremultiplyingandconnectorsarebecomingcrucial.In2017,analyticsplatformswillbeevaluatedbasedontheirabilitytoprovidelivedirectconnectivitytothesedisparatesources.FURTHERREADING:Variety,notvolume,isdrivingbigdatainitiatives5SparkandmachinelearninglightupbigdataApacheSpark,onceacomponentoftheHadoopecosystem,isnowbecomingthebig-dataplatformofchoiceforenterprises.Inasurveyofdataarchitects,ITmanagers,andBIanalysts,nearly70%oftherespondentsfavoredSparkoverincumbentMapReduce,whichisbatch-orientedanddoesn’tlenditselftointeractiveapplicationsorreal-timestreamprocessing.Thesebig-compute-on-big-datacapabilitieshaveelevatedplatformsfeaturingcomputation-intensivemachinelearni