搜索
HUNANUNIVERSITY毕业设计(论文)设计论文题目：提高数据仓库中数据质量的研究学生姓名：高忠山学生学号：20041610505专业班级：软件开发班学院名称：软件学院指导老师：申煜湘学院院长：林亚平2008年5月20日湖南大学毕业设计(论文)第I页提高数据仓库中数据质量的研究摘要数据仓库是一个面向主题的、集成的、相对稳定的、能够反映历史变化并且能够支持管理决策的数据的集合。维护数据仓库中数据的正确性是非常重要的环节，而ETL是构建数据仓库的重要组成部分，要占据60%左右的工作量。提高数据仓库中的数据质量有利于保障数据仓库的可靠性，也有利于企业长期的总结和应用数据。提高数据质量的方法很多，鉴于本项目在实际工程应用中的经验总结和实践积累，本文主要侧重于利用ETL调度以及校验的方式来提高数据仓库中的数据质量。首先，利用ETL调度原理实现数据的自动化调度，使数据成为一个流动的数据流，数据所在的数据库作为ETL流的起点，保存最终高质量数据的数据仓库作为ETL数据流的终点。其次，对于已经形成ETL数据流的数据生成流实例信息，对完成ETL流程处理的数据构造历史信息，从而保证能够对ETL流进行监控。再次，构造ETL流程的处理逻辑，这是构造高质量数据的核心部分，根据实际的经验构造自动化的校验程序，使数据流流过时校验程序能够纠正错误数据，最终达到获得高质量数据的目的。最后，经过EMC项目的实施情况证明，非法数据在以ETL流的形式流经校验程序时都能够按照提前设定得到处理，保证了数据仓库中数据的稳定性。关键词：ETL，EMC（ETL管理中心），数据仓库湖南大学毕业设计(论文)第II页TheResearchinImprovingtheQualityofDataintheDataWarehouseAbstractAuthor:GaoZhongshanTutor:ShenYuxiangDataWarehouseisadatacollectionwhichissubject-oriented,integrated,andrelativelystable.Itcanreflectthechangesofhistoryandsupportthemanagementdecision.Itisanimportantsegmenttomaintaintheaccuracyofthedatawarehouse.However,ETLisanimportantparttobuilddatawarehouseandoccupyabout60percentoftheload.Theimprovementofthequalityinthedatawarehousehelpstoguaranteethereliabilityandalsohelptheenterprisetosummarizeandusethedataforalongperiodoftime.Therearemanymethodstoimprovethequalityofdata.However,intheviewoftheexperiencesummaryandpracticeaccumulationinthepracticalapplication,thepapermainlyfocusesontheETLdispatcherandwayofchecktoimprovedataqualityofdatawarehouse.Firstofall,theautomationofdatamanagementwhichisrealizedbyETLprinciplemakethedataintoadatastream.,thedatabasewherethedataexistsbecomeasstartingpointofETL,savingthedatawarehousewithultimatehigh-qualitydataastheendofETL.Second,forthedatawhichtheETLdataflowhasbeenformedtogenerateexample,forthedatawhichcompleteETLprocessesstructuretohandlinghistoricalinformation,thusensuringthattheETLflowcanmonitoring.Third,buildingtheprocessinglogicofETLtectonicprocessesisthecoreofconstructinghigh-qualitydata,constructingtheautomatedcalibrationproceduresbasedontheactualexperience,sothatthecalibrationprocedurescancorrecterroneousdatawhenpassingby,andfinallyachievingthegoalofaccessthehigh-qualitydata.Finally,theimplementationoftheEMCprojecthasprovedthattheillegaldataintheformofETLrunthroughcalibrationprocedurescanbehandledinaccordancewiththesetinadvance,ensuringthestabilityofdatainwarehousedata.Keywords:ETL,EMC(ETLManageCenter),DataWareHouse湖南大学毕业设计(论文)第III页目录1.绪论.......................................................................................................................................11.1研究背景......................................................................................................................11.2国内外研究现状..........................................................................................................21.3本文的主要研究内容和内容组织..............................................................................32.数据仓库以及ETL...............................................................................................................42.1数据仓库（DataWareHouse,DW）..........................................................................42.2ETL（Extract,Transform,loading）...........................................................................62.2.1ETL简介.............................................................................................................62.2.2ETL过程特点.....................................................................................................62.2.3数据质量保证....................................................................................................72.2.4元数据：拓展新星应用....................................................................................83.EMC项目中的ETL应用...................................................................................................103.1业务背景....................................................................................................................103.2实例概述....................................................................................................................123.3总体流程....................................................................................................................133.4接口表........................................................................................................................133.4企业数据迁移（EDM）模型层...............................................................................143.5事实表层....................................................................................................................153.6数据集市（DM）层..................................................................................................153.7多维分析（OLAP）层..............................................................................................163.8总体处理流程、反馈过程........................................................................................164.EMC中数据调度校验的应用............................................................................................174.1需求分析...........................................................