20081226_数据仓库技术架构

SPDBProjectTraining数据仓库技术架构黄予辉2008年12月26日Agenda•架构设计原理•整体架构说明•方案说明•物理体系架构•架构前瞻架构立方逻辑架构层物理业务信息应用技术当前转换目标操作的顺序定义的等级起止经架构设计原则•简单化•纯粹化•标准化•扩展性•保障性•一致性?SourceSystemERDIfavailableDataDefinitions…?SourceSystemERDIfavailableDataDefinitions…Source/StagingCORESemantic/BusinessConceptualModelingLogicalModelingPhysicalModelingCustCustAcctAcctLocatLocatProdProdERD-ILDMCandidateKeyModelSysAFileBEtcSysAFileCSysAFileDSysBFileAKeyModelSourceTablesSrcAFileASrcAFileBSrcAFileCCustomizedLDMPartyPartyAGRAGREventEventProdProdFinanceFinanceAssetAssetCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXORDERORDERNUMBERORDERDATESTATUSORDERITEMBACKORDEREDQUANTITYITEMITEMNUMBERQUANTITYDESCRIPTIONORDERITEMSHIPPEDQUANTITYSHIPDATEPDM(3NF)EndUserAccessCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONSummaryVIEWINDEXAGGJINUSIEtc.ExtendedPDMUseCaseRulesAcceptFundReviewApproveBusinessProcessBIOBusinessRequirements?SourceSystemERDIfavailableDataDefinitions…?SourceSystemERDIfavailableDataDefinitions…Source/StagingCORESemantic/BusinessConceptualModelingLogicalModelingPhysicalModelingCustCustAcctAcctLocatLocatProdProdERD-ILDMCustCustAcctAcctLocatLocatProdProdCustCustAcctAcctLocatLocatProdProdERD-ILDMCandidateKeyModelSysAFileBEtcSysAFileCSysAFileDSysBFileAKeyModelSysAFileBEtcSysAFileCSysAFileDSysBFileASysAFileBEtcSysAFileCSysAFileDSysBFileAKeyModelSourceTablesSrcAFileASrcAFileBSrcAFileCSourceTablesSrcAFileASrcAFileBSrcAFileCCustomizedLDMPartyPartyAGRAGREventEventProdProdFinanceFinanceAssetAssetCustomizedLDMPartyPartyAGRAGREventEventProdProdFinanceFinanceAssetAssetPartyPartyAGRAGREventEventProdProdFinanceFinanceAssetAssetCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXORDERORDERNUMBERORDERDATESTATUSORDERITEMBACKORDEREDQUANTITYITEMITEMNUMBERQUANTITYDESCRIPTIONORDERITEMSHIPPEDQUANTITYSHIPDATEPDM(3NF)CUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXORDERORDERNUMBERORDERDATESTATUSORDERITEMBACKORDEREDQUANTITYITEMITEMNUMBERQUANTITYDESCRIPTIONORDERITEMSHIPPEDQUANTITYSHIPDATEPDM(3NF)EndUserAccessCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONEndUserAccessCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXCUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONITEMITEMNUMBERQUANTITYDESCRIPTIONSummaryVIEWINDEXAGGJINUSIEtc.ExtendedPDMSummaryVIEWINDEXAGGJINUSIEtc.INDEXAGGJINUSIEtc.ExtendedPDMUseCaseRulesAcceptFundReviewApproveBusinessProcessBIOBusinessRequirementsUseCaseRulesAcceptFundReviewApproveBusinessProcessAcceptFundReviewApproveAcceptFundReviewApproveBusinessProcessBIOBIOBusinessRequirements体系架构目的•提供一组集中的数据仓库架构集合–一般从概念参考架构和逻辑层次入手增强对一个经典数据仓库环境所提供的能力的理解和认识建立理想的逻辑设计，以体现架构中各种服务环节的能力为将来数据仓库架构的实现、物理化和部署奠定基础架构定义•信息技术系统架构(通用定义)–一个系统各组件的结构，包括它们之间的内部关系，以及一些原则和指导方针，用来管理系统架构内的组件及内部关系的质量、设计和不断变化的过程，使其最终达到一个理想的状态。•数据仓库架构–专为满足企业信息需要的一种系统架构，包括定义数据仓库能力，组件、流程以及相应的原则，并为数据仓库设计的实施提供参考和依据。目标•建立一个数据仓库的体系架构:遵循企业的IT目标，并成为一个可集成多种产品和服务的提供者可以按照业务运营的水平和方式来支持数据仓库环境的运行和管理帮助所有用户能够更好地使用数据仓库所提供信息用于提高业务成效，并提供访问的便利性推动并保证数据仓库体系内的一致性和可重用性为企业级数据仓库的能力提供文档说明为适用将来架构的扩展提供一个基准参考架构描述•参考架构将涵盖一个经典数据仓库环境所提供的主要能力–例如，它定义了哪些要素组成了现代数据仓库的环境•它建立一套完整的术语和数据仓库面向服务的结构，将为企业内的业务和IT部门所普遍使用•在参考架构中建立了五大主要服务类别，如下所示：DataManagementDataAcquisitionWarehouseInfrastructureWarehousingApplicationsInformationDelivery逻辑架构描述•逻辑架构的设计用于支持参考架构所定义的服务能力•更加关注这些功能组件如何进行整合，以及如何支持环境内的数据流和元数据流•逻辑架构为正在进行的数据仓库实现和部署提供一个重要的基础DataManagementDataAcquisitionWarehouseManagementWarehousingApplicationsInformationDeliveryMetadataFlowsWarehousePhysical&SemanticStructuresTechnicalDesignSemanticBusinessLogicalDataModelAgenda•架构设计原理•整体架构说明•方案说明•物理体系架构•架构前瞻CUSTOMERCUSTOMERNUMBERCUSTOMERNAMECUSTOMERCITYCUSTOMERPOSTCUSTOMERSTCUSTOMERADDRCUSTOMERPHONECUSTOMERFAXORDERORDERNUMBERORDERDATESTATUSORDERITEMBACKORDEREDQUANTITYITEMITEMNUMBERQUANTITYDESCRIPTIONORDERITEMSHIPPEDQUANTITYSHIPDATECo-locatedDependentMartDimensionalViewLogicalMartTeradataWarehouseFramework–Sinceforever…DataLoadingKnowledgeDiscovery/DataMiningBusinessUsersITUsersAssembleManageAnswerITUsersBusine