互联网公司技术架构资料新浪动态应用平台后端系统与

•DevOps@SinaἧLamp项目托管平台ἧ已托管项目500+ἧ日均请求10B+ἧSOA陷阱•Memcached“太”好用了ἧRDBMS难以ģ展ἧ困扰ሹ们的“Ẓ喀琉斯之踵”ἧ解耦ἧ分而治之ἧ...太多名词来描述优点和必要性ἧ当目标太多的时候，是不是容易⁇成і了做而做？Service1Service2UserService3Service4Service5Service6Service7Service8Service9Service10ClientService1ZooKeeperService2Service2DiscoveryNamingACLQuotaConfigurationCenterOpsDNSWebServerDNSMasqApacheDNSQueryWorkerServerDNSMasqQueueSubscriberDNSQueryMessageQueueUpdatingAPIUpdateSyncPurgeServicePurgeReqDNSCachePurgeMonitorStatusUpdate•CPythonNode.JSPowerDNSDNSMasqMongoDBἧDNS协议，更新时通知所有客户端，缓存一致性强ἧ多样化APIἧ无监控框ว，࿸结合API设计开发ἧA记ᕢ无法返回端口号ἧሹ的首ၷ最多要取50+1+4+2个用户的信息ἧሹ的首ၷ最少要取50X2+3个计数ዸἧі了访问ቴ度大量使用并依赖Memcachedἧ如果有一台Memcached服务ዸ宕机的话...ZookeeperClusterWebFront-EndApachephp-meta-extphp-mc-extmetadzklocalcacheCacheNodeWatcherMCMCMCUpdatingNotificationCacheNodeWatcherMCMCMCLegendServerProcessClusterExtensionNodeRegister&HeartbeatMemcachedCommand•ThriftCC++PythonZooKeeperἧ分片信息保存在ZooKeeper中，并在Metad缓存ἧউ持引擎级主⎫主复制ἧ连接数LRU（PatchfromSAEۤ•Memcached-LikeSDKἧԯ来特性：ἧ跨IDC复制（开发中ۤἧ数据迁移和无缝ģ容（也ᱎۤἧFlashCached（也ᱎۤἧ更好的管ၯ性ἧMySQL很棒，但是...ἧRange଑ĈHashἧ开发改造成本ἧ不合“规范”的Ĉ┐ቷ•MasterHAManagementAPIZooKeeperFEClusterWeb/Command-linetoolsMySQLClusterMySQLClusterMySQLClusterMayaMayaἧNode.JĈ、C、Flex௲Bḋson、P௾thon、ἧĈ┐ቷ路ҹἧ流量统计、配̩、限制ἧ数据库预拆分（不是自动拆分ۤἧ主库半自动切换、从库自动切换ἧĈ┐ቷ审计、自动DDቷἧ每年新增PHP项目100+ἧ3000+台前端服务ዸ100亿+Hits/天ἧ45000+ᥳ虚拟主机配置30G代码（GZip后ۤἧ3000+台数据库服务ዸ200亿+数据库请求/天ἧ6000+个MySQL实例总存储量2P+MakeDisastersManualἧ配置ዃ部᫪越统一越容易自动化ἧኚ服务ዸ按照功能划分角色ἧі每一个角色指定相应的配置ᅎ件ἧ角色和配置ᅎ件的分类尽量ℛ洁ἧ差异配置利用程序来自动化处ၯhttpd-vhost.conf_tmplvhost_config.plFENode/etc/httpd-vhost.confἧ那༫创造一个配置ᅎ件吧ἧ当然要尽量创造一个全局统一的ἧ索性ዃ监控程序结合૜来怎么样？•INIVSYAMLἧ༫比如Memcached![MBlog_Userinfo]ip=10.55.22.100:1000010.55.22.101:1000010.55.22.102:1000010.55.22.103:10000memsize=10.55.22.100:1G10.55.22.101:1G10.55.22.102:1G10.55.22.103:1Gconn_limit=10.55.22.100:1W10.55.22.101:1W10.55.22.102:1W10.55.22.103:1W[MBlog_Counter]ip=10.55.22.100:1000110.55.22.101:1000110.55.22.102:1000110.55.22.103:10001memsize=10.55.22.100:1G10.55.22.101:1G10.55.22.102:1G10.55.22.103:1Gconn_limit=10.55.22.100:1W10.55.22.101:1W10.55.22.102:1W10.55.22.103:1Wmc.confMCNode/etc/init.d/memcached/etc/cron.d/check_mcMonitorNode/etc/monitor.d/check_mc.pyReportingNode/etc/reporting.d/get_mc_stats.pyAutomationἧ自动化了很多个点ἧ但是运维是一个体系ἧ但当ሹ们尝试把点联૜来的时候ἧሹ们发现不是点不够多ἧ༫是重复发明了轮༬Frameworkἧ自动化运维不是“另一个”系统ἧ它是对你ว构可运维性的更高要求ἧ࿸要ኚ可运维性作і软件和ว构设计的主要考量ᐦᕫ之一ἧ运维工具也是软件和ว构的一部分RevolutionOfMartyredElitesit’sROMEAPIsAssetInventoryOrchestrationConfigurationManagementNodegetun-appliedchangesgenerateconfigcreateorchestrationjobsshrome@10.71.1.2xxxcreateOrchestration(targets=['10.71.1.2','10.71.1.3'],commands=['updatevhost.conf','restarthttpd'])5readConfig(criteria={'cmdb':'D2012887'}files=['vhost.conf'])readAsset('criteria'={'cmdb':'D2012887'},'fields'={'addresses.internal':true})341readRole(criteria={'applied':false},fields={'included':true})2retrievetherelatedinfoAssetInventoryOrchestrationConfigurationManagementNodesomeupdategettherelatedinfogenerateconfigcreateorchestrationjobsshrome@10.71.1.2xxx1createOrchestration(target='10.71.1.2',commands=['updateall','initialize'])5readConfig(criteria={'cmdb':'D20120601x'})createRole(name='D20120601x',includes=['mysql','xd'],ports=[3310,3324])readAsset('criteria'={'cmdb':'D20120601x'},'fields'={'addresses.internal':true})342•HTTP-basedRESTfulAPIwithJSONpayload•AvarietyofMongoDB-likeCRUD•Node.JS•Connect&Express•MongoDB•Ops-DevOps•Scrum•BDD•CI•工具大部分来自于Atlassian(比如Jira)Q&A