互联网公司技术架构资料.新浪微博.构建高性能的微博系统

BuildHighPerformanceWeiboSystem@TimYang••140•••QCon2010•WDC2010Agenda1234Part1()MySQL•••hash••user_timelineindex••••5•MySQL+cache•••cache••NoSQL“Databasesarespecializing–the“onesizefitsall”approachnolongerapplies.”MongoDBNoSQL?••Redis•MongoDB••Cassandra•HBase•MongoDB•Redis•HBase•CassandraRedis-••snapshot-•vm-•diskstore-•aof-Redis-•string:keyvalueredisObject16bytes/item•list:40bytes/item•hash:zipmap(64)•set/sortedsetRedis-Replication•rdb•Redis-Redis-••failover•list/set(optional)•MySQL•NoSQLMySQLcachePart2()→→Webcachecache••“Webjsoncachejson()2~5K,xml10k,protobuf500”3600:47(495)|(134)RDBMS→Keyvalue(JSON)→Protocolbuffers(binary)JSON•DB•Cache•MessageQueue•APIPB•Numeric:varint,from1byte••Java,C++,Python...••cache•“192.168.0.1”→“0xc0a80000”→varintBenchmarkTextText()“WewouldliketoprovidepublicAPIsthatacceptprotocolbuffersaswellasXML,bothbecauseitismoreefficientandbecausewe'rejustgoingtoconvertthatXMLtoprotocolbuffersonourendanyway.”-Google•••“”•LAMP•1ms••MQstat•MQProcessorstat?“1”•Timeline•••cache•RAM1.2.3.4.cachemaster5.replication•5•5••5•••5••••RAMisthenewdisk•“PerconaServernowbothSQLandNOSQLHandlerSocket,100rps12cores/24threadsand380GBofRAM,mysqlRedisNoSQL”2010-12-2500:22(9)|(2)@jackbillow“1.PKlookup2.cache”@kobe“innodb”@TimYang“InnoDBAdaptiveHashIndexeshash”•NoSQL•binlogRedis•jbinlogfl/jbinlogPart3app•••spam••Text••GZIP•Part4“@Zheng:“”//@:@bian(49)|(15)”21410:36(22)|(10)••••••••••cache•••MySQLRedisMemcachedTimeline••RAM•Q&A@TimYang杭州站·∙ 2011年10月20日~22日（6月启动）QCon北京站官方网站和资料下载