Google File System(Chinese version)

GoogleFileSystemalexGoogleGFSGFSGFSGFSGFSGFSGoogleTBD[4]:3—D1.GoogleGoogle(GoogleFileSystem–GFS)GFSGFSGFSGFSbugbugGFSGBwebTBKBI/OBlockAPIGFSGFSGoogleGFS1000300TB2.2.1••100MBGB•KB1MBKB••alexwell-defined”-“•2.2GFSAPIPOSIXAPIGFS”-”3.43.32.3GFSMasteralexMasterGFSMasterMasterMasterMasterMasterChunk1Linux(user-level)ChunkGFSChunkChunkMasterChunk64ChunkChunkChunklinuxChunk3MasterChunkChunkMasterChunk(alexBDBlease)Chunk(alexorphanedchunks)ChunkChunkMasterChunkChunkChunkGFSGFSAPIMasterChunkMasterChunkPOSIXAPIGFSAPILinuxvnodeChunkChunkChunkLinux2.4MasterMasterMasterChunkMasterMasterMasterMasterChunkChunk1ChunkChunkChunkMasterMasterChunkChunkkeyChunkChunkMasterChunkMasterChunkChunkMaster2.5ChunkChunk64MBBlocksizeChunkLinuxChunkChunkChunkMasterMaterChunkChunkChunkTBChunkChunkChunkTCPChunkMaster2.6.1ChunkChunkChunkChunkChunkChunkGFSGFSsingle-chunkChunk2.6MasteralexMasterMasterMaster3ChunkChunkChunkMasterChunkMasterMasterMasterMasterChunkMasterChunkChunkChunk2.6.1MasterMasterChunkChunkChunkChunk4.34.4ChunkMasterMaster6464MBChunkChunkChunkChunk64Master2.6.2ChunkMasterChunkChunkMasterChunkMasterChunkChunkChunkMasterChunkChunkMasterChunkChunkChunkMasterChunkChunk()Chunk2.6.3GFSalexLSNChunk(4.5)ChunkMasterMasterMasteralexMasterCheckpoint(alexCheckpoint)CheckpointalexMasterCheckpointCheckpointCheckpointB-CheckpointMasterCheckpointMasterCheckpointCheckpointCheckpoint1CheckpointMasterCheckpointCheckpointalexcatastrophesCheckpointCheckpoint2.7GFSGFSGFS2.7.1GFSMaster4.1Master2.6.3regionalexregion1region“”regionregion“”regionregionregionregionregionregionGFS3.3alexGFSGFSregionGFSregionGFSregionGFSaChunk3.1bChunkChunk4.5MasterChunkChunkChunkChunkReaderalexReaderWriterGFSMasterChunkGFSMasterChunk“”ChunkChecksum5.24.3ChunkGFSChunkGFSalexMasterChunk2.7.2GFSCheckpointCheckpointCheckpointReadersCheckpointregionregionCheckpointWriterReader-“”WriterReadersWritersChecksumReaderChecksum()webI/OalexThesefunctionalitiesforrecordI/OGoogleReader3.MasterMasterChunk3.1leasealexleaseChunkChunkleaseMasterChunkChunkChunkChunkMasterChunkMaster60ChunkChunkMasterMasterChunkMasterMasterMasterChunkChunk21.MasterChunkChunkMaster2.MasterChunksecondaryChunkChunkMaster3.ChunkLRUChunkChunk3.24.ChunkChunkalex“”5.ChunkChunkThisimagecannotcurrentlybedisplayed.6.Chunk7.ChunkalexChunkChunkChunkChunkregion37ChunkGFSregionChunkregion2.73.2ChunkChunkChunkeginter-switchChunkS1S4ChunkS1S1S2S2S4S2S2S3S4IP“”TCPChunkBRB/T+RLTL100MbpsTL1ms1MB80ms3.3GFS–regionregionGFSbyteGFSGFSUnixO_APPEND/3.1ChunkChunkChunkChunkChunk64MBChunkChunkChunkChunk1/4ChunkChunkChunkChunk–GFSChunkChunkChunkChunkMasterChunkregion2.7.23.4(alexCOW)“”AFSalexAFSAndrewFileSystemcopy-on-writeMasterChunkChunkMasterMasterChunkMasterMasterChunkChunkCMasterMasterChunkeC1(alex1.Snapshot)MasterChunkC`MasterChunkCChunkC`ChunkChunkChunkChunk100Mb3ChunkMasterChunkC`ChunkChunk4.MasterMasterChunkChunkChunkChunkChunk4.1MasterChunkChunkMasterregionGFSGFSUnixGFSMaster/d1/d2/…/dn/leaf/d1/d1/d2…/d1/d2/…/dn/d1/d2/…/dn/leafleaf/home/user/save/user/home/user/foo/home/save/home/user/save/user/home/home/user/home/user/foo/home/user””inode4.2GFSChunkChunkChunkChunkChunkChunk4.3ChunkChunkMasterChunkMaster1ChunkChunk2Chunk””ChunkChunkWriter””Chunk3ChunkChunkMasterChunkChunkChunkChunkChunkChunkChunkChunkliveChunkChunk4.4ChunkChunkMasterChunkChunk””ChunkMasterChunkChunkChunkMasterMasterChunkChunkMasterMasterChunk4.4GFSGFSChunk4.4.1MasterMasterMaster“”MasterChunkalexThiseffectivelyseversitslinkstoallitschunksChunkMasterChunkChunkChunkMasterChunkMasterChunkChunkMasterChunkChunk4.4.2GFSChunkMasterChunkLinuxChunkMaster””ChunkChunkChunkMasterMasterChunkalexmetadataMasterChunkMasterMaster4.5ChunkChunkMasterChunkMasterChunkChunkMasterChunkChunkMasterChunkMasterChunkChunkMasterMasterChunkMasterMasterChunkMasterChunkChunkChunkChunkChunk5.GFSGFS5.1GFS5.1.1MasterChunkkill(alexaminorhiccup)6.6.25.1.2ChunkChunkChunk3ChunkChksum5.2MasterChunkalexChunk3ChunkErasurecodesalexErasurecodesbuffer5.1.3MasterMasterMasterMastercheckpointMasterMasterMasterMasterGFSMasterMastergfs-testDNSMasterMasterGFS“”Master“”“”Master“”Master1“”MasterChunk“”MasterMasterMaster“”MasterChunkChunk“”MasterChunk“”Master“”MasterMaster5.2ChunkChecksumGFS7ChunkChunkGFSGFS(alexbyte-wise)ChunkChecksumChunk64KB32ChecksumChecksumChunkChunkChecksumChunkChecksumChunkMasterMasterMasterChunkChecksumGFSChecksumblockChunkChunksumI/OChecksumI/OChecksumChunkChecksumChecksumChecksumChecksumChecksumChunkChecksumChecksumChunkChunkChunkChunkMasterChunkMasterMaster5.3GFSChunkRPCRPCRPC6.GFSGoogleGFS6.11Master2Master16Chunk16GFSGFSChunkPIII1.4GHz2GB80G/5400rpm100MbpsHP2524GFS19161Gbps6.1.1NGFS320GB4MBregion2561GBChunk32GB10%Linux95%3aN1Gbps125MB/S100Mbps12.5MB/s10MB/s80%1694MB/s75%6MB/s80%75%Chunk6.1.2NN1MB1GB3b67MB/sbyte16Chunk3Chunk12.5MB/s6.3MBChunk1635MB/s2.2MB/sChunk16166.1.33cNChunkChunk6.0MB/s164.8MB/sNMNMChunkChunk6.2GoogleAMBTBBBTB””6.2.1ChunkTB“”Chunk18TB52TBB“”BChunk6.2.2ChunkGB64KBChecksumChunkChunk4.5MasterMB100MasterGFSMasterChunkChunkChunkalexCOWcopy-on-writeChunkMaster50MB100MBMa