How Google Drive Syncs Files With Chunking and Deduplication

You edit one paragraph in a 50 MB document and it syncs in a second. You upload a file your coworker already uploaded and it finishes instantly. Neither of these is magic. Both fall out of two ideas: break files into chunks, and address those chunks by their content. Once you see how file sync really works, cloud storage stops looking mysterious and starts looking like a clever application of hashing.

The core problem

Naive sync uploads the whole file every time anything changes. Change one byte in a large file and you resend all of it. Across many users and many edits, that wastes enormous bandwidth and storage, and it makes sync feel slow. The system has to answer two questions cheaply: what changed in this file, and have we seen this data before.

Key design decisions

Split files into chunks. Instead of treating a file as one blob, cut it into pieces, often a few megabytes each. Now a file is an ordered list of chunk references plus some metadata. When you edit the file, only the chunks that changed need to move. Sync one modified chunk, not the whole file.

Address each chunk by the hash of its content. Compute a strong hash (something like SHA-256) of each chunk and use that hash as its identity. This is content-addressable storage. Two chunks with identical bytes produce the same hash, so they are the same object. This single decision gives you deduplication for free: before uploading a chunk, ask the server if it already has that hash. If yes, skip the upload and just record the reference.

Separate metadata from block storage. The metadata service knows the file tree, versions, permissions, and the chunk list for each file. The block store holds the actual chunk bytes, keyed by hash, and is happy being dumb and enormous. This split lets you scale them independently. Metadata is small, relational, and consistency-sensitive. Blocks are huge, immutable, and live in object storage.

Sync deltas, not whole files. The client keeps a local index of the chunks it has. When a file changes, it recomputes chunk hashes, diffs against the last known set, and uploads only the new chunks, then commits a new file version pointing at the updated chunk list. Because chunks are immutable and content-addressed, old versions keep working as long as their chunks exist.

The consistency question

Two devices edit the same file offline, then both come online. Now you have a conflict. Drive-style systems resolve this at the metadata layer with versioning: each change creates a new version with a parent pointer, and when two changes share a parent you either merge, pick a winner, or keep both as conflicted copies. The block store is never the thing that conflicts because chunks are immutable; conflicts are always about which chunk list is the current one. Delivering changes to other devices quickly usually means a notification channel: when a file's metadata changes, subscribed clients get a nudge to pull the new version rather than polling constantly.

A note on chunk boundaries

Fixed-size chunks have a weakness. Insert a byte at the front of a file and every chunk boundary shifts, so every chunk hash changes and you resend everything. Content-defined chunking (using a rolling hash to place boundaries based on the data itself) makes boundaries stable under insertions, so an early edit only touches nearby chunks. This is the trick that makes dedup robust in the real world.

How the real systems do it

Dropbox and Google Drive both chunk files, hash the chunks, and store blocks separately from metadata. Dedup happens across the whole system, so a popular file that many people have is stored once at the block level. Immutable content-addressed blocks make caching, CDN distribution, and integrity checks straightforward, because the hash both names the data and verifies it. The metadata service, backed by a strongly consistent database, is where versions, sharing, and permissions live.

The takeaway for an interview: chunk the file, name each chunk by its content hash, split metadata from blocks, and sync only the deltas. Almost every nice property of cloud sync is a consequence of those four moves. I wrote the full breakdown, with the sync protocol and the metadata schema, here: https://www.systemdesign.academy/interview/design-google-drive