Huny's Dev Blog
huny.dev

Why gzip SHA-256 Changes for Identical Content: Reproducible Compression in Python and GNU gzip

Reproduce how gzip timestamps and filenames change artifact hashes, then stabilize repeated output with explicit Python metadata and GNU gzip -n.

Hun Jang
Hun Jang Nov 22, 2026

If the SHA-256 of a gzip artifact changes despite identical source bytes, inspect the header's timestamp and original filename as well as the compressed data. For a reproducible single gzip artifact in Linux CI, use GNU gzip's -n option or explicitly set metadata with Python's GzipFile(filename="", mtime=0, ...). The examples below compare changed and fixed metadata and define what was actually verified.

AI-generated conceptual illustration of removing variable metadata tags from two archive containers holding identical data. This is not an exact gzip format diagram.

Written and documentation checked on 2026-11-22. The tested environment was Linux x86_64, Python 3.12.14, zlib runtime 1.3.2, and GNU gzip 1.13. References are the Python 3.12 documentation and GNU gzip manual; the latest patch version shown in the documentation is distinct from the version executed here.

Identical source content is not an identical deployment artifact

Even when generated file content stays unchanged, a changed compressed artifact can be treated as a different file by caches or change detection keyed on its artifact hash. First decide what you are comparing.

Comparison target
Question
Check used here
Decompressed bytes
Is the source content identical?
Direct byte comparison against the source
Entire gzip bytes
Is the deployment artifact identical?
Full byte and SHA-256 comparison
Gzip header
Did information outside the content change?
Inspect MTIME and the FNAME flag

A gzip member includes header information such as MTIME, OS, and an optional filename in addition to compressed blocks. MTIME 0 means no timestamp is provided. Identical source content therefore does not imply identical gzip bytes. RFC 1952 member format

Change time and filename separately in Python

Save this as gzip_repro.py and run python3 gzip_repro.py. Explicit timestamps 1 and 2 isolate the time field without requiring a delay. The example does not read or modify user files.

The actual output was:

Only the timestamp differs between a and b; only the filename differs between a and c. Both cases passed assertions checking that their compressed bytes and SHA-256 values differ, while all five outputs decompressed to the same source. fixed1 and fixed2 produced identical results in the same environment after setting the header time to 0 and excluding the filename.

With Python's fileobj specified, filename determines the original name included in the header. Passing an empty string avoids relying on the file object's name. Fix mtime explicitly as well. Read the result after the with block so it includes data written when the stream closes. GzipFile constructor documentation

The header checks inspect only MTIME and FNAME in the gzip member created by this example. They are not a complete parser for arbitrary gzip files.

Exclude metadata with GNU gzip -n

When compressing a regular file, GNU gzip normally stores the original name and modification time. -n requests that this information not be stored; -c writes to standard output while preserving the input. The compression level is also fixed at -6 here. GNU gzip options

This check operates entirely inside a separate temporary directory. It compresses identical content under two different input names, then compares both the compressed results and the restored source. The temporary directory is removed afterward.

The actual output was as follows. Both cmp commands must succeed before the final message is printed.

This compares GNU gzip outputs with each other. It neither verifies nor requires byte-for-byte equality between Python and GNU gzip outputs.

What to pin in CI, and what remains outside the guarantee

Removing the filename and timestamp does not require different compressors or versions to produce the same compressed stream. In CI, pin source bytes, compressor and runtime versions, compression level, and invocation method together, then compare the actual outputs. The successful scope here is repeated execution in the environment above using the same method.

Do not assume that Python's gzip.compress() and GzipFile always follow the same implementation path. The Python 3.12 documentation notes that gzip.compress(..., mtime=0) uses a zlib path and can produce a different OS header byte. This example uses explicit GzipFile settings instead of that API. Python 3.12 compress documentation

For .tar.gz, changing the outer gzip layer does not fix differences in the inner tar's file order, timestamps, or ownership metadata. Compare tar bytes before compression in that case. Reproducible Builds guidance on archive metadata

Before deployment, generate the same input twice and compare the entire compressed artifacts, then decompress both and compare them against the source. A changed compressed result does not immediately prove changed source content; matching decompressed content does not make the deployment artifact's hash difference irrelevant. Performance, compression ratio, and reproducibility across different operating systems or versions were not measured here.

한국어 버전: 같은 원본인데 gzip SHA-256이 달라질 때: 시간·파일명 제거로 재현 가능한 압축 만들기같은 원본인데 gzip SHA-256이 달라질 때: 시간·파일명 제거로 재현 가능한 압축 만들기

추천 글