Skip header

Compression Algorithm Testing (2021-11-02)

.Zip It Good!

brook > dive > you can zip it

I conducted some tests to see which compression formats squeeze certain file formats better. Because apparently even my backups need to be optimized.

PeaZip 7.32

Initial testing was conducted in July 2020 with PeaZip, which incorporates its own PEA format alongside 7-Zip (gzip, bzip2, xz, zip, 7z) and other FOSS algorithms (BCM, Brotli, FreeArc, ZPaq, zstandard).

Text

A folder containing mostly HTML files downloaded from AO3, plus some ASCII plaintext files:

LEAST SIZE REDUCTION --------- MOST SIZE REDUCTION
pea, zpaq, zip, gz, xz, br, zst, 7z, bz2, arc, bcm

SLOWEST ---------------------------------- FASTEST
bz2, gz, zip, 7z, zst, arc, br, bcm, zpaq, xz, pea

Images

PNGs and JPGs plus the occasional GIF:

LEAST SIZE REDUCTION --------- MOST SIZE REDUCTION
zpaq, pea, gz, zip, xz, bcm, bz2, arc, br, 7z, zst

SLOWEST ---------------------------------- FASTEST
bz2, br, zip, 7z, gz, bcm, arc, zst, pea, xz, zpaq

Videos

Mostly MP4S with the occasional FLV and WEBM:

LEAST SIZE REDUCTION --------- MOST SIZE REDUCTION
zpaq, arc, pea, xz, gz, bz2, zip, 7z, br, zst, bcm

SLOWEST ---------------------------------- FASTEST
bz2, br, zip, gz, 7z, arc, bcm, zst, pea, xz, zpaq

Flash

Purely SWF animations:

LEAST SIZE REDUCTION --------- MOST SIZE REDUCTION
zpaq, bcm, pea, bz2, zip, gz, arc, xz, br, zst, 7z

SLOWEST ---------------------------------- FASTEST
bz2, br, zip, gz, 7z, bcm, arc, zst, pea, xz, zpaq

Others

PDF

1.36GB of PDF files, compressed with 7-Zip ZS 1.5.0 R1 (gzip, bzip2, xz, zip, 7z, lizard, lz4, lz5, zstandard):

LARGER --------------- SMALLER
lz5, lz4, gz, liz, 7z, zst, xz (1.19GB)

SLOWEST -------------- FASTEST
gz, 7z, lz5, xz, liz, zst, lz4

I did not test bzip2 as it was previously shown to be the slowest algorithm available.

HTML Dumps

With PeaZip 7.90, I tested out some archived message boards that were mostly HTML with very little CSS or images:

LEAST SIZE REDUCTION --------- MOST SIZE REDUCTION
zip, pea, gz, bz2, br, zst, zpaq, xz, 7z, bcm, arc

XML/JSON

I threw 7-Zip ZS 1.5.0 R1's formats at a small collection of bookmark files and JSON/OPML settings exports:

LARGER ------------------------- SMALLER
lz5, lz4, liz, zip, gz, bz2, zst, 7z, xz

Conclusions

For text files and websites under 5GB in size, the FreeArc 0.67 alpha from 2014 absolutely dunks on everything else, even seven years after it was made. For larger website mirrors, 7z was able to achieve smaller sizes.

For images, videos, and animations, zstandard offers if not the best compression ratio, then the best speed/size compromise. But, if you absolutely must have the smallest backups, use BCM for video, and 7z for SWFs.

To save a bit of space, I replaced PeaZip with 7-Zip ZS, but kept FreeArc's CLI utility around to crunch them HTTrack backups since it's exceptional at doing so. The rest of my backups are stored in 7Zs for general files, ZSTs for images, and XZ for PDFs.


[Listen]Listen to "Modern Talking - In 100 Years..." on YouTube?
τ