gziptest.sh part 2: multi-threaded compression benchmarks
Second table below replaces compression and decompression cpu utilisation columns with speed in megabytes per second (MB/s).
Findings:
Tar compressed /usr partition produced a 1500MB sized usr.tar file which was compressed using the above compression methods. Compression levels 1 to 5 were tested. The results pretty much mirrored the same patterns of the MySQL sql backup file tests.
- Pigz compared to gzip was 2x times faster for compression and 1.79x times faster for decompression. But both ended up with 25.5% compression ratio.
- Bzip2, pbzip2 and lbzip2 all resulted in level 5 compression ratio of 22.7% so again smaller compressed file sizes than pigz, zip and gzip. However, bzip2 was slowest taking 482.8 seconds to complete at level 5 compression levels. Pbzip2 was 3.06x times faster than bzip2. Again, lbzip2 was fastest – 5.3x times faster than bzip2 and 1.74x times faster than pbzip2. But memory consumption for lbzip2 was again the highest of the 3 at 1,106MB peak memory usage.
- Lzip and plzip were on average the best in terms of compression ratio from level 1 to 4 and at level 5 managed 17.9% compression ratio only just behind p7zip’s (LZMA 7z) compression ratio of 15.6%. Plzp was 3.4x times faster than single threaded lzip. But memory usage was alot higher peaking at 929MB memory usage for compression at level 5.
- P7zip again produced strange results for LZMA 7z tests where only compression levels 1, 3 and 5 resulted in reduction in compressed file sizes. Level 5 compression produced the most noticeable results with lowest compression ratio of all tests at 15.6% but did so at the expense of compression time 2.77x times slower than plzip and increased peak compression memory usage from level 1 & 2 12.56MB, to level 3 & 4 40.2MB to 770.4MB at level 5 which is still 17% less than plzip.
- P7zip bzip2 tests were middle of the road being 3.2x times faster than bzip2 and ever so slightly faster than pbzip2 but still 1.66x times slower than lbzip2 – 150.81 seconds vs 90.50 seconds at level 5 compression.
Tar gziptest.sh summary:
- For speed again pigz is the best choice – compression a 1500MB tar file in 31.34 seconds.
- For compression ratios with plzip has best overall balance for compression ratio vs memory usage. For for instance just at level 1 compression, compression times are still 1.19x times faster than lbzip2 level 5 compression but result in 1/3 the peak compressed memory usage and slightly better compression ratios – lbzip2 level 5 at 22.7% vs plzip level 1 at 21.4%.
