PIGZ: Parallel GZIP
Posted by Sachin Garg on 13th April 2007 | Permanent Link
Mark Adler, maintainer of popular zlib library has released a multicore capable Parallel Implementation of GZip, nicknamed PIGZ. Version 1.5 implements nearly all of gzip’s functionality, including decompression of .gz and .Z (Unix compress) files.
You can get the latest code, it needs latest version of zlib (1.2.3) to build.
April 13th, 2007 at 11:24 pm
That link won’t work. The latest distributed version is 1.6, and the link is http://zlib.net/pigz16.c.gz.
It will also optionally compress to the zlib format.
April 30th, 2007 at 11:33 am
Mark has released another update, version 1.7 decompresses the first entry of a zip file (if deflated), and compresses to a single-entry zip file if requested with –zip.
http://zlib.net/pigz17.c.gz
March 25th, 2008 at 1:18 pm
http://zlib.net/pigz16.c.gz is broken, can you point me to a different location?
Cheers,
kris
March 25th, 2008 at 6:40 pm
Just use the link to version 1.7, its in the comment above yours.
March 26th, 2008 at 2:01 pm
I wish to use this opportunity and ask Mark Adler a question. In 1990 Martucci published article regarding usage of simple predictor in combination with entropy encoder for compression of images. Following this article it is possible to write image compression routine of 300 lines that makes photo images to be compressed as 2.5 to 1. Three years later PNG format was created that required 20 times longer code and compressed photo images typically as 1.7 to 1. PNG format happened to be incorporated into major browsers. 16 years after mentioned publication Microsoft introduced HD Photo that compress images to the same ratio as in Marucci method published in 1990. And few years ago JPEG2000 anounced format that is about 7% better than Martucci suggested in 1990. The question is
Wouldn’t that be nice to incorporate new format into all browsers that require 300 lines of coding, so programmers will be able to generate images on the fly in web applications without using so called new approach of Microsoft that is called Silverlight.
Thanks,
Andrew Polar
June 14th, 2008 at 2:11 am
According to my personal benchmarks, pigz doesn’t speed up compression, unlike pbzip. In fact, in some cases it’s performance may be slower than single-threaded version!
June 16th, 2008 at 5:30 am
Girlfriend: What is your hardware/software setup?
October 19th, 2008 at 4:50 pm
Version 2.0 of the parallel implementation of gzip called “pigz” is
available here:
http://zlib.net/pigz/
pigz exploits multiple processors and multiple cores for compression.
The use of threads has been completely rewritten to be more efficient
and clean. It compiles on Unixish operating systems that provide the
zlib and pthread libraries.
If you have success or failure in compiling and using pigz, I would
appreciate knowing about either way. Thanks.
October 21st, 2008 at 3:05 am
Nice article. Thanks. :) Eugene
October 23rd, 2009 at 10:12 am
Does this run on Windows systems? Is the compression faster than something like gzip?
October 30th, 2009 at 12:16 pm
You will need to compile it yourself (its just code but it should work on windows).
It will work 2 times faster by using both cores if your CPU is dual core. 4x faster if you have a quad core, but no speed up if you have a single-core CPU.
July 29th, 2010 at 8:17 pm
You cant extract the resulting .tar file from http://zlib.net/pigz/ on an AIX server.
# tar -xvf pigz-2.1.6.tar.tar
tar: 0511-169 A directory checksum error on media; 0 not equal to 72695.
[root@nzaxs22ch-cl] : /opt/pigz
Are there limitations as to what O/S can use this?
July 29th, 2010 at 8:37 pm
Ignore my last post.
The download randomly changed the filename to “tar -xvf pigz-2.1.6.tar.tar” when it should have been “tar -xvf pigz-2.1.6.tar.gz”
…. go figure
August 16th, 2010 at 7:50 pm
Just wondering how I can compile pigz “statically” in fedora and then copy it to a Redhat machine - because RH still uses zlib version 1.2 but pigz requires 1.3.