PIGZ: Parallel GZIP
Posted by Sachin Garg on 13th April 2007 | Permanent Link
Mark Adler, maintainer of popular zlib library has released a multicore capable Parallel Implementation of GZip, nicknamed PIGZ. Version 1.5 implements nearly all of gzip’s functionality, including decompression of .gz and .Z (Unix compress) files.
You can get the latest code, it needs latest version of zlib (1.2.3) to build.
April 13, 2007 11:24 pm
That link won’t work. The latest distributed version is 1.6, and the link is http://zlib.net/pigz16.c.gz.
It will also optionally compress to the zlib format.
April 30, 2007 11:33 am
Mark has released another update, version 1.7 decompresses the first entry of a zip file (if deflated), and compresses to a single-entry zip file if requested with –zip.
http://zlib.net/pigz17.c.gz
March 25, 2008 1:18 pm
http://zlib.net/pigz16.c.gz is broken, can you point me to a different location?
Cheers,
kris
March 25, 2008 6:40 pm
Just use the link to version 1.7, its in the comment above yours.
March 26, 2008 2:01 pm
I wish to use this opportunity and ask Mark Adler a question. In 1990 Martucci published article regarding usage of simple predictor in combination with entropy encoder for compression of images. Following this article it is possible to write image compression routine of 300 lines that makes photo images to be compressed as 2.5 to 1. Three years later PNG format was created that required 20 times longer code and compressed photo images typically as 1.7 to 1. PNG format happened to be incorporated into major browsers. 16 years after mentioned publication Microsoft introduced HD Photo that compress images to the same ratio as in Marucci method published in 1990. And few years ago JPEG2000 anounced format that is about 7% better than Martucci suggested in 1990. The question is
Wouldn’t that be nice to incorporate new format into all browsers that require 300 lines of coding, so programmers will be able to generate images on the fly in web applications without using so called new approach of Microsoft that is called Silverlight.
Thanks,
Andrew Polar
June 14, 2008 2:11 am
According to my personal benchmarks, pigz doesn’t speed up compression, unlike pbzip. In fact, in some cases it’s performance may be slower than single-threaded version!
June 16, 2008 5:30 am
Girlfriend: What is your hardware/software setup?
October 19, 2008 4:50 pm
Version 2.0 of the parallel implementation of gzip called “pigz” is
available here:
http://zlib.net/pigz/
pigz exploits multiple processors and multiple cores for compression.
The use of threads has been completely rewritten to be more efficient
and clean. It compiles on Unixish operating systems that provide the
zlib and pthread libraries.
If you have success or failure in compiling and using pigz, I would
appreciate knowing about either way. Thanks.
October 21, 2008 3:05 am
Nice article. Thanks. :) Eugene
October 23, 2009 10:12 am
Does this run on Windows systems? Is the compression faster than something like gzip?
October 30, 2009 12:16 pm
You will need to compile it yourself (its just code but it should work on windows).
It will work 2 times faster by using both cores if your CPU is dual core. 4x faster if you have a quad core, but no speed up if you have a single-core CPU.
July 29, 2010 8:17 pm
You cant extract the resulting .tar file from http://zlib.net/pigz/ on an AIX server.
# tar -xvf pigz-2.1.6.tar.tar
tar: 0511-169 A directory checksum error on media; 0 not equal to 72695.
[root@nzaxs22ch-cl] : /opt/pigz
Are there limitations as to what O/S can use this?
July 29, 2010 8:37 pm
Ignore my last post.
The download randomly changed the filename to “tar -xvf pigz-2.1.6.tar.tar” when it should have been “tar -xvf pigz-2.1.6.tar.gz”
…. go figure
August 16, 2010 7:50 pm
Just wondering how I can compile pigz “statically” in fedora and then copy it to a Redhat machine - because RH still uses zlib version 1.2 but pigz requires 1.3.
July 22, 2011 2:46 am
Hi all,
I’ve compiled pigz 2.1.6 on AIX 5.3 but it won’t decompress gz files larger than 2GB. any idea how to enable large file support when compiling pigz on AIX?? I’ve not install gnu c compiler in it.. using the stock c compiler from aix.
Thanks.
January 30, 2012 1:15 pm
Artikel-dealer.de…
[...]PIGZ: Parallel GZIP - The Data Compression News Blog - c10n.info[...]…
January 31, 2012 12:20 am
News…
[...]PIGZ: Parallel GZIP - The Data Compression News Blog - c10n.info[...]…
February 3, 2012 9:25 pm
amd nvidia benchmark…
[...]PIGZ: Parallel GZIP - The Data Compression News Blog - c10n.info[...]…
May 22, 2013 12:59 am
Denn auf unserem Geschenke des weiteren Geschenkideen Blog präsentieren unsereins schöne Dinge
und auch Wohnkamine. Wir reisen umher und sehen kontinuierlich mal
was uns gefällt und das präsentieren wir hier. Freilich können Sie die meisten Produkte auch sofort
kaufen, doch ausschließlich über amazon, damit der
Einkauf sicher ist!