There are 4 Parallel bzip2 Implementations
Posted by Sachin Garg on 11th February 2006 | Permanent Link
A follow up to the recent pbzip2 update, I came across a few more similar solutions. None of these have been integrated (yet) in the official tree.
A Parallel (MPI) version, its also called pbzip2. This one doesn’t needs a multiprocessor or multicore machine. You can use multiple PCs on a network to collabrate with MPI. And there are two more SMP versions called BZIP2SMP and SMPBZIP2.
I wonder how all 4 of these compare to each other. I hope the benchmark maintainers are listening, they will like to take them for a test drive.
February 19th, 2008 at 3:52 pm
pbzip2 creates _slightly_ larger files then the others. bunzip2 can decompress files written by pbzip2 and vice-versa. However using pbzip2 to decompress pbzip2 compressed files is much much faster. pbzip2 was the fastest compared to bzip2smp and smpbzip2 - but sadly does not support reading from streams or sockets.
August 21st, 2008 at 9:17 am
I’d appreciate if somebody with strong benchmarking experience and good test cases evaluated lbunzip2.
August 24th, 2008 at 5:10 pm
Lacos, your work looks interesting. It would be really interesting to see comparisons of all these implementations.
I am hoping that as all are opensource, the best ideas can probably be mixed up to get an even better implementation.