The Data Compression News Blog

All about the most recent compression techniques, algorithms, patents, products, tools and events.

Subscribe

Posts: RSS Feed
Comments: RSS Feed

Recent Posts

  • Google Snaps Up On2 (8 Comments)

  • Bijective BWT (34 Comments)

    David Scott has written a bijective BWT transform, which brings all the advantages of bijectiveness to BWT based compressors. Among other things, making BWT more suitable for compression-before-encryption and also give (slightly) better compression.

  • Asymmetric Binary System (170 Comments)

    Jarek Duda’s “Asymmetric Binary System” promises to be an alternate to arithmetic coding, having all the advantages, but being much simpler. Matt has coded a PAQ based compressor using ABS for back-end encoding. Update: Andrew Polar has written an alternate implementation of ABS.

  • Precomp: More Compression for your Compressed Files (5 Comments)

    So many of today’s files are already compressed (using old, outdated algorithms) that newer algorithms don’t even get a chance to touch them. Christian Schneider’s Precomp comes to rescue by undoing the harm.

  • On2 Technologies is Hiring (3 Comments)

    There aren’t too many companies working on cutting edge codecs, and of those few this one is hiring. Best of luck.

There are 4 Parallel bzip2 Implementations

Posted by Sachin Garg on 11th February 2006 | Permanent Link

A follow up to the recent pbzip2 update, I came across a few more similar solutions. None of these have been integrated (yet) in the official tree.

A Parallel (MPI) version, its also called pbzip2. This one doesn’t needs a multiprocessor or multicore machine. You can use multiple PCs on a network to collabrate with MPI. And there are two more SMP versions called BZIP2SMP and SMPBZIP2.

I wonder how all 4 of these compare to each other. I hope the benchmark maintainers are listening, they will like to take them for a test drive.

3 Responses to “There are 4 Parallel bzip2 Implementations”

  1. Antonio Says:

    pbzip2 creates _slightly_ larger files then the others. bunzip2 can decompress files written by pbzip2 and vice-versa. However using pbzip2 to decompress pbzip2 compressed files is much much faster. pbzip2 was the fastest compared to bzip2smp and smpbzip2 - but sadly does not support reading from streams or sockets.


  2. lacos Says:

    I’d appreciate if somebody with strong benchmarking experience and good test cases evaluated lbunzip2.


  3. Sachin Garg Says:

    Lacos, your work looks interesting. It would be really interesting to see comparisons of all these implementations.

    I am hoping that as all are opensource, the best ideas can probably be mixed up to get an even better implementation.