The Data Compression News Blog

All about the most recent compression techniques, algorithms, patents, products, tools and events.

Subscribe

Posts: RSS Feed
Comments: RSS Feed

Recent Posts

  • Google Snaps Up On2 (3 Comments)

  • Bijective BWT (29 Comments)

    David Scott has written a bijective BWT transform, which brings all the advantages of bijectiveness to BWT based compressors. Among other things, making BWT more suitable for compression-before-encryption and also give (slightly) better compression.

  • Asymmetric Binary System (170 Comments)

    Jarek Duda’s “Asymmetric Binary System” promises to be an alternate to arithmetic coding, having all the advantages, but being much simpler. Matt has coded a PAQ based compressor using ABS for back-end encoding. Update: Andrew Polar has written an alternate implementation of ABS.

  • Precomp: More Compression for your Compressed Files (5 Comments)

    So many of today’s files are already compressed (using old, outdated algorithms) that newer algorithms don’t even get a chance to touch them. Christian Schneider’s Precomp comes to rescue by undoing the harm.

  • On2 Technologies is Hiring (1 Comment)

    There aren’t too many companies working on cutting edge codecs, and of those few this one is hiring. Best of luck.

There are 4 Parallel bzip2 Implementations

Posted by Sachin Garg on 11th February 2006 | Permanent Link

A follow up to the recent pbzip2 update, I came across a few more similar solutions. None of these have been integrated (yet) in the official tree.

A Parallel (MPI) version, its also called pbzip2. This one doesn’t needs a multiprocessor or multicore machine. You can use multiple PCs on a network to collabrate with MPI. And there are two more SMP versions called BZIP2SMP and SMPBZIP2.

I wonder how all 4 of these compare to each other. I hope the benchmark maintainers are listening, they will like to take them for a test drive.

3 Responses to “There are 4 Parallel bzip2 Implementations”

  1. Antonio Says:
    February 19, 2008 3:52 pm

    pbzip2 creates _slightly_ larger files then the others. bunzip2 can decompress files written by pbzip2 and vice-versa. However using pbzip2 to decompress pbzip2 compressed files is much much faster. pbzip2 was the fastest compared to bzip2smp and smpbzip2 - but sadly does not support reading from streams or sockets.


  2. lacos Says:
    August 21, 2008 9:17 am

    I’d appreciate if somebody with strong benchmarking experience and good test cases evaluated lbunzip2.


  3. Sachin Garg Says:
    August 24, 2008 5:10 pm

    Lacos, your work looks interesting. It would be really interesting to see comparisons of all these implementations.

    I am hoping that as all are opensource, the best ideas can probably be mixed up to get an even better implementation.


This entry was posted on Saturday, February 11th, 2006 at 5:54 am by Sachin Garg. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.