The Data Compression News Blog

All about the most recent compression techniques, algorithms, patents, products, tools and events.

Subscribe

Posts: RSS Feed
Comments: RSS Feed

Recent Posts

  • Google Snaps Up On2 (8 Comments)

  • Bijective BWT (34 Comments)

    David Scott has written a bijective BWT transform, which brings all the advantages of bijectiveness to BWT based compressors. Among other things, making BWT more suitable for compression-before-encryption and also give (slightly) better compression.

  • Asymmetric Binary System (170 Comments)

    Jarek Duda’s “Asymmetric Binary System” promises to be an alternate to arithmetic coding, having all the advantages, but being much simpler. Matt has coded a PAQ based compressor using ABS for back-end encoding. Update: Andrew Polar has written an alternate implementation of ABS.

  • Precomp: More Compression for your Compressed Files (6 Comments)

    So many of today’s files are already compressed (using old, outdated algorithms) that newer algorithms don’t even get a chance to touch them. Christian Schneider’s Precomp comes to rescue by undoing the harm.

  • On2 Technologies is Hiring (4 Comments)

    There aren’t too many companies working on cutting edge codecs, and of those few this one is hiring. Best of luck.

18 Years of ZIP format: Happy Birthday

Posted by Sachin Garg on 19th July 2006 | Permanent Link

The ZIP format celebrated its 18th birthday recently. Considering the fact that it was designed in times when disk capacities were measured in 10s of megabytes and network bandwidth was 300-1200 bauds, it sure has come a long way.

ZIP format’s single major strength was open specifications which resulted in remarkable stability and compatibility over all these years. Large file support was the only change needed (made to have ZIP files larger than 2 GB in size).

The ZIP file format was originally created in 1989 by Phil Katz, founder of PKWARE, after a prolonged legal dispute between PKWare and System Enhancement Associates (SEA) over the trademark name “ARC” (short for “Archive”), the file name extension .arc and copyright issues over SEA’s published code. (The name zip (meaning speed) implied that their product would be faster than ARC and other compression formats of the time).

Soon open source implementation of Phil Katz’s “deflate” and “inflate” routines was released by the Info-ZIP project under a BSD license. This resulted in a horde of PKZIP imitators, establishing the ZIP file format as a defacto industry standard.

In the mid 1990s, as GUIs became more popular, WinZip became popular by pitching a graphical user interface. And in the late 1990s, various file manager software products started integrating support for the ZIP format into the file manager user interface (Windows Explorer, the Mac OS Finder, GNOME, KDE, and others).

Mr. Katz died of complications from chronic alcoholism in 2000. A sad end to a true pioneer in the field of data compression.

Thinking about all the advancements made in data compression algorithms research, it doesn’t feels good to know that the solution used by most people is so way behind the current state-of-the-art.

But then many more recent formats/algorithms are trying to hit it big, but haven’t been very successful, primarily due to lack of open specifications. The next best open-source option, bzip2, has gained popularity but couldn’t displace ZIP as it is considerably slower than ZIP’s Deflate algorithm.

However the ZIP format has also been splintering recently. In 2003, both PKWare and WinZip introduced their own ‘incompatible’ encryption extensions to format. More recently, WinZip and PKWare added new algorithms to ZIP format. PKWare introduced Deflate64, and WinZip incorporated bzip2 and ppmd as algorithm options in ZIP archives. This splintering of format, with more than one incompatible variations going around, might prove to be fatal.

These sugar coated ‘advancements’, which result in incompatible archives, were probably just an attempt to use the ZIP format’s brand value. Anyway, most users continue to use the plain old ZIP for reasons of compatibility.

These attempts to hack newer technology in the aging ZIP format clearly show that the format is, well, showing age. But there is still no doubt to believe that it will still go on strongly for many more years to come, for same reasons which have maintained it as an obvious choice till now.

31 Responses to “18 Years of ZIP format: Happy Birthday”

  1. Krisjohn Says:

    Note that the ARC issue was not a trademark dispute. SEA claimed that Katz ripped off their source code with no attribution. Katz subsequently framed the battle as a “David and Goliath” fight which resulted in a somewhat upsetting (and ill-informed) boycott. The BBS Documentary ( http://www.bbsdocumentary.com/ ) covers this in the last episode as best as it can given that Phil Katz is no longer with us.


  2. Sachin Garg Says:

    Yes, I have read about that “big faceless corporation crushing a small guy” image which Katz gave to SEA vs PKWare case, when actually both were same sized companies. An interesting move.

    Another interesting bit is that the same reasons why ZIP became the defacto standard (open-specifications and open-source Info-ZIP implementation) resulted in PKWare’s inability to make much profit from it.


  3. Sandman Says:

    Is it really a surprise that money can’t be made by creating something open and free?

    and which also doesn’t needs support ;-)


  4. An old guy Says:

    It wasn’t just an ill-informed boycott. Not many people care about ethical issues behind technology. Think Linux and Microsoft, its just a small loud minority which says it cares.

    ZIP was technically superior to ARC.


  5. Informed Says:

    Actually Phill died in 2000 and his company PKWARE is going strong, its now doing data encryption using zip files and is growing fast. PKWARE still controls the App note for zip so any changes that are made come from them. But i agree phill won the PR battle.

    Interesting note, the computer that Phil used to write the ZIP standard on (deflate and inflate routines) still exsists and I have seen it, still looks good. Its referred to as the “Mexican” because Phil bought the thing from a BBS out of Mexico and had it shipped to Milwaukee.


  6. Mike Says:

    Instead of what happened 18 years ago, shouldn’t we be more worried about what PKWare and WinZip are doing to the format now?


  7. Mike Says:

    But when most users are using WinZip, does controlling the appnote makes any difference?


  8. Sachin Garg Says:

    You may find this page by ARC author interesting

    http://www.esva.net/~thom/philkatz.html

    ps. I have fixed the date in article above.


  9. Informed Says:

    Mike,

    You have to remember that winzip operates on a windows platform as well as PKZIP, there are numerous other platforms that the world runs on, UNIX, Linux, I5OS mainframes, and Z series IBM Mainframes and all of them use the .zip standard..winzip has no product for them while PKWARE does…In the case of Mainframes its absolutly essiential when you talk about the size of files they use there, i.e. hundres of Gigabytes and Terabytes. Also many applications use the deflate routine inside hidden from you and I.

    So calling the appnote in the correct way is essitential to making sure if I zip a file somewhere on some platform and you unzip it on your desktop with winzip, it will work. Also think about how many .zip archives are out there right now. How big of a job would it be to go back and change them to some new compression format?


  10. Krisjohn Says:

    While I huge proportion of Windows users use WinZip, not nearly enough of them use the latest version with the PPMd format.


  11. freeko Says:

    actually the deflate/inflate routine in many formats compression/image/doc.. (like in 7-zip,tif,png,svgz,all open-office formats,wmz,..)


  12. A. Hacker Says:

    While zip is a great format, this splintering will hopefully cause a newer and still free format to be created for the future. The 4GB limitation is becoming more troublesome.

    I’d also like to see LZMA and some of the other higher performance algorithms incorporated. unit or solid compression would be kind of nice as an option in certain cases


  13. Evangelist Says:

    ZIP is faster than most modern algorithms. This is why nothing ‘modern’ could replace ZIP.

    It’s old but still has its advantages.


  14. Sachin Garg Says:

    Another Story discussing PKWare’s focus on “securing” the ZIP format.


  15. Sum Yung Gai Says:

    I started with PKXARC back in 1985, and I remember this history well. There’s another reason why PKXARC was so much faster than SEA ARC, and that’s because SEA ARC did *everything* on disk, but PKXARC did a lot of it in DRAM. *That* was the major optimization trick. Remember, back then in 1985, most people couldn’t afford hard disks; everyone had floppies, and floppies are about as fast as glacial ice in the North Pole.

    Anyway, later, I went to Info-ZIP on OS/2 v2.1 back in 1993, when I wanted to use native OS/2 tools. I used Info-ZIP OS/2 on Windows NT (3.5, 3.51, and 4.0) for years, since it worked so well on that platform, too.

    Now, I use zip/unzip on GNU/Linux, and it’s still working just great. The ZIP format may be old, but it still works, is still relevant, and is used on just about any platform that you can think of. With ZIP, I don’t have to worry about whether or not Platform X has support. I can distribute anything that I want in a .ZIP archive, and anybody can use it.

    If ever there were an example of why truly open file formats are necessary, ZIP is its poster child. You hear that, Microsoft?


  16. antonio Says:

    hi my zipfile option on my desktop doesnt work at all anymore, like ill go to my desktop and right click. ill go to new and new zip file wont be there. where did it go?? i really need it

    if you are going to answer my question please email me at fo_shizzle655@yahoo.com

    thank you


  17. Coobegreeri Says:

    I lost my job right on my birthday with a greeting card. I heard about people loosingt they jobs - but I didn’t think it’s going to happened with me. I start looking for a new job. I got some money for now and I pretty good worker so I don’t think that going to be a big problem. I got couple offers so I think Life come back in shape.


  18. Pourquoi je hais le format zip. « No-life Says:

    [...] Non pas parce que le logiciel le plus couramment employé pour en générer, Winzip, a fabriqué dans son coin un modèle incompatible après des années de paix http://www.c10n.info/archives/430. [...]


  19. Best running shoes 2014 Says:

    Thank you for the good writeup. It in fact was
    a amusement account it. Look advanced to
    more added agreeable from you! By the way, how could we communicate?


  20. top food processors 2014 Says:

    Valuable information. Lucky me I discovered your web site by accident, and
    I am stunned why this accident did not took place earlier!
    I bookmarked it.


  21. bestcrossbowreviews2014.net Says:

    I just like the helpful info you provide on your articles.
    I will bookmark your blog and test once more here regularly.
    I am relatively certain I’ll be told a lot of new stuff right
    here! Best of luck for the next!


  22. hair dryer reviews 2014 Says:

    I must thank you for the efforts you’ve put in penning this site.
    I really hope to view the same high-grade blog posts from you later on as well.
    In fact, your creative writing abilities has
    motivated me to get my own blog now ;)


  23. bestfooddehydratorreviews2014.com Says:

    Heya! I just wanted to ask if you ever have any problems with hackers?

    My last blog (wordpress) was hacked and I ended up losing many months of hard work due to no
    backup. Do you have any solutions to protect against hackers?


  24. Best lawn mower 2014 Says:

    These are actually impressive ideas in about blogging. You have touched some fastidious factors here.

    Any way keep up wrinting.


  25. Best home theater receiver 2014 Says:

    With havin so much content and articles do you ever run into any problems of plagorism or copyright
    infringement? My website has a lot of exclusive content I’ve either created myself or outsourced but it seems a lot
    of it is popping it up all over the internet without my agreement.
    Do you know any solutions to help stop content from being stolen? I’d certainly appreciate
    it.


  26. Bestjuicersreviews2014.net Says:

    Hello Dear, are you truly visiting this site daily, if so then you
    will without doubt take fastidious knowledge.


  27. best dslr camera black friday deals Says:

    It’s fantastic that you are getting thoughts from this piece of writing as
    well as from our dialogue made here.


  28. best gas grill 2014 Says:

    Very good information. Lucky me I found your website by accident
    (stumbleupon). I have saved it for later!


  29. chloe sims best hair Care Products for fine hair Says:

    ve booked your holiday, filled in your EHIC card and paid for your travel insurance.

    A tanning lotions is their hottest selling product and provides you with amazing results within a few applications
    for short time periods. I have included the link below if you would like to read the study.

    my web blog: chloe sims best hair Care Products for fine hair


  30. best mattresses 2014 Says:

    Appreciation to my father who shared with me on the topic of this
    web site, this web site is truly amazing.


  31. hcg weight loss One month Says:

    While you are in the diet phase where you are taking your sublingual
    h - CG, you will need to be much disciplined in order to get the results you want.
    Under typical situations, our system is rather responsive.

    Could the HCG shot be the secret to losing weight and keeping it off.

    Here is my site … hcg weight loss One month