Yeah
Posted by Mark Nelson on 5th April 2006 | Permanent Link
Man, it’s depressing to see make people make data compression claims that aren’t benchmarked. They end up looking like clowns, and all they do is stir up the hoi polloi. The most recent stupidity was first reported on Slashdot, referencing a blog entry titled Coolest Product at Storage Networking World (so far). Oh guess what, they backup your data with 25:1 compression.
I just want to say, hey d00dz, backup this at 25:1, and until you manage that, please STFU.
And shame on you, StorageMojo for being a chump and broadcasting this bullshit. A pox on you and your house.
Don’t even get me started on Euclid Discoveries.
April 5th, 2006 at 6:11 pm
I left Slashdot after yet another perpetual motion story. I don’t think you need to debunk anything just because it’s posted there.
April 6th, 2006 at 12:41 am
Mark, this time there is no talk about data compression claims. What the author at StorageMojo (shame on him/her!) wrote is confusing the term compression instead of data reduction.
Diligent claims are rather in the sense of optimization techniques applied to the data set to be stored, and that belongs to another category where several product already exists, at least in the WAN arena.
April 10th, 2006 at 12:11 am
I have actually written a new whitepaper proving a new means of lossless compression. The basic maths are outside the normal scope of math but it does work.
A serious amount of compression can be achieved via cycling on random binary data. The odds are incredibly high against a random occurence that can stop the compression, and those are extremely easy to handle, they compress via other means quite easily.
My url talks about it Security1
There is an open challenge to disprove my claims, via actually using my created methods. There is also an open reward for the first to create functional software.
April 10th, 2006 at 8:02 am
Michael Harrington, a pox on thee!
You have two strikes against you.
1) “The basic maths are outside the normal scope of math”. There is no such thing as being “outside the normal scope of math”, this is just BS talk, total garbage.
2) “An open reward for the first to create functional software”. This only means that your so-called white paper is senseless crap that has nothing even resembling an algorithm in it.
May 6th, 2006 at 12:41 pm
Mike,
I wrote the blog post you excoriate. I appreciate the link, if not the sentiments as I am still getting a steady stream of visitors from your site, presumably to gratify their sense of superiority. Or maybe they just want to learn something about a compression.
I think it is fine for professionals to use a specialized vocabulary to make subtle points - that is the mark of a professional, after all. But that doesn’t mean it is right for the general IT buying public.
Data Domain, Diligent and others have a very cool technology that actually works to dramatically reduce the size of backups, making disk to disk backup economically feasible. Which, given the flakiness of tape backup, is a Very Good Thing. But they are having a hard time communicating that fact because, being very well versed in compression technology themselves, they refuse to use the word that your average IT manager would understand: compression.
This is why techies need smart marketing people: to get them to do what *works* instead of what is *correct* and *doesn’t* work. Dominated by smart techies, neither company is doing what is takes to really sell their product.
No one barfs over calling (maybe you do, though) MPEG-4 “compression” even though it is functionally equivalent to what DD, Diligent, et.al. do. I understand that MPEG-4 is essentially a toolkit of techniques optimized for certain scenarios, and not Shannon-style compression. And guess what, I and the other millions of people using it don’t care.
Check out the last post of mine on the subject. I welcome your comments.
Sincerely,
Robin Harris
May 6th, 2006 at 1:29 pm
well, my comment was unnecessarily harsh, but I sometimes get a bit overwhelmed by the level of scamming in PR on this topic.
Your scrutiny is correct, these guys are saying that they use existing data as a dictionary to compress files.
However, their claim of 25:1 is bogus in this respect: if I create a new word document, database, spreadsheet, or whatever, and then have it backed up, their system will not back it up at 25:1 ratios.
The way they cook up numbers like this is by the misleading notion that they might be able to backup the next version of that file at 25:1.
True enough, but not that remarkable, and it is definitely not going to reduce your backup storage requirements by 25. The interesting quesiton is how much your storage requirements will be reduced in comparison to a normal backup system using file timestamps and zip-like compression. I suspect the results would not be phenomenal.
May 6th, 2006 at 1:31 pm
And let me add, my first comment was that they should be benchmarking their claims, and I stand by that. With no reasonable benchmarks, their claims are worthless.
May 8th, 2006 at 10:19 am
Are there benchmarks for this kind of compression? Check out Data Domain’s pretty good pdf whitepaper page 14 where they state: The first full backup results in 3-4x data reduction. File level incrementals will see 6-7x and subsequent full backups will see 50-60x data reduction ratios. The aggregate with Weekly Fulls and Daily Incrementals is 20x.
If it doesn’t exist it would be useful to create such a benchmark because, if it works, this technology is going to be very popular.
May 8th, 2006 at 11:35 am
Just to give you an idea of how unimpressive this might be, the backup program I use on my PC reported this when I last ran on Friday night:
5/5/2006 5:29:21 PM Backup completed. 2001 of 2001 files securely backed up. Total Backup Size: 4.74 GB, Optimized Backup Size: 323.5 MB, Compressed Backup Size: 241.8 MB.
Now, that’s 20:1 compression, right?