Comments for The Data Compression News Blog

Comment on Scientific-Atlanta Expands Advanced Digital Video Compression Technology Portfolio With Acquisition of UB Video Inc. by Earl Colby Pottinger

Earl Colby Pottinger — Thu, 13 Nov 2008 03:37:48 +0000

Anyone have any idea how much money was involved?

Comment on Bijective BWT by David A. Scott

David A. Scott — Tue, 21 Oct 2008 16:54:50 +0000

There is an article at http://arxiv.org/PS_cache/cs/pdf/0502/0502073v1.pdf that is very close to talking about the indexless BWT transform. It may be worth reading if one is interested in that sort of thing.

Comment on PIGZ: Parallel GZIP by Eugene

Eugene — Tue, 21 Oct 2008 10:05:42 +0000

Nice article. Thanks. :) Eugene

Comment on PIGZ: Parallel GZIP by Mark Adler

Mark Adler — Sun, 19 Oct 2008 23:50:48 +0000

Version 2.0 of the parallel implementation of gzip called “pigz” is
available here:

http://zlib.net/pigz/

pigz exploits multiple processors and multiple cores for compression.
The use of threads has been completely rewritten to be more efficient
and clean. It compiles on Unixish operating systems that provide the
zlib and pthread libraries.

If you have success or failure in compiling and using pigz, I would
appreciate knowing about either way. Thanks.

Comment on My Very Bad Euclid Discoveries Experience by Earl Colby Pottinger

Earl Colby Pottinger — Sat, 18 Oct 2008 16:30:11 +0000

Oh,yes. I would also like to add, my main OS is BeOS/Haiku, my girl-friend’s OS is XP, my ex-girl-friend’s OS is Vista, my best friend’s OS is Linux.

At any time I can plug this TB drive into any of these systems to read/write files for exchange and sharing with ZERO SOFTWARE issues.

All of these so-called super compressors claim support for only a limited number of OSes and expect one to pay for a copy for each OS. Not to mention all the possible software conflicts to deal with.

So again, who needs these sky-in-the-pie systems when we have workable plug and play systems right now?

Comment on My Very Bad Euclid Discoveries Experience by tardo

tardo — Sat, 18 Oct 2008 16:16:35 +0000

Apparently Euc. has sent out new hype investor letter trying to settle investors. Obvious that a select few are giving the co.s talking points for the week or month.Others are to chime in how great euc. is.If thsi pos fails pumpers will have to answer for it from state,fed authorities.

Comment on My Very Bad Euclid Discoveries Experience by Earl Colby Pottinger

Earl Colby Pottinger — Sat, 18 Oct 2008 00:16:12 +0000

I would just like to point out in reading their latest flyer that Business Depot is now selling 1 TB size external drives now.

And I only have 360 GB in files on my computers, why did I need Euclid’s software again?

Comment on My Very Bad Euclid Discoveries Experience by Bardo

Bardo — Fri, 17 Oct 2008 12:50:54 +0000

What a relevant date for ‘helpisontheway’s’ message.

Comment on This is what Google uses for compression by Google, Bigtable, Compression, Zippy and BMDiff « Kevin Burton’s NEW FeedBlog

Mon, 13 Oct 2008 04:41:58 +0000

[...] I remembered some notes about compression in the original Bigtable paper and decided to dig a bit deeper. [...]

Comment on Another Failed Compression Promise: NearZero by Stevo

Stevo — Mon, 06 Oct 2008 23:20:32 +0000

Thanks for the offer Jessie, I’m just another disgruntled ex-employee who was promised the earth and left short when wages wern’t paid.

Comment on Asymmetric Binary System by Jarek

Jarek — Wed, 01 Oct 2008 13:32:17 +0000

To visualize this connection, let’s think about theoretical limit of bits of redundancy we have to add for bit of information for assumed statistical error distribution to be able to full correct the file.
To find this threshold, let’s think about simpler looking question: how many information is stored in such uncertain bit?
Let’s take the simplest error distribution model - for each bit probability that it’s switched is equal e (near zero), so if we see ‘1′ we know that with probability 1-e it’s really ‘1′, and with probability e it’s 0.
So if we would know which of this cases we have, what is worth h(e)=-e lg(e) - (1-e) lg(1-e),we would have whole bit.
So such uncertain bit is worth 1-h(e) bits.
So to transfer n real bits, we have to use at least n/(1-h(e)) these uncertain bits - the theoretical limit to be able to read a message is (asymptotically)
h(e)/(1-h(e)) additional bits of redundancy /bit of information.

So a perfect data correction coder for e=1/100 error probability, would need only additional 0.088 bits/bit to be able to restore message.
Hamming 4+3 instead of using additional 0.75 bits/bit, looses 16bits/kilobyte with the same error distribution
… we see that there is a lot to do…

Comment on Asymmetric Binary System by Jarek

Jarek — Wed, 01 Oct 2008 11:57:27 +0000

I’ve just realized that Hamming and tripling bits are some special cases of ANS based data correction :)
They are kind of degenerated cases because the intermediate state after bit transfer, has only one possibility, making the blocks independent.

If it could have many values, we would transfer some redundancy - the ‘blocks’ would be somehow connected and if in one of them would occur more errors, we could use this connection to see that something is wrong and use some unused redundancy from succeeding blocks to correct it - we use the assumption that according to error distribution, the succeeding bits are with large probability correct.
It correspond better to the real probability distribution of errors than Hamming, in which all 8 possibilities - 7bits were correct or one of bits were changed - are treated equally … and we are forgetting about the possibility of more errors.

I apology for putting last posts on this blog (more information is on the linked forum), but I’ve realized that data compression and correction are highly connected.
In the first we want to remove some statistical, difficult to practical use redundancy …
in the second we want to distribute uniformly some deterministic, easy to use redundancy …
And in both cases, the closer to the real statistics we are - of symbols or errors, the closer to the optimal coding we are - the less bit we need …

Comment on Asymmetric Binary System by Jarek

Jarek — Mon, 29 Sep 2008 08:19:18 +0000

I’ve just realized that we can use huge freedom of choice for the functions for ANS to improve latency of data correction - we can make that if the forbidden symbol occurs, we are sure that if there was only single error, it was among bits used to decode this symbol.
The trick is that the forbidden symbol usually dominate in the coding tables, so we can make that if for given transferred bits we would get allowed symbol, for each sequence differ on one bit (Hamming distance 1) we would get the forbidden symbol.
I’m describing it on the forum I’ve linked.

Comment on Another Failed Compression Promise: NearZero by Bookmarks about Fraud

Bookmarks about Fraud — Sat, 27 Sep 2008 04:00:07 +0000

[...] - bookmarked by 4 members originally found by djretcon on 2008-09-10 Comment on Another Failed Compression Promise: NearZero by John… http://www.c10n.info/archives/579 - bookmarked by 5 members originally found by Kaiafas00 on [...]

Comment on Asymmetric Binary System by Jarek

Jarek — Fri, 26 Sep 2008 21:28:40 +0000

I’m sorry for the link, I’ve just noticed it has changed, here is the correct one:
http://www.thescienceforum.com/Data-correction-methods-resistant-to-pessimistic-cases-13416t.php