Microsoft’s HD Photo not better than Jpeg 2000?
Posted by Sachin Garg on 23rd October 2006 | Permanent Link
Update: WMPhoto is now known as HD Photo, and might become Jpeg XR in future.
A report by Graphics and Media Lab at Moscow State University (MSU) concludes “WMP’s quality is quite average, and there are no signs of “revolution” in compression ratio” and that “some one-year old implementations of JPEG 2000 significantly outperform WMPhoto in objective and subjective comparison”.
But this is NOT the final word. Bill Crow, program manager for Windows Media Photo gives an interesting review of this comparison.
Microsoft introduced this new image format back in May 2006, and it was touted as a ‘Jpeg killer’. Both better image quality at better compression ratios, and faster performance were claimed. They didn’t talked too much about the newer Jpeg2000 standard but it was always mentioned by them in presentations and they ofcourse thought WMP is better.
The results put together by MSU researchers change the scene, making it clear that WMP isn’t better than JPEG2000 (atleast as far as quality is concerned).
In his review of this comparison, Bill Crow (program manager for Windows Media Photo) has raised some interesting and legitimate points.
Bill said:
I was a bit surprised with the range of quality differences among the different implementations of JPEG-2000. The design of basic Windows Media Photo compression algorithm doesn’t leave room for that big a range of differences among different implementations.
Admittedly, I too was surprised by this when I saw MSU’s first report comparing the Jpeg 2000 codecs in September 2005.
But what this means here is that Jpeg2000 being a flexible standard, leaves enough room for improvement and future Jpeg2000 implementations might be even better than this, but with WMP this is the best you will ever get.
His comment on all compression errors not being equal is also interesting. In the slide he presented at WinHEC, I can see what he says:
Windows Media Photo delivers a more random distribution of error from lossy compression, independent of the frequency of the image content. The end result is artifacts that are closer to random noise, and in many cases, this is significantly less damaging to an image, especially when it will be encoded/decoded multiple times.
This definitely makes sense, but the visual inspection results by MSU guys leave enough room for debate here too.
His last comment on the ‘validity’ of images used for comparison is again valid. I would myself want to see results on a more modern and more detailed set of images. Most popular images used in image compression research are very old and a upgrade will be very refreshing (I hope he publishes his test set soon).
However, although these images are old and have lesser resolution than that supported by modern cameras, they still have an acceptable level of both noise and detail. My opinion/prediction is that the newer test set won’t change the relative results too much. It might change the absolute ratings (ratio/quality/etc…) but I don’t expect the relative standings of Jpeg2000 and WMP to change much (I am ready to be proven wrong here).
But one question is still left unanswered among all this. While Microsoft has always claimed “low-complexity” and “performance” as advantages of WMP, I have not seen any hard results on how much better they are, compared to both JPEG and JPEG2000. Their presentations always mention that it is ‘faster’, but in forthcoming comparisons I expect to see some results telling ‘how much’.
With Jpeg2000 winning the ‘quality’ battle, performance is the only point left with WMP. It still remains to be seen how much better they are on that front (or if the choice of test set changes results).
October 24, 2006 3:27 am
A third party test set will be better. What if they include in it only such images on which their algorithm works better.
October 24, 2006 7:53 am
Yes, a third party test set will be better. Even better will be to have multiple public test sets from independent sources.
But till that happens, I will be happy if the first one comes from MS.
(In any case, intentionally or unintentionally, most algorithms are tuned towards the files commonly used for benchmarking)
October 27, 2006 1:32 am
It would be great too to get some ’social comparation’ results. Such as:
- license politics and costs;
- cpu and memory performance;
- number of platforms technology [will be] available on;
- losless versions compare.
November 4, 2006 1:15 am
Windows Media Photo…
Windows Media Photo - новый формат для хранения cжатых изображений, разработанный Microsoft. Утверждается, что этот формат обладает самым лучшим качест……
August 28, 2007 10:03 am
When image compression scientists are coming to the point of making scientifically proven test, they act as if they’ve never heard of statistics and hypotheses test. There is a big science about that. If corporation (or individual) A says that it achieved better result than B the hypothesis is formulated that says “B makes better compression than A”. The hypothesis must sound this way, not the other. Then statistician randomly collects image samples for testing (I say again randomly, it is important) at the size 30 images or more (less number is allowed but may raise questions and open debates). Then the test is conducted and recorded. Then mean and standard deviation is computed for results for A and B methods. Even if mean for A is really better this is not a prove yet. Then data are plugged into a formula (some standard error tables are used too) that tells what might be a probability that the result is obtained by chance. And if the probability of obtaining better result for A by chance is less than 5 percent the statistician conclude that “statement that B makes better compression than A” is not true. Sorry if someone see this as kind of weird logic, this is standard statistics that is used all over the planet. And it is level of community college not even university. I explained it here in some sort of common sense language, omitting details, tying to provide a concept so we can see that tests are not even close to what should be called scientific prove.
August 29, 2007 7:52 am
I quickly ran 5 different programs to compress kodak image set. The result is on http://www.ezcodesample.com/pre_image_reduction.html
Microsoft WMP is called there HD Photo. The program obtained from Bill Craw blog. I provided also for comparisson my own implementation of Marticci median adaptive predictor with arithmetic coding suggested in 1990. For those who did not see this predictor I can say that it is few lines of ‘c’ code. Arithmetic encoders are available for free I used one of them (it is not even mine). This program that gives few percent less compression than JPEG2000 and better than WMP could be written in 1990 and have size near 500 lines if we ignore format manipulation code.
August 29, 2007 10:05 am
You might also want to look a the effect of device noise on compression efficiency. We found that minimally lossless J2K encoding primarily strips out noise from the image. The result is much more controlled than the Median filtering Photoshop offers. Wavelets at work, I guess…
You can confirm this via image differencing using the Photoshop Calculation… command and looking at the resultant Histogram. Even with high end-scan backs, file sizes can drop noticeably, sometimes by more a factor of 3 over losslessly encoded images.
Wouldn’t device noise show up as low-order bit effects, with variation due to color (via combining the relative noisiness of R, G, & B)channels?