WebM vs. H.264: A Closer Look
In my tests, x264 produced slightly higher quality than the H.264 files produced by Squeeze with the Main Concept encoder, which was slightly better than WebM (Figure 4; click on the image for a larger version). In my view, however, slight differences in quality are irrelevant if the typical viewer wouldn't notice the difference absent side by side comparisons at normal data rates.
To explain, I produce my 720p test file at 800Kbps, which means that I'm allocating .029 bits per pixel in the 29.97 frame per second file. In comparison, YouTube produces their 720p H.264 video at about 2Mbps, which means an allocation of .072 bits per pixel, 2.5 times higher than mine. Why are my test files compressed so highly? Because if the data rate is high enough, all technologies look good and it's impossible to differentiate.
Figure 4. Three comparison images produced with x264, WebM and H.264 using the Main Concept codec (click image to see a larger version).
What's the most relevant test? In my recent review of video files produced by a range of broadcast and corporate sites, the lowest bits per pixel allocation that I found was 0.43, with most well above .07. Apple produces their iPad advertisements at .168 bits per pixel, about five times higher than my test file, while ESPN produces at .173 and CNN at .106. The notoriously penurious Tiger Woods publishes at .136, though perhaps he'll tighten this up after losing $750 million in his divorce.
Long story short, if YouTube produces their WebM-based 720p files at 2Mbps at 720p, only the most discriminating viewer will be able to distinguish it from H.264, and then, only with side by side comparisons, which, of course, viewers never have. It's not whether one technology is better than another; it's whether it's sufficiently better to make a difference for the typical viewer.
I should point out that some highly respected sources don't share this opinion. For example, the Graphics and Media Lab of Moscow State University produces a codec comparison every year; this year it included VP8. In terms of H.264 encoding quality, the report concludes that "The x264 encoder demonstrates better quality on average, and MainConcept shows slightly lower quality," which was my primary motivation for including x264 in this evaluation. Regarding VP8, the report concludes "When comparing VP8 and x264 VP8 also shows 5-25 lower encoding speed with 20-30% lower quality at average." I just didn't see that.
Then there's x264 developer Jason Garrett-Glaser's extensive analysis of VP8 and comparison to x264, which concludes that x264 is 28% better than VP8, though his comparisons seem to focus on 1080p delivery, so it's unclear how much you can generalize these results to streaming. In any event, Garrett-Glaser's analysis is wonderful reading for anyone who wants to understand the inner workings of the VP8 codec and WebM spec, as well as the patent issues that WebM may be facing.
Both these comparisons rely primarily on automated quality measurements like Peak Signal to Noise ratio (PSNR) and Structured Similarly (SSIM) which compare the encoded frame to the original and produce a comparative numerical score. I produce the files with the different technologies, making sure that they're within 5% of target data rate without dropped frames. I then grab frames for comparison purposes and watch the files side by side to assess the presence of motion artifacts. You can view my HD comparisons and my SD Comparisons—and comparative frame grabs—and draw your own conclusions.
Overall, I'm sure that Garrett-Glaser can coax more quality out of x264 than I can, but find comfort in the fact that the Moscow study concluded that MainConcept "shows slightly lower quality" than x264, which is consistent with my results. Certainly if you're using a MainConcept-based tool like Squeeze, the quality difference between VP8 and H.264 will be meaningless at most relevant data rates.
Related Articles
Google's attempt to clarify its decision to drop H.264 from Chrome in favor of WebM creates even more questions than it answers
17 Jan 2011
With Google's announcement that it's dropping H.264 support in Chrome in favor of WebM, it's time to start looking at the format. Here's a look at how to get the best WebM quality.
12 Jan 2011
With WebM, Google hasn't created any new revenue opportunities, opened any new markets or increased the size of the pie. They've just made it more expensive to get your share, all in the highly ethereal pursuit of "open codec technologies."
12 Jan 2011
Pay no attention to the man behind the Mac. HTML5 won't be a serious consideration for at least a few years.
02 Sep 2010
The standards body extended in perpetuity the royalty-free license on internet video that's free to users from 2015
26 Aug 2010
VP8 is now free, but if the quality is substandard, who cares? Well, it turns out that the quality isn't substandard, so that's not an issue, but neither is it twice the quality of H.264 at half the bandwidth. See for yourself.
20 May 2010