WebM vs. H.264: A First Look
Then there's x264 developer Jason Garrett-Glaser's extensive analysis (http://x264dev.multimedia.cx/?p=377#more-377) of VP8 and comparison to x264, which concludes that x264 is 28% better than VP8, though his comparisons seem to focus on 1080p delivery, so it's unclear how much you can generalise these results to streaming. In any event, Garrett-Glaser's analysis is wonderful reading for anyone who wants to understand the inner workings of the VP8 codec and WebM spec, as well as the patent issues that WebM may be facing.
B
oth these comparisons rely primarily on automated quality measurements such as Peak Signal to Noise ratio (PSNR) and Structured Similarly (SSIM), which compare the encoded frame to the original and produce a comparative numerical score. I produce the files with the different technologies, making sure that they're within 5% of the target data rate without dropped frames. I then grab frames for comparison purposes and watch the files side by side to assess the presence of motion artifacts. You can view my HD comparisons at www.doceo.com/HD_Comps. html and my SD comparisons at www.doceo.com/SD_Comps.html, and comparative frame grabs are at www.streaminglearningcenter.com. Draw your own conclusions.
Overall, I'm sure that Garrett-Glaser can coax more quality out of x264 than I can. But find comfort in the fact that the Moscow study concluded that MainConcept "shows slightly lower quality" than x264, which is consistent with my results. Certainly, if you're using a MainConcept-based tool such as Squeeze, the quality difference between VP8 and H.264 will be meaningless at most relevant data rates.
Playback Requirements
Which takes us neatly back to playback requirements, which is where we started. Let's introduce the computers and browsers.
Briefly, I tested on four computers. The MacBook Pro has a 3.06GHz Core 2 Duo CPU, 8GB RAM, and was running OS 10.6.2 with an NVIDIA GeForce 9600M graphic chip. The iMac has a 2.0GHz Core 2 Duo CPU, 2.5GB of RAM, and an ATI Radeon X1600 graphics chip while running OS 10.6.
The HP 8710w was running a 64-bit version of Windows 7 on a 2.2GHz Intel Core 2 Duo CPU with 2GB of RAM and an NVIDIA Quadro FX 1600M graphics controller. The Acer Aspire One is a netbook running Windows XP Home Edition with a 1.60GHz Intel Atom CPU, 1GB of RAM, and an integrated Intel 945 Express Chipset for graphics. All computers except the Aspire netbook were measured playing back the 720p file, and Table 3 shows the browsers used in the respective tests.
During testing, I followed this procedure:
• I turned off as many background processes as possible.
• I updated my graphics card drivers.
• First, I loaded the page and waited until the video completely downloaded (watching the little bar thingie on the bottom of the player).
• Next, I played the file, monitoring and recording the CPU usage on Windows Task Manager and percentage idle on the
Activity Monitor. Obviously, to compute CPU utilisation for the Macs, I subtracted the percentage idle from 100%.
Table 4 shows the results grouped by format and playback environment. Pulling some conclusions out of these disparate numbers is challenging, but here goes. Most obvious is the fractured nature of the HTML5 market, which still lacks a single codec that can play across all platforms. With Microsoft only supporting VP8 playback on IE 9 if the codec is otherwise installed, and Apple squarely in the H.264 camp, Google's open sourcing of VP8 has done nothing to reduce this logjam. As you have probably heard, Adobe has announced that a future version of the Flash Player will support WebM at some point; that may end up being the only easy way to display WebM video across all relevant browsers. However, one suspects that you could buy a Tastee Freez in Hades before WebM will play on the walled garden of Apple iDevices.
If you're a Mac owner who consumes lots of video, you can see that results vary by browser and if your Flash implementation is hardware accelerated. If it is, Safari is your best option; if not, go with Chrome. As a random thought, people watching Flash-video on a nonaccelerated Mac via Opera or Safari probably consider Flash a CPU hog-I wonder if their opinions would change if they did switch to Chrome. Ditto for HTML5-based H.264 playback on the Mac, where Safari is the efficiency king and Chrome the laggard.
Regarding WebM, on the platforms without Flash GPU acceleration (the iMac and Aspire), the most efficient WebM implementation required less CPU than the most efficient Flash implementation, though the WebM implementations varied more widely. Absent hardware acceleration, CPU utilisation on playback appears to be a wash.
As I mentioned at the beginning of this article, when playing a WebM 720p file on the HP 8710w via Media Player, CPU requirements dropped to 18%, about a third of the requirement of the most efficient browser and within spitting distance of GPU-accelerated Flash playback. On the Aspire for the 640x480 file, CPU load dropped to 24%; 30% with hardware acceleration disabled, which is much lower than any other tested technology. If any of the browser vendors can enable this level of efficiency for web-based playback, WebM would have a significant competitive advantage over H.264, whether Flash- or HTML5-based.
If not, given that many newer computers and mobile devices offer some measure of Flash acceleration, WebM may trail in quality, encoding speed, and playback efficiency.
Analysis
As it stands today, WebM's value proposition is basically that it's free and better than Ogg Theora; though it's way too tough to decode as compared to the rapidly expanding base of GPU-accelerated H.264. If the browser vendors and Google can reduce CPU playback requirements to levels shown in Media Player, however, the story changes considerably, at least for pay-per-view and subscription-based video distributors, who currently pay a fee to deliver H.264 video. Ironically, given the lack of universal HTML5 browser support, the most efficient way to distribute WebM to your customers may be via the Flash Player, assuming that the Flash Player's CPU playback requirements are competitive. You'll probably still have to write cheques to MPEG-LA for delivering to Apple's iDevices, however.
As you probably know, H.264 is royalty-free for free internet distribution, at least until 2015, so there's no financial incentive to switch. If your organisation wants to migrate toward HTML5, WebM doesn't provide that single codec solution, is still lower quality than H.264 (however small the difference), and takes longer to encode. Overall, for those not charging for their video, H.264 is still a better solution, and given the rapidly increasing size of the GPU-accelerated installed base, it will likely remain so, unless and until Google creates distribution channels that you can't access with H.264.