HTML5 has entered the online video market, which is both exciting and challenging for developers in the industry. With the HTML5 specification and the various browser implementations in constant flux, we at LongTail Video spend a signficant amount of time understanding the limitations of the technology and optimizing our own products for HTML5.
In developing the JW Player, we perform routine tests across the various browsers and devices to help determine the current state of online video. Our State of HTML5 Video Report is a compilation of our research and latest test results, focused on HTML5 Video playback. We are excited to share our findings with other developers/users in the industry as we explore just what HTML5 can and cannot support.
We have grouped our test results into the few topics we find to be the most critical for online video. We hope that you will benefit from our findings, and as always, look forward to feedback from the online video community.
1. Market Share of Browsers and Devices
We kick off this report with the aggregated data on the market share of each browser and device, and the modes they currently support. It is often difficult to get a solid snapshot of this, due to the major discrepancies between the two leading data sources, StatCounter and NetMarketShare. Market shares also vary greatly between different geographic locations.
Below is the StatCounter data for November 2011, combining all desktop and mobile devices with at least 1% of the market share:
2/3 of the market is already supporting HTML5. That being said, Flash is not obsolete. On the desktop, Internet Explorer 6/7/8 make up a large pecentage of the market share (28%), and are here to stay for at least a few more years. Since IE 6/7/8 do not support HTML5, alternatives like Flash remain critical for video playback. As for the other browsers, they already sufficiently support HTML5 video.
Mobile phones and tablets have emerged as a new category over the last few years. Currently, only the iOS and Android market shares are relevant. Both support HTML5 video. Android still supports Flash, but as announced recently, future phones will no longer include the plugin.
Connected TVs and set-top boxes are not yet a factor. Popular devices (XBox, PS3, Apple TV, Roku) have neither web browsers nor app markets. This may change in 2012 as Apple and Google roll out new products.
2. Media Formats
One of the biggest challenges with HTML5 is the fragmented support for audio/video formats. Here’s the current breakdown for HTML5 mode:
MP4 support may drop dramatically when Chrome officially drops MP4. Although this was announced back in January 2011, it is yet to happen, and thus harder to predict.
Both iOS and Android only support MP4 video. This will remain the case for any mobile device, until WebM decoders are built into hardware and integrated into phones; See the WebM blog for progress on that effort.
Every browser supports the <source> tag for loading multiple sources. Our tests show that including the type attribute prevents some preloading, but breaks compatibility with Android 2.2. Setting the codecs in the type attribute has no impact in any browser.
We have not included the Ogg video format in our tests. This format is not widely used and of lower quality than MP4 and WebM. Firefox 3.6, which is quickly phasing out, is the only browser version that supports Ogg but not WebM today (5% market share in December 2011).
3. Tag Attributes
The HTML5 video tag supports several attributes, and most are already supported to date across the various browsers and devices. Next to the width, height and src, these are:
The HTML5 video tag supports several attributes, and most are already supported to date across the various browsers and devices. Next to the width, height and src, these are:
Firefox has not rolled out support for loop yet, but otherwise the tag attributes will work on the desktop. Note that the brand new attribute muted is not yet widely supported, but we predict that soon it will be.
The mobile browsers ignore preload, autoplay and muted, but iOS 5 started supporting loop. Another difference between iOS 4 and 5 is that iOS 4 always preloads video using several range requests, while iOS 5 never preloads the video.
The video controls of each browser look different, but provide the same options: a play/pause toggle, a position slider and a volume slider. Safari provides 2 additional buttons: fullscreen and 30-seconds-back.
The video controls on mobile devices differ greatly from those on desktop browsers:
- On the iPad, controls are still quite similar, though there is no volume slider (volume is set with hardware buttons). Similar to Safari, there is a fullscreen button.
- On the iPhone, only a round play button in the middle of the poster is shown. When clicking, the video starts playing in fullscreen. When exiting fullscreen, the round play button reappears.
- On Android 2.2, no controls are displayed. This means a video can only be started using custom controls built in JavaScript. Like the iPhone, playback is fullscreen only.
- On Android 2.3, a control bar is displayed. The small play button must be clicked to trigger playback; clicking the poster does nothing. Though an improvement over Android 2.2, it still presents a major UX issue.
4. Fullscreen Playback
While a minor feature at first sight, fullscreen playback is essential to the success of HTML5 video. Fullscreen video enhances the visual experience and increases viewer engagement. HTML5 support for fullscreen playback is still in its infancy, as seen in this table below:
Fullscreen support using built-in controls varies across the various desktop browsers. For example, Firefox has a right-click menu item and Safari a controlbar button. For mobile, fullscreen is assorted based on particular device. The iPad has always featured a fullscreen toggle, and video playback on the iPhone/Android is by default fullscreen.
The W3C recently started working on a fullscreen API specification. Using this API, it is possible to render any HTML element in fullscreen, so custom controls can be added over/around a video. Safari and Chrome support this API in their latest versions, using the webkit vendor prefix. A similar implementation (using moz) will land in Firefox 10.
The aspect ratio of a user’s monitor is often different from that of the video element. Therefore, the ability to control how the video will fit to the screen is also important. With iOS, a built-in control is available to toggle between having the video contain and cover the screen. In Opera, the CSS3 object-fit property can be used to achieve the same. In other browsers, the video is always scaled to contain the screen.
5. Adaptive Streaming
Adaptive streaming is a core component of online video. It enables buffer control, mid-stream quality adjustments, live/dvr streaming and security through encryption and/or DRM. Adaptive streaming is not part of the HTML5 specification, but browsers can support it by loading manifests from an HTML5 tag. Currently iOS is the only platform with adaptive streaming support in HTML5:
Apple’s HTTP Live Streaming protocol is currently supported on Safari/iOS. Android has preliminary HLS support too, but in our experience it is quite buggy and very much in its infancy. MPEG DASH is the protocol developed by MPEG to standardize adaptive streaming. It is brand new and not yet supported by any browser.
In order to actually measure and/or influence adaptive streaming behavior, QOS metrics must be made available. Several proposals float around W3C/WhatWG, but to date only Firefox has vendor-specific support for the number of frames parsed, decoded and presented.
Note that every HTML5 browser supports seeking to not-yet-downloaded portions of the video by using HTTP 1.1 range-requests. This reduces the need for adaptive streaming, as it enables seeking in longer-form videos.
6. Accessibility
None of the browsers support full-featured accessibility in HTML5. Since HTML5 is native to browsers, in theory it can achieve a greater level of accessibility than plugins like Flash. In order to make a video accessible it must be controllable via the keyboard, and capable of rendering closed captions and audio descriptions. The latter are enabled through the HTML5 <track> element. The table below outlines the state of keyboard & text track support in HTML5 mode, across the various browsers and devices.
HTML5 Video elements can be controlled using the keyboard on IE, Firefox and Opera. IE/Firefox focus the entire video tag at once, using SPACE to play/pause the video, plus LEFT/RIGHT and UP/DOWN to control seeking and volume. In Opera, each individual control can be focused through tabbing.
Though in the process of being developed, HTML5 Text Track support has not yet landed in any browser. However, every browser has some type of beta version with preliminary <track> support. Once Text Track support lands in a production browser, we’ll update this section with more details and improved test cases.
The the HTML5 Video specification defines two alternative ways to playback closed captions – subtitles and descriptions with video elements. The first one leverages videos with multiple inline audio/video/text tracks. The second one leverages the synchronized playback of multiple <audio> or <video> elements. No browser has yet committed to implement either of these mechanisms.
