The commercial media industry is undergoing a major transition as content providers move away from proprietary web plug-in based delivery mechanisms (such as Flash or Silverlight), and replace them with unified plug-in free video players that are based on HTML5 specifications and commercial media encoding capabilities. Browsers are moving away from plug-ins as well, as Chrome is with NPAPI and Microsoft Edge is with ActiveX, and toward more secure extension models.
The transition to plug-in free media has been enabled through the recent development of new specifications:
- From W3C: Media Source Extensions for adaptive streaming and Encrypted Media Extensions for content protection.
- From the Moving Picture Experts Group (MPEG): DASH and Common Encryption (CENC).
These specs were designed and developed to enable interoperable streaming to a variety of media platforms and devices. By focusing on interoperable solutions, content providers are able to reduce costs and at the same time users are able to access the content they want on the device they prefer using the app or web browser of their choice. Microsoft believes that this is a huge benefit to both content producers and consumers, and is committed to supporting companies that make this transition.
This is a long blog, and we don’t want you to miss a topic that interests you. Here’s a glimpse of what we’ll cover:
- Some information on Microsoft Edge and Silverlight
- An overview of interoperable web media.
- Some challenges and options to address them:
- The simplest form of DASH streaming.
- A website demo that uses a library to play Smooth content.
- Services from Azure Media Services that can help.
- A simple method for creating a Universal Windows Platform (UWP) app based on website code.
- A demo UWP that integrates video playback with Cortana voice commands.
Microsoft Edge and Silverlight
Support for ActiveX has been discontinued in Microsoft Edge, and that includes removing support for Silverlight. The reasons for this have been discussed in previous blogs and include the emergence of viable and secure media solutions based on HTML5 extensions. Microsoft continues to support Silverlight, and Silverlight out-of-browser apps can continue to use it. Silverlight will also continue to be supported in Internet Explorer 11, so sites continue to have Silverlight options in Windows 10. At the same time, we encourage companies that are using Silverlight for media to begin the transition to DASH/MSE/CENC/EME based designs and to follow a single, DRM-interoperable encoding work flow enabled by CENC. This represents the most broadly interoperable solution across browsers, platforms, content and devices going forward.
Interoperable Media across Browsers
Plug-ins like Silverlight were intended to support interoperable media by providing versions for every browser they supported. This became more difficult as devices and platforms that support browsers multiplied. Now, as the old plug-in models are being removed, replacements for them are needed. For media, a great forward looking replacement can be based on DASH, MSE, EME and CENC.
Windows 10 and Microsoft Edge support DASH, MSE, EME and CENC natively, and other major browsers ship implementations of MSE and CENC compliant EME. This support allows developers to build plug-in free web video apps that runs across a huge range of platforms and devices, with each MSE/EME implementation built on top of a different media pipeline and DRM provider.
DRM Providers Can Differ by Browser (Click to enlarge)
In the days when DRM systems used proprietary file formats and encryption methods, this variation in DRM providers by browser would have presented a significant issue. With the development and use of Common Encryption (CENC), the problem is substantially reduced because the files are compressed in standard formats and encrypted using global industry standards. The service provider issues the keys and licenses necessary to consume the content in a given browser, but the website code, content and encryption keys are common across all of them, regardless of which DRM is in use. An example of such an implementation is DASH.js, the open source industry reference player used to prove out these technologies and which serves as the basis for many players across the web today.
As shown above, Microsoft’s PlayReady DRM supports two modes of DRM protection: “Software DRM”, which uses our traditional software protected media path, and “Hardware DRM”, which moves the media path into secured hardware when available on the device. Hardware DRM was designed to meet the highest requirements for commercial media content protection, and will allow streaming of the highest quality content available. Not all devices will be Hardware DRM capable, but sites built on MSE/EME can accommodate the difference and stream the best possible content quality supported by the browser or device.
Support from Microsoft
Like any new technology, the transition to DASH/MSE/EME/CENC can present some challenges. These include:
- MSE works by allowing a JavaScript client to set one or more sourceBuffer(s) as the source on a media element, and dynamically download and append media segments to the sourceBuffer. This provides sites precise control over the experience, but also requires a larger investment in site development.
- Large existing media libraries have been encoded in formats which are not directly supported by MSE/EME. These libraries must be either supported in some way on the new APIs or re-encoded. Silverlight Smooth Streaming is an example format which was designed and built for sites using Silverlight plug-ins. A solution that could play this content directly would be useful for any Silverlight replacement technology.
- MSE/EME are maturing, but still undergoing change that may present interop challenges working with different media formats and across browsers.
As part of helping the industry move to interoperable media delivery solutions, we are investing in technologies to address these challenges.
DASH Type 1: MSE Made Easy
DASH content usually consists of media files encoded at various quality levels and a manifest that provides information on the files to the media application. An MSE based player is then responsible for parsing these files, downloading the most appropriate content and feeding it into the media element’s sourceBuffer(s). This is very flexible, but requires either investment in authoring site MSE implementations or use of a reference MSE implementation such as the aforementioned DASH.js.
There is another much easier option: native DASH streaming, where the site code simply sets the manifest as the media element source, and playback is automatically managed by the browser’s built-in streaming engine. This approach allows web developers to leverage the learnings and investments made by browser makers and easily provide a premium media experience on their site. We have added support for native DASH streaming to Windows 10 and Microsoft Edge, and more details on this are available in our previous blog: Simplified Adaptive Video Streaming: Announcing support for HLS and DASH in Windows 10.
A DASH JavaScript Library That Plays Smooth Content
A number of websites have large media libraries encoded in the Smooth Streaming format and are looking to move to an interoperable HTML5-based solution. One possible solution is to use a library that supports their current content through MSE/EME without re-encoding. Libraries are available now that are capable of playing back Smooth Streaming content using MSE and EME. For example, a version of the “hasplayer.js” library can do just that and is available for download at https://github.com/eiximenis/hasplayer.js.
This hasplayer.js library is based on dash.js, and enables playback of both clear and protected Smooth content using PlayReady on Microsoft Edge. It is a client-side JavaScript library that provides content and manifest translations and is cross browser compatible. Thanks to the inclusion of EME poly-fill support, it may be easily extended to support other browser’s DRM solutions as well.
Here’s a sample of JavaScript that uses hasplayer.js to retrieve and play a DASH or Smooth media file:
That makes it very simple to support streaming Smooth content on a website. We’ve provided a sample in the Contoso Video Sample GitHub Repository that uses this library to play video. You can try the webpage yourself at the Contoso Video Demo Website.
• Microsoft Edge Rendering
• Chakra JavaScript Engine
• HTML/CSS/JS from server
It is possible to provide a client-side translation library for Smooth Streaming content because the Protected Interoperable File Format (PIFF) underlying the Smooth Streaming protocol was a primary input into the ISO Base Media File Format (ISOBMFF) spec used with DASH and because PIFF introduced the multi-DRM protocol which was standardized as ISO MPEG Common Encryption (CENC). Two PIFF formats are in broad use today – PIFF 1.1 and PIFF 1.3 – and the hasplayer.js Smooth Streaming MSE/EME library supports both formats. These formats are supported by on-the-fly conversion from the PIFF format to the Common Media Format (CMF) used with DASH. This ensures that all browser content played back by the library is conformant with DASH CMF and capable of playback in all MSE-capable browsers.
Media Services
Some content owners would prefer to focus on producing great content, not the technical details required to deliver their content to consumers. These companies may wish to leverage hosted media services that prepare content for web delivery, handle the streaming logic and player UI, and manage the DRM license servers. Azure Media Services offers this capability today, with support for both PlayReady and Widevine DRM systems. This service provides support for both Video on Demand (VoD) and live streaming. A single high quality file/stream is provided to Azure, where it undergoes dynamic compression and encryption into CENC protected content that can be streamed to end devices, and a complete player solution is available for developers to simply add to their website. Some details of this service are newly announced in Azure Media Services delivers Widevine encrypted stream by partnering with castLabs.
Hosted Web Apps
One powerful advantage of moving to DASH/MSE/EME/CENC streaming is that the same code running on your website can be packaged as a Universal Windows Platform (UWP) app. UWP applications run on all devices with Windows 10. A website developer can create both an interoperable website based player and a Windows app that uses the same code, but offers a richer experience using Windows platform capabilities. Their common code will thus be able to handle UI and media streaming details, AND take advantage of capabilities only available to apps through WinRT APIs.
These hosted web apps:
- Are offered through the Windows Store
- Are able to interact with Cortana (“Contoso Video play elephants”)
- Are able to offer notifications (“NBA Finals streaming starts in 15 minutes”)
- Have access to advanced adaptive streaming support
- Have access to enhanced content protection for Full HD and Ultra-HD video playback
- Are able to light up live tiles
- And more!
We talked about the power of hosted apps in our talk on Hosted web apps and web platform innovations at the Microsoft Edge Web Summit 2015. You can also find reference documentation on MSDN.
Hosted App Demo
We’ve taken the demo Contoso Video website shown above and packaged it as a demo UWP app that takes advantage of Windows Platform APIs. This demo shows how simple it is to take the basic video player and integrate Cortana Voice Commands. The demo also customizes the colors used in the app’s title bar. All of the JavaScript code is part of the HTML website which is deployed as part of the standard web development workflow.
• Retain Microsoft Edge Rendering
• Retain Chakra JavaScript Engine
• Retain HTML/CSS/JS from server or local
• Add access native Windows APIs – Cortana, notifications, customizations & more…
• Offered in the Windows Store Catalog
Three files are needed to integrate Cortana in your Hosted Web Applications (HWA): A Voice Command Definition (VCD) file, a JavaScript file, and an HTML file.
Voice Command Definition (VCD) file
The Voice Command Definition (VCD) File specifies the actions you want to support with voice commands. The code below informs Cortana of the app name (Contoso Video), that the app supports a “play” command, and how the “playing” state should be represented in the app UI.
The JavaScript File
JavaScript must listen for the Windows Activation event and check for VoiceCommands.
The HTML File
The HTML file must add a meta element pointing to a VCD file on your server.
With the addition of a VCD, and updates to the website HTML and JavaScript, our Contoso Video website demo can now be packaged as a Universal Windows Platform application that will run across every device that runs Windows 10. Further, users can launch the app to play a video by saying “Contoso, play Tears of Steel”. Cortana will understand the command, launch the Contoso Video app and start playing the video “Caves of Steel”. The app also has a customized view on the app bar.
Contoso Video in Cortana
Contoso Video in the App Menu
The full source for the Contoso Video website and demo app is available in the Contoso Video Sample GitHub Repository.
Conclusion
DASH/MSE/EME/CENC offer compelling advantages over plug-in based solutions, and we are quickly moving towards an era of broadly interoperable streaming media. Both content providers and consumers will benefit from this transition. While the adoption of these technologies may present short-term challenges, the features and options discussed in this blog are provided to assist companies as they make this change.
We’re eager for your feedback so we can further improve our streaming media offerings, and are looking forward to seeing what you do with the tools and approaches we’ve discussed here!
– David Mebane, Program Manager, Windows Media
– Jerry Smith, Senior Program Manager, Microsoft Edge
– Kevin Hill, Senior Program Manager, Microsoft Edge