As we announced last month, Project Spartan will be the new browser across all Windows 10 devices, from phones to tablets, PCs and beyond. You’ll hear about the new browser’s features in the coming months but in this post, we want to tell you more about what motivated us to build a new rendering engine focused on interoperability with other modern browsers ― all in the name of making the Web “just work” for our customers. This new rendering engine was designed with Project Spartan in mind, but will also be available in Internet Explorer on Windows 10 for enterprises and other customers who require legacy extensibility support.

Modernizing IE while not breaking the Web

Internet Explorer has been the Microsoft Web browser for 20 years and has evolved to work with ever-changing Web sites over that time. Since 2009, we’ve been systematically bringing major investments to the browser to modernize each of the browser subsystems: from fast new JavaScript and layout engines, GPU-accelerated rendering and 3D graphics, and multi-threaded touch input to improved F12 developer tools and backwards compatibility with Enterprise Mode.

Big changes in 5 years: Fish IE (GPU rendering), Chakra fastest on SunSpider (JavaScript performance), Touch Effects (multi-touch input), new F12 developer tools, Assassin’s Creed Pirates (WebGL)

As we’ve been rolling out these significant changes with major versions of IE, we also did our best to abide by the mantra of “don’t break the Web.” This is an ambitious goal given that the Web consists of over 44 billion Web sites, so we prioritized compatibility testing on the Web’s top 9000 Web sites globally which accounts for roughly 88% of Web traffic. This represented the sweet spot of where the “head of the Web” becomes the “long tail” – allowing us to focus our testing resources on the most impactful sites. Prior to release, we would ensure a compatibility pass rate greater than previous releases of IE and our competitors on those top 9000 sites.

The long tail matters

And yet, even as we released new and improved versions of IE, we heard complaints about some sites being broken in IE – from family members, co-workers in other parts of Microsoft, or online discussions. As we examined these sites, they were often not on our top 9000 list. They were issues like not being able to complete a reservation online for the barbershop down the street, or not being able to log in to see the schedule for the local kids soccer league.

In Windows 10 planning, we set out to tackle this apparent discrepancy – how could our real-world compatibility be declining when our compatibility testing data shows that we are improving?

Rethinking our assumptions

As we dug in, we uncovered a series of issues that led us to realize that we needed to significantly rethink our approach to compatibility with the Web at large:

• Legacy vs. modern. While we were pushing ahead with new HTML5 features, we were also expected to preserve compatibility with old versions of IE, particularly for enterprise Web apps. Limited compatibility was provided through document compatibility modes within Trident, however compatibility could not be guaranteed and it provided consistent obstacles towards fixing long-standing IE-specific behaviors. Furthermore, fixing long standing interoperability bugs with other modern browsers could actually break sites who have coded to the IE-specific behavior.
• CV list. Our compatibility pass rates were dependent on the presence of the compatibility view list. This allowed us to “fix” broken sites by forcing them into old document modes which emulated legacy IE behaviors. However, this approach requires testing and maintenance, and doesn’t scale well beyond the top sites.
• X-UA-Compatible. Some sites forced an older document mode using the “x-ua-compatible” header. However, rather than using it as a temporary stopgap, they would rely upon that to keep that version of the site working in future versions of IE while they developed an evergreen code path of their site for other modern browsers.
• Standards focus. Our focus on building new HTML5 features was to comply with Web standards, which in turn should lead to interoperability among browsers. However, interpretations of the standards document could easily vary, leading to real-world interoperability gaps and ultimately more bug fixing for Web developers and more broken sites for customers.

A break from the past

Faced with these issues, we were inspired to have “courage in the face of reality”. We needed a plan to make it easy for Web developers to build compatible sites regardless of which browser they develop first for. We needed a plan which ensured that our customers have a good experience regardless of whether they browse the head or tail of the Web. We needed a plan which gave enterprise customers a highly backward compatible browser regardless of how quickly we pushed forward with modern HTML5 features.

In order to really address these challenges, we realized that we couldn’t just incrementally improve on our previous approach, we needed a break from the past – all without losing the major investments that we had been making since 2009.

A pragmatic approach

The break meant bringing up a new Web rendering engine, free from 20 years of Internet Explorer legacy, which has real-world interoperability with other modern browsers as its primary focus – and thus our rallying cry for Windows 10 became “the Web just works.” This pragmatic viewpoint, which was initially proven out by our work in Windows Phone 8.1 Update, meant that Web standards would continue to be important but should function in the background to drive real-world interoperability between browsers.

This interoperability-focused approach brought the obvious question of adopting an existing open-source rendering engine such as WebKit. While there were some advantages, upon further investigation it was not the right path forward for two important reasons. First, the Web is built on the principle of multiple independent, yet interoperable implementations of Web standards and we felt it was important to counter movement towards a monoculture on the Web. Second, given the engineering effort required, we found that we could deliver an interoperability focused engine to customers significantly faster if we started from our own engine (especially if unshackled from legacy compatibility concerns), rather than building up a new browser around an open-source engine. We will continue to look at open source and shared source models where it makes sense and look forward to sharing more details in coming posts.

A new Web rendering engine is born

As detailed in Jacob Rossi’s article for Smashing Magazine, the new engine began as a fork of MSHTML.dll but has since diverged very quickly. By making this split, we were able to keep the major subsystem investments made over the last several years, while allowing us to remove document modes and other legacy IE behaviors from the new engine. On the other hand, our legacy engine (MSHTML.dll) can remain largely unchanged (outside of security and other high priority fixes) to help guarantee legacy compatibility for our enterprise customers. We also built up capabilities to switch between the legacy and new rendering engines seamlessly.

A clean break also necessitates a new user-agent string to ensure that no IE-specific code was being sent. This built upon a long browser history of using whatever tokens are necessary to get the desired content from servers. Although this meant a lower compatibility rate initially, it was also useful to reveal interoperability issues that needed to be fixed!

Finally, since legacy IE code is no longer being sent, we are able to drastically reduce our reliance on the CV list. This means that our top site compatibility rate will match our long tail compatibility much more closely.

Fixing patterns instead of sites

However, a new engine was not enough – we also needed to significantly revamp how we find, track and fix issues on the long tail of the Web. To do so, we do daily analysis on trillions of URLs crawled in conjunction with Bing to detect patterns that exist in the head of the Web and the tail of the Web. By fixing these patterns, sites just end up working. This data is augmented by thousands of daily feedback reports from users via the “smiley face” icon.

In addition, we revised our internal engineering processes to prioritize real-world interoperability issues uncovered by our data analysis. With these processes in place, we set about fixing over 3000 interoperability bugs and adding over 40 new Web standards (to date) to make sure we deliver on our goals.

We don’t see this interoperability effort having an end date – we’ll be continuously checking the data and rolling out improvements to the new rendering engine. For users that upgrade to Windows 10, the engine will be evergreen, meaning that it will be kept current with Windows 10 as a service.

A community effort

Our mission to create a Web that “just works” won’t be successful without your help. That’s why the Project Spartan journey has included a number of ways for you to get involved. Here’s a quick rundown of ways you can help:

• Join the Windows Insider Program to get the latest Windows 10 previews and test your Web sites on our new rendering engine by enabling experimental Web features in the about:flags page. If you don’t have a device to install it on, try out RemoteIE which will stream our latest browser from the Azure cloud to Windows, iOS or Android devices.
• If you see any site issues, “Send a Frown” using the smiley icon so we can track it. If you have a more detailed description of the issue, you can open a bug on Connect.
• Check out our Web platform roadmap at status.modern.ie. If anything is missing, submit a feature request or vote for an existing request at our UserVoice site.
• Send us a message on Twitter @IEDevChat, or join a monthly #AskIE tweetchat (next one on 2/26 @ 10AM PST). For help on more detailed issues, ask a question on our StackOverflow page.

We believe the break from IE’s past we’ve made to create a new rendering engine will help make the browsing experience for our customers better, and make building Web sites that just work across browsers easier for Web developers. We’ll be sharing more details on specific aspects of the approach outlined above in the coming weeks and look forward to your continued participation in this journey with us.

— Charles Morris, Program Manager Lead, Project Spartan