On April 12-13, Microsoft hosted a record-setting Test the Web Forward event to advance the Web by creating interoperability tests. Dozens of volunteers from Adobe, AT&T, Blackberry, Mozilla, and many other local companies joined us at our Seattle offices to learn about Web standards testing, how to write CSS and HTML tests, as well as the tools available for test suite management. Attendees from around the country - and even Canada – contributed to create a record-breaking 514 overall new tests.

Why Tests?

The quality and correctness of different browsers’ HTML and CSS standards compliance continues to vary widely. The W3C requires independent tests of all normative requirements in a specification in order to move a W3C Web specification from a candidate recommendation to an official recommendation. These tests are used to ensure that at least two browsers fully support each normative statement. As you might imagine, creating all of these tests is daunting; HTML5 is expected to need well over 100,000 tests, to say nothing of CSS3 modules, WebApps, Media Extensions, etc. We have submitted thousands of test cases for HTML, CSS, and SVG that can be viewed at the W3C and the Internet Explorer Test Center, however more tests are still needed. These tests benefit all browsers – and ultimately the entire Web developer community – by ensuring a consistent, predictable behavior. As different browsers improve their same-markup support, we can all realize the promise of HTML5 and CSS3.

A few years ago, several members of the standards community turned to crowd-sourcing to help create new tests, this resulted in Test the Web Forward events.

With the sponsorship of major players like Microsoft, Adobe, Google, and Mozilla, the Web community has come together, running local test-writing sprints around the world—France, China, Australia, and the US. Each sprint not only generates hundreds of tests, but also engages with and educates Web developers about the specifications that make up the Web platform.

Inside the Seattle Event

Our friends at Adobe were instrumental in setting up a successful event, building upon their experience in running previous events. In Seattle we kicked off the hack-a-thon on Friday night, with inspirational and informative presentations from Mozilla's fantasai (Elika Etemad), Adobe's Rebecca Hauck, and Microsoft's Kris Krueger, explaining why we need tests, what type of tests are available, and how to create them. Here’s a quick outline:

Stand-alone tests typically rely on visual verification: if a failure condition occurs, red content will show.

Reference tests compare a test against a visual reference that has no dependency on the feature being tested. Note that the test includes a link to the reference test against which is should be compared. For example, if you wanted to test that DIVs render background colors correctly, you might make a ref test using TABLEs.

Object Model tests depend on a JavaScript test harness; they verify that the object model reflects what static style sheets specify. For instance, this CSS media query test.

These presentations were followed by 2-minute pitches from Saturday's session test leaders on why participants might want to pick a particular focus (CSS Flexbox, Pointer Events, CSS Transforms, CSS OM, Backgrounds & Borders, Exclusions, or HTML5), though session participants were free to write tests against any API or spec they felt passionate about.

Following breakfast the next morning, participants broke up into three rooms with session leaders helping out in each. Each area was staffed by experts (in addition to the speakers from the previous evening): Arron Eicholz (Microsoft, CSS); Jacob Rossi (Microsoft, Pointer Events); Sylvain Galineau (Adobe [formerly Microsoft], CSS); Alan Stearns (Adobe, CSS); Dave Methvin (President of jQuery, HTML).

The leaders instructed everyone on how to determine where tests were needed and how to create code that tested the specific assertion that we wanted to capture. Volunteers could either work on their own, work in small groups, or get 1:1 help with the experts.

When all was said and done, the sprint generated 514 submitted tests, just edging out the record set by the Paris test sprint and setting a new bar for upcoming sprints to beat. After a few celebratory drinks, the end of the night saw the raffling off of a Surface Pro which was won by a student volunteer who joined us from the University of Washington.

You Can Help Too!

In IE10, we have added support for a long list of new standard features across CSS, HTML, SVG and the DOM. We have published some of our test cases for these new features on our IE Test Center. We will be submitting more, but we still need help from the community to get the right tests written and move these specs forward.

We are excited to be part of the community working towards a more innovative and interoperable Web. We support several initiatives in this direction, like the recent donation of JavaScript documentation to Webplatform.org, or our continued efforts to simplify cross-browser testing with modern.ie. If you want to help move the Web forward, too, come and join us at one of the next Test the Web Forward events! In the meanwhile, you can learn how to contribute tests, or review existing tests online. To hear about upcoming events and to stay in touch with the rest of the Test the Web Forward community, please subscribe to our W3C mailing list: public-testtwf. If test writing sounds too intense, but you are still knowledgeable and passionate about the Web, you can get involved with the WebPlatform Docs project and help document the Web.

For more info and updates, follow our Internet Explorer developer relations handle on Twitter @IEDevChat, this initiative’s handle @testthewebfwd and in particular #testtwf.

We will keep you posted on upcoming events and we are looking forward to meeting you soon!

—John Jansen, Kris Krueger, Arron Eicholz, and Jacob Rossi – Internet Explorer