EtchMark is a new take on the classic Etch-A-Sketch drawing toy, showcasing IE11’s improved support for touch and emerging Web standards (including Pointer Events and Device Orientation). In this post, we’ll walk through several features that you can easily add to your own sites to build an experience that feels smooth and natural with touch, mouse, pen, and keyboard – and even responds to device shaking.

Structure of the Demo

EtchMark enables you to draw anything you want on the screen using touch, mouse, pen, or the arrow keys.  The drawing surface is an HTML5 canvas element that we update any time a knob turns.  In benchmark mode, we use the requestAnimationFrame API, which provides a smooth 60 frames per second animation loop and longer battery life.  The drop shadows for the knobs are created using SVG filters. IE11’s hardware acceleration moves much of this work to the GPU, which leads to a blazing fast experience.  Check out the video below to see these features in action, and then we’ll dive in and see how it’s built.

EtchMark uses HTML5 canvas, requestAnimationFrame, SVG filters, Pointer Events and Device Orientation APIs to create a new take on a classic toy

Touch, Mouse, Keyboard, and Pen Using Pointer Events

Pointer Events enable you to build experiences that work equally well with mouse, keyboard, pen, and touch – all by coding against a single API. Pointer Events are supported across the full range of Windows devices and are coming soon to other browsers as well.  The Pointer Events specification is now a Candidate Recommendation at the W3C, and IE11 supports an un-prefixed version of the standard.

To get started, the first thing we need to do is wire up our pointer events in Knob.js. First we check for the standard un-prefixed version, and if that check fails we fall back to the prefixed version needed to enable IE10 support.  In the example below, hitTarget is a simple div containing the knob image, sized to be a bit bigger so the user has space to land her finger easily: 

    if (navigator.pointerEnabled)

    {

        this.hitTarget.addEventListener("pointerdown",   pointerDown.bind(this));

        this.hitTarget.addEventListener("pointerup",     pointerUp.bind(this));

        this.hitTarget.addEventListener("pointercancel", pointerCancel.bind(this));

        this.hitTarget.addEventListener("pointermove",   pointerMove.bind(this));

    }

    elseif (navigator.msPointerEnabled)

    {

        this.hitTarget.addEventListener("MSPointerDown",   pointerDown.bind(this));

        this.hitTarget.addEventListener("MSPointerUp",     pointerUp.bind(this));

        this.hitTarget.addEventListener("MSPointerCancel", pointerCancel.bind(this));

        this.hitTarget.addEventListener("MSPointerMove",   pointerMove.bind(this));

    }

Similarly, we add the correct fallback for setPointerCapture to Element.prototype to ensure it also works on IE10:

    Element.prototype.setPointerCapture = Element.prototype.setPointerCapture || Element.prototype.msSetPointerCapture;

Next let’s handle our pointerDown event.  The first thing we do is call setPointerCapture on this.hitTarget.  We want to capture the pointer so that all subsequent pointer events are handled by this element. It also ensures that other elements don’t fire events even if the pointer moves into their bounds.  Without this, we’d run into problems when the user’s finger is on the edge of the image and the containing div: sometimes the image would get the pointer event and other times the div would.  This would result in a jagged experience where the knob jumps around. Capturing the pointer is an easy way to solve this.

Pointer capture also works nicely when the user puts her finger down on the knob and then gradually moves off the hit target while continuing to rotate.  Even if the finger isn’t lifted until it has moved several inches off the hit target, the rotation still feels smooth and natural.

The last thing to note about setPointerCapture is that we pass in the event’s pointerId property.  This enables us to support multiple pointers, meaning the user can use a finger on each knob simultaneously without interfering with the other knob’s events. The support for multiple knobs means that when the user rotates both knobs at once, she gets a freeform drawing rather than only seeing vertical and horizontal lines.

We also want to set two flags on this, which points to our Knob object (the flags are per-knob):

• pointerEventInProgress - tells us whether or not the pointer is down
• firstContact - tells us if the user has just put their finger down

    function pointerDown(evt)

    {

        this.hitTarget.setPointerCapture(evt.pointerId);

        this.pointerEventInProgress = true;

        this.firstContact = true;

    }

Finally we want to reset the pointerEventInProgress flag when the user lifts up her finger (or mouse/pen):

    function pointerUp(evt)

    {

        this.pointerEventInProgress = false;

    }

    function pointerCancel(evt)

    {

        this.pointerEventInProgress = false;

    }

PointerCancel can occur in two different ways. The first is when the system has determined that a pointer is unlikely to continue producing events (for example, due to a hardware event). The event also fires if the pointerDown event has already occurred and then the pointer is used to manipulate the page viewport (for example by panning or zooming).  For completeness it is always recommended to implement both pointerUp and pointerCancel.

With up, down, and cancel events wired up, we’re now ready to implement support for pointerMove.  We use the firstContact flag so that we don’t over-rotate when the user first touches her finger down. After firstContact is cleared, we just calculate the movement deltas of the finger.  We use trigonometry to turn our start and end co-ordinates into a rotation angle, which we then pass along to our drawing function:

    function pointerMove(evt)

    {

        //centerX and centerY are the centers of the hit target (div containing the knob)

        evt.x -= this.centerX;

        evt.y -= this.centerY;

        if (this.pointerEventInProgress)

        {

            //Trigonometry calculations to figure out rotation angle

            var startXDiff = this.pointerEventInitialX - this.centerX;

            var startYDiff = this.pointerEventInitialY - this.centerY;

            var endXDiff = evt.x - this.centerX;

            var endYDiff = evt.y - this.centerY;

            var s1 = startYDiff / startXDiff;

            var s2 = endYDiff / endXDiff;

            var smoothnessFactor = 2;

            var rotationAngle = -Math.atan((s1 - s2) / (1 + s1 * s2)) / smoothnessFactor;

            if (!isNaN(rotationAngle) && rotationAngle !== 0&& !this.firstContact)

            {

                //it’s a real rotation value, so rotate the knob and draw to the screen

                this.doRotate({ rotation: rotationAngle, nonGesture: true });

            }

            //current x and y values become initial x and y values for the next event

            this.pointerEventInitialX = evt.x;

            this.pointerEventInitialY = evt.y;

            this.firstContact = false;

        }

    }

By implementing four simple event handlers, we have now created a touch experience that feels natural and sticks to your finger.  It supports multiple pointers and enables the user to manipulate both knobs simultaneously to produce a freeform drawing.  Best of all, because we used Pointer Events, the same code also works for mouse, pen, and keyboard.

Getting more fingers in the game: Adding Gesture Support

The Pointer Events code that we wrote above works great if the user rotates the knob using one finger, but what if she rotates it using two fingers?  We had to use trigonometry to calculate the rotation angle, and calculating the correct angle with a second moving finger gets even more complex.  Rather than trying to write this complex code ourselves, we take advantage of IE11’s MSGesture support.

    if (window.MSGesture)

    {

        var gesture = new MSGesture();

        gesture.target = this.hitTarget;

        this.hitTarget.addEventListener("MSGestureChange", handleGesture.bind(this));

        this.hitTarget.addEventListener("MSPointerDown", function (evt)

        {

            // adds the current mouse, pen, or touch contact for gesture recognition

            gesture.addPointer(evt.pointerId);

        });

    }

With the events wired up, we can now handle gesture events:

    function handleGesture(evt)

    {

        if (evt.rotation !== 0)

        {

            //evt.nonGesture is a flag we defined in the pointerMove method above.

            //It will be true when we’re handling a pointer event, and false when

            //we’re handling an MSGestureChange event

            if (!evt.nonGesture)

            {

                //set to false if we got here via Gesture so flag is in correct state

                this.pointerEventInProgress = false;

            }

            var angleInDegrees = evt.rotation * 180 / Math.PI;

            //rotate the knob visually

            this.rotate(angleInDegrees);

            //draw based on how much we rotated

            this.imageSketcher.draw(this.elementName, angleInDegrees);

        }

    }

As you can see, MSGesture gives us a simple rotation property which represents the angle in radians, so we don’t have to do all the math ourselves manually.  This now gives us support for two-finger rotation that feels natural and sticks to your finger.

Device Motion: Adding a Little Shake

IE11 supports the W3C DeviceOrientation Event Specification, which enables us to access information about a device’s physical orientation and movement.  When a device is being moved or rotated (or more accurately, accelerated), the devicemotion event is fired at the window and provides acceleration (both with and without the effects of gravitational acceleration on the device, expressed in meters/second2) in the x, y, and z axis.  It also provides the rate of change in the alpha, beta, and gamma rotation angles in degrees/second.

In this case, we want to erase the screen anytime the user shakes the device.  To do this, the first thing we do is wire up the devicemotion event (in this case we’re using jQuery):

    $(window).on("devicemotion", detectShaking);

Next, we detect if the user has moved the device in any direction with an acceleration greater than our threshold value.  Because we need to detect shaking, we have a counter to ensure that there are two such fast movements in a row.  If we detect two fast movements, we erase the screen:

    var nAccelerationsInARow = 0;

    var detectShaking = function (evt)

    {

        var accl = evt.originalEvent.acceleration;

        var threshold = 6;

        if (accl.x > threshold || accl.y > threshold || accl.z> threshold)

        {

            nAccelerationsInARow++;

            if (nAccelerationsInARow > 1)

            {

                eraseScreen();

                nAccelerationsInARow = 0;

            }

        }

        else

        {

            nAccelerationsInARow = 0;

        }

    }

For more information on device orientation and motion, please see this post on the IE Blog.

Orientation Lock

IE11 also introduces support for the Screen Orientation API and features like Orientation Lock.  Since EtchMark is also a performance benchmark, we want to keep our canvas size the same across screen resolutions so that we’re doing the same amount of work on every device.  This can make things really tight on smaller screens, particularly in portrait mode.  To enable the best experience, we simply lock the orientation to landscape:

    window.screen.setOrientationLock("landscape");

This way, no matter which way the user rotates the device, she always sees it in landscape mode.  You can also use screen.unlockOrientation to remove the orientation lock.

Looking Ahead

Interoperable and standards-based techniques like Pointer Events and device orientation events enable new exciting possibilities for your web sites. IE11’s excellent touch support delivers a smooth stick-to-your-finger experience and interactivity. You can go even further with IE11 and MSGesture, making scenarios like calculating two-finger rotation angles as simple as accessing a property. Give these techniques a try on your own site, and we look forward to your feedback.

Jon Aneja
Program Manager, Internet Explorer