Liven up your app with location and sensors

If you’re developing an app for Windows 8 tablets or convertibles, you have access to cutting edge hardware and software services for sensors and location. By tapping into these hardware capabilities, you can add value to your apps and also make them exciting and useful.

Imagine an app that automatically loads a 3D environment based on the user’s current location, and then allows them to navigate the 3D world (pan, tilt) by moving the tablet around in 3D space. The pictures here show a Photosynth panorama app that uses sensor fusion to allow natural interactions between the orientation of the device and the Photosynth panorama. Sensor fusion combines the outputs of an accelerometer, a gyroscope and a magnetometer to provide a better rotation experience than any of these sensors alone and is exposed through the Windows.Devices.Sensors.OrientationSensor runtime type.

When the user holds the device level, they see a view looking straight ahead in the Photosynth panorama:

When the user tilts the device upwards, the view in the panorama is tilted upwards too:

The same type of interactions occurs when the user moves the device left or right. It’s as if the camera in the panorama is directly tied to the orientation of the tablet. You can imagine there’s quite a few cool things that you can do with these capabilities including first-person games, mapping experiences, pedestrian navigation, and many others. So let’s take a look at how you go about implementing this type of functionality for Windows 8.

Why use sensors

At the fall 2011 //Build conference, our team shared details about hardware support for location and sensors in Windows 8, and also information about the Windows Runtime APIs for location and sensors. With sensor support in Windows 8, users will enjoy system-level features like screen auto-rotation, and automatic brightness control. While these features are great, it’s location-aware and sensor-aware apps that really create excitement. With Windows 8, location and sensors enable a diverse set of experiences including navigation and compass scenarios, casual games, augmented reality, interaction motion control for games, and many more.

All Windows 8 certified tablets and convertibles will include these sensors:

• Ambient light sensor (ALS)
• 9-axis sensor fusion system which uses this sensor hardware:
• 3D accelerometer
• 3D gyro
• 3D magnetometer

In addition, all Windows 8 PCs include the Windows Location Provider (software location lookup service), and GPS (for PCs that have mobile broadband hardware).

What this means is that you can access an impressive array of sensor and location capabilities. It also means that capabilities may differ depending on the PC form-factor, so you need to consider which hardware form factors you are targeting with your apps.

Using location in your app

Adding location support to an app is easy and straightforward. The first thing you need to do is to add a location capability declaration in the app’s manifest file. This declaration lets the system know that the app may try to access the current location. Of course, the user controls the final approval over whether the app can access the current location. You can add this declaration via Visual Studio (double-click package.appxmanifest in Solution Explorer, click the Capabilities tab, then check the Location checkbox in the list).

Here’s what the manifest editor looks like after you declare the Location capability:

Here’s what the snippet looks like in your package.appxmanifest.xml file when you declare the location capability: (this is done for you when you use the manifest editor in Visual Studio)

<Capabilities>
  <DeviceCapability Name="location" />
Capabilities>

Once you have declared the capability for location in your app, you can now start writing the code to access location data. Location functionality is built into the Windows Runtime APIs, and can be accessed via the Windows.Devices.Geolocation namespace. Or, you can use the W3C API for location via JavaScript. In this post we’ll show the Windows-specific variety. For more details about using the W3C API for location, see Quickstart: detecting location using HTML5 (Metro style apps using JavaScript and HTML).

There are two fundamental ways that you can access location data: a single-shot lookup, or a stream of events that are raised when the system detects a change in location. Let’s take a quick look at each of these access models so that you can decide which model is better for your app.

Single shot location access

If you only need to access location on-demand, for example to find points of interest in the user’s area, annotating content with location info, showing users position on a map, etc., you can do this with a handful of lines of code, and also ensure that you are not using up valuable battery life. Let’s look at a code snippet that demonstrates the single shot lookup.

JavaScript:

var loc = new Windows.Devices.Geolocation.Geolocator();
loc.positionAccuracy = Windows.Devices.Geolocation.PositionAccuracy.high;
loc.getGeopositionAsync().done(getPositionHandler, errorHandler);

function getPositionHandler(pos) {
    var lat = pos.coordinate.latitude;
    var long = pos.coordinate.longitude;
    var acc = pos.coordinate.accuracy;
}

function errorHandler(err) {
    // handle error
}

C#:

using Windows.Devices.Geolocation;

var loc = new Geolocator();

try
{
    loc.DesiredAccuracy = PositionAccuracy.High;
    Geoposition pos = await loc.GetGeopositionAsync();

    var lat = pos.Coordinate.Latitude;
    var long = pos.Coordinate.Longitude;
    var acc = pos.Coordinate.Accuracy;
}
catch (System.UnauthorizedAccessException)
{
    // handle error
}

Here we see:

• Creating a Geolocator object (root object for location)
• Asynchronously invoking an operation to retrieve the current location, with error handling if access to location is denied.

Note that you can cancel an in-progress call to GetGeopositionAsync if you need to. See the Windows Runtime Geolocation SDK sample for example code that demonstrates this. For more info about single shot location access, see the Geolocator.GetGeopositionAsync() | getGeopositionAsync() method and the Geolocation SDK sample.

Continuous location access

If your app needs continuous access to location data, for example turn-by-turn navigation, alerting the user when points of interest are nearby, route logging etc., you’ll want to add an event listener that the app calls when one or more location providers on the system detects a change in location. Here’s how to listen to changes in location.

JavaScript:

var loc = new Windows.Devices.Geolocation.Geolocator();
loc.desiredAccuracy = Windows.Devices.Geolocation.PositionAccuracy.high;
loc.addEventListener("positionchanged", onPositionChanged);

function onPositionChanged(e) {
    var lat = e.position.coordinate.latitude;
    var long = e.position.coordinate.longitude;
    var acc = e.position.accuracy;
}

C#:

using Windows.Devices.Geolocation;

private Geolocator loc = new Geolocator();

private void StartTracking(object sender, RoutedEventArgs e)
{
    loc.DesiredAccuracy = PositionAccuracy.High;
    loc.PositionChanged += new TypedEventHandler(OnPositionChanged);
}

private void OnPositionChanged(Geolocator sender, PositionChangedEventArgs e)
{
    Geoposition pos = e.Position;

    var lat = pos.Coordinate.Latitude;
    var long = pos.Coordinate.Longitude;
    var acc = pos.Coordinate.Accuracy;
}

Here we have essentially the same code as we had with single shot lookup, but instead of performing an async operation, we add an event listener, passing a delegate to handle the event. Now whenever a change in position is detected, the delegate gets called and we can use location data however our app needs to from this method. If you want to have influence over how sensitive the event is to movement, you can set the MovementThreshold property on the Geolocator object.

Another aspect of fine-tuning location API behavior is the ability to set the DesiredAccuracy property on the Geolocator object. If your app doesn’t need GPS-level accuracy (a weather app for example), set this property to Default to save battery by limiting the use of GPS devices in determining position.

For more info on adding location awareness to your app, see the Geolocation sample , Guidelines for location aware applications and Windows.Devices.Geolocation namespace docs on MSDN.

Sensors

Windows 8 offers an extensive and easy to use set of APIs for sensors. These classes are found in the Windows.Devices.Sensors namespace, and represent the corresponding underlying sensor hardware:

• LightSensor – ALS sensor
• Accelerometer – 3D accelerometer (acceleration)
• Gyrometer – 3D gyro sensor (rotational speed)
• Compass – tilt compensated heading (true north and magnetic north)
• Inclinometer – device tilt (yaw, pitch, roll)
• SimpleOrientationSensor – portrait/landscape/face-up/face-down
• OrientationSensor – 9-axis (sensor fusion) device orientation data (quaternion, rotation matrix)

Remember that interactive Photosynth panorama that I started off talking about? It turns out that we can use sensor fusion via the Windows Runtime APIs to implement these types of interactions. Because we want a seamless and responsive relationship between device orientation and the camera or viewpoint in our Photosynth panorama, we need to use sensor fusion via the Windows.Devices.Sensors.OrientationSensor class. To learn more about the underlying technology used to support sensor fusion, see the Supporting sensors on Windows 8. The OrientationSensor class exposes two data fields that are a powerful representation of device orientation: the 4-element vector called the quaternion, and a 3x3 matrix of values called the rotation matrix. These data fields both represent equivalent data, but package the data in different formats.

The specifics for implementing support for pans and tilts via quaternion or rotation matrix data (transforms, etc) are outside the scope of this blog post, but I will cover how to access the data via the Windows Runtime API.

All of the sensor objects in the Windows Runtime API for sensors follow pretty much the same pattern. First, you create the sensor object. Next, you perform initialization and apply preferences (setting the desired report interval for instance) and then access data either by a synchronous call, or by adding an event listener.

Here’s how to access data synchronously.

JavaScript:

var sensor = Windows.Devices.Sensors.OrientationSensor.getDefault();
if (sensor) {
    // Select a report interval that works for the app and that the sensor supports.
    var minimumReportInterval = sensor.minimumReportInterval;
    var reportInterval = minimumReportInterval > 20 ? minimumReportInterval : 20;

    // Set the report interval to enable the sensor for polling
    sensor.reportInterval = reportInterval;

    var reading = sensor.getCurrentReading();
    if (reading) {
        var quaternion = reading.quaternion;
        var x = quaternion.x;
        var y = quaternion.y;
        var z = quaternion.z;
        var w = quaternion.w;
    }
}

C#:

var sensor = Windows.Devices.Sensors.OrientationSensor.getDefault();
if (sensor) {
    // Select a report interval that works for the app and that the sensor supports.
    var minimumReportInterval = sensor.minimumReportInterval;
    var reportInterval = minimumReportInterval > 20 ? minimumReportInterval : 20;

    // Set the report interval to enable the sensor for polling
    sensor.reportInterval = reportInterval;

    var reading = sensor.getCurrentReading();
    if (reading) {
        var quaternion = reading.quaternion;
        var x = quaternion.x;
        var y = quaternion.y;
        var z = quaternion.z;
        var w = quaternion.w;
    }
}

This type of access could be preferred if you only need data under certain circumstances (such as tagging a device orientation on an object in response to a user action) or when you need to poll for data on a timer (a render loop in a game for instance). Even when polling for data, you should set the report interval as shown here so that the hardware/driver for the sensors can refresh data at the rate your app needs it.

Next I’ll show responding to reading changed events.

JavaScript:

var sensor = Windows.Devices.Sensors.OrientationSensor.getDefault();
if (sensor) {
    var minimumReportInterval = sensor.minimumReportInterval;
    var reportInterval = minimumReportInterval > 20 ? minimumReportInterval : 20;
    sensor.reportInterval = reportInterval;

    sensor.addEventListener("readingchanged", onDataChanged);
}

function onDataChanged(e) {
    var quaternion = e.reading.quaternion; 
    var x = quaternion.x;
    var y = quaternion.y;
    var z = quaternion.z;
    var w = quaternion.w;
}

C#:

using Windows.Devices.Sensors;

private OrientationSensor sensor = OrientationSensor.GetDefault();

private void EnableReadings(object sender, RoutedEventArgs e)
{
    if (sensor != null)
    {
        var minReportInterval = sensor.MinimumReportInterval;
        var desiredReportInterval = minReportInterval > 20 ? minReportInterval : 20;

        sensor.ReportInterval = desiredReportInterval;

        sensor.ReadingChanged += new TypedEventHandler(ReadingChanged);
    }
}

private void ReadingChanged(object sender, OrientationSensorReadingChangedEventArgs e)
{
    SensorQuaternion quaternion = e.Reading.Quaternion;
    var x = quaternion.X;
    var y = quaternion.Y;
    var z = quaternion.Z;
    var w = quaternion.W;
}

In this code snippet we obtain an OrientationSensor object, and then set the report interval either to 20 or the minimum supported value (fastest rate at which sensor hardware can supply data), whichever is larger. This way we get data at the intervals as close as possible to every 20 milliseconds, as an example, for this app. Use an interval that is appropriate for your app (only as fast as you need data) so that you don’t negatively affect power and performance.

In the event listener delegate, we apply the appropriate transforms using the quaternion data [x,y,z,w] to support motion/orientation interactions in our app. The equivalent rotation matrix data is also available. While this transform code is outside the scope of this blog, there are several online sources of info about using quaternion data for gaming scenarios, robotics, and much more. The advantage of using quaternion (or rotation matrix) data over Inclinometer data is the continuity of the rotation data, avoiding “gimbal lock” inherent in Euler angle calculations.

Conclusion

Adding sensor and location functionality to your app can light up new and exciting experiences for users. With Windows 8 you have access to comprehensive hardware and software support, Visual Studio, easy to use Windows Runtime APIs, and a variety of programming languages to choose from for sensor- and location-enhanced apps. This makes implementing these features in your app both advantageous and straightforward. So what are you going to build?

--Gavin Gear, Program Manager

--Dino Natucci, Program Manager