This blog post was authored by Craig Horsfield, a Senior SDET on the Windows Phone Test and Operations team.
- Adam
Testing your app throughout the development process helps to ensure that your app has a high level of performance and quality. Testing also helps to make sure your app is effectively represented in the Windows Phone Store. A small investment in the key areas described in this post can give your app positive results in your market, and help bypass common errors early in the development process. This document will outline testing approaches for key test areas that you should consider.
Beta testing
After you have thoroughly tested your app using the emulator and on a Windows Phone device, it’s important to also test your app as a beta app. Submit your beta app for beta distribution, where you can continue to test your app based on the beta and code flow experience. Additionally, you can include beta tester to help review your app. For more info, see Beta testing your app and in-app products.
Installation and launch
Testing your app’s install and launch process is a vital area that can often be overlooked. By following these steps, you can minimize installation and launch issues that are not handled by your app. There are a few scenarios to consider when testing the initial launch experience and resume paths scenarios. One of these is the new Windows Phone 8 fast app resume scenario that a Windows Phone 8 app can opt into.
The following tests can help verify your app’s install, launch, and resume functions.
Test scenario | Details |
1. Verify the app on the first launch. | Note any End User License Agreement (EULA) and any other one-time notifications that appear in the launch UI. |
2. Reinstall the app. | Verify that all one-time notifications appear again on a subsequent install and launch. |
3. Start the app, and then return to the Windows Phone Start screen. | Launch the app again from a Live Tile, or from the phone’s App list. · Verify the default scenario, in which the old app is terminated, and a new instance of the app starts. · For Windows Phone 8 apps, if the app uses fast app resume for Windows Phone 8, in which ActivationPolicy="Resume", then the app resumes instead of starting a new instance. Verify that the app returns to the correct page in the app. |
Fast Application Switching (FAS) and tombstoning
When a Windows Phone app is moved to the back stack based on the app’s navigation history, the app is suspended. When the app resumes, it returns to the foreground. Apps on the back stack are tombstoned when there is resource pressure on the device. The exception occurs when running location-tracking apps in the background for Windows Phone 8. Windows Phone 8 location-tracking apps continue to run in this scenario. Your app needs to handle both of these cases, and you should test this function in your app. FAS occurs when the user leaves the app to go to Start, Search, or some other path based on the launchers and choosers you’ve used in your app to support navigation. For more info, see Multitasking for Windows Phone and App activation and deactivation for Windows Phone.
The following tests can help verify your app’s FAS and tombstoning functions.
Test scenario | Details |
1. Test the phone’s Start button and Back button. | Launch the app, and then press the Start button to verify that the Windows Phone start page appears. Press the Back button to resume the app. Verify that the app resumes in the required time, and in the state it was in before you navigated away from the app. |
2. Test the Camera button. | Launch the app, and then press the Camera button to verify that the camera starts. Press the Back button to resume the app. Verify that the app resumes in the required time (10 seconds) and that is in the state it was before you navigated away from the app. |
3. Test locking the phone using the Power button. | Launch the app, and then press the Power button to lock the phone. Unlock the phone, and then verify that the app resumes in the required time, and that the app is in the state it was in before you locked the phone. |
4. Perform the same test cases for FAS and tombstone state. | To force a tombstone path, in Visual Studio, in the project Properties windows, click the Debug tab, and then select Tombstone upon deactivation while debugging. |
5. Test deactivating and closing your app. | Make sure the app doesn’t take too long to process the Application_Deactivatedand Application_Closingevents, so that the app can save its state correctly. Avoid doing a lot of work in the Application_Closing code path, because processing time and access to the UI thread are limited. |
6. Test background execution of your app. | For continued background execution in location tracking apps, you’ll want to test the app’s RunningInBackgroundevent handler. In addition to running the preceding tests, check for the following conditions: · Background execution occurs when the app enters the back stack. · Start the app and move it to the back stack and resume the app. Verify that the app was not suspended and that it continued to run. |
App navigation
You should navigate all paths in the app to test its navigation. You especially want to be sure that your app exits and handles navigation interruptions as intended.
Test scenario | Details |
1. Test the Back button. | Your app should exit when you press the Back button on the top most page. |
2. Test navigation interruptions. | Your app should not try to initiate new navigation while your app is currently navigating. Test rapid navigations in the app, and navigating in the app while pages are loading. |
Don’t throw unhandled exceptions to exit the app at a page that isn’t at the top of the page stack. Instead, navigate over the page by removing it from the back stack; you don’t want to display the page on the exit path, or call API available in Application.Terminate in Windows Phone 8 apps. This triggers an app crash, and a Watson error report. You might see the crashes indicated in the app reports available in the Store.
Launchers and choosers
Launchers in your Windows Phone app give users the ability to perform common tasks, such as launching the Bing Maps app, launching Media Player, or launching the Store or Marketplace. Launchers do not return data or status to your app. With a chooser, on the other hand, users can return data to your app. For instance, users can choose return info from the Contacts app. For all launchers and choosers, test your app to be sure the launcher or chooser works correctly, and that it doesn’t trigger related errors.
Test scenario | Details |
1. Test navigating to and from the launcher or chooser. | In the app, navigate to the launcher or chooser. Immediately go back to the previous page or screen. The app should return to the page and state the navigation was initiated from. |
2. Test the complete scenario when using the launcher or chooser. | In the app, navigate to the launcher or chooser. Complete the scenario, i.e., select a photo or send mail. When the action is completed, the app should return to the preceding app state. |
3. Test FAS scenarios when navigating to a launcher or chooser. | Navigating to a launcher or chooser can triggers an FAS scenario. See the related section earlier in this post for additional test scenarios related to FAS. |
UI and layout
You need to test to make sure all elements of your UI and their layout work as intended, and meet app submission requirements, such as supporting Light Theme and Dark Theme. Test each control you use in your app, and the overall appearance of your app on the phone.
The following tests will help verify your app’s UI and layout.
Test scenario | Details |
1. Test your app in the Light Theme. | Ensure all text is readable and all UI is visible. |
2. Test your app in the Dark Theme. | Ensure all text is readable and all UI is visible. |
3. Test the UI for text clipping. | You may need to set the text to wrap or make other layout changes. |
4. Test your Splash screen display. | If you use a splash screen in your app, verify that it’s visible when the app starts, and that it displays for the expected time. |
5. Test each animation. | Ensure all animations are smooth. If they appear slow or stop unexpectedly, see if the app is doing excessive work on a background task or thread. |
6. Test each screen orientation. | Verify that the layout looks as you intended in portrait and landscape orientations, and that each page can display and move between orientations. |
In part 2 of this series, we’ll discuss additional areas and testing approaches to consider related to push notification; Live Tiles; background agents; media, audio, and video; geolocation, and resource usage and performance.