Coded UI Test Extensibility V2

In VS2013 Update 2 we enhanced the Coded UI Test extensibility model to support plugins built for Windowless controls including controls in Modern Apps. These series of blogs should give an overview of the extensibility model exposed in Coded UI Test. Also, this blog explains a new extensibility QFE that we introduced earlier. In the current blog I will build upon the context from the above two links. These are the major goals that the new extensibility changes target:

1. Windowless control support including extensibility for Modern Apps.
2. Control Specific Support for only controls that the extension targets.
3. Support API’s that are not closely tied to window handles.

Design Changes

For the above feature set, changes were made in the logic for Coded UI Tests authoring, playback and navigation. These changes involve the engine calling into a new set of API’s detailed below. These changes should be backward compatible and should continue to work for existing plugins since the base extensible points have default implementations for the new API’s to redirect to the existing API set.

Authoring

The Coded UI Test Technology Abstraction Layer (TAL) is now responsible for stitching the query ID of a control based on its corresponding UIA hierarchy. To elaborate on that the TAL would get a controls UIA hierarchy in terms of AutomationElements and then query each of the registered plugins for support for each node in the hierarchy. When a plugin claims support for a node through the GetControlSupportLevel API (surfaced in UITechnologyManagerProxy) it would then be queried again to create a query ID (from the Query Id property surfaced in UITechnologyElementProxy) for the range of nodes it supports. The TAL then would stitch the query ID’s (generated by the plugins) for controls in the hierarchy to obtain a leaf level control’s complete query ID. With this change TAL manages the generation of a complete query ID for an element as opposed to the earlier logic where individual plugins manage the generation of a query ID until it reaches a control in the hierarchy that it does not support. With this change, plugins providing control specific support for an intermediate control in the hierarchy take precedence over plugins providing default support.

 The below flowchart summarizes this change. Consider T_x to be technologies and E_xx to be elements in the UIA hierarchy of a control.

 Playback/Navigation

Playback/Navigation is fairly the same as earlier except when there is a technology switch in the hierarchy in such cases, TAL would communicate across technologies via an AutomationElement which is now the common factor as opposed to a window handle earlier.

API Set

To better support Windowless controls, the following are the set of new API’s that we introduced.

Technology Manager

In UITechnologyManagerProxy/UITechnologyManager we added overrides for methods that were window handle based earlier to now also take AutomationElement:

1. public virtual IUITechnologyElement GetElementFromPoint(int pointX, int pointY, AutomationElement ceilingElement)
   
   This method gets called from TAL and expects the plugin to return the element at the given point after creating its query ID. It is a requirement on the plugins to create a Query ID only until the ceiling element is reached. The ceiling element prevents plugins from encroaching into controls in the hierarchy that are better supported by other plugins. This API is called for the leaf nodes in the hierarchy.
   
   
2. public virtual IUITechnologyElement GetElementFromAutomationElement(AutomationElement element, AutomationElement ceilingElement)
   
   This method gets called for intermediate elements in the hierarchy Query ID creation here follows the same strategy as earlier, having ceiling element as the limit for creating the query ID.
   
   
3. public virtual IUITechnologyElement GetFocusedElement(AutomationElement ceilingElement)
   
   This is needed when recording so that the engine knows which control is being acted upon.
   
   
4. public virtual int GetControlSupportLevel(AutomationElement element)
   
   Gets the support level of this technology manager for the control corresponding to the given Automation Element. The framework uses this function to select the right technology manager for the control.

Technology Element

In UITechnologyElementProxy/UITechnologyElement we added

1. public virtual AutomationElement AutomationElement
   
   We use this AutomationElement value when we need to convert between technologies during playback.

In scenarios where a plugin does not implement these list of API’s we have default implementation to redirect to use the existing API functionalities based on window handles.

Resources

1. This blog helps troubleshoot issues for plugin writers.

2. Attached to this blog is a sample extension written for modern apps.