There have been several questions on the different scenarios related to running automated tests in various workflows and which versions can / cannot be used.
Let’s broadly use the following 2 categories of scenarios:
- Test Controller/Test Agent (TC/TA) scenarios – this refers to on-premises load testing, running automated tests in the XAML build workflows, triggering an on-demand execution of automated tests using MTM and creating and managing lab environments using Visual Studio Lab Management.
- Team Build / Release Management scenarios – this is for running functional tests in the Team Build / Release Management workflows, using the ‘Visual Studio Test Agent Deployment’ and ‘Run Functional Tests’ tasks from the task catalog.
Agents for Visual Studio 2013 Update 5
- Can be used for TC/TA scenarios
- Cannot be used for Team Build/Release Management scenarios
- Can run tests that were built using Visual Studio 2013 and Visual Studio 2015.
Agents for Visual Studio 2015 and Agents for Visual Studio 2015 Update 3
- Cannot be used for TC/TA scenarios
- Can be used for Team Build / Release Management scenarios
- Can run tests that were built using Visual Studio 2015 and Visual Studio 2013.
Agents for Visual Studio 2017 Preview
- Can be used for TC/TA scenarios
- Cannot be used for Team Build / Release Management scenarios in the current Preview release. The support for this scenario will be added in the Visual Studio 2017 RTM timeframe.
- Can run tests that were built using Visual Studio 2017 RC.
- ‘Visual Studio Test Agent Deployment’ and ‘Run Functional Tests’ tasks in the Team Build / Release Management scenarios use the Agents for Visual Studio 2015 Update 3 package. As such, the tasks in their current form can be used for running tests built using Visual Studio 2015 and Visual Studio 2013. In the TFS 2017 Update 1 timeframe, the tasks will be updated to leverage Agents for Visual Studio 2017 so that you can then also run tests built using Visual Studio 2017 in Team Build/Release Management.
Content for the various download locations for these SKUs is being updated as well.