2018 April Release

How to Run Multiple Tests From Command LinePermanent link for this heading

It is possible to run multiple test from a predefined list on a defined pool of Fabasoft app.test Agents using the Fabasoft app.test Starter. The Fabasoft app.test Starter takes care of scheduling jobs onto the agents based on the OS on which the test should be executed and in which browser the test should run. To be able to schedule the tests according to those properties Fabasoft app.test Starter needs to know the OS and the installed browser of each agent that should run tests. The scheduling supports multiple backends to test against with a specified number of scopes.

The Fabasoft app.test Starter jar file needs to be in the same directory as the jar of Fabasoft app.test Console Player as it uses the Console Player classes to execute the tests.

The Fabasoft app.test Starter uses a template configuration where specific parts are overwritten at runtime (e.g. Agent hostname, used scope, …).

How to Use the Fabasoft app.test StarterPermanent link for this heading

The Fabasoft app.test Starter is started with the following command line:

java -jar <path to apptest-starter.jar> -config:<path to template config> -agentlist:<path to agent list> -testlist:<path to test list> -schedule:<path to schedule> -scopes:<first scope>-<last scope>

The parameters are described in the following table:

Parameter

Definition

config

Defines the path to the template configuration file, which specifies the settings that are used for each test.

agentlist

Defines the path to the list of agents that should be used to execute the tests defined in the test list. An example agent list can be found below.

testlist

Defines the path to the list of tests that should be executed. An example test list can be found below.

schedule

Defines which tests are not allowed to run concurrently. An example schedule can be found below.

scopes

Defines the scopes to use for the tests. Scopes are given in the min-max format (e.g. 1-10) and are the same for all backends found in the test list.

Example Agent ListPermanent link for this heading

The agent list is formatted as CSV file and defines the agents that are used to execute the tests. To be able to schedule the tests correctly Fabasoft app.test Starter needs to know the hostname, OS and the supported browsers of each agent. Each line of the list corresponds to one Fabasoft app.test Agent and is constructed as follows:

<hostname or ip>[:port];<OS>;<browser1> <browser2>

The possible OS and browser values are predefined:

Parameter

Possible Values

OS

WINDOWS, LINUX, MAC

Browser

MOZILLA_FIREFOX, MICROSOFT_INTERNET_EXPLORER, GOOGLE_CHROME, APPLE_SAFARI

The following example shows a complete agent list:

win1.example.com;WINDOWS;MOZILLA_FIREFOX MICROSOFT_INTERNET_EXPLORER GOOGLE_CHROME

win2.example.com;WINDOWS;MOZILLA_FIREFOX MICROSOFT_INTERNET_EXPLORER GOOGLE_CHROME

lin1.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com:1234;LINUX;MOZILLA_FIREFOX

mac1.example.com;MAC;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

mac2.example.com;MAC:1122;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

If no port is specified, the port defined in the given config is used.

Example Test ListPermanent link for this heading

The test list is like the agent list formatted as CSV file and represents one test per line that should be executed by app.test Starter. Each test entry holds the path to the .ducx-test file, the OS on which the test should run, the browser that should be used and the backend URL to test against. This leads to the following line format:

<path to .ducx-test>;<OS>;<browser>;<backend URL>[;<delay in ms>]

For OS and browser the same predefined values are used as defined in 5.5.1.1 Example agent list”.

The following example shows a complete test list:

D:\path\to\example.ducx-test;WINDOWS;MOZILLA_FIREFOX;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;MICROSOFT_INTERNET_EXPLORER;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;LINUX;MOZILLA_FIREFOX;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;MAC;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;LINUX;GOOGLE_CHROME;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;MAC;APPLE_SAFARI;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;MOZILLA_FIREFOX;https://backend1.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;GOOGLE_CHROME;https://backend2.ex.com/folio

D:\path\to\example.ducx-test;WINDOWS;MICROSOFT_INTERNET_EXPLORER;https://backend1.ex.com/folio

Example SchedulePermanent link for this heading

A schedule defines groups of tests that are not allowed to run together. Exclusive tests of a group never run with other exclusive or protected tests of that group. Protected tests may run together with other protected tests but not with exclusive tests. For test names, stars can be used as wildcards.

# Group Wastebasket
Wastebasket;exclusive;*GlobalWastebasketACLSet
Wastebasket;exclusive;*TeamroomWastebasket
Wastebasket;protected;*ContactManagement

# Group License
License
;exclusive;*LicenseCheck
License;protected;*

Example Output of Fabasoft app.test StarterPermanent link for this heading

0/100 - 0008 - F0341_Test1 - 10.10.72.80 - Windows/Chrome backend1.example.com

OK - 0004 - F0046_Test2 - 10.10.72.87 - Windows/Chrome backend1.example.com

Agent: mac1.example.com is not alive

...

Finished: 100 - NOK: 3 - OK: 97 - Unschedulable: 0 - Duration: 00:25:44

When a test is started on an agent, Fabasoft app.test Starter prints the current state of the execution list (<failed tests>/<number of tests>) and information about the started test (<scope> - <test name> - <agent hostname> - <OS>/<browser> - <backend hostname>). When a test finishes the result of the test (OK or NOK) is printed alongside with the same information about the test as described before.

If a Fabasoft app.test Agent becomes unresponsive Fabasoft app.test Starter starts polling the agent in increasing periods of time until it becomes responsive again or all tests are finished. If there are tests that were scheduled but could not be started because of an agent becoming unavailable, a notification is printed at the end of the run.

After all tests are finished, a summary is printed including the duration.