2021 September Release

Fabasoft app.test StarterPermanent link for this heading

It is possible to run multiple test from a predefined list on a defined pool of Fabasoft app.test Agents using the Fabasoft app.test Starter. The Fabasoft app.test Starter takes care of scheduling jobs onto the agents based on the OS on which the test should be executed and in which browser the test should run. To be able to schedule the tests according to those properties Fabasoft app.test Starter needs to know the OS and the installed browser of each agent that should run tests. The scheduling supports multiple backends to test against with a specified number of scopes.

The Fabasoft app.test Starter JAR file needs to be in the same directory as the JAR of Fabasoft app.test Console Player as it uses the console player classes to execute the tests.

The Fabasoft app.test Starter uses a template configuration where specific parts are overwritten at runtime (e.g. Agent hostname, used scope, …).

How to Use the Fabasoft app.test StarterPermanent link for this heading

The Fabasoft app.test Starter is started with the following command line:

Command Line

java -jar <path to apptest-starter.jar> -config:<path to template config> -agentlist:<path to agent list> -testlist:<path to test list> -schedule:<path to schedule> -scopes:<first scope>-<last scope> -tag:<comma separated tags to execute> -excludetags:<comma separated tags to exclude>

The parameters are described in the following table:

Parameter

Definition

config

Defines the path to the template apptest-config file, which specifies the settings that are used for each test.

agentlist

Defines the path to the list of agents that should be used to execute the tests defined in the test list. An example agent list can be found below.

testlist

Defines the path to the list of tests that should be executed. An example test list can be found below.

schedule

Defines which tests are not allowed to run concurrently. An example schedule can be found below.

scopes

Defines the scopes to use for the tests. Scopes are given in the min-max format (e.g. 1-10) and are the same for all backends found in the test list.

tag

A list of tags that will be executed. Tags can be applied as options to the test keyword ([tag="p0529, ccs_dam"]).

excludetags

A list of tags that will be omitted during execution. This list will be applied after the list provided in the tag parameter.

Example Agent List

The agent list is formatted as CSV file and defines the agents that are used to execute the tests. To be able to schedule the tests correctly Fabasoft app.test Starter needs to know the hostname with optional port for the agent and optional port for the adaptor, OS and the supported browsers of each agent. Each line of the list corresponds to one Fabasoft app.test Agent and is constructed as follows:

Syntax

<hostname or ip>[:port][:adaptorport];<OS>;<browser1> <browser2>

The possible OS and browser values are predefined:

Parameter

Possible Values

OS

WINDOWS, LINUX, MAC

Browser

APPLE_SAFARI,
GOOGLE_CHROME,
GOOGLE_CHROME_HEADLESS,
MICROSOFT_EDGE_CHROMIUM,
MOZILLA_FIREFOX,
MOZILLA_FIREFOX_HEADLESS

The following example shows a complete agent list:

Example

win1.example.com;WINDOWS;MOZILLA_FIREFOX MICROSOFT_EDGE_CHROMIUM
win2.example.com;WINDOWS;MOZILLA_FIREFOX
MICROSOFT_EDGE_CHROMIUM
lin1.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com;LINUX;MOZILLA_FIREFOX

lin2.example.com:1234;LINUX;MOZILLA_FIREFOX

mac1.example.com;MAC;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

mac2.example.com:1122:4360;MAC;GOOGLE_CHROME MOZILLA_FIREFOX APPLE_SAFARI

If no ports are specified, the ports defined in the given config are used.

Example Test List

The test list is like the agent list formatted as CSV file and represents one test per line that should be executed by app.test Starter. Each test entry holds the path to the test file, the OS on which the test should run, the browser to use, the backend URL to test against and optionally the path to a parameter table file. This leads to the following line format:

Syntax

<path to .apptest>;<OS>;<browser>;<backend URL>[;<path to params>]

The following example shows a complete test list:

Example

D:\path\to\ex.apptest;WINDOWS;MOZILLA_FIREFOX;https://b1.ex.com/folio
D:\path\to\ex.apptest;WINDOWS;GOOGLE_CHROME;https://b2.ex.com/folio

D:\path\to\ex.apptest;MAC;APPLE_SAFARI;https://b1.ex.com/folio;D:\path\to\params.apptest

D:\path\to\ex.apptest;LINUX;MOZILLA_FIREFOX;https://b2.ex.com/folio

D:\path\to\ex.apptest;MAC;GOOGLE_CHROME;https://b2.ex.com/folio

D:\path\to\ex.apptest;LINUX;GOOGLE_CHROME;https://b1.ex.com/folio

D:\path\to\ex.apptest;WINDOWS;MOZILLA_FIREFOX;https://b1.ex.com/folio

D:\path\to\ex.apptest;WINDOWS;GOOGLE_CHROME;https://b2.ex.com/folio

D:\path\to\ex.apptest;WINDOWS;
MICROSOFT_EDGE_CHROMIUM;https://b1.ex.com/folio

Example Schedule

A schedule defines groups of tests that are not allowed to run together. Exclusive tests of a group never run with other exclusive or protected tests of that group. Protected tests may run together with other protected tests but not with exclusive tests. For test names, stars can be used as wildcards.

The final category will be executed on all backends. The path to the test file has to be defined relative to the schedule file (no wildcards are allowed).

Example

# Group Wastebasket
Wastebasket;exclusive;*GlobalWastebasketACLSet
Wastebasket;exclusive;*TeamroomWastebasket
Wastebasket;protected;*ContactManagement

# Group License
License;exclusive;*LicenseCheck
License;protected;*

FinalTestGroup;final;Tests/finalize.apptest

Example Output of Fabasoft app.test StarterPermanent link for this heading

Example

0/0/100 - 0008 - F0341_Test1 - 10.10.72.80 - Windows/Chrome – b1.example.com
OK - 0004 - F0046_Test2 - 10.10.72.87 - Windows/Chrome – b1.example.com
Agent: mac1.example.com is not alive
...
Finished: 100 - NOK: 3 - OK: 97 - Unschedulable: 0 - Duration: 00:25:44

When a test is started on an agent, Fabasoft app.test Starter prints the current state of the execution list (<failed tests>/<succeeded tests>/<number of tests>) and information about the started test (<scope> - <test name> - <agent hostname> - <OS>/<browser> - <backend hostname>). When a test finishes the result of the test (OK or NOK) is printed alongside with the same information about the test as described before.

If a Fabasoft app.test Agent becomes unresponsive Fabasoft app.test Starter starts polling the agent in increasing periods of time until it becomes responsive again or all tests are finished. If there are tests that were scheduled but could not be started because of an agent becoming unavailable, a notification is printed at the end of the run.

After all tests are finished, a summary is printed including the duration.