Gearset provides a public External Testing API as part of the Automation API for receiving testing data from any third-party testing product. For testing data to be displayed in Gearset, an integration must be created to transform the data from the third-party product and send it to Gearset via the API.
The External Testing data is currently used for display purposes only, which means it currently cannot be used to gate (block) deployments.
External Testing data in Gearset
In Gearset, the External test runs are stored against the CI job runs.
The External tests status label on an environment in Pipelines:
The External tests status column in CI jobs:
Each CI job run can contain the details of any number of External test runs. The status label shown in the Pipelines and CI Jobs sections is an aggregate over all the External test runs with data in Gearset. Clicking on the status label opens the individual External test run details.
The External test results page lists all the test data available for a CI job run:
What testing data can be sent to Gearset
The following data can be sent to and displayed in Gearset via the External Testing API:
Provider Name - A mandatory free text field for identifying the testing provider in the UI.
Provider Run ID - An optional free text external ID to allow identifying a test run in the third-party product.
Results URL - An optional URL to the test results page in the third-party product. If provided, a button to open the link will be in the External test results table.
Status Class - A mandatory Gearset status classification used to determine how the status is displayed in the Pipelines and CI Jobs sections. All External test runs must be mapped to one of the following values:
Succeeded - all tests have finished and passed (or not run).
Failed - some critical tests have failed (or not run).
Warning - some tests have failed, but these are not critical.
In Progress - the test run is in progress.
Scheduled - the test run is scheduled for execution, but has not yet started.
Not Run - there is a test run that should be run, but it's not scheduled yet.
It's the caller's choice of how to map the test run results, but we recommend sticking to the interpretation mentioned above.
Status - The optional (short) status value of the test run as it is shown in the third-party product. This is the status displayed in the External test results. If missing, the Status Class is shown instead.
Status Message - An optional (longer) informative message about the current status of the test run, if available from the third-party product (e.g., if there is an error, the error message). If provided, there will be a tooltip with the message when hovering over the Status in the External test results.
Start Time - The start time of the test run.
End Time - The end time of the test run.
Getting Started With the External Testing API
Generate a Gearset API access token must be created.
Set up an outgoing webhook to send the CI job run ID on completion of a deployment.
Use the External Testing API documentation to implement (or test) the Gearset integration.