Suites API

Execute suites and retrieve results via CI/CD and other external systems.

List all suites

List all suites in your account.

Request

ET https://api.reflect.run/v1/suites

Response

{
  "suites": {
    "data": [
      {
        "name": "Regression Tests",
        "suiteId": "regression-tests",
        "created": 1638416940
      }
    ]
  }
}

Response Fields

Table 1. API listing suites responses

Field

Type

Description

suites

object

A wrapped collection of abridged suites.

suites.data.name

string

The suite name.

suites.data.suiteId

string

The suite identifier.

suites.data.created number

number

The suite's creation timestamp as an epoch in seconds.



List suite executions

List recent executions for a given suite.

Request

GET https://api.reflect.run/v1/suites/<suite-id>/executions

Response

{
  "suiteId": "regression-tests",
  "executions": {
    "data": [
      {
        "executionId": 47,
        "originExecutionId": 45,
        "attemptNumber": 2,
        "url": "https://app.reflect.run/suites/regression-tests/executions/45/attempts/2?accountId=...",
        "isFinished": true,
        "status": "passed"
      },
      {
        "executionId": 42,
        "attemptNumber": 1,
        "url": "https://app.reflect.run/suites/regression-tests/executions/42?accountId=...",
        "isFinished": false,
        "status": "pending"
      }
    ]
  }
}

Response Fields

Table 2. API listing suite executions

Field

Type

Description

suiteId

string

The identifier.

executions

object

A wrapped collection of abbreviated executions.

executions.data[].execution

number

The execution identifier for this attempt.

executions.data[].originExecutionId

optional number

The identifier of the original execution. Present only when the returned execution is a retry attempt. Absent for the original executions. Use it to correlate a retry with the original execution visible in the user interface.

executions.data[].attemptNumber

number

The attempt number. 1 for the original execution, 2 for the first retry, and so on.

executions.data[].url

string

The in-app URL for this execution. Includes /attempts/<n> when the attemptNumber is greater than 1 .

executions.data[].isFinished

boolean

The completion of the execution.

executions.data[].status

string

The status of the execution. One of: pending , canceled , passed, or failed.



Execute a suite

Triggers an execution of the specified suite.

The suite-id value for your suite should match the ID value displayed on the Suite Detail page, including casing. (See Integrating via API.)

Reflect identifies certain properties of a scheduled execution as “reserved”. These reserved fields can be overridden with values provided by the API request:

  • hostnames: Allows you to specify a replacement hostname value for a given target hostname.

  • cookies: Allows you to specify cookies that will be set when loading the test’s initial URL. A cookie consists of the following fields:

    • name (required string): The name of the cookie to be set.

    • value (required string): The cookie’s value.

    • domain (optional string): Host to which the cookie will be sent. Defaults to the host of the test’s initial URL.

    • expires (optional number): Epoch timestamp (in milliseconds) indicating when the cookie should be deleted. If unspecified, the cookie becomes a session cookie.

    • httpOnly (optional boolean): Indicates the cookie should be inaccessible to JavaScript on the page and only sent to the server.

    • maxAge (optional number): Number of milliseconds until the cookie expires. If both expires and maxAge are set, maxAge has precedence.

    • path (optional string): A URL path that must exist in a request URL in order to include the cookie in a request.

    • secure (optional boolean): Indicates that the cookie should only be sent to the server when a request is made with https:.

  • headers: Allows you to specify headers that will be sent when loading the test’s initial URL.

    • name (required string): The name of the header.

    • value (required string): The header value.

    • persist (optional boolean): Specifies whether this header should only be set in the initial request or for every subsequent HTTP request (ex: when overriding the Accept-Language header). If set to true, the Authorization header is only set for requests that match the hostname of the test’s original URL. Defaults to false.

  • localStorage and sessionStorage: Allows you to specify local and session storage values to be set on the test’s initial URL.

The variables field allows you to specify a definition for a variable or parameter used in the tests.

Finally, if your Reflect account has OAuthed with GitHub, you can specify a repository, its owner, and a commit SHA in the request to have Reflect automatically post a commit status to GitHub as the suite’s execution progresses.

Request

POST https://api.reflect.run/v1/suites/<suite-id>/executions

Response

{
  "overrides": {
       "agent": {
      "name": "agent-name"
    },
     "hostnames": [{
      "original": "prod.myapp.com",
      "replacement": "staging.myapp.com"
    }],
    "cookies": [{
      "name": "my-favorite-cookie",
      "value": "chocolate-chip",
      "domain": "myapp.com",
      "expires": 123456789,
      "httpOnly": false,
      "maxAge": 123,
      "path": "/",
      "secure": true
    }],
    "headers": [{
      "name": "X-Custom-Header",
      "value": "custom-value",
      "persist": false
    }],
    "localStorage": [{
      "key": "my-local-key",
      "value": "local-value"
    }],
    "sessionStorage": [{
      "key": "my-session-key",
      "value": "session-value"
    }]
  },
  "variables": {
    "username1": "user+${alpha(8)}@example.com",
    "password": "acompletelyunguessablepassword"
  },
  "gitHub": {
    "owner": "repository-owner",
    "repo": "my-repository-name",
    "sha": "0f4212320f2cb6734583ebef3a4928d78d4f3fde"
  }
}

Request Fields

overrides optional object: Overrides to apply to properties of all tests executed in this scheduled execution.

  • agent (object, optional)

    • name (string, required) : The name of the Tunnel/Agent to use for this execution.

variables optional object: Collection of ('name', 'definition') variable overrides to apply to this scheduled execution.

gitHub optional object: An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.

Response

{
  "suiteId": "regression-tests",
  "executionId": 3878,
  "overrides": {
    "hostnames": [
      {
        "original": "prod.myapp.com",
        "replacement": "staging.myapp.com"
      }
    ]
  },
  "gitHub": {
    "owner": "repository-owner",
    "repo": "my-repository-name",
    "sha": "0f4212320f2cb6734583ebef3a4928d78d4f3fde"
  },
  "url": "https://app.reflect.run/suites/regression-tests/executions/3878",
  "isFinished": false,
  "status": "pending",
  "tests": {
    "data": [],
    "bookmark": ""
  }
}

Response Fields

suiteId string: The identifier specified in the execution request.

executionId number: The unique numeric identifier assigned to the suite execution.

overrides optional object: Overrides applied to properties of all tests executed in this suite.

gitHub optional object: An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.

url string: The in-app URL that displays information and results about the suite execution.

isFinished boolean: A boolean indicating whether the suite execution has completed.

status string: One of: pending, canceled, passed or failed. Summarizes the state of the execution from its test instances.

tests object: A wrapped collection of test instances (test definition &amp; execution configuration) with individual test runs.

tests.bookmark optional string: Reserved for future - used to read additional test instances.

Execute a data-driven suite

Similar to the execute a suite endpoint, this endpoint invokes a Reflect test suite with a set of optional overrides.

However, this endpoint has the additional capability of accepting a set of data-driven values in a CSV file. Those values are used to override the (test, parameters) pairs for a specific Run Tests action in the Suite. Multiple CSV files can be sent to override different Run Tests actions.

Note

This endpoint requires sending data in a multipart request, consisting of:

Request

POST https://api.reflect.run/v1/suites/<suite-id>/executions/data-driven

Or via curl through the following command:

curl \
  -H 'X-API-KEY: <API-KEY>' \
  -F 'config={"actionNameByFilename": {"one.csv": "run-tests-1", "two.csv": "run-tests-2"}};type=application/json' \
  -F fileone=@one.csv \
  -F filetwo=@two.csv \
  https://api.reflect.run/v1/suites/<suite-id>/executions/data-driven

Request CSV file format

Each CSV file in the request should be in the following format:

Test Name or ID,Parameter One,Parameter Two,Parameter Three
1,value one,${var(one)},value three
1,,value two,${var(two)}
two,value one,value two,value three

The first column is the ID or name of a Test, all columns that follow are values for the parameter names specified in the header row of the CSV.

Note that any column whose value is left empty will not be overridden. For example, in the second data row ‘Parameter One’ is not overridden with any value.

Request JSON Fields

actionNameByFilename object: Collection of ('CSV file name', 'Custom Action Name') pairs, where the 'Custom Action Name' is the custom name you have assigned to a Run Tests action in the Suite.

overrides optional object: Overrides to apply to properties of all tests executed in this scheduled execution. See 'Execute a suite' for details on the properties that can be overridden.

variables optional object: Collection of ('name', 'definition') variable overrides to apply to this scheduled execution.

gitHub optional object: An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.

Response Fields

suiteId string: The identifier specified in the execution request.

executionId number: The unique numeric identifier assigned to the suite execution.

overrides optional obejct: Overrides applied to properties of all tests executed in this suite.

gitHub optional object: An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.

url string: The in-app URL that displays information and results about the suite execution.

isFinished boolean: A boolean indicating whether the suite execution has completed.

status string: One of: pending, canceled, passed or failed. Summarizes the state of the execution from its test instances.

tests object: A wrapped collection of test instances (test definition &amp; execution configuration) with individual test runs.

tests.bookmark optional string: Reserved for future - used to read additional test instances.

Get execution status

Returns the completion status of the specified suite execution. You can retrieve the status of a specific execution attempt by using the optional attempt query parameter.

Request:

GET https://api.reflect.run/v1/suites/<suite-id>/executions/<execution-id>?attempt=<n>
Table 3. Get execution request query parameters

Parameter

Type

Required

Description

attempt

number

No

The attempt number to retrieve.

Attempt 1 is the original execution, attempt 2 is the first retry, and so on. If the value exceeds the total number of attempts, the latest attempt is returned.

Omitting the parameter returns the latest attempt.



Response

{
  "suiteId": "regression-tests",
  "executionId": 36,
  "url": "https://app.reflect.run/suites/regression-tests/executions/36/attempts/2?accountId=...",
  "isFinished": true,
  "status": "passed",
  "tests": { ... }
}

Response Fields

Table 4. Get execution responses

Field

Type

Description

suiteId

string

The identifier specified in the execution request.

executionId

number

The unique numeric identifier assigned to the suite execution.

overrides

optional object

Overrides applied to properties of all tests executed in this suite.

gitHub

optional object

An object specifying the GitHub repository name, its owner, and the commit SHA to post the status of the execution.

url

string

The in-app URL that displays information and results about the suite execution. Includes /attempts/<n> when the attempt number is greater than 1 .

isFinished

boolean

A boolean indicating whether the suite execution has completed.

status

string

Reflects the state as of the requested attempt. One of: pending , canceled , passed, or failed.

tests

object

A wrapped collection of test instances (test definition and execution configuration) with individual test runs.

tests.bookmark

optional string

Reserved for future - used to read additional test instances.

tests.data.instanceId

number

The identifier for a logical test instance (definition and configuration) in the execution.

tests.data.testId

number

The identifier for the test.

tests.data.actionName

optional string

The name assigned to the Run Tests action, the instance originated from, if one was given.

tests.data.startingUrl

string

The initial URL used by the test, accounting for any user-specified overrides.

tests.data.status

string

Summarizes the outcome of the test instance from its runs. One of: pending, passed, or failed.

tests.data.runs

object

An array of test runs for a single test instance, with completion timestamp and status.

tests.data.runs.browser

string

The browser used to execute this test run. One of: Chrome, Edge, Firefox, or Safari.

tests.data.runs.status

string

The outcome of the test run. Either passed or failed.

tests.data.runs.variable

object

The variables that were used or created by the test run.

tests.data.runs.startTime

number

Timestamp of when the test run started.

tests.data.runs.endTime

number

Timestamp of when the test run stopped.

tests.data.runs.runTime

number

Total time that the test run spent running (endTime - startTime).

tests.data.runs.timestamp

number

Timestamp when the test run's execution was recorded in Reflect.

tests.data.runs.videoUrl

string

The URL of a video recording of the test run.

tests.data.runs.stepCount

number

The number of steps in the executed definition.

tests.data.runs.failedStepIndex

optional object

A dictionary containing the failed steps from the test run. The key represents the step index (zero-based), and the value is a summary error message describing the failure encountered at that step. Only defined when the status of the run is failed.



Table 5. Error responses for the get execution status

Status

Description

400

The attempt parameter is less than 1 .

401

Authentication failed.

404

Unknown suite or execution.



Cancel an execution

Attempts to cancel the specified suite execution.

Request

PATCH https://api.reflect.run/v1/suites/<suite-id>/executions/<execution-id>/cancel

Response

{
  "suiteId": "regression-tests",
  "executionId": 3878,
  "success": true
}

Response Fields

suiteId string: The identifier specified in the execution request.

executionId number: The unique numeric identifier assigned to the suite execution.

success boolean: A boolean indicating whether the suite execution was successfully canceled.

Upload a Mobile Build

Request

POST https://uploads-api.reflect.run/v1/mobile/builds?name=<build-name>

This endpoint allows you to upload a new mobile app build (for example, an .apk or .ipa) to Reflect Grid and receive a build ID. The returned build ID can then be used when executing a mobile test suite via the Suites API.

This is commonly used in CI/CD workflows to dynamically test newly generated mobile builds without manual updates. Check Reflect Suite Execution API: Trigger Tests with Specific Mobile Builds to learn more.

Publication date: