> ## Documentation Index
> Fetch the complete documentation index at: https://checksum.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CI/CD Integration

> Run your Checksum suite automatically on pull requests, merges, and schedules. The simplest option on GitHub is the Checksum GitHub Action. You can also run the CLI on your own runners (GitHub Actions or GitLab), or call the REST API from any pipeline. Every option can gate the build on the result and send failures to auto-healing.

## At a glance

| Option | Where the tests run | Good for |
| - | - | - |
| [GitHub Action](#run-checksum-with-the-github-action) | Checksum's cloud, in one step with nothing to install | GitHub repos that want to trigger a run, gate on it, and auto-heal |
| [CLI in GitHub Actions](#run-the-cli-in-github-actions) | Your runner | Running on your own infrastructure, or reaching an app on your network |
| [CLI in GitLab CI/CD](#run-the-cli-in-gitlab-ci/cd) | Your runner | GitLab projects |
| [REST API](#trigger-and-gate-a-run-from-any-ci-system) | Checksum's cloud, triggered by a script | Jenkins, CircleCI, Buildkite, Azure Pipelines, or anything else |

<div className="ai-ref">
  <Accordion title="Reference for AI: CI/CD at a glance" icon="robot">
    | Interface | Entry point | Where tests execute | Best for |
    | - | - | - | - |
    | GitHub Action | `uses: checksum-ai/test-run-action@v2` | Checksum's cloud (one step, no installs) | GitHub repos that need to trigger a run, gate on it, and auto-heal |
    | CLI | `npx checksumai test` in GitHub Actions | Your runner (Node + Playwright install per job) | Execution on your own infrastructure, or local network access to the app under test |
    | CLI | `npx checksumai test` in GitLab CI/CD | Your runner | GitLab projects |
    | REST API | `POST https://api.checksum.ai/public-api/v2/execution/grep` + `GET https://api.checksum.ai/public-api/v1/execution/status/run/{runId}` | Checksum's cloud (script trigger + poll) | Jenkins, CircleCI, Buildkite, Azure Pipelines, internal orchestrators, or anything not listed |
    | Auto-heal | `auto-heal: true` · `--cksm-auto-heal` · `autoHeal` | Checksum's cloud | Fixing tests that failed in CI and opening a PR |

    * Secret: store the project API key as a CI secret named `CHECKSUM_API_KEY`. Checksum sets up the tests repository during onboarding, so there is nothing to initialize.
    * Cross-repo: if the tests live in a different repository than the pipeline, you also need a Personal Access Token with read access to the tests repo.
    * Gate on `verdict`: pass the build only when `verdict` is `"pass"`. Don't gate on the `passed`/`failed` counts.
    * No cancel: cancelling a CI job only stops polling. An API-triggered Checksum run continues until it finishes.
    * Sharding requires `checksumai` 4.4.0 or later on the tests branch.
  </Accordion>
</div>

<div className="part dev"><span className="part-icon">{"</>"}</span><div><div className="part-title">Developer guide</div><div className="part-sub">GitHub Action, the CLI in GitHub Actions and GitLab, the REST API, and auto-heal</div></div></div>

## Run Checksum with the GitHub Action

The [`checksum-ai/test-run-action`](https://github.com/checksum-ai/test-run-action) action starts a Checksum run in a single step, with auto-heal on failure built in. The tests run in Checksum's cloud, so your job doesn't need Node, Playwright, or browsers. Add your [API key](/docs/authentication#your-api-key) as a repository secret named `CHECKSUM_API_KEY` (from **Settings → Project Settings** in the Checksum web app; in GitHub, go to **Settings → Secrets and variables → Actions**), then add this workflow:

```yaml .github/workflows/checksum.yml theme={null}
name: Checksum tests
on: pull_request

permissions:
  contents: read
  pull-requests: read

jobs:
  checksum:
    runs-on: ubuntu-latest
    steps:
      - uses: checksum-ai/test-run-action@v2
        with:
          api-key: ${{ secrets.CHECKSUM_API_KEY }}
          grep: 'checkout'
          auto-heal: true
```

That's the whole step. The action finds the source PR on its own (from the event payload on `pull_request`, or via the GitHub API on `push`) and passes it to auto-heal, so healing progress shows up as a comment on the right PR. Checksum posts these PR comments itself through its GitHub App, so your workflow doesn't need its own comment steps or `pull-requests: write`.

<Tip>
  **Have a coding agent build it**

  The [CI Setup Prompts](/docs/ci-setup-prompts) are fill-in-the-blanks prompts that have Claude Code, Cursor, or Copilot write this workflow for your repository, including a preview URL, sharding, and auto-heal.
</Tip>

Choose **which tests to run** with exactly one of these inputs: `grep` (a name pattern), `affected` (only tests affected by the PR's changes), `suite-ids`, `test-ids`, or `collection-id`. The endpoints behind them are described in [Running Tests → Execution endpoints](/docs/running-tests#start-a-cloud-run-from-the-rest-api), and the complete inputs/outputs reference and changelog are in the [action's README](https://github.com/checksum-ai/test-run-action#readme).

<Warning>
  **`@v2` is a breaking release**

  An empty test selection (for example, a `grep` that matches nothing) now **fails** the step instead of passing. Read the [v2.0.0 release notes](https://github.com/checksum-ai/test-run-action/releases/tag/v2.0.0) for the full list of behavior changes before upgrading from `@v1`.
</Warning>

### Heal failures, with or without a PR

With `auto-heal: true`, a failed run starts [Auto-Healing](/docs/auto-healing) automatically. By default the fixes arrive as a PR, and progress is reported as a comment on the originating PR, with no extra wiring on `pull_request` events. To start heal sessions without auto-creating a PR, add `auto-create-pr: false`:

```yaml theme={null}
- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: 'checkout'
    auto-heal: true
    auto-create-pr: false
```

### Make the workflow wait and pass or fail

By default the action exits as soon as Checksum accepts the run (about 15 seconds), and you hear about results through the PR comment. Set `wait: true` if the workflow check itself should pass or fail on the outcome. The step succeeds only when the run's `verdict` is `pass`. A failed run, an empty selection, a cancelled run, an infrastructure error, or a timeout all fail the step.

```yaml theme={null}
- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: 'checkout'
    auto-heal: true
    wait: true
```

<Tip>
  **Runner minutes**

  `wait: true` holds a runner for the whole run, typically 5–25 minutes. If runner cost matters, keep `wait: false` and rely on the PR-comment notification.
</Tip>

### Split a large run across shards

Set `shard-count` (2–40, in `grep` and `affected` modes) to run the tests in parallel and merge the results into one verdict:

```yaml theme={null}
- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: '@smoke'
    shard-count: 8
    wait: true
    wait-timeout-seconds: 1800
```

<Warning>
  **Before your first sharded run**

  Update `checksumai` on your tests branch to **4.4.0 or later** and commit it (`npm install checksumai@latest` is the safe default). Older versions can fail to merge shard reports, which delays the final `verdict` well past a normal run. With `wait: true` and no `wait-timeout-seconds`, the step then runs until the job's own `timeout-minutes`. Always set `wait-timeout-seconds` (or a job-level `timeout-minutes`) alongside `wait: true`.
</Warning>

Sharding works together with `auto-heal` from action **v2.1.0** onward: once the shards merge, a merged run that ends `failed` is healed just like a non-sharded run. If the shards never merge, the heal is never evaluated. Older action versions reject the combination; `@v2` already resolves to the latest 2.x. Check your suite against [Sharding](/docs/sharding) before using a large shard count.

### Test each PR's preview deployment

If every PR is deployed to its own preview URL, pass `env-overrides` (grep mode only) to point the run at it:

```yaml theme={null}
- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: 'checkout'
    auto-heal: true
    env-overrides: |
      {"BASE_URL": "https://pr-${{ github.event.pull_request.number }}.preview.example.com"}
```

### Choose the tests-repo branch

In `grep` mode, `branch` picks the branch of your **tests** repository that the run checks out. Leave it out to use the tests repo's default branch. Set it explicitly when the workflow runs from your application repo, so the run (and any heal PR, which defaults to the run's branch) uses your tests repo's integration branch rather than the application PR's branch:

```yaml theme={null}
- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: 'checkout'
    branch: main          # tests-repo branch, not the app PR branch
    auto-heal: true
```

### Pin the action version

Use `@v2` for compatible updates (recommended), `@v2.0.0` for a release-specific tag, or a full commit SHA if you need a guarantee of immutability, since tags can technically be moved by the repo owner (see [GitHub's guidance on pinning third-party actions](https://docs.github.com/en/actions/reference/security/secure-use#using-third-party-actions)). `@v1` still exists with the pre-sharding behavior but no longer gets updates.

<div className="ai-ref">
  <Accordion title="Reference for AI: checksum-ai/test-run-action@v2" icon="robot">
    Wraps the public-API execution endpoints in a single step with built-in auto-heal on failure. Tests execute in Checksum's cloud, so no Node, Playwright, or browsers are needed on the runner. Source PR auto-detected: from the event payload on `pull_request`, via the GitHub API on `push`.

    #### Required secrets

    | Secret | Where to get it | Required |
    | - | - | - |
    | `CHECKSUM_API_KEY` | Checksum web app → **Settings → Project Settings**. Add under GitHub **Settings → Secrets and variables → Actions**. | Yes |

    #### Action inputs

    | Input | Required | Default | Description |
    | - | - | - | - |
    | `api-key` | Yes | — | Checksum API key, e.g. `${{ secrets.CHECKSUM_API_KEY }}` |
    | `grep` | One execution mode | — | Name pattern (substring or regex). You want to filter tests by name pattern. Calls `POST /public-api/v2/execution/grep`. |
    | `affected` | One execution mode | — | You want to run only the tests affected by the PR's changed files. Calls `POST /public-api/v1/affected-tests`, then a grep dispatch. |
    | `suite-ids` | One execution mode | — | You want to run one or more specific test suites, as comma-separated UUIDs. Calls `POST /public-api/v1/execution/suite`. |
    | `test-ids` | One execution mode | — | You want to run a specific list of tests by UUID. Calls `POST /public-api/v1/execution/tests`. |
    | `collection-id` | One execution mode | — | You want to run a saved collection. Calls `POST /public-api/v1/execution/collection/{id}` (`/collection/:id`). |
    | `auto-heal` | No | — | `true` runs Auto-Healing when the run fails. Healing progress is reported as a comment on the originating PR. |
    | `auto-create-pr` | No | Healing opens a PR | `false` starts heal sessions without auto-creating a PR |
    | `wait` | No | Exit when the dispatch is accepted (\~15s) | `true` waits for the run and exits on its `verdict`. Holds a runner for the full run (typically 5–25 minutes). |
    | `wait-timeout-seconds` | No | — | Upper bound on how long `wait: true` holds the runner. Always set it with `wait: true`. |
    | `shard-count` | No | — | `2`–`40`. Honored in `grep` and `affected` modes. Composes with `auto-heal` from action v2.1.0. |
    | `env-overrides` | No | — | JSON object of per-run environment variables. `grep` mode only. |
    | `branch` | No | Tests repo's default branch | Branch of the **tests** repository to check out for the run. `grep` mode only. Set it to the tests repo's integration branch when the workflow runs from application code. |
    | `pr-number` | No | Auto-detected on `pull_request` / `pull_request_target` | Source PR that receives Checksum's comments. Set it only on other events. |
    | `repo-name` | No | From `github.repository` | Source repository name, bare repo only (no owner). |
    | `metadata` | No | — | JSON object of free-form metadata attached to each healing session, e.g. correlation IDs. |
    | `poll-interval-seconds` | No | `15` | How often to poll status when `wait` is `true`. |
    | `github-token` | No | `${{ github.token }}` | Looks up the open PR for the branch on `push` events so `pr-number` can be resolved. Needs `pull-requests: read`. Ignored on `pull_request`. |

    Set exactly one execution mode: `grep`, `affected`, `suite-ids`, `test-ids`, or `collection-id`.

    #### Action outputs

    | Output | Description |
    | - | - |
    | `verdict` | Server-computed CI verdict when `wait` is `true`: `pass`, `fail`, or `pending` (only if it timed out). The step's exit code gates on this. Empty when `wait` is `false`. |
    | `status` | Raw final run status when `wait` is `true` (e.g. `passed`, `healed`, `failed`, `process-error`, `cancelled`, `timeout`). Empty when `wait` is `false`. |
    | `test-run-id` | Test run UUID, returned at dispatch for sharded and non-sharded runs. Poll it with `GET /public-api/v1/execution/status/run/{runId}`. |
    | `job-name` | Job name of a non-sharded run; empty for a sharded run. Prefer `test-run-id`. |
    | `affected-test-ids` | JSON array of test IDs from `/affected-tests` when `affected` is `true`. |
    | `grep-pattern` | Grep pattern derived from the affected test IDs. Set only when `affected` found tests. |

    #### Step exit with `wait: true`

    | Outcome | Step exit |
    | - | - |
    | `verdict: pass` | success |
    | `verdict: fail` (a failed run, an empty selection, a cancelled run, or an infra/process error) | failure |
    | timeout (`verdict: pending`, the run never reached a terminal status) | failure |

    #### Version pins

    | Reference | What you get |
    | - | - |
    | `@v2` | Compatible updates (recommended) |
    | `@v2.0.0` | A release-specific tag |
    | A full commit SHA | The only guarantee of immutability |
    | `@v1` | Pre-sharding behavior, no longer updated |

    Rules: `@v2` fails the step on an empty selection. Sharding requires `checksumai` ≥ 4.4.0 on the tests branch; older versions can fail to merge shard reports and delay the terminal `verdict`. Older action versions reject `shard-count` + `auto-heal` client-side. From application-code CI, set `branch` to the tests repo's integration branch. PR comments come from Checksum's GitHub App, so workflows need no comment steps and only `pull-requests: read`.
  </Accordion>
</div>

## Run the CLI in GitHub Actions

Use this when the tests should run on **your own** runner, for example to reach an app that's only available on your network. Create `.github/workflows/checksum-tests.yml` in your tests repository:

```yaml .github/workflows/checksum-tests.yml theme={null}
name: Run Checksum Tests

on:
  # Choose your triggers:
  # schedule:
  #   - cron: '0 0 * * *'  # Runs every day at 00:00 UTC
  workflow_dispatch:        # Allows manual triggering

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install npm dependencies
        run: npm install

      - name: Install Playwright with dependencies
        run: npx playwright install --with-deps

      - name: Download .env from Checksum
        run: npx checksumai dotenv --download --api-key="${{ secrets.CHECKSUM_API_KEY }}"

      - name: Run Checksum tests
        run: npx checksumai test
        env:
          CHECKSUM_API_KEY: ${{ secrets.CHECKSUM_API_KEY }}
          USERNAME: ${{ secrets.USERNAME }}
          PASSWORD: ${{ secrets.PASSWORD }}
          LOGIN_URL: ${{ secrets.LOGIN_URL }}
          BASE_URL: ${{ secrets.BASE_URL }}
          CI: true
```

Add the secrets the workflow uses under **Settings → Secrets and variables → Actions**: `CHECKSUM_API_KEY` (required), plus the test user's `USERNAME` and `PASSWORD`, your application's `BASE_URL`, and its `LOGIN_URL`. Setting `CI: true` turns on report uploads and auto-heal PRs by default (see [checksum.config.ts](/docs/test-repository#configure-checksum-config-ts)). Variables you set explicitly always override the downloaded `.env` ([Environment variables](/docs/environments#work-with-environment-variables)). To run only the tests a PR affects, add `--cksm-affected` to the test command ([Selecting tests](/docs/running-tests#pick-which-tests-to-run)).

### If your tests live in another repository

When the workflow runs in your application repo but the tests live elsewhere, check out the tests repo with a Personal Access Token stored as `TEST_REPO_PAT`:

```yaml theme={null}
steps:
  - uses: actions/checkout@v4

  - name: Checkout test repo
    uses: actions/checkout@v4
    with:
      repository: org/your-test-repo
      ref: main
      token: ${{ secrets.TEST_REPO_PAT }}
      path: .

  # ... rest of the steps
```

### Try the workflow

1. Commit the workflow file to your main branch.
2. Go to **Actions**, select the workflow, and click **Run workflow**.
3. Watch the run to confirm tests execute and the report appears in [Test Results](/docs/results-and-reports).

<div className="ai-ref">
  <Accordion title="Reference for AI: CLI in GitHub Actions" icon="robot">
    Steps in order: `actions/checkout@v4` → `npm install` → `npx playwright install --with-deps` → `npx checksumai dotenv --download --api-key="${{ secrets.CHECKSUM_API_KEY }}"` → `npx checksumai test` with env vars below. Optional flags: `--cksm-affected` (PR-affected tests only), `--cksm-auto-heal` (heal failures).

    #### Secrets for the CLI workflow

    | Secret | Value | Required |
    | - | - | - |
    | `CHECKSUM_API_KEY` | Checksum API key (**Settings → Project Settings**) | Yes |
    | `USERNAME` | Test user login | Used by the workflow file |
    | `PASSWORD` | Test user password | Used by the workflow file |
    | `BASE_URL` | Your application URL | Used by the workflow file |
    | `LOGIN_URL` | Your login page URL | Used by the workflow file |
    | `TEST_REPO_PAT` | Personal Access Token with repo permissions | Cross-repo only |

    `CI: true` sets defaults `hostReports: true` and `autoHealPRs: true`. Explicit env vars take precedence over the downloaded `.env`.
  </Accordion>
</div>

## Run the CLI in GitLab CI/CD

Add this job to your `.gitlab-ci.yml`:

```yaml .gitlab-ci.yml theme={null}
image: node:20-bookworm

stages:
  - test

run-checksum-tests:
  stage: test
  rules:
    # - if: '$CI_PIPELINE_SOURCE == "schedule"'  # Runs for scheduled pipelines
    - if: '$CI_PIPELINE_SOURCE == "web"'          # Allows manual triggering
      when: manual

  before_script:
    - npm ci
    - npx playwright install --with-deps
    - npx checksumai dotenv --download --api-key="${CHECKSUM_API_KEY}"

  script:
    - npx checksumai test

  variables:
    CHECKSUM_API_KEY: $CHECKSUM_API_KEY
    USERNAME: $USERNAME
    PASSWORD: $PASSWORD
    LOGIN_URL: $LOGIN_URL
    BASE_URL: $BASE_URL
    CI: "true"

  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
```

Add the same variables under your GitLab project's **Settings → CI/CD → Variables**. In merge-request pipelines, `--cksm-affected` picks up the target branch from `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` automatically.

### If your tests live in another project

Clone the tests project with an access token stored as `TEST_REPO_PAT`:

```yaml theme={null}
  before_script:
    - git clone "https://gitlab-ci-token:${TEST_REPO_PAT}@gitlab.com/org/your-test-repo.git" tests
    - cd tests
    - npm ci
    - npx playwright install --with-deps
    - npx checksumai dotenv --download --api-key="${CHECKSUM_API_KEY}"
```

<div className="ai-ref">
  <Accordion title="Reference for AI: CLI in GitLab CI/CD" icon="robot">
    Image `node:20-bookworm`. `before_script`: `npm ci` → `npx playwright install --with-deps` → `npx checksumai dotenv --download --api-key="${CHECKSUM_API_KEY}"`. `script`: `npx checksumai test`. Set `CI: "true"`. In merge-request pipelines, `--cksm-affected` reads `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`.

    #### GitLab CI/CD variables

    | Variable | Value | Required |
    | - | - | - |
    | `CHECKSUM_API_KEY` | Checksum API key (**Settings → Project Settings**) | Yes |
    | `USERNAME` | Test user login | Used by the workflow file |
    | `PASSWORD` | Test user password | Used by the workflow file |
    | `BASE_URL` | Your application URL | Used by the workflow file |
    | `LOGIN_URL` | Your login page URL | Used by the workflow file |
    | `TEST_REPO_PAT` | Access token with read access to the tests project | If the tests are in another project |

    Location: GitLab project → **Settings → CI/CD → Variables**.
  </Accordion>
</div>

## Trigger and gate a run from any CI system

Any CI system that can run `curl` can use Checksum. Start a run, check its status until it finishes, and pass the build only when the `verdict` is `"pass"`. On GitHub, the [GitHub Action](#run-checksum-with-the-github-action) does exactly this for you and is usually simpler.

1. **Start the run.** For PR checks, use `POST https://api.checksum.ai/public-api/v2/execution/grep`, which accepts the PR `branch`, a preview URL in `envOverrides`, and a `shardCount` in one request. Save the `runId` it returns. The other ways to start a run are in [Execution endpoints](/docs/running-tests#start-a-cloud-run-from-the-rest-api).
2. **Check the status** with `GET https://api.checksum.ai/public-api/v1/execution/status/run/{runId}` until `isTerminal` is `true` ([Run status](/docs/running-tests#check-whether-a-run-passed)).
3. **Gate on `verdict`,** not on the passed/failed counts. The verdict is only computed after sharded results merge, and it treats an empty selection as a failure.
4. **Heal, if you like,** by including an `autoHeal` block when you start the run, or by calling `POST https://api.checksum.ai/public-api/v1/auto-heal` afterward ([Auto-Healing](/docs/auto-healing#choose-how-to-start-healing)).

Here is a complete, sharded PR check. It's written in GitHub Actions syntax; adapt the `${{ … }}` expressions to your CI system's variables. It needs a `CHECKSUM_API_KEY` secret, `jq` on the runner (preinstalled on `ubuntu-latest`), and a per-PR preview URL.

### Example: a sharded PR check with curl

```yaml .github/workflows/checksum-sharded.yml theme={null}
name: Checksum sharded PR tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  checksum:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - name: Trigger sharded Checksum run
        id: trigger
        run: |
          RESPONSE=$(curl -sf -X POST https://api.checksum.ai/public-api/v2/execution/grep \
            -H "Authorization: Bearer ${{ secrets.CHECKSUM_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d "$(jq -n \
              --arg grep "@smoke" \
              --arg branch "${{ github.head_ref }}" \
              --arg baseUrl "https://pr-${{ github.event.pull_request.number }}.preview.example.com" \
              '{
                grep: $grep,
                branch: $branch,
                envOverrides: { BASE_URL: $baseUrl },
                shardCount: 8
              }')")
          echo "run_id=$(echo "$RESPONSE" | jq -r .runId)" >> "$GITHUB_OUTPUT"

      - name: Poll Checksum verdict
        run: |
          RUN_ID="${{ steps.trigger.outputs.run_id }}"
          while true; do
            STATUS=$(curl -sf "https://api.checksum.ai/public-api/v1/execution/status/run/$RUN_ID" \
              -H "Authorization: Bearer ${{ secrets.CHECKSUM_API_KEY }}")
            echo "$STATUS" | jq '{status, phase, isTerminal, verdict, passed, failed, recovered, bug}'

            if [[ "$(echo "$STATUS" | jq -r .isTerminal)" == "true" ]]; then
              break
            fi

            sleep 15
          done

          [[ "$(echo "$STATUS" | jq -r .verdict)" == "pass" ]]
```

<Warning>
  **API runs can't be cancelled**

  Cancelling your CI job (for example with `concurrency: cancel-in-progress`) only stops your polling. The Checksum run continues until it finishes. Plan your concurrency settings with this in mind.
</Warning>

<Accordion title="Example: full suite with auto-heal, gated on the verdict">
  Starts a full-suite run with **auto-heal on failure**, polls until it finishes, and exits non-zero unless the verdict is `pass`. When the run fails, Checksum starts healing automatically, so no separate heal call is needed. Needs `curl`, `jq`, and `CHECKSUM_API_KEY`.

  ```bash theme={null}
  #!/usr/bin/env bash
  set -euo pipefail

  BASE="https://api.checksum.ai/public-api/v1"
  TESTS_REPO="acme-co/my-tests"
  TESTS_BRANCH="main"

  # 1. Start a full suite run and opt into heal-on-failure
  RUN_ID=$(curl -sf -X POST "$BASE/execution/suite" \
    -H "Authorization: Bearer $CHECKSUM_API_KEY" \
    -H "Content-Type: application/json" \
    -d "$(jq -n --arg repo "$TESTS_REPO" --arg branch "$TESTS_BRANCH" \
          '{autoHeal: {autoCreatePR: true, repoName: $repo, branch: $branch}}')" \
    | jq -r '.runId')
  echo "Started run: $RUN_ID"

  # 2. Poll until the run is terminal
  while true; do
    RESULT=$(curl -sf "$BASE/execution/status/run/$RUN_ID" \
      -H "Authorization: Bearer $CHECKSUM_API_KEY")
    echo "Status: $(echo "$RESULT" | jq -r '.status')"
    [[ "$(echo "$RESULT" | jq -r '.isTerminal')" == "true" ]] && break
    sleep 15
  done

  # 3. Gate on the verdict, not the counts
  VERDICT=$(echo "$RESULT" | jq -r '.verdict')
  if [[ "$VERDICT" == "pass" ]]; then
    echo "All tests passed."
  else
    echo "Run did not pass (verdict: $VERDICT). Auto-heal was dispatched with the run."
    exit 1
  fi
  ```

  To heal a run that **already finished** without `autoHeal`, call `POST /auto-heal` with `testRunId` set to the run's UUID (the `runId` above), not the dispatch `name` (see the next example).
</Accordion>

<Accordion title="Example: other calls you can chain in a pipeline">
  ```bash theme={null}
  # Full suite, sharded 4 ways
  curl -X POST https://api.checksum.ai/public-api/v1/execution/suite \
    -H "Authorization: Bearer $CHECKSUM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"shardCount": 4}'
  # → { "runId": "9f2c7a4e-8b31-4d6a-a2f0-3c5e1b7d9a42", "name": null, "sharded": true }

  # Per-test results once terminal
  curl https://api.checksum.ai/public-api/v1/test-runs/$RUN_ID/results \
    -H "Authorization: Bearer $CHECKSUM_API_KEY"

  # Heal a run that finished without autoHeal
  curl -X POST https://api.checksum.ai/public-api/v1/auto-heal \
    -H "Authorization: Bearer $CHECKSUM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"testRunId": "'"$RUN_ID"'", "autoCreatePR": true, "branch": "main", "prNumber": 42, "repoName": "<owner>/<repo>"}'
  # → { "batchId": "...", "sessionIds": [...], "failureCount": 3, "testIds": [...] }

  # Poll healing
  curl https://api.checksum.ai/public-api/v1/auto-heal/batch/$BATCH_ID \
    -H "Authorization: Bearer $CHECKSUM_API_KEY"
  ```

  The per-test results are covered in [Results API](/docs/results-and-reports#get-results-with-the-rest-api), and healing in [Auto-Healing](/docs/auto-healing#choose-how-to-start-healing).
</Accordion>

<div className="ai-ref">
  <Accordion title="Reference for AI: REST API in any CI system" icon="robot">
    All calls send `Authorization: Bearer $CHECKSUM_API_KEY` and, with a body, `Content-Type: application/json`.

    | Step | Call | What to do with the result |
    | - | - | - |
    | 1. Trigger | `POST https://api.checksum.ai/public-api/v2/execution/grep` (or another execution endpoint) | For PR checks, use v2 grep: it accepts `branch`, `envOverrides`, and `shardCount` in one request. Save `runId`. |
    | 2. Poll | `GET https://api.checksum.ai/public-api/v1/execution/status/run/{runId}` | Repeat (e.g. every 15 s) until `isTerminal` is `true`. |
    | 3. Gate | Read `verdict` | Pass only on `verdict: "pass"`. Not the `passed`/`failed` counts: the verdict is computed only after sharded results merge, and an empty selection is a failure. |
    | 4. Heal (optional) | `autoHeal` block in step 1, or `POST https://api.checksum.ai/public-api/v1/auto-heal` afterward | See [Auto-Healing](/docs/auto-healing#choose-how-to-start-healing). |

    * `envOverrides`: application variables only. `CI` and keys starting with `CHECKSUM_` are rejected with `400`.
    * v1 execution endpoints can shard, but they run against the project's configured branch and environment. For a PR branch with a preview URL, use v2 grep.
    * `POST /v1/execution/suite` with `{"shardCount": 4}` returns `{ "runId": "…", "name": null, "sharded": true }`.
    * `POST /v1/auto-heal` returns `{ "batchId", "sessionIds", "failureCount", "testIds" }`. Poll `GET /v1/auto-heal/batch/{batchId}`.
    * API-triggered runs can't be cancelled through the public API. Cancelling the CI job only stops polling.
  </Accordion>
</div>

## Auto-heal tests that fail in CI

Checksum can fix the tests that failed in a CI run and open a PR with the fixes. You can opt in for a single run, or ask Checksum to turn it on for the whole project.

**For one run,** add `auto-heal: true` to the GitHub Action, add `--cksm-auto-heal` to the CLI test command, or include an `autoHeal` block when you start a run through the API. With the CLI in GitHub Actions or GitLab, the repository, branch, and PR number are detected from the CI environment:

```yaml theme={null}
- name: Run Checksum tests
  run: npx checksumai test --cksm-auto-heal
  env:
    CHECKSUM_API_KEY: ${{ secrets.CHECKSUM_API_KEY }}
    BASE_URL: ${{ secrets.BASE_URL }}
    CI: true
```

The full `--cksm-auto-heal*` flag set is in [Running Tests → `test` flags](/docs/running-tests#useful-flags).

### Heal every failing CI run (project-wide)

<Info>
  **Enabled by Checksum**

  Checksum can turn on automatic healing for your whole project, so every failing CI run is healed without per-pipeline flags. Ask your Checksum team to enable it.
</Info>

Once it's enabled, set this variable in each pipeline whose runs should be treated as CI runs:

```yaml theme={null}
env:
  CHECKSUM_RUNTIME_REQUEST_REASON: cicd
```

A per-run opt-in (CLI flag, action input, or API `autoHeal`) takes priority over the project-wide setting for that run. See [Auto-Healing](/docs/auto-healing) for how healing works.

<div className="ai-ref">
  <Accordion title="Reference for AI: auto-heal in CI" icon="robot">
    #### Per-run opt-in

    | Interface | How to opt in |
    | - | - |
    | GitHub Action | `auto-heal: true` (optionally `auto-create-pr: false`) |
    | CLI (GitHub Actions or GitLab) | `npx checksumai test --cksm-auto-heal`. Repo, branch, and PR number are auto-detected from the CI environment. |
    | REST API | `autoHeal` block in the execution request body ([autoHeal block](/docs/auto-healing#heal-automatically-if-a-run-fails)) |

    #### Project-wide

    Enabled by Checksum on request. Then set `CHECKSUM_RUNTIME_REQUEST_REASON: cicd` in each pipeline whose runs are CI runs. Precedence: a per-run opt-in overrides the project-wide setting for that run.
  </Accordion>
</div>

<div className="part bg"><span className="part-icon">i</span><div><div className="part-title">Planning your pipeline</div><div className="part-sub">When to run, and how to fix common CI problems</div></div></div>

## When to run your tests

| Strategy | When to use | GitHub | GitLab |
| - | - | - | - |
| **Manual trigger** | Testing and debugging the pipeline | `workflow_dispatch` | `when: manual` |
| **On schedule** | Nightly or regular health checks | `schedule: cron` | `$CI_PIPELINE_SOURCE == "schedule"` |
| **On PR** | Gate changes before merge (pair with `--cksm-affected` or `affected`) | `pull_request` | `merge_request_event` |
| **On merge to main** | Verify after deployments | `push: branches: [main]` | `$CI_PIPELINE_SOURCE == "push"` |

<Tip>
  Start with a manual trigger to verify the setup, then add scheduled or PR/merge-triggered runs. For schedules Checksum runs for you, without your own CI, see [Running Tests → Scheduled runs](/docs/running-tests#scheduled-runs).
</Tip>

## Troubleshooting

<AccordionGroup>
  <Accordion title="The action step fails immediately with an empty selection">
    As of `@v2`, a `grep` (or affected set) that matches no tests fails the step. Check your pattern with `npx checksumai test --cksm-affected-dry-run` or run the grep locally.
  </Accordion>

  <Accordion title="A sharded run never reaches a verdict">
    The tests branch has a `checksumai` version older than 4.4.0, so shard reports can't merge. Upgrade, commit, and re-run. Always bound `wait: true` with `wait-timeout-seconds`.
  </Accordion>

  <Accordion title="Heal PR opened against the wrong branch">
    When the pipeline runs from application code, point the run at your tests repo's integration branch (often `main`). With the GitHub Action, set the `branch` input: healing defaults to the run's branch. With the REST API, set `autoHeal.branch`; with the CLI, `--cksm-auto-heal-branch`. See [Where healing runs](/docs/auto-healing#choose-where-the-fix-pr-lands).
  </Accordion>

  <Accordion title="Cancelling the workflow didn't stop the run">
    Expected behavior. API-triggered runs can't currently be cancelled through the public API.
  </Accordion>

  <Accordion title="Reports don't show up in the dashboard">
    Report upload defaults to on only when `CI=true`. Set `CI: true` in the job, or set `options.hostReports: true` in [checksum.config.ts](/docs/test-repository#configure-checksum-config-ts).
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="Running Tests" icon="play" href="/docs/running-tests">
    Execution and status endpoints, test selection, and verdicts.
  </Card>

  <Card title="Sharding" icon="layer-group" href="/docs/sharding">
    Parallelize large suites safely.
  </Card>

  <Card title="Auto-Healing" icon="wand-magic-sparkles" href="/docs/auto-healing">
    What happens after a failing run.
  </Card>

  <Card title="The Checksum CLI" icon="code" href="/docs/running-tests#run-tests-with-the-cli">
    Every `checksumai` command and flag.
  </Card>

  <Card title="CI Setup Prompts" icon="robot" href="/docs/ci-setup-prompts">
    Have a coding agent build these workflows.
  </Card>
</CardGroup>