Auto-Healing - Checksum AI

At a glance

Start healing from	How	When it starts
CLI	Add `--cksm-auto-heal` to `npx checksumai test`	When that run fails
GitHub Action	Set `auto-heal: true`	When that run fails
Every CI run	Ask Checksum to enable it, then mark your pipelines	Whenever a CI run fails
REST API	Add `autoHeal` to a run, or call `POST /auto-heal` later	On failure, or on demand
Coding agent (MCP)	Ask it to heal your latest failed run	On demand
Web app	Feature Health Dashboard agent actions	On demand

Reference for AI: auto-healing at a glance

Interface	Entry point	When healing starts	Key constraint
CLI	`npx checksumai test --cksm-auto-heal`	When that run fails	Repo, branch, and PR are auto-detected in GitHub Actions and GitLab CI
GitHub Action	`auto-heal: true`	When that run fails	With `shard-count`, requires action v2.1.0+
Project-wide	`CHECKSUM_RUNTIME_REQUEST_REASON: cicd`	Every failing CI (or scheduled) run	Enabled by Checksum
REST API	`autoHeal` block on any execution request	When that run ends `failed`	Omit the block to opt out
REST API	`POST https://api.checksum.ai/public-api/v1/auto-heal`	After a run finished	`testRunId` must be the run UUID, not the job `name`. `repoName` required unless `autoCreatePR: false`.
REST API	`GET https://api.checksum.ai/public-api/v1/auto-heal/batch/{batchId}`	—	Poll until `allTerminal` is `true`
MCP	`checksum_test_heal`	After a run finished	The run must have failures
Web app	Feature Health Dashboard → agent actions	On demand, for a bug or test	—

Precedence: a per-run opt-in (CLI flag, action input, or API autoHeal) takes priority over project-wide auto-heal settings for that run.
Target branch: branch is one branch in the tests repository. Defaults to the run’s recorded branch. If the run came from app-code CI, set it to the tests repo’s integration branch (often main).
Output: a PR from a generated checksumai/<id> branch into branch, when autoCreatePR is true (the default).
Honest triage: real product bugs get a bug verdict and an @bug tag instead of being rewritten to go green.
Sharding: a merged sharded run that ends failed is healed. If shards never merge (checksumai older than 4.4.0), the heal is never evaluated.

Developer guide

Trigger healing from the CLI, CI, the REST API, or MCP, and control where the fix PR lands

Choose how to start healing

There are two moments to ask for healing. You can opt in when you start a run, so healing kicks off on its own if the run fails. Or you can heal a run that already finished. Checksum can also turn healing on for every CI run in your project. If a run has its own opt-in (a CLI flag, an action input, or an API autoHeal block), that takes priority over the project-wide setting for that run.

Opt in when the run starts	Heal a run that already finished
CLI flag, GitHub Action input, API `autoHeal` block, or project-wide for CI	`POST /auto-heal`, your coding agent, or the Health Dashboard

Heal failures from the CLI

Add --cksm-auto-heal to your test command. If the run fails, Checksum heals the failing tests and, by default, opens a PR with the fixes:

# Run tests and auto-heal failures in CI (repo/branch/PR auto-detected)
npx checksumai test --cksm-auto-heal

# Heal failures but don't open a PR
npx checksumai test --cksm-auto-heal --cksm-auto-heal-create-pr=false

In GitHub Actions and GitLab CI, the repository, branch, and PR number are detected automatically, so --cksm-auto-heal alone is usually all you need. Outside CI, you can set them yourself with the extra flags in the reference below.

Reference for AI: --cksm-auto-heal flags

Flag	Description
`--cksm-auto-heal`	After the run, heal any failing tests. By default this opens a PR with the fixes.
`--cksm-auto-heal-create-pr[=false]`	Whether healing opens a PR. Defaults to `true` with `--cksm-auto-heal`. `=false` heals without a PR.
`--cksm-auto-heal-repo-name=<owner>/<repo>`	Repo hosting the source PR for progress comments (usually app code). Auto-detected in CI.
`--cksm-auto-heal-branch=<branch>`	Branch in the tests repository the healed PR targets. Auto-detected in CI.
`--cksm-auto-heal-pr-number=<number>`	Existing PR to post healing progress on. Auto-detected for PR CI events.

Auto-detection of repo, branch, and PR number works in GitHub Actions and GitLab CI.

Heal failures from the GitHub Action

Set auto-heal: true on the Checksum action. The action works out the PR and repository from the workflow, so healing progress appears as a comment on the PR that triggered it:

- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: 'checkout'
    auto-heal: true
    # auto-create-pr: false   # heal without opening a PR

If you also shard the run with shard-count, you need action v2.1.0 or later. When the workflow runs from application code, set the action’s branch input to your tests repo’s integration branch so the heal PR lands there (see Choose where the fix PR lands). More in CI/CD Integration, or have a coding agent build the workflow with CI Setup Prompts → Prompt 1.

Heal every failing CI run

Enabled by ChecksumChecksum can turn on automatic healing for your project so every failing CI run is healed without per-pipeline flags. Ask your Checksum team to enable it. Scheduled runs can be healed the same way (a project-level setting).

Once it’s enabled, mark your CI pipelines so Checksum knows their runs are CI runs:

env:
  CHECKSUM_RUNTIME_REQUEST_REASON: cicd

The repository, branch, and PR number are picked up from the CI environment.

Heal from the REST API

From the API you can either ask for healing as part of starting a run, or heal a run afterward. Either way you get a healing batch to follow. Every request sends Authorization: Bearer $CHECKSUM_API_KEY.

Heal automatically if a run fails

Add an autoHeal block to the request that starts the run (any of the execution endpoints). If the run ends failed, Checksum starts healing on its own; there’s no second call. Leave the block out to opt out.

curl -X POST https://api.checksum.ai/public-api/v1/execution/suite \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"autoHeal": {"autoCreatePR": true, "branch": "main", "repoName": "acme-co/my-tests"}}'

The block can carry the same options as a heal request (whether to open a PR, which branch, which PR to comment on, and extra context), for example:

{
  "autoHeal": {
    "autoCreatePR": true,
    "branch": "main",
    "prNumber": 42,
    "repoName": "acme-co/webapp",
    "metadata": { "source": "nightly" }
  }
}

Then follow the run with GET /execution/status/run/{runId}. It also works with sharded runs: once the shards merge, a failed run is healed.

Heal a run that already finished

If a run finished without autoHeal, ask for healing afterward. Pass the run’s ID (the runId from when you started it, or the id of the latest run), not its job name:

curl -X POST https://api.checksum.ai/public-api/v1/auto-heal \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "testRunId": "'"$TEST_RUN_ID"'",
    "autoCreatePR": true,
    "branch": "main",
    "prNumber": 42,
    "repoName": "<owner>/<repo>",
    "metadata": { "context": "post-deploy healing" }
  }'

{
  "batchId": "batch-xyz-789",
  "sessionIds": ["sess-1", "sess-2", "sess-3"],
  "failureCount": 3,
  "testIds": ["test-abc-123", "test-def-456", "test-ghi-789"]
}

Checksum starts a healing session for the failing tests and returns a batchId. Because it opens a PR by default, include repoName unless you set "autoCreatePR": false.

Follow healing progress

Check the batch every so often until allTerminal is true. Each session’s prUrl links the fix PR once it opens.

curl https://api.checksum.ai/public-api/v1/auto-heal/batch/$BATCH_ID \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"

{
  "batchId": "batch-xyz-789",
  "status": "in_progress",
  "testRunId": "run-id",
  "totalSessions": 3,
  "completedCount": 1,
  "failedCount": 0,
  "allTerminal": false,
  "sessions": [
    { "sessionId": "sess-1", "checksumTestId": "test-abc-123", "status": "completed", "prUrl": "https://github.com/org/repo/pull/42" },
    { "sessionId": "sess-2", "checksumTestId": "test-def-456", "status": "running",   "prUrl": null },
    { "sessionId": "sess-3", "checksumTestId": "test-ghi-789", "status": "running",   "prUrl": null }
  ]
}

The batch moves from pending to in_progress and ends as completed or failed. Generation batches report progress in the same shape, without testRunId and sessions[].checksumTestId.

Example: a script that heals the latest failed run and waits for the fix PRs

Needs curl, jq, CHECKSUM_API_KEY, and REPO set to the <owner>/<repo> used for status comments. Set TESTS_BRANCH to your tests repo’s integration branch.

#!/usr/bin/env bash
set -euo pipefail
BASE="https://api.checksum.ai/public-api/v1"
AUTH="Authorization: Bearer $CHECKSUM_API_KEY"

# 1. Pick the run to heal (latest completed, non-manual run unless TEST_RUN_ID is set)
TEST_RUN_ID="${TEST_RUN_ID:-$(curl -sf "$BASE/test-runs/latest" -H "$AUTH" | jq -r .id)}"

# 2. Start healing
BATCH_ID=$(curl -sf -X POST "$BASE/auto-heal" -H "$AUTH" -H "Content-Type: application/json" \
  -d "$(jq -n --arg run "$TEST_RUN_ID" --arg repo "$REPO" --arg br "${TESTS_BRANCH:-main}" \
        '{testRunId: $run, autoCreatePR: true, branch: $br, repoName: $repo}')" | jq -r .batchId)
echo "Healing batch: $BATCH_ID"

# 3. Poll until every session is finished
until [[ "$(curl -sf "$BASE/auto-heal/batch/$BATCH_ID" -H "$AUTH" \
  | tee /tmp/heal.json | jq -r .allTerminal)" == "true" ]]; do
  sleep 30
done

# 4. Print each healed test and its PR
jq -r '.sessions[] | "\(.checksumTestId)  \(.status)  \(.prUrl // "no PR")"' /tmp/heal.json

Reference for AI: healing REST API

Headers on every request: Authorization: Bearer $CHECKSUM_API_KEY; with a body, Content-Type: application/json.

autoHeal block (execution request bodies)

Accepted on POST https://api.checksum.ai/public-api/v1/execution/suite, POST https://api.checksum.ai/public-api/v1/execution/collection/{id}, POST https://api.checksum.ai/public-api/v1/execution/tests, POST https://api.checksum.ai/public-api/v2/execution/grep. Presence means heal-on-failure: if the run ends failed, a healing batch starts with no separate call. Omit to opt out.

Field	Type	Required	Description
`autoHeal.autoCreatePR`	boolean	No	Open a PR in the tests repo when healing finishes. Default `true`.
`autoHeal.branch`	string	No	Tests-repo branch where the heal PR lands
`autoHeal.prNumber`	number	No	Existing PR to post healing progress on. Pair with `repoName`.
`autoHeal.repoName`	string	Conditional	`<owner>/<repo>` that hosts `prNumber`. Required when `autoCreatePR` is `true` (the default) or `prNumber` is set.
`autoHeal.metadata`	object	No	Key/value context added to the agent’s prompt

On grep runs, top-level branch = what is checked out for the test run; autoHeal.branch = where the heal PR lands. Works with shardCount: the merged run is healed; if shards never merge (outdated checksumai), the heal is never evaluated. Next: poll GET /execution/status/run/{runId}.

`POST https://api.checksum.ai/public-api/v1/auto-heal`

Starts healing for the failing tests in a finished run.

Body field	Type	Required	Description
`testRunId`	string	Yes	The run to heal: its UUID (the `runId` from an execution call, the `id` from `GET /test-runs/latest`, or `testRunId` from v2 job-name status). Not the dispatch `name`.
`autoCreatePR`	boolean	No	Open a PR in the tests repo when healing finishes. Default `true`.
`branch`	string	No	Tests-repo branch that healing clones, commits to, and targets the PR against. Defaults to the run’s recorded branch.
`prNumber`	number	No	Existing PR to post healing progress on. Pair with `repoName`.
`repoName`	string	Conditional	`<owner>/<repo>` that hosts `prNumber`. Required when `autoCreatePR` is `true` (the default) or `prNumber` is set. Omit only with `"autoCreatePR": false` and no `prNumber`.
`metadata`	object	No	Key/value context added to the agent’s prompt

Response field	Type	Description
`batchId`	string	Pass to `GET /auto-heal/batch/{batchId}`
`sessionIds`	string[]	The healing agent sessions that were started
`failureCount`	number	Number of failing tests being healed
`testIds`	string[]	Checksum test IDs of the failing tests

Error	Cause	Fix
`400`	`repoName` missing while `autoCreatePR` is `true` (the default) or `prNumber` is set	Pass `repoName`, or set `"autoCreatePR": false` without `prNumber`

Next: poll the batch.

`GET https://api.checksum.ai/public-api/v1/auto-heal/batch/{batchId}`

Returns the progress of a healing batch. Generation batches (GET /auto-generate/batch/{batchId}) return the same shape, minus testRunId and sessions[].checksumTestId. Path parameter batchId (string, required): the batch ID returned by the trigger. Poll until allTerminal is true.

Response field	Type	Description
`batchId`	string	The batch being reported
`status`	enum	`pending` (healing hasn’t started), `in_progress` (one or more sessions running), `completed` (all sessions finished successfully), `failed` (one or more sessions failed)
`testRunId`	string	The run being healed
`totalSessions`	number	Sessions in the batch
`completedCount`	number	Sessions that finished successfully
`failedCount`	number	Sessions that failed
`allTerminal`	boolean	`true` once every session has finished. Stop polling.
`sessions[].sessionId`	string	Healing session ID
`sessions[].checksumTestId`	string	The test that session is healing
`sessions[].status`	string	That session’s state, e.g. `running`, `completed`
`sessions[].prUrl`	string \| null	URL of the fix PR once opened, otherwise `null`

Choose where the fix PR lands

Healing always works in your tests repository. The branch you pass is the one branch healing clones, commits to, and opens its PR against; the PR comes from a generated checksumai/<id> branch. If you don’t pass one, Checksum uses the branch the run was recorded on. That only works when the run executed against a tests-repo branch. If the run came from your application’s CI (on a branch like feature/foo), set branch to your tests repo’s integration branch, often main. These settings behave the same on POST /auto-heal, the autoHeal block, and the CLI flags.

Choose where progress is posted

Separately, prNumber and repoName tell Checksum which pull request should receive progress comments while healing runs. They don’t change what’s cloned or where the fix PR lands. Usually they point at the PR in your application repo that triggered the CI run, but they can point at the tests repo if the source PR is there.

Reference for AI: healing targets

Where healing runs (tests repository)

Parameter	Type	Required	Description
`testRunId`	string	Yes (heal API)	The test run to heal
`autoCreatePR`	boolean	No	Open a PR in the tests repo when healing finishes. Default `true`.
`branch`	string	No	One branch in the tests repository. Healing clones it, commits fixes there, and opens the healed PR against it. Defaults to the run’s recorded branch, which only works if the run executed against a tests-repo branch. If the run came from app-code CI (e.g. `feature/foo`), set it to the tests repo’s integration branch (often `main`). There is no per-repository branch field.

With autoCreatePR: true, the PR opens from a generated checksumai/<id> branch into branch. Same semantics on POST /auto-heal, the autoHeal block, and --cksm-auto-heal-branch. On grep runs, a top-level branch controls the run’s checkout, not the heal PR target.

Where to post status (correlation only)

Parameter	Type	Required	Description
`prNumber`	number	No	Existing PR to associate this batch with. Progress comments post there. Doesn’t change the clone target or the healed-PR target. Pair with `repoName`.
`repoName`	string	Conditional	`<owner>/<repo>` containing `prNumber`. Usually the app-code repo when CI ran from application code, or the tests repo if the source PR is there. Matched against the project’s connected Git integrations; not a second branch selector. Required when `autoCreatePR` is `true` or `prNumber` is set.
`metadata`	object	No	Key/value context included in the agent’s prompt

Ask your coding agent to heal

With the Checksum MCP server connected, just ask:

My latest Checksum run has failures — heal them and open a PR.

Your agent finds the failed run if you don’t name one, then starts one healing session covering every failing test in it and, by default, opens a PR with the fixes. Healing works from a run that already finished, so there’s nothing to push first. If the run has nothing to heal, it tells you why. You can ask for progress or send extra instructions while it works, and ask for Deep mode if you want to approve a plan first.

Reference for AI: checksum_test_heal

Item	Value
Tool	`checksum_test_heal` (changes things: starts a billable cloud run and can open a PR)
Input	`testRunId` of a run that has failures. Find one with `checksum_test_run_list`.
Behavior	Opens one heal session covering every failing test in the run and, by default, a PR with the fixes. Works from a finished run: nothing to push first.
Optional input	`deepMode: true` for Deep mode (pauses for plan approval via `checksum_session_approve`)
Nothing to heal	Returns `No tests to heal in test run: <id> — <reason>`
Follow-up tools	`checksum_session_status` (track), `checksum_session_prompt` (steer), `checksum_session_create_pr` (open a PR if the session finished without one)
Heal on run completion	`checksum_test_run` with `autoHeal: true` starts healing sessions when the run finishes (works with `shardCount`). Those sessions don’t open a PR on their own: call `checksum_session_create_pr`.

▦

In the Checksum web app

Start healing from the dashboard and follow it to the PR

Heal from the Feature Health Dashboard

From the Feature Health Dashboard

From a bug row or an expanded bug, start healing (or another agent workflow) on the grouped set of failures. From any test row, start an agent session for that single test when your project has agent workflows enabled. See Agent actions. Each healing session appears under Agent Sessions. Tests under healing show Under Healing on the dashboard.

Test states while healing

While healing runs, each test on the dashboard is in one of three states:

State	Meaning
Bug	Confirmed application issue. It won’t be healed.
Clear	The test is healthy
Under Healing	A healing session is working on a fix

How it works

Triage, healing modes, what gets fixed, and reviewing the PR

How healing works

Auto-healing flow: failing run, triage, fix, verify, PR

Failing runTests failed and couldn’t be recovered at runtime

→

TriageEach failure classified as a test issue or an app bug

→

FixTest issues repaired and re-run to verify

→

PRHealed tests delivered to your tests repo

Triage: bug or healable?

The agent’s first step is to triage every failure in the run. For each failing test it:

Reads the test results and error context (screenshots, error messages, stack traces)
Reads the relevant test code and application code
Classifies the failure:
- Test issue: the app is fine but the test needs fixing (selector drift, timing, stale setup, assertion drift). These go on to the fix stage.
- Application bug: a real defect in your product. The agent submits a bug verdict and tags the test @bug in source. It’s tracked in the Feature Health Dashboard for your team.

If triage finds nothing test-side to fix (every failure is an app bug, or nothing failed), the fix stage is skipped.

Healing is honestIf a test fails because your app genuinely broke, healing reports a bug. It doesn’t rewrite the test to go green.

Deep vs Standard healing

Triage → Fix

Triage classifies each failure and submits verdicts for real bugs. Fix repairs test-local issues (selectors, timing, setup, assertions), runs the tests to verify, and commits. Best for selector changes, timing fixes, and minor assertion updates.

Plan first, then repair

Interview/Plan → Knowledge Base Update → Implementation → Review → Checksumify → Verify. The agent plans the fixes and learns what changed in your app before repairing. Best for major refactors, flow changes, and many related failures.

What the agent fixes

Issue	How it’s healed
Selector drift	Updates `data-testid`, role, or text selectors to match the current DOM
Timing / waits	Replaces arbitrary waits with Playwright web-first assertions
Assertion mismatches	Updates expected values to match current app behavior
Setup / cleanup changes	Adjusts data setup and cleanup when API endpoints or data models change
Flow changes	Adds, removes, or reorders steps to match the current user flow
Page layout changes	Adapts to restructured pages, moved elements, or new UI components
Authentication flow changes	Handles updated login, SSO, or multi-factor auth steps
API endpoint changes	Updates API calls in data setup/cleanup when backend endpoints change

Review healed tests

Read the diff

See what changed and why. The PR description explains each fix.

Run locally (optional)

Check out the checksumai/<id> branch and run the affected tests.

Merge

Once merged, the tests are green again, and the dashboard shows them as Clear.

Healing feedback

After reviewing healed tests, you can give feedback on the quality of the healing. Checksum uses it to improve healing accuracy over time.

Notifications

Healing batches emit two events you can route to Slack, Teams, Discord, or Google Chat with notification connectors:

Event	Fires when
Auto-Heal Started	A healing batch starts after a failing run
Auto-Heal Completed	The batch finishes (every session reached a terminal state, or the batch failed)

RoutingNew connectors don’t route any events until you turn them on. Enable Auto-Heal Started and Auto-Heal Completed for each connector in the routing matrix. See Notifications & Slack.

Troubleshooting

The heal PR targets the wrong branch

The run came from app-code CI, so its recorded branch (e.g. feature/foo) doesn’t exist in the tests repo. Set branch / autoHeal.branch / --cksm-auto-heal-branch to your tests repo’s integration branch.

400: repoName required

autoCreatePR defaults to true, which requires repoName. Pass it, or set "autoCreatePR": false without prNumber.

I passed the job name and healing didn't start

testRunId must be the run UUID (runId), not the dispatch name. For v2 grep runs polled by job name, use the testRunId field from GET /public-api/v2/execution/status/:jobName. See Which ID is which.

Healing ran but no fix PR appeared

Triage may have classified every failure as an application bug. Check the Feature Health Dashboard. Also check whether autoCreatePR was false.

Sharded run failed but healing never started

The shards never merged, usually because checksumai is older than 4.4.0, so autoHeal was never evaluated. See Sharding.

Auto-Recovery

Runtime fixes during execution.

Feature Health Dashboard

Bugs found by triage.

CI/CD Integration

Heal failing CI runs automatically.

​At a glance

​Choose how to start healing

​Heal failures from the CLI

​Heal failures from the GitHub Action

​Heal every failing CI run

​Heal from the REST API

​Heal automatically if a run fails

​Heal a run that already finished

​Follow healing progress

​autoHeal block (execution request bodies)

​POST https://api.checksum.ai/public-api/v1/auto-heal

​GET https://api.checksum.ai/public-api/v1/auto-heal/batch/{batchId}

​Choose where the fix PR lands

​Choose where progress is posted

​Where healing runs (tests repository)

​Where to post status (correlation only)

​Ask your coding agent to heal

​Heal from the Feature Health Dashboard

​From the Feature Health Dashboard

​Test states while healing

​How healing works

​Triage: bug or healable?

​Deep vs Standard healing

Triage → Fix

Plan first, then repair

​What the agent fixes

​Review healed tests

​Healing feedback

​Notifications

​Troubleshooting

​Related

Auto-Recovery

Feature Health Dashboard

CI/CD Integration

At a glance

Choose how to start healing

Heal failures from the CLI

Heal failures from the GitHub Action

Heal every failing CI run

Heal from the REST API

Heal automatically if a run fails

Heal a run that already finished

Follow healing progress

autoHeal block (execution request bodies)

`POST https://api.checksum.ai/public-api/v1/auto-heal`

`GET https://api.checksum.ai/public-api/v1/auto-heal/batch/{batchId}`

Choose where the fix PR lands

Choose where progress is posted

Where healing runs (tests repository)

Where to post status (correlation only)

Ask your coding agent to heal

Heal from the Feature Health Dashboard

From the Feature Health Dashboard

Test states while healing

How healing works

Triage: bug or healable?

Deep vs Standard healing

What the agent fixes

Review healed tests

Healing feedback

Notifications

Troubleshooting

Related