Sharding

​

At a glance

​

Update checksumai first

npm install checksumai@latest
git commit -am "Update checksumai for sharded runs" && git push

​

Shard a run from the GitHub Action

- uses: checksum-ai/test-run-action@v2
  with:
    api-key: ${{ secrets.CHECKSUM_API_KEY }}
    grep: '@smoke'
    shard-count: 8
    wait: true
    wait-timeout-seconds: 1800

​

Shard a run from the REST API

​

Shard the full suite

curl -X POST https://api.checksum.ai/public-api/v1/execution/suite \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"shardCount": 4}'
# → { "runId": "9f2c7a4e-8b31-4d6a-a2f0-3c5e1b7d9a42", "name": null, "sharded": true }

​

Shard a PR branch against its preview URL

curl -X POST https://api.checksum.ai/public-api/v2/execution/grep \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "grep": "@smoke",
    "branch": "feature/checkout-preview",
    "envOverrides": { "BASE_URL": "https://pr-42.preview.example.com" },
    "shardCount": 8
  }'

​

Follow a sharded run

curl https://api.checksum.ai/public-api/v1/execution/status/run/$RUN_ID \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"

{
  "status": "running", "isTerminal": false, "verdict": "pending",
  "phase": "sharding", "executedCount": 0,
  "sharded": true, "shardTotal": 8,
  "passed": 0, "failed": 0, "recovered": 0, "bug": 0, "failureReason": null
}

​

How sharding works

• Parallelism is controlled only by the shard count. Each shard always runs one Playwright worker.
• Status reporting is unified. Poll GET https://api.checksum.ai/public-api/v1/execution/status/run/{runId}. Its phase moves through sharding → merging → complete, and verdict is computed only after the merge (see Run status). The legacy job-name status endpoint doesn’t support sharded runs.
• Auto-heal works with sharding. Once the shards merge, a merged run that ends failed is healed exactly like a non-sharded run. A lost shard doesn’t stop the surviving shards’ failures from being healed.

​

Is your suite ready to shard?

​

Every test owns its data

• When the setup isn’t what the test is verifying, create the data through your API rather than a UI form, with a unique name like <test-id>-<random-suffix>. Never use a fixed human-readable name another test might also create or match.
• Don’t assert on the state of a shared resource (“there are 3 items in the list”). Assert on your own uniquely named item.
• Avoid a single default project, workspace, or org that every test reads and writes. If several tests mutate the same record, one shard’s setup or cleanup can race another shard’s assertions.
• Clean up what you created in a best-effort teardown step. Log a cleanup failure rather than failing the test, so leftover data doesn’t go unnoticed.

​

Session and login state

• Measure before you assume. Check how many concurrent sessions your backend actually supports on one account. Some backends safely deduplicate concurrent logins into one valid session, while others invalidate the previous session on each new login. Open N sessions concurrently and confirm none get logged out before you pick a shard count. Add test users per role in Environments.
• Restore saved sessions early. If you restore cookies or local storage instead of logging in through the UI, apply them before the first navigation, and don’t rewrite cookie domains you don’t need to. Some SSO providers run a silent login check on page load that can overwrite a session restored too late or with a mangled cookie domain. This shows up as an intermittent “logged out” failure that only happens in parallel runs.

​

Backend and environment capacity

• Check for rate limits, connection-pool limits, or per-account concurrency limits on your backend and staging/test environment.
• If you only see timeouts in sharded runs, don’t raise the timeout first. Rule out a data or session race (two shards fighting over the same entity or account), because it fails the same way real saturation does. If the failure still reproduces with isolated data, it’s a capacity signal, and raising the timeout is the right fix.
• Retries add wall-clock time to every retried test. On a suite with a few very slow tests, they can cancel out a meaningful part of the speedup.

​

Avoid test.describe.serial as a workaround

​

Know your ceiling

• A shard’s runtime depends on the files assigned to it. If one file is much larger or slower than the rest, its shard sets your minimum wall-clock time no matter how many shards you add. Splitting an oversized file into smaller files gives sharding more to balance.
• The merge takes time. Once every shard finishes, Checksum combines the reports. Budget a short merge step on top of shard runtime.
• Compare runs fairly. A shard-count change is only comparable to an earlier run if the test count, environment, and retry settings also match.
• Verify in real CI. Auth and session handling can behave differently on a headless CI runner than locally. Confirm with an actual sharded run before you rely on a shard count.

​

Current limits

​

Roll out safely

​

Troubleshooting

​

Related