> ## Documentation Index > Fetch the complete documentation index at: https://checksum.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # CI Setup Prompts for AI Agents > Fill in a prompt, paste it into your coding agent (Claude Code, Cursor, Copilot, or any other), and it builds the Checksum GitHub Actions workflow for your repository. One prompt sets up a sharded test run on every pull request with auto-healing; the other starts test generation when a pull request opens. Each prompt carries the requirements agents most often get wrong, and tells the agent to ask you about your setup before it writes anything. ## At a glance | Prompt | What the agent builds | Preview URL it can use | Before you start | | - | - | - | - | | [Shared context](#shared-context) | Nothing on its own. Paste it before either prompt. | You choose A, B or C here | `CHECKSUM_API_KEY` saved as a GitHub Actions secret | | [Prompt 1](#prompt-1-sharded-test-run-on-pr-with-auto-healing) | A sharded test run on every PR, gated on the verdict, with [auto-healing](/docs/auto-healing) | A, B, or C (C only while the job waits) | `checksumai` 4.4.0 or later on the tests branch | | [Prompt 2](#prompt-2-test-generation-when-a-pr-opens) | [Test generation](/docs/generate-tests) when a PR opens | A or B only: it must outlive the session | The app repo connected in [Git Integration](/docs/git-integration) |
| Prompt | Output | Entry point | Key constraint | | - | - | - | - | | Shared context | Setup values and rules for either prompt | — | Step 0: the agent asks one round of questions before writing any file | | Prompt 1 | `.github/workflows/checksum-tests.yml` | `uses: checksum-ai/test-run-action@v2` (REST `POST /public-api/v2/execution/grep` only for non-GitHub CI) | `checksumai` ≥ 4.4.0 on the tests branch; gate on `verdict`; `branch` = tests-repo branch | | Prompt 2 | `.github/workflows/checksum-generate.yml` | `POST https://api.checksum.ai/public-api/v1/auto-generate` | Persistent preview (A or B); `prNumber` + `repoName` + `branch` required | * Preview options: A) predictable URL pattern, B) URL from an existing deploy workflow, C) temporary preview built inside the job. C is not allowed for generation. * PR comments come from Checksum, not the workflow: the action passes the PR on `pull_request` events, and generation posts its own "Checksum Auto-Generate" comment. No comment steps, no `pull-requests: write`. * Permissions: `contents: read`, `pull-requests: read`. Skip fork PRs with an `if:` (no secrets). * Cancelling the workflow doesn't stop a Checksum run.
{""}
Developer guide
Pick a preview, fill in the prompts, and paste them into your coding agent
## How to use the prompts Replace every `{{PLACEHOLDER}}`. If you don't know a value, write `ASK ME` or leave the placeholder as it is. The agent always asks about your setup before it writes anything (see "Step 0" in the shared context). Keep the `[OPTIONAL: …]` blocks that apply and delete the rest. The **Fixed requirements** sections come from these docs and cover what agents most often get wrong. Paste the shared context first, then Prompt 1, Prompt 2, or both. ## Choose a preview URL Both workflows point Checksum at the PR's preview URL. How long that URL has to stay up depends on which workflow uses it: | Workflow | How long the preview must stay up | Temporary in-job preview (container + tunnel)? | | - | - | - | | **Prompt 1: test run + auto-heal** | For the whole test run (usually 5–25 min, longer when sharded). Auto-healing also runs in Checksum's cloud after the run fails, so if it's on, a persistent URL is the safer choice. | Only with `wait: true`, and only if auto-heal is off or you accept the risk described in Prompt 1 | | **Prompt 2: test generation** | **Until generation finishes. That can be several hours**, because an agent in Checksum's cloud writes, verifies and delivers the tests long after the workflow has triggered it. | **No.** The URL would be gone before the agent is done. | **Generation needs a persistent preview URL** For generation on PR open, the preview has to stay up for at least as long as a generation session. For example, Firebase Hosting preview channels last 7 days by default, and Vercel, Netlify, Cloudflare Pages and Render keep per-PR preview deployments until the PR closes. Temporary URLs, such as a tunnel the job opens or a preview that's deleted when the workflow ends, **won't work for generation**. ## Shared context Paste this before either prompt. It describes your repositories, secrets and preview, and tells the agent to ask before it builds. ```text theme={null} Context for this task: - App repo: {{OWNER/APP_REPO}} - Tests repo: {{OWNER/TESTS_REPO | "same repo, tests in {{TESTS_DIR}}"}} - Tests repo integration branch: {{TESTS_BRANCH, usually main}} - Checksum API key secret name: {{SECRET_NAME, default CHECKSUM_API_KEY}} (already set in GitHub → Settings → Secrets and variables → Actions) - [OPTIONAL: Tests in a separate repo → PAT secret name: {{TEST_REPO_PAT}}] PR preview URL (required, both workflows run against it). Pick one and delete the others: A) Predictable URL pattern: {{URL_PATTERN, e.g. https://pr-${PR_NUMBER}.preview.example.com}}. B) Preview deployed by an existing workflow/step: {{WORKFLOW_OR_STEP_NAME}}, which exposes the URL as {{OUTPUT_NAME | "a deployment status" | "a PR comment"}}. Provider: {{Vercel | Netlify | Cloudflare Pages | Firebase preview channel | Render | Amplify | other}}. C) Temporary preview built inside the job (e.g. container + tunnel). Build command: {{BUILD_CMD}}, serve command: {{SERVE_CMD}}, port: {{PORT}}. Hosting rules the server must copy: {{ROUTING_FILE_OR_TABLE}}. NOT allowed for test generation (Prompt 2). - How long the preview stays up: {{LIFETIME, e.g. "7 days (Firebase channel, expires: 7d)" | "until the PR closes" | "only while the job runs"}} - Does the same URL get redeployed on every push to the PR? {{yes | no | don't know}} - Env var the tests read the app URL from: {{BASE_URL_VAR, default BASE_URL}} Reference docs (read before writing anything): {{DOCS_PATH_OR_URL}} Pages: ci-integration, sharding, auto-healing, generate-tests. Step 0: ask before building (mandatory, always do this first): Before you write or change any file, send me ONE round of questions, using your question tool if you have one, and wait for my answers. Ask about: - every placeholder above or below that is still in {{…}} form, says ASK ME, or offers several options I haven't narrowed to one; - any dynamic setup that is ambiguous, contradicts itself, or doesn't match what you find in the repo (e.g. I named a deploy workflow that doesn't exist, or the preview lifetime is shorter than the workflow needs); - a confirmation of the preview setup: the option (A/B/C), the lifetime, and how the workflow gets the URL, even if I filled them in. Put a recommended answer on each question where you have one. Don't guess an answer and carry on; don't assume defaults for dynamic values without asking. If an answer opens a new question, ask it before you build. Rules: write files only under .github/. Don't commit. Don't invent a secret, URL or repo name. When you finish, list every file you created, every secret/variable it needs, and each assumption you made. ``` ## Prompt 1: Sharded test run on PR, with auto-healing Builds a workflow that runs your suite against the PR's preview as a [sharded](/docs/sharding) run, passes or fails the check on the run's `verdict`, and [heals](/docs/auto-healing) failures. It uses the [GitHub Action](/docs/ci-integration#run-checksum-with-the-github-action) by default. ```text theme={null} Create .github/workflows/{{FILE_NAME, e.g. checksum-tests.yml}} that runs the Checksum suite against the PR's preview URL as sharded runs when a PR is opened or updated, and auto-heals failures. Trigger: pull_request, types [{{opened, synchronize, reopened}}] [OPTIONAL: only for base branches {{BRANCHES}}] [OPTIONAL: only for paths {{PATHS}}] [OPTIONAL: skip draft PRs] [OPTIONAL: skip PRs from Checksum's own heal branches (checksumai/*)] Execution approach (pick one): - GitHub Action (default): checksum-ai/test-run-action@{{v2 | pinned SHA}} Its `branch` input picks the tests-repo branch (grep mode), and it passes the PR to Checksum on its own, so Checksum comments on the PR. Use it whenever the workflow runs on GitHub. - REST API with curl (only if the workflow will later move to another CI) Test selection (exactly one): {{grep: '' | affected}} Sharding works in grep and affected modes, but the preview URL override (env-overrides) is grep only. If I pick affected, ask me in Step 0 how the preview URL should reach the run. Shard count: {{2–40; start with 2–4 on a new suite}} Gate the PR check on the result: {{yes → wait | no → PR comment only}} Wait timeout: {{seconds, e.g. 1800}} Auto-heal opens a fix PR: {{yes | no → auto-create-pr: false}} Concurrency per PR: {{cancel-in-progress | queue | none}} Fixed requirements (from the docs): 1. The tests branch must have checksumai >= 4.4.0 committed. Otherwise shard reports never merge, no verdict comes back, and auto-heal never runs. Check the tests repo's package.json and tell me if it's older. Don't upgrade it yourself. 2. Sharding + auto-heal together needs action v2.1.0+. @v2 already resolves to that. If you pin a SHA, confirm it's >= v2.1.0. 3. With wait: true, always set wait-timeout-seconds AND a job-level timeout-minutes a bit longer than it. 4. Pass or fail on the run's verdict ("pass"), never on passed/failed counts. In REST mode, poll GET /public-api/v1/execution/status/run/{runId} until isTerminal is true, then check the verdict. 5. The preview URL goes in env-overrides (Action, grep mode only) or envOverrides (REST, v2 grep) as {{BASE_URL_VAR}}. Keys named CI or starting with CHECKSUM_ are rejected with a 400. 6. For REST, trigger with POST /public-api/v2/execution/grep, sending branch, envOverrides, shardCount and an autoHeal block in one request. Don't use the v1 endpoints: they ignore the PR branch and preview URL. autoHeal.branch = {{TESTS_BRANCH}} (the tests repo branch, NOT the app PR branch). autoHeal.repoName + prNumber = the app PR, so progress comments land on it. 7. Cancelling the workflow does not stop the Checksum run. Write a comment in the YAML next to the concurrency settings saying so. 8. Permissions: least privilege (contents: read, pull-requests: read). PR comments come from Checksum itself (the action's PR-comment pipeline and auto-heal progress). Don't add comment scripts, sticky- comment helpers or comment steps. Just make sure the PR reaches Checksum: the action detects it on pull_request events (set pr-number / repo-name only on other events); in REST mode, send autoHeal.repoName + prNumber. 9. PRs from forks don't get secrets. Skip them cleanly with an `if:` instead of failing. 10. Before handing the URL to Checksum, check that it returns 200 for both the root and one deep route: {{DEEP_ROUTE, e.g. /accounts/probe}}. Fail with a clear message if it doesn't. A preview that's still deploying, or has already expired, fails every test. Preview-specific requirements (keep the one matching A–C above): A) Build the URL from the pattern using the PR number or head ref. Wait until the preview for this commit is live before starting the run. B) Make the Checksum job depend on the deploy job and read the URL from {{OUTPUT_NAME}}. Fail with a clear message if the URL is empty. Don't guess or construct it. C) The preview only lives as long as the job, so wait MUST be true. Tear the preview down in an `if: always()` step. Auto-heal sessions run in Checksum's cloud after the run and verify their fixes by running tests. The docs don't say which URL they use, so a temporary preview may already be gone by then. Tell me about this risk in Step 0 and ask whether to keep auto-heal on with C or switch to a persistent preview. ``` ## Prompt 2: Test generation when a PR opens Has the agent pick between Checksum's built-in auto-generate, a workflow that calls [the generation API](/docs/generate-tests#start-generation-from-the-rest-api), or a `/checksum generate` comment, and build only the one that fits. The default is the API workflow, since it's the only one that can pass a per-PR preview URL. ```text theme={null} Set up Checksum test generation for new PRs in {{OWNER/APP_REPO}}, running against the PR's persistent preview URL. Preview URL rules for generation (check these in Step 0): - Generation is done by an agent in Checksum's cloud. The workflow only starts it, and the agent can keep working for several hours. It needs the preview URL for that whole time. - The preview must be persistent: option A or B, with a lifetime of at least {{MIN_LIFETIME, e.g. 24 hours}}. Examples: a Firebase Hosting preview channel (expires defaults to 7d), or Vercel / Netlify / Cloudflare Pages / Render per-PR previews that last until the PR closes. - Option C (a temporary preview built inside the job) is NOT allowed. If that's all I have, stop and tell me I need a persistent preview first, with a suggestion for my provider. Don't build a workflow that would point the agent at a URL that disappears. - If I haven't said how long the preview lasts, or it's shorter than MIN_LIFETIME, ask me in Step 0 before building. - Never tear down or expire the preview in this workflow. - If the same URL is redeployed on every push, the agent may see newer code partway through a session. Point this out in Step 0 and ask whether that's acceptable, or whether a per-commit URL should be used. First, tell me which of these fits and why, then build only that one: 1. Built-in auto-generate on PR open. Checksum's team enables this for the project, so no workflow is needed. Confirm with me that the project's configured environment (or a preview override) is what the agent should test against, then stop. 2. Workflow that calls POST /public-api/v1/auto-generate with the preview URL in envOverrides (default, and the only option here that can pass a per-PR URL). 3. Workflow that posts a "/checksum generate {{INSTRUCTIONS}}" PR comment. This needs the Checksum GitHub App on the repo, and the comment has to come from an account with write access. Check whether GITHUB_TOKEN's bot qualifies. A comment can't pass envOverrides, so this option can't point at the preview URL. Use option 2 unless I say otherwise. For option 2, create .github/workflows/{{FILE_NAME, e.g. checksum-generate.yml}}: Trigger: pull_request, types [{{opened, ready_for_review}}] Don't include synchronize unless I say so: each push would start another billable generation session. [OPTIONAL: only when the PR has label {{LABEL}}] [OPTIONAL: only for paths {{PATHS}}] Extra agent context (metadata): {{e.g. {"triggeredBy":"ci"} | none}} Wait for completion: {{no → rely on Checksum's sticky PR comment (default, recommended because a session can take hours) | yes → poll, timeout {{MINUTES, max 360 on GitHub-hosted runners}}}} Fixed requirements (from the docs): 1. Request body: prNumber (number), repoName "/" of the repo hosting the PR, branch = the PR HEAD ref (github.head_ref), not the base branch, and envOverrides: { "{{BASE_URL_VAR}}": "" }. prNumber, repoName and branch are all required, or no PR gets opened. repoName must be connected in Checksum's Git Integration. 2. Only start generation once the preview for this commit is live: check that the root and {{DEEP_ROUTE}} return 200. If the preview comes from another workflow/step, depend on it and read the URL from its output. Fail with a clear message if the URL is empty or unreachable. 3. Build the JSON with jq; never interpolate strings by hand. Send Authorization: Bearer ${{ secrets.{{SECRET_NAME}} }} and Content-Type: application/json. Expect 202 with a batchId. Fail the step on any other status and print the response body. 4. Default: don't wait. Write batchId, links.session and the preview URL (with its expiry, if known) to the job summary, and let the job finish. Checksum reports progress in its sticky PR comment. If waiting: poll GET /public-api/v1/auto-generate/batch/{batchId} every 10–30 s until allTerminal is true, fail if status is failed, print each session's prUrl, and bound it with timeout-minutes. 5. envOverrides: only application variables; CI and CHECKSUM_* are rejected with a 400. 6. Skip draft PRs (Checksum's own auto-generate does the same). 7. Prevent loops: skip PRs whose head branch starts with ChecksumAI-generated-test- or checksumai/ (Checksum's own generation and heal PRs). This matters most when the tests live in the app repo. 8. Skip fork PRs (no secrets). Least-privilege permissions (contents: read, pull-requests: read). 9. Generation works from pushed commits only. That's always true on a PR event, but mention it in a YAML comment, along with a note that the preview must outlive the generation session. 10. Don't add comment scripts or comment steps. Checksum posts and updates its own "Checksum Auto-Generate" comment on the PR from the prNumber + repoName in the request; a second comment from the workflow makes one generation look like two. ```
i
How it works
What the fixed requirements protect against, and common problems
## Why the fixed requirements are there | Requirement | What goes wrong without it | Details | | - | - | - | | `checksumai` 4.4.0 or later on the tests branch | Shard reports never merge, no verdict comes back, and auto-heal is never evaluated | [Sharding](/docs/sharding#update-checksumai-first) | | Gate on `verdict`, not counts | An empty selection or an unmerged sharded run can look like a pass | [CI/CD Integration](/docs/ci-integration#make-the-workflow-wait-and-pass-or-fail) | | Tests-repo `branch`, not the app PR branch | The run, or the heal PR, targets a branch that doesn't exist in the tests repo | [Where healing runs](/docs/auto-healing#choose-where-the-fix-pr-lands) | | Check the preview returns 200 before the run | A preview that's still deploying, or already expired, fails every test | [Choose a preview URL](#choose-a-preview-url) | | No comment scripts or comment steps | Duplicate comments; one generation looks like two. Checksum already comments on the PR. | [GitHub Action](/docs/ci-integration#run-checksum-with-the-github-action), [Generate Tests](/docs/generate-tests#start-generation-from-the-rest-api) | | No `synchronize` trigger for generation | Every push starts another billable generation session | [Generate Tests](/docs/generate-tests) | ## Troubleshooting Checksum posts the comments itself, so the PR has to reach it. With the GitHub Action on a `pull_request` event this is automatic; on other events, set `pr-number` and `repo-name`. With the REST API, send `autoHeal.repoName` + `autoHeal.prNumber` (test runs) or `prNumber` + `repoName` (generation). The repository must be connected in [Git Integration](/docs/git-integration). No tests matched the selection, which counts as a failure. Check that the tests repo has tests on the `branch` you run against and that your `grep` pattern matches them. Prompt 2 tells it to stop instead. If it didn't, remove the workflow: the preview disappears long before the generation session ends. Set up a persistent preview first (see [Choose a preview URL](#choose-a-preview-url)). ## Related The GitHub Action, the CLI, and the REST API. The generation API and PR comments. What happens after a failing run. Connect your agent to Checksum.