Generate Tests

At a glance

Start from	What you do	What you get
REST API	Send a PR’s number, repo, and branch to `POST /auto-generate`	A PR with tests
Pull request	Comment `/checksum generate`	A PR with tests
Coding agent (MCP)	Ask in plain English	A PR, or tests to review in the session
Slack	Mention `@checksum` in a thread	A PR when the thread has PR context
Web app	Click Generate on saved test flows	A PR with tests
Chrome extension	Record yourself clicking through a flow, optionally narrating	One test, or a batch split into flows, to review in the session

Reference for AI: generation at a glance

Interface	Entry point	Input	Opens a PR
REST API	`POST https://api.checksum.ai/public-api/v1/auto-generate`	`prNumber` + `repoName` + `branch` (head)	Yes
REST API (poll)	`GET https://api.checksum.ai/public-api/v1/auto-generate/batch/{batchId}`	`batchId`	n/a
GitHub	PR comment `/checksum generate [instructions]`	PR diff + instructions	Yes
Automatic	Auto-generate on PR open (enabled by Checksum, non-draft PRs)	PR diff	Yes
MCP	`checksum_test_generate`	PR (number + repo + branch), `extraInstructions`, or existing specs (`userStoryIds` / `collectionNames`)	PR-based runs by default; others with `autoCreatePR`
Slack	`@checksum <request>` in a thread	Thread conversation	With PR context only
Web app	Test Generation → Generate	Saved test flows	Yes
Chrome extension	Checksum Capture (Beta): record a walkthrough	Recorded DOM actions + optional voice narration	From the session, on demand

Auth: every REST call sends Authorization: Bearer $CHECKSUM_API_KEY. Key location: Settings → Project Settings.
Async: POST /auto-generate returns 202 with batchId. Poll GET /auto-generate/batch/{batchId} every 10–30 s until allTerminal is true.
Cloud generation clones from the Git provider, so it only sees pushed commits.
PR-based generation always diffs against the PR head branch.
Output per flow: one *.checksum.md story + one *.checksum.spec.ts test on a new branch, delivered as a PR.

How generation works

Every trigger starts an agent session, which runs this pipeline:

1 · PlanDeep mode only: interview + plan

→

2 · ImplementStory .checksum.md + test .checksum.spec.ts

→

3 · ReviewSelf-review for quality and correctness

→

4 · VerifyActually runs the tests against your environment

→

5 · DeliverPull request to your tests repo

Test Generation Flow — The test generation pipeline, from trigger to pull request.

Checksum’s world model is built from real user-behavior patterns, and the agent reaches ~97% test accuracy. Generated tests include data setup and cleanup, selectors grounded in your source code, and a production-ready structure. Unlike local coding agents, Checksum runs in the cloud 24/7, generating and verifying tests autonomously at scale. You can start many sessions in parallel and come back to finished PRs.

Detect first for broad coverageTo cover a whole feature area, run detection first. You can then review and prioritize the proposed flows before any code is written. For one quick, targeted test, generate straight from a manual flow or a PR.

Start generation

Pick where you want to start. Every option starts the same agent session and pipeline. They differ in what you give the agent and whether a PR opens automatically.

Start generation from the REST API

Use the API when a release bot, an internal tool, or a custom CI step should kick off test generation. You tell Checksum which pull request to cover. It replies right away with a batchId, works in the background, and opens a pull request in your tests repository when the tests are ready. Progress also appears as a single comment on the source PR, posted by Checksum, so your CI step doesn’t need to comment. To have a coding agent build this CI step for you, use CI Setup Prompts → Prompt 2.

POSThttps://api.checksum.ai/public-api/v1/auto-generate

curl -X POST https://api.checksum.ai/public-api/v1/auto-generate \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prNumber": 42,
    "repoName": "acme-co/webapp",
    "branch": "feature/checkout-redesign",
    "metadata": { "triggeredBy": "release-bot" },
    "envOverrides": { "BASE_URL": "https://preview.example.com/pr-42/" }
  }'

Only the first three fields are required:

Field	What it’s for
`prNumber`, `repoName`, `branch`	The pull request to cover: its number, its repository (`owner/repo`), and its head branch
`metadata`	Optional context for the agent, such as who triggered the run
`envOverrides`	Optional variables for this run only, typically a preview URL like `{"BASE_URL": "https://pr-42.preview.example.com"}`

Checksum answers with 202 Accepted:

{
  "batchId": "batch-abc",
  "sessionId": "sess-xyz",
  "links": { "session": "https://app.checksum.ai/..." }
}

Keep the batchId to check on progress. links.session opens the live session in the web app if you want to watch.

Check on progress

Ask for the batch every 10–30 seconds until allTerminal is true. When it finishes, each session’s prUrl points to the new pull request.

GET

curl https://api.checksum.ai/public-api/v1/auto-generate/batch/$BATCH_ID \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"

{
  "batchId": "batch-abc",
  "status": "in_progress",
  "totalSessions": 1,
  "completedCount": 0,
  "failedCount": 0,
  "allTerminal": false,
  "sessions": [
    { "sessionId": "sess-xyz", "status": "running", "prUrl": null }
  ]
}

The batch moves from pending to in_progress, then ends as completed or failed. It has the same shape as a healing batch, without the healing-only fields testRunId and sessions[].checksumTestId.

Example: a script that generates tests for a PR and waits for the result

Needs curl, jq, and the variables CHECKSUM_API_KEY, PR_NUMBER, REPO (owner/repo), and HEAD_BRANCH.

#!/usr/bin/env bash
set -euo pipefail
BASE="https://api.checksum.ai/public-api/v1"

# 1. Start generation for the PR
BATCH=$(curl -sf -X POST "$BASE/auto-generate" \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" -H "Content-Type: application/json" \
  -d "$(jq -n --argjson pr "$PR_NUMBER" --arg repo "$REPO" --arg br "$HEAD_BRANCH" \
        '{prNumber: $pr, repoName: $repo, branch: $br}')" | jq -r .batchId)
echo "Started batch: $BATCH"

# 2. Poll until every session is finished
until [[ "$(curl -sf "$BASE/auto-generate/batch/$BATCH" \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" | tee /tmp/batch.json | jq -r .allTerminal)" == "true" ]]; do
  sleep 30
done

# 3. Print each session's result and PR link
jq -r '.sessions[] | "\(.status)  \(.prUrl // "no PR")"' /tmp/batch.json

Reference for AI: generation REST API

POST `https://api.checksum.ai/public-api/v1/auto-generate`

Starts a generation agent session for a pull request. Opens a PR in the tests repository when finished. Progress is posted as one sticky comment on the source PR. Generates from a PR only: saved test flows require the web app, and description-only runs require MCP.

Header	Value
`Authorization`	`Bearer $CHECKSUM_API_KEY`
`Content-Type`	`application/json`

Body field	Type	Required	Description
`prNumber`	number	Yes	Pull request to generate tests for. Used as agent context and for the sticky progress comment.
`repoName`	string	Yes	Repository that hosts the PR, `<owner>/<repo>`. Must be connected in Git Integration.
`branch`	string	Yes	The PR’s head branch. The agent diffs against it.
`metadata`	object	No	Free-form key/value pairs forwarded to the agent as context.
`envOverrides`	object (string → string)	No	Per-session environment variables layered on the project environment. Keys named `CI` or starting with `CHECKSUM_` are rejected with `400`.

Response field (202)	Type	Description
`batchId`	string	Pass to `GET /auto-generate/batch/{batchId}`
`sessionId`	string	The generating agent session
`links.session`	string (URL)	Live session in the web app

Error	Cause	Fix
`400`	Reserved key in `envOverrides` (`CI`, `CHECKSUM_*`)	Pass only application variables, e.g. `BASE_URL`
`401`	Missing or invalid API key	Send `Authorization: Bearer $CHECKSUM_API_KEY`. Verify with `GET /me`.

Next: poll GET /public-api/v1/auto-generate/batch/{batchId}.

GET `https://api.checksum.ai/public-api/v1/auto-generate/batch/{batchId}`

Path parameter	Type	Description
`batchId`	string	From `POST /auto-generate`

Response field (200)	Type	Description
`batchId`	string	The batch being reported
`status`	enum	`pending` (not started), `in_progress` (at least one session running), `completed` (all sessions succeeded), `failed` (one or more sessions failed)
`totalSessions`	number	Sessions in the batch
`completedCount`	number	Sessions that finished successfully
`failedCount`	number	Sessions that failed
`allTerminal`	boolean	`true` once every session has finished. Stop polling.
`sessions[].sessionId`	string	Agent session ID
`sessions[].status`	string	Session state, e.g. `running`, `completed`
`sessions[].prUrl`	string \| null	Tests-repo PR URL once opened, otherwise `null`

Unlike a healing batch, a generation batch has no testRunId and no sessions[].checksumTestId.Polling rule: every 10–30 seconds until allTerminal is true. On failed, open links.session from the start response.

Ask from a pull request

If the Checksum GitHub App is installed on a repository, anyone with write access can ask for tests right on the pull request. Comment:

/checksum generate

Add instructions on the same line to steer the agent:

/checksum generate focus on the new SSO login flow and error states

Checksum reacts with 👀 while it starts, 🚀 once the session is running, or 👎 if it can’t start (usually missing permissions or the app isn’t installed). Progress goes in one sticky comment that is updated in place, so the thread stays tidy. The agent always works from the PR’s head branch, so feature-branch PRs get tests for the right code.

Automation welcomeBots and automation accounts work just like teammates. PRs opened by Dependabot, Renovate, or your own release bot can get generated tests. Any bot account with write access can post /checksum generate, or call POST /auto-generate, to start the agent whenever you need it.

Auto-generate when a PR opens

Enabled by Checksum Checksum can turn on automatic generation for every non-draft PR. It works the same as /checksum generate, without anyone commenting. Draft PRs are skipped. Ask your Checksum team to enable it for your project.

Reference for AI: /checksum generate

Item	Value
Syntax	`/checksum generate [free-text instructions on the same line]` as a PR comment
Requirement	Checksum GitHub App installed on the repository (Git Integration)
Requirement	Commenter (person or bot account) has write access to the repository
Target code	Always the PR head branch. Checksum fetches the head ref from GitHub.
Progress	One sticky PR comment, edited in place
Output	A PR in the tests repository
Automatic variant	Auto-generate on PR open: non-draft PRs only, enabled per project by Checksum

Reaction	Meaning
👀	Command received, session starting
🚀	Session started successfully
👎	Command rejected (missing permissions, or the app isn’t installed)

Ask your coding agent (MCP)

With the Checksum MCP server connected, you can ask Claude Code, Cursor, or VS Code for tests in plain English:

Generate Checksum tests for PR 142 in acme/web on branch feature/checkout.

Give your agent a pull request (number, repository, and branch) and Checksum opens a PR with the tests. If you ask twice for the same PR, it joins the run already in progress. Or describe a flow, such as “the login flow with an invalid password”, and the tests stay in the session for you to review without a PR. Or target specs you already have, such as a collection or the flows detection just found, and Checksum generates tests for exactly those. For a description or targeted run, ask for a PR up front, or once you’ve reviewed the session. Your agent can also point a run at a preview deployment, switch on Deep mode, report progress, and pass along follow-up instructions while the run is going.

Push firstCloud generation checks out your repo from your Git provider, so it only sees pushed commits. Commit and push before you ask. The Claude Code plugin does this for you.

Reference for AI: checksum_test_generate

MCP server: https://api.checksum.ai/public-api/mcp (Streamable HTTP). Tool: checksum_test_generate. Returns a batchId (a sessionId for a targeted run). It changes things: it starts a billable cloud run and can open a PR.

Input	Required	Behavior
PR number + repository + branch	All three together, or one of the options below	Generates for the PR and opens a PR with the tests. A repeat request for the same PR joins the run in progress.
`extraInstructions`	Instead of a PR	Describes the flow to cover. Tests stay in the session unless `autoCreatePR` is `true`.
`userStoryIds` and/or `collectionNames`	Instead of a PR	Targeted generation for exactly those specs. Can’t be combined with a PR. Returns a `sessionId`.
`codeRepoNames`, `repoBranches`	No	Which connected code repositories (and branches) a targeted run reads
`autoCreatePR`	No	`true` opens a PR when a description or targeted run finishes
`deepMode`	No	`true` for Deep mode: the session pauses for plan approval (`checksum_session_approve`). Default: Standard
`envOverrides`	No	Points the run at another environment, e.g. a preview deployment

Related tool	Use
`checksum_session_status`	Progress, the agent’s latest messages, changed files, and `prUrl`. Poll until the run finishes.
`checksum_session_prompt`	Send a follow-up instruction to a running session
`checksum_session_approve` / `checksum_session_answer`	Approve a deep-mode plan / answer the agent’s questions
`checksum_session_create_pr`	Open the PR for a finished session that didn’t open one

Precondition: commits must be pushed. The cloud run clones from the Git provider.

Mention `@checksum` in Slack

Once the Checksum Slack app is installed (Notifications & Slack), mention it in a thread in a channel it’s been invited to:

@checksum add E2E coverage for the new checkout discount flow

Checksum reads the whole thread, not just your message, and uses it as the agent’s instructions. The bot replies with a link to follow the session. If the thread includes pull-request context, the tests arrive as a PR; otherwise they wait in the session for you to review. Mention the bot in a thread rather than a top-level message, and be specific: “Add tests for guest checkout with an expired card” works much better than “add tests.”

Record a walkthrough with the Chrome extension

Beta The Checksum Capture Chrome extension turns one walkthrough of your app into a batch of tests. Click through the flow you want covered, and talk through it if you like. Checksum hands the recording to the agent, which starts a generation session from it.It’s not a 1:1 recorder like Playwright’s codegen, which turns each click into one line of one test. Checksum treats your recording as training material for the agent. The agent learns the flow, the screens, and what you care about, then writes the tests it implies. When you split the recording into flows, one recording can produce 10, 20, 30 or more tests, depending on what you ask for and how deep and broad the flow is.

Install the extension

Add Checksum Capture (Beta) to Chrome from the Chrome Web Store.

Connect it to your project

Open the extension and paste your project’s API key, which you’ll find on the Settings page in the Checksum web app. The key tells Checksum which project the recording and the generated tests belong to.

Go to your app

In the same browser, go straight to the page of your app where the flow starts.

Set up the session

Name the session, then choose how the agent should turn your recording into tests:

Exactly what I recorded (one test): one test that follows your walkthrough.
Split into flows (one test per flow): the agent breaks the recording, and your narration, into separate flows and writes a test for each. This is how one recording becomes a batch of tests.

To talk while you record, turn on Record microphone narration. Chrome asks for microphone access the first time.

Record and click through

Start recording and use the app the way a user would. The extension captures your actions from the page’s DOM as you go. If you turned on narration, talk the agent through what matters (see below).

Stop, and let the agent generate

When you stop, the recording is sent to Checksum and an agent session starts generating tests from it.

Review and open a PR

The generated tests appear on the session’s page in the Checksum web app. Review them there, then open a pull request to your tests repository from the session when you’re ready.

What to say while you record

Narration is the best way to tell the agent things it can’t learn from clicks alone. Use it to:

Call out edge cases: “An expired card should show an error here, not a spinner.”
State assertions: “After saving, the total at the top should update and the badge should say Draft.”
Share app knowledge: requirements, business rules, or behavior that a user wouldn’t notice, such as “Only admins see this tab” or “This field is required for EU customers.”
Ask for variations: “There are four plan types and two billing periods. Capture every combination.” The agent expands the one path you showed into a test for each variation.

Say how much you wantWith Split into flows, the size of the batch follows your ask. One quiet walkthrough gives a focused set of tests. Narrating variations, roles, and error cases gives the agent the breadth to write many more.

Reference for AI: Checksum Capture Chrome extension

Item	Value
Name	Checksum Capture (Beta), a Chrome extension
Install	`https://chromewebstore.google.com/detail/checksum-capture-beta/pnooemkcjngfobcfkmfpmoglbocjiadi`
Auth	Connected to a project with the project’s API key, found on the Settings page of the web app (API keys)
Session setup	Session name · output mode: Exactly what I recorded (one test) or Split into flows (one test per flow) · optional Record microphone narration
Captures	User actions from the DOM while recording, plus optional microphone narration
Output	Starts a generation agent session that uses the recording as context. “Exactly what I recorded” gives one test. “Split into flows” gives a batch, one test per flow (often 10–30+).
Delivery	Tests stay on the agent session page in the web app. No PR is opened automatically: open one from the session.
Not	A 1:1 action-to-code recorder (unlike Playwright codegen)
Narration uses	Edge cases, assertions, requirements and app knowledge, and requests to cover variations or combinations

What gets generated

For each flow, the agent writes two files to a new branch in your tests repository:

File	Purpose
`*.checksum.md`	The story: a human-readable spec with frontmatter, data setup, data cleanup, and numbered steps with verifications. It is the source of truth for what the test does.
`*.checksum.spec.ts`	The test: Playwright TypeScript using Checksum fixtures. Pure Playwright underneath, so you can run it anywhere.

The branch is named like ChecksumAI-generated-test-20260315143022, and the PR description explains the flow being covered. See Story & Test Format for both file formats.

Generation statuses

Each test flow in the web app shows where its generation stands:

Status	Meaning
Idle	No generation in progress for this flow
Queued	Generation is queued and will start shortly
Running	The agent is planning, implementing, or reviewing
Verifying	The agent is executing the tests to prove they pass
Completed	Tests are generated and the PR is ready
Failed	Generation hit an error. Open the session for details.
Aborted	Generation was cancelled
Edited	The generated tests were manually edited afterward
PR Merged	You merged the generated-tests PR
PR Closed	You closed the PR without merging

API and MCP callers see the equivalent state on the batch (pending → in_progress → completed / failed) and on each session. See Agent Sessions for the session lifecycle.

Review and merge

Generated tests arrive as a normal pull request. Checksum’s agent behaves like another developer on your team. By default, nothing reaches your main branch without your review. If you turn on auto-merge, Checksum’s PRs merge without a human review and the steps below are optional.

Read the PR

Review the story file first, since it’s the fastest way to confirm the agent understood the flow. Then review the test code.

Run it yourself (optional)

Check out the branch and run npx checksumai test -g "<test title>". See Running Tests.

Request changes

Edit the files directly, or re-trigger generation with more specific instructions. A flow you edit shows the Edited status.

Merge

Once merged, the tests join your suite and are covered by auto-maintenance.

Troubleshooting

The API returned 400 for envOverrides

envOverrides only accepts your application’s own variables. Names reserved by Checksum, including CI and anything starting with CHECKSUM_, are rejected.

Tests were generated but no PR was opened

Checksum opens a PR only when it knows the pull request number, repository, and branch. Slack mentions without PR context, MCP runs from a description or targeted specs (without autoCreatePR), and Chrome extension recordings leave the tests in the session. Open the session and create the PR from there (PR Eligible state), ask your coding agent to call checksum_session_create_pr, or re-run against a PR.

The tests cover old code

Cloud sessions clone from your Git provider. Push your commits, then trigger generation again.

/checksum generate got a 👎 reaction

The commenter needs write access, and the Checksum GitHub App must be installed on that repository. Check Settings → Git Integration for an Active status.

The session failed with "application unreachable"

The agent couldn’t load your environment URL. Check that the URL in Environments is correct and up. If the environment is behind a VPN or firewall, confirm Checksum’s IP addresses are allowlisted (see Security & Access).

Detect Test Flows

Let the agent propose what to test.

Deep vs Standard

Trade speed for thoroughness.

Agent Sessions

Answer questions, approve plans, and track progress.

Start Here

Managing Your Project

Test Generation

Test Execution

Maintenance & Monitoring

Integrations

Security & Admin

At a glance

How generation works

Start generation

Start generation from the REST API

Check on progress

POST `https://api.checksum.ai/public-api/v1/auto-generate`

GET `https://api.checksum.ai/public-api/v1/auto-generate/batch/{batchId}`

Ask from a pull request

Auto-generate when a PR opens

Ask your coding agent (MCP)

Mention `@checksum` in Slack

Record a walkthrough with the Chrome extension

What to say while you record

Generate from a collection

What gets generated

Generation statuses

Review and merge

Troubleshooting

Detect Test Flows

Deep vs Standard

Agent Sessions

​At a glance

​How generation works

​Start generation

​Start generation from the REST API

​Check on progress

​POST https://api.checksum.ai/public-api/v1/auto-generate

​GET https://api.checksum.ai/public-api/v1/auto-generate/batch/{batchId}

​Ask from a pull request

​Auto-generate when a PR opens

​Ask your coding agent (MCP)

​Mention @checksum in Slack

​Record a walkthrough with the Chrome extension

​What to say while you record

​Generate from a collection

​What gets generated

​Generation statuses

​Review and merge

​Troubleshooting

​Related

Detect Test Flows

Deep vs Standard

Agent Sessions

At a glance

How generation works

Start generation

Start generation from the REST API

Check on progress

POST `https://api.checksum.ai/public-api/v1/auto-generate`

GET `https://api.checksum.ai/public-api/v1/auto-generate/batch/{batchId}`

Ask from a pull request

Auto-generate when a PR opens

Ask your coding agent (MCP)

Mention `@checksum` in Slack

Record a walkthrough with the Chrome extension

What to say while you record

Generate from a collection

What gets generated

Generation statuses

Review and merge

Troubleshooting

Related