Story & Test Format

At a glance

File	What’s in it	Who reads it
`*.checksum.md`	The test in plain language: setup, cleanup, and steps	Anyone: QA, PMs, reviewers
`*.checksum.spec.ts`	The Playwright code that runs the story	Engineers and CI

Reference for AI: story and test file contract

Part of the contract	Value
File names	`tests/<name>.checksum.md` (story) + `tests/<name>.checksum.spec.ts` (test), in the `checksum/` project folder
Story frontmatter (YAML)	`title`, `checksumTestId`, `startUrl`, `envUser`
Story sections (Markdown)	`## Data Setup`, `## Data Cleanup`, `## Steps`. Each step has an `Action` and a `Verify`.
Test import	`import { init } from "@checksum-ai/runtime";`
`init()` returns	`test`, `defineChecksumTest`, `login`, `expect`, `checksumAI`
Test fixtures	`page` (Playwright page), `vs` (variable store for sharing data between steps)
Test declaration	`test(defineChecksumTest("<title>", "<checksumTestId>"), async ({ page, vs }) => { … })`
Run	`npx checksumai test -g "<title>"`

Frontmatter field	Type	Description
`title`	string	Name of the test scenario; same string passed to `defineChecksumTest`
`checksumTestId`	string (e.g. `OFJlm`)	Unique test ID. Same ID in `defineChecksumTest`, in run results (`checksumTestId`), and in `POST /public-api/v1/execution/tests` `testIds`
`startUrl`	string (path)	URL path where the test begins
`envUser`	object (`username`)	References an environment test user

The story is the source of truth: healing and future regeneration read it as the statement of intent. Keep story and test in step when editing.
IDs must match between frontmatter checksumTestId and defineChecksumTest(title, id).
Pure Playwright underneath: replacing the Checksum imports with standard Playwright imports runs the test with vanilla Playwright.
Product-bug failures are tagged @bug in the test source by auto-healing.

Two files per test

File	Audience	Role
`*.checksum.md`	Anyone: QA, PMs, reviewers	The source of truth for what the test does: frontmatter, data setup, data cleanup, and steps with verifications
`*.checksum.spec.ts`	Engineers, CI	Standard Playwright TypeScript, generated from the story and run by the Checksum CLI or plain Playwright

Story files (`.checksum.md`)

Story files describe the test scenario in structured Markdown with YAML frontmatter.

user-can-create-opportunity.checksum.md

---
title: "User Can Create Opportunity"
checksumTestId: OFJlm
startUrl: /opportunities
envUser:
  username: checksum@example.mailosaur.net
---

## Data Setup
**Method:** API

- POST `/api/contacts` - Create contact with name "John Doe"

## Data Cleanup
**Method:** API

- DELETE `/api/contacts/{contactId}` - Remove test contact

## Steps

1. **Navigate to Opportunities**
   - Action: Go to the opportunities page
   - Verify: Opportunities list is visible

2. **Create New Opportunity**
   - Action: Click "New Opportunity", fill in the form with contact "John Doe"
   - Verify: Form fields are populated correctly

3. **Save and Confirm**
   - Action: Click "Save"
   - Verify: New opportunity appears in the list with contact name

Frontmatter fields

Field	Description
`title`	Name of the test scenario
`checksumTestId`	Unique identifier for the test. It is the same ID used in `defineChecksumTest`, in run results (`checksumTestId`), and in `POST /execution/tests` (see IDs).
`startUrl`	The URL path where the test begins
`envUser`	Credentials to use. References an environment user (see Test users).

Sections

Data Setup: how to prepare test data before the test runs (API calls, database seeds)
Data Cleanup: how to clean up afterward (delete created records)
Steps: the actions the user takes, each with a verification

Test files (`.checksum.spec.ts`)

Test files are standard Playwright TypeScript tests, generated from the story. You can run them with the Checksum CLI or directly with Playwright.

user-can-create-opportunity.checksum.spec.ts

import { init } from "@checksum-ai/runtime";

const { test, defineChecksumTest, login, expect, checksumAI } = init();

test(
  defineChecksumTest("User Can Create Opportunity", "OFJlm"),
  async ({ page, vs }) => {
    // Step 1: Navigate to Opportunities
    await page.goto("/opportunities");
    await expect(page.getByRole("heading", { name: "Opportunities" })).toBeVisible();

    // Step 2: Create New Opportunity
    await page.getByRole("button", { name: "New Opportunity" }).click();
    await page.getByLabel("Contact").fill("John Doe");

    // Step 3: Save and Confirm
    await page.getByRole("button", { name: "Save" }).click();
    await expect(page.getByText("John Doe")).toBeVisible();
  }
);

Checksum fixtures

Tests use Checksum fixtures (init() from @checksum-ai/runtime). These add automatic recovery, smart selector recovery, report uploading, and a variable store for sharing data between steps. Underneath, the tests are pure Playwright. To run them with vanilla Playwright, replace the Checksum imports with standard Playwright imports. There’s no vendor lock-in.

Page Object Model

Generated tests can also use the Page Object Model pattern for better organization:

pages/opportunities.page.ts

import { Page, Locator } from "@playwright/test";

export class OpportunitiesPage {
  readonly newButton: Locator;
  readonly contactField: Locator;
  readonly saveButton: Locator;

  constructor(private page: Page) {
    this.newButton = page.getByRole("button", { name: "New Opportunity" });
    this.contactField = page.getByLabel("Contact");
    this.saveButton = page.getByRole("button", { name: "Save" });
  }

  async createOpportunity(contactName: string) {
    await this.newButton.click();
    await this.contactField.fill(contactName);
    await this.saveButton.click();
  }
}

tests/opportunities.checksum.spec.ts

import { init } from "@checksum-ai/runtime";
import { OpportunitiesPage } from "../pages/opportunities.page";

const { test, defineChecksumTest, expect } = init();

test(
  defineChecksumTest("User Can Create Opportunity", "OFJlm"),
  async ({ page }) => {
    const opportunities = new OpportunitiesPage(page);
    await page.goto("/opportunities");
    await opportunities.createOpportunity("John Doe");
    await expect(page.getByText("John Doe")).toBeVisible();
  }
);

Key characteristics

Built on Playwright: the @checksum-ai/runtime wrapper adds Checksum features (auto-recovery, reporting, variable store) on top of Playwright.
Grounded selectors: selectors come from your actual app code (data-testid, role attributes, text content), not guesses.
defineChecksumTest: wraps each test with a name and a Checksum test ID for tracking and reporting.
Variable store (vs): a shared store for passing data between steps, such as created record IDs for cleanup.
Web-first assertions: Playwright’s built-in assertions (toBeVisible(), toHaveText()) retry automatically.

Where the files live

Generated tests go in the tests/ folder of your Checksum test project, next to checksum.config.ts (see Test Repository & Config):

checksum/
├── checksum.config.ts
├── playwright.config.ts
├── login.ts
└── tests/
    ├── user-can-create-opportunity.checksum.md
    └── user-can-create-opportunity.checksum.spec.ts

Each generation arrives on its own branch, e.g. ChecksumAI-generated-test-20260315143022, as a pull request. When auto-healing later fixes a test, it edits these same files in a separate PR. Failures classified as product bugs are tagged @bug in the source (see Auto-Healing).

Editing generated tests

Your repository is the source of truth. You can edit the story or the test directly, and Checksum picks up the change on its next sync. A flow whose tests you’ve edited shows the Edited status in the web app. Keep the story and the test in step, because healing and future regeneration read the story as the statement of intent.

Running generated tests

# Run all tests
npx checksumai test

# Run a specific test by name
npx checksumai test -g "User Can Create Opportunity"

See Running Tests for the CLI, CI, and REST options, including every test flag.

Generate Tests

Produce these files.

Test Repository & Config

Project layout and checksum.config.ts.

Running Tests

CLI, CI, and REST API.

Start Here

Managing Your Project

Test Generation

Test Execution

Maintenance & Monitoring

Integrations

Security & Admin

At a glance

Two files per test

Story files (`.checksum.md`)

Frontmatter fields

Sections

Test files (`.checksum.spec.ts`)

Checksum fixtures

Page Object Model

Key characteristics

Where the files live

Editing generated tests

Running generated tests

Generate Tests

Test Repository & Config

Running Tests

​At a glance

​Two files per test

​Story files (.checksum.md)

​Frontmatter fields

​Sections

​Test files (.checksum.spec.ts)

​Checksum fixtures

​Page Object Model

​Key characteristics

​Where the files live

​Editing generated tests

​Running generated tests

​Related

Generate Tests

Test Repository & Config

Running Tests

At a glance

Two files per test

Story files (`.checksum.md`)

Frontmatter fields

Sections

Test files (`.checksum.spec.ts`)

Checksum fixtures

Page Object Model

Key characteristics

Where the files live

Editing generated tests

Running generated tests

Related