Generated Tests

Checksum’s API tests aren’t a black box. Every test is standard Python written with pytest and committed to your repository, so you can open any test and read exactly what it does, edit it, and run it independently of Checksum.

Where the tests live

Generated tests are organized under a checksum/ directory in your repo:

Path	Contents
`checksum/tests/`	The test files (`test_*.py`), one per suite
`checksum/utils/`	Shared utility functions the tests import
`checksum/archives/`	Archived tests, kept out of runs
`checksum/api-specs/`	Your uploaded sources and the generated endpoint catalog

What a generated test looks like

A test is a function whose docstring spells out the Journey, Steps, Payloads, and Assertions (the same summary you see on the API Tests page). The body drives the journey by calling utility functions and asserting the result of each step. Each test receives an authenticated HTTP client.

from checksum.utils.cleanup import delete_customer_if_present
from checksum.utils.customers import (
    create_customer,
    retrieve_customer,
    delete_customer,
    unique_customer_fields,
)


def test_customer_create_retrieve_delete(authenticated_client):
    """
    Journey: Create a customer, retrieve it by id, then delete it.

    Steps:
      1. POST /v1/customers          Create a customer with generated fields
      2. GET /v1/customers/{id}      Retrieve it by the id returned from step 1
      3. DELETE /v1/customers/{id}   Delete the same customer
    """
    fields = unique_customer_fields()
    customer_id = None
    try:
        created = create_customer(
            authenticated_client,
            {"name": fields["name"], "email": fields["email"]},
            run_id=fields["run_id"],
        )
        assert created.status_code == 200
        customer_id = created.json()["id"]            # capture a dynamic value

        fetched = retrieve_customer(authenticated_client, customer_id)
        assert fetched.status_code == 200
        assert fetched.json()["email"] == fields["email"]

        deleted = delete_customer(authenticated_client, customer_id, run_id=fields["run_id"])
        assert deleted.status_code == 200
        assert deleted.json()["deleted"] is True
        customer_id = None
    finally:
        delete_customer_if_present(authenticated_client, customer_id)

Journeys, not isolated requests

Testing an endpoint tells you it returns a 200. Testing a journey tells you your product actually works. A real user creates a resource, updates it, triggers a downstream effect, and expects the state to stay consistent throughout. Checksum chains those calls together so the bugs that only appear across a sequence get caught. Dynamic values are wired automatically. When a step creates a resource, the test captures the values the API returns (IDs, tokens, references) and passes them into later steps, so you never hardcode an ID. Generated data is uniquified per run (note the run_id above), which keeps tests stable instead of brittle against a single frozen database state.

Assertions

The agent derives assertions from what each step should produce:

Status codes the call is expected to return.
Response structure and field values in the body.
Cross-step persistence, verifying that state created in one step is correctly reflected in later steps (the same id comes back on retrieve, the updated email persists on re-fetch, and so on).

It covers happy paths and failure cases. Alongside the successful flow, the agent generates tests for invalid inputs, missing records, unauthorized requests, and mid-journey errors. Verifying that your API fails correctly matters as much as verifying that it succeeds.

Payloads

Request bodies for POST and PUT calls come from your spec: field names, types, constraints, and example values. Where your source provides examples the agent uses them; where it doesn’t, it generates values consistent with the field definitions.

Shared utility functions

Reusable setup, teardown, and helper logic lives in utility functions under checksum/utils/, which the tests import and call rather than duplicating in every test. A utility is just a small, readable function:

# checksum/utils/customers.py
def create_customer(client, payload, *, run_id):
    return client.post(
        "/v1/customers",
        data=payload,
        headers={"Idempotency-Key": idempotency_key(f"create-customer-{run_id}")},
    )

Cleanup runs in a try/finally using a utility (like delete_customer_if_present above), so a run never leaves state behind even if a step fails.

How they’re organized

Level	What it is
Test suite	A file of related test cases grouped by API flow or domain.
Test case	A single journey: a named sequence of calls verifying one scenario end to end.
Test step	One API call within a journey.
Utility function	Shared setup, teardown, or helper logic that cases import and call.

Browse this on the API Tests page, or open the raw files in API Files.

You own the tests

Because the tests are plain pytest in your repo, they run anywhere pytest runs: locally during development or natively in your CI pipeline, like any other test. Edit them freely; the agent is aware of manual edits and flags an affected test for your review rather than silently overwriting your changes.

Get Started

Create & Review

Configure

Run & Monitor

CI & Automation

Where the tests live

What a generated test looks like

Journeys, not isolated requests

Assertions

Payloads

Shared utility functions

How they’re organized

You own the tests

Next Steps

​Where the tests live

​What a generated test looks like

​Journeys, not isolated requests

​Assertions

​Payloads

​Shared utility functions

​How they’re organized

​You own the tests

​Next Steps

Where the tests live

What a generated test looks like

Journeys, not isolated requests

Assertions

Payloads

Shared utility functions

How they’re organized

You own the tests

Next Steps