CI Integration

Run your API tests automatically in CI with the Checksum API Testing GitHub Action. It triggers a run of your suites in Checksum’s cloud, waits for the result, and returns pass/fail counts you can act on, so a broken API blocks the merge.

Setup

Get your project API key from the Checksum web app (Settings → Project Settings).
Add it to your repository as a secret named CHECKSUM_API_KEY (Settings → Secrets and variables → Actions).
Add the workflow below.

Store the API key as a CI secret. Never commit it to your repository.

GitHub Actions

Create .github/workflows/checksum-api-tests.yml:

name: API Testing with Checksum AI

on:
  pull_request:
    types: [opened, synchronize, reopened]
  workflow_dispatch: {}

jobs:
  checksum-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Run Checksum test suites
        uses: checksum-ai/checksum-action/api-testing@main
        with:
          checksum_api_key: ${{ secrets.CHECKSUM_API_KEY }}
          suite-ids: ""

By default (suite-ids: "") the Action runs every suite that has generated code, the same as Run Suite in the app. The run appears on the API Test Runs page, and failures flow to Triage as usual.

Inputs

Input	Required	Default	Description
`checksum_api_key`	Yes	None	Your Checksum project API key
`suite-ids`	No	`""`	Comma-separated suite UUIDs to run. Empty runs all suites with code
`base-url`	No	`https://aiagents.checksum.ai`	Checksum API base URL
`poll-interval-seconds`	No	`10`	How often the Action polls for run status
`timeout-seconds`	No	`600`	Max time to wait for the run to finish

Outputs

Output	Description
`test-run-id`	ID of the test run
`status`	Final status of the run
`passed-count`	Number of passed tests
`failed-count`	Number of failed tests
`healed-count`	Number of healed tests
`bug-count`	Number of tests marked as bugs
`error-count`	Number of tests that ended in error
`result-url`	Link to view the run in Checksum

Run specific suites

Pass a comma-separated list of suite UUIDs to run only those:

- name: Run specific test suites
  uses: checksum-ai/checksum-action/api-testing@main
  with:
    checksum_api_key: ${{ secrets.CHECKSUM_API_KEY }}
    suite-ids: "uuid-1,uuid-2,uuid-3"

Gate the build on results

Give the step an id, then read its outputs in later steps to log the result and fail the build when tests fail:

- name: Run Checksum tests
  id: checksum
  uses: checksum-ai/checksum-action/api-testing@main
  with:
    checksum_api_key: ${{ secrets.CHECKSUM_API_KEY }}

- name: Fail if tests failed
  if: ${{ steps.checksum.outputs.failed-count != '0' || steps.checksum.outputs.error-count != '0' }}
  run: |
    echo "Tests failed. See ${{ steps.checksum.outputs.result-url }}"
    exit 1

This fails the build on either failed or errored tests. To also block on confirmed bugs, add || steps.checksum.outputs.bug-count != '0' to the condition.

Use result-url in a PR comment or job summary so reviewers can jump straight to the run report.

Get Started

Create & Review

Configure

Run & Monitor

CI & Automation

Setup

GitHub Actions

Inputs

Outputs

Run specific suites

Gate the build on results

Next Steps

​Setup

​GitHub Actions

​Inputs

​Outputs

​Run specific suites

​Gate the build on results

​Next Steps

Setup

GitHub Actions

Inputs

Outputs

Run specific suites

Gate the build on results

Next Steps