Skip to main content

​
At a glance

​
Checksum endpoints to allow from your network

  • Code repository: read-only. Tests repository: read + write; every change arrives as a pull request.
  • API key: one per project, sent as Authorization: Bearer $CHECKSUM_API_KEY. Store it only in CI secrets or a secret manager.
  • MCP browser sign-in: scoped to the projects the user approves, re-checked on every request.
  • Inbound: Checksum reaches the test environment from fixed IP addresses. The list is provided by the Checksum contact during onboarding, or on request from support@checksum.ai.
Developer guide
Network, repository scopes, credentials, API keys, and MCP tokens

​
Network access

Two directions of traffic are involved: your tools calling Checksum, and Checksum reaching your test environment.

​
Let your tools reach Checksum

These run inside your network and call Checksum: If traffic to api.checksum.ai hangs or fails on a corporate network, ask IT to allow it over HTTPS (port 443). A TLS-inspecting proxy can break connections by re-signing certificates, so add api.checksum.ai to its bypass list. To test:

​
Let Checksum reach your test environment

Checksum’s AI agents and Playwright runners must be able to load your UAT/test environment. If it’s behind a VPN, firewall, or similar protection, allowlist Checksum’s IP addresses. Your Checksum contact provides the current list during onboarding. You can also request it from support@checksum.ai. Also make sure bot-protection, WAF, or rate-limiting rules don’t block automated Playwright traffic from Checksum’s addresses. During onboarding, Checksum checks that the bot isn’t blocked and that the AI can reach your site. Fix any access blockers before the POV kickoff date so they don’t delay your evaluation. See Onboarding & Proof of Value.
Symptom of a blocked environmentAn agent session that ends Failed because “the application under test was unreachable” is almost always a network or allowlist problem.
  • Connectivity test: curl -I https://api.checksum.ai/public-api/mcp
  • TLS-inspecting proxies must have api.checksum.ai on their bypass list, because certificate re-signing breaks connections.
  • Symptom of a blocked environment: an agent session ends Failed with “the application under test was unreachable”.

​
Repository access

Checksum connects to two kinds of repositories, and each gets a different access level: On GitHub you install the Checksum GitHub App on the repositories you choose, so you never pick scopes yourself. GitLab, Bitbucket, and Azure DevOps use an access token with read-only scopes for the code repository. For the exact minimum scopes per provider, open Reference for AI: repository scopes, or see Git Integration → Permissions. For tests-repository permissions, protected-branch notes, and connection steps, see Git Integration → Permissions.
You stay in control of what mergesYour repository is always the source of truth. By default, every generated or healed test arrives as a PR that you review and merge on your terms. If you turn on auto-merge, Checksum’s PRs merge without a human review. See Test Repository & Config.
  • Code repository access is read-only on every provider. Checksum never writes to it.
  • Tests repository access is read + write: it writes generated and healed tests, creates branches, opens pull requests, comments, and may auto-merge. Tests-repository scopes and protected-branch notes: Git Integration → Permissions.
  • Azure DevOps supports code repositories only.

​
Data and credentials

​
Test user credentials and variables

  • Test users (username/password per environment) and custom environment variables are configured under Settings → Testing Environment. They are stored securely and used only during test execution. See Environments & Test Users.
  • Your tests repository reads credentials and variables from a local .env file that Checksum maintains and you download with npx checksumai dotenv --download, so you don’t commit secrets to source control. Adding them in the web app doesn’t update .env. Keep .env out of git.
  • In CI, pass credentials as CI secrets. Variables set explicitly take precedence over the downloaded .env.
  • Use dedicated test accounts in a UAT/test environment, not real users’ accounts.

​
Source code

  • Checksum clones your repositories from your connected Git provider in the cloud. Code is never uploaded from a developer’s machine, including when you use the MCP server, which only carries instructions and results.

​
Test artifacts

  • When report hosting is enabled (by default when CI=true), the CLI uploads results, videos, screenshots, HAR files (network traffic), and Playwright traces to the Checksum dashboard. Local runs keep reports local unless you enable hostReports. See Results, Reports & Traces.
  • HAR files and traces can capture request and response data from your test environment. Keep that in mind when choosing test data.
  • Report and trace links returned through MCP are signed and expire.

​
API keys

Each project has a project API key (Settings → Project Settings). The CLI, CI pipelines, the REST API, and API-key MCP connections all use it. See API Keys & Authentication.
Treat the key as a secretAnyone with the key can run tests, access your project, start billable cloud runs, and open pull requests on your repo. Store it only in CI secrets or a secret manager, and never in committed files.
  • An API key is tied to exactly one project.
  • If a key may have leaked, rotate it in Settings → Project Settings and update your CI secrets.
  • MCP clients that store a key in a config file (Claude Code --header, Cursor mcp.json) store it in plain text. Prefer browser sign-in, or VS Code’s inputs prompt.

​
MCP connections

  • Browser sign-in is scoped. At approval time you choose which projects a client may act on. A client can never reach a project you didn’t approve.
  • Access is re-checked on every request. If you lose access to a project, the client loses it too.
  • Connections belong to the user, not the project. Review or revoke them from the MCP connections card on My Profile. Revoking cuts the client off from every project you granted.
  • Tools that cost money or open PRs are marked as such, so clients can ask for confirmation first.
Full details: MCP server.
▦
In the Checksum web app
Roles, admin actions, and revoking access

​
Workspace roles and admin actions

Webhook URLs and bot tokens entered into connectors are masked after creation, and bot tokens are never shown again. The Slack bot can only see channels it has been invited to. Install it only in Slack workspaces your organization owns.

​
Revoking access

i
How it works
User sign-in and related pages

​
Single sign-on

Users can sign in with their company Microsoft account. Checksum requests only openid, email, and User.Read (delegated). It stores only the user’s name and email, and holds no Microsoft tokens after sign-in. You can restrict access to assigned users or groups in Entra ID. Full reviewer guide: Microsoft Entra ID SSO.

Git permissions

Exact scopes per provider and repository type.

Microsoft Entra ID SSO

A step-by-step guide for IT reviewers.

Onboarding checklist

What to provide before kickoff.

API Keys & Authentication

Base URL, headers, and IDs.