@asserthive/webtest-ai

WebTest AI

CLI-first browser testing for teams that want AI assistance without handing pass/fail truth to a browser agent.

WebTest AI turns human-readable specs into deterministic browser runs through a portable execution-driver layer. Reports preserve screenshots, traces, quality signals, and integration-friendly evidence. Model-backed workflows are optional and bounded to proposal flows such as discovery, recording cleanup, and reviewed healing.

It is published under the AssertHive package scope:

npm install --save-dev @asserthive/webtest-ai@beta
npx webtest-ai --help

For a one-off invocation without installing it first:

npx @asserthive/webtest-ai@beta --help

Why It Exists

Modern product teams are moving toward agentic development, but browser quality still needs evidence, repeatability, and reviewable changes. WebTest AI is built around that line:

• humans and agents can write simple specs
• the runner converts those specs into known action types
• deterministic browser execution owns pass/fail truth
• reports preserve evidence that people and agents can inspect
• healing and model workflows produce proposals instead of silently rewriting tests

Compared with prompt-only browser agents, WebTest AI is designed to be a stable quality layer: agent-ready, deterministic, auditable, customizable, and safe to run locally, in CI, or through MCP-enabled agent workflows.

Capabilities

• Markdown suites with multiple tests per file
• deterministic browser execution through the bundled Playwright driver
• customizable execution-driver boundary for future or project-specific browser runtimes
• accessibility-tree action surfaces for resilient role/name-oriented interaction
• opt-in a11y and web-vitals quality checks
• screenshots, traces, JSON reports, HTML reports, and CI summaries
• guarded healing proposals backed by local UI inventory
• recorder for turning browser interaction into spec drafts
• model-backed discovery and cleanup when an explicit model profile is configured
• MCP server for agents to read specs/reports/artifacts and run explicit debug tools

Quick Start

Install the beta package and the Chromium browser:

npm install --save-dev @asserthive/webtest-ai@beta
npx playwright install chromium

Create a local config when you want to override defaults:

cp webtest-ai.config.example.json webtest-ai.config.json

Create a small suite at specs/homepage.md:

mkdir -p specs

---
suite: homepage
baseUrl: https://example.com
tags: [smoke]
---

# Homepage Loads

## Steps
1. Open "/"

## Expected
1. Assert text "Example Domain"

Run it:

npx webtest-ai run --suite specs/homepage.md

Commands

• webtest-ai run: run a Markdown suite or directory of suites
• webtest-ai debug: run with Playwright Inspector enabled
• webtest-ai record: capture a browser flow into a Markdown spec draft
• webtest-ai heal: review and approve verified UI inventory proposals
• webtest-ai discover: generate bounded model-backed test-flow proposals
• webtest-ai report: export reports or render CI summaries
• webtest-ai mcp: start the MCP server over stdio

Detailed command docs live at the WebTest AI docs, with a first-run guide at the getting started guide. The beta changelog lives at the WebTest AI changelog.

Local-First Trust Model

WebTest AI does not call a model by default. Deterministic runs, assertions, screenshots, quality checks, and reports work locally. Model-backed features require an explicit model profile and are limited to proposal-style workflows.

Generated artifacts can contain application evidence. The built-in reports redact common headers, query params, emails, and bearer tokens, but raw Playwright traces may still contain data from the browser session. Treat artifacts/ as test evidence and apply your normal retention policy.

Config

By default WebTest AI looks for:

webtest-ai.config.json

Missing config files are allowed; built-in defaults are used. The example config is safe to commit:

webtest-ai.config.example.json

Local runtime memory is written under .webtest-ai/ by default and should not be committed.

For the full configuration and API reference, see the config and API reference.

Beta Rollout

WebTest AI is still in early beta. The npm package is published under the beta tag while the source repository remains private during early feedback.

Try the CLI with:

npm install --save-dev @asserthive/webtest-ai@beta
npx playwright install chromium
npx webtest-ai --help

The docs home for the package is the WebTest AI docs.

License

MIT AssertHive