@agents-forge/aiqa

AIQA — AI Assisted Quality Engineering

One command. One Claude session. Full AI-assisted QA suite — requirements, test plan, test cases, Playwright scripts, a selector-validated review, and an execution report.

npx @agents-forge/aiqa https://www.yourwebsite.com

---

What makes this different

AIQA is a Super Agent — a single Claude session that orchestrates five specialist subagents, not a chain of disconnected tools. The Super Agent holds the whole pipeline in one context, so insights and snapshots flow in memory from one subagent to the next:

            ┌──────────────────────────────────────────────┐
            │            AIQA — Super Agent                 │
            │        (one Claude session, shared memory)    │
            └──────────────────────────────────────────────┘
                                  │
   analyst → qa-planner → qa-engineer → qa-reviewer → qa-reporter
   └──────────────────── subagents ───────────────────────────┘

The subagents

Subagent	Role	Output
analyst	Explores the site with playwright-cli, captures accessibility snapshots of every page	requirements.md
qa-planner	Uses snapshots to assess real UI complexity and assign risk-based priorities	test_plan.md
qa-engineer	Generates test cases + Playwright scripts using actual element refs — no guessing	test-cases/, test-scripts/
qa-reviewer	Validates every selector in the specs against the captured snapshots	review_report.md
qa-reporter	Runs the suite, captures failure snapshots, writes the stakeholder report	summary_report.md

Every subagent works against the same playwright-cli accessibility snapshots the analyst captured — so the test scripts reference real DOM elements, and the reviewer can flag any selector that doesn't exist in the ground-truth snapshots.

For a deep dive into how the Super Agent orchestrates the subagents, the runtime flow, and the key design decisions, see docs/architecture.md.

---

Install

npm install @agents-forge/aiqa
npm install --save-dev @playwright/test
npm install -g @playwright/cli@latest
playwright-cli install --skills
npx playwright install chromium

---

Authentication

Auto-detected — no config needed:

Method	Setup
Anthropic API key	ANTHROPIC_API_KEY=sk-ant-... in .env
Claude subscription	npm install -g @anthropic-ai/claude-code → claude login
GitHub Copilot	gh auth login

---

Usage

# Full pipeline — all five subagents
npx @agents-forge/aiqa https://my-app.com

# Smoke tests only
npx @agents-forge/aiqa https://my-app.com --grep @smoke

# Regression suite to custom dir
npx @agents-forge/aiqa https://my-app.com --grep @regression --dir ./aiqa-output

# Skip the review subagent
npx @agents-forge/aiqa https://my-app.com --skip qa-reviewer

# Resume after a crash
npx @agents-forge/aiqa https://my-app.com --resume

# Interactive — prompts for the URL and an optional existing requirements doc
npx @agents-forge/aiqa

# Unattended / CI — don't prompt before overwriting playwright.config.ts
npx @agents-forge/aiqa https://my-app.com --force

# Inside VS Code — open the generated docs in Markdown preview when done
npx @agents-forge/aiqa https://my-app.com --open

# Verbose mode
npx @agents-forge/aiqa https://my-app.com --verbose

---

Bring your own requirements (optional)

You can seed the pipeline with an existing requirements document — the analyst still explores the site and then merges the two, rather than skipping exploration:

1. Drop your requirements .md file into .aiqa/requirements/ (the folder is created for you on first run).
2. Run AIQA normally (or just npx @agents-forge/aiqa and answer the prompts).
3. The analyst explores the URL, then writes .aiqa/requirements/requirements.md with two clearly-labelled parts:
   • Part A — Existing Requirements (carried over from your doc, tagged [existing])
   • Part B — Newly Discovered Requirements (the addon found by exploring, tagged [discovered])

Nothing from your document is dropped, and the merge happens automatically — no mid-run prompt.

Naming: your doc can have any name. If you happen to name it requirements.md (the same as the generated output), AIQA preserves your copy as existing-requirements.md first, then writes the merged result to requirements.md — so your original is never lost.

---

Installed into an existing project?

AIQA writes everything it generates under .aiqa/. The only file placed at your repo root is playwright.config.ts (so npx playwright test can auto-discover it). Since it's typically run inside an existing project, the Super Agent guards that one file:

• If a playwright.config.ts already exists, AIQA asks once, before the run, whether it may overwrite it.
• Decline, and your existing config is left untouched — the pipeline runs against it as-is.
• In a non-interactive shell (CI, piped) it never silently overwrites — pass --force (or force: true) to opt in.
• Use --dir <path> to run the whole pipeline (and its .aiqa/ folder) in a different directory.

---

What it produces

Everything lands under .aiqa/ so your repo root stays clean — only playwright.config.ts is written to the root (so npx playwright test finds it):

.aiqa/
├── session.json                  ← pipeline state (resumable)
├── auth-state.json               ← saved login session (if any)
├── snapshots/
│   ├── home.yml                  ← accessibility trees (ground truth)
│   ├── login.yml
│   └── failure-<test>.yml        ← captured on test failure
├── requirements/
│   └── requirements.md           ← analyst subagent: business analysis
├── test-plan/
│   └── test_plan.md              ← qa-planner subagent: strategy + risk analysis
├── test-cases/
│   ├── login.md                  ← qa-engineer subagent: plain-English test cases
│   └── checkout.md
├── test-scripts/
│   ├── login.spec.ts             ← qa-engineer subagent: Playwright scripts (snapshot-accurate)
│   └── checkout.spec.ts
└── reports/
    ├── review_report.md          ← qa-reviewer subagent: selector validation + quality review
    ├── summary_report.md         ← qa-reporter subagent: stakeholder report
    ├── test-results/             ← results.json, results.xml, traces, failure screenshots
    └── playwright-report/        ← Playwright HTML report (npx playwright show-report .aiqa/reports/playwright-report)

playwright.config.ts              ← multi-browser config at repo root (overwrite-guarded)

---

Programmatic Usage

import { runAIQA } from "@agents-forge/aiqa";

const result = await runAIQA({
  target: "https://my-app.com",
  grep: "@smoke",
  skip: ["qa-reviewer"],
  cwd: "./aiqa-output",
  resume: false,
  force: false,
  open: false,
  existingRequirements: ".aiqa/requirements/my-reqs.md",
  model: "claude-sonnet-4-6",
  paths: { reports: "reports" },
  verbose: false,
});

console.log(`Passed: ${result.passed}`);
console.log(`Session: ${result.sessionFile}`);

Option	Meaning
target	URL to analyse (required)
grep	Run only tests matching a tag, e.g. @smoke
skip	Subagents to skip, by name
cwd	Output directory (where .aiqa/ is created)
resume	Resume a previous run
force	Skip the playwright.config.ts overwrite prompt
open	Open the generated docs in VS Code Markdown preview
existingRequirements	Path to an existing requirements .md to merge in
model	Claude model id
paths	Override the .aiqa base + subfolder names
verbose	Show tool-call details

---

Configuration — aiqa.config.ts

Drop an aiqa.config.ts (or .mjs / .js / .json) in your project root to set defaults. CLI flags override the config file, which overrides the built-in defaults. Loaded at runtime via jiti — no build step.

import { defineConfig } from "@agents-forge/aiqa";

export default defineConfig({
  // Run defaults (any of these is overridden by the matching CLI flag)
  url: "https://my-app.com",
  grep: "@smoke",
  skip: ["qa-reviewer"],
  open: true,
  force: false,

  // Model
  model: "claude-sonnet-4-6",

  // Rename the .aiqa base + subfolders to taste
  paths: {
    base: ".aiqa",
    requirements: "requirements",
    plan: "test-plan",
    testCases: "test-cases",
    testScripts: "test-scripts",
    reports: "reports",
  },
});

defineConfig() is optional but gives you typed autocomplete. Folder renames flow everywhere automatically — including the generated playwright.config.ts paths.

---

Skipping subagents

Each stage is a subagent you can skip by name (analyst, qa-planner, qa-engineer, qa-reviewer, qa-reporter):

# Analysis + planning only (no tests)
npx @agents-forge/aiqa https://my-app.com --skip qa-engineer,qa-reviewer,qa-reporter

# Skip the review subagent
npx @agents-forge/aiqa https://my-app.com --skip qa-reviewer

# Write tests but don't run them (skip the reporter subagent)
npx @agents-forge/aiqa https://my-app.com --skip qa-reporter

---

Resumable pipeline

If the Super Agent crashes mid-run:

npx @agents-forge/aiqa https://my-app.com --resume

The session file at .aiqa/session.json tracks which subagents completed. On --resume, any subagent whose output already exists on disk is marked done and skipped, so the pipeline picks up where it left off.

---

Troubleshooting

Error	Fix
playwright-cli not found	npm install -g @playwright/cli@latest
Browser not installed	npx playwright install chromium
Authentication failed	Set ANTHROPIC_API_KEY or run claude login
Pipeline crashed	Re-run with --resume
Won't overwrite my config in CI	Pass --force (non-interactive runs never overwrite playwright.config.ts silently)
Want to preview docs in VS Code	Run inside VS Code with --open (opens requirements / test plan / summary in Markdown preview)
Want to run tests interactively	npx playwright test --ui

---

License

MIT