@testrelic/playwright-analytics
TestRelic
A test analytics toolkit that generates structured JSON and HTML reports from your Playwright test runs — capturing navigation timelines, API call tracking, network statistics, failure diagnostics, and CI metadata. Supports browser E2E tests, API-only tests, and unified workflows that combine both.
What It Does
When you run your Playwright tests with the TestRelic reporter and fixture, you get a JSON + HTML report that captures:
- Navigation timeline — Every URL visited during each test, in chronological order, with navigation type detection (goto, link click, back/forward, SPA route changes, hash changes)
- API call tracking — Captures HTTP method, URL, status code, request/response headers and bodies, response times, and assertions for every API call
- Network statistics — Total requests, failed requests, bytes transferred, and resource type breakdowns (scripts, images, stylesheets, fonts, XHR)
- Test results — Pass/fail/flaky status, duration, retry count, and tags for every test
- Failure diagnostics — Error messages, source code snippets pointing to the exact failure line, and optional stack traces
- CI metadata — Auto-detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build ID, commit SHA, and branch
- Sensitive data redaction — Automatically scrubs AWS keys, Bearer tokens, private keys, credential URLs, and sensitive API headers/body fields
- Action steps — Captures Playwright test steps (actions, assertions, custom steps) with video timestamp sync
- Console log capture — Collects browser console messages (log, warn, error) per test
- Screenshots & videos — Automatically captures screenshots and video recordings, organized per test in an
artifacts/folder alongside the report - HTML report — Self-contained interactive HTML report with test grid, filter bar, and expandable drawer showing navigation timeline, API call details, and artifacts
Quick Start
1. Install
Install @testrelic/playwright-analytics alongside @playwright/test (a required peer dependency):
# npm
npm install -D @playwright/test @testrelic/playwright-analytics
# pnpm
pnpm add -D @playwright/test @testrelic/playwright-analytics
# yarn
yarn add -D @playwright/test @testrelic/playwright-analyticsIf your project already has @playwright/test installed, just add @testrelic/playwright-analytics. Any version >=1.35.0 of Playwright is compatible.
Installation notes
Peer dependency:
@playwright/testmust be a direct dependency in the same project. If you see a peer dependency warning from pnpm or npm after installing only@testrelic/playwright-analytics, install@playwright/testexplicitly as shown above — this is a tooling hint, not a TestRelic bug.Postinstall: On a fresh install (outside CI), the package prints a line like
[testrelic] Created .testrelic/testrelic-config.json. This is intentional — it scaffolds a config file at your project root for cloud upload. Updatetestrelic-repo.nameto match your project and setTESTRELIC_API_KEYin your environment before using cloud features. In CI environments whereCI=trueorCI=1, the script is skipped automatically; create the config file manually if needed. See Cloud Integration for full details.
2. Add the reporter to your Playwright config
Option A: Use defineConfig (recommended) — automatically enables video, screenshots, trace capture, and the TestRelic reporter:
// playwright.config.ts
import { defineConfig } from '@testrelic/playwright-analytics';
export default defineConfig({
testDir: './tests',
// video: 'on', screenshot: 'on', trace: 'on', and the TestRelic reporter
// are all configured automatically. Override any of these in `use:` if needed.
});You can pass reporter-specific options as a second argument:
export default defineConfig(
{ testDir: './tests' },
{ includeStackTrace: true, includeCodeSnippets: true },
);Option B: Manual setup — configure Playwright and the reporter separately:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
includeStackTrace: true,
includeCodeSnippets: true,
includeNetworkStats: true,
}],
],
});Note: When using
defineConfigfrom@testrelic/playwright-analytics, Playwright's defaultlistreporter is preserved automatically so CLI commands likenpx playwright test --listcontinue to work.
3. Use the fixture in your tests
TestRelic provides two fixture modes depending on your testing needs:
E2E tests (browser) — uses page for navigation tracking:
import { test, expect } from '@testrelic/playwright-analytics/fixture';
test('homepage loads correctly', { tag: ['@e2e'] }, async ({ page }) => {
await page.goto('https://example.com');
await expect(page.locator('h1')).toBeVisible();
});API tests — uses request for API call tracking:
import { test as base } from '@playwright/test';
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
import { expect } from '@testrelic/playwright-analytics/fixture';
const test = base.extend(testRelicApiFixture);
test('fetch posts', { tag: ['@api'] }, async ({ request }) => {
const response = await request.get('https://api.example.com/posts');
expect(response.status()).toBe(200);
});Unified tests (browser + API) — uses both page and request together:
import { test, expect } from '@testrelic/playwright-analytics/fixture';
test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
// API call
const apiResponse = await request.get('https://api.example.com/user/1');
const user = await apiResponse.json();
// Browser navigation
await page.goto('https://example.com/profile');
await expect(page.locator('.user-name')).toHaveText(user.name);
});4. Run your tests
npx playwright testThe JSON report will be written to ./test-results/analytics-timeline.json and the HTML report to ./test-results/analytics-timeline.html.
Troubleshooting
require() of ES Module … fixture.js
Symptom: Running npx playwright test throws an error like:
Error [ERR_REQUIRE_ESM]: require() of ES Module .../node_modules/@testrelic/playwright-analytics/dist/fixture.js not supported.
Cause: Your project's TypeScript or Playwright configuration compiles test files as CommonJS. When a CJS module calls require('@testrelic/playwright-analytics/fixture'), some toolchains mis-resolve the subpath export to the ESM build (fixture.js) instead of the CJS build (fixture.cjs).
Fix — Option A (recommended): Run tests as ESM
Add "type": "module" to your project's package.json, or configure TypeScript to emit ESM:
// package.json
{ "type": "module" }// tsconfig.json
{
"compilerOptions": {
"module": "NodeNext", // or "ESNext"
"moduleResolution": "NodeNext"
}
}Fix — Option B: Use the explicit CJS subpath
Import the CJS build via the dedicated subpath export (no deep node_modules path needed):
// base-test.ts or your fixture wrapper
import { test, expect } from '@testrelic/playwright-analytics/fixture/cjs';// api tests
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture/cjs';This is stable across version upgrades and pnpm/npm layouts.
[testrelic] No API key configured
This message is printed when TESTRELIC_API_KEY is not set in the environment the test process sees. Local HTML/JSON reports are still written — this only affects cloud upload. Set the key in your shell or CI secrets; see Cloud Integration.
Cloud Integration
TestRelic can upload test results, action steps, artifacts (screenshots, videos), and network logs directly to the TestRelic cloud platform for team-wide visibility, historical tracking, and the interactive Steps dashboard.
Setup
Step 1: Get your API key
Sign in to platform.testrelic.ai and generate an API key from Settings → API Keys.
Step 2: Create the config file
Installing the package automatically creates .testrelic/testrelic-config.json in your project root via a postinstall script. If it wasn't created (e.g. in CI with CI=true), create it manually:
{
"cloud": {
"apiKey": "$TESTRELIC_API_KEY",
"endpoint": "https://platform.testrelic.ai/api/v1",
"upload": "both"
},
"testrelic-repo": {
"name": "my-project"
}
}Replace my-project with the name of your repository as it should appear in the cloud dashboard. The apiKey value of "$TESTRELIC_API_KEY" is automatically resolved from the environment variable of the same name at runtime — no secrets in your config file.
Step 3: Set your API key in the environment
export TESTRELIC_API_KEY=your_api_key_hereIn CI, add TESTRELIC_API_KEY as a repository secret and pass it to the test job.
Step 4: Enable cloud upload in your Playwright config
// playwright.config.ts
export default defineConfig({
reporter: [
['list'],
[
'@testrelic/playwright-analytics',
{
cloud: {
apiKey: process.env.TESTRELIC_API_KEY,
endpoint: 'https://platform.testrelic.ai/api/v1',
upload: 'both',
uploadArtifacts: true,
artifactMaxSizeMb: 50,
},
includeActionSteps: true,
includeCodeSnippets: true,
includeNetworkStats: true,
includeArtifacts: true,
captureResponseBody: true,
captureRequestBody: true,
},
],
],
use: {
video: 'on',
screenshot: 'on',
trace: 'on',
},
});Config File Reference
The .testrelic/testrelic-config.json file is the project-level configuration for cloud integration. It should be committed to version control.
Full schema:
{
"cloud": {
"apiKey": "$TESTRELIC_API_KEY",
"endpoint": "https://platform.testrelic.ai/api/v1",
"upload": "both",
"timeout": 30000
},
"testrelic-repo": {
"name": "my-project"
},
"queue": {
"maxAge": "7d",
"directory": ".testrelic/queue"
}
}| Key | Description |
|---|---|
cloud.apiKey |
API key for authentication. Use "$TESTRELIC_API_KEY" to read from the environment variable. |
cloud.endpoint |
Cloud API endpoint. Defaults to https://platform.testrelic.ai/api/v1. |
cloud.upload |
Upload strategy — see Upload Strategy below. |
cloud.timeout |
Request timeout in milliseconds (default: 30000). |
testrelic-repo.name |
Repository/project name shown in the cloud dashboard. |
queue.maxAge |
How long to retain queued failed uploads before discarding (default: "7d"). |
queue.directory |
Directory for queued uploads (default: .testrelic/queue). |
Upload Strategy
The upload field controls how test data is sent to the cloud:
| Value | Behaviour |
|---|---|
"realtime" |
Streams test events to the cloud as they happen. Fast feedback but skips the batch payload. |
"batch" |
Sends a single complete JSON payload after all tests finish. Includes full step data. |
"both" |
Recommended. Streams realtime events AND sends the complete batch payload at the end. Required for the Steps tab in the cloud dashboard to be populated. |
Important: Using
upload: "realtime"alone skips the batch payload that the cloud platform uses to populate the Steps, navigation timeline, and network stats views. Use"both"to ensure all data appears in the dashboard.
Environment Variables
Environment variables take the highest priority and override both reporter options and the config file:
| Variable | Description |
|---|---|
TESTRELIC_API_KEY |
API key — always set this in CI secrets rather than hardcoding it. |
TESTRELIC_CLOUD_ENDPOINT |
Override the API endpoint (e.g. for a self-hosted or stage deployment). |
TESTRELIC_UPLOAD_STRATEGY |
Override the upload strategy (realtime, batch, or both). |
TESTRELIC_CLOUD_TIMEOUT |
Override the request timeout in milliseconds. |
Priority order (highest wins):
- Environment variables
- Reporter
cloud:options inplaywright.config.ts .testrelic/testrelic-config.json- Built-in defaults
.gitignore Guidance
The .testrelic/ directory serves multiple purposes. Only the config file should be committed:
# Commit: .testrelic/testrelic-config.json
# Ignore runtime directories:
.testrelic/queue/
.testrelic/cache/
Migrating from Previous Versions
Config file location (v2.3.x → v2.4.0)
The config file has moved from a flat .testrelic file to .testrelic/testrelic-config.json. This prevents a filesystem conflict where the SDK needed to create a .testrelic/ directory (for queue and cache) while the config file occupied the same path.
If you have an existing .testrelic flat file, it still works — the SDK reads it with a deprecation warning:
[testrelic] Deprecation: config file ".testrelic" has moved to ".testrelic/testrelic-config.json".
Please migrate your config file to the new location.
To migrate, rename the file and move it into the directory:
mkdir -p .testrelic
mv .testrelic .testrelic/testrelic-config.json # on Unix/macOSproject key renamed to testrelic-repo (v2.3.x → v2.4.0)
The project config key has been renamed to testrelic-repo for clarity. The old key still works with a deprecation warning:
[testrelic] Deprecation: "project" key in config is deprecated. Rename it to "testrelic-repo".
Update your config file:
// Before
{ "project": { "name": "my-project" } }
// After
{ "testrelic-repo": { "name": "my-project" } }Output Example
{
"schemaVersion": "1.0.0",
"testRunId": "797128f5-c86d-466c-8d6d-8ec62dfc70b6",
"startedAt": "2026-02-07T10:41:28.759Z",
"completedAt": "2026-02-07T10:41:36.794Z",
"totalDuration": 8035,
"summary": {
"total": 6,
"passed": 5,
"failed": 1,
"flaky": 0,
"skipped": 0
},
"ci": {
"provider": "github-actions",
"buildId": "12345678",
"commitSha": "abc123def456",
"branch": "main"
},
"timeline": [
{
"url": "https://en.wikipedia.org/wiki/Main_Page",
"navigationType": "goto",
"visitedAt": "2026-02-07T10:41:29.844Z",
"duration": 216,
"specFile": "tests/homepage.spec.ts",
"domContentLoadedAt": "2026-02-07T10:41:30.200Z",
"networkStats": {
"totalRequests": 40,
"failedRequests": 0,
"totalBytes": 1289736,
"byType": {
"xhr": 0,
"document": 1,
"script": 9,
"stylesheet": 2,
"image": 27,
"font": 2,
"other": 0
}
},
"tests": [
{
"title": "homepage.spec.ts > Homepage > loads correctly",
"status": "passed",
"duration": 1028,
"failure": null
}
]
}
]
}Configuration
All options are passed as the second element of the reporter tuple:
General Options
| Option | Type | Default | Description |
|---|---|---|---|
outputPath |
string |
./test-results/analytics-timeline.json |
Where to write the JSON report |
includeStackTrace |
boolean |
false |
Include full stack traces in failure diagnostics |
includeCodeSnippets |
boolean |
true |
Include source code snippets around the failure line |
codeContextLines |
number |
3 |
Lines of context above/below the failure line |
includeNetworkStats |
boolean |
true |
Track network requests and bytes per navigation |
includeArtifacts |
boolean |
true |
Include screenshots and video in the report |
includeActionSteps |
boolean |
true |
Capture Playwright test steps (actions, assertions, custom steps) with video sync |
captureConsoleLogs |
boolean |
true |
Capture browser console messages (log, warn, error) per test |
navigationTypes |
NavigationType[] | null |
null (all) |
Filter timeline to specific navigation types |
redactPatterns |
(string | RegExp)[] |
Built-in patterns | Additional patterns to redact from error output |
testRunId |
string | null |
null (auto UUID) |
Override the test run ID |
metadata |
Record<string, unknown> | null |
null |
Attach custom metadata to the report |
openReport |
boolean |
true |
Auto-open the HTML report after test run |
quiet |
boolean |
false |
Suppress TestRelic banner output |
API Tracking Options
| Option | Type | Default | Description |
|---|---|---|---|
trackApiCalls |
boolean |
true |
Enable/disable API call tracking entirely |
captureRequestHeaders |
boolean |
true |
Capture request headers for each API call |
captureResponseHeaders |
boolean |
true |
Capture response headers for each API call |
captureRequestBody |
boolean |
true |
Capture request body/payload for each API call |
captureResponseBody |
boolean |
true |
Capture response body for each API call |
captureAssertions |
boolean |
true |
Track assertion results (pass/fail with expected/actual values) |
redactHeaders |
string[] |
['authorization', 'cookie', 'set-cookie', 'x-api-key'] |
Header names to redact (case-insensitive) |
redactBodyFields |
string[] |
['password', 'secret', 'token', 'apiKey', 'api_key'] |
Body field names whose values are replaced with [REDACTED] |
apiIncludeUrls |
(string | RegExp)[] |
[] (all URLs) |
Only track API calls matching these patterns |
apiExcludeUrls |
(string | RegExp)[] |
[] (none excluded) |
Exclude API calls matching these patterns |
Cloud Upload Options
Passed inside the cloud: key of the reporter options. These are merged with (and take priority over) the .testrelic/testrelic-config.json values.
| Option | Type | Default | Description |
|---|---|---|---|
cloud.apiKey |
string |
— | API key for cloud authentication. Prefer process.env.TESTRELIC_API_KEY. |
cloud.endpoint |
string |
https://platform.testrelic.ai/api/v1 |
Cloud API endpoint. |
cloud.upload |
'realtime' | 'batch' | 'both' |
'batch' |
Upload strategy. Use 'both' to populate all cloud dashboard views including Steps. |
cloud.timeout |
number |
30000 |
Request timeout in milliseconds. |
cloud.uploadArtifacts |
boolean |
true |
Upload screenshots and videos to cloud storage. |
cloud.artifactMaxSizeMb |
number |
50 |
Maximum size per artifact file in MB. Files exceeding this are skipped. |
cloud.queueDirectory |
string |
.testrelic/queue |
Directory for queued uploads that failed and will be retried. |
cloud.queueMaxAge |
string |
'7d' |
Maximum age for queued items before they are discarded (e.g. '7d', '24h'). |
Configuration Examples
Default (zero config) — captures everything with built-in redaction:
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
}],
]Minimal capture (no bodies) — reduces report size, tracks only method/URL/status/timing:
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
captureRequestBody: false,
captureResponseBody: false,
}],
]URL filtering — only track specific endpoints:
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
apiIncludeUrls: ['**/api/**'],
apiExcludeUrls: ['**/health', '**/metrics'],
}],
]Custom redaction — add your own sensitive header/body field names:
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
redactHeaders: ['authorization', 'cookie', 'set-cookie', 'x-api-key', 'x-custom-secret'],
redactBodyFields: ['password', 'secret', 'token', 'ssn', 'creditCard'],
}],
]Disable API tracking — browser-only report with no API call data:
reporter: [
['list'],
['@testrelic/playwright-analytics', {
outputPath: './test-results/analytics-timeline.json',
trackApiCalls: false,
}],
]Screenshots & Videos
When using defineConfig from @testrelic/playwright-analytics, video recording and screenshot capture are enabled by default (video: 'on', screenshot: 'on'). If using manual config, enable them in Playwright's use section:
export default defineConfig({
use: {
video: 'on',
screenshot: 'on',
},
reporter: [['list'], ['@testrelic/playwright-analytics']],
});Artifacts are automatically copied to an artifacts/ folder next to your report:
test-results/
├── analytics-timeline.json
├── analytics-timeline.html
└── artifacts/
├── homepage-loads-correctly/
│ ├── screenshot.png
│ └── video.webm
└── search-works--retry-1/
├── screenshot.png
└── video.webm
- Screenshots are captured at the end of each test (
.png) - Videos record the full browser session (
.webm) - Retried tests get separate folders with a
--retry-Nsuffix - The HTML report embeds links to these artifacts with inline image/video viewers
- Disable artifact collection with
includeArtifacts: false
Action Steps
TestRelic captures Playwright test steps — actions (click, fill, goto), assertions (expect), and custom steps (test.step()). Steps are timestamped and synced with the video timeline so you can jump to the exact moment in the recording.
Enable/disable with the includeActionSteps option (default: true):
['@testrelic/playwright-analytics', {
includeActionSteps: true,
}]In the HTML report, action steps appear in the test drawer alongside the navigation timeline, showing:
- Step category (action, assertion, or custom)
- Step title and duration
- Video timestamp for playback sync
Console Log Capture
TestRelic captures browser console messages (log, warn, error) per test. Useful for debugging test failures without re-running.
Enable/disable with the captureConsoleLogs option (default: true):
['@testrelic/playwright-analytics', {
captureConsoleLogs: true,
}]Network Statistics
Each navigation in the timeline includes detailed network statistics:
- Total requests — number of network requests during the navigation
- Failed requests — count and URLs of failed requests (useful for spotting broken assets)
- Bytes transferred — total bytes received
- Resource breakdown — requests grouped by type:
xhr,document,script,stylesheet,image,font,other
Disable with includeNetworkStats: false.
Viewing Reports
For small test suites (< 50 tests), the report is a self-contained HTML file — just open it in your browser.
For large test suites (50+ tests), TestRelic uses streaming mode to avoid memory issues. Test details are stored as separate files and loaded on demand via a local server. After tests complete, the terminal shows:
[testrelic] View report: npx testrelic serve ./test-results/.testrelic-report
Run that command and open http://127.0.0.1:9323 in your browser. The server auto-starts after test runs when not in CI, but if you need to restart it later:
npx testrelic serve ./test-results/.testrelic-report
npx testrelic serve ./test-results/.testrelic-report --port 9400 # custom portMerging Shard Reports
When running Playwright with sharding, each shard produces its own report. Merge them with the CLI:
npx testrelic merge shard-1.json shard-2.json shard-3.json -o merged-report.jsonOr programmatically:
import { mergeReports } from '@testrelic/playwright-analytics/merge';
await mergeReports(
['shard-1.json', 'shard-2.json'],
{ output: 'merged-report.json' }
);Testing Modes
TestRelic supports three testing modes, each with its own fixture:
E2E Testing (Browser)
Use the unified fixture which provides page with navigation tracking — page load timing, DOM content loaded, network idle detection, and network request statistics.
import { test, expect } from '@testrelic/playwright-analytics/fixture';
test('search works', { tag: ['@e2e'] }, async ({ page }) => {
await page.goto('https://example.com');
await page.fill('#search', 'query');
await page.click('button[type=submit]');
});API Testing
Use the API-only fixture which provides request without any browser dependency — tracks HTTP method, URL, status code, headers, bodies, response times, and assertions.
import { test as base } from '@playwright/test';
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
import { expect } from '@testrelic/playwright-analytics/fixture';
const test = base.extend(testRelicApiFixture);
test('CRUD operations', { tag: ['@api'] }, async ({ request }) => {
const response = await request.post('https://api.example.com/posts', {
data: { title: 'New Post', body: 'Content' },
});
expect(response.status()).toBe(201);
});Custom Page Fixtures (authenticated pages, etc.)
If your tests create pages manually via browser.newContext() / context.newPage() (e.g., for auth with storageState), the built-in page fixture is bypassed and no network/console/navigation data is captured. Use trackPage() to enable tracking on any manually-created page:
import { test as base, expect, trackPage } from '@testrelic/playwright-analytics/fixture';
type MyFixtures = {
authenticatedPage: Page;
};
export const test = base.extend<MyFixtures>({
authenticatedPage: async ({ browser }, use, testInfo) => {
const ctx = await browser.newContext({ storageState: 'auth.json' });
const page = await ctx.newPage();
const cleanup = await trackPage(page, testInfo); // ← enables tracking
await use(page);
await cleanup(); // ← flushes data to report
await ctx.close();
},
});Important: Without
trackPage(), custom page fixtures will only show screenshots, videos, and action steps — but no network requests, console logs, or navigation timeline.
Unified Testing (Browser + API)
Use the unified fixture which provides both page and request — the TestRelic report shows navigation timeline AND API call details for the same test. This is ideal for:
- Contract testing: Verify API data matches what the browser renders
- Setup/teardown via API: Use API calls to set up test data, then verify in the browser
- Full-stack workflows: Test end-to-end flows that span both UI and API layers
import { test, expect } from '@testrelic/playwright-analytics/fixture';
test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
// Fetch data via API
const apiResponse = await request.get('https://api.example.com/user/1');
const user = await apiResponse.json();
// Verify it renders correctly in the browser
await page.goto('https://example.com/profile');
await expect(page.locator('.user-name')).toHaveText(user.name);
});Examples
Full working examples for E2E, API, and unified testing are available in the testrelic-sdk-examples repository:
| Example | Type | What It Demonstrates |
|---|---|---|
| api-testing | API only | CRUD operations, API chaining, response assertions, config options, error handling |
| unified-testing | E2E + API | Browser navigation and API calls in the same test |
| wikipedia | E2E | Homepage, search, link navigation, media-heavy pages |
| flipkart | E2E | Homepage, product search, product pages |
| E2E | Search queries, results navigation |
Each example is a standalone project — clone and run:
git clone https://github.com/testrelic-ai/testrelic-sdk-examples.git
cd testrelic-sdk-examples/api-testing
npm install
npx playwright testPrerequisites
- Node.js >= 18
- Playwright >= 1.35.0
License
MIT