npm.io
2.11.1 • Published yesterdayCLI

@testrelic/playwright-analytics

Licence
MIT
Version
2.11.1
Deps
1
Size
4.3 MB
Vulns
0
Weekly
905
Install scriptsThis package runs scripts during installation (preinstall/install/postinstall)

TestRelic

A test analytics toolkit that generates structured JSON and HTML reports from your Playwright test runs — capturing navigation timelines, API call tracking, network statistics, failure diagnostics, and CI metadata. Supports browser E2E tests, API-only tests, and unified workflows that combine both.

What It Does

When you run your Playwright tests with the TestRelic reporter and fixture, you get a JSON + HTML report that captures:

  • Navigation timeline — Every URL visited during each test, in chronological order, with navigation type detection (goto, link click, back/forward, SPA route changes, hash changes)
  • API call tracking — Captures HTTP method, URL, status code, request/response headers and bodies, response times, and assertions for every API call
  • Network statistics — Total requests, failed requests, bytes transferred, and resource type breakdowns (scripts, images, stylesheets, fonts, XHR)
  • Test results — Pass/fail/flaky status, duration, retry count, and tags for every test
  • Failure diagnostics — Error messages, source code snippets pointing to the exact failure line, and optional stack traces
  • CI metadata — Auto-detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build ID, commit SHA, and branch
  • Sensitive data redaction — Automatically scrubs AWS keys, Bearer tokens, private keys, credential URLs, and sensitive API headers/body fields
  • Action steps — Captures Playwright test steps (actions, assertions, custom steps) with video timestamp sync
  • Console log capture — Collects browser console messages (log, warn, error) per test
  • Screenshots & videos — Automatically captures screenshots and video recordings, organized per test in an artifacts/ folder alongside the report
  • HTML report — Self-contained interactive HTML report with test grid, filter bar, and expandable drawer showing navigation timeline, API call details, and artifacts

Quick Start

1. Install

Install @testrelic/playwright-analytics alongside @playwright/test (a required peer dependency):

# npm
npm install -D @playwright/test @testrelic/playwright-analytics

# pnpm
pnpm add -D @playwright/test @testrelic/playwright-analytics

# yarn
yarn add -D @playwright/test @testrelic/playwright-analytics

If your project already has @playwright/test installed, just add @testrelic/playwright-analytics. Any version >=1.35.0 of Playwright is compatible.

Installation notes
  • Peer dependency: @playwright/test must be a direct dependency in the same project. If you see a peer dependency warning from pnpm or npm after installing only @testrelic/playwright-analytics, install @playwright/test explicitly as shown above — this is a tooling hint, not a TestRelic bug.

  • Postinstall: On a fresh install (outside CI), the package prints a line like [testrelic] Created .testrelic/testrelic-config.json. This is intentional — it scaffolds a config file at your project root for cloud upload. Update testrelic-repo.name to match your project and set TESTRELIC_API_KEY in your environment before using cloud features. In CI environments where CI=true or CI=1, the script is skipped automatically; create the config file manually if needed. See Cloud Integration for full details.

2. Add the reporter to your Playwright config

Option A: Use defineConfig (recommended) — automatically enables video, screenshots, trace capture, and the TestRelic reporter:

// playwright.config.ts
import { defineConfig } from '@testrelic/playwright-analytics';

export default defineConfig({
  testDir: './tests',
  // video: 'on', screenshot: 'on', trace: 'on', and the TestRelic reporter
  // are all configured automatically. Override any of these in `use:` if needed.
});

You can pass reporter-specific options as a second argument:

export default defineConfig(
  { testDir: './tests' },
  { includeStackTrace: true, includeCodeSnippets: true },
);

Option B: Manual setup — configure Playwright and the reporter separately:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['@testrelic/playwright-analytics', {
      outputPath: './test-results/analytics-timeline.json',
      includeStackTrace: true,
      includeCodeSnippets: true,
      includeNetworkStats: true,
    }],
  ],
});

Note: When using defineConfig from @testrelic/playwright-analytics, Playwright's default list reporter is preserved automatically so CLI commands like npx playwright test --list continue to work.

3. Use the fixture in your tests

TestRelic provides two fixture modes depending on your testing needs:

E2E tests (browser) — uses page for navigation tracking:

import { test, expect } from '@testrelic/playwright-analytics/fixture';

test('homepage loads correctly', { tag: ['@e2e'] }, async ({ page }) => {
  await page.goto('https://example.com');
  await expect(page.locator('h1')).toBeVisible();
});

API tests — uses request for API call tracking:

import { test as base } from '@playwright/test';
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
import { expect } from '@testrelic/playwright-analytics/fixture';

const test = base.extend(testRelicApiFixture);

test('fetch posts', { tag: ['@api'] }, async ({ request }) => {
  const response = await request.get('https://api.example.com/posts');
  expect(response.status()).toBe(200);
});

Unified tests (browser + API) — uses both page and request together:

import { test, expect } from '@testrelic/playwright-analytics/fixture';

test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
  // API call
  const apiResponse = await request.get('https://api.example.com/user/1');
  const user = await apiResponse.json();

  // Browser navigation
  await page.goto('https://example.com/profile');
  await expect(page.locator('.user-name')).toHaveText(user.name);
});
4. Run your tests
npx playwright test

The JSON report will be written to ./test-results/analytics-timeline.json and the HTML report to ./test-results/analytics-timeline.html.

Troubleshooting

require() of ES Module … fixture.js

Symptom: Running npx playwright test throws an error like:

Error [ERR_REQUIRE_ESM]: require() of ES Module .../node_modules/@testrelic/playwright-analytics/dist/fixture.js not supported.

Cause: Your project's TypeScript or Playwright configuration compiles test files as CommonJS. When a CJS module calls require('@testrelic/playwright-analytics/fixture'), some toolchains mis-resolve the subpath export to the ESM build (fixture.js) instead of the CJS build (fixture.cjs).

Fix — Option A (recommended): Run tests as ESM

Add "type": "module" to your project's package.json, or configure TypeScript to emit ESM:

// package.json
{ "type": "module" }
// tsconfig.json
{
  "compilerOptions": {
    "module": "NodeNext",   // or "ESNext"
    "moduleResolution": "NodeNext"
  }
}

Fix — Option B: Use the explicit CJS subpath

Import the CJS build via the dedicated subpath export (no deep node_modules path needed):

// base-test.ts or your fixture wrapper
import { test, expect } from '@testrelic/playwright-analytics/fixture/cjs';
// api tests
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture/cjs';

This is stable across version upgrades and pnpm/npm layouts.

[testrelic] No API key configured

This message is printed when TESTRELIC_API_KEY is not set in the environment the test process sees. Local HTML/JSON reports are still written — this only affects cloud upload. Set the key in your shell or CI secrets; see Cloud Integration.

Cloud Integration

TestRelic can upload test results, action steps, artifacts (screenshots, videos), and network logs directly to the TestRelic cloud platform for team-wide visibility, historical tracking, and the interactive Steps dashboard.

Setup

Step 1: Get your API key

Sign in to platform.testrelic.ai and generate an API key from Settings → API Keys.

Step 2: Create the config file

Installing the package automatically creates .testrelic/testrelic-config.json in your project root via a postinstall script. If it wasn't created (e.g. in CI with CI=true), create it manually:

{
  "cloud": {
    "apiKey": "$TESTRELIC_API_KEY",
    "endpoint": "https://platform.testrelic.ai/api/v1",
    "upload": "both"
  },
  "testrelic-repo": {
    "name": "my-project"
  }
}

Replace my-project with the name of your repository as it should appear in the cloud dashboard. The apiKey value of "$TESTRELIC_API_KEY" is automatically resolved from the environment variable of the same name at runtime — no secrets in your config file.

Step 3: Set your API key in the environment

export TESTRELIC_API_KEY=your_api_key_here

In CI, add TESTRELIC_API_KEY as a repository secret and pass it to the test job.

Step 4: Enable cloud upload in your Playwright config

// playwright.config.ts
export default defineConfig({
  reporter: [
    ['list'],
    [
      '@testrelic/playwright-analytics',
      {
        cloud: {
          apiKey: process.env.TESTRELIC_API_KEY,
          endpoint: 'https://platform.testrelic.ai/api/v1',
          upload: 'both',
          uploadArtifacts: true,
          artifactMaxSizeMb: 50,
        },
        includeActionSteps: true,
        includeCodeSnippets: true,
        includeNetworkStats: true,
        includeArtifacts: true,
        captureResponseBody: true,
        captureRequestBody: true,
      },
    ],
  ],
  use: {
    video: 'on',
    screenshot: 'on',
    trace: 'on',
  },
});
Config File Reference

The .testrelic/testrelic-config.json file is the project-level configuration for cloud integration. It should be committed to version control.

Full schema:

{
  "cloud": {
    "apiKey": "$TESTRELIC_API_KEY",
    "endpoint": "https://platform.testrelic.ai/api/v1",
    "upload": "both",
    "timeout": 30000
  },
  "testrelic-repo": {
    "name": "my-project"
  },
  "queue": {
    "maxAge": "7d",
    "directory": ".testrelic/queue"
  }
}
Key Description
cloud.apiKey API key for authentication. Use "$TESTRELIC_API_KEY" to read from the environment variable.
cloud.endpoint Cloud API endpoint. Defaults to https://platform.testrelic.ai/api/v1.
cloud.upload Upload strategy — see Upload Strategy below.
cloud.timeout Request timeout in milliseconds (default: 30000).
testrelic-repo.name Repository/project name shown in the cloud dashboard.
queue.maxAge How long to retain queued failed uploads before discarding (default: "7d").
queue.directory Directory for queued uploads (default: .testrelic/queue).
Upload Strategy

The upload field controls how test data is sent to the cloud:

Value Behaviour
"realtime" Streams test events to the cloud as they happen. Fast feedback but skips the batch payload.
"batch" Sends a single complete JSON payload after all tests finish. Includes full step data.
"both" Recommended. Streams realtime events AND sends the complete batch payload at the end. Required for the Steps tab in the cloud dashboard to be populated.

Important: Using upload: "realtime" alone skips the batch payload that the cloud platform uses to populate the Steps, navigation timeline, and network stats views. Use "both" to ensure all data appears in the dashboard.

Environment Variables

Environment variables take the highest priority and override both reporter options and the config file:

Variable Description
TESTRELIC_API_KEY API key — always set this in CI secrets rather than hardcoding it.
TESTRELIC_CLOUD_ENDPOINT Override the API endpoint (e.g. for a self-hosted or stage deployment).
TESTRELIC_UPLOAD_STRATEGY Override the upload strategy (realtime, batch, or both).
TESTRELIC_CLOUD_TIMEOUT Override the request timeout in milliseconds.

Priority order (highest wins):

  1. Environment variables
  2. Reporter cloud: options in playwright.config.ts
  3. .testrelic/testrelic-config.json
  4. Built-in defaults
.gitignore Guidance

The .testrelic/ directory serves multiple purposes. Only the config file should be committed:

# Commit: .testrelic/testrelic-config.json
# Ignore runtime directories:
.testrelic/queue/
.testrelic/cache/
Migrating from Previous Versions

Config file location (v2.3.x → v2.4.0)

The config file has moved from a flat .testrelic file to .testrelic/testrelic-config.json. This prevents a filesystem conflict where the SDK needed to create a .testrelic/ directory (for queue and cache) while the config file occupied the same path.

If you have an existing .testrelic flat file, it still works — the SDK reads it with a deprecation warning:

[testrelic] Deprecation: config file ".testrelic" has moved to ".testrelic/testrelic-config.json".
Please migrate your config file to the new location.

To migrate, rename the file and move it into the directory:

mkdir -p .testrelic
mv .testrelic .testrelic/testrelic-config.json   # on Unix/macOS

project key renamed to testrelic-repo (v2.3.x → v2.4.0)

The project config key has been renamed to testrelic-repo for clarity. The old key still works with a deprecation warning:

[testrelic] Deprecation: "project" key in config is deprecated. Rename it to "testrelic-repo".

Update your config file:

// Before
{ "project": { "name": "my-project" } }

// After
{ "testrelic-repo": { "name": "my-project" } }

Output Example

{
  "schemaVersion": "1.0.0",
  "testRunId": "797128f5-c86d-466c-8d6d-8ec62dfc70b6",
  "startedAt": "2026-02-07T10:41:28.759Z",
  "completedAt": "2026-02-07T10:41:36.794Z",
  "totalDuration": 8035,
  "summary": {
    "total": 6,
    "passed": 5,
    "failed": 1,
    "flaky": 0,
    "skipped": 0
  },
  "ci": {
    "provider": "github-actions",
    "buildId": "12345678",
    "commitSha": "abc123def456",
    "branch": "main"
  },
  "timeline": [
    {
      "url": "https://en.wikipedia.org/wiki/Main_Page",
      "navigationType": "goto",
      "visitedAt": "2026-02-07T10:41:29.844Z",
      "duration": 216,
      "specFile": "tests/homepage.spec.ts",
      "domContentLoadedAt": "2026-02-07T10:41:30.200Z",
      "networkStats": {
        "totalRequests": 40,
        "failedRequests": 0,
        "totalBytes": 1289736,
        "byType": {
          "xhr": 0,
          "document": 1,
          "script": 9,
          "stylesheet": 2,
          "image": 27,
          "font": 2,
          "other": 0
        }
      },
      "tests": [
        {
          "title": "homepage.spec.ts > Homepage > loads correctly",
          "status": "passed",
          "duration": 1028,
          "failure": null
        }
      ]
    }
  ]
}

Configuration

All options are passed as the second element of the reporter tuple:

General Options
Option Type Default Description
outputPath string ./test-results/analytics-timeline.json Where to write the JSON report
includeStackTrace boolean false Include full stack traces in failure diagnostics
includeCodeSnippets boolean true Include source code snippets around the failure line
codeContextLines number 3 Lines of context above/below the failure line
includeNetworkStats boolean true Track network requests and bytes per navigation
includeArtifacts boolean true Include screenshots and video in the report
includeActionSteps boolean true Capture Playwright test steps (actions, assertions, custom steps) with video sync
captureConsoleLogs boolean true Capture browser console messages (log, warn, error) per test
navigationTypes NavigationType[] | null null (all) Filter timeline to specific navigation types
redactPatterns (string | RegExp)[] Built-in patterns Additional patterns to redact from error output
testRunId string | null null (auto UUID) Override the test run ID
metadata Record<string, unknown> | null null Attach custom metadata to the report
openReport boolean true Auto-open the HTML report after test run
quiet boolean false Suppress TestRelic banner output
API Tracking Options
Option Type Default Description
trackApiCalls boolean true Enable/disable API call tracking entirely
captureRequestHeaders boolean true Capture request headers for each API call
captureResponseHeaders boolean true Capture response headers for each API call
captureRequestBody boolean true Capture request body/payload for each API call
captureResponseBody boolean true Capture response body for each API call
captureAssertions boolean true Track assertion results (pass/fail with expected/actual values)
redactHeaders string[] ['authorization', 'cookie', 'set-cookie', 'x-api-key'] Header names to redact (case-insensitive)
redactBodyFields string[] ['password', 'secret', 'token', 'apiKey', 'api_key'] Body field names whose values are replaced with [REDACTED]
apiIncludeUrls (string | RegExp)[] [] (all URLs) Only track API calls matching these patterns
apiExcludeUrls (string | RegExp)[] [] (none excluded) Exclude API calls matching these patterns
Cloud Upload Options

Passed inside the cloud: key of the reporter options. These are merged with (and take priority over) the .testrelic/testrelic-config.json values.

Option Type Default Description
cloud.apiKey string API key for cloud authentication. Prefer process.env.TESTRELIC_API_KEY.
cloud.endpoint string https://platform.testrelic.ai/api/v1 Cloud API endpoint.
cloud.upload 'realtime' | 'batch' | 'both' 'batch' Upload strategy. Use 'both' to populate all cloud dashboard views including Steps.
cloud.timeout number 30000 Request timeout in milliseconds.
cloud.uploadArtifacts boolean true Upload screenshots and videos to cloud storage.
cloud.artifactMaxSizeMb number 50 Maximum size per artifact file in MB. Files exceeding this are skipped.
cloud.queueDirectory string .testrelic/queue Directory for queued uploads that failed and will be retried.
cloud.queueMaxAge string '7d' Maximum age for queued items before they are discarded (e.g. '7d', '24h').
Configuration Examples

Default (zero config) — captures everything with built-in redaction:

reporter: [
  ['list'],
  ['@testrelic/playwright-analytics', {
    outputPath: './test-results/analytics-timeline.json',
  }],
]

Minimal capture (no bodies) — reduces report size, tracks only method/URL/status/timing:

reporter: [
  ['list'],
  ['@testrelic/playwright-analytics', {
    outputPath: './test-results/analytics-timeline.json',
    captureRequestBody: false,
    captureResponseBody: false,
  }],
]

URL filtering — only track specific endpoints:

reporter: [
  ['list'],
  ['@testrelic/playwright-analytics', {
    outputPath: './test-results/analytics-timeline.json',
    apiIncludeUrls: ['**/api/**'],
    apiExcludeUrls: ['**/health', '**/metrics'],
  }],
]

Custom redaction — add your own sensitive header/body field names:

reporter: [
  ['list'],
  ['@testrelic/playwright-analytics', {
    outputPath: './test-results/analytics-timeline.json',
    redactHeaders: ['authorization', 'cookie', 'set-cookie', 'x-api-key', 'x-custom-secret'],
    redactBodyFields: ['password', 'secret', 'token', 'ssn', 'creditCard'],
  }],
]

Disable API tracking — browser-only report with no API call data:

reporter: [
  ['list'],
  ['@testrelic/playwright-analytics', {
    outputPath: './test-results/analytics-timeline.json',
    trackApiCalls: false,
  }],
]

Screenshots & Videos

When using defineConfig from @testrelic/playwright-analytics, video recording and screenshot capture are enabled by default (video: 'on', screenshot: 'on'). If using manual config, enable them in Playwright's use section:

export default defineConfig({
  use: {
    video: 'on',
    screenshot: 'on',
  },
  reporter: [['list'], ['@testrelic/playwright-analytics']],
});

Artifacts are automatically copied to an artifacts/ folder next to your report:

test-results/
├── analytics-timeline.json
├── analytics-timeline.html
└── artifacts/
    ├── homepage-loads-correctly/
    │   ├── screenshot.png
    │   └── video.webm
    └── search-works--retry-1/
        ├── screenshot.png
        └── video.webm
  • Screenshots are captured at the end of each test (.png)
  • Videos record the full browser session (.webm)
  • Retried tests get separate folders with a --retry-N suffix
  • The HTML report embeds links to these artifacts with inline image/video viewers
  • Disable artifact collection with includeArtifacts: false

Action Steps

TestRelic captures Playwright test steps — actions (click, fill, goto), assertions (expect), and custom steps (test.step()). Steps are timestamped and synced with the video timeline so you can jump to the exact moment in the recording.

Enable/disable with the includeActionSteps option (default: true):

['@testrelic/playwright-analytics', {
  includeActionSteps: true,
}]

In the HTML report, action steps appear in the test drawer alongside the navigation timeline, showing:

  • Step category (action, assertion, or custom)
  • Step title and duration
  • Video timestamp for playback sync

Console Log Capture

TestRelic captures browser console messages (log, warn, error) per test. Useful for debugging test failures without re-running.

Enable/disable with the captureConsoleLogs option (default: true):

['@testrelic/playwright-analytics', {
  captureConsoleLogs: true,
}]

Network Statistics

Each navigation in the timeline includes detailed network statistics:

  • Total requests — number of network requests during the navigation
  • Failed requests — count and URLs of failed requests (useful for spotting broken assets)
  • Bytes transferred — total bytes received
  • Resource breakdown — requests grouped by type: xhr, document, script, stylesheet, image, font, other

Disable with includeNetworkStats: false.

Viewing Reports

For small test suites (< 50 tests), the report is a self-contained HTML file — just open it in your browser.

For large test suites (50+ tests), TestRelic uses streaming mode to avoid memory issues. Test details are stored as separate files and loaded on demand via a local server. After tests complete, the terminal shows:

[testrelic] View report: npx testrelic serve ./test-results/.testrelic-report

Run that command and open http://127.0.0.1:9323 in your browser. The server auto-starts after test runs when not in CI, but if you need to restart it later:

npx testrelic serve ./test-results/.testrelic-report
npx testrelic serve ./test-results/.testrelic-report --port 9400  # custom port

Merging Shard Reports

When running Playwright with sharding, each shard produces its own report. Merge them with the CLI:

npx testrelic merge shard-1.json shard-2.json shard-3.json -o merged-report.json

Or programmatically:

import { mergeReports } from '@testrelic/playwright-analytics/merge';

await mergeReports(
  ['shard-1.json', 'shard-2.json'],
  { output: 'merged-report.json' }
);

Testing Modes

TestRelic supports three testing modes, each with its own fixture:

E2E Testing (Browser)

Use the unified fixture which provides page with navigation tracking — page load timing, DOM content loaded, network idle detection, and network request statistics.

import { test, expect } from '@testrelic/playwright-analytics/fixture';

test('search works', { tag: ['@e2e'] }, async ({ page }) => {
  await page.goto('https://example.com');
  await page.fill('#search', 'query');
  await page.click('button[type=submit]');
});
API Testing

Use the API-only fixture which provides request without any browser dependency — tracks HTTP method, URL, status code, headers, bodies, response times, and assertions.

import { test as base } from '@playwright/test';
import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
import { expect } from '@testrelic/playwright-analytics/fixture';

const test = base.extend(testRelicApiFixture);

test('CRUD operations', { tag: ['@api'] }, async ({ request }) => {
  const response = await request.post('https://api.example.com/posts', {
    data: { title: 'New Post', body: 'Content' },
  });
  expect(response.status()).toBe(201);
});
Custom Page Fixtures (authenticated pages, etc.)

If your tests create pages manually via browser.newContext() / context.newPage() (e.g., for auth with storageState), the built-in page fixture is bypassed and no network/console/navigation data is captured. Use trackPage() to enable tracking on any manually-created page:

import { test as base, expect, trackPage } from '@testrelic/playwright-analytics/fixture';

type MyFixtures = {
  authenticatedPage: Page;
};

export const test = base.extend<MyFixtures>({
  authenticatedPage: async ({ browser }, use, testInfo) => {
    const ctx = await browser.newContext({ storageState: 'auth.json' });
    const page = await ctx.newPage();
    const cleanup = await trackPage(page, testInfo); // ← enables tracking
    await use(page);
    await cleanup(); // ← flushes data to report
    await ctx.close();
  },
});

Important: Without trackPage(), custom page fixtures will only show screenshots, videos, and action steps — but no network requests, console logs, or navigation timeline.

Unified Testing (Browser + API)

Use the unified fixture which provides both page and request — the TestRelic report shows navigation timeline AND API call details for the same test. This is ideal for:

  • Contract testing: Verify API data matches what the browser renders
  • Setup/teardown via API: Use API calls to set up test data, then verify in the browser
  • Full-stack workflows: Test end-to-end flows that span both UI and API layers
import { test, expect } from '@testrelic/playwright-analytics/fixture';

test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
  // Fetch data via API
  const apiResponse = await request.get('https://api.example.com/user/1');
  const user = await apiResponse.json();

  // Verify it renders correctly in the browser
  await page.goto('https://example.com/profile');
  await expect(page.locator('.user-name')).toHaveText(user.name);
});

Examples

Full working examples for E2E, API, and unified testing are available in the testrelic-sdk-examples repository:

Example Type What It Demonstrates
api-testing API only CRUD operations, API chaining, response assertions, config options, error handling
unified-testing E2E + API Browser navigation and API calls in the same test
wikipedia E2E Homepage, search, link navigation, media-heavy pages
flipkart E2E Homepage, product search, product pages
google E2E Search queries, results navigation

Each example is a standalone project — clone and run:

git clone https://github.com/testrelic-ai/testrelic-sdk-examples.git
cd testrelic-sdk-examples/api-testing
npm install
npx playwright test

Prerequisites

  • Node.js >= 18
  • Playwright >= 1.35.0

License

MIT

Keywords