@checkly/playwright-reporter

@checkly/playwright-reporter

Official Playwright reporter for Checkly. Automatically upload test results, screenshots, videos, and traces to gain visibility into your end-to-end tests.

Fully compatible with Playwright's native JSON reporter - use it as a drop-in replacement that adds Checkly integration.

Installation

npm install --save-dev @checkly/playwright-reporter

Requirements:

• Node.js >= 18.0.0
• Playwright >= 1.40.0
• A Checkly account

Quick Start

1. Get Your Credentials

1. Go to Account Settings > General to find your Account ID
2. Go to User Settings > API Keys to create an API key

2. Configure Playwright

Add the reporter to your playwright.config.ts:

import { defineConfig } from '@playwright/test';
import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    ['list'],
    createChecklyReporter(),
  ],
});

Tip: Using createChecklyReporter() provides full intellisense for configuration options. Alternatively, use the array syntax: ['@checkly/playwright-reporter', { ... }]

3. Set Environment Variables

export CHECKLY_API_KEY=your_api_key
export CHECKLY_ACCOUNT_ID=your_account_id

Or pass them inline:

CHECKLY_API_KEY=... CHECKLY_ACCOUNT_ID=... npx playwright test

4. Run Tests

npx playwright test

You'll see a session URL in the output:

View test results: https://app.checklyhq.com/test-sessions/abc123

[...test execution...]

✓ Check the results: https://app.checklyhq.com/test-sessions/abc123

Configuration Options

import { createChecklyReporter } from '@checkly/playwright-reporter';

createChecklyReporter({
  outputDir: 'test-results',
  sessionName: 'My Test Suite',
  dryRun: false,
  verbose: false,
  showProgress: true,
  showSummaryTable: true,
})

Option	Type	Default	Description
apiKey	string	CHECKLY_API_KEY env var	Checkly API key
accountId	string	CHECKLY_ACCOUNT_ID env var	Checkly account ID
outputDir	string	Playwright's outputDir	Directory for assets, JSON, and ZIP
sessionName	string | function	Auto-generated	Custom session name
dryRun	boolean	false	Create ZIP without uploading
verbose	boolean	false	Enable debug logging
showProgress	boolean	true	Show real-time test progress in the terminal
printSteps	boolean	false	Print individual test steps (requires showProgress)
showSummaryTable	boolean	true	Show per-project breakdown table after the run
scrubbing	object | false	See Secret Scrubbing	Configure secret scrubbing for reports and traces
testCommand	string	Auto-detected	Override the test command shown in the Checkly UI

Output files (written to outputDir):

• checkly-report.json - JSON test report
• checkly-report.zip - ZIP archive with report and assets

Environment Variables

Variable	Description
CHECKLY_API_KEY	Your Checkly API key
CHECKLY_ACCOUNT_ID	Your Checkly account ID
CHECKLY_REPORTER_VERBOSE	Set to true for detailed debug output
CHECKLY_REPORTER_DRY_RUN	Set to true to skip uploading
CHECKLY_REPORTER_OUTPUT_DIR	Override the output directory
CHECKLY_REPORTER_SHOW_SUMMARY_TABLE	Set to false to hide the summary table
CHECKLY_TEST_COMMAND	Override the test command shown in the Checkly UI

Environment variables take precedence over options passed in the config.

What Gets Uploaded

• Test results and status (passed/failed/flaky)
• Execution duration and timing
• Screenshots (on failure or explicit capture)
• Video recordings
• Playwright traces
• Console logs and network requests (extracted from traces)

Secret Scrubbing

The reporter automatically scrubs sensitive values from JSON reports, trace files, and logs before upload.

By default, any environment variable whose name starts with CHECKLY_SECRET_ is scrubbed. You can configure additional scrubbing strategies:

createChecklyReporter({
  scrubbing: {
    envVars: ['DB_PASSWORD', 'STRIPE_KEY'],
    autoDetect: true,
    prefix: 'CHECKLY_SECRET_',
    replacement: '[REDACTED]',
  },
})

Option	Type	Default	Description
envVars	string[]	[]	Environment variable names whose runtime values should be scrubbed
autoDetect	boolean	false	Scrub env vars matching common secret patterns (SECRET, KEY, TOKEN, PASSWORD, CREDENTIAL, AUTH, PRIVATE, API)
prefix	string | false | null	'CHECKLY_SECRET_'	Scrub all env vars starting with this prefix. Set to false or '' to disable
replacement	string	'*********'	Custom replacement string for scrubbed values

Set scrubbing: false to disable scrubbing entirely.

Flaky Test Detection

The reporter automatically detects flaky tests:

• Flaky test: A test that failed initially but passed after retry
• Degraded status: Set when there are flaky tests but no complete failures
• Overall status: passed if all tests eventually pass, failed otherwise

Running Locally (No Upload)

For local development without uploading to Checkly:

# Simply don't set CHECKLY_API_KEY or CHECKLY_ACCOUNT_ID
npx playwright test

What happens:

• Reporter creates checkly-report.zip locally
• Upload is automatically skipped
• Tests run normally
• You can inspect the ZIP file for debugging

Dry Run Mode

Create local JSON and ZIP files without uploading:

createChecklyReporter({
  dryRun: true,
})

Conditional dry run (skip uploads when no credentials):

createChecklyReporter({
  dryRun: !process.env.CHECKLY_API_KEY,
})

CI/CD Integration

GitHub Actions

name: Playwright Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        env:
          CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
          CHECKLY_ACCOUNT_ID: ${{ secrets.CHECKLY_ACCOUNT_ID }}
        run: npx playwright test

GitLab CI

test:
  image: mcr.microsoft.com/playwright:v1.57.0-jammy
  stage: test
  script:
    - npm ci
    - npx playwright test
  variables:
    CHECKLY_API_KEY: $CHECKLY_API_KEY
    CHECKLY_ACCOUNT_ID: $CHECKLY_ACCOUNT_ID

Setting up variables:

1. Go to your GitLab project
2. Navigate to Settings > CI/CD > Variables
3. Add CHECKLY_API_KEY (check "Mask variable")
4. Add CHECKLY_ACCOUNT_ID (check "Mask variable")

Multiple Reporters

Combine with other Playwright reporters:

import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    createChecklyReporter(),
    ['html', { outputFolder: 'playwright-report' }],
    ['list'],
    ['junit', { outputFile: 'test-results/junit.xml' }],
  ],
});

Understanding Test Sessions

The reporter creates suite-level test sessions, not individual test file results.

When you run npx playwright test:

1. One test session is created, named after your directory
2. One test result for the entire run
3. All assets uploaded together in a single ZIP file

Benefits:

• Consolidated view of your entire test suite
• Efficient storage
• Track overall pass/fail rates over time

Security

Always use environment variables for credentials. Never commit API keys to version control.

// DON'T DO THIS
createChecklyReporter({
  apiKey: 'chk_api_...',  // Hardcoded credentials!
})

// DO THIS
createChecklyReporter()  // Reads from environment

Use the Secret Scrubbing feature to ensure sensitive values don't leak into uploaded reports and traces.

Documentation

For detailed documentation, visit checklyhq.com/docs/detect/testing/playwright-reporter.

License

Apache 2.0