StageWright: The Intelligent Playwright Reporter

As Playwright usage expands across teams, environments, and CI pipelines, reporting needs naturally become more sophisticated. StageWright is designed to meet that need by turning standard Playwright results into a more structured and actionable reporting experience. Instead of focusing only on individual test outcomes, StageWright helps QA teams and engineering stakeholders understand broader patterns such as stability, retries, performance changes, and historical trends. This added visibility makes it easier to review test results, share insights, and support better release decisions.

While Playwright’s built-in HTML reporter is useful for quick inspection, StageWright extends reporting with capabilities that are better suited to growing test suites and collaborative QA workflows. This blog explores how StageWright adds structure, clarity, and actionable insight to Playwright reporting for growing QA teams.

What Is StageWright?

StageWright is an intelligent reporting layer for Playwright Test. You install it as a dev dependency and add a single entry to your playwright.config.ts, and run your tests as usual. However, instead of the default output, you get a polished, single-file HTML report that you can open in any browser, share with your team, or upload to a CI artifact store.

What makes StageWright “smart” is what happens beyond the basic pass/fail summary.

• Stability Grades: Every test gets an A–F grade based on historical pass rate, retry frequency, and duration variance.
• Retry & Flakiness Analysis: Automatically detects and flags tests that only pass after retries.
• Run Comparison: Compares the current run against a baseline, helping identify regressions instantly.
• Trend Analytics: Tracks pass rates, durations, and flakiness across builds.
• Artifact Gallery: Centralizes screenshots, videos, and trace files.
• AI Failure Analysis: Available in paid tiers for clustering failures by root cause.

StageWright is compatible with Playwright Test v1.40 and above and runs on Node.js version 18 or higher.

Getting Started with StageWright

The setup process for StageWright is designed to be simple and efficient. In just a few steps, you can move from basic test output to a fully interactive report.

Step 1: Install the package

npm install playwright-smart-reporter --save-dev

Step 2: Add it to your Playwright config

Open playwright.config.ts and add StageWright to the reporters array. Importantly, it works alongside existing reporters rather than replacing them.

import { defineConfig } from '@playwright/test';

export default defineConfig({
 reporter: [
   ['list'],
   ['playwright-smart-reporter', {
     outputFile: 'smart-report.html',
     title: 'My Test Suite',
   }],
 ],
});

Step 3: Run your tests

npx playwright test

Then open the report:

open smart-report.html

At this point, you’ll have a fully self-contained HTML report. Since no server or build step is required, you can easily share it across your team or attach it to CI artifacts.

Pro Tip:

Although the default output is smart-report.html, it’s recommended to store reports in a dedicated folder, such as test-results/report.html for better organization.

Configuration Reference: Why It Matters More Than You Think

Once you have a basic report working, configuration becomes essential. In fact, this is where StageWright starts delivering its full value.

Core options you’ll use most

• HistoryFile: Stores run history and enables trend analytics, run comparison, and stability grading. Without it, you lose historical visibility.
• MaxHistoryRuns: Controls how many runs are stored. Typically, 50–100 works well.
• EnableRetryAnalysis: Tracks retries and identifies flaky tests.
• FilterPwApiSteps: Removes unnecessary noise from reports, improving readability.
• PerformanceThreshold: Flags tests with performance regression.
• EnableNetworkLogs: Captures network activity when needed for debugging.

Environment variables

In addition to config options, StageWright supports environment variables, which are particularly useful in CI environments.

• SMART_REPORTER_LICENSE_KEY: Enables paid features
• STAGEWRIGHT_TITLE / STAGEWRIGHT_OUTPUT: Customize reports dynamically
• CI: Enables CI-optimized behavior automatically

CI Behavior:

When running in CI, StageWright reduces report size, disables interactive hints, and injects build metadata such as commit SHA and branch details.

Stability Grades: A Report Card for Your Test Suite

One of the most valuable features of StageWright is its Stability Grades system. Instead of treating all tests equally, it evaluates them based on reliability over time.

Each test is graded using:

• Pass rate
• Retry rate
• Duration variance

This is calculated using the following formula:

Grade Score = (passRate × 0.6)
           + (1 - retryRate) × 0.25
           + (1 - durationVariance) × 0.15

Because the pass rate has the highest weight, it strongly influences the final score. However, retries and performance variability also contribute to a more realistic assessment.

As a result, teams can quickly identify unstable tests and prioritize fixes effectively.

Run Comparison: Catch Regressions Before They Reach Production

Another key feature of StageWright is Run Comparison. Instead of manually comparing results, it automatically highlights differences between runs.

Tests are categorized as follows:

• New Failure
• Regression
• Fixed
• New Test
• Removed
• Stable Pass / Stable Fail

Additionally, performance changes are tracked, making it easier to detect slowdowns.

Because of this, debugging becomes faster and more focused.

Retry Analysis: Flakiness, Measured

Retries can sometimes create a false sense of stability. However, StageWright ensures that these hidden issues are visible.

A test that fails initially but passes on retry is marked as flaky. While it may not fail the build, it is still flagged for attention.

The report also highlights the following:

• Total retries
• Flaky test percentage
• Time spent on retries
• Most retried tests

Over time, this helps teams reduce flakiness and improve overall reliability.

Trend Analytics: The Long View on Suite Health

While individual runs provide immediate feedback, trend analytics offer long-term insights.

StageWright tracks:

• Pass rate trends
• Duration trends
• Flakiness trends

Moreover, it detects degradation automatically, helping teams identify issues early.

As a result, teams can move from reactive debugging to proactive improvement.

CI Integration: Built for Real Pipelines

StageWright integrates seamlessly with modern CI platforms such as GitHub Actions, GitLab CI, Jenkins, and CircleCI.

Importantly, no additional plugins are required. Instead, it runs as part of your existing workflow.

To maximize its value:

• Always upload reports (even on failure)
• Cache history files
• Maintain report retention

This ensures consistency and visibility across builds.

Annotations: Metadata That Shows Up in Your Reports

StageWright supports Playwright annotations, allowing teams to add metadata directly to tests.

test.info().annotations.push(
 { type: 'priority', description: 'P1' },
 { type: 'team', description: 'payments' }
);

This makes it easier to filter tests by priority, ownership, or related tickets. Consequently, debugging and triaging become more efficient.

Starter Features: What’s Behind the License Key

StageWright also offers advanced capabilities through its Starter and Pro plans.

These include:

• AI failure clustering
• Quality gates
• Flaky test quarantine
• Export formats
• Notifications
• Custom branding
• Live execution view
• Accessibility scanning

Importantly, these features integrate seamlessly without requiring separate configurations.

Conclusion: Why StageWright Matters

Ultimately, QA automation is only as effective as your ability to understand test results. StageWright transforms Playwright reporting into a structured, insight-driven process. Instead of relying on logs and guesswork, teams gain clear visibility into test stability, performance, and trends. As a result, teams can prioritize effectively, reduce flakiness, and improve release confidence.

Frequently Asked Questions