Blog

Playwright Report Portal Integration Guide

by Rajesh K | Apr 24, 2025 | Automation Testing, Blog, Featured, Latest Post | 1 comment

Test automation frameworks like Playwright have revolutionized automation testing for browser-based applications with their speed,, reliability, and cross-browser support. However, while Playwright excels at test execution, its default reporting capabilities can leave teams wanting more when it comes to actionable insights and collaboration. Enter ReportPortal, a powerful, open-source test reporting platform designed to transform raw test data into meaningful, real-time analytics. This guide dives deep into Playwright Report Portal Integration, offering a step-by-step approach to setting up smart test reporting. Whether you’re a QA engineer, developer, or DevOps professional, this integration will empower your team to monitor test results effectively, collaborate seamlessly, and make data-driven decisions. Let’s explore why Playwright Report Portal Integration is a game-changer and how you can implement it from scratch.

What is ReportPortal?

ReportPortal is an open-source, centralized reporting platform that enhances test automation by providing real-time, interactive, and collaborative test result analysis. Unlike traditional reporting tools that generate static logs or CI pipeline artifacts, ReportPortal aggregates test data from multiple runs, frameworks, and environments, presenting it in a user-friendly dashboard. It supports Playwright Report Portal Integration along with other popular test frameworks like Selenium, Cypress, and more, as well as CI/CD tools like Jenkins, GitHub Actions, and GitLab CI.

Key Features of ReportPortal:

• Real-Time Reporting: View test results as they execute, with live updates on pass/fail statuses, durations, and errors.
• Historical Trend Analysis: Track test performance over time to identify flaky tests or recurring issues.
• Collaboration Tools: Share test reports with team members, add comments, and assign issues for resolution.
• Custom Attributes and Filters: Tag tests with metadata (e.g., environment, feature, or priority) for advanced filtering and analysis.
• Integration Capabilities: Seamlessly connects with CI pipelines, issue trackers (e.g., Jira), and test automation frameworks.
• AI-Powered Insights: Leverage defect pattern analysis to categorize failures (e.g., product bugs, automation issues, or system errors).

ReportPortal is particularly valuable for distributed teams or projects with complex test suites, as it centralizes reporting and reduces the time spent deciphering raw test logs.

Why Choose ReportPortal for Playwright?

Playwright is renowned for its robust API, cross-browser compatibility, and built-in features like auto-waiting and parallel execution. However, its default reporters (e.g., list, JSON, or HTML) are limited to basic console outputs or static files, which can be cumbersome for large teams or long-running test suites. ReportPortal addresses these limitations by offering:

Benefits of Using ReportPortal with Playwright:

• Enhanced Visibility: Real-time dashboards provide a clear overview of test execution, including pass/fail ratios, execution times, and failure details.
• Collaboration and Accountability: Team members can comment on test results, assign defects, and link issues to bug trackers, fostering better communication.
• Trend Analysis: Identify patterns in test failures (e.g., flaky tests or environment-specific issues) to improve test reliability.
• Customizable Reporting: Use attributes and filters to slice and dice test data based on project needs (e.g., by browser, environment, or feature).
• CI/CD Integration: Integrate with CI pipelines to automatically publish test results, making it easier to monitor quality in continuous delivery workflows.
• Multimedia Support: Attach screenshots, videos, and logs to test results for easier debugging, especially for failed tests.

By combining Playwright’s execution power with ReportPortal’s intelligent reporting, teams can streamline their QA processes, reduce debugging time, and deliver higher-quality software.

Step-by-Step Guide: Playwright Report Portal Integration Made Easy

Let’s walk through the process of setting up Playwright with ReportPortal to create a seamless test reporting pipeline.

Prerequisites

Before starting, ensure you have:

• Node.js and npm installed (version 14 or higher recommended).
• A Playwright project set up. If you don’t have one, initialize it with:
npm init playwright@latest

• Access to a ReportPortal instance. You can:
• Use the demo instance at https://demo.reportportal.io for testing.
• Set up a local instance using Docker (refer to ReportPortal’s official documentation).
• Use a hosted instance if your organization provides one.
• A personal API token from ReportPortal (more on this below).

Step 1: Install Dependencies

In your Playwright project directory, install the necessary packages:

npm install -D @playwright/test @reportportal/agent-js-playwright

• @playwright/test: The official Playwright test runner.
• @reportportal/agent-js-playwright: The ReportPortal agent for Playwright integration.

Step 2: Configure Playwright with ReportPortal

Modify your playwright.config.js file to include the ReportPortal reporter. Here’s a sample configuration:

// playwright.config.js
const config = {
  testDir: './tests',
  reporter: [
    ['list'], // Optional: Displays test results in the console
    [
      '@reportportal/agent-js-playwright',
      {
        apiKey: 'your_reportportal_api_key', // Replace with your ReportPortal API key
        endpoint: 'https://demo.reportportal.io/api/v1', // ReportPortal instance URL (must include /api/v1)
        project: 'your_project_name', // Case-sensitive project name in ReportPortal
        launch: 'Playwright Launch - ReportPortal', // Name of the test launch
        description: 'Sample Playwright + ReportPortal integration',
        attributes: [
          { key: 'framework', value: 'playwright' },
          { key: 'env', value: 'dev' },
        ],
        debug: false, // Set to true for troubleshooting
      },
    ],
  ],
  use: {
    browserName: 'chromium', // Default browser
    headless: true, // Run tests in headless mode
    screenshot: 'on', // Capture screenshots for all tests
    video: 'retain-on-failure', // Record videos for failed tests
  },
};

module.exports = config;

How to Find Your ReportPortal API Key

1. Log in to your ReportPortal instance.

2. Click your user avatar in the top-right corner and select Profile.

3. Scroll to the API Keys section and generate a new key.

4. Copy the key and paste it into the apiKey field in the config above.

Note: The endpoint URL must include /api/v1. For example, if your ReportPortal instance is hosted at https://your-rp-instance.com, the endpoint should be https://your-rp-instance.com/api/v1.

Step 3: Write a Sample Test

Create a test file at tests/sample.spec.js to verify the integration. Here’s an example:

// tests/sample.spec.js
const { test, expect } = require('@playwright/test');

test('Google search works', async ({ page }) => {
  await page.goto('https://www.google.com');
  await page.locator('input[name="q"]').fill('Playwright automation');
  await page.keyboard.press('Enter');
  await expect(page).toHaveTitle(/Playwright/i);
});

This test navigates to Google, searches for “Playwright automation,” and verifies that the page title contains “Playwright.”

Step 4: Run the Tests

Execute your tests using the Playwright CLI:

npx playwright test

During execution, the ReportPortal agent will send test results to your ReportPortal instance in real time. Once the tests complete:

1. Log in to your ReportPortal instance.

2. Navigate to the project dashboard and locate the launch named Playwright Launch – ReportPortal.

3. Open the launch to view detailed test results, including:

• Test statuses (pass/fail).
• Execution times.
• Screenshots and videos (if enabled).
• Logs and error messages.
• custom attributes (e.g., framework: playwright, env: dev).

Step 5: Explore ReportPortal’s Features

With your tests running, take advantage of ReportPortal’s advanced features:

• Filter Results: Use attributes to filter tests by browser, environment, or other metadata.
• Analyze Trends: View historical test runs to identify flaky tests or recurring failures.
• Collaborate: Add comments to test results or assign defects to team members.
• Integrate with CI/CD: Configure your CI pipeline (e.g., Jenkins or GitHub Actions) to automatically publish test results to ReportPortal.

Troubleshooting Tips for Playwright Report Portal Integration

Tests not appearing in ReportPortal?

• Verify your apiKey and endpoint in playwright.config.js.
• Ensure the project name matches exactly with your ReportPortal project.
• Enable debug: true in the reporter config to log detailed output.

Screenshots or videos missing?

• Confirm that screenshot: ‘on’ and video: ‘retain-on-failure’ are set in the use section of playwright.config.js.

Connection errors?

• Check your network connectivity and the ReportPortal instance’s availability.
• If using a self-hosted instance, ensure the server is running and accessible.

Alternatives to ReportPortal

While ReportPortal is a robust choice, other tools can serve as alternatives depending on your team’s needs. Here are a few notable options:

Allure Report:

• Overview: An open-source reporting framework that generates visually appealing, static HTML reports.
• Pros: Easy to set up, supports multiple frameworks (including Playwright), and offers detailed step-by-step reports.
• Cons: Lacks real-time reporting and collaboration features. Reports are generated post-execution, making it less suitable for live monitoring.
• Best For: Teams looking for a lightweight, offline reporting solution.

TestRail:

• Overview: A test management platform with reporting and integration capabilities for automation frameworks.
• Pros: Comprehensive test case management, reporting, and integration with CI tools.
• Cons: Primarily a paid tool, with limited real-time reporting compared to ReportPortal.
• Best For: Teams needing a full-fledged test management system alongside reporting.

Zephyr Scale:

• Overview: A Jira-integrated test management and reporting tool for manual and automated tests.
• Pros: Tight integration with Jira, robust reporting, and support for automation results.
• Cons: Requires a paid license and may feel complex for smaller teams focused solely on reporting.
• Best For: Enterprises already using Jira for project management.

Custom Dashboards (e.g., Grafana or Kibana):

• Overview: Build custom reporting dashboards using observability tools like Grafana or Kibana, integrated with test automation results.
• Pros: Highly customizable and scalable for advanced use cases.
• Cons: Requires significant setup and maintenance effort, including data ingestion pipelines.
• Best For: Teams with strong DevOps expertise and custom reporting needs.

While these alternatives have their strengths, ReportPortal stands out for its real-time capabilities, collaboration features, and ease of integration with Playwright, making it an excellent choice for teams prioritizing live test monitoring and analytics.

Conclusion

Integrating Playwright with ReportPortal unlocks a new level of efficiency and collaboration in test automation. By combining Playwright’s robust testing capabilities with ReportPortal’s real-time reporting, trend analysis, and team collaboration features, you can streamline your QA process, reduce debugging time, and ensure higher-quality software releases. This setup is particularly valuable for distributed teams, large-scale projects, or organizations adopting CI/CD practices. Whether you’re just starting with test automation or looking to enhance your existing Playwright setup, ReportPortal offers a scalable, user-friendly solution to make your test results actionable. Follow the steps outlined in this guide to get started, and explore ReportPortal’s advanced features to tailor reporting to your team’s needs.

Ready to take your test reporting to the next level? Set up Playwright with ReportPortal today and experience the power of smart test analytics!

Frequently Asked Questions

Streamlining Automated Testing with Github Actions

by Arthur Williams | Oct 21, 2024 | Automation Testing, Blog, Featured, Latest Post | 0 comments

Automated testing plays a big role in software development today. GitHub Actions is a useful tool for continuous integration (CI). When developers use GitHub Actions for automated testing, it makes their testing processes easier. This leads to better code quality and helps speed up deployment.

Key Highlights

• Learn how to automate your testing processes with GitHub Actions. This will make your software development quicker and better.
• We will help you set up your first workflow. You will also learn key ideas and how to use advanced features.
• This complete guide is great for beginners and for people who want to enhance their test automation with GitHub Actions.
• You can see practical examples, get help with issues, and find the best ways to work. This will help you improve your testing workflow.
• Discover how simple it is to connect with test management tools. This can really boost your team’s testing and reporting skills.

Understanding GitHub Actions and Automated Testing

In software development, testing is very important. Test automation helps developers test their code fast and accurately. When you use test automation with good CI/CD tools, like GitHub Actions, it improves the development process a lot.
GitHub Actions helps teams work automatically. This includes test automation. You can begin automated tests when certain events happen. For example, tests can run when someone pushes code or makes a pull request. This ensures that every change is checked carefully.

The Importance of Automation in Software Development

Software development should happen quickly. This is why automation is so important. Testing everything by hand each time there is a change takes a long time. It can also lead to mistakes.
Test automation solves this issue by running test cases without help. This allows developers to focus on other important tasks. They can spend time adding new features or fixing bugs.
GitHub Actions is a powerful tool. It helps you to automate your testing processes. It works nicely with your GitHub repository. You can run automated tests each time you push changes to the code.

Overview of GitHub Actions as a CI/CD Tool

GitHub Actions is a strong tool for CI and CD. It connects well with GitHub. You can design custom workflows. These workflows are groups of steps that happen automatically when certain events take place.
In continuous integration, GitHub Actions is very helpful for improving test execution. It allows you to automate the steps of building, testing, and deploying your projects. When you make a change in the code and push it to your new repository’s main branch, it can kick off a workflow that will, by default, run tests, including any related to Pull Requests (PR), build your application, and deploy it either to a staging area or to production.
This automation makes sure your code is always checked and added. It helps to lower the chances of problems. This also makes the development process easier.

Preparing for Automated Testing with GitHub Actions

Before you start making your automated testing workflow, let’s make sure you have everything ready. This will help your setup run smoothly and be successful.
You need a GitHub account. You also need a repository for your code. It helps to know some basic Git commands while you go through this process.

What You Need to Get Started: Accounts and Tools

If you don’t have a repository, start by making a new one in your GitHub account. This repository will be the main place for your code, tests, and workflow setups.
Next, choose a test automation framework that suits your project’s technology. Some popular choices are Jest for JavaScript, pytest for Python, and JUnit for Java. Each option has a unique way of writing tests.
Make sure your project has the right dependencies. If you use npm as your package manager, run npm ci. This command will install all the necessary packages from your package.json file.

Configuring Your GitHub Repository for Actions

With your repository ready, click on the “Actions” tab. Here, you can manage and set up your workflows. You will organize the automated tasks right here.
GitHub Actions searches for files that organize workflows in your repository. You can locate these files in the .github/workflows directory. They use YAML format. This format explains how to carry out the steps and gives instructions for your automated tasks.
When you create a new YAML file in this directory, you add a new workflow to your repository. This workflow begins when certain events happen. These events might be code pushes or pull requests.

Creating Workflow on GitHub Actions

Pre-Requisites:

• Push the “Postman” collection and “Environment” file in repository.
• Install “Newman” in your system.

Create a new workflow:

• Open your GitHub repository.
• Click on the “Actions” tab on the top.
• Click on “New workflow” in the actions page.
• Click on “Configure” button within “Simple Workflow” in “New workflow” page.
• You can navigate to the “.github/workflow” directory , where we can configure the default “blank.yml” file.
• Based on the requirements we can configure the “.yml” file, for example if you want to triggers a particular branch whenever the deployment is done, we need to “configure” the branch name in the “.yml” file.
• We can configure the workflow to be triggered based on specific events, such as whenever a push or pull request occurs in the specified branch.



• Add steps to install NodeJS and Newman in the .yml file



• If you want to run the particular collection in your branch, configure the “.yml” file using the below command:



• To generate an HTML report, you must include steps to install the htmlextra dependency and establishing a folder to store the report.

The screenshot below demonstrates creating a folder to save the report:

The screenshot below illustrates copying the generated HTML report:

• Once the configuration setup is completed click on “Commit changes”



• Create a new branch and raise an “PR” to the appropriate branch where you want the workflow.
• Accept the “PR” from the respective branch.
• After the “Workflow” is added (or) merged in the respective branch, it will auto trigger the configured file (or) folder every time whenever the deployment is done.

Report Verification:

• Once the execution is completed, we can see the report in the “Actions” tab.
• The recent executions are displayed at the top (or) the recent workflows are displayed in the left side of the “Actions” panel.
• Click on the “Workflow”.
• Click on “build” where we can see the entire test report.
• The “html” report is generated under “Artifacts” at the bottom of the workflow run.



• When you click on the report, it will be getting download in your local system as a zip file.

Issues Faced:

• Sometimes the htmlextra report will not be generated if any of the previous steps or any of the tests getting failed in your postman collection, to handle this error we need to handle the issue.
• To fix the issue we need to handle it with the “if” condition.

Enhancing Your Workflow with Advanced Features

Now that you have a simple testing workflow set up, let’s look at how we can make it better. We can improve it by using advanced features from GitHub Actions.
These features let you run tests at the same time. They also help speed up build times. This can make your CI/CD pipeline easier and faster.

Incorporating Parallel Testing for Efficiency

As your test suite gets bigger, it takes more time to run UI tests. GitHub Actions can help make this easier. It allows you to run your new configuration tests in parallel, which is a great way to cut down the time needed for your tests. By breaking your test suite into smaller parts, you can use several runners to run these parts simultaneously and you can even use a test automation tool to subscribe to notifications about the test run ID and the progress.
This helps you receive feedback more quickly. You don’t need to wait for all the tests to end. You can gain insights into certain parts fast.

Here are some ways to use parallel testing:

• Split by Test Files: Divide your test suite into several files. You can set up GitHub Actions to run them all together.
• Split by Test Types: If you group your tests by type, like unit, integration, or end-to-end, run each group together.
• Use a Test Runner with Parallel Support: Some test runners can run tests at the same time. This makes it easier to set up.

Utilizing Cache to Speed Up Builds

Caching is important in GitHub Actions. It helps speed up your build processes. When you save dependencies, build artifacts, or other files that you use often, it can save you time. You won’t have to download or create them again.
Here are some tips for using caching:

• Find Cachable Dependencies: Look for dependencies that do not change. You can store them in cache. This means you will not need to download them again.
• Use Actions That Cache Automatically: Some actions, like actions/setup-node, have built-in caching features. This makes things easier.
• Handle Cache Well: Make sure to clear your cache regularly. This helps you save space and avoid problems from old files.

Monitoring and Managing Your Automated Tests

It is important to keep an eye on the health and success of automated tests. This is as important as creating them. When you understand the results of the workflow, you can repair any tests that fail. This practice helps to keep a strong CI pipeline.
By paying close attention and taking good care of things, you can make sure your tests give the right results. This helps find and fix any problems quickly.

Understanding Workflow Results and Logs

GitHub Actions helps you see each workflow run in a simple way. It shows you the status of every job and step in that workflow. You can easily find this information in the “Actions” tab of your repository.
When you click on a specific workflow run, you can see logs for each job and step. The logs show the commands that were used, the results they produced, and any error messages. This information is helpful if you need to solve problems.
You might want to connect to a test management tool. These tools can help you better report and analyze data. They can show trends in test results and keep track of test coverage. They can also create detailed reports. This makes your test management much simpler.

Debugging Failing Tests and Common Issues

Failing tests are common. They help you see where your code can get better. It is really important to fix these failures well.
Check the logs from GitHub Actions. Focus on the error messages and stack traces. They often provide helpful clues about what caused the issue.
Here is a table that lists some common problems and how to fix them:

Conclusion

In conclusion, using automated testing with GitHub Actions greatly enhances your software development process by improving speed, reliability, and efficiency. Embracing automation allows teams to streamline repetitive tasks and focus on innovation. Tools like parallel testing further optimize workflows, ensuring code consistency. Regularly monitoring your tests will continuously improve quality. If you require similar automation testing services to boost your development cycle, reach out to Codoid for expert solutions tailored to your needs. Codoid can help you implement cutting-edge testing frameworks and automation strategies to enhance your software’s performance.

Frequently Asked Questions