Blog

Testbeats Integration with Playwright: A Comprehensive Guide

by Jacob | Mar 20, 2025 | Automation Testing, Blog, Latest Post | 1 comment

Testbeats is a powerful test reporting and analytics platform that enhances automation testing and test execution monitoring by providing detailed insights, real-time alerts, and seamless integration with automation frameworks. When integrated with Playwright, Testbeats simplifies test result publishing, ensures instant notifications via communication tools like Slack, Microsoft Teams, and Google Chat, and offers structured reports for better decision-making. One of the key advantages of Testbeats is its ability to work seamlessly with CucumberJS, a behavior-driven development (BDD) framework that runs on Node.js using the Gherkin syntax. This makes it an ideal solution for teams looking to combine Playwright’s automation capabilities with structured and collaborative test execution. By using Testbeats, QA teams and developers can streamline their workflows, minimize debugging time, and enhance visibility into test outcomes, ultimately improving software reliability in agile and CI/CD environments.

This blog explores the key features of Testbeats, highlights its benefits, and demonstrates how it enhances Playwright test automation with real-time alerts, streamlined reporting, and comprehensive test analytics.

Key Features of Testbeats

• Automated Test Execution Tracking – Captures and organizes test execution data from multiple automation frameworks, ensuring a structured and systematic approach to test result management.
• Multi-Platform Integration – Seamlessly connects with various test automation frameworks, making it a versatile solution for teams using different testing tools.
• Customizable Notifications – Allows users to configure notifications based on test outcomes, ensuring relevant stakeholders receive updates as needed.
• Advanced Test Result Filtering – Enables filtering of test reports based on status, execution time, and test categories, simplifying test analysis.
• Historical Data and Trend Analysis – Maintains test execution history, helping teams track performance trends over time for better decision-making.
• Security & Role-Based Access Control – Provides secure access management, ensuring only authorized users can view or modify test results.
• Exportable Reports – Allows exporting test execution reports in various formats (CSV, JSON, PDF), making it easier to share insights across teams.

Highlights of Testbeats

1. Streamlined Test Reporting – Simplifies publishing and managing test results from various frameworks, enhancing collaboration and accessibility.

2. Real-Time Alerts – Sends instant notifications to Google Chat, Slack, and Microsoft Teams, keeping teams informed about test execution status.

3. Comprehensive Reporting – Provides in-depth test execution reports on the Testbeats portal, offering actionable insights and analytics.

4. Seamless CucumberJS Integration – Supports behavior-driven development (BDD) with CucumberJS, enabling efficient execution and structured test reporting.

By leveraging these features and highlights, Testbeats enhances automation workflows, improves test visibility, and ensures seamless communication within development and QA teams. Now, let’s dive into the integration setup and execution process

Guide to Testbeats Integrating with Playwright

Prerequisites

Before proceeding with the integration, ensure that you have the following essential components set up:

• Node.js installed (v14 or later)
• Playwright installed in your project
• A Testbeats account
• API Key from Testbeats

Step 1: Sign in to TestBeats

• Go to the TestBeats website and sign in with your credentials.
• Once signed in, create an organization.
• Navigate to Settings under the Profile section.
• In the Keys section, you will find your API Key — copy it for later use.

Step 2: Setting Up a Google Chat Webhook

Google Chat webhooks allow TestBeats to send test execution updates directly to a chat space.

Create a Webhook in Google Chat

• Open Google Chat on the web and select the chat space where you want to receive notifications.
• Click on the space name and select Manage Webhooks.
• Click Add Webhook and provide a name (e.g., “Test Execution Alerts”).
• Google Chat will generate a Webhook URL. Copy this URL for later use.

Step 3: Create a TestBeats Configuration File

In your Playwright Cucumber framework, create a configuration file named testbeats.config.json in the root directory.

Sample Configuration for Google Chat Webhook

{
"api_key": "your_api_key",
 "targets": [
        {
            "name": "chat",
            "inputs": {
                "url": "your_google_chat_webhook_url",
                "title": "Test Execution Report",
                "only_failures": false
            }
        }
    ],
    "extensions": [
        {
            "name": "quick-chart-test-summary"
        },
        {
            "name": "ci-info"
        }
    ],
    "results": [
        {
            "type": "cucumber",
            "files": ["reports/cucumber-report.json"]
        }
    ]
}

Key Configuration Details:

• “api_key” – Your TestBeats API Key.
• “url” – Paste the Google Chat webhook URL here.
• “only_failures” – If set to true, only failed tests trigger notifications.
• “files” – Path to your Cucumber JSON report.

Step 4: Running and Publishing Test Results

1. Run Tests in Playwright

Execute your test scenarios with:

npx cucumber-js --tags "@smoke"

After execution, a cucumber-report.json file will be generated in the reports folder.

2. Publish Test Results to TestBeats

Send results to TestBeats and Google Chat using:

npx testbeats@latest publish -c testbeats.config.json

Now, your Google Chat space will receive real-time notifications about test execution!

Step 5: Verify Test Reports in TestBeats

• Log in to TestBeats.
• Click on the “Projects” tab on the left.
• Select your project to view test execution details.
• Passed and failed tests will be displayed in the report.

Example Notification in Google Chat

After execution, a message like this will appear in your Google Chat space:

Conclusion:

Integrating Playwright with Testbeats makes test automation more efficient by providing real-time alerts, structured test tracking, and detailed analytics. This setup improves collaboration, simplifies debugging, and helps teams quickly identify issues. Automated notifications via Google Chat or other tools keep stakeholders updated on test results, making it ideal for agile and CI/CD workflows. Codoid, a leading software testing company, specializes in automation, performance, and AI-driven testing. With expertise in Playwright, Selenium, and Cypress, Codoid offers end-to-end testing solutions, including API, mobile, and cloud-based testing, ensuring high-quality digital experiences.

Frequently Asked Questions

TestComplete Tutorial: Expert Tips for Success

by Mollie Brown | Mar 14, 2025 | Automation Testing, Blog, Latest Post | 7 comments

Ensuring the quality, reliability, and performance of applications is more critical than ever. As applications become more complex, manual testing alone is no longer sufficient to keep up with rapid release cycles. Automated testing has emerged as a game-changer, enabling teams to streamline their testing workflows, reduce manual effort, and improve test coverage while accelerating software delivery. Among the various automation tools available,TestComplete, developed by SmartBear, stands out as a feature-rich and versatile solution for automating tests across multiple platforms, including desktop, web, and mobile applications. It supports both scripted and scriptless automation, making it accessible to beginners and experienced testers alike.

Whether you are new to test automation or looking to enhance your skills, this step-by-step tutorial series will guide you through the essential functionalities of TestComplete and help you become proficient in leveraging its powerful features.

Key Features of TestComplete

• Cross-Platform Testing – Supports testing across desktop, web, and mobile applications.
• Multiple Scripting Languages – Allows test automation using Python, JavaScript, VBScript, JScript, and DelphiScript.
• Scriptless Test Automation – Provides keyword-driven and record-and-replay testing options for beginners.
• Advanced Object Recognition – Uses AI-based algorithms to identify UI elements, even when their properties change.
• Data-Driven Testing – Enables running tests with different data sets to improve test coverage.
• Seamless CI/CD Integration – Works with tools like Jenkins, Azure DevOps, and Git for continuous testing.
• Parallel and Distributed Testing – Runs tests simultaneously across multiple environments to save time.

Why Use TestComplete?

• User-Friendly Interface – Suitable for both beginners and experienced testers.
• Supports Multiple Technologies – Works with apps built on .NET, Java, Delphi, WPF, Angular, React, etc.
• Reduces Manual Effort – Automates repetitive tests, allowing teams to focus on critical testing areas.
• Improves Software Quality – Ensures applications are stable, reliable, and bug-free before release.

Getting Started with TestComplete

Starting your TestComplete journey is easy. You can get a free trial for 30 days. This lets you see what it can do before deciding. To get started, just visit the official SmartBear website for download and installation steps. Make sure to check the system requirements first to see if it works with your computer.

After installing, TestComplete will help you create your first testing project. Its simple design makes it easy to set up your testing space. This is true even for people who are new to software testing tools.

System Requirements and Installation Guide

Before you start installing TestComplete, it is important to check the system requirements. This helps ensure it will run smoothly and prevents any unexpected compatibility problems. You can find the detailed system requirements on the SmartBear website, but here is a quick summary:

• Operating System: Use Windows 10 or Windows Server 2016 or newer. Make sure the system architecture (32-bit or 64-bit) matches the version of TestComplete you want to install.
• Hardware: A dual-core processor with a clock speed of 2 GHz or more is best for good performance. You should have at least 2 GB of RAM, but 4 GB or more is better, especially for larger projects.
• Disk Space: You need at least 1 GB of free disk space to install TestComplete. It’s smart to have more space for project files and test materials.

Once you meet these system needs, the installation itself is usually easy. SmartBear offers guides on their website. Generally, all you need to do is download the installer that fits your system, run it as an administrator, agree to the license, choose where to install, and follow the instructions on the screen.

Setting Up Your First Test Environment

Follow these simple steps to set up your test environment and run your first test in TestComplete.

Install TestComplete

• Download and install TestComplete from the SmartBear website.
• Activate your license or start a free trial.

Prepare Your Testing Environment

• Make sure your application (web, desktop, or mobile) is ready for testing.
• Set up any test data if needed.
• If testing a web or mobile app, configure the required browser or emulator.

Check Plugin Availability

• After installation, open TestComplete.
• Go to File → Install Extensions and ensure that necessary plugins are enabled.
• For web automation, enable Web Testing Plugin.
• For mobile automation, enable Mobile Testing Plugin.

Plugins are essential for ensuring TestComplete can interact with the type of application you want to test.

Creat New Project

• Open TestComplete and click “New Project”.



• On the Enter project attributes page of the wizard, you specify the name, location and scripting language of the project, as well as some additional settings:



• Project Name – Specifies the name of the project. TestComplete will automatically add the .mds extension to this name when creating the project file.
• Project Location – specifies the Folder where the Project file will be created.
• Scripting Language – Select the scripting language for your project once selected you can’t change the Project language So choose wisely. You can choose any one of scripting languages Javascript, Python, VBScript
• Use XPath and CSS selectors for web objects – Having this option enabled is compulsory for creating cross platform web test that is, tests that can be run in remote environments that use web browsers not supported by TestComplete directly, like Safari, and operating systems and platforms, like Windows, Linux, Unix, Mac OS, mobile Android and iOS.
• Tested Application – select this checkbox if you want to add your desktop or mobile application to the tested application list of your new project. You can also add a tested application at any time later.
• BDD Files – Select this check box to import your BDD feature files to your project to automate them. You can also import files at any time after you create the project
• Select the Application Type based on what you are testing:
• Desktop Application → For Windows-based applications.
• Web Application → For testing websites and web applications (supports Chrome, Edge, Firefox, etc.).
• Mobile Application → For testing Android and iOS apps (requires a connected device/emulator).
• Enter a Project Name and select a save location.
• Click “Create” to set up the project.

TestComplete will now generate project files, including test logs, name mappings, and test scripts.

Adding the Application Under Test (AUT)

To automate tests, TestComplete needs to recognize the Application Under Test (AUT).

For Desktop Applications:

• Go to Project Explorer → Tested Applications.
• Click “Add”, then select “Add Application”.
• Browse and select the .exe file of your desktop application.
• Click OK to add it.

For Web Applications:

• Navigate to Tested Applications → Click “Add”.
• Enter the URL of the web application.
• Select the browser where the test will run (Chrome, Edge, Firefox, etc.).
• Click OK to save.

For Mobile Applications:

• Connect an Android/iOS device to your computer.
• In TestComplete, navigate to Mobile Devices → Connect Device.
• Select the application package or install the app on your device.

Now, TestComplete knows which application to launch and test.

Understanding Object Spy & Object Browser

TestComplete interacts with applications by identifying UI elements like buttons, text fields, checkboxes, etc. It does this using:

Object Spy (To Identify UI Elements)

• Click Object Spy from the TestComplete toolbar.
• Drag the crosshair icon over the UI element you want to inspect.
• TestComplete will display:
• Element properties (ID, name, type, etc.)
• Available methods (Click, SetText, etc.)
• Click “Map Object” to save it for automation scripts.

Object Spy helps TestComplete recognize elements even if their location changes.

Object Browser (To View All UI Elements)

• Open View → Object Browser.
• Browse through the application’s UI hierarchy.
• Click any object to view its properties and available actions.

Object Browser is useful for debugging test failures and understanding UI structure.

Creating a Test in TestComplete

TestComplete allows different ways to create automated tests.

Method 1: Record and Playback (No Coding Required)

• Click “Record” in the toolbar.
• Perform actions on your application (click buttons, enter text, etc.).
• Click “Stop” to save the recorded test.
• Click Run to execute the recorded test.

Great for beginners or those who want quick test automation without scripting!

Method 2: Keyword-Driven Testing (Step-by-Step Actions)

• Open Keyword Test Editor.
• Add actions like Click, Input, Verify, etc. using a graphical interface.
• Arrange steps in order and save the test.
• Run the test and check results.

Ideal for testers who prefer a structured, visual test flow.

Method 3: Scripted Testing (Python, JavaScript, VBScript, etc.)

• Right-click “Test Scripts” → Select “New Script”.
• Write a script using a supported language (Python example)
def test_login():
    TestedApps.MyApp.Run()
    Aliases.MyApp.LoginPage.Username.SetText("user")
    Aliases.MyApp.LoginPage.Password.SetText("password")
    Aliases.MyApp.LoginPage.LoginButton.Click()

• Save and run the script.

Best for advanced users who need flexibility and customization.

Running the Test

• Click Run to start the test execution.
• TestComplete will launch the application and perform actions based on the recorded/scripted steps.
• You can pause, stop, or debug the test at any point.

Running a test executes the automation script and interacts with the UI elements as per the defined steps.

Viewing Execution Results

Once the test completes, TestComplete generates a Test Log that provides:

• ✅Pass/Fail Status – Displays if the test succeeded or failed.
• 📷Screenshots – Captures test execution steps.
• ⚠️Error Messages – Shows failure reasons (if any).
• 📊Execution Time & Performance Metrics – Helps analyze test speed.

Here some of Pros & Cons of TestComplete

Pros

• Supports a wide range of platforms, including Windows, macOS, iOS, and Android.
• Allows for data-driven testing, enabling tests to be run with multiple data sets to ensure comprehensive coverage.
• Supports parallel execution of tests, speeding up the overall testing process.
• Generates detailed test reports and logs, helping testers analyze results and track issues efficiently.
• Can test web, desktop, and mobile applications.

Cons

• Mastering all the functionalities, especially advanced scripting, can take time.
• TestComplete can be a bit expensive compared to some other testing tools.
• It can be resource-intensive, requiring robust hardware for optimal performance, especially when running multiple tests in parallel.
• Despite advanced object recognition, there can still be issues with recognizing dynamic or complex UI elements, requiring manual adjustments.

Conclusion

Test automation is essential for ensuring software quality, increasing efficiency, and reducing manual effort. Among the many automation tools available, TestComplete, developed by SmartBear, is a powerful and flexible solution for testing desktop, web, and mobile applications. In this tutorial, we covered key aspects of using TestComplete, including installation, project setup, test creation, execution, and result analysis. We also explored how to add an Application Under Test (AUT), use Object Spy and Object Browser to identify UI elements, and implement different testing methods such as record-and-playback, keyword-driven testing, and scripting. Additionally, we discussed best practices like name mapping, test modularization, CI/CD integration, and data-driven testing to ensure stable and efficient automation.

As a leading software testing company, Codoid specializes in test automation, performance testing, and QA consulting. With extensive expertise in TestComplete and other advanced automation tools, Codoid helps businesses improve software quality, speed up testing cycles, and build strong automation strategies. Whether you’re new to automation or looking to enhance your existing test framework, Codoid offers expert guidance for achieving reliable and scalable automation solutions.

This blog provided an overview of TestComplete’s capabilities, but there’s much more to explore. Stay tuned for upcoming blogs, where we’ll dive deeper into advanced scripting, data-driven testing, CI/CD integration, and handling dynamic UI elements in TestComplete.

Frequently Asked Questions