Getting Started With Visual Regression Testing Using Playwright

Visual bugs impact the user experience negatively. For instance, a resized image might inadvertently cover a button. While these bugs often don’t affect an application’s functionality, they can be tricky for developers to identify. Visual regression testing is a tool to help detect these bugs early in the development process.

The playwright makes visual regression testing straightforward. With as little as two lines of code, you can address basic visual issues. When you further integrate Playwright’s automation capabilities with LambdaTest’s digital experience testing platform, you can establish a comprehensive visual regression testing process. This approach not only ensures a high-quality user experience but also prevents visual bugs from affecting your users. Dive in and start using Playwright for visual regression testing to guarantee that your application consistently looks its best.

Visual Regression Testing: What Is It?

Visual Regression Testing is primarily utilized to prevent unintentional alterations in the visual presentation of an application. For instance, imagine you possess a website with predetermined page layouts. You direct your testing framework to capture screenshots of each page and establish a guideline that mandates the pages to consistently mirror the recently captured screenshots. 

The ideal outcome is that your website perpetually mirrors the flawless appearance of the screenshots for every user. Nevertheless, there are instances when glitches can lead to deviations between your website’s appearance and the reference screenshots. This could stem from a failed deployment, modifications in CSS that trigger rendering problems on outdated browsers, or other bugs disrupting the user experience. 

VRT empowers you to systematically identify and address these regression-related issues. It promptly notifies you of any shifts in your website’s design.

Importance Of Visual Regression Testing

Visual regression testing is indispensable for several reasons:

Improves User Experience: Visual defects can detrimentally affect user experiences, causing dissatisfaction and frustration. By identifying and rectifying visual regressions, we can offer a consistent and smooth user interface, fostering user engagement.

Identifies UI Inconsistencies: For large-scale applications, it can be tough to maintain visual uniformity across various pages. Visual regression testing pinpoints UI discrepancies, ensuring compliance with design norms and brand identity.

Catches Unnoticed Bugs: Standard functional tests might not uncover all visual anomalies, making visual regression testing vital for spotting subtle visual alterations that could result in undetected bugs or design issues.

Enhances Continuous Integration (CI/CD): Incorporating visual regression tests into the CI/CD process facilitates automated testing with every code modification. This enables early identification of visual regressions, preventing their migration into the production environment.

Reduces Time And Effort: Manual visual inspection can be laborious and prone to errors. Automated visual regression testing lessens manual workload and broadens test scope, boosting testing efficiency.

How Does Visual Regression Testing Generally Work?

When you run the script for the first time, it captures a screenshot of a component or the entire page and stores it for future comparison. In subsequent executions, the script takes another screenshot and compares it with the previously saved one. If discrepancies are detected, the execution is flagged as a failure, and the difference is highlighted through a diff image. After assessing the differences, you can choose to treat them as bugs and address them accordingly, or you can interpret them as expected and intentional changes. In the latter case, you have the option to replace the previous screenshot with the new one so that an updated reference is available for the next execution.

What Is Playwright?

Playwright is an open-source automation library created by Microsoft, designed specifically for testing and automating web applications in modern web browsers. It offers a robust and intuitive API that enables developers and testers to engage with web pages, emulate user actions, and verify the behaviour of applications. For visual tests, Playwright has an integrated feature that leverages the pixelmatch library like a diffing engine. Additionally, Playwright facilitates the execution of these tests across various screen sizes and browsers, ensuring comprehensive coverage of all scenarios. This configuration can be set in the playwright.config file.

How To Use Playwright For Visual Regression Testing?

You can effectively integrate Playwright with testing frameworks like Jest or Mocha, both of which provide strong support for automating Playwright tasks. To enable browser interaction and simulate user actions such as button clicks and text input, you simply insert a code snippet within your Jest test or Mocha scripts that connects to the browser being tested through Playwright. For Jest, you can make use of the “jest-playwright-preset” preset to streamline the Playwright connection. In Mocha testing, establish the Playwright connection within the “beforeEach” section of your test, as demonstrated in the following code snippet:

const pw = require(‘playwright-core’)

beforeEach(async function() {

    browser = await pw.chromium.connectOverCDP({

        wsEndpoint: ‘wss://cloud.testingbot.com?browserName=edge&browserVersion=latest’

    })

    context = await browser.newContext()

    page = await context.newPage(‘https://testingbot.com’)

})

Setting Up Visual Regression Testing Using Playwright

Playwright Test, a toolkit that enables automated test execution, is a component of Playwright. A screenshot of the current page is taken and saved as a baseline screenshot, acting as the reference image, when using Playwright Test to perform tests. New screenshots are rigorously evaluated against the reference image utilizing pixel-level matching in subsequent tests. Consider an instance where a golden image has been established (reference screenshot) to provide an illustration. You can use a code snippet like the one shown below to compare it to a screenshot recently taken using Playwright:

expect(await page.screenshot()).toMatchSnapshot(‘golden.png’)

The playwright employs the page object to execute the screenshot function, capturing an image for saving. Subsequently, it leverages the pixelmatch library to analyze the captured screenshots, identifying any visual disparities between them. Utilizing the threshold option enables fine-tuning of pixel matching within the screenshots. Refer to the provided code snippet for a more comprehensive illustration:

// example.spec.js

const { test, expect } = require(‘@playwright/test’);

test(‘example test’, async ({ page }) => {

  await page.goto(‘https://testingbot.com’);

  expect(await page.screenshot(‘.hero’)).toMatchSnapshot(‘hero.png’, { threshold: 0.2 });

});

How To Implement A Simple Playwright Visual Test?

Using Playwright for visual tests is beneficial if your team is already familiar with it for other testing levels. If you’re new to Playwright, follow the documentation to install it from https://playwright.dev/docs/intro, with TypeScript as an example. Ensure you have NodeJS installed and an Integrated Development Environment (IDE) like Visual Studio Code. If you use yarn, install Playwright with: yarn create playwright

Next, create a .spec file, for instance, example.spec.ts:

import { test, expect } from ‘@playwright/test’;

test(‘example test’, async ({ page }) => {

  await page.goto(‘https://playwright.dev’);

  await expect(page).toHaveScreenshot();

});

To execute the script, use: npx playwright test

When running this script initially, you’ll see the following test runner output:

Error: A snapshot missing at example.spec.ts-snapshots/example-test-1-chromium-darwin.png, writing actual.

This occurs because it captures the page screenshot, and when it attempts to compare it with a baseline screenshot, it doesn’t find one locally with that name. Therefore, it saves the screenshot for comparison in subsequent test runs. If you run the test again in the future and it fails, it indicates a visual discrepancy between the two screenshots. You must assess if this is due to a bug or a deliberate code change. If it’s an intentional change, execute the command with the flag — update-snapshots: npx playwright test –update-snapshots

How Does Screenshot Nomenclature Work?

The process first creates a folder with the same name as the .spec file if it doesn’t already exist. It then saves the screenshot with a name, for instance, example-test-1 (an auto-generated name for the snapshot that corresponds to the test’s name – you can replace it with a custom name by providing it as an argument to the toHaveScreenshot() method). In addition to example-test-1, the process also appends the browser name and the platform, which let us consider, for this example, ‘chromium’ and ‘darwin’, respectively (these values can’t be customized).

Please note that your local Operating System (OS) should match the OS of your Continuous Integration (CI) system, or you should run these tests in Docker. This is to ensure consistency, as the screen display could vary between different OSs used in local and CI environments.

How To Customize Your Visual Comparison?

In the example we shared earlier, the script takes a screenshot of the entire page and compares it with an existing screenshot. However, this is just one use case – you may want to test only a specific element on the page. To do this, you should utilize Playwright’s locators. You can also set various options that will alter the behavior of the toHaveScreenshot() method, as provided by pixel match:

maxDiffPixels: The acceptable ratio of differing pixels to the total number of pixels, ranging from 0 to 1 (no default setting);

threshold: The tolerable perceived color difference in the YIQ color space between the same pixel in the compared images, ranging from zero (strict) to one (lax). The default is 0.2;

maxDiffPixelRatio: The acceptable quantity of pixels that may differ (no default setting).

For example, using the maxDiffPixels option, the syntax would be:

await expect(page).toHaveScreenshot({ maxDiffPixels: 100 });

To test a particular UI element, substitute the page parameter within the expect() function with a distinct element locator. Additionally, if you intend to verify whether the content of an element aligns with a reference text or any form of binary data, Playwright Test will automatically identify the content type and implement the suitable comparison algorithm. For instance:

expect(await page.textContent(‘.hero__title’)).toMatchSnapshot(‘heroText.txt’);

A Bit About Reporting Visualization

Playwright offers several built-in reporters to cater to various requirements; you have the option to incorporate custom reporters and the ability to use third-party reporters. For example, if you run these tests using Allure as the reporter, you can quickly identify the issue by viewing the screen difference in failed tests, using either the ‘Show diff’ or ‘Show overlay’ option.

Conclusion

Visual regression testing stands as a pivotal approach in the world of software testing, ensuring our applications not only function as intended but also look the part. With tools like Playwright, setting up and executing these tests has never been more seamless. As we’ve traversed through the basics of integrating Playwright for visual regression, it’s evident that such tools empower developers and testers alike to catch visual discrepancies early and maintain consistent user experiences. As you continue your journey, always be inquisitive, keep iterating on your test strategies, and leverage the full potential of Playwright to fortify the visual integrity of your applications.

Read Also:

• Take Note Of These Steps To Develop Your Brand
• Creating a Customer-Centric Experience in Real Estate
• 5 Lead Generation Secrets Emerging Brands Can Rely On