Our Shift From Cypress to Playwright in Testing

In our post about extensive-react-boilerplate updates, we mentioned that we migrated e2e-testing from Cypress to Playwright. Now, let's delve a little deeper into this change.

At the time of writing the automated tests, we had a small amount of functionality to cover and didn't face significant limitations when using Cypress. Yet, we decided to turn our attention to Playwright for several reasons. We wanted to explore the framework created by Microsoft and understand why it is gaining popularity. Additionally, similar to the case when we added MongoDB support, we received requests from the community and colleagues who wanted to start a project based on boilerplate with Playwright tests.

As we began the process of migrating tests, we acknowledged that the volume of tests was insignificant. Therefore, we decided to rewrite the tests manually in order to familiarize ourselves with the new framework in more detail. 

Familiarizing Ourselves With a New Framework

First, we start discussing the documentation. We can confidently say that Cypress's documentation surpasses Playwright. Cypress documentation is very detailed and contains numerous examples and tutorials. There is also an entire project on GitHub with examples for every action that can be performed on a typical website. Additionally, the Cypress community is larger in comparison to Playwright. Although experienced developers may be content with the information provided in Playwright’s documentation, less experienced developers may find learning Cypress more enjoyable.

Moving on to setting up the configuration file. We don’t find significant differences between the two frameworks. We only need to configure the timeouts and the base URL. We also explored some new capabilities that Playwright offers in this regard, such as:

• Setting timeouts for each test, including test and Before/After hooks:

// playwright.config.ts
export default defineConfig({
...
  timeout: 2 * 60 * 1000,
...
});

• Support of testing on WebKit, which is based on Apple Safari, whereas Cypress lacks such support

Playwright also has the ability to start a local development server with your project before running tests, which can be easily implemented using the webServer parameter. 

  webServer: {
    command: process.env.CI
      ? "npm run build:e2e && npm run start"
      : "npm run dev",
    url: "http://127.0.0.1:3000",
    reuseExistingServer: !process.env.CI,
  },

Next, we write our first test. The difference in syntax between the two frameworks is notable. Cypress uses chainable syntax and has its own implementation of asynchrony, while Playwright supports the ECMAScript 2015 standard (ES6) and works with convenient async/await construction for asynchronous functions.

Here is a Playwright code sample:

  test("should be successful open page and check data is displayed", async ({
    page,
  }) => {
    await page.getByTestId("profile-menu-item").click();
    await page.getByTestId("user-profile").click();
    await page.waitForURL(/\/profile/);
    await expect(page.getByTestId("user-name")).toHaveText(
      `${firstName} ${lastName}`
    );
    await expect(page.getByTestId("user-email")).toHaveText(email, {
      ignoreCase: true,
    });
    await page.getByTestId("edit-profile").click();
    await page.waitForURL(/\/profile\/edit/);
    await expect(page.getByTestId("first-name").locator("input")).toHaveValue(
      firstName
    );
    await expect(page.getByTestId("last-name").locator("input")).toHaveValue(
      lastName
    )

And here is Cypress:

  it("should be successful open page and check data is displayed", () => {
    cy.getBySel("PersonIcon").click();
    cy.getBySel("user-profile").click();
    cy.location("pathname").should("include", "/profile");
    cy.getBySel("user-name").should("contain.text", firstName + " " + lastName);
    cy.getBySel("user-email").should("contain.text", email);
    cy.getBySel("edit-profile").click();
    cy.location("pathname").should("include", "/profile/edit");
    cy.get('[data-testid="first-name"] input').should("contain.value", firstName);
    cy.get('[data-testid="last-name"] input').should("contain.value", lastName);
  });

Framework Comparisons

When it comes to running the tests, we notice the architectural differences between the frameworks. 

• Cypress executes commands inside the browser, which gives it easy access to important components such as DOM, local storage, and window objects. On the other hand, Playwright uses a client-server architecture and communicates with browsers via a WebSocket connection.
• After rewriting all the tests, we ran them all and observed that by default, Playwright runs tests in parallel, providing this feature for free. In contrast, Cypress performs parallelization only for different machines, and it is a paid feature. 
• Running the same tests in both frameworks revealed that Playwright completed the tests more quickly than Cypress. 

We conducted the tests and found that Playwright completed them in 2.7 minutes:

While Cypress took 3.82 minutes, showing a significant time difference in favor of Playwright.

Conclusion

Considering all the above points, one might wonder why we decided to change the framework. Although we did not see significant benefits at that moment, we took into account the future of our project and potential projects that will be built on top of boilerplates from the bcboilerplates ecosystem. From this perspective, Playwright seemed more promising than Cypress due to its parallelization of tests, higher speed, the possibility of testing mobile applications, the ability to use programming languages other than JS and TS, and the backing of a major player like Microsoft.