Categories
Playwright

Five reasons why Playwright is better than Cypress

It’s pretty obvious I am not a Cypress fan – heck I wrote a whole post about it 2 years ago.

I’ve since become very keen on Playwright and I thought I’d revisit Cypress to compare it with Playwright and see whether any of my long-standing gripes about Cypress have been addressed in 2+ years since I’ve used it.

Versions compared:

Cypress: 8.7.0 on Electron 93
Playwright: 1.16.0

Reason 1: Playwright is so much faster than Cypress

Up to 4x faster.

Take this simple example taken directly from the Cypress doco:

describe('My First Test', () => {
    it('clicking "type" shows the right headings', () => {
      cy.visit('https://example.cypress.io')
  
      cy.contains('type').click()
  
      // Should be on a new URL which includes '/commands/actions'
      cy.url().should('include', '/commands/actions')
  
      // Get an input, type into it and verify that the value has been updated
      cy.get('.action-email')
        .type('fake@email.com')
        .should('have.value', 'fake@email.com')
    })
})

Takes 8 seconds running on my M1 Macbook Air

 ✔  demo-spec.js                             00:08

I wrote the same example in Playwright:

import { test, expect } from '@playwright/test'

test.describe.parallel('My First Test', () => {
  test('clicking "type" shows the right headings', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    // Should be on a new URL which includes '/commands/actions'
    await page.waitForURL(/.+\/commands\/actions$/)
    // Get an input, type into it and verify that the value has been updated
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
})

Running 1 test using 1 worker

  ✓  scenarios/compare.cypress.spec.ts:4:3 › My First Test › clicking "type" shows the right headings (1s)


  1 passed (2s)

4 times faster 😎

Reason 2: Playwright has first-class support for parallel running of tests on a single machine both locally and on CI without a subscription or account required

Cypress on the other hand requires you to set your tests to “Record” everything they do which sends everything to Cypress (or Sorry Cypress) if you want to run in parallel, and using a single machine is not recommended (despite it working perfectly in Playwright using my Macbook Air M1)

“While parallel tests can also technically run on a single machine, we do not recommend it since this machine would require significant resources to run your tests efficiently.”

https://docs.cypress.io/guides/guides/parallelization#Overview

Either this is true and Cypress is so bloated that running in parallel locally needs crazy expensive hardware, or the Cypress team want users to pay for their parallel execution service. Or both?

Reason 3: Playwright supports parallel tests within a single test file – to run Cypress tests in parallel you need to split them across files.

For example, in Playwright I can run these 8 tests in parallel:

import { test, expect } from '@playwright/test'

test.describe.parallel('My First Test', () => {
  test('clicking "type" shows the right headings 1', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 2', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 3', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 4', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 5', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 6', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 7', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
  test('clicking "type" shows the right headings 8', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
})

which takes only 6 seconds on my Macbook Air 😎

> npx playwright test "./scenarios/compare.cypress.bulk.spec.ts"

Running 8 tests using 4 workers

  ✓  scenarios/compare.cypress.bulk.spec.ts:4:3 › My First Test › clicking "type" shows the right headings 1 (4s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:11:3 › My First Test › clicking "type" shows the right headings 2 (4s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:18:3 › My First Test › clicking "type" shows the right headings 3 (4s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:25:3 › My First Test › clicking "type" shows the right headings 4 (4s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:46:3 › My First Test › clicking "type" shows the right headings 7 (1s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:32:3 › My First Test › clicking "type" shows the right headings 5 (1s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:39:3 › My First Test › clicking "type" shows the right headings 6 (1s)
  ✓  scenarios/compare.cypress.bulk.spec.ts:53:3 › My First Test › clicking "type" shows the right headings 8 (1s)

  8 passed (6s)

To possibly run these in parallel (using a server) in Cypress I’d need to structure these as individual files each with a single spec:

And to run them on the same machine I ran these 8 scenarios in Playwright in 6 seconds takes 45 seconds! 7.5 times slower!

Reason 4: Playwright fully supports async/await syntax for clean, readable code.

This is the Playwright code:

import { test, expect } from '@playwright/test'

test.describe.parallel('My First Test', () => {
  test('clicking "type" shows the right headings', async ({ page }) => {
    await page.goto('https://example.cypress.io')
    await page.click('a:has-text("type")')
    await page.waitForURL(/.+\/commands\/actions$/)
    await page.fill('.action-email', 'fake@email.com')
    await expect(page.locator('.action-email')).toHaveValue('fake@email.com')
  })
})

No chaining, no hard to understand code, no magic. You can see where test and expect come from, you can see where page comes from.

The same test in Cypress:

describe('My First Test', () => {
    it('clicking "type" shows the right headings', () => {
      cy.visit('https://example.cypress.io')
    
      cy.contains('type').click()
  
      // Should be on a new URL which includes '/commands/actions'
      cy.url().should('include', '/commands/actions')
  
      // Get an input, type into it and verify that the value has been updated
      cy.get('.action-email')
        .type('fake@email.com')
        .should('have.value', 'fake@email.com')
    })
})

Note how the function calls chain together – in this simple example it’s not too bad but it quickly gets out of hand. Also note there’s no indication that this is synchronous code – no await on any calls. Also where does this cy thing come from – oh yes that’s magic. Same for describe, again magic. No imports but lots of magic.

Reason 5: Playwright doesn’t need plugins

There’s so many limitations in Cypress there’s pretty much a plugin for everything. And there’s still so may trade-offs that can’t be be fixed by a plugin.

Need to send the tab key to the browser? There’s a plugin for that. And it doesn’t work well at all according to the comments on the Github issue for lack of tab key support open since 2016 (and still open since I last wrote about Cypress).

Need to upload a file? There’s also a plugin for that.

Need to run tests n-times in a row to measure repeatability? There’s a plugin for that.

Fortunately Playwright supports all of these things, and many more, natively. No plug-ins needed 😎

Summary

As you can see I’m still not a Cypress fan all these years later 😀

It’s just that now there’s a tool which is so much better and freely available to use.

So I’ll stand by my summary from 2 years ago which I can repeat here:

From a distance Cypress looks like a polished tool for automated testing – I just think it’s incorrectly marketed as an end-to-end testing tool when it’s really only good for component testing. There are too many limitations in the tool in acting like a real user to use it to create true end-to-end automated tests.

https://alisterbscott.com/2019/07/06/my-thoughts-on-cypress-io/