Categories
e2e Testing

Playwright + Jest = 💖

We were looking at a possible replacement for our dated Protractor + Cucumber e2e testing framework. As we move away from Angular to React microapps we have found that Protractor doesn’t work very well/efficiently and cucumber isn’t giving us any benefits.

It was a good opportunity to do some research/tinkering to answer the question lingering in my mind: in 2020 what e2e testing tool would I use by default for a dynamic react based web application?

After some experimentation it came down to Puppeteer + Jest or Playwright + Jest. I’ll compare those in this post.

Why Jest?

As explained previously we don’t need a BDD framework, but we do need something that allows us to specify our tests, create assertions and run these tests (in parallel). Jest, particularly when using the Jest Circus test runner, seems the most mature tool in this regard in 2020. Whilst Jest is often associated with React as it handles the snapshot test results also, it doesn’t need to be used with React and it’s possible and easy to use it as a standalone Node.js testing library.

Parallel Support

Jest by default runs tests across files in parallel and uses the available resources to scale appropriately using processes/threads. I’ve found this particularly good as I write independent e2e tests which can scale through parallelism, and using Puppeteer and Playwright you can spawn new incognito browser contexts to run these.

Auto-Retry Support

Whilst I believe in writing deterministic e2e automated tests, since e2e tests are full-stack there are often external dependencies and services beyond our control (like a third party domain provider) and I’d rather prioritize test reliability over test perfection. With this mind I think it’s important to be able to automatically retry a single failing test scenario before failing a build, and the Jest Circus test runner supports this:

jest.retryTimes(1)

test('can wait for an element to appear', async () => {
  global.page = await pages.spawnPage()
  await nav.visitHomePage(global.page)
  await global.page.waitForSelector('#elementappearschild', { visible: true, timeout: 5000 })
}, jestTimeoutMS)

HTML Reports and Screenshots

Nice looking HTML reports are easy to achieve by using jest-stare and screenshots are easy to generate using Jest Circus hooks.

Playwright or Puppeteer?

I’m a fan of Puppeteer however Playwright is a much nicer browser automation library. Whilst it adds support for Firefox and Webkit, even if you’re running your e2e tests in one browser (Chromium) I’d still recommend Playwright over Puppeteer any day of the week. Here’s why:

Automatic Waiting

I’ve found the automatic waiting in Playwright just works™️ Especially when dealing with dynamically rendered react web apps I’ve found my Puppeteer code looks is full of page.waitFor calls to make it run reliably:

await page.waitFor('#loadedchild', { visible: true, timeout: 5000 })

Whilst this is occasionally necessary in Playwright (in particular waiting for an iFrame to switch into), I’ve found it’s almost never required which reminds me of good old Watir days.

Nicer API

Whilst the APIs are similar, Playwright is just nicer to use. Take grabbing some text from a div, in Puppeteer:

await page.goto(`${config.get('baseURL')}`)
await page.waitFor('#loadedchild', { visible: true, timeout: 5000 })
const element = await page.$('#loadedchild')
const text = await (await element.getProperty('textContent')).jsonValue()
expect(text).toBe('Loaded!')

In Playwright:

await page.goto(`${config.get('baseURL')}`)
const text = await page.textContent('#loadedchild')
expect(text).toBe('Loaded!')

Summary

If I was writing an e2e web testing framework from scratch in 2020 I would use Playwright + Jest. Playwright offers automatic waiting and a nice API, whilst Jest offers a solid runner with parallel support, automatic retries and the ability to easily generate HTML reports and capture screenshots.

I’ve created one repository for Playwright + Jest #
And another for Puppeteer + Jest to compare #

As an aside we put changing node libraries on hold for our e2e test framework and will look at some test infrastructure improvements instead.

Categories
e2e Testing

Puppeteer Jest Demo

I’m trying to work out what would be the ideal combination for an e2e web testing framework in Node.

I wanted to try Jest with Puppeteer (I’d previously used Mocha) and I also wanted to try avoiding Babel for transpiling and using JavaScript directly so I came up with this example on GitHub.

I think I’ll try using TypeScript for my next tinker project.