Categories
Playwright

API Testing using Playwright

I’m a big fan of being able to call APIs during e2e tests to set up data, verify things, do things and all that jazz.

Up until now I’ve been mixing in Supertest with Playwright to call APIs within tests, until now…

Playwright 1.16 includes the ability to call APIs both independently and using the page browser object (which sends the currently stored cookies for API requests).

I’ve updated my example TypeScript project to include these API calls directly using both page and request which looks like:

test('can POST a REST API and check response using approval style', async ({ request }) => {
    const response = await request.post('https://my-json-server.typicode.com/webdriverjsdemo/webdriverjsdemo.github.io/posts', { data: { title: 'Post 4' } })
    expect(response.status()).toBe(201)
    const body = await response.text()
    expect(body).toMatchSnapshot('post4.txt')
  })

  test('can POST a REST API and check response using assertion style (using page)', async ({ page }) => {
    const response = await page.request.post('https://my-json-server.typicode.com/webdriverjsdemo/webdriverjsdemo.github.io/posts', { data: { title: 'Post 4' } })
    expect(response.status()).toBe(201)
    const body = JSON.parse(await response.text())
    expect(body.id).toBe(4)
    expect(body.title).toBe('Post 4')
  })

It’s nice to be able to remove another dependency from my e2e tests and still allow calling APIs using a nice API and reusing the existing objects that Playwright provides. I am continually blown away by how Playwright continues getting more and more awesome 😎

Categories
Playwright

Selecting hard to identify elements with Playwright

Say you have a web app which displays a table of dynamic data that looks like this:

Admire my beautiful design skills!

And the DOM looks like:

<table>
    <theader>
      <tr>
        <th>Status</th>
        <th>Name</th>
        <th>Actions</th>
      </tr>
    </theader>
    <tbody>
      <tr>
        <td>OPEN</td>
        <td>Cactus</td>
        <td>
          <div>
            <button>CLICK ME</button>
          </div>
        </td>
      </tr>
      <tr>
        <td>CLOSED</td>
        <td>Succulent</td>
        <td>
          <div>
            <button>CLICK ME</button>
          </div>
        </td>
      </tr>
      <tr>
        <td>OPEN</td>
        <td>Aloe</td>
        <td>
          <div>
            <button>CLICK ME</button>
          </div>
        </td>
      </tr>
      <tr>
        <td>OPEN</td>
        <td>Agave</td>
        <td>
          <div>
            <button>CLICK ME</button>
          </div>
        </td>
      </tr>
    </tbody>
  </table>

Say you’re writing an e2e test that adds an item to the table dynamically (with a new name each time) and you need to click the button on the corresponding line to complete your e2e test.

How do you select the button based upon the name of the item you just created? Let’s save Agave?

I feel like the best possible approach is to add a data- attribute to the buttons so you can know exactly which one to click. Something like <button data-name="Agave">Agave</button"

A previous article I wrote on using data attributes

But if you can’t add that value to each row – what’s the next best thing to do?

Playwright has a pretty neat test recorder (even though I don’t like test recorders) – so let’s see what it generates

I launch Playwright Inspector (which has the Record function) with

npx playwright open https://webdriverjsdemo.github.io/table/

After clicking Record, and then clicking the four buttons on the table in order from top-bottom, the Inspector tool gives me:

Clicking the four buttons here
  await page.click('text=CLICK ME');
  await page.click('text=Succulent CLOSED CLICK ME >> button');
  await page.click('text=Aloe OPEN CLICK ME >> button');
  await page.click('text=Agave OPEN CLICK ME >> button');

The issue with this generated code is it relies on the status values which can change, also if the status column was removed in a future change it would fail which makes the tests brittle.

We can use the Playwright Inspector “Explore” feature to enter a selector value and see whether it finds what we’re looking for in the DOM.

If we took one of the selectors above and tried to remove the bit we don’t care about from the text you can see it doesn’t find what we want:

text=Agave >> button

Doing some research I found there’s an option to find any element containing some text, we can choose the row containing the text “Agave” (or whatever the name of the record we created is) and then choose the button belonging to that row:

tr:has-text("Agave") >> button

The important thing to note here is we are selecting the row, even though the cell contains the text we’re looking for.

I think this is probably the best option if we don’t have ability to change the DOM and add data attributes – it’s the least brittle as any additional columns could be added to the table and it would still work.

Some other options I investigated were locators based on position

For example, you can write a selector that finds the buttons to the right of the text we’re looking for:

button:right-of(:text("Agave"))

But as you can see from the image above it locates all the buttons as they’re all technically to the right of the text – just not immediately to the right.

Summary

Whilst it’s good practice to add testability features to your app, Playwright also offers powerful ways to select elements based on text and DOM layout if you can’t. You can read all about the Playwright selectors here.

Categories
Playwright

Running parallel Playwright Tests within a single spec file

Playwright by default runs all tests within a single spec file in order using the same worker, and runs tests in different spec files in parallel.

You can now use the test.describe.parallel block to specify that all tests contained in the block are independent and can be run in parallel. This can mean faster test runs.

There’s also test.describe.serial to specify the tests contained should be run one after another, and the subsequent tests won’t be run if an earlier test fails.

There’s also test.step which can be used to break tests down further into a series of steps.

I’ve updated my example TypeScript tests to use the parallel format.

Categories
Aside

Microsoft Teams

Most of my meetings nowadays are held online.

I hate being late to meetings, but I also like getting things done. When my calendar gives me the reminder about a meeting 10 mins before I’ve developed a habit of joining the meeting straight away so I am already in the meeting when other people join. That way I can continue my work without forgetting to join right on time. I’ve found if I don’t do that I’ll dismiss the notification, get caught up in my work (flow) and forget to join.

This was all well and good at my previous company where we used Google Meet. But my current company uses Microsoft Teams and when I join 10 mins early everyone else invited to the meeting gets a notification telling them I’ve started the meeting! Who designed Microsoft Teams thinking that was a good idea!?!

Any ideas how I can work with this?

Categories
Automated Testing Playwright

Playwright Trace Viewer

One of the things that people really like about Cypress is the ability to see a step by step replay of what happened with snapshots of the DOM as it ran.

Well Playwright also offers this 😎

An easy way to get access to these is to set your Playwright config to rerun failed tests and to capture a trace on the second attempt:

playwright.config.ts:

module.exports = {
  use: {
    headless: true,
    screenshot: "only-on-failure",
    video: "retry-with-video",
    trace: "on-first-retry",
  },
  retries: 1,
};

A trace zip file will be generated on a failed 2nd attempt.

I uploaded an example here. You simply run

npx playwright show-trace ./doco/exampletrace.zip

which displays an interactive step by step trace viewer 😎

An example trace viewer
Categories
Automated Testing Playwright

TypeScript!

I recently changed jobs 🎉

One of the biggest things I noticed since changing jobs nearly 3 years ago was the prevalence of TypeScript: JavaScript With Syntax For Types. Most, if not all, of the jobs I were looking at were dev teams using TypeScript – driven by the increasing use full stack Node.js.

It’s only week 2 in my new job but I’ve already set up an e2e testing framework using Playwright and it’s all in TypeScript 😊 I thought I’d share the main differences and I’ve created a sample repository to do so!

To use TypeScript on a front-end codebase running in a browser you need to compile/transpile it into JavaScript (which a browser understands).

Fortunately Playwright doesn’t require this step as Playwright runs your TypeScript code directly.

The first thing to do is add typescript to your project and add a tsconfig.json file to your project.

I use Playwright Test – and the main difference between a TypeScript spec file (*.spec.ts) and a JavaScript one (*.spec.js) is the import statements:

import { test, expect } from '@playwright/test'
import { visitHomePage } from '../lib/actions/nav'

test('can wait for an element to appear', async ({ page }) => {
  await visitHomePage(page)
  await page.waitForSelector('#elementappearschild', { state: 'visible', timeout: 5000 })
})

test('can use an element that appears after on page load', async ({ page }) => {
  await visitHomePage(page)
  const text = await page.textContent('#loadedchild')
  expect(text).toBe('Loaded!')
})

whereas in JavaScript the first two lines were:

const nav = require('../lib/actions/nav')
const { test, expect } = require('@playwright/test')

The main differences are in the library code.

import { Page } from '@playwright/test'
import config from 'config'

export async function visitHomePage (page: Page) {
  return await page.goto(`${config.get('baseURL')}`)
}

export async function goToPath (page: Page, path: string) {
  return await page.goto(`${config.get('baseURL')}/${path}`)
}

You will see the types being added, fortunately Playwright provides type definitions which are imported here.

Previously, this same file looked like:

const config = require('config')

async function visitHomePage (page) {
  return await page.goto(`${config.get('baseURL')}`)
}

async function goToPath (page, path) {
  return await page.goto(`${config.get('baseURL')}/${path}`)
}

module.exports = {
  visitHomePage,
  goToPath
}

The other main thing to update is if you use classes you define types against all the properties.

Everything else is about the same.

My fully working TypeScript example of Playwright Test is here: https://github.com/alisterscott/playwright-test-ts-demo

Enjoy TypeScripting 😀

Categories
Automated Testing

“Never Trust an Automated Test You Haven’t Seen Fail”

My number one philosophy when developing automated tests is:

“Never Trust an Automated Test You Haven’t Seen Fail”

I’ve found it can be easy to write a test that always passes, so forcing a test to fail as a starting point gives me confidence that it can fail.

I know of two ways to make a test fail:

  1. Change the implementation to fail and make sure the test fails at first – fixing the implementation fixes the test
  2. Change the assertion to make a test fail, make sure the actual result is correct and then update the assertion to make the test pass

I don’t really care what method I use to know a test fails, but I make sure I always make it fail. Never trust a test that hasn’t failed.

Categories
CSharp e2e Testing

Playwright in C# (.NET Core)

Whilst I was doing some reading of the Playwright docs I noticed they have C# bindings (as well as Python & Java, but not Ruby) – and since it’s been a couple of years since I’ve used C# I thought I’d take a look at how it works – especially considering .NET Core has support for Mac which makes working in C# .NET so much easier for me.

First I downloaded Visual Studio for Mac Community Edition which was pretty easy to install and this included the .NET Core framework which includes the dotnet command line tool.

One thing about .NET Core is there’s a lot more command line options to do things.

Installing Playwright?

dotnet tool install --global Microsoft.Playwright.CLI
playwright install

Creating a new NUnit Project?

dotnet new nunit -n PlaywrightNunitDemo

Adding Playwright, Building and Running Your Tests?

dotnet add package Microsoft.Playwright.NUnit
dotnet build
dotnet test

My tests end up looking like this:

using System.Threading.Tasks;
using Microsoft.Playwright.NUnit;
using NUnit.Framework;
using PlaywrightNunitDemo.lib;

namespace PlaywrightNunitDemo
{
    [Parallelizable(ParallelScope.Self)]
    public class Scenario02 : PageTest
    {
        [Test]
        public async Task CanCheckForErrors()
        { 
            string errors = await AppHelpers.VisitURLGetErrors(Page, "/error");
            Assert.AreEqual(": Purple Monkey Dishwasher Error", errors);
        }

        [Test]
        public async Task CanCheckForNoErrors()
        {
            string errors = await AppHelpers.VisitURLGetErrors(Page);
            Assert.AreEqual(string.Empty, errors);
        }
    }
}

and my reusable Playwright code can live in a class with static methods:

using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Playwright;

namespace PlaywrightNunitDemo.lib
{
    public class AppHelpers
    {
        public static async Task<IResponse> VisitURL(IPage page, string path = "/")
        { 
            var config = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
            string url = config["BASE_URL"] + path;
            return await page.GotoAsync(url);  
        }

        public static async Task<string> VisitURLGetErrors(IPage page, string path = "/")
        {
            var errors = "";
            page.PageError += (_, exception) => { errors = errors + exception; };
            await VisitURL(page, path);
            return errors;
        }
    }
}

Summary

I quite enjoyed working in C# in previous roles. There’s something nice about typed languages which makes you confident when writing the code that things will do what you expect. Having .NET Core easily accessible on a Mac and being able to use Playwright makes C# even nicer.

Example Code

My example code is: https://github.com/alisterscott/PlaywrightNunitDemo

I’ll see if I can get this running on Linux in CircleCI soon.

Categories
Career

Technical coding interviews for testers?

About a year ago I was asked to apply to work at a startup here in Brisbane. I’ve never really been that keen on the idea of working for a startup as I prefer stability over exhilaration, I’m not looking for rapid promotion and I don’t really care for the speculative wealth that startup employee stock options provide – but I was curious about the job as it was working with a well-respected person I’d known for a while so I went ahead with an interview.

Little did I know that it would be a coding technical interview – where candidates – testers included – connect to a web session where a blank IDE is opened and you’re given a problem verbally and you need to write code to solve it on the spot.

I can’t remember the exact problem(s) but they were something like writing a way to reverse strings, do custom sorts and some fairly complex coding problems.

The reason I don’t remember is I totally blanked. As an anxious person with post traumatic stress disorder, staring at an empty JavaScript IDE whilst the interviewer watches every keystroke you type to solve the problem on the spot felt a lot like this to:

Coding Interview?

The interview was like fuel on my burning imposter syndrome fire and the next six months of my life was spent ruminating and questioning whether I could even do my job as a technical QA.

This was despite the facts I highlighted when applying for the job:

  • I have 16 public GitHub code repositories including 7 in JavaScipt, 4 in Ruby and 1 in C#
  • I’ve been writing this blog since 2006 and have shared numerous code samples across years of blogging.
  • I successfully completed a 3 month trial to work at Automattic/WordPress.com during which I built the foundation for the first ever automated e2e test suite to be successful at WordPress.com

I’d suggest to startups considering to do technical on-the-spot coding examinations whether these are absolutely necessary considering the candidate and knowing that people may not perform well in that situation.

Whilst I raised this feedback after the interview and I could have actually progressed along (despite flunking the coding exercises) I decided I didn’t want to work somewhere that would do that to potential employees. I’ve never had to sit next to someone in my working career and have them watch every keystroke I made writing code from scratch. Instead I’ve worked at great places that have measured me by my great outputs (thoughts/code/prose), and outcomes (software quality) without worrying about how I generate those.

I noticed the person who asked me to apply left the startup a few months after my interview anyway 🙃

Aside: perhaps the diagram above may also explain how some people, including myself, don’t like pair-programming as the main way of delivering code?

Categories
Playwright

Playwright’s “retry with video”

Another feature I love in Playwright Test is the retry with video option.

In your playwright.config.js file you specify:

module.exports = {
  use: {
    headless: true,
    screenshot: 'only-on-failure',
    video: 'retry-with-video'
  },
  retries: 1
}

The video option combined with the retry option means that if a test fails Playwright will re-run it whilst recording a video of that re-run.

If the test fails on the second attempt it will capture the video in the test results (alongside the screenshot if you config it to capture screenshots also).

It’s a webm video that is lightweight, easy to capture during CI, and easy to share: