Categories
Automated Acceptance Testing Automated Testing Business Analysis Conferences

Adventures of end-to-end automated web testing in Node.js

This is a presentation I gave at ATTAC in Melbourne on Friday 24th May 2019. The full Google Slides are available here.

Good morning everyone and thanks for having me along to speak.
My name is Alister Scott and I’m from Brisbane. I work as a QA generalist at a software company in Brisbane called Console, and I write a blog called WatirMelon. Today I’m going to be sharing my knowledge of end to end automated testing in Node.js. I’ve created very simple but working demos of the main tools I’ll be discussing today and I’ve put these on GitHub as separate repositories you can easily clone and play around with: github.com/alisterscott

Outside of work I enjoy hiking, often to the summits of mountains, and spending time with my wife and our three young kids.
In 2015 I started a paid trial at Automattic. My trial project – on which I would be assessed to gain full time employment – was to establish an automated e2e testing framework for WordPress.com – the first of its kind for Automattic. I quickly spun up a framework using Watir in Ruby – because that’s what I knew – and it worked. However I quickly gained some feedback that of the hundred+ developers at Automattic almost none knew Ruby and creating shared ownership for e2e tests would be a key measure of success so I had to rethink. At the time Automattic was moving from primarily developing in PHP towards Node.js meaning full stack JavaScript and it made sense that the e2e tests were also developed in Node.js.

This of course made my trial project a lot more difficult than I had originally thought as I had to teach myself Node.js and understand what testing tools existed in this space.

Fast forward to 2019 and earlier this year I decided to change jobs as I could no longer travel for work. When I started looking at job advertisements for testing and QE positions in Brisbane I noticed just how many mentioned Node.js as their technology stack of choice: in the 3.5 years at Automattic Node.js had become increasingly popular at other companies as well.
But automated e2e testing in Node.js was and is really hard. Much much harder than I was used to in Ruby and Watir where things just worked.

It’s slightly better in 2019 than 2015, however there are some reasons why it’s hard.
There’s also the paradox of choice when it comes to tooling specifically for e2e web testing. Searching for selenium and WebDriver on NPM provides a list of dozens of libraries – and the official Selenium bindings (WebDriverJs) don’t appear when searching for WebDriver. It’s all very confusing.

My aim today is to create some clarity in this space.
I believe there’s more e2e test tools in Node.js that something like Java or C#. I’ll talk about the four most popular/mature ones.
Cypress.io is better suited to component testing than true end-to-end testing. Cypress.io also doesn’t support cross-domain stuff – so beware if you’re doing anything like that.
You’ll need your own test runner and assertion library.
I’ve distilled these four tools down into an easy to read visual.
And an even easier to understand flow chart.
Categories
Conferences

ATTAC 2019: Australian Test & Tech Automation Conference

I’m excited to be presenting at this year’s ATTAC (Australian Test & Tech Automation Conference) in Melbourne on 24th May on establishing automated e2e web testing in Node.js:

Node.js is so hot right now. Whether you’re looking at quality engineering jobs at a new and upcoming startup, or at an established corporate, chances are you’ll see Node.js on the tech stack. But establishing automated e2e web testing in Node.js is hard since it’s very different to traditional web automation technology. In this talk Alister will use nearly four years of Node.js experience to bring clarity to the somewhat confusing Node.js landscape, going into detail on things like asynchronous promises, ECMAScript transpiling, npm vs yarn and monorepos.

Adventures of end-to-end automated web testing in Node.js #

I attended the first conference last year in Melbourne and loved the format (single day, single track), affordability (tickets from AUD$199) and technical subject matter.

Early bird tickets are on sale now. I hope to see you there!

Categories
Automated Testing Conferences

→ Scaling Selenium to infinity using AWS Lambda

A great presentation from the recent 2018 Selenium Conference with a few shout-outs to this blog. I really like what Wes and Kurt were able to achieve using AWS Lambda – amazingly fast e2e tests running in parallel.

Categories
Automated Testing Conferences

TestBash Sydney: Automated e2e Testing at WordPress.com

This is an approximate transcript of the talk I delivered at TestBash in Sydney on Friday 19th October 2018.

Alister Scott Test Bash Sydney Oct 2018-1-1

Today I’d like to share my story about how we started with automated end to end testing at WordPress.com since I started at Automattic over 3 years ago.

Alister Scott Test Bash Sydney Oct 2018 2-2

Categories
Automattic Conferences

Testbash Australia 2018

I only speak at one conference a year and this year that conference will be the first ever Australian Testbash in Sydney on October 19, 2018:

TestBash_Australia_2018_Adverts_DOJO_EVENT_BANNER.png

My talk:

At WordPress.com we constantly deliver changes to our millions of customers – in the past month alone we released our React web client 563 times; over 18 releases every day. We don’t conduct any manual regression testing, and we only employ 5 software testers in a company of ~680 people with ~230 developers across . So, how do we ensure our customers get a consistently great user experience with all this rapid change?

Our automated end-to-end (e2e) tests give us confidence to release small frequent changes constantly to all our different applications and platforms knowing that our customers can continue to use our products to achieve their desires.

Alister will share how these automated tests are written and work, some of the benefits and challenges we’ve seen in implementing these tests, and what we have planned for future iterations.

Takeaways: How to effectively use automated end-to-end testing to ensure a consistent user experience in high frequency delivery environments

Grab a ticket before they sell out #

Categories
Conferences

Staying focused with one conference presentation per year

I recently saw Basecamp made their employee handbook available online1 and this statement stood out to me in their moonlighting guide:

Not OK

2. Go out on a regular speaking circuit tour which is going to require multiple days of travel multiple times a year. That’s too disruptive to our own work schedule and to your fellow teammates.

For this exact reason, without knowing, I created my own personal rule2 of only committing to doing one conference presentation per year.

Categories
Automattic Conferences

The future of testing is distributed

This are my slides during my presentation on distributed testing at the ANZTB Test 2017 Conference in Wellington, New Zealand last Friday 5th May 2017.

Alister Scott - The Future of Testing is Distributed FINAL(1) 01

Categories
Ask Me Anything Conferences Life Presentations

AMA: Any more talks planned?

Anonymous asks..

I learned who you were by watching your Google automation talk last year in 2015. Your presentations are really nice. Are you planning on giving any other presentations this year or next year?

My response…

My short answer is no.

My long answer is also no because I actually don’t actually enjoy giving presentations at all. I wrote about my battles with anxiety last year and whilst I am 90% better than I was, last year I committed to present three talks in less than two months which resulted in me having panic attacks about giving these talks. This wasn’t fair on my wife or children who I need to support on a day-to-day basis.

Each talk requires a huge amount of preparation and since my personality leans towards perfectionism I wanted to make sure each talk was as good as it could be, so I wrote every word of each talk and (unsuccessfully) tried to memorise these. This resulted in me delivering the talks partly reading what I’d prepared, which I wasn’t happy about as I was comparing myself to others who delivered their talks without notes.

The reason people give talks is that speech is an amazingly effective communications tool – probably the most so – yet it’s a drastically inefficient communications tool – each minute of a talk requires at least an hour of preparation. I much prefer written communication as I find confidence in writing, and I hope with frequent, thoughtful updates to my blog I can reach a wide audience and still be effective in spreading new ideas.

Categories
Ask Me Anything Automated Testing Conferences

AMA: why flaky tests for GTAC?

dmcnulla asks…

I enjoyed your GTAC presentation on “your tests are not flaky“. How did you pick your topic?

My response…

Firstly thanks for the compliment, you’re very kind.

When I came up for the topic I was working on a system where we were practicing continuous delivery by frequently doing production releases. As we began releasing more frequently the business expected this and so the reliability of our automated tests became more important. We wouldn’t release on a failed build since we were working on a high volume eCommerce site where a small bug could cause an outage costing a very large amount of revenue. We didn’t have a team of testers to fall back onto for any manual regression testing, so we were 100% dependent on our automated tests.

Even though we were clever about building testability into our system, we still had too many full-stack automated tests which would create non-deterministic results.

I believe everyone looks at the same thing slightly differently as we each have a unique lens that we see our world through and everyone’s lens has varying degrees of difference:

“Each of us tends to think we see things as they are, that we are objective. But this is not the case. We see the world, not as it is, but as we are—or, as we are conditioned to see it. When we open our mouths to describe what we see, we in effect describe ourselves, our perceptions, our paradigms.”

~ Stephen R. Covey

As people who were developing and maintaining tests, we were looking at our non-deterministic tests as the test’s fault. What we didn’t do was look through another lens to see that it could actually be the fault of our system as we had built it instead.

This aha! moment struck me when we released a bad build to production that had passed all automated QA by someone re-running our automated tests a number of times (to get them to pass).

We were blinded by perceived ‘test flakiness’: we refused to believe our problems were something else, so I thought it would be a good topic to present. From the feedback I received both at and after the event, it seems I am very much not alone.

 

Categories
Automated Testing Conferences

Highlights from CukeUp! Sydney 2015

I spent the last two days at CukeUp! 2015 in Sydney, in the beautiful but hot Cell Block Theater.

Here’s some of my highlights.