Categories
Career

Technical coding interviews for testers?

About a year ago I was asked to apply to work at a startup here in Brisbane. I’ve never really been that keen on the idea of working for a startup as I prefer stability over exhilaration, I’m not looking for rapid promotion and I don’t really care for the speculative wealth that startup employee stock options provide – but I was curious about the job as it was working with a well-respected person I’d known for a while so I went ahead with an interview.

Little did I know that it would be a coding technical interview – where candidates – testers included – connect to a web session where a blank IDE is opened and you’re given a problem verbally and you need to write code to solve it on the spot.

I can’t remember the exact problem(s) but they were something like writing a way to reverse strings, do custom sorts and some fairly complex coding problems.

The reason I don’t remember is I totally blanked. As an anxious person with post traumatic stress disorder, staring at an empty JavaScript IDE whilst the interviewer watches every keystroke you type to solve the problem on the spot felt a lot like this to:

Coding Interview?

The interview was like fuel on my burning imposter syndrome fire and the next six months of my life was spent ruminating and questioning whether I could even do my job as a technical QA.

This was despite the facts I highlighted when applying for the job:

  • I have 16 public GitHub code repositories including 7 in JavaScipt, 4 in Ruby and 1 in C#
  • I’ve been writing this blog since 2006 and have shared numerous code samples across years of blogging.
  • I successfully completed a 3 month trial to work at Automattic/WordPress.com during which I built the foundation for the first ever automated e2e test suite to be successful at WordPress.com

I’d suggest to startups considering to do technical on-the-spot coding examinations whether these are absolutely necessary considering the candidate and knowing that people may not perform well in that situation.

Whilst I raised this feedback after the interview and I could have actually progressed along (despite flunking the coding exercises) I decided I didn’t want to work somewhere that would do that to potential employees. I’ve never had to sit next to someone in my working career and have them watch every keystroke I made writing code from scratch. Instead I’ve worked at great places that have measured me by my great outputs (thoughts/code/prose), and outcomes (software quality) without worrying about how I generate those.

I noticed the person who asked me to apply left the startup a few months after my interview anyway 🙃

Aside: perhaps the diagram above may also explain how some people, including myself, don’t like pair-programming as the main way of delivering code?

Categories
Career

Software Testing Jobs June 2021

Even though I’m not looking for a new job, I like to browse and read job advertisements as I’ve found it’s a good way to take control of your career as it gives you insights into what future employers are looking for in currently advertised roles.

In this light, I recently did some keyword analysis of the 20 most recent software testing jobs advertised in Brisbane, Australia (the job market seems 🔥 so these were all advertised in the last week or so) attempting to find some insights into what is in demand and what’s happening in the world of software testing (in Brisbane, Australia).

Type of Jobs

Most of the 20 most recent jobs were permanent full time roles. As a part time employee myself I found it interesting none offered or mentioned the flexibility of part time hours.

Salary Advertised

Only 4 of the 20 roles included any information about salary. This saddens me as it pushes the burden of salary negotiation back onto the candidate (I am yet to meet an employer who wants to pay more than bare necessary!)

The four roles that indicated salary were:

  1. $80,000 – $99,999 AUD (Software Quality Assurance Specialist)
  2. $95,000 – $110,000 AUD (Senior Test Analyst)
  3. $80,000 – $109,999 AUD (QA Engineer)
  4. “Up to” $125,000 AUD (Senior QA Engineer)

Location of Work

Despite 2020/2021 being touted as the years the world went fully remote, interestingly a majority of the roles indicated working in the office full time, 6 of 20 offering a “hybrid” model typically being 1-2 days WFH per week (often described as a “work life balance” or “flexibility” perk), and only 1 of 20 was fully remote (no office or expectation to work in the office). What happened?

Job Category

There was a good split between roles advertised as test management, test engineering, and test analysis. However, almost all roles required test automation skills (see next section).

Test Automation Skills Required

This was probably the most and least surprising result to me. Least surprising as there’s been a big shift to technical testing roles in the industry which I’ve been talking about for some time. And most surprising in that even the roles categorized as “test management” and “test analysis” predominantly required test automation skills. Of the four that didn’t mention the requirement for test automation skills three were test analysis and only one was test management.

An example job description:

“As a Test Manager, you will be a hands-on leader with a broad skill-set and knowledge in test methodologies and automation development. “

Qualifications/Certifications

  • 3 of 20 jobs required tertiary IT qualifications
  • 1 of 20 jobs required ISTQB certifications

Automated Testing Tools

  • 6 of 20 jobs mentioned Selenium
  • 1 of 20 jobs mentioned Cypress 😢
  • 1 of 20 mentioned TOSCA 😢😢
  • 0 of 20 mentioned Playwright or Puppeteer 😢😢😢

Buzzword Bingo

  • 9 of the 20 jobs mentioned “flexibility” in their role
  • 11 of 20 mentioned “agile”
  • 7 of 20 mentioned either “high-performing, fast-paced, dynamic or fast-moving” teams and culture

Favourite Description

My favourite description of culture was:

“<employer name> recognises the critically important role of testers in the delivery of top quality applications. Our testers have to be sharp to hunt out issues and work with developers and business stakeholders to deliver truly awesome applications into production, but also mentoring others in the development of their own QA mindset. “

followed by

“We recognise that life is easier when cash flow is easy. That’s why we’ll pay you weekly – not fortnightly or monthly like other companies.”

Least Favourite Descriptions

(in no particular order – verbatim with emphasis added)

  • “You will feel happier & healthier for working at <company name>!
  • “As a Test Manager, you will be a hands-on leader with a broad skill-set and knowledge in test methodologies and automation development.”
  • “Salary and super flexible based on experience
  • Supervise the delivery of high quality production optimised code. “
  • “We are a friendly bunch here at XXX, so we expect you to be a good collaborator, strong communicator and willing to roll up your sleeves and focus on reaching our common goals.”
  • “We are looking for an experienced Automation specialist to join this organisation and help in ensuring thigh quality delivery of their software.” 🍗
  • “You will be working ac across multiple projects within a high paced environment, contributing to team success and having plenty of flexibility within a great company.” 🆙
  • “You’re not afraid of hard work. Curly challenges. Sometimes inconvenient hours.” 😬
  • “During your interview, you’ll be able to tell us the capital city of Bolivia” 🤦‍♂️

Closing Thoughts

I enjoyed this exercise more than I thought. The things that most surprised me was how few jobs were remote in 2021, and also how many roles required test automation skills, particularly jobs at the test management level: is every test manager role “hands on” nowadays?

Categories
Automated Testing Career Software Testing

Creating a skills-matrix for t-shaped testers

I believe the expression “jack of all trades, master of none” is a misnomer, as I’ve mentioned previously. Being good at two or more complimentary skills is better than being excellent at just one, in my opinion.

But what about being excellent at one skill, and still being good at two or more? Why can’t we be both?

Jason Yip describes a T-shaped person and the benefits that having t-shaped people on teams brings:

A T-shaped person is capable in many things and expert in, at least, one.
As opposed to an expert in one thing (I-shaped) or a “jack of all trades, master of none” generalist, a “t-shaped person” is an expert in at least one thing but also somewhat capable in many other things. An alternate phrase for “t-shaped” is “generalizing specialist”.

jason yip
image by Jason Yip

Ideally we’d like to have a team of t-shaped testers in Flow Patrol at Automattic. But how do we get to this end goal?

I recently embarked on an exercise to measure and benchmark our skills and do just this with our team. Here’s the steps we took.

Step One – Devise Desired Team Skills

The first thing we did was come up with a list of skills that we have in the team and would like to have in the team. These can be ‘hard’ skills like a specific programming languages and ‘soft’ skills like triaging bugs. In a standard co-located team this would be as easy as conducting a brainstorming session and using affinity grouping to discover these skills. In a distributed environment I wrote a blog post to my team’s channel and had individual members comment with a list of skills they thought appropriate, and then I did the grouping and came up with a draft list of skills and groups.

Step Two – Self-assess against a team skills matrix

Once I had a final list of skills and groups (see below for full list), I put together a matrix (in a Google Spreadsheet) that listed team members on the x-axis, and the skills on the y-axis, and came up with a skill level rating. Our internal systems use a three level scale (Newbie, Comfortable, Expert) which we didn’t think was broad enough so we decided upon five levels:

1. Limited
2. Basic
3. Good
4. Strong
5. Expert

 

skills_matrix
Team Skills Matrix

I hadn’t seen Jason yip’s visual representation at that point in time, otherwise I may have used something like that, which has five similar levels:

matrix jason yip
Image by Jason Yip

Step Three – Publish results and cross-skill

Once we had the self assessments done we could then publish the data within our organisation and use the benchmark to cross-skill people in the team. In a co-located environment this could involve pair programming, in a distributed one it could involve mentoring and reviewing other team member’s work.

Have you done a skills matrix for your team? How did you do it? What did you discover?


Full List of Skills and Skill Groups for Flow Patrol at Automattic

Automattic Product Knowledge
WordPress Core
WordPress.com Simple Sites
WordPress.com Atomic Sites
Jetpack
Woocommerce
Simplenote
Mobile Apps
Human Software Testing
Flow Mapping
Bug Triage & Prioritization
Exploratory Testing (pre-release)
Dogfooding
Cross-browser Cross-device Testing
Facilitating Beta/Community Testing
Facilitating User Testing
Usability Testing
Accessibility Testing
Automated Testing
Automated End-to-end Browser Testing
Automated API/Integration Testing
Automated Unit Testing
Automated Visual Regression Testing
Android Automated Testing
iOS Automated Testing
Programming Languages
JavaScript
PHP
Shell Scripting
Objective C
Swift
Android/Kotlin
Testing Tools/Frameworks
Mocha
WebDriverJS
Git/Github
CircleCI
TravisCI
Team City (CI)
Mailosaur
Applitools
VIP Go
Docker
Other
i18n Testing
Performance Testing
Security Testing
User advocacy – empathy and compassion
Mentoring/onboarding
Project Management
Product Management
Product Development 
Calypso
Jetpack
WP.com API PHP
Woocommerce
iOS App
Android App

 

Categories
Ask Me Anything Automated Testing Career Exploratory Testing

AMA: What sets exceptional QA testers apart?

Dayana asks…

I wondered if you could tell me what sets exceptional QA testers apart? Not just personality or work ethic traits, but specific skills and programming knowledge that will be very valuable to a team?

My response…

I think exceptional QA testers, as explained recently, aren’t people who are exceptional at just one thing, eg. testing, but good at lots of things.

So an exceptional QA tester, in my opinion, will typically have (at least good) skills in the following things:

  1. Skills in human exploratory testing: an exceptional QA tester has the ability to effectively find the most important bugs fast. Whilst this skill can be developed, I have found it’s mostly a mindset.
  2. Skills in developing automated tests: an exceptional QA tester will have programming skills needed to develop automated tests and I would recommend these to typically match the programming language(s) that programmers in your organization use. For example, skills in automated testing in .NET if your company primarily uses Microsoft .NET. Although, someone with strong programming skills in one language (eg. ruby) should be able to transfer these skills to another language (eg. python).
  3. Knowledge/Experience in your business domain: an exceptional QA tester will fully understand your business domain and keep this context in mind whilst testing a product and raising issues. An exceptional tester is always testing your system – just as I am testing WordPress.com publishing this post.
  4. An empathetic mindset: we design and develop software for real people and real life. An exceptional QA tester will test with this in mind.
Categories
Automated Testing Career Software Testing Test Automation

(Not) Lying about Writing Code

I recently saw this quote in an article by Nikita Hasis on Medium.

“If Your Test Leaders Aren’t Telling You To Write Code, They Are Lying!
Even if it’s by omission.

There’s this argument, almost daily, about whether software testers should learn programming. I’ll jump right in. It is unimaginable that someone would tell you NOT to learn something. That’s the first, and probably shittiest lie that inexperienced testers get fed. It’s further unimaginable, and downright irresponsible to tell people not to learn something that is very clearly where a large, well-paying, and above all interesting part of the industry is heading. Wanna work on innovative, data-driven projects with smart and driven people? You probably need to pull up terminal and at least get your toes wet, y’all.

The worst part of the lie is that it imposes that coding is a difficult grind and will only cause more problems than it solves. I even saw Alister Scott’s blog post referenced as an argument against coding, ironic as it is.”

~ Nikita Hasis (Medium)

Since Medium is a walled garden that doesn’t allow you to leave a comment without creating an account I’ll leave my response here instead (where anyone is free to comment however they like).

Categories
Ask Me Anything Automated Testing Career Exploratory Testing

AMA: the future of QA roles

Lroy asks…

How, according to you, is the future of QA roles going to look like. I currently work as a QA in a matured agile team where the devs are responsible enough to practice TDD and write automated tests while I pair with them. I pair with them on test strategy, performance testing, a bit of security testing. It is definitely an interesting shift to see how QA was perceived to how it is now. I understand that it is not like this in every company. But how do you see this role pan to to be? Thanks PS: Thank you for taking time to do AMA, it is really interesting to read your responses. I have enjoyed reading your blog for couple of years now.

Categories
Ask Me Anything Career

AMA: describing our work

Chris Stanbridge asks…

Hi Alister, Long time fan of this site and your work – you’ve become the quality go-to source of truth for our scrum team. I think I have a similar role as your day job, but I have a quandary: I’m unable to describe what I do at work. Friends and family lose attention part way through my explanation, and I’ve found it has made me appear to be a crashing bore. I’ve developed a recurring nightmare where I fail to summarise the nature of my work to my demons (game-show hosts or stand-up comedians who pick me out of a crowd) who just don’t get it. I thought I was being clever by saying I use a “robot” to check if computer code works, but the picture painted using that word is considerably off the mark. If only I was a plumber… I’m sure you can provide me with a concise blurb I can memorise and plagiarise for the next neighborhood barbeque that’ll make me appear as interesting as I think I am 🙂 Cheers, Chris PS: Please keep up the excellent and entertaining blog – you’re an inspiration and an education.

My response…

Well, my job title at Automattic is actually Excellence Wrangler which is both awesome (in that no-one has heard of it before), and terrible (in that I have to explain what it is to everyone who asks).

I will often say something quick like “I work with various software teams to create a consistently pleasant user experience on WordPress.com”.

If that’s not sufficient detail I can explain that we continually do lots of testing of various WordPress.com user flows across different devices and browsers trying to find issues and bugs before our users do. We also develop automated scripts that run all the time to help us find these issues quickly since we make changes so frequently.

I’m not sure if that will help you, but since you’re in Brisbane too, let’s catch up and discuss further over lunch and/or tea 😀

 

Categories
Ask Me Anything Career

AMA: QA tools and resources when starting out

Nanthida asks...

Hey Alister!
I’m new to software testing, having come from a Support background, and I’d love to know what tools and/ resources worked best for you when you were starting out? 🙂

My response…

I very recently answered an almost identical question, so I will point you to my response.

If you have any follow-up questions, feel free to leave a comment below.

Categories
Ask Me Anything Career Software Testing

AMA: how to cope as a solo tester

Bodda asks…

hey, i like your blog, i’m reading your blog on a daily basis, and i just want to ask you what i can i do to enhance my skills, knowledge and to be a be good in functional testing (manual and automation) IF i’m the ONLY tester in my current company (performing all testing activities), but i feel that i have a mess in my head and lots to learn to be up to date with the last trends.
my question now “What to learn, When(everyday?) and How in case your are the only tester in your company”.

i hope to answer my question soon to make my head calm down

My response…

I have fond memories of a project where I worked as the solo tester on a software delivery team; we had something like 8 developers (including one lead who took on iteration management tasks) and me as a tester. That was it. I loved it because we established really good unhindered rapport with our business stakeholders, and I was always busy!

But getting back to your question: when I was working in that situation it was vital that we had developers working on as much test automation as possible, alongside functional development, since there was just too much functional testing to do for a single person also doing test automation. I found myself spending about 80% of my time just testing new functionality and spot-testing different browsers/devices etc. The remaining 20% I spent ensuring we had good regression test coverage through the automated tests that developers were writing and I was helping maintain. So first and foremost I would strongly encourage you to get the developers you work with to have as much responsibility as possible for automated tests.

If you have this in place you should be able to take a deep breath and spend more time doing quality testing. I have found the best way to learn is on the job, you’ll be in a good position to do this, and often you’ll learn best by making your own mistakes.

learning - 1
via The New Yorker

If you want to know more about how to become better at what you do, I’ve shared some tips in a previous answer. There’s also my Pride & Paradev book which I wrote whilst working as a solo tester on a team.

All the best.

Categories
Ask Me Anything Career

AMA: junior QA professional development

Mark asks…

What advice would you give to a talented junior QA who wanted to advance their career in software testing? Assume that they have no formal training in testing and have worked their way up from a junior IT role in an unrelated area.

My response…

I strongly believe you create what you will, so if someone’s willing to take control of their own testing career then they will have a mighty fine one at that.

I find there’s two key components to a career in software testing, and they’re quite opposite roles/skills so it’s worthwhile focusing on both:

  1. Developing skills in human exploratory testing: effectively finding the most important bugs fast. Whilst this skill can developed, I have found it’s mostly a mindset.
  2. Developing skills in test automation: having the ability to collaborate on and implement automated regression tests so a tester can spend more time on point 1.

In terms of resources for professional development for human testing for a tester I would recommend the following:

Books: some books cover the basics: agile testing, specification by example, pride and paradev.

Blogs: I’ve mentioned some blogs I read. There’s also the fantastic 5blogs feed as well which covers some great testing articles each day – so they can follow that.

Conferences: there’s a good site which lists every testing conference, and coming up in Australia is Australian Testing Days 2016 and the ANZTB Conference both next month and both in Melbourne, and CukeUp! Australia later in the year. Internationally I recommend GTAC and the Selenium Conference(s).

In terms of test automation skills, I am finding software developers are much more commonly interested in automated testing than I have found say 5 or 10 years ago. This has two benefits:

  1. Software developers can up-skill testers on their technical programming and automation tasks; and
  2. Software developers can take on some of this responsibility so technical testers can spend more time testing

In terms of building up test automation skills on one’s own, it’s never been easier or more accessible.

Two of the most popular testing tools of all time (Selenium/WebDriver and Cucumber/Specflow) are free and open source and available in all mainstream programming languages and platforms so there really isn’t an excuse not to learn them on one’s own.

If someone was just starting out and platform-agnostic I’d recommend starting with cucumber and watir-webdriver in ruby (because it’s so easy) and buy Cheezy’s book ‘Cucumber & Cheese‘ to learn how.

If you combine all of these together with a willing attitude it’s easy to develop great testing skills.

Finally, Nas has some good career advice:

I know I can
Be what I wanna be
If I work hard at it
I’ll be where I wanna be