Categories
Specification by Example

Given/When/Then with Style?

Gojko Adzic has started a series of challenges on the Specflow blog where readers can submit solutions to common problems writing automated tests using gherkin (Given/When/Then).

The first challenge is:

“What’s the best way to describe a Given for a value that’s not supposed to be there? For example, starting the specification from a point of a user who does not yet exist in a database, or a product that’s not been set up. “Given a user that doesn’t exist” sounds vague, but “Given user John that does not exist” sounds silly – how can we talk about John if he doesn’t exist yet?”


Gojko Adzic
Given user John does not exist
When John tries to register
Then the registration is successful

Given user John exists
When John tries to register
Then the registration fails

How would you improve the scenario of our example to solve the challenge of this week?

Gojko Adzic

Interestingly this problem highlights the issues I found using Gherkin to specify business rules and examples I previously wrote about:

“I’m not keen on writing specifications in gherkin (Given/When/Then) as I think it’s too generic and frequently makes the feature specifications too verbose – and takes emphasis away from the critical parts.”

Example of lightweight feature documentation

I prefer bullet points and tables.

I’d specify the above scenarios as follows.


User Registration

Rules

  1. A user can only register a single account per unique email address
  2. A user’s email address is their username
  3. A user can register using their Google or Facebook account without having to enter a username or password
  4. A user must enter a password unless they are registering using Google or Facebook
  5. Passwords must contain at least 4 letters and 4 digits
  6. Passwords are maximum length of 10 and can contain any ASCII characters

Registration Scenarios/Examples

ScenarioExample Email AddressResult
New user signs up with a new email new.customer@hotmail.com✅ Registration Successful
New user signs up with a new email and invalid password new.customer@hotmail.com❌ Registration Unsuccessful – Validation Error Displayed and no account created
User attempts to sign up with email already registered email for account with password existing.customer@hotmail.com❌ Registration Unsuccessful – Reset password page displayed with email prefilled
User attempts to sign up with email already registered email for account with Google log-inexisting.customer@gmail.com❌ Registration Unsuccessful – Log in with Google page displayed

It’s tempting to put passwords into the above table, which is what I originally did, but I’ve found it’s better to specify those separately (in the same document).


Passwords Scenarios/Examples

Password ScenarioExample PasswordValid/Invalid
Less than 8 characters123
Less than 4 digitsaaaabbbb
Less than 4 letters123456aa
No emoji – ASCII Chars Only1234abcd💀
More than 10 characters123456abcdef
10 Characters with ASCII1234abcd%$
4 digits and 4 lettersabcd1234
Special characters don’t count as lettersabc$1234

Summary

Whilst these scenarios aren’t gherkin I don’t think they need to be. They could be taken by developers and written as unit tests alongside the code. I wouldn’t have an e2e automated test that tests this logic at this level – it’s complex and an expensive waste of time.

I’ll be interested to see the follow up posts about the best way to specify these in gherkin, as I can’t see a clearer way than just using bullet points and tables 🙌

Categories
Ask Me Anything ATDD Business Driven Cucumber Specflow Specification by Example

AMA: adopting a TDD/BDD approach

João Monteiro asks…

I recently joined a small company where I am the only QA and the test automation suite was already written in a given/when/then style but rarely (if not at all) gets read by the rest of the team (product or developers). Any tips on how to mentor the team to adopt a BDD approach? Do you recommend any tools / framework to share the features in a centralised place easily accessible by the rest of the team an not just on the tests repository?

Categories
Business Driven Specification by Example

The color of acceptance is gray

James Shore recently wrote some brillant words about acceptance testing:

I think “acceptance” is actually a nuanced problem that is fuzzy, social, and negotiable. Using tests to mediate this problem is a bad idea, in my opinion. I’d rather see “acceptance” be done through face-to-face conversations before, after, and during development of code, centering around whiteboard sketches (earlier) and manual demonstrations (later) rather than automated tests.

To rephrase: “acceptance” should be a conversation, and it’s one that we should allow to grow and change as the customer sees the software and refines her understanding of what she wants. Testing is too limited, and too rigid. Asking customers to read and write acceptance tests is a poor use of their time, skill, and inclinations.

This is pretty much where my head is at right now around automating acceptance tests. Automated tests are black and white, acceptance is gray.

“The color of truth is gray.”
~ André Gide

I prefer to have a handful of end to end automated functional tests that cover the typical journey of a user than a large set of acceptance tests constantly in a state of flux as the system is being developed and acceptance is being defined and changed.

We need to take feedback from the customer that we are building the right thing and ensure our automated tests model this, not make them responsible for specifying the actual tests.