Categories
Agile Software Dev

BDD in 2020

It’s 2020: can we officially declare Behaviour Driven Development (BDD) dead?

How did we get here?

“There is no trap so deadly as the trap you set for yourself.”

Raymond Chandler, Long Goodbye

BDD’s lure was the promised land of collaboration between business and technology defining expected behaviour as executable specifications (automated tests). These automated tests defined our software development processes: outside-in software development, behaviour driven development, acceptance driven test development and specifications by example.

BDD’s trap was a bunch of Cucumber specifications written after the fact that create another layer to automate tests in a browser, hidden away in source code which are seldom, if ever, seen by the business, and either too comprehensive, flaky and expensive, or narrowly focused but lacking in enough detail for a complete executable specification of a system.

Sure BDD tests can drive things besides the browser, but the overhead and complexity of maintaining these outweighs the benefits they serve, especially when there is still a need for more expensive integration/e2e tests anyway.

So what actually works in 2020?

I’ve found this process seems to work really well for modern software development in 2020:

  1. Every new piece of work has a kick-off discussion with the product owner, the tester and the developer(s)
  2. The tester usually facilitates this meeting and aims to produce written lightweight documentation during the discussion containing the following information: background info, scenarios, business rules, examples (edge cases), questions and decisions.
  3. This documentation is enough for development work to begin, and the tester to plan how they’ll test it.
  4. The documentation is updated as work progresses: questions arise and decisions are made (and documented).
  5. The developer(s) is/are responsible for making sure there is automated test coverage for these business rules at the cheapest level of testing, these are typically automated unit and/or automated integration tests. These aren’t written in business readable language, they’re test code alongside production code.
  6. The developer also needs to ensure any existing automated end-to-end (e2e) tests are updated for the work that is happening – CI builds highlight these impacts/failures.
  7. When the feature, or a working part of the feature, is ready for testing the developer and tester do a quick handover/walkthrough making sure what was done is what was discussed.
  8. The tester uses the lightweight documentation as a basis for their testing, but adds extra rigour in testing edge cases/unknowns, other impacted areas, different browsers, performance, and devices and anything else they think could be risky.
  9. Bugs that are found and determined by the product owner to be crucial are fixed immediately, some bugs become “fast follows” which are the next thing to ship. Some bugs aren’t fixed.
  10. The tester gives a walkthrough to the product owner and sometimes UX designer to make sure what’s been done is acceptable – if the feature is new/risky it’s launched to a subset of customers and can be disabled at a flick of a toggle.
  11. When an entirely new feature is added to the system and it’s important enough to warrant an (expensive) automated e2e test – the tester works on adding that in amongst their other work. This isn’t a blocker to release and provides a safety blanket for future changes. This isn’t a comprehensive specification of the system – it’s an e2e automated test that covers a business critical flow over multiple moving parts and again doesn’t need to be business readable.
  12. The specifications/documentation lives on in a wiki page and can be referred back to when testing future changes to this feature, or when support tickets or bugs are found in production.

BDD isn’t working for us, what can I do now?

  1. Make sure you commit to having kick-offs and encourage robust discussion creating lightweight documentation as you go – remove uncertainty and create clarity by making and documenting questions and decisions.
  2. Keep your lightweight specifications up to date as you go – don’t worry about making them executable – let people write automated tests in the way that is the most efficient for them.
  3. Only use automated e2e tests to cover key user flows in the most efficient way – don’t use these to create/maintain comprehensive living documentation.

Also See:

Categories
Automated Acceptance Testing Specflow WebDriver

Five automated acceptance test anti-patterns

Whilst being involved with lots of people writing automated acceptance tests using tools like SpecFlow and WebDriver I’ve seen some ‘anti-patterns’ emerge that can make these tests non-deterministic (flaky), very fragile to change and less efficient to run.

Here’s five ‘anti-patterns’ I’ve seen and what you can do instead.

Anti-pattern One: Not using page-objects

Page objects are just a design pattern to ensure automated UI tests use reusable, modular code. Not using them, eg, writing WebDriver code directly in step definitions, means any changes to your UI will require updates in lots of different places instead of the one ‘page’ class.

Bad

[When(@"I buy some '(.*)' tea")]
public void WhenIBuySomeTea(string typeOfTea)
{
Driver.FindElement(By.Id("tea-"+typeOfTea)).Click();
Driver.FindElement(By.Id("buy")).Click();
}

Better

[When(@"I buy some '(.*)' tea")]
public void WhenIBuySomeTea(string typeOfTea)
{
     MenuPage.BuyTea(typeOfTea);
}

Complicated set up scenarios within the tests themselves

Whilst there’s a place for automated end-to-end scenarios (I call these user journies), I prefer most acceptance tests to jump straight to the point.

Bad

Scenario: Accept Visa and Mastercard for Australia
 Given I am on the home page for Australia
 And I choose the tea menu
 And I select some 'green tea'
 And I add the tea to my basket
 And I choose to checkout
 Then I should see 'visa' is accepted
 And I should see 'mastercard' is accepted

Better

This usually requires adding some special functionality to your app, but the ability for testing to ‘jump’ to certain pages with data automatically set up makes automated tests much easier to read and maintain.

Scenario: Accept Visa and Mastercard for Australia
 Given I am the checkout page for Australia
 Then I should see 'visa' is accepted
 And I should see 'mastercard' is accepted

Using complicated x-path or CSS selectors

Using element identification selectors that have long chains from the DOM in them leads to fragile tests, as any change to that chain in the DOM will break your tests.

Bad

private static readonly By TeaTypeSelector =
            By.CssSelector(
                "#input-tea-type > div > div.TeaSearchRow > div.TeaSearchCell.no > div:nth-child(2) > label");

Better

Identify by ‘id’ (unique) or ‘class’. If there’s multiple elements in a group, create a parent container and iterate through them.

private static readonly By TeaTypeSelector = By.Id("teaType");

Directly executing JavaScript

Since WebDriver can directly execute any arbitrary JavaScript, it can be tempting to bypass DOM manipulation and just run the JavaScript.

Bad

public void RemoveTea(string teaType)
{
  (driver as IJavaScriptExecutor).ExecuteScript(string.Format("viewModel.tea.types.removeTeaType(\"{0}\");", teaType));
  }

Better

It is much better to let the WebDriver control the browser elements which should fire the correct JavaScript events and call the JavaScript, as that way you avoid having your ‘test’ JavaScript in sync to your ‘real’ JavaScript.

public void RemoveTea(string teaType)
{
  driver.FindElement(By.Id("remove-"+teaType)).Click();
}

Embedding implementation detail in your features/scenarios

Acceptance test scenarios are meant to convey intention over implementation. If you start seeing things like URLs in your test scenarios you’re focusing on implementation.

Bad


 Scenario: Social media links displayed on checkout page
   Given I am the checkout page for Australia
   Then I should see a link to 'http://twitter.com/beautifultea'
   And I should see a link to 'https://facebook.com/beautifultea'
 

Better

Hide implementation detail in the steps (or pages, or config) and make your scenarios about the test intention.


 Scenario: Social media links displayed on checkout page
   Given I am the checkout page for Australia
   Then I should see a link to the Beautiful Tea Twitter account
   And I should see a link to the Beautiful Tea Facebook page
 

I hope you’ve enjoyed these anti-patterns. Leave a comment below if you have any of your own.