Hi Alister, I have been reading your blog for quite a while now. I find it quite insightful and engaging.
I wanted to get your opinion on a question (rather observation) I have after doing BDD on few projects as a tester.
If testing is acceptance driven/BDD, then all of the business rules are captured in acceptance criteria in stories and then in feature files when they are automated. Does this makes test cases redundant? Are they actually needed anymore?
In terms of traceability, a feature file clearly maps to a story and hence a business requirement so unless there is a legal requirement to have test cases documented by the client, I feel test cases to be a redundant effort, unless I am missing something. Very interested to hear your experience/thoughts on this.
My short answer is yes, test cases are redundant.
My long answer is also yes, here’s why:
Ideally I think every software system should aim to have no (zero) manual regression testing, that is you can do a software release and be confident you haven’t introduced any regressions. This isn’t 100% test coverage: this often isn’t practical, zero manual regression is having (just) enough automated tests for you to not have to do any manual regression testing. You still have to manually test new features of course, it’s the existing ones you don’t need to worry about.
When you’re in this situation, there’s no need to have manual test cases, since your automated tests act as living executable specifications of your system, and manual test cases would be redundant.
But what about new functionality? In a continuous delivery model, there’s no time to write manual test cases then execute them, and this is pointless anyway as they won’t live on past that user story which will have automated tests developed alongside it, so I typically test against the acceptance criteria defined for a user story (hopefully in collaboration with you), and add notes to a user story to show what other edge cases I envisioned and what I uncovered during story testing.
I have found having this level of documentation is more than enough even for the audit-heavy environments, as I have found auditors prefer to see automated test results against every build, then a pile of test cases in excel spreadsheets that are out of date.