Do you have set up (inexpensive) infrastructure to store data collected in your automated tests? We are currently using using selenium Java webdriver to automate our tests and IntelliJ as our IDE. We create data from scratch for each and every test case 🙁
I’m a little confused by the question and whether it’s about test data: data is that is needed by the automated tests, or test results data: insights into the results of our automated tests. So I’ll answer both 😀
Infrastructure to manage test data
Our tests run on specific test accounts and sites on production databases. Since our tests are end-to-end in fashion, we try to make our tests have as few dependencies as possible on existing data. Often an end-to-end scenario will involve creating, viewing, editing and deleting something. If we don’t do all of this by our UI we can use hooks that either use services or database jobs to clean up the data. I explained this in more detail previously.
Infrastructure to manage test results data
We use CircleCI for automated end-to-end tests. We have a number of projects that run different types of end-to-end tests from the same code repository for different purposes (canary tests, visual-diff tests, full regression tests for example).
We generate x-unit test results (from Mocha/Magellan) which CircleCI uses to provide insights into our test results such as this:
You can also drill down into slowest tests and most failed tests etc.
Since all our tests are open source you can view these build insights yourself!
We’re pretty happy with the insights we get from CircleCI at the moment so we don’t see a need to currently develop anything ourself.