close
Punished By Rewards Punished By Rewards
Search
The Cost of Acceptance Tests

[dropcap]S[/dropcap]ometimes you read a section from a book and feel as if the author had directly interviewed you. Such is the case when I read the following passage from a very good book on Continuous Delivery. Here’s the passage.
[quote]

In many organizations where automated functional testing is done at all, a common practice is to have a separate team dedicated to the production and maintenance of the test suite. As described at length in Chapter 4, “Implementing a Testing Strategy”, this is a bad idea. The most problematic outcome is that the developers don’t feel as if they own the acceptance tests. As a result, they tend not to pay attention to the failure of this stage of the deployment pipeline, which leads to it being broken for long periods of time. Acceptance tests written without developer involvement also tend to be tightly coupled to the UI and thus brittle and badly factored, because the testers don’t have any insight into the UI’s under- lying design and lack the skills to create abstraction layers or run acceptance tests against a public API.

The reality is that the whole team owns the acceptance tests, in the same way as the whole team owns every stage of the pipeline. If the acceptance tests fail, the whole team should stop and fix them immediately.

[one_third][/one_third]One important corollary of this practice is that developers must be able to run automated acceptance tests on their development environments. It should be easy for a developer who finds an acceptance test failure to fix it easily on their own machine and verify the fix by running that acceptance test locally. The most common obstacles to this are insufficient licenses for the testing software being used and an application architecture that prevents the system from being deployed on a development environment so that the acceptance tests can be run against it. If your automated acceptance testing strategy is to succeed in the long term, these kinds of obstacles need to be removed.

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation – page 125

[/quote]

I won’t tip my hand too much here, but I will say this. I know exactly what is meant by the above and it’s dead on. The only piece that is missing is that UAT are often not run on developer machines also due to the time it takes to set them up, run them and then restore a dev’s box back to his original working condition.

Comments are closed.