Wednesday, June 12, 2013

Take ambiguity out of software quality determination: use Acceptance Criteria and test results as the only opinion of quality

Software quality, or the quality of any product that people use, is always a matter of opinion.

Opinions usually differ for a variety of reasons.

For any product, there is no such thing as quality, or any guarantee of of it. Then how can quality be determined good or bad?

Easy answer: Acceptance Criteria.

The great thing is that can be applied not only to functional tests cases but also performance-related test cases.

If the product is tested correctly and meets the Acceptance Criteria of the test cases which have high enough priority that they MUST BE TESTED, which is determined during the first 30% of an Agile sprint by the team stakeholders (Test, Development, Product Management, anyone else), then the product is ready to be released to the customer with acceptable quality.

The stakeholders are responsible for determining all possible test cases and which of those test cases are critical to be run per sprint. If opinions differ for the priority of test case or acceptance criteria or whatever else, then the Product Manager has final decision making power in each case.

So keep the team focused on collaboratively determining test cases, steps to test, and acceptance criteria. The test results become the definitive answer on the "quality" of the product.


  1. Hi, Michael...

    I see some possible pitfalls in your suggestions here. Could you help me out by telling me what you've done to study testing?

  2. Hi Michael, thanks for the comments. What pitfalls have you identified? I would like to respond to each one. In regards to studying testing, My "study" in testing is being at the forefront of testing activities as both a software developer, architect, and SQA/QE for the past 10 years for several companies and have seen it done wrong (mostly), and cases where it was done right. I have learned from these best practices (well, my opinion of best practices) based on work experience.