Saturday, May 25, 2013

Successful QA using Agile process - QA milestones during a sprint

Most companies are just now starting to integrate the Agile process into their software development teams, but I have seen that companies are mostly unaware of what the QA activity milestones are within each sprint.

Most of us know the typical activities of QA for a project, tasks like:

1) Create a test plan
2) Create test cases
3) Review and approve test cases with Product Management and Development team.
4) Execute tests manually
5) Perhaps even automate some of them
6) Update wiki
7) Update Story status (using something like Rally, for example)

So when do these tasks need to be done and finished within each sprint? This is the topic of this blog post. Following the milestone dates in the example is critical to the success of insuring the quality is met.

Let's say, for example, your Agile team is using 2 week sprints, or 10 working days. (It's also easier to demonstrate task time in regards to percentage using a 10-day sprint rather than a 3 week or 15 day sprint.)

Day 1-3 tasks:

1) Create test plan and test cases for each story. The test cases are determined by the QA team, Product Manager, and development team.
2) Document the test cases for each story that has QA tasks. Purpose (what is being tested), Test Case Steps, and Acceptance Criteria (what needs to happen in order to determine that the test case passed during testing) needs to be very clearly documented.
3) An email, with test case documents and/or links to all story test cases is sent to the Product Manager and development team with a request to review and approve the test cases.
4) The approval emails must be received by end of day 3 for all stories.

Milestone 1 - finished by first 30% of sprint duration: Approval of all identified Test Cases and Acceptance Criteria for all stories by all stakeholders involved.

Day 3/4 -7 tasks:

1) The time for testing a story will be different. It is entirely dependent on the size of the development task. There may be times when no mock testing can be done, or anything to prepare or be proactive until the story is ready to be tested.

Milestone 2 - Make sure that the development team sends an email to the QA person indicating when development is finished and when each story is ready to be tested in the QA environment. This is very important because the QA team must know asap when a story is ready to be tested at the earliest possible time.

2) Proactive writing of mock tests until development has completed their part of the story, if possible.
3) Updating of wiki and Agile story progress
4) Execute the test cases when story is ready to be tested in QA environment
5) Reporting of bugs found during testing

If the QA person has not received email and the story is not ready to be tested by the 60% (6 days) timeline, that is introducing risk into the quality of the product, quite simply because the QA team may not have adequate time to test the story. This risk needs to be relayed to the all involved stakeholders, so send an email indicating this.

Milestone 3 - All sprint story testing needs to be completed by the 70% (7th day) stage.

Day 8 - 10 tasks

1) Validation of fixed bugs
2) All regression testing run and determination of any bugs found.

Milestone 4 - Must verify no P1-P2 bugs exist in QA environment by 80% stage (8th day). Your team needs to determine if even P2 bugs are allowed and known into the product at this point.

3) Push the product into the staging environment. All testing needs to occur and be finished within the next 24 hours.

Milestone 5 - Must verify no P1-P2 bugs exist in the Staging environment by 90% stage (9th day). Your team needs to determine if even P2 bugs are allowed and known into the product at this point.

The final day is for story completion updates in Agile tool (Rally, etc) and updating of test case results/documentation in test case management tool (QMetry, etc). Update wiki to document any updating of test procedure changes. Send emails updating all stakeholders of final test results. Include links to any Continuous Integration test results for proof of any automated testing that might have occured also.

Milestone 6 - Assuming no P1, P2 bugs were found in staging environment testing, send email to all stakeholders that the production release can proceed and that QA is finished for all stories.

The milestone completion dates are very important, especially if your team is using 2-week, or 10 day, sprints. 

Even 1 day delays can introduce much risk into the product quality. You, as the QA person, are responsible for managing these timelines and doing as much as possible to reduce product stability risk.


  1. Hi Michael,
    I think I'm misunderstanding. While I don't think there is a standard definition of agile so it's different for all,what I read is a well described Waterfall.

    Part of the reason I have that perception is use of words/phrases/sentences like 'QA milestones', 'development team sends an email to the QA person', 'You, as the QA person, are responsible....' and so on.

    Based off what I've read there's not a lot of:
    talking going on
    team/shared responsibility

    There is a lot of:
    your responsibility syndrome

    It seems there should be (whole) team milestones, less email, a lot more talking, shared responsibility.

    Testers have a main focus of providing information and a role to play however responsibilities for everything lay with everybody.

  2. Hi Tony,

    thanks for the comments.

    I can understand the perception that this is classic waterfall method "shoved" into a 2 week Agile Sprint.

    I think there are "must-do" activities (what I referred to the completion of them as "milestones") for Testing resources to do per Sprint, and the main focus is achieving these milestone dates on time. Using the Agile process for developing and testing software is actually more difficult to achieve than previous methods. It is a BETTER way of developing and testing software, but the timeframes are often compressed, especially for 10-day sprints. The time for testing activities are also ambiguous and usually hurried, because testers dont have adequate time to test, most often because story implementation from the development team are very rarely put into the testing environment onschedule, usually after expected schedule, but yet the deadline does not extend, thus reducing the amount of time for testing activities.

    The process is extremely collaborative in all phases. For example, the first 30% is the team deciding colloboratively on tests cases and priority. The last part is about the entire team being aware of acceptance criteria being met and making decisions on what to do if bugs are found and to back out problematic code.

    Approval emails are important to provide a shared responsibility when things go wrong and for communication purposes when dev is finished onb a story and ready for testing. For example, how many times has someone asked "why didnt we test this?" or "Oh, I didnt know that feature was ready to be tested" If something is missed, the testing person isnt the first person blamed. Following these milestones is about collaboration and shared responsibility as a team and allow for team improvement if things aren't working.

    In Agile, and on most teams, there is no project manager. So the Testing person needs to drive the team to make these milestones for the betterment of the product and the team.