Test Review Process When Working With Different Suppliers

When we talk about testing deliverables, there is a huge list and reviewing them is one of the major activity. When you review anything means assuring the quality. Test Reviews play an important role in the software testing lifecycle for any team to do the things right in the very first attempt avoiding potential catastrophe it could lead to in the next phases.

Test reviews also help in making sure the test team is adhering to set standards and expectations for optimum quality delivery in the same time with the similar resources available.

Aman has already emphasised much on the importance of reviews in Software Testing in his article on QST about some simple tips to write effective test cases.

Here in this article, I am going to explain how to review the various tabs and sections in ALM. I am going to talk about reviewing each of the testing activities managed in ALM – starting right from creating a requirement to logging a defect in ALM.

I am working as a QA Reviewer for my employee where we have different suppliers providing services for us.

Making sure each one of them follows a set standard or template for HP ALM and logs the right amount of information is quite crucial. This ensures re-usability of test artifacts and also a uniform of details for each single section in ALM.

I am using HP ALM for my article but you can follow this check-list during test review process for any Test Management Tool.

So let’s get started – here is a list of things you should take care of as a QA review lead during test review process!

Requirement Phase Tab

Below are the various things you should score in requirement phase tab in ALM.

#1. Make sure that you create Requirements correctly in ALM

Checking the requirements creation is not that simple as it looks like! As a reviewer (I feel) you should throw some light on many related things. Some of them are mentioned below…

For example, name of the requirement should be meaningful and easy to understand, details of the requirement should be selected correctly such as Name, type of the requirement etc.

#2. Requirements should be linked to Test plan, Test Lab and Defects

People generally forget to link the requirements which is one of the easy and important activity.

As I mentioned above, you should always focus on linking of Requirements to other sections (Test Plan, Test Lab, Defects etc) and this should come out as a review feedback if you find this aspect missing.

You would be already aware of Requirement Traceability Matrix (RTM) which tells the overall status of requirements – How many test cases executed, How many passed & failed and how many defects are outstanding for any particular requirement.

The reason I am emphasising to make sure requirements are mapped correctly, in all different tabs in ALM is this – ALM gives you a provision to generate the RTM by just linking the requirements correctly in the first place.

So now you can make out how important it is to link the requirement.

#3. Checking the Test coverage in the Requirements Tab

When I say checking the test coverage, I am referring to the amount of testing performed for the particular requirement by the set of the test cases.

One of the ways to find out the test coverage for particular requirement is to check the Test coverage Tab in Requirement Tab in ALM which will give you the status of the test cases against that requirement.

This comes in real handy for reporting, so make sure Test Cases are correctly mapped to Requirements.

#4. Checking the Linked Defects against the Requirements

Linked Defect in Requirement Tab in ALM tells you – how many defects, in total, are open for the selected requirement.

#5. Making sure that Requirement is assigned to the correct Target Release & Cycle

Many teams do not bother about assigning the requirement to correct Release & Cycle. It’s up to the Test Manager or the reviewer if they want their team to put this check as it helps while reporting.

Assigning the requirements to correct Release & Cycle will give you the provision to get the overall status of release in Management tab in ALM.

Test Plan Tab

As a reviewer, when I start reviewing the Test Plan, there are many things that I consider into review check-list. Here are the list of things to be reviewed in Test Plan.

#1. Checking  the Test Plan is created under the Test Plan tab in ALM

Good process always says to create the Test Plan. Naming of the folder can be same as requirement name created in Requirement Tab. To know more about naming convention you can refer to this article.

#2. Test Cases or test scripts are created and mapped against the Requirements

Test cases or Scripts are the same thing – it’s just the way how people call it. Test cases should be simple and easy to understand. I won’t go in detail on how to write test cases or scripts as it’s a separate and huge topic to talk about.

Once the test cases are created make sure that they are mapped to requirements without fail. Once the mapping is done, you can see the mapped requirement under the Requirement coverage Tab in test plan.

#3. Correct Details are filled against for each Test Case under the Details Tab

Filling the proper Details based on your project requirements, you can convey how you want your team to fill out the details.

For example, status of the test case should be changing based on your Test Phase. If you are in Designing phase, status should be ‘Design’. In review phase, test case status should be ‘Review’ and if it is ready to execute then the Status should be ‘Ready’ and so on.

#4. Checking the Linked Defects to make sure that Defects are linked correctly

If any defects have an interdependency, such as if re-test of one defect is dependent on resolution of the other, linking the defects correctly helps. As a reviewer you should make sure this is done as it helps in reporting.

Test Lab Tab

Test Lab is actually where you can keep a track of the execution. In my experience, I have seen people who run the test case but forget to update the status in ALM. It is a mandatory step to update the execution status in ALM and keeping it updated with correct status.

As a reviewer you can check the below things in the Test Lab tab in ALM.

#1. Checking the mapping against the Test Plan

As I mentioned earlier, mapping is one of the important part in ALM. As a QA reviewer, you need to make sure that test sets are mapped correctly to the Test Cases in Test Plan.

#2. Checking the Test Execution and Defects mapping

As a reviewer, you should see the execution status is updated with Passed or Failed or Blocked. You need to make sure that if the test steps are passed, Actual result should be updated for relevant test steps.

Also, you need to make sure that if any test step failed and defect is raised then it is raised from that particular step. Some projects, first they raise the defect in Defect Tab and link it later. It depends how you want to link. In my current project I have asked my team to raise a defect from Test Lab linking the Test Step to defect.

Also, you should see the defects linked to the test cases under the Linked Defects Tab in Test Lab.

Defects Tab

One of the main activity which you would do as a tester is, identify defects.

Management of defects is one of the critical activity. Throughout my experience, I have seen that Defect management plays very important role as that is a way to convey to the stakeholders, how stable an application is.

Defect is very import exit criteria to go any project live. As a reviewer, below things should be reviewed.

#1. Defects are raised as per the template set by your company.

You need to make sure that defect are raised properly with all the relevant information which make complete sense to the resolver group.

In my job, in some instances, I have seen people just putting 1 line which does not sense to anyone. Most of the time I have seen Dev team is not happy about testing team raising such defects.

I have been involved in Defect Management team and we came up with common agreed template which will be used across the projects and believe me this has made everyone’s life so easy.

So, you can also agree and come up with your template agreed upon by Testing, Development, Review, Defect Management / Release Management teams and business teams, where applicable (you can include any other teams who are involved in this whole process)

As a reviewer, you need to make sure that Defects are raised as per the template and the defects are linked to test cased in Test Lab and that test evidences are attached to the defects.

#2. Defect Status

Most importantly, make sure defect statuses are selected properly and as per the agreed template.

If the Defects are raised following the template, you can pull out the report from ALM easily and send it across to all the stakeholders. This is really useful for the people who just want to know the overall status of projects and the outstanding defects.

So here, I have talked about one of the main deliverables from suppliers where I have tried to explain as much as I could. I have tried to keep it simple so that anyone who is new to the Test Review Process, can easily understand.

I hope you find this article useful and helpful. If you do, please share it with your colleagues, team members and friends and on social media.

Please feel free to ask any questions you have and I will try my best to answer them.

I will be discussing about reviewing rest of the deliverables in my next article. So, stay tuned till then!