Thursday, January 13, 2011

What is traceability Matrix and template structure

Traceability matrix is a document/spreadsheet prepared in a project to map the test case against the requirements. Requirements may be in terms of use case , business requirement document(BRD) etc as per the guidelines in that project.

Traceability matrix template
Sl.No | Requirement ID |Requirement Document Name(UseCase/BRD)|Scope(In Scope/Out of Scope)|TestCase|Comments

Best Practice For Test case Writing



Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written. The basic objective of writing test cases is to validate the testing coverage of the application. The test cases should be written in enough detail that they could be given to a new team member who would be able to quickly start to carry out the tests and find defects.

The most extensive effort in preparing to test an application, is writing test cases. Documenting the test cases prior to test execution ensures that the tester does the ‘homework’ and is prepared for the ‘attack’ on the Application Under Test.

“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.”

There are levels in which each test case will fall in order to avoid duplication efforts.
Level 1: In this level you will write the basic test cases from the available specification and user documentation.

Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.

Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10

Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionality's to test rather than remaining busy with regression testing.

Factors Involved in Test Case Quality

Measuring quality is always a daunting task: The term ”quality” has to be defined first. Here, quality refers to the appropriateness of test cases to check the quality of a system
which in turn is difficult to analyse. Therefore, the discussion focuses on three different aspects of test case quality:
First, we argue that it should strongly be differentiated what kind of object is in the focus of measures, secondly, we structure different dimensions of test case quality by means of the available artefacts and documents in different phases of the development process and finally we distinguish relative and absolute measurements.

The above diagrams depicts that the test cases should be simple and easier to understand and should be informative covering all the requirements .Regarding the testing styles we should use only simple language or style to write the Test cases so that any one can easily understand without any ambiguity


Steps To be followed for writing good test cases.

1. Read the requirements clearly and completely.If we have any questions in the Requirements it should be clarified by appropriate person (e.g Customer or Business Team). And also, it is good practice to gather some basic domain knowledge before getting into reading requirements and writing Test Cases. And also, we can have discussion/meeting with developers/business team

2. Have Good Understanding of the design and implementation.

3. Before starting writing test case become familiar with the Application Under Test (AUT).

4. Keep Traceability Matrix to make sure that we have written test cases for covering all

5. we should use only simple language or style to write the Test cases so that any one can easily understand without any ambiguity

6. Test Description should be short and it should uniquely represent the current test scenario without any ambiguity.

7. Cover all the positive scenario first and then think about all possibilities of negative scenarios to have test cases which will effectively find most of the bugs.

8. For doing this we can refer alternate flow section of use case document, and we can think about different data, boundary conditions, different navigations paths and multi user environment.

9. Follow Naming Convention – Naming Convention helps you to differentiate between function ,control, dbname etc..It helps in easy identification of different controls ,functions ,labels etc..

10.        Give some sample test data that will be useful for executing the test cases.
11.        Always update the test case along with the new build.
        

Anatomy of a Test Case
Fields in test cases:

Test case id:
Requirement # / Section:
Objective: [What is to be verified? ]
Assumptions & Prerequisites
Test Procedure Steps to be executed:
Test data: Variables and their values
Expected result:
Actual result:
Status : Pass/Fail
Comments:


Test Case Design Techniques

While developing test case , the motto should be to develop optimum test case
 

Types Of Test cases

Functional Test case
Functional Test case is designed to  measure the quality of the functional components of the system .Test Cases verify that the system behaves correctly from the user / business perspective and functions according to the requirements, models, story boards, or any other design paradigm used to specify the application. The functional  test cases must determine if each component or business event: performs in accordance to the specifications, responds correctly to all conditions that may be presented by incoming events / data, moves data correctly from one business event to the next (including data stores), and that business events are initiated in the order required to meet the business objectives of the system.
The approach followed for functional test case development is as mentioned above for in section Steps for writing good test cases.

GUI  Test Case
Functional Test Case mainly  covers and focus on  the functionality of the system  and while executing these we may miss some GUI defects.
In order to avoid this  circumstance some organizations do adapt to write separate GUI test cases which ensure  us that the test suite fully exercises the GUI itself.

The approach followed for writing GUI test case is referring the UI prototype  and test case mainly focus on below three parameters
1 - Windows Compliance Standards
1.1. Application
1.2. For Each Window in the Application
1.3. Text Boxes
1.4. Option (Radio buttons)
1.5. Check Boxes
1.6. Command Buttons
1.7. Drop Down List Boxes
1.8. Combo Boxes
1.9.    List Boxes
2 - Tester's Screen Validation Checklist
2.1. Aesthetic Conditions
2.2. Validation Conditions
2.3. Navigation Conditions
2.4. Usability Conditions
2.5. Data Integrity Conditions
2.6. Modes (Editable Read-only) Conditions
2.7. General Conditions
2.8. Specific Field Tests
2.8.1. Date Field Checks
2.8.2. Numeric Fields
2.8.3.               Alpha Field Checks
3 - Validation Testing - Standard Actions
3.1. On every Screen
3.2. Shortcut keys / Hot Keys
3.3.    Control Shortcut Keys

Web Service Test Case
Web services are typically application programming interfaces (API) or web APIs that are accessed via Hypertext Transfer Protocol and executed on a remote system hosting the requested services.
The basic Web services platform is XML + HTTP.

XML provides a language which can be used between different platforms and programming languages and still express complex messages and functions.

The HTTP protocol is the most used Internet protocol.

Mainly in SOA projects , we write web services test cases.
The test engineer should do a detailed study of the Web service  specification and any related use case documentation. This review of the web services specification from a test perspective typically uncovers numerous errors in the implementation before a single test case is written.
To write web service we need to go through the SOA web service document and need to figure out for which all are exposed services , the test cases needed to be written for it.
We use the all pair tools for the combination of parameter values  and formulating the number of test cases needed to written for each web service.
By following the above approach web service test case  is developed detailing the test conditions and expected results for each web service 


API Test case
An API (Application Programming Interface) is a collection of software functions and procedures, called API calls, that can be executed by other software applications. All of the tests for a given API should be in a single file; and it makes sense to name the file after the API under test. This makes it easier for others to find and locate all of the test cases for a specific function. In a comment at the top of the file, you should include the declaration of the APIs. This will allow you to reference the parameters and return type quickly. Each test case should be as self-contained and isolated from dependencies as possible.

Performance Test Case
The performance test case  is designed to  focuses on the response time. Resource utilization of server component  and through put of the application under test. The performance test case describes the types of users and number of users of each type that will be simulated in a performance test. Test Designer should go through the non functional requirement document before authoring the  performance test case. The following test case template followed  for performance test cases
Test Suite
Test  Case ID
Use Case ID
Requirement ID
Test case Description
Prerequisites
Test Procedure
Test Data
Expected results
Other setting: will define the users load and number of iteration
Remarks