The strategy and testing approach used by software development organizations to validate the quality and functionality of their products is shown in a variety of testing documents. In addition to strategy and approach, most organizations also produce documents to report test results and defect tracking information. Some of these testing documents are created by developers but it is more common for the quality assurance (QA) group to generate them. Towards the end of the testing process, the QA team typically compiles a release package, which is the compilation of testing documents created throughout the process. This package often includes a recommendation as to whether and how the software application should be released. These documents include but are not limited to the following:
- Test Strategy
- Requirements Analysis
- Functional Specification
- Test Plan
- Test Case
- Test Results
- Issues and Defects
- Project Overview
For most organizations, the Test Strategy is the first document prepared for a project by QA. Since it should be updated throughout the lifecycle of the project, it is considered to be a living document. This testing document provides details on the approach that QA intends to follow for testing the product that is being developed. It specified the test methodology used, the testing types that will be executed (functions, interface, etc.), priority of testing, product configurations to be tested, as well as the risk associated with testing. Another important part of the test strategy is details of the testing setup. This section identifies the hardware, software, and personnel required for testing. The test strategy document also contains a schedule for the project that includes completion dates for all other documents, delivery dates, release dates, etc. as well as defines the criteria for project completion.
The Requirements Analysis Document defines the functionality that was included in the project. It basically describes what the product is and what it is supposed to do. Developers use this document to create the functional specification and it is often prepared or reviewed by sales and marketing teams, and product managers. The QA team should use it as part of preparation of the test strategy. As vital as it is, some projects do not have requirements documents and this could have dire consequences to development. If it does not exist, this fact should be indicated in the project overview.
The Functional Specification testing document explains how the project requirements will be implemented. It outlines how new product features will be implemented and any database queries that are required as part of the project. The QA team usually uses the functional specification to create the test plan and business rules. Though it is a very important part of the SDLC, some projects do not have functional specification documents and this could have dire consequences to development. If it does not exist, this fact should be indicated in the project overview.
The Test Plan document is typically created after the testing strategy and functional specification documents are completed and it provides the testing approach to testing that QA will employ to validate the functionality and quality of the product before it is released. Though the exact content may vary from organization to organization, most test plans provide the following information:
- General project description
- Project objectives
- Schedule of testing
- Resource requirements (hardware, software and personnel)
- Listing of features being tested (also those that are not being tested)
- Details of test approach
- Lists of deliverable items (test cases and scripts)
- Test dependencies
- Risks of testing
- Description of defect tracking process
- Milestone criteria
The Test Case document provides a specific set of steps that QA testers must perform for each test category in order to validate the feature. It is prioritized, includes the expected results, and should be limited to one function. A test case must provide details of the test setup, describe the test environment, provide details of the test steps/procedure, specify pass/fail criteria, and reference the results of the test. Most test case formats include the following details:
- ID number – Number identifying the test case
- Priority – Assigned order if precedence
- Purpose – Basis for the test case
- Steps – Sequence of steps that the tester must follow to perform test case
- Expected Results – What the tester should see when test case is executed
- Actual Result – What the tester actually witnesses when test case is executed
- Status – Indicates whether the test case passed, failed, etc.
The Test Results document simply reports the results of the executed test cases. Test results documents should at least include the following:
- Software version checked
- Scope of test
- Database version used (if applicable)
- Installation paths (If applicable)
- Name of tester(s)
- Date test performed
- Results of test
- Issues/Defects (if applicable)
Issues and Defects
The Issues and Defects testing document is most often used to aid technical support. It outlines all known application problems and defects. It details existing workarounds to problems, and lists bugs in the various stages of the defect tracking process. Though this document varies from organization to organization, most include the following:
- Known Issues- This part of the issues and defects document identifies issues that the organization is aware of but have decided to delay correction or ignore. If any exist, workarounds are included. This information is used primarily for technical support.
- Open Defects – This includes a list of defects that are either in-work or are expected to be fixed at some point.
- Deferred Defects – Deferred defects are problems that will not be addressed in the current release but are expected to be addressed later.
- Pending Defects – Pending defects are waiting a decision from management before being addressed by a developer.
- Fixed Defects – These are defects that have been fixed but are awaiting QA verification.
- Closed Defects – These are defects that QA has verified as fixed.
The Project Overview document provides a synopsis and scope of the project, problems encountered during testing, and whether release of the product is recommended or not. The project overview document can be thought of as a response to the test strategy document and should indicate the areas of the strategy that was successful as well as those that required revision. It can also include suggestions for improvements to the process for the next project cycle.