Concepts: Acceptance Testing
Acceptance testing is the final test action before deploying the software. The goal of acceptance testing is to verify that the software is ready and can be used by your end users to perform those functions and tasks for which the software was built. There are three common strategies for implementing an acceptance test. They are:
The strategy you select is often based on the contractual requirements, organizational and corporate standards, and application domain.
Formal Acceptance Testing
Formal acceptance testing is a highly managed process and is often an extension of the system test. The tests are planned and designed as carefully as, and in the same detail as, system testing. The test cases chosen should be a subset of those performed in system test. It's important not to deviate in any way from the chosen test cases. In many organizations, formal acceptance testing is fully automated.
The activities and artifacts are the same as for system testing. In some organizations, the development organization (or its independent test group), with the representatives of the end-user organization, performs the acceptance test. In other organizations, acceptance testing is performed completely by the end-user organization or an objective group of people chosen by the end-user organization.
The benefits of this form of testing are:
- The functions and features to be tested are known.
- The details of the tests are known and can be measured.
- The tests can be automated, which permits regression testing.
- The progress of the tests can be measured and monitored.
- The acceptability criteria are known.
The disadvantages include:
- Requires significant resources and planning.
- The tests may be a re-implementation of system tests.
- The testing may not uncover subjective defects in the software, since you're only looking for defects you expect to find.
Informal Acceptance Testing
In informal acceptance testing, the test procedures for performing the test are not as rigorously defined as for formal acceptance testing. The functions and business tasks to be explored are identified and documented, but there are no particular test cases to follow. The individual tester determines what to do. This approach to acceptance testing is not as controlled as formal testing and is more subjective than the formal one.
Informal acceptance testing is most frequently performed by the end-user organization.
The benefits of this form of testing are:
- The functions and features to be tested are known.
- The progress of the tests can be measured and monitored.
- The acceptability criteria are known.
- You will uncover more subjective defects than with formal acceptance testing.
The disadvantages include:
- Resources, planning, and management resources are required.
- You have no control over what test cases are used.
- End users may conform to the way the system works and not see the defects.
- End users might focus on comparing the new system to a legacy system, rather than looking for defects.
- Resources for acceptance testing are not under the control of the project and could be constricted.
Beta Testing
Beta testing is the least controlled of the three acceptance test strategies. In beta testing, the amount of detail, the data, and the approach taken is entirely up to the individual tester. Each tester is responsible for creating his or her own environment, selecting his or her data, and determining what functions, features, or tasks to explore. Each tester is responsible for identifying his or her own criteria for whether to accept the system in its current state or not.
Beta testing is implemented by end users, often with little or no management from the development (or other non end-user) organization. Beta testing is the most subjective of all acceptance test strategies.
The benefits of this form of testing are:
- Testing is implemented by end users.
- There are large volumes of potential test resources.
- There is increased customer satisfaction for those who participate.
- You uncover more subjective defects than with formal or informal acceptance testing.
The disadvantages include:
- You might not test all functions or features.
- Test progress is difficult to measure.
- End users might conform to the way the system works and not see or report the defects.
- End users may focus on comparing the new system to a legacy system, rather than looking for defects.
- Resources for acceptance testing are not under the control of the project and could be constricted.
- Acceptability criteria are not known.
- You need increased support resources to manage the beta testers.
