Acceptance testing is the final phase in any systems development project. It comes after the product or solution has been developed and (hopefully) tested thoroughly by its creator. The purpose of acceptance testing is to ensure that the requirements are met.
At this point, it’s up to the client to verify that the system lives up to the agreed-upon specifications, as well as any implicit or expected requirements. Here are eight guidelines to follow when running your own acceptance testing sessions.
1) Requirements will Change
Requirements will change—this is a gospel truth. Formulate a strategy and establish a process for dealing with changing requirements. The process you formulate will need to help you capture, organize, prioritize, and manage many versions of new and changing requirements.
The process needs to cover how you intend to handle any requirements changes that are bound to appear during testing. Affirm the changes and think of them as lessons learned rather than failures. After all, requirements are what the acceptance tests are intended to approve.
2) Staff to Test
Unless you’re deliberately aiming for failure, testing cannot wait until the end of the project—it needs to begin as early as the procurement process. The cost of finding and fixing problems when they’re on the drawing board is negligible, compared to shoehorning in requirements discovered when the system is ready for deployment.
Depending on the size of the project, you’ll need at least one experienced person on the team whose focus is solely on reviewing and testing. This ensures that quality never takes a back seat to other needs. You probably already have system test managers and system testers, but acceptance test managers and acceptance testers are just as important.
3) Have Acceptance Testers Check for Usability Issues Early-On
Acceptance testers are extremely helpful to have in system testing, since they’re highly adept at spotting usability gaps and opportunities for improving the user interface. However, they can start usability testing much earlier—as soon as there is a user interface to look at.
The interface can be as basic as a simple workflow, a single system module or a form, embodied in a simple sketch or an advanced prototype. It’s nearly impossible to fix logical errors in workflow when they’ve been implemented throughout the system, so bring in your acceptance testers early to catch these logical errors before they spread.
4) Make Time to Test
All too often, employees are expected to test a system in tandem with their regular duties. Validating functionality and testing new workflows take time, so it’s important that any employees involved in acceptance testing truly understand the process (and its importance) and have set aside sufficient blocks of time in their schedule.
Requirements will change—this is a gospel truth
You can set up a special testing room to keep testers from being distracted by other tasks. Boost testers’ focus and motivation by explaining why testing is important and training them in effective testing techniques, specifically within the framework of the project.
The developer and client should agree on the preconditions for acceptance testing long before testing starts to ensure that employees don’t waste time testing without the proper processes and tools in place.
5) Establish Law and Order with a Shared Test Management Tool
Testing isn’t about randomly kicking tires, it’s about working against a discrete list of identified risks and priorities, and then tackling them in a logical order, one by one.
Testers document identified defects and deviations and report them to the person tasked with correcting them. After they’ve been corrected, the testers need to monitor and retest the problem spots, individually and in context.
If the project involves several people, they need a shared data structure, and should coordinate their work with a test management tool right from the start. Ideally, the tool should be able to handle the entire chain from requirements through test cases to bug reporting and provide summary statistics and metrics. Both client and developer should use the same tool. I suggest trying out our tool, ReQtest.
6) Create New Test Cases
Developing test cases as soon as requirements are developed is great for emphasizing the importance of testing, but those initial cases are seldom sufficient; they need to be supplemented throughout the project.
It’s certainly tempting to reuse test cases from previous testing levels in acceptance testing, however each level of testing has a unique purpose and acceptance testing focuses on usage of the whole system.
You need to describe the expected results in the test cases; without knowing what the results are supposed to be, it’s much harder to determine if the output is accurate. Retesting is easier if you can use the same test cases again, and a testing tool makes it easier to both maintain and use test cases.
7) A Separate Test Environment
If you expect testing to provide accurate results, you will need to set up a test environment that’s identical to—but separate from—the production environment. A common problem is that the test environment has fewer servers, or that other systems are not installed in the test environment. If the environment deviates from the production environement, your results might be misleading.
Investing in two sets of hardware and licenses can seem expensive, and make it tempting to install and test directly in a live environment, or even to skip acceptance testing altogether. However, such shortsightedness can cause users to lose confidence in the system and ultimately drive the cost much higher than if the investment was made earlier, and more wisely. Pay a little now or a lot later.
8) “Typical” is not Always Realistic—Take Time to Create Test Data
Test managers often expect “typical” testers to be able to provide “typical” data that’s representative of the real world. In my experience, this approach works in simple contexts, but in complex situations, where complex transactions and dependencies need to be managed and measured, realistic test data doesn’t come from typical cases.
You’ll benefit enormously from a test data strategy that includes a test database of anonymous data that you use and update throughout testing. Extract selected parts as needed and use them as much as possible during acceptance testing.
You should also take the time to create procedures that restore the test data back to its initial state. In complex systems, obtaining relevant test data can be a project in and of itself, and you should plan more than adequate time to do the job right.
Image of man hands courtesy Shutterstock
