Testing server affinity | Why is it complex

Introduction to performance testing

Performance testing, also known as load testing, is testing of the responsiveness of a system. For example, performance testing can be used to ascertain how quickly can a system respond, how many responses a system provide can in a given unit of time, how long a certain level of responsiveness can be maintained, at what cost, and so on.

The objective of performance testing is not to test the proper functioning of a given component or product in a single user test scenario. This aspect of testing should be covered by function testing. We assume that the single user function testing has already passed successfully, prior to performance testing that function.

All the responses received during performance testing should be error free or the number of errors should be within an acceptable margin of error. During performance testing these errors could be due to concurrent users accessing the system or due to other performance defects. If a given system has good performance but the number of defective responses (due to functional, performance, or other defects) are above the margin of error, then such a performance test will be deemed to have failed. A failed test case is not always undesirable. For example, it can be useful in ascertaining capacity limits during stress testing.

The aim of performance testing is to:

• Establish the performance characteristics of the application, such as...
  • transaction response time
  • transaction rates
  • CPU utilization

  Profiling can be used on those slow requests to help determine the root cause of their poor response times.

• Identify system bottlenecks.

• Determine system capacity.

• Demonstrate system scalability by:
  • Measuring CPU and throughput at various loads
  • Employing techniques to fully utilize available hardware
  • Driving machines to >90% utilization
  • Validating that hardware and software configuration is optimal

Methodical performance testing and recording of results reveals system capacity and provides insight into system scalability. A performance reference model, obtained from the execution of a controlled test plan, provides scaling predictability. Thus, when a variable, such as the number of users, is increased, the reference model may flag the need for additional hardware capacity.

The performance test plan is a methodical testing approach that starts with acquisition of baseline information, and then adds capacity to determine how the system scales for all components. By controlled workload escalation, the defined test procedures expose constrained resources, or bottlenecks, which cause the overall system to begin displaying non-linear performance characteristics. The bottleneck then becomes the focus of drill down testing and analysis to alleviate the constraint, after which the process is repeated. Commonly restrained resources include thread pools, connection pools, and available memory.

Performance test scenarios are similar to, and sometimes overlap, other types of testing, but with the focus on developing a deep understanding of system performance.

There are typically two desired results of a performance test:

• Gaining knowledge of how a system performs

• Using that knowledge to improve the performance of the system by tuning the components, or perhaps by adding additional hardware capacity