Test tools introduction | Performance test tools classification

22.1.1 How to select test tool

With so many stress testing tools available today, how can you choose the one that is most appropriate for your application? Some of the points to consider when evaluating stress testing tools include:

1. Client interaction

   The stress testing tool must be able to handle the features and protocols that your application uses.

2. Simulation of multiple clients

   This is the most basic functionality of a stress testing tool.

3. Scripted execution with the ability to edit scripts

   If you cannot script the interaction between the client and the server, then you cannot handle anything except the most simple client requests. The ability to edit the scripts is essential. Minor changes should not require you to go through the process of regenerating a script.

4. Session support

   If a stress testing tool does not support sessions or cookies, it is not very useful, and may not be able to stress test Java and WebSphere applications.

5. Configurable numbers of users

   The stress testing tool should let you specify how many simulated users are running each script or set of tasks, including allowing you to vary the number of simulated users over time. Many stress testing tools enable you to start with a small number of users and ramp up slowly to a higher numbers of users.

6. Reporting: success, errors, and failures

   The tool that you choose must have a defined way to identify a successful interaction, as well as failure and error conditions. An error might be getting no Web page back at all, whereas a failure might be getting the wrong data back on the page.

7. Page display and playback

   A useful feature in many stress testing tools is the capability to inspect some of the pages that are being sent to the simulated users or to replay entire test scripts. You can then be confident that the stress test is functioning as you expect.

8. Exporting test results

   After running a stress test, you may want to be able to analyze the test results using various tools that are external to the stress testing tool, including spreadsheets and custom analysis scripts. Most stress testing tools include extensive built-in analysis functions, but being able to export the data gives you more flexibility to analyze and catalog the data in arbitrary ways.

9. Think time

   Real-world users do not request one Web page immediately after another. There are generally delays between viewing one Web page and the next. The term think time is the standard way of expressing the addition of a delay into a test script to more realistically simulate user behavior. Many stress testing tools support randomly generated think times based on a statistical distribution.

10. Variable data

    Live users do not work with the same set of data on each interaction with your application. During a stress test, this should also be true of your simulated users. It is easier to make your simulated users appear to be working with varied data if the stress testing tool supports data input from lists, files, or a database.

11. Script recording

    Rather than writing scripts, it is much easier to manually run through a session with your browser and have that session recorded for later editing. Most stress testing tools include provisions for capturing manual interaction with your application.

12. Analysis tools <

    Measuring performance is only half the story. The other, and perhaps more important, half of stress testing is analyzing the performance data. The type of analysis tools and degree of detailed analysis you can perform depend directly on what analysis tools are supported by the tool that you select. Therefore, evaluate this support in the tools that you are considering carefully.

13. Load distribution

    Your deployed application may well need to support hundreds of concurrent users once in production. How can you simulate this level of traffic in a stress testing environment? A typical workstation running a stress testing tool will likely begin bottlenecking once approximately 200 virtual users are running. To simulate a greater number of users, you can distribute the stress testing load across multiple workstations. Many of the available stress testing tools support distribution of load, and you will certainly want this feature for large-scale stress testing.

14. Measuring server-side statistics

    The basic stress testing tool measurement is client-based response times from client/server interactions. However, you may also want to gather other statistics, such as the CPU utilization or page faulting rates. With this server-side data, you can then do useful things like view client response times in the context of server load and throughput statistics.