Common test execution steps
- Set up the test environment.
Setting up the test environment includes setting up everything that is required to run a performance test. This includes setting up a test WebSphere Commerce environment to mimick or simulate your target WebSphere Commerce production environment, as well as the test client software.
This also assumes that you have developed the automation scripts that will be executed using your client software. If you are building a new site you will also need to pre-create test data for your scripts to run on.
- Tune up the test environment.
Throughout this book there are various performance tuning considerations that should be applied to optimize your site's performance.
- Warm up the test environment.
Before you can run the formal target performance test case warm up the environment.
Warm-up is required to initialize that various caches in your system, especially Dynamic Cache, as well as to allow your database to build any access plans that may be required. For convenience we call out these two key aspects in the flow chart below and call them as application server warm-up and database warm-up, respectively.
To warm-up your test environment start your automation test with a single user and run up to a few users while monitoring your cache status. Once the hit ratio stabilizes, your site has been warmed up.
After you have done this a few times you should know the rules of thumb (for example, the number of users required to run a warm-up test for a given duration) to warm up your site, and you may not need to constantly monitor your site's operation to confirm whether warm-up was done successfully.
- a href="37-05.htm">Reorganize the database and optimize database statistics.
Your test generates data and, as such, you should run the DB2 RUNSTATS and REORG utilities to optimize your database.
- Back up the test environment.
Before continuing any further IBM recommends that you back up your environment, including your database. This would allow you to return to a baseline environment to compare the impact of various performance tuning or fixes that you may apply later on during the performance testing.
- Execute the performance test.
If you backed up your environment, as mentioned in the previous step, or if you are restarting your test from a backed up environment, then you need to warm up your site a little bit to load the Dynamic Cache and the file system cache, and so on. You can do this with even a single user test for a few iterations of your test scenarios.
In the remainder of this chapter we discuss the execution of various performance tests.
- Analyze the test results.
This subject is discussed in detail in Chapter 24, Analyzing test results and solving performance problems.
Note: Warming-up your test environment with the same users as the users included in your automation test can skew your performance test, depending on the duration of your warm-up, the duration of your test, and the amount of test data in the database. As such, IBM recommends that users used for warming up your test environment not be included in the target performance test case.
