9.3.3 Execution and results
As stated at the opening of the Benchmarking example section, the purpose of our tests is to create a benchmark for a WebSphere Commerce system with DynaCache, as well as to show performance improvements that DynaCache brings. As discussed in Obtaining the baseline results and measuring improvements, we therefore need a baseline test without DynaCache and a test with the finalized parameter set, reflected by the cachespec.xml descriptor, to serve as the new DynaCache baseline for any possible changes at a future time. Sometimes, we also require additional tests, for example, to confirm that we get performance benefit from such changes or to investigate system behavior under a different set of circumstances.
We ran four tests in preparation of this book:
| 100 concurrent users without DynaCache |
|
| 100 concurrent users with caching the product display page only |
|
| 100 concurrent users with the cachespec.xml file provided as part of the WebSphere Commerce consumer direct starter store |
|
| 200 concurrent users with the cachespec.xml file provided as part of the WebSphere Commerce consumer direct starter store |
We ran each test for 3 hours, using the JVM heap size of 1GB and identical scenarios, ratios, and think times (see Test data set and scenario).