200 concurrent users using cachespec.xml
]The test with sample cachespec.xml shows that our test environment is very capable of handling 100 users: the page response times are low, as is the CPU usage on all the servers. We can therefore use the test results as the final benchmark of WebSphere Commerce with DynaCache.
What happens if we increase the number of concurrent users, however? Will the throughput (number of pages) and CPU usage increase linearly? What about the response time?
The final benchmark-creation test, which we tackle in this section, is an illustration of the kinds of additional tests that we would typically include in a benchmark suite, intended to investigate issues of performance under stress, scalability, and so on. Here, we double the number of concurrent users to see how the system reacts.
Table | -9 and Table | -10 and Figure | -6 summarize the results of the test.
Table 9-9
| CPU usage | Minimum JVM heap usage | Hard disk usage increase | |
|---|---|---|---|
| Commerce server 1 | 27.99% | 450MB | |
| Commerce server 2 | 28.02% | 430MB | |
| Database server | 13.74% | 101MB | |
| Web server | 10.2% |
Resource usage in the test with 200 users test and sample cachespec.xml
Table 9-10
| Web action | Average response time (second) | Count | Total time (second) |
|---|---|---|---|
| Overall | 0.938 | 468,303 | 439268.214 |
| Select shipping address | 1.145 | 5,837 | 6683.365 |
| Add shipping address | 0.988 | 4,129 | 4079.452 |
| Display category | 1.004 | 211,692 | 212538.768 |
| Log off | 0.801 | 10,872 | 8708.472 |
| Log on | 0.858 | 6,044 | 5185.752 |
| Go to log on page | 0.797 | 6,044 | 4817.068 |
| Add item to cart | 1.666 | 14,876 | 24783.416 |
| View billing address | 1.749 | 12,118 | 21194.382 |
| Process order | 1.85 | 1,306 | 2416.1 |
| View order summary | 2.354 | 2,689 | 6329.906 |
| Add payment info | 1.839 | 1,306 | 2401.734 |
| Display product | 0.757 | 70,515 | 53379.855 |
| View shipping method | 1.328 | 12,107 | 16078.096 |
| Display top category | 0.655 | 70,617 | 46254.135 |
| Visit store front | 0.478 | 27,261 | 13030.758 |
| Register user | 0.845 | 5,445 | 4601.025 |
| Go to register page | 1.201 | 5,445 | 6539.445 |
response times in the test with 200 users and sample cachespec.xml
Figure 9-6 Response times in the tests with 100 and 200 users and sample cachespec.xml
The test results show that the throughput, response time, and CPU usage do not increase linearly. Doubling the number of concurrent users does not double the throughput, even though a considerable amount of unused resources remains.
As a next step, we would run more tests to determine the saturation point.
The results also confirm the common-sense observation that the response time is unlikely to stay unchanged when the number of concurrent users increases - but we need to take more measurements to determine why the response time is increasing.