Workload characteristics
It goes without saying that a benchmark test is only as good as its input. It is very difficult to predict Web site traffic and weight the importance of different kinds of traffic. It is a good idea to break down the workload into different categories so that the results can be analyzed.
| Choose a good mix of test cases |
Good test cases reflect the typical workload of a site. Selecting the most frequent and the most important user activities is key. Because the benchmark will be used as a reference point and is unlikely to be recreated, we need to make sure that it is comparable with other performance-type tests and contains enough information for comparison purposes.
|
| Account for error conditions |
Benchmarking setup should also include considerations for error conditions, such as a wrong password or wrong credit card information supplied by the user since a typical system behaves differently upon encountering these. Your own experience should convince you that handling invalid passwords and credit card information is nearly as common as entering the correct information first time.
|
| Test for system failures |
We must always have a plan for partial or total system failure. How long does the system need to resume normal operation? What kind of information will be lost when the system fails? If, as a preventative measure, we were to perform periodic system backups, how long will they take and will they fit into the desired maintenance window?