Information to be recorded
The baseline result forms the basis of a benchmark. Usually, we also run additional tests, characterized by changes to workload, configuration, features, or hardware. These additional tests are all part of the benchmark. For example:
| The size of workload can be used to find the saturation point and study how the system behaves under stress. The type of workload can reflect the business model, such as B2B versus B2C. |
|
| We should test any newly-introduced as well as any special-purpose features if we expect them to impact performance or if their impacts are unknown. |
|
| The optimal values of DynaCache parameters depend on a number of aspects, including workload size and type, hardware, and software. Changes to configuration usually require a lot of experimentation. We should keep thorough records of all the changes introduced in each test run in order to get the best tuning. |
|
| Running tests with different hardware configurations (for example, a different number of CPUs or amount of memory) provides important insight into system scalability. |
|
| System performance is bound to change after long-term operation, due, in no small part, to changes to the database and logs. Running a test with a larger database and logs will give us an idea of system performance after extended periods of operation. |