Cloud credits only: Size Cloud Load Generators
Sizing Cloud Load Generators is a crucial step to define the number of User Paths which a Cloud Load Generator can handle. The number of User Paths depends on many aspects:
- Technologies used in the application to test
- Complexity of the application to test
- Network connection capacity
- Etc.
Starting a ramp-up test of the application with one Cloud Load Generator using an on-demand session in one given geographical region makes it possible to benchmark the Load Generator. NeoLoad monitoring data, and particularly thresholds on the CPU and on the memory of the Load Generator machine, help identify the maximum number of User Paths for the Load Generator.
After identifying the number of User Paths that one Cloud Load Generator can handle and the number of User Paths for the test, it is easy to calculate the number of Cloud Load Generators necessary to run the test.
For more information about on-demand Cloud sessions, see Create an on-demand Cloud session. For more information about ramp-up tests, see Load variation policy. For more information about monitoring thresholds, see Set and configure a counter alert threshold.
To pinpoint the load limits of a Load Generator, it is necessary to watch its behavior under the ramp-up test load, and specifically:
- CPU
- Memory
- Throughput
- Hit rate
- Virtual Users load
- Tip: By default, NeoLoad sets thresholds when 80% and 90% of the Load Generator CPU and memory are used. When the CPU or memory figures exceed the thresholds for 15 seconds or more, an alert is raised.
The first significant breaking point in the CPU, memory, throughput or hit rate metrics represents the performance limit of the Load Generator. This point must be correlated with the number of Virtual Users generated. Beyond this number of Virtual Users, serious issues may well occur with the Load Generators.
For example, a sizing test is started with a Large Cloud Load Generator. The ramp-up test adds 1 Virtual User every second. The Graphs tab in the Runtime section or the Graphs tab in the Results section shows that the hit rate curve slows down on the 22nd minute. This shift occurs with 1,300 Virtual Users (1 Virtual User per second for 22 minutes). The size limit of the Large Cloud Load Generator is 1,300 Virtual Users. To test the application with 20,000 Virtual Users, it is probably necessary to use at least 16 Large Cloud Load Generators (20,000 / 1,300).
For example, a sizing test is started with a Medium Cloud Load Generator. The ramp-up test adds 1 Virtual Users every second. The Graphs tab in the Runtime section or in the Results section shows that the memory curve slows down on the 6th minute. This shift occurs with 360 Virtual Users (1 Virtual User per second for 6 minutes). The size limit of the Medium Cloud Load Generator is 360 Virtual Users. To test the application with 20,000 Virtual Users, it is probably necessary to use at least 56 Medium Cloud Load Generators (20,000 / 360).
- Tip: A 20% to 30% security margin out of the sizing limits is recommended for the Load Generators in the real heavy load test.