+

Search Tips   |   Advanced Search

Validate a server response


Understand the context

Problems or errors occurring during the execution of a test scenario will bias your results, making them partially or -at worst- completely unusable. It is therefore fundamental to make sure that errors occurring during a scenario be clearly identified and traced. NeoLoad automatically identifies and reports HTTP-related errors or severe errors due to networking or application failures. However, NeoLoad cannot identify errors due to application logic, but provides a powerful validating mechanism that will let you locate unexpected or erroneous behavior. An application logic error can trigger chain reactions that could, in the end, cause an HTTP error: catching application errors as soon as they appear will make it much easier to understand and analyze the problem. The sooner an erroneous behavior is identified and traced, the quicker the resolution will be. Because NeoLoad provides mechanisms to validate application behavior, this technique can, to some extent, be used to achieve functional testing.

Finally, from a purely load testing point of view, it is also important to identify pages that are time costly. This section describes how to check that an HTTP response does not exceed a certain amount of time. This is consistent with non-functional requirements that are often expressed by users or clients and that you must ensure.

NeoLoad validations are expressed either for each HTTP request or for each Container or Virtual User. See Global Validation. The validation mechanism allows you to set conditions that must be met by the HTTP response to a specific HTTP request . NeoLoad provides three types of criteria by which validations can be expressed: time, content size and, lastly, the response content itself. This tutorial describes how these three criteria can be added and configured. It also shows you how to test your validations and where to configure NeoLoad behavior in the event the validation fails.


Add validations to an HTTP response

The example relies on the very simple HTML source code that follows

<HTML> 
<BODY> 
   <H1>Order references</H1> 
   <P> Thank you, your order has 
      been successfully submitted : 
      <B>#I45-020206-45RT347</B> 
   </P> 
<BODY> 
</HTML>

You may want your validation to apply to the contents of the HTTP response making sure that an order identifier has been correctly returned and also checking that the format of the order identifier is correct. The second validation will apply to a non-functional requirement: confirming and placing the order is a resource-greedy and time-consuming operation, we may want to make sure the response time does not exceed 2 seconds.


Define validations to an HTTP response

  1. Select the HTTP request for which the response will be validated:

    • Select the Design section.

    • Select the Virtual Users tab.

    • Unfold the appropriate HTML page node.

    • Select the HTTP request for which the response will be validated.

      In this case it is the/loadtest/validation request. An HTML page most of the time is composed of several HTTP requests (images, .css files, and so on). In the example, the page only has one request and you will be validating the response to that request.

    • Take a glance at the response.

    • At this stage -and before going to the validation steps- it usually is a good idea to take a look at the contents of the response you are intending to validate:

      • Click on the Advanced button.

      • Select the Response Contents tab. In the example, it shows the contents of the HTTP response you want to validate. As mentioned previously, we may want to apply your validation to the presence and correctness of the order identifier.

      In the example the contents of the response is simple and short but a realistic response could be much longer. You can search for specific text by right-clicking in the contents panel and selecting the Search option or using the CTRL+F shortcut keys. This will open a search panel at the bottom of the dialog box in which you can enter the target text and search for occurrences of that text.

    • Close the Advanced dialog box.

  2. Open the Validation dialog:

    • Click on the Validation button.

    • The Validation dialog has three panels, each defining one of the types of validation NeoLoad provides: assertions on the duration, the length of the content and assertions on the content itself. As described earlier, these are the three criteria NeoLoad uses to validate a response. In this case you will leave the size criteria aside. This could be used to validate image contents for instance, making sure the image returned has the required size.

    • At this stage you define the Duration criteria and enter "2000 ms". This is consistent with your requirements. The duration is the amount of time between the moment the HTTP request is sent and the moment the entire HTTP response has been received.

    • Add content validation:

      • Add an assertion

        Click on the + (plus) button in the Reply Content panel. NeoLoad automatically adds an assertion, called assertion_1 in this case. An assertion expresses a condition that must be met by the HTTP response. As you will see later, you can tell NeoLoad to stop when this assertion fails. This behavior is configured in the Virtual User containing this HTTP request. NeoLoad will stop executing those Virtual Users. In all cases, NeoLoad indicates, during the execution of a load test, where and why an assertion has failed.

      • Define the assertion

        Select the contains the regexp option. An assertion expresses a condition that must be met by the HTTP response. In this case the content we may want to validate is dynamic: an order identifier generated for each order that has been confirmed. You therefore choose the contains the regexp option. The default option contains would have been sufficient had you been looking for some static text. You will use a regular expression to identify the order identifier and make sure the identifier is correctly constructed. Using regular expressions is NeoLoad basic technique to match and extract dynamic content.

      • Define a regular expression

        Click on the wrench-like button. This opens the RegExp picker dialog box. The box is divided into two parts, the first one containing the contents of the HTTP response and the second the regular expression you will be defining to match part of the contents. While you are entering the regular expression, or part of it, NeoLoad dynamically tries to match the expression you are typing in with the contents of the response. This makes it easy to test the regular expression you are defining.

      In the example, we may want to match the order id. This identifier is made up of three distinct parts:

      A prefix, such as #I45, always starting with the character # and followed by a sequence of exactly three letters or digits. You use the #(.{3}) regular expression to match this part. The parentheses are used for grouping an expression. The dot character matches any character other than the new line character. The {3} expression indicates that there are exactly three characters other than the new line character.

      A middle part, such as -020206, starting with a dash and followed by a sequence of exactly six digits. You use the -(\d{6}) regular expression to match this part. The \d matches any digit and the {6} says there are exactly 6 digits. You could have refined this expression to match a date and force the month and day parts to match only consistent values.

      A suffix, such as -45RT347, starting with a dash and followed by a sequence of exactly seven letters or digits. You use the -(.{7}) regular expression to match this part. You could obviously improve this regular expression by being much more precise and forcing your expression to match a set of uppercase characters or digits. In this case we would use the following expression ([A-Z|0-9]{7}) , where the A-Z sub-expression matches any character from A to Z, 0-9 matches all the digits from 0 to 9 and the | between the two sub-expressions indicates that one of the sub-expressions must match. Finally, the square brackets define a character range. For this example you will keep to the simpler form.

      NeoLoad has highlighted the content that matches the regular expression you have defined.

      In the example the contents of the response is simple and short but a realistic response could be much longer. You can search for specific text by right-clicking in the contents panel and selecting the Search option or using the CTRL + F shortcut keys. This will open a search panel at the bottom of the dialog box in which you can enter the target text and search for occurrences of that text.

      The validation is now configured. To quickly and easily spot requests that have associated validations, select your Virtual User, right-click on its node and select the Flag requests item. NeoLoad adds a check mark to all the nodes (records, HTML pages and HTTP requests) in the User Paths using validations.

  3. Test your validation.

    Erroneous or incorrectly-declared validations generate as many problems as no validations at all, so it is important to make sure validations are consistent. This isn't always easy, especially with complex scenarios. Most of the time though, a foolproof way to test your validations is to test the Virtual User containing the requests that have response validations. The following procedure describes how:

    • Open the Check Virtual User dialog by clicking the Check button.

    • Test the Virtual User by clicking the Start Checking button.

      NeoLoad displays all the pages and requests that have been played back during the test. In this case all has gone well and no assertions have failed. This does not completely guarantee that your validations have been correctly configured, but it does give you a good indication.

    • When you select the failed response and select the Assertion option, NeoLoad displays the assertions that have failed and the reasons why they failed.

  4. Define NeoLoad behavior when assertions fail:

    • Although this final step is not about configuring validations, it is directly related to this subject, since it details how and where you can configure NeoLoad behavior when assertions fail.

    • Virtual Users are made up of pages and their associated HTTP requests and are grouped in User Paths. Virtual Users are in turn used to build Populations and, finally, Populations are used to build load scenarios. Virtual Users are the building blocks on which you can define how NeoLoad will behave when an assertion fails. You can either tell NeoLoad to ignore the failure and continue executing the Virtual User, or force NeoLoad to abort the Virtual User when such a failure occurs. The second option is strongly recommended as it ensures that load tests do not run pointlessly and makes it much easier for you to identify and determine the reasons for the assertion failures.

      For example, we may have decided to abort Virtual User execution when both errors and assertion failures occur. Errors are HTTP errors, such as 404 Page Not Found errors. They are automatically detected by NeoLoad.


Related links

For more information about creating Virtual Users, see Virtual Users.

For more information about regular expressions, see Regular expressions.


Home