Socket - Testing

Socket performance testing guidelines

Before testing the performance of TCP/IP socket-based applications, set up your test environment and incorporate these guidelines to produce reliable performance tests.

Limitations

You can use this extension to test applications that run in a client-server model, where the test simulates multiple clients that connect to one or several servers. Other models, such as peer-to-peer networks, are not supported.

IBM Rational Performance Tester does not support socket recording in the 64 bit versions of Microsoft Windows 2003 and Windows XP.

Performance

When you deploy performance tests, use a relevant number of vusers on a given computer is important. For example, if you deploy too many vusers on a single computer, the results will reflect more the load of the test computer than the load of the server.

For best results with performance tests on an average test computer with a 1 GHz processor and 1 GB of RAM, do not exceed 1000 concurrent vusers.

If you exceed the number of vusers that a single test computer can run, the measured performance of the server will be affected by the performance of the test computer, which will invalidate the final results.

When editing a schedule for long performance tests, use these guidelines:

• In the schedule editor, reduce the Test log level to None.
• In the schedule editor, set the Statistics sample interval to approximately 1/60 of the run time, for example 12 minutes for an estimated 12-hour session.
• When possible, use loops inside test suites rather than loops in the schedule. Using loops inside test suites avoids connection problems that might occur over long duration tests and emphasizes measurement of the send and receive activity rather than connection and close activity.

SSL/TLS Authentication

Socket tests support simple or strong SSL or Transport Layer Security (TLS) authentication mechanisms, also called server authentication and client authentication.

For server authentication, the client must determine whether the server can be trusted. When you are recording or running a socket test with a proxy recorder, the proxy recorder sits between the server and the client. Therefore, you must "trick" the client application into behaving as though the proxy recorder is the certified server by performing either one of the following actions:

• Configure the SSL or TLS settings of the recorder proxy to authenticate itself as the actual server to the client and as the client to the service. This means that you must supply the recording proxy with the adequate certificates.
• Configure a managed client (an external client application) to accept the proxy recorder as though it were the certified server. The recording wizard provides a link to download and import an IBM RPTcertificate into the client application.

For client authentication, the server must authenticate the test client according to its root authority. Therefore, you must provide the client certificate that is expected by the server to authenticate the proxy recorder or the test agent as a certified client.

TN3270 performance testing guidelines

Before testing the performance of TN3270 terminal applications, set up your test environment and incorporate these guidelines to produce reliable performance tests.

Limitations

You can use this extension to test applications that run on a TN3270 terminal emulation client, where the test simulates multiple terminals that connect to one or several servers.

These TN3270 terminal emulation packages are supported:

• IBM Personal Communications
• Attachmate EXTRA! X-treme

Performance

When deploying your performance tests, use a relevant number of vusers on a given computer. For example, if you deploy too many vusers on a single computer, the results reflect more the load of the test computer than the load of the server.

For best results with performance tests on an average test computer with a 1 GHz processor and 1 GB of RAM, do not exceed 1000 concurrent vusers.

If you exceed the number of vusers that a single test computer can run, the measured performance of the server is affected by the performance of the test computer, which invalidates the final results.

When editing a schedule for long performance tests, use these recommendations:

• In the schedule editor, reduce the Test log level setting to None.
• In the schedule editor, set the Statistics sample interval value to approximately 1/60 of the run time, for example 12 minutes for an estimated 12-hour session.
• When possible, use loops inside test suites rather than loops in the schedule. Using loops inside test suites avoids connection problems that might occur over long-duration tests and emphasizes measurement of the send and receive activity rather than connection and close activity.

Record a socket API performance test

You can record a socket API test from any client program on your computer. When you record, the recording wizard automatically starts the client program and records all the data that transits through the socket API.

Tests are stored in performance test projects. If your workspace does not contain a performance test project, the test creation wizard creates one with a name that you can change. To store a test in a specific project, verify that the project exists before you record the test.

Ensure that you have a working client program and that you can connect to the server.

Ensure that the session that you are recording is reproducible. This means that when the recorded actions are replayed by the test, the same responses from the server will be received.

IBM Rational Performance Tester does not support socket recording in the 64 bit versions of Microsoft Windows 2003 and Windows XP.

To record a socket test:

1. In the Performance Test perspective, click the New Test from Recording toolbar button or click File > New > Test from Recording.
2. In the New Test from Recording wizard, click Create a test from a new recording, select Socket Test, and click Next. If you are recording sensitive data, you can make a selection in Recording encryption level.
3. On the Select Location page, select the project and folder to create the test in, type a name for the test, and click Next. If necessary, click Create Parent Folder to create a new project or folder.
4. On the Select Client Application page, select the type of client program to use to record the test:
   • To specify any client program that is located on your computer, select Managed Application, and click Next.

     On the Managed Application Options page, click Browse to specify the Program path. If necessary, specify the Working directory, and inArguments type the command-line arguments that the program requires.

     If the program requires user input from a command-line interface, select Open console for user input.

   • To record a TN3270 terminal emulation session, select IBM Personal Communication or Attachmate EXTRA! X-treme if these programs are installed, and click Next.

     If required, specify a session file to start the TN3270 session.

     Using this method to record a TN3270 session produces a low-level socket API performance test that is based on the TN3270 protocol traffic.

   • To record an HTTP session, select Microsoft Internet Explorer or Mozilla Firefox, and click Next.

     If you choose Mozilla Firefox, you can specify a Firefox profile.

     Using this method to record an HTTP session produces a socket API performance test that is based on the HTTP traffic.

5. If the application uses SSL and Transport Layer Security (TLS) authentication to authenticate the server or the client application, specify the following options, and click Next:
   • Select The server requires a specific client certificate if you are using client authentication. Specify a certificate keystore file name and password. If multiple certificates are required, click Multiple certificates and specify a certificate keystore file name and password for each host name and port.
   • Select The client requires a specific server certificate to provide the certificate keystore file name of the server and a password for each host name and port.

     If you do not provide the server certificate, you must configure the client application to authenticate the certificate of the proxy recorder as though the proxy recorder were the actual server. Click Save this certificate to save the certificate generated by IBM RPT, and import the .cer file into the client application.

   If necessary, select whether to use SSL 3.0 and TLS 1.0 encryption.
6. If this is the first time that you record a socket API performance test, read the Privacy Warning, and select Accept to proceed.
7. Click Finish to start recording. A progress window opens while the client program starts.
8. Use the client program to perform the actions to test. You can use the Recorder Test Annotations toolbar to add comments, record synchronizations, or take screen captures during the recording.
   • To add a comment to the recorded test, click the Insert comment icon .
   • To add a screen capture to the recorded test, click the Capture screen icon . Screen and window captures make your tests easier to read and help you visualize the recorded test. You can change the settings for screen captures and add comments to images.
   • To manually add a test synchronization to the recording, click the Insert synchronization icon .
   • To manually add a transaction folder to the recording, click the Start Transaction icon and Stop Transaction icon to start and stop the transaction.
   • To insert a split point into the recorded test, click the Split point icon . With split points, you can generate multiple tests from a single recording, which you can replay in a different order with a schedule.
9. When you have finished test actions in the program, stop the recorder. You can do this by closing the client program or by clicking the Stop push button in the Recorder Control view. A progress window opens while the test is generated. On completion, the Recorder Control view displays the Test generation completed message, the Test Navigator lists your test, and the test opens in the test editor.

Record a TN3270 performance test

You can record a TN3270 test from a terminal emulation client. When you record, the recording wizard automatically starts the terminal emulation client and records all the screen and input activity that transits the socket connection.

Ensure that you have a TN3270 terminal emulation program installed on the local computer.

Tests are stored in performance test projects. If your workspace does not contain a performance test project, the test-creation wizard creates one with a name that you can change. To store a test in a specific project, verify that the project exists before you record the test.

Ensure that the session that you are recording is reproducible. This means that when the recorded actions are replayed by the test, the same responses from the server will be received.

To record a socket test:

1. In the Performance Test perspective, click the New Test from Recording toolbar button or click File > New > Test from Recording.
2. In the New Test from Recording wizard, click Create a test from a new recording, select TN3270 Test, and click Next. If you are recording sensitive data, you can select a Recording encryption level.
3. On the Select Location page, select the project and folder where to save the new test, type a name for the test, and click Next. If necessary, click Create Parent Folder to create a new project or folder
4. On the Select Client Application page, select the type of client program to use to record the test:
   • In most cases, select IBM Personal Communication or Attachmate EXTRA! X-treme, and click Next.

     If required, specify a session file to start the TN3270 session.

   • If you are using other TN3270 terminal emulation software, select Managed Application, and click Next.

     On the Managed Application Options page, click Browse to specify the program path. If necessary, specify the Working directory, and in Arguments, type the command-line arguments that the program requires.

     If the program requires user input from a command-line interface, select Open console for user input.

5. If this is the first time you record a socket API performance test, read the Privacy Warning, and select Accept to proceed.
6. Click Finish to start recording. A progress window opens while the TN3270 terminal program starts.
7. Use the TN3270 terminal program to perform the actions to test. You can use the Recorder Test Annotations toolbar to add comments, record synchronizations, or take screen captures during the recording.
   • To add a comment to the recorded test, click the Insert comment icon .
   • To add a screen capture to the recorded test, click the icon Capture screen. Screen and window captures make your tests easier to read and help you visualize the recorded test. You can change the settings for screen captures and add a comment to the image.
   • To manually add a test synchronization to the recording, click the Insert synchronization icon .
   • To manually add a transaction folder to the recording, click the Start Transaction icon and Stop Transaction icon to start and stop the transaction.
   • To insert a split point into the recorded test, click the Split point icon . With split points, you can generate multiple tests from a single recording, which you can replay in a different order with a schedule.
8. When you have finished test actions in the program, stop the recorder. You can do this by closing the TN3270 terminal program or by clicking the Stop push button in the Recorder Control view. A progress window opens while the test is generated. On completion, the Recorder Control view displays the Test generation completed message, the Test Navigator lists your test, and the test opens in the test editor.

Change test generation preferences

You can change the way that the test recorder organizes multiple send and receive elements in a new socket test by changing test generation preferences. To improve the readability of your test, you can merge consecutive send or receive elements that use the same connection.

To change the way that test elements are organized by default in a new test, you can change the test generation preferences before recording the test.

To merge or reorganize elements in an existing test, you can use the Organize wizard.

To merge send or receive elements in a new socket test:

1. Click Window > Preferences > Test > Test Generation > Socket Test Generation. The Socket Test Generation preferences window opens.
2. Select Strategies. You can create multiple organization strategies for handling different applications. Only one strategy is active during the recording.
3. Select Default Strategy or click New to create an organization strategy.
4. Click Settings.
5. In Edit Socket Strategy Settings, specify how you want the test recorder to generate multiple send and receive elements:

   Send elements

   Merge consecutive send elements

   Select this option to merge together all the consecutive socket send elements that use the same connection.

   Manipulate data with custom code

   Select this option to force all the selected send elements to enable the Manipulate data with custom code setting with the specified Class name of a custom Java. class that uses the API to process data in the socket send element.

   Receive Actions

   Do not merge

   Select this option to keep receive elements unmodified as they are initially recorded.

   Merge consecutive receive elements

   Select this option to merge together all the consecutive socket receive elements that use the same connection.

   Keep only last receive element

   Discard all multiple consecutive receive elements except the last one recorded.

   Response timeout

   The maximum delay (in seconds) to receive the first byte of the response. If no data is received before the end of the response timeout delay, the receive action produces an error in the test log. The response timeout counter starts when the receive action starts after the think time; the counter is interrupted when the first byte is received.

   End policy

   This option specifies when to stop receiving data and to move to the next test element.

   • Receives exact number of bytes: The receive action stops when the recorded number of bytes is received. Specify a Timeout (in seconds) after which the receive action produces an error in the test log, if the correct number of bytes is not received. If Link data size is enabled, the receive action expects the number of bytes displayed in the Data area. If Link data size is disabled, the receive action expects the number of bytes displayed in Bytes. This is the default setting
   • Receives until end of stream: The receive action stops when the connection is closed by the remote computer. If Accepts empty response is selected, then the reception of a single byte is not required and the Response Timeout is ignored. Specify a Timeout (in seconds) after which the receive action produces an error in the test log, if the correct number of bytes is not received.
   • Matches a string: The receive action stops when a specified sequence of bytes is received. Specify a Timeout (in seconds) after which the receive action produces an error in the test log, if the correct number of bytes is not received.
   • Recognizes a regular expression: The receive action stops when a sequence of bytes that matches a regular expression is received. Specify a Timeout (in seconds) after which the receive action produces an error in the test log, if the correct number of bytes is not received.
   • Delegated to custom code: The receive action stops when a condition is met in a custom Java class. This setting allows great flexibility, but requires coding of a custom Java class following the Rational Performance Tester extension API. Click Generate Code to generate a template based on the API or View Code to open the specified class in the Java editor.

   Except when the Receives until end of stream policy is in force, receive actions produce an error in the test log when the connection is closed by the remote computer.

   Timeout

   For end policies that have a Timeout setting, this setting specifies a delay (in seconds) after which the receive action produces an error in the test log if the end policy criteria is not met. The timeout counter starts when the first byte is received.

6. Click OK to apply the changes, and close the Preferences window.