MyCo Sample Performance Test Plan
Rational Performance Tester v8.1
WebSphere Portal v6.1

 

+

Search Tips   |   Advanced Search

 

  1. Develop test plan
  2. Setup RPT infrastructure
  3. Develop RPT scripts for the business transactions
  4. Execute load test scenarios
  5. Review and recommendations



Overview

The following contains tasks and links to procedures for executing performance testing using a Rational Performance Tester v8



Tasks


Develop test plan

Tasks Status
Review architecture diagrams and documentation
Define performance test objectives
Set scope
Confirm high level approach
Define test cases
Create test ids
Plan system activity monitoring
Define tuning procedures
Set DynaCache replication type

Define targeted maximum CPU and memory utilization at steady state
Define Test settings
Define schedule settings
Define stages
Review failover considerations
Determine web-based location for HTML Performance reports.
Define performance requirements in Tests



Setup RPT infrastructure

Tasks Status
Review WAS and portal logs for errors
Set up remote agents
If using IPs, configure hosts file
Verify sufficient TCP/IP ports
For secure sites, disable security warnings
Disable any firewalls between workbench and agent computers
Set heap preferences
Increase memory allocation on remote computers
Deploy monitoring tools
Configure dynamic cache monitor
Configure extended cache monitor
Set up collection infrastructure



Develop RPT scripts for the business transactions

Tasks Status
Create projects
Record Tests
Provide tests with variable data
Insert response code verification points into Tests
Create schedules
Run Tests to confirm success
Review Test logs for errors



Execute load test scenarios

Tasks Status
Start performance testing
        Sched1 1/2 hr. 30 Users
        Sched2 1 hr. 100 Users
        Sched3 4 hrs. 350 Users.
        Sched4 2-3 days. Long running test for memory.



Review and recommendations

Tasks Status
Capture and deliver test results
Evaluate reports in steady-stage range
Test Results - Iterations
Test Results - Final numbers
Review schedule performance reports
Make export backups of projects.
Compare reports
Prepare final executive summary report



Define performance test objectives

  1. Baseline and tune Portal 6.1 Base (out of box) performance
  2. Baseline, compare, and tune Portal 6.1 Base with custom themes
  3. Performance test and tune custom portlets and WCM content (including Personalization)
  4. Determine a repeatable performance test process for Go-Live



Scope

  1. Develop portal test plan and process
  2. Develop RPT scripts for the business transactions.
  3. Define and execute load test scenario
  4. Deploy monitoring tools to collect vital statistics from the servers during the tests.
  5. Monitor portal's system and server resources during the tests.
  6. Identify bottleneck and work with appropriate teams to resolve it
  7. Analyze the performance test results and prepare preliminary report for test run.
  8. Prepare final executive summary report at the end of the engagement.



High level testing approach

  1. On stage, baseline without custom applications. No tuning after baseline.
  2. On stage, baseline with custom apps
  3. On stage, baseline with and without Wily
  4. On stage, test without caching and then with caching
  5. On stage, ramp up to 350 users
  6. On prod, testing on prod to verify instance is set up correctly.

MyCo will create and circulate a schedule to account for load testing periods and availability

MyCo will make tuning adjustments suggested by testing team in a reasonable amount of time.

Load tests may be executed for up to 2-3 days for long-term performance and memory leak tests.

Tuning adjustments requested by testing team should be implemented without requiring specific proof that the adjustment is required. Making adjustments, even in the absence of specific proof, is generally more expeditious than attempting to design a Test to isolate a single aspect of a single component within the system.

MyCo will add hardware and additional licenses of Portal if the system is deemed unable to meet the required performance goals after other reasonable tuning options have been exhausted

MyCo will be responsible for any code changes required for under-performing service or back end interfaces.



System Activity Monitoring

The following activities will be monitored during load testing process to draw a conclusion on the system performance and to fine-tune the same for better performance.

Client side:

Server side:

  • Database



    Capture deliver test results

    For each Test execution, the following summary reports will be delivered to MyCo:

    • Test Result - Pass/Fail
    • Test Summary
    • Transaction Response time - 90th Percentile
    • Client Side Statistics.
    • Server Resources Graph.
    • Error details (If any)



    Run Schedules

    1. Prior to running a Schedule...

      1. Configure browser settings
      2. Verify the RPT Agent controller is running
      3. Verify monitoring tools are working
      4. Split login into it's own Test and insert before Loop

    2. Run schedule

    3. Post run...

      1. Review logs to verify error-free run
      2. Review monitoring metrics
      3. Tune portal

    Schedules will be run for multiple iterations for early identification of any potential bottlenecks in the scripts.

    To reuse tests in prod environment, we will reset server connection variables.



    Define tuning procedures

    MyCo and testing team will work together to tune the overall portal environment.

    Testing team will tune...

    MyCo will tuning...

    • network
    • databases
    • Active Directory
    • back end systems
    • operating system



    Load Test Performance Goals

    Phase Resp. time SLA Target volume and transaction Numbers Concurrent active users Simultaneous users Performance Critical Transactions Ramp up / Ramp down rate Backend dependencies
    Baseline TBD 3384 pages /hr 382 unique logins /hr TBD TBD Test HTTP Server page
    Test Portal
    Portlet Page
    TBD N/A
    Baseline + Theme TBD 3384 pages /hr 382 unique logins /hr TBD TBD Test HTTP Server page
    Test Portal
    Portlet Page | Theme
    TBD N/A
    custom TBD 8101 pages /hr 906 unique logins /hr TBD TBD Login/Logout Registration Activation
    WCM pages other iteration-2 portlets
    TBD SunOne LDAP SeeBeyond




    Test Settings

    The following is a list of desired recorded Tests...


    Test1 Test2 Test3
    Define datapools DP1, DP2, ... DP1, DP2, ... DP1, DP2, ...
    Enable tests to use datapools


    Associate test values with datapools


    Define performance requirements in tests and schedules None None None
    Timeout 240000 ms (4 min) 240000 ms (4 min) 240000 ms (4 min)
    Timeout action Log error and continue execution Log error and continue execution Log error and continue execution
    Disable secondary requests Yes Yes Yes
    Clear cookie cache when the test starts Yes Yes Yes

     

    Schedule settings


    Baseline Baseline_Theme MSS MSS_Prod
    Host stage.myco.com stage.myco.com stage.myco.com prod.myco.com
    Host_Port 10040 10040 10040 10040
    Remote locations TBD TBD TBD TBD
    Stage users 30 100 400 400
    Stage durations .5 hr 1 hr 4 hr 4 hr
    UserGroup1 size 30% 30% 30% 30%
    UserGroup2 size 70% 70% 70% 70%
    UserGroup1 location TBD TDB TDB TDB
    UserGroup2 location TDB TDB TDB TDB
    Loop duration Infinite Infinite Infinite Infinite
    Loop iterations 4/minute 4/minute 4/minute 4/minute
    Logging levels All All All TBD
    Ramp up 5 users / minute 5 users / minute 5 users / minute 5 users / minute
    Delay time 1 sec 1 sec 1 sec 1 sec
    Error handling TBD TBD TBD TBD
    Think time 10 secs 10 secs 10 secs 10 secs
    Perf requirements TBD TBD TBD TBD
    Theme Bundle Custom Custom Custom