MyCo Sample Performance Test Plan
Rational Performance Tester v8.1
WebSphere Portal v6.1
- Develop test plan
- Setup RPT infrastructure
- Develop RPT scripts for the business transactions
- Execute load test scenarios
- Review and recommendations
Overview
The following contains tasks and links to procedures for executing performance testing using a Rational Performance Tester v8
Tasks
Develop test plan
Tasks Status Review architecture diagrams and documentation
Define performance test objectives
Set scope
Confirm high level approach
Define test cases
Create test ids
Plan system activity monitoring
Define tuning procedures
Set DynaCache replication type
Define targeted maximum CPU and memory utilization at steady state
Define Test settings
Define schedule settings
Define stages
Review failover considerations
Determine web-based location for HTML Performance reports.
Define performance requirements in Tests
Setup RPT infrastructure
Develop RPT scripts for the business transactions
Tasks Status Create projects
Record Tests
Provide tests with variable data
Insert response code verification points into Tests
Create schedules
Run Tests to confirm success
Review Test logs for errors
Execute load test scenarios
Tasks Status Start performance testing
Sched1 1/2 hr. 30 Users Sched2 1 hr. 100 Users Sched3 4 hrs. 350 Users. Sched4 2-3 days. Long running test for memory.
Review and recommendations
Tasks Status Capture and deliver test results
Evaluate reports in steady-stage range
Test Results - Iterations
Test Results - Final numbers
Review schedule performance reports
Make export backups of projects.
Compare reports
Prepare final executive summary report
Define performance test objectives
- Baseline and tune Portal 6.1 Base (out of box) performance
- Baseline, compare, and tune Portal 6.1 Base with custom themes
- Performance test and tune custom portlets and WCM content (including Personalization)
- Determine a repeatable performance test process for Go-Live
Scope
- Develop portal test plan and process
- Develop RPT scripts for the business transactions.
- Define and execute load test scenario
- Deploy monitoring tools to collect vital statistics from the servers during the tests.
- Monitor portal's system and server resources during the tests.
- Identify bottleneck and work with appropriate teams to resolve it
- Analyze the performance test results and prepare preliminary report for test run.
- Prepare final executive summary report at the end of the engagement.
High level testing approach
- On stage, baseline without custom applications. No tuning after baseline.
- On stage, baseline with custom apps
- On stage, baseline with and without Wily
- On stage, test without caching and then with caching
- On stage, ramp up to 350 users
- On prod, testing on prod to verify instance is set up correctly.
MyCo will create and circulate a schedule to account for load testing periods and availability
MyCo will make tuning adjustments suggested by testing team in a reasonable amount of time.
Load tests may be executed for up to 2-3 days for long-term performance and memory leak tests.
Tuning adjustments requested by testing team should be implemented without requiring specific proof that the adjustment is required. Making adjustments, even in the absence of specific proof, is generally more expeditious than attempting to design a Test to isolate a single aspect of a single component within the system.
MyCo will add hardware and additional licenses of Portal if the system is deemed unable to meet the required performance goals after other reasonable tuning options have been exhausted
MyCo will be responsible for any code changes required for under-performing service or back end interfaces.
System Activity Monitoring
The following activities will be monitored during load testing process to draw a conclusion on the system performance and to fine-tune the same for better performance.
Client side:
- Running Vusers.
- Transaction response time.
- Error Statistics.
- Total Successful/Failed Transactions.
Server side:
- System Resources
- Processor Utilization
- Memory Utilization
- Portal Server Resources
- JVM(Garbage Collection Pattern)
- Connection Pool Size
- Thread Pool Size
- Dynacache Usage
- If the response time goals are not met, further monitoring will be performed to isolate the bottleneck.
Database
Capture deliver test results
For each Test execution, the following summary reports will be delivered to MyCo:
- Test Result - Pass/Fail
- Test Summary
- Transaction Response time - 90th Percentile
- Client Side Statistics.
- Server Resources Graph.
- Error details (If any)
Run Schedules
- Prior to running a Schedule...
- Configure browser settings
- Verify the RPT Agent controller is running
- Verify monitoring tools are working
- Split login into it's own Test and insert before Loop
- Run schedule
- Post run...
Schedules will be run for multiple iterations for early identification of any potential bottlenecks in the scripts.
To reuse tests in prod environment, we will reset server connection variables.
Define tuning procedures
MyCo and testing team will work together to tune the overall portal environment.
Testing team will tune...
- Portal
- JVM
- web servers
- caches
- Adjustments to developed code
MyCo will tuning...
- network
- databases
- Active Directory
- back end systems
- operating system
Load Test Performance Goals
Phase Resp. time SLA Target volume and transaction Numbers Concurrent active users Simultaneous users Performance Critical Transactions Ramp up / Ramp down rate Backend dependencies Baseline TBD 3384 pages /hr 382 unique logins /hr TBD TBD Test HTTP Server page
Test Portal
Portlet PageTBD N/A Baseline + Theme TBD 3384 pages /hr 382 unique logins /hr TBD TBD Test HTTP Server page
Test Portal
Portlet Page | ThemeTBD N/A custom TBD 8101 pages /hr 906 unique logins /hr TBD TBD Login/Logout Registration Activation
WCM pages other iteration-2 portletsTBD SunOne LDAP SeeBeyond
Test Settings
The following is a list of desired recorded Tests...
Test1 Test2 Test3 Define datapools DP1, DP2, ... DP1, DP2, ... DP1, DP2, ... Enable tests to use datapools
Associate test values with datapools
Define performance requirements in tests and schedules None None None Timeout 240000 ms (4 min) 240000 ms (4 min) 240000 ms (4 min) Timeout action Log error and continue execution Log error and continue execution Log error and continue execution Disable secondary requests Yes Yes Yes Clear cookie cache when the test starts Yes Yes Yes
Schedule settings