Configure the autonomic request flow manager

Operating Systems: AIX, HP-UX, Linux, Solaris, Windows, z/OS

Configure the autonomic request flow manager

You can fine tune the autonomic request flow manager (ARFM) by changing the default settings in the administrative console.

Before you begin

To change the settings on the autonomic request flow manager, have operator, configurator, or administrator administrative privileges. Operators can only view the information on the configuration tab, but can change the settings on the runtime tab. Configurators can change settings on the configuration tab, but cannot change settings on the runtime tab. Administrators have all privileges.
When security is enabled, some fields are not editable without proper security authorization.

About this task
The autonomic request flow manager contains the following components:

A compute power controller per target cell, such as a cell to which an ARFM gateway directly sends work. This is an HAManagedItem that runs in any node agent, on demand router (ODR), or deployment manager.
A gateway per used combination of protocol family, proxy process, and deployment target. A gateway runs in its proxy process. For HTTP and SIP, the proxy processes are the ODRs; for JMS and IIOP, the proxy processes are the WebSphere^® Application Server application servers.
A work factor estimator per target cell. This is an HAManagedItem that can run in any node agent, ODR, or deployment manager.
The gateways intercept and queue the incoming HTTP, SIP, and IIOP requests, while the controller provides control signals, or directions, to the gateways and the placement controller. The work profiler continually estimates the computational requirements of the various kinds of requests, based on observations of the system in operation. Working together, these components properly prioritize incoming requests.
If you are using the Compute Grid product with the WebSphere Virtual Enterprise product, the dynamic application placement function with the job scheduler is supported. The application placement controller, along with the scheduler and autonomic request flow manager, provides overload protection of servers as long as both online and batch workloads are on dynamic clusters. This overload protection is not supported for static clusters member. Since batch jobs can consume a lot of CPU and run for a long period of time, utilization limit might be exceeded.
When the WebSphere Virtual Enterprise product is installed with the Compute Grid product, the application placement controller is consulted by the job scheduler during its endpoint selection process. The custom property, UseAPCEndpointSelection; value = false, can be configured on the job scheduler to disable the application placement controller/job scheduler integration. Use this custom property to disable the application placement controller during the job scheduler's endpoint selection process.
Dynamic placement function with job scheduler is not supported on z/OS servers.
To modify the ARFM default settings in the administrative console page, click Operational policies > Autonomic managers > Autonomic request flow manager.

Procedure

Modify the appropriate ARFM settings as described in the table following this procedure.
Click OK or Apply when you have completed your changes.
Click Save to save the changes to the master repository.
Test the settings you have just defined and iterate as often as necessary to get the request flow performance that you want.

Example
The following table provides specific guidance for configuring each setting.

Field Purpose Tips for setting
Aggregation period Each ARFM gateway broadcasts aggregated statistics periodically, and this parameter specifies the period. The statistics reported by the gateways support: the runtime charting in the administrative console, the operation of ARFM controllers, the operation of the application placement controller, and the operation of work profilers.
When setting the aggregation period, ensure the value is high enough to allow for the collection of a sufficient number of performance samples. Samples are collected by the gateways for each request. A few hundred samples are necessary to produce a good statistical measure.
Use an example - requests associated with a service class run in 250 milliseconds, and on average 10 requests run concurrently. The concurrency value is calculated automatically, based on the cluster size and the resources in the environment. The concurrency value can be seen on the visualization panels, under the Runtime Operations category in the console. As a result, the service class handles about 40 requests per second. Therefore, setting the aggregation period value to 15 seconds results in the collection of 600 samples for each aggregation period. The metrics provided by a 600 sample survey are useful and reliable.
Setting an aggregation period value too low results in unreliable performance metrics. Performance metrics derived from fewer samples are more noisy and less reliable, then a higher sample size. Because the ARFM controller is activated when new statistics are produced, setting an aggregation period value that is too long results in less frequent recomputation of the control settings. Therefore, WebSphere Extended Deploymentbecomes less responsive to sudden changes in traffic intensities and patterns.
Control cycle length minimum This parameter defines how often the ARFM controller is activated. Controller activation is the process of evaluating inputs and producing new control settings as a result of the input received. The activation process for an ARFM controller is initiated when new statistics are received from one of its gateways AND the elapsed time since the previous activation is greater than or equal to the control cycle minimum length, or the controller has never activated before. This setting determines the control cycle length giving it a lower bound. For example, if you have just one ODR and set the aggregation period to 30 seconds and the control cycle minimum length to 60 seconds, you might find that one activation occurs at 12:00:00.0 and the next occurs 90.1 seconds later at 12:01:30.1 because the previous statistics arrival time was 12:00:59.9. To ensure a reliable control cycle of around 60 seconds, you would set the control cycle minimum length to 58 or 59 seconds.
Smoothing window This setting defines how sensitive the ARFM controller reaction is to the incoming gateway statistics, by allowing a concatenation of gateway statistics. For any gateway, its ARFM controller uses a running average of the last few statistics reports from that gateway. The smoothing window controls the number of reports that are combined.
A low smoothing window setting makes the controller more sensitive and react more quickly. However, a low parameter also creates a sensitive reaction to noise, or anomalies, in the data.
The product of the smoothing window and the aggregation period should be roughly the same as the actual control cycle length, which is sometimes slightly greater than the configured control cycle minimum length.
Maximum queue length
This parameter is used to bound the length of each ARFM queue to a maximum number of requests that may be held in queue. ARFM divides all incoming traffic into flows, and has a separate queue for each flow. Flow particulars include requests that have a particular service class, are served on a particular deployment target, or go through a particular ODR.
When a request arrives and its queue is full, the request is rejected.
A lower parameter in this field increases the possibility that a request will be rejected due to short-term traffic bursts, while a higher parameter in this field may allow requests to linger longer in the queues. Queued requests consume memory. The default setting is 1000, but you may want to experiment with this setting to find the one that is a best match for your environment.

Maximum CPU utilization
Maximum CPU usage

The ARFM provides overload protection, in addition to its prioritization capabilities. An ARFM will queue requests in its gateways to avoid overloading the application servers.
For this release, load is determined in terms of CPU utilization on the first tier of application servers. The maximum CPU utilization parameter tells ARFM how heavily to load the servers. During severe peak conditions this utilization limit may be exceeded briefly.

Higher values give better resource utilization; lower values give more robust operation. Real load is noisy and variable. The performance management techniques in WebSphere Virtual Enterprise react to changes in the load, but with some time delay. During that reaction time, the system might operate outside its configured region; this includes having higher CPU utilization than configured. Operation with one application server at 100% CPU utilization for multiple minutes has been observed to break some internal communication mechanisms, to the detriment of many features.
The performance management in this release of WebSphere Extended Deployment does not work well if the first tier of application server machines are loaded with other work besides WebSphere requests that arrive through HTTP through the ODRs.
This setting affects application placement. If the total predicted demand is above the Maximum CPU utilization limit, the placement controller uniformly reduces the demand of all the dynamic clusters before calculating best placement.
Admission control for CPU overload protection
The purpose of admission control for CPU overload protection is to deliberately not accept dialogs based on judgments concerning how much can be accepted without overloading the compute power in the nodes being managed and compromising the response time of the accepted messages.
Admission control for CPU overload protection applies only to HTTP and SIP; it does not apply to IIOP and JMS.
Enable it when queuing for CPU overload protection is not enough; when it is important to make deliberate refusals of some offered load.
Disabled by default. To configure:

Define service policies with achievable performance goals, and set the goal type of the policies either to response time or percentile, not discretionary.
In the ARFM panel, set the CPU utilization limit to no higher than 90%. Select the third button for Rejection Policy. The rejection policy determines whether the admission control for CPU overload protection is enabled and, if so, how the response time threshold used for admission control is related to the response time threshold that appears in the performance goal.
At the cell level, set a cell custom property named arfmInitialMsgDlgRatio. The value is a decimal-formatted float that is the initial estimate for the ratio of each of the dialog-continuing message flows to the dialog-initiating message flow within the same (protocol family, deployment target). That is, it is the number of incoming follow-up messages per dialog. Set arfmInitialMsgDlgRatio to a value that is comparable among the collection of all dialog-continuing message flows.
arfmInitialMsgDlgRatio is also relevant to queuing for CPU overload protection and differentiated service. This custom property is also relevant when dialog orientation for CPU overload protection and differentiated service is enabled.
Save your changes.

The admission control for CPU overload protection is working if, in a heavily loaded system, the CPU utilization is about the same as the setting for CPU overload protection.
Memory overload protection: Maximum percentage of the WebSphere Application Server heap size to useMemory overload protection: Maximum percentage of the WebSphere Application Server heap size to use. See Memory overload protection for more information.
Specifies the maximum percentage of the heap size to be used for each application server.

Maximum percentage of the WebSphere Application Server heap size to use. Set the value to less than 100. For WebSphere Extended Deployment V6.1 and earlier, the default is 95%.
Request rejection policy
Specifies the behavior for HTTP, SOAP and SIP requests that are associated with a performance goal when an overload condition is detected.
In V6.1, this value is not active. Requests are not rejected for CPU overload protection.

Choose among the options to determine when to reject messages to prevent the CPU from being overloaded. You can reject no messages, or specify a rejection threshold value that determines when to reject messages. The default is to reject no messages.
Discretionary work is assumed to have a response time threshold of 60 seconds.

What to do next

Use the mustGather documents to troubleshoot the autonomic request flow manager and application placement. The Support team provides and maintains the mustGather documents for each version of WebSphere Extended Deployment.

Related concepts

Memory overload protection

Related tasks

Manage the WebSphere Virtual Enterprise environment
Configure speed factors in multiple tier configurations
Defining a service policy

Related reference

Autonomic request flow manager custom properties
Autonomic request flow manager advanced custom properties
Administrative roles and privileges
Autonomic request flow manager cell-wide custom properties
arfmController.py script

Field	Purpose	Tips for setting
Aggregation period	Each ARFM gateway broadcasts aggregated statistics periodically, and this parameter specifies the period. The statistics reported by the gateways support: the runtime charting in the administrative console, the operation of ARFM controllers, the operation of the application placement controller, and the operation of work profilers.	When setting the aggregation period, ensure the value is high enough to allow for the collection of a sufficient number of performance samples. Samples are collected by the gateways for each request. A few hundred samples are necessary to produce a good statistical measure. Use an example - requests associated with a service class run in 250 milliseconds, and on average 10 requests run concurrently. The concurrency value is calculated automatically, based on the cluster size and the resources in the environment. The concurrency value can be seen on the visualization panels, under the Runtime Operations category in the console. As a result, the service class handles about 40 requests per second. Therefore, setting the aggregation period value to 15 seconds results in the collection of 600 samples for each aggregation period. The metrics provided by a 600 sample survey are useful and reliable. Setting an aggregation period value too low results in unreliable performance metrics. Performance metrics derived from fewer samples are more noisy and less reliable, then a higher sample size. Because the ARFM controller is activated when new statistics are produced, setting an aggregation period value that is too long results in less frequent recomputation of the control settings. Therefore, WebSphere Extended Deploymentbecomes less responsive to sudden changes in traffic intensities and patterns.
Control cycle length minimum	This parameter defines how often the ARFM controller is activated. Controller activation is the process of evaluating inputs and producing new control settings as a result of the input received. The activation process for an ARFM controller is initiated when new statistics are received from one of its gateways AND the elapsed time since the previous activation is greater than or equal to the control cycle minimum length, or the controller has never activated before.	This setting determines the control cycle length giving it a lower bound. For example, if you have just one ODR and set the aggregation period to 30 seconds and the control cycle minimum length to 60 seconds, you might find that one activation occurs at 12:00:00.0 and the next occurs 90.1 seconds later at 12:01:30.1 because the previous statistics arrival time was 12:00:59.9. To ensure a reliable control cycle of around 60 seconds, you would set the control cycle minimum length to 58 or 59 seconds.
Smoothing window	This setting defines how sensitive the ARFM controller reaction is to the incoming gateway statistics, by allowing a concatenation of gateway statistics. For any gateway, its ARFM controller uses a running average of the last few statistics reports from that gateway. The smoothing window controls the number of reports that are combined.	A low smoothing window setting makes the controller more sensitive and react more quickly. However, a low parameter also creates a sensitive reaction to noise, or anomalies, in the data. The product of the smoothing window and the aggregation period should be roughly the same as the actual control cycle length, which is sometimes slightly greater than the configured control cycle minimum length.
Maximum queue length	This parameter is used to bound the length of each ARFM queue to a maximum number of requests that may be held in queue. ARFM divides all incoming traffic into flows, and has a separate queue for each flow. Flow particulars include requests that have a particular service class, are served on a particular deployment target, or go through a particular ODR. When a request arrives and its queue is full, the request is rejected.	A lower parameter in this field increases the possibility that a request will be rejected due to short-term traffic bursts, while a higher parameter in this field may allow requests to linger longer in the queues. Queued requests consume memory. The default setting is 1000, but you may want to experiment with this setting to find the one that is a best match for your environment.
Maximum CPU utilization Maximum CPU usage	The ARFM provides overload protection, in addition to its prioritization capabilities. An ARFM will queue requests in its gateways to avoid overloading the application servers. For this release, load is determined in terms of CPU utilization on the first tier of application servers. The maximum CPU utilization parameter tells ARFM how heavily to load the servers. During severe peak conditions this utilization limit may be exceeded briefly.	Higher values give better resource utilization; lower values give more robust operation. Real load is noisy and variable. The performance management techniques in WebSphere Virtual Enterprise react to changes in the load, but with some time delay. During that reaction time, the system might operate outside its configured region; this includes having higher CPU utilization than configured. Operation with one application server at 100% CPU utilization for multiple minutes has been observed to break some internal communication mechanisms, to the detriment of many features. The performance management in this release of WebSphere Extended Deployment does not work well if the first tier of application server machines are loaded with other work besides WebSphere requests that arrive through HTTP through the ODRs. This setting affects application placement. If the total predicted demand is above the Maximum CPU utilization limit, the placement controller uniformly reduces the demand of all the dynamic clusters before calculating best placement.
Admission control for CPU overload protection	The purpose of admission control for CPU overload protection is to deliberately not accept dialogs based on judgments concerning how much can be accepted without overloading the compute power in the nodes being managed and compromising the response time of the accepted messages. Admission control for CPU overload protection applies only to HTTP and SIP; it does not apply to IIOP and JMS. Enable it when queuing for CPU overload protection is not enough; when it is important to make deliberate refusals of some offered load.	Disabled by default. To configure: Define service policies with achievable performance goals, and set the goal type of the policies either to response time or percentile, not discretionary. In the ARFM panel, set the CPU utilization limit to no higher than 90%. Select the third button for Rejection Policy. The rejection policy determines whether the admission control for CPU overload protection is enabled and, if so, how the response time threshold used for admission control is related to the response time threshold that appears in the performance goal. At the cell level, set a cell custom property named arfmInitialMsgDlgRatio. The value is a decimal-formatted float that is the initial estimate for the ratio of each of the dialog-continuing message flows to the dialog-initiating message flow within the same (protocol family, deployment target). That is, it is the number of incoming follow-up messages per dialog. Set arfmInitialMsgDlgRatio to a value that is comparable among the collection of all dialog-continuing message flows. arfmInitialMsgDlgRatio is also relevant to queuing for CPU overload protection and differentiated service. This custom property is also relevant when dialog orientation for CPU overload protection and differentiated service is enabled. Save your changes. The admission control for CPU overload protection is working if, in a heavily loaded system, the CPU utilization is about the same as the setting for CPU overload protection.
Memory overload protection: Maximum percentage of the WebSphere Application Server heap size to useMemory overload protection: Maximum percentage of the WebSphere Application Server heap size to use. See Memory overload protection for more information.	Specifies the maximum percentage of the heap size to be used for each application server.	Maximum percentage of the WebSphere Application Server heap size to use. Set the value to less than 100. For WebSphere Extended Deployment V6.1 and earlier, the default is 95%.
Request rejection policy	Specifies the behavior for HTTP, SOAP and SIP requests that are associated with a performance goal when an overload condition is detected. In V6.1, this value is not active. Requests are not rejected for CPU overload protection.	Choose among the options to determine when to reject messages to prevent the CPU from being overloaded. You can reject no messages, or specify a rejection threshold value that determines when to reject messages. The default is to reject no messages. Discretionary work is assumed to have a response time threshold of 60 seconds.