+

Search Tips   |   Advanced Search

Operating Systems: AIX, HP-UX, Linux, Solaris, Windows, z/OS

 

Configure emergency throttle


The on demand router (ODR) and associated autonomic managers are able to support business goals in times of intense request flows by making smart decisions about the work coming into the server. The autonomic request flow manager (ARFM) controls HTTP request prioritization in the ODR. At times, emergency conditions result when certain sensors detect such overloaded situations. These overload situations include extremely high node utilization, intermittent communication failures between ARFM controller and request scheduling gateways, and intermittent communication failures between AsyncPMI monitoring data producers and the gateways. To prevent prolonging of these conditions, should they occur, and the accompanying degradation in performance, the gateways are equipped with emergency throttle controllers that control, and safeguard request dispatch rates to backend nodes. ARFM is handled in the back end for IIOP/JMS requests.

The ARFM contains two parts: a controller and a gateway. The ARFM function is implemented, for each node group, by a controller plus a collection of gateways in the ODRs. The ARFM controller (triggered by the eWLM controller if available on the system) might initiate typical throttling directives to the gateways. In a typical mode, throttling directives come from the ARFM controller by way of the RatesMessages, and are immediately enforced at the gateway by the throttle controller.

A throttle is attached to each queue in the gateway, and is not in the throttle state by default. When an emergency occurs or when rate messages arrive from the ARFM controller, it receives directives from the throttle controller and changes to the throttled state.

In the event that one or more overload sensors detect overload condition, in spite of typical throttling, the gateway throttle controller enters emergency mode. An emergency blackout sensor senses communication failures between an ARFM controller and request scheduling gateways, or communication failures between AsyncPMI monitoring data producers and the gateways. The term blackout means that the sensor does not receive expected messages. In emergency mode, the throttle controller gradually reduces the dispatch rates of the gateway queues until the overloaded sensor(s) stops firing. Then it gradually restores the rates to their original, pre-emergency mode settings. While doing this, the throttle controller ensures that rate directives from ARFM controller are never exceeded, thereby preserving the integrity of throttling decisions made by different controllers. Working together, these components can properly limit incoming requests.

Multiple sensors detect emergency conditions, resulting in the throttle controller going into emergency mode. Each sensor can be in one of two states: fired or unfired. During an emergency, there are two phases for the throttle controller: emergency_throttle and emergency_unthrottle. During the emergency_throttle phase, the throttle reduces all queue rates as long as one of the sensors still fires. In the emergency_unthrottle phase, all the sensors return to the unfired state and gradually restore all queue rates to their original values they had before entering emergency mode. Emergency throttling is enabled by default (EnableEmergencyThrottlling=true), you can disable it by modifying a configuration file on the ODR host called arfm.cfg located in WAS_HOME/profiles/node/properties/arfm.cfg. To disable the emergency mode:

  1. Edit arfm.cfg.

  2. Add EnableEmergencyThrottling=false.
Enforcing rate directives from ARFM controller (initiated by eWLM) is enabled by default. You can disable it by modifying arfm.cfg as follows:

  1. Edit arfm.cfg.

  2. Add EnableExternalThrottling=false.
Other configuration parameters you can add to arfm.cfg :

  1. EmergencyRateChangeStep=x where x is an integer in the range 0 to 100 specifying the percentage change in rate in each step of the gradual reduction/increase of throttle rate.

    • Default is EmergencyRateChangeStep=20

  2. EmergencyRateChangeInterval=x where x is the time between two successive rate change steps in emergency mode in milliseconds.

    • Default is EmergencyRateChangeInterval=15000

  3. EmergencyBlackoutMultiplier=x where x is a multiplier multiplied by different normal message cycles used as input to emergency blackout sensor. The EmergencyBlackoutMultiplier is a configuration parameter which tells the sensor indirectly how long it should wait before firing. This interval is determined as WebSphere XD (multiplication) of this EmergencyBlackoutMultiplier and the normal anticipated interval between successive messages.

    • Default is EmergencyBlackoutMultiplier=2

  4. EmergencyCPUUtilLimit=x where x is an integer in the range 0 to 100 specifying the CPU utilization watermark, on backend nodes, that triggers emergency throttling.

    • Default is EmergencyCPUUtilLimit=100

  5. TokenBucketSizeMillis=x. This parameter indirectly specifies how many tokens can be accumulated in a queue's token bucket.

    • Default is TokenBucketSizeMillis=1000



Related tasks

Configure the autonomic request flow manager