Monitor and tune the application placement controller

Monitor and tune the application placement controller

The application placement controller is designed to work with the default settings. However, there can be times when fine tuning the application placement controller becomes necessary to ensure the best results.

Before we can tune application placement, we need to have dynamic application placement running. See: configuring dynamic application placement
APC administrative roles...

Monitor: Can view the information.
Operator: Can view the information on the configuration tab. Can change the settings on the runtime tab.
Configurator: Can change the configuration but not the runtime settings.
Administrator: Has all privileges.

We tune the application placement controller...

To adjust the amount of time before runtime tasks expire
To adjust the amount of time that passes before starting or stopping a server is considered a failure
To adjust the amount of time between placement changes for applications

Tasks

In the administrative console, click...
Operational policies > Autonomic managers > Application placement controller

Determine whether we want to modify the persistent settings in the configuration tab, which are permanent between stops and starts of the application placement controller, or test runtime settings in the runtime tab, which last for the life of the application placement controller. We can choose to modify the runtime settings and then find those settings are actually what we want to use as permanent settings. We can make runtime settings persistent settings by clicking Save to repository.
Modify the application placement controller settings, as needed.
The table following these steps contains the general configuration settings.

Define custom properties to change placement behavior.
We can specify the reservedMemoryFixed and reservedMemoryPercent custom properties to define how much node.memory to reserve for processes that are not related to WebSphere Application Server or Intelligent Management.

Click Apply or OK. Save our changes.
Iterate and modify the settings as needed.
Make changes until you discover the best possible configuration for the environment.

Setting Description
Enable Enable or disables the application placement controller. If we disable the application placement controller, we are disabling all autonomic operations for the dynamic clusters. After the placement controller is disabled, no dynamic changes occur with regard to the size and placement of applications on the dynamic cluster. This action is equivalent to turning all dynamic clusters to static clusters.
Approval timeout Controls how long the runtime task sits in the queue awaiting action before it expires automatically. Acceptable time values are 1 to 60 minutes. This setting is particular to the supervised operating mode. When the operating environment is set to run in supervised mode, the application placement controller creates tasks, but must wait for approval from the system administrator before making changes.
The application placement controller treats runtime tasks that are timed out as user-rejected tasks.

Server operation timeout Represents the amount of time, in minutes, that the application placement controller waits for a start or stop operation to complete before the operation is considered a failure.
Set this value to the predicted worst case time to start or stop a server.
Acceptable values are between 1 to 60 minutes. If a server fails to start before the timeout, the server is put into maintenance mode. The application placement controller avoids trying to start that server again until we manually fix the start issue and manually start the server. When the server starts, the application placement controller detects that the server has started.
Servers are not automatically placed in maintenance mode when the servers fail to start before the server operation timeout. Instead, a runtime task is generated a when the server fails to start. The application placement controller does not attempt to start this server again until the server is successfully started manually. When we successfully start the server manually, the application placement controller receives a notification that the server has started.

If we want the servers to be put into maintenance mode, we can set the maintenanceModeOnOperationFail custom property on the application placement controller.

To take servers out of maintenance mode after starting successfully, we can specify the unsetMaintenanceModeAfterStart custom property on the application placement controller.

Minimum time between placement change Amount of time that the application placement controller waits before initiating a new batch of changes. The application placement controller might wait for a batch of changes after completing previous changes, or encountering a timeout. Acceptable values can range from 1 minute to 24 hours.
When setting this value, consider the overhead associated with starting and stopping servers. Start and stop servers can take several minutes and might introduce an additional load to the nodes.

If we allow the placement controllers to readjust application placement too often, the added overhead negates the increased performance gains that can be earned by readjusting the size of dynamic clusters. For example, if a server takes one minute to start, and the setting for the minimum time between placement changes is 20 minutes, then placement changes have a performance impact of approximately 5%.

Set this value at least 20 to 30 times larger than the time necessary to start a server. A value greater than several hours prevents application placement changes from happening more than once a day. If we believe that traffic load and application demands adjust several times during a day, we might want to allow placement changes to occur more frequently.

The application placement controller is tuned for its best performance.

Related:

Overview of application placement
Configure dynamic application placement
Intelligent Management: administrative roles and privileges
PlacementControllerProcs.jacl script
Intelligent Management: application placement custom properties

Setting	Description
Enable	Enable or disables the application placement controller. If we disable the application placement controller, we are disabling all autonomic operations for the dynamic clusters. After the placement controller is disabled, no dynamic changes occur with regard to the size and placement of applications on the dynamic cluster. This action is equivalent to turning all dynamic clusters to static clusters.
Approval timeout	Controls how long the runtime task sits in the queue awaiting action before it expires automatically. Acceptable time values are 1 to 60 minutes. This setting is particular to the supervised operating mode. When the operating environment is set to run in supervised mode, the application placement controller creates tasks, but must wait for approval from the system administrator before making changes. The application placement controller treats runtime tasks that are timed out as user-rejected tasks.
Server operation timeout	Represents the amount of time, in minutes, that the application placement controller waits for a start or stop operation to complete before the operation is considered a failure. Set this value to the predicted worst case time to start or stop a server. Acceptable values are between 1 to 60 minutes. If a server fails to start before the timeout, the server is put into maintenance mode. The application placement controller avoids trying to start that server again until we manually fix the start issue and manually start the server. When the server starts, the application placement controller detects that the server has started. Servers are not automatically placed in maintenance mode when the servers fail to start before the server operation timeout. Instead, a runtime task is generated a when the server fails to start. The application placement controller does not attempt to start this server again until the server is successfully started manually. When we successfully start the server manually, the application placement controller receives a notification that the server has started. If we want the servers to be put into maintenance mode, we can set the maintenanceModeOnOperationFail custom property on the application placement controller. To take servers out of maintenance mode after starting successfully, we can specify the unsetMaintenanceModeAfterStart custom property on the application placement controller.
Minimum time between placement change	Amount of time that the application placement controller waits before initiating a new batch of changes. The application placement controller might wait for a batch of changes after completing previous changes, or encountering a timeout. Acceptable values can range from 1 minute to 24 hours. When setting this value, consider the overhead associated with starting and stopping servers. Start and stop servers can take several minutes and might introduce an additional load to the nodes. If we allow the placement controllers to readjust application placement too often, the added overhead negates the increased performance gains that can be earned by readjusting the size of dynamic clusters. For example, if a server takes one minute to start, and the setting for the minimum time between placement changes is 20 minutes, then placement changes have a performance impact of approximately 5%. Set this value at least 20 to 30 times larger than the time necessary to start a server. A value greater than several hours prevents application placement changes from happening more than once a day. If we believe that traffic load and application demands adjust several times during a day, we might want to allow placement changes to occur more frequently.