Highly available agents in Managed File Transfer

We can configure standard or bridge agents in MFT to run in a high availability (HA) configuration. A pair of agent instances with identical configurations are involved in the HA setup, where one instance is running on one machine while another instance is running on a different machine. Both instances are configured to connect to the same agent queue manager.

Overview

Only one, called the active instance, of the two instances, is processing file transfers, while the other instance, called the standby instance, is in a state whereby it can complete its initialization and take over running the file transfers, but does not process any file transfers.

When an active instance fails or loses connectivity to the queue manager, the standby instance completes its initialization, becomes active and starts processing file transfers. Any inflight transfers that were in progress when the active instance failed resume from the last known check point.

The following illustration shows a common configuration of active and standby agents:

Notes:

1. One instance of an agent is running on two different machines, with one of the instances as an active instance, and the other as the standby instance.
2. Only the active instance processes transfers. The standby instance is in an idle state, waiting for the active instance to go down.
3. The same set of agent queues is shared between two instances of an agent.

The active-standby agent instance mechanism works by taking a lock on a shared resource, like IBM MQ multi-instance queue managers. The agent instance that takes a lock on the shared resource becomes the active instance while the other instance (that fails to take a lock) becomes a standby instance.

The shared resource here is a new queue, SYSTEM.FTE.HA.<agent name>. This queue gets created automatically when an IBM MQ Version 9.1.4 or later agent is configured.

How the process works

To create an HA agent, you create an agent with identical configuration parameters on two machines by running the fteCreateAgent or fteCreateBridgeAgent command using the additional -x parameter together with the highlyAvailable agent property in the agent.properties file set to true. Notes:

• Both configurations must point to the same agent queue manager.
• Required agent queues must be created only once on the agent queue manager.

See the fteCreateAgent command for more information on the -x parameter and the agent.properties file for more information on the highlyAvailable agent property.

Note that running either command creates an MQSC file containing the scripts that are required to create IBM MQ objects in the agent queue manager and the <SYSTEM.FTE.HA.agent name queue whether or not you specify the -x parameter.

While creating a highly available agent configuration, the fteCreateAgent or fteCreateBridgeAgent command checks the existence of an instance of the same agent present elsewhere by subscribing to the SYSTEM.FTE/Agents/agent name topic. If an instance of the same agent is found, then either command creates the required configuration on the file system but does not publish the creation of the agent again.

When an agent starts in HA mode:

1. The agent attempts to open the SYSTEM.FTE.HA.agent name queue in an exclusive GET mode.
2. If the agent opens the SYSTEM.FTE.HA.agent name queue successfully, this becomes the active instance of an agent, and a further startup process continues.
3. If the attempt to open the SYSTEM.FTE.HA.agent name queue in an exclusive GET mode fails with the MQRC_OBJECT_IN_USE reason code, it means there is already an active instance of the agent running elsewhere. Therefore, this instance becomes the standby instance of the agent.

   The standby instance attempts to open the SYSTEM.FTE.HA.<agent name> queue at specified intervals. An additional agent property standbyPollInterval is provided for this purpose in the agent.properties file.

   With the default value, the standby instance attempts to open the SYSTEM.FTE.HA.agent name queue every five seconds. This repeats until the instance succeeds in opening the SYSTEM.FTE.HA.agent name queue, or is stopped using the fteStopAgent command.

Multiple standby instances

All the standby instances attempt to take the SYSTEM.FTE.HA.agent name queue in an exclusive GET mode, and the instance that is successful, after the active instance fails, becomes the active instance.

The active instance maintains information of all known standby instances, and publishes the information as part of the agent status publication. The output of fteShowAgentDetails command, the agent GET REST API response, and IBM MQ Explorer MFT plug in display information about all standby instances.

See the example outputs of the fteShowAgentDetails command and the agent GET REST API response for more information.

See MFT agent status messages for examples of agent status information in XML format.

Version requirement

The active and standby agents must be IBM MQ Version 9.1.4 or higher. Attention:

• We cannot configure or start IBM MQ versions prior to IBM MQ Version 9.1.4 in high availability modes.
• Both active and standby instances must run the same version of the code.

The version of the active and standby instances is validated to ensure that both instances are of the same version. A temporary dynamic queue is used for communication between the instances. Two agent properties, dynamicQueuePrefix and modelQueueName, defined in the agent.properties file, generate the name of the temporary dynamic queue.

• Required information for highly available agents in Managed File Transfer
  There are various types of information that we need to know about standard or bridge MFT agents that are running in a high availability configuration. This information includes the different methods by which the agent starts, how to identify the instance of the agent in the log file, and status information for the agent.
• Discarding instances, failover, and maintenance in highly available agents
  Highly available Managed File Transfer instances can be discarded, can be failed in various different ways, and might need maintenance.

Parent topic: Configure Managed File Transfer