Monitor MFT resources

Monitor MFT resources

We can monitor Managed File Transfer resources; for example, a queue or a directory. When a condition on this resource is satisfied, the resource monitor starts a task, such as a file transfer. We can create a resource monitor by using the fteCreateMonitor command or the Monitors view in the Managed File Transfer plug-in for IBM MQ Explorer.

About this task

A common scenario is to monitor a directory for the presence of a trigger file. An external application might be processing multiple files and placing them in a known source directory. When the application has completed its processing, it indicates that the files are ready to be transferred, or otherwise acted upon, by placing a trigger file into a monitored location. The trigger file can be detected by a Managed File Transfer monitor and the transfer of those files from the source directory to another Managed File Transfer Agent is initiated.
Two examples of monitoring a directory are as follows:

Monitor for a trigger file (for example trigger.file) and then transfer a wildcard (for example, *.zip)
Monitor for *.zip and then transfer ${FilePath} (for example, the file that triggered the transfer). For more details about variable substitution, see Customizing MFT tasks with variable substitution.
Do not create a monitor that monitors for *.zip, and then transfers *.zip. The monitor tries to start a transfer of *.zip for every .zip file on your system. That is, the monitor generates * number of transfers for *.zip.
To see an example of creating a resource monitor to monitor a directory, see Monitor a directory and using variable substitution.
An example of monitoring a queue is as follows:

An external application might be generating messages and placing them on a known queue with the same group ID. When the application has completed putting messages on the queue, it indicates that the group is complete. The complete group of messages can be detected by a Managed File Transfer monitor and the transfer of the group of messages from the source queue to a file is initiated.
For an example of creating a resource monitor to monitor a queue, see Example: Configuring an MFT resource. Managed File Transfer resource monitoring uses the following terminology:

monitor

A process that polls a resource (such as a directory or queue) at a predefined regular interval to see if the resource contents have changed. If they have, the contents are compared with the set of conditions for this monitor. If there is a match, the task for this monitor is started.

resource

The system resource the monitor examines every poll interval to be compared with the trigger conditions. Queues, directories, or nested directory structures can be the monitored resource.

condition

An expression that is evaluated (typically against the content of the monitored resource). If the expression evaluates to true, the condition contributes to the overall trigger condition.

trigger condition

The overall condition, which is satisfied when all conditions are satisfied. When the trigger condition is satisfied the task can proceed.

task

The operation that is started when the trigger condition or set of conditions is satisfied. Supported tasks are file transfer and command call.

trigger file

A file that is placed in a monitored directory to indicate that a task (typically a transfer) can begin. For example, it might indicate that all the files to be processed have arrived in a known location and can be transferred or otherwise acted upon. The name of the trigger file can be used to specify the files to be transferred by using variable substitution. For more information, see Customizing MFT tasks with variable substitution.
The trigger file is also known as ready file or go file. However, in this documentation it is always referred to as the trigger file.

From IBM MQ Version 9.0.3, Managed File Transfer includes resource monitor logging. For more information, see Logging MFT resource monitors.

Resource monitoring is not supported on protocol bridge agents or Connect:Direct® bridge agents.

MFT resource monitoring concepts
An overview of the key concepts of the Managed File Transfer resource monitoring feature.
Configure MFT monitor tasks to start commands and scripts
Resource monitors are not limited to performing file transfers as their associated task. We can also configure the monitor to call other commands from the monitoring agent, including executable programs, Ant scripts or JCL jobs. To call commands, edit the monitor task definition XML to include one or more command elements with corresponding command call parameters, such as arguments and properties.
Monitor a directory and using variable substitution
We can monitor a directory using the fteCreateMonitor command. The value of a substitution variable can be substituted in the task XML definition and used to define the transfer behavior.
Example: Configuring an MFT resource
We can specify an IBM MQ queue as the resource to be monitored by a resource monitor by using the -mq parameter with the fteCreateMonitor command.
Customizing MFT tasks with variable substitution
When the trigger conditions of an active resource monitor are satisfied, the defined task is called. In addition to calling the transfer or command task with the same destination agent or the same destination file name every time, we can also modify the task definition at run time. You do this by inserting variable names into the task definition XML. When the monitor determines that the trigger conditions are satisfied, and that the task definition contains variable names, it substitutes the variable names with the variable values, and then calls the task.
Examples: Variable substitution
Examples of variable substitution using XML and IBM MQ Explorer.
Monitor a queue and using variable substitution
We can monitor a queue and transfer messages from the monitored queue to a file by using the fteCreateMonitor command. The value of any IBM MQ message property in the first message to be read from the monitored queue can be substituted in the task XML definition and used to define the transfer behavior.
Monitor retry behavior for message-to-file transfers
If a message-to-file transfer that is triggered by a resource monitor fails and leaves the message group that triggered the monitor on the queue, that transfer is resubmitted at subsequent poll intervals. The number of times that the transfer is resubmitted is limited by the monitorGroupRetryLimit property of the monitoring agent.
Use a trigger file
We can use the contents of a trigger file in a resource monitor to define a set of files to transfer in a single transfer request. Each time a matching trigger file is detected, its contents are parsed for source file paths and optionally for destination file paths. These file paths are then used to define file items in the task transfer XML file that you specify, which is submitted as single transfer request to the agent. The definition of the resource monitor determines whether trigger content is enabled.
Logging MFT resource monitors
From IBM MQ Version 9.0.3, Managed File Transfer includes an improved method for obtaining diagnostic information about resource monitors.
Backing up and restoring MFT resource monitors
We can back up the resource monitors to have available for future use by exporting their definitions to an XML file that we can then import to create a new resource monitor from the backup.
Parent topic: Administer MFT

Related reference

Guidance for configuring an MFT resource monitor to avoid overloading an agent
fteCreateMonitor: create an MFT resource monitor
fteListMonitors: list MFT resource monitors
fteDeleteMonitor: delete an MFT resource monitor
MFT monitor request message formats