MFT stand-alone file logger format

The format of message information written by the file logger can be defined in the FileLoggerFormat.xml file.

The configuration directory for the logger is located in MQ_DATA_PATH/mqft/config/coordination_qmgr_name/loggers/logger_name. When creating a new file logger, a version of this file is created that contains a default set of definitions used by the file logger. For more information about the default log format definition, see MFT stand-alone file logger default log format.

To specify your own custom log format, edit the FileLoggerFormat.xml file.

A custom log format definition

A log format definition consists of a set of message types with each message type having a format definition. A format definition for a message type consists of a set of inserts provided in XPATH format and a separator that is used to separate each insert. The ordering of the inserts determines the order in which the content is placed in the lines generated for output to the log files. For example, this is the definition for the callStarted message type:

<callStarted>
  <format>
    <inserts>
      <insert type="user"   width="19" ignoreNull="false">/transaction/action/
       @time</insert>
      <insert type="user"   width="48" ignoreNull="false">/transaction/@ID</insert>
      <insert type="system" width="6"  ignoreNull="false">type</insert>
      <insert type="user"   width="0"  ignoreNull="false">/transaction/agent/
       @agent</insert>
      <insert type="user"   width="0"  ignoreNull="false">/transaction/agent/@QMgr</insert>
      <insert type="user"   width="0"  ignoreNull="false">/transaction/job/name</insert>
      <insert type="user"   width="0"  ignoreNull="true">/transaction/transferSet/
       call/command/@type</insert>
      <insert type="user"   width="0"  ignoreNull="true">/transaction/transferSet/
      call/command/@name</insert>
      <insert type="system" width="0"  ignoreNull="true">callArguments</insert>
    </inserts>
    <separator></separator>
  </format>
</callStarted>

This format produces a line in the log file like this:

2011-11-25T10:53:04;414d5120514d5f67627468696e6b20206466cf4e20004f02;[CSTR];
  AGENT1;AGENT_QM;Managed Call;executable;echo;call test;

The inserts provided in the format definition are in the order in which the information appears on the line in the log file. For more information on the XML schema defining the format for the FileLoggerFormat.xml file, see Stand-alone file logger format XSD.

Message types

The FTE agents write a range of different message types to the SYSTEM.FTE/Log sub-topic. For more information, see SYSTEM.FTE topic. The log file definition can contain format definitions for these types of messages:

callCompleted
callStarted
monitorAction
monitorCreate
monitorFired
notAuthorized
scheduleDelete
scheduleExpire
scheduleSkipped
scheduleSubmitInfo
scheduleSubmitTransfer
scheduleSubmitTransferSet
transferStarted
transferCancelled
transferComplete
transferDelete
transferProgress

The format of the messages can vary. The majority of message types write a single line in the log file for each log message consumed from the SYSTEM.FTE/Log sub-topic. This leads to the simple case where the XPATH addresses provided in the log format definition relate to the root of the message. These are the message types that use this method to write output:

callCompleted
callStarted
monitorAction
monitorCreate
monitorFired
notAuthorized
scheduleDelete
scheduleExpire
scheduleSkipped
scheduleSubmitInfo
scheduleSubmitTransfer
transferStarted
transferCancelled
transferComplete
transferDelete

The other method used to write a log message uses multiple lines to represent the items in a transfer set within a log message. In this case the format provided is applied to each item in the transfer set within the log message. To include information that is specific to each item within the transfer set, then the XPATH provided is required to use the item as its XPATH root. These are the message types that use this method to write output:

scheduleSubmitTransferSet
transferProgress

A line of output is written for each item in the transfer set. Information that we want to be fixed for all items in a transfer set can still use XPATH addresses relative to the root of the log message. In the following simplified transferProgress format definition example it's the timestamp and transfer ID that are fixed. Any information that is relative to an item as its root will vary for each line written. In this example the source and destination file information for each item are written.

<transferProgress>
      <format>
        <inserts>
          <insert type="user"   width="19" ignoreNull="false">/transaction/action/
           @time</insert>
          <insert type="user"   width="48" ignoreNull="false">/transaction/@ID</insert>
          <insert type="system" width="6"  ignoreNull="false">type</insert>
          <insert type="user"   width="3"  ignoreNull="true">status/@resultCode</insert>
          <insert type="user"   width="0"  ignoreNull="false">source/file | 
           source/queue</insert>
          <insert type="user"   width="0"  ignoreNull="false">source/file/@size | 
           source/queue/@size</insert>
          <insert type="user"   width="5"  ignoreNull="true">source/@type</insert>
          <insert type="user"   width="6"  ignoreNull="true">source/@disposition</insert>
          <insert type="user"   width="0"  ignoreNull="false">destination/file | 
            destination/queue</insert>
          <insert type="user"   width="0"  ignoreNull="false">destination/file/@size |
            destination/queue/@size</insert>
          <insert type="user"   width="5"  ignoreNull="true">destination/@type</insert>
          <insert type="user"   width="9"  ignoreNull="true">destination/@exist</insert>
          <insert type="user"   width="0"  ignoreNull="true">status/supplement</insert>
        </inserts>
        <separator></separator>
      </format>
    </transferProgress>

This produces a log file entry of one or more lines in this format:

2011-11-25T13:45:16;414d5120514d5f67627468696e6b20206466cf4e20033702;[TPRO];0  
  ;/src/test1.file;3575;file;leave ;/dest/test1.file;3575;file;overwrite;;
2011-11-25T13:45:16;414d5120514d5f67627468696e6b20206466cf4e20033702;[TPRO];0  
  ;/src/test2.file;3575;file;leave ;/dest/test2.file;3575;file;overwrite;;

Insert Format

There are two types of insert available when defining a format for a message type: user and system. The type of an insert is defined in the type attribute of the insert element. Both types of inserts can also have their layout customized using the width and ignoreNull attributes of the insert element. For example:

<insert type="user" width="48" ignoreNull="false">/transaction/@ID</insert>

In this example, the insert takes the information found in the log message at /transaction/@ID and trims or pads it to 48 characters before writing it to the log. If the content of /transaction/@ID is null it writes the string null after padding it to 48 characters because the ignoreNull attribute is set to false. If ignoreNull is set to true the empty string, padded to 48 characters, is written instead. Setting width="0" means the column width is not trimmed, it does not mean that the width is trimmed to 0. The ignoreNull attribute can be used in this way to detect in the log when a null is found when it was not expected. This can be useful when debugging a new log file definition.

User defined inserts

A user insert contains an XPATH address for the information to be written in that insert. This address refers to a piece of information found in the FTE log message. For more information about log message formats, see:

System defined inserts

System defined inserts contain a keyword that refers to a piece of information that either cannot be found in the log message or is not easy to define using the XPATH language.

Supported systems inserts are:

type - Writes the type of the log message in a short format.
callArguments - Writes the set of arguments supplied to a managed call in a space separated format.
transferMetaData - Writes the set of metadata entries defined for a transfer in a comma separated key=value format.

The following table lists the value of "type" for system defined inserts for each message type.

Message type	Value of "type" system insert
callCompleted	[CCOM]
callStarted	[CSTR]
monitorAction	[MACT]
monitorCreate	[MCRT]
monitorFired	[MFIR]
notAuthorized	[AUTH]
scheduleDelete	[SDEL]
scheduleExpire	[SEXP]
scheduleSkipped	[SSKP]
scheduleSubmitInfo	[SSIN]
scheduleSubmitTransfer	[SSTR]
scheduleSubmitTransferSet	[SSTS]
transferStarted	[TSTR]
transferCancelled	[TCAN]
transferComplete	[TCOM]
transferDelete	[TDEL]
transferProgress	[TPRO]

Excluding message types from the MFT stand-alone file logger
To exclude a certain message type from the file logger output, we can use empty message type elements.
Defining custom formats for the MFT stand-alone file logger
It is possible to define a subset of custom message types within a log format definition to reduce the amount of configuration required to customize your log file format.
Reducing duplicate messages in the MFT stand-alone file logger
Duplicate log messages can occur in the log of the stand-alone file logger. By using the logger.properties file we can tune the stand-alone file logger and reduce the number of duplicates.