Reducing duplicate messages in the MFT stand-alone file logger

Duplicate log messages can occur in the log of the stand-alone file logger. By using the logger.properties file we can tune the stand-alone file logger and reduce the number of duplicates.

Duplicate messages in the file logger log

In the case of a failure, a log message might be written to the log of the stand-alone file logger without the consumption of the log message from the SYSTEM.FTE/Log# topic being committed to IBM MQ . If this happens, when the stand-alone file logger restarts it will retrieve the same message a second time and write it to the log file again. Plan to handle the possibility of these duplicates when looking at the log files either manually or when processing them automatically. To assist in the detection of duplicates, the stand-alone file logger outputs the following message to the log file when it starts:

BFGDB0054I: The file logger has successfully started

Duplicates always happen around the start time of the stand-alone file logger, because this is when the last message read before the previous instance failed is processed. By knowing when the new instance has started we can detect if duplicates should be expected, and whether they need to be handled or not.

Reducing the number of duplicates

The stand-alone file logger groups together log messages it processes into transactions to improve performance. This batch size is the maximum number of duplicate messages you may see in the case of a failure. To reduce the number of duplicates we can tune the following property in the logger.properties file:

wmqfte.max.transaction.messages

For example, by setting this to 1 the maximum number of duplicated messages is reduced to 1. Be aware that modifying this value has an effect on the performance of our stand-alone file logger so thorough testing is required to ensure this does not adversely affect the system.

The logger.properties file is in the MQ_DATA_PATH/mqft/config/coordination_qmgr_name/loggers/logger_name directory. For more information on available properties and their effects, see MFT logger configuration properties