FFST: IBM MQ for UNIX and Linux systems

Describes the name, location, and contents of the First Failure Support Technology ( FFST ) files for UNIX and Linux systems.

For IBM MQ on UNIX and Linux systems, FFST information is recorded in a file in the /var/mqm/errors directory.

An FFST file contains one or more records. Each FFST record contains information about an error that is normally severe, and possibly unrecoverable. These records indicate either a configuration problem with the system or an IBM MQ internal error.

FFST files are named AMQ nnnnn.mm.FDC, where:

nnnnn: Is the ID of the process reporting the error
mm: Starts at 0. If the full file name already exists, this value is incremented by one until a unique FFST file name is found. An FFST file name can already exist if a process is reused.

An instance of a process will write all FFST information to the same FFST file. If multiple errors occur during a single execution of the process, an FFST file can contain many records.

In order to read the contents of a FFST file, you must be either the creator of the file, or a member of the mqm group.

When a process writes an FFST record, it also sends a record to syslog. The record contains the name of the FFST file to assist in automatic problem tracking. The syslog entry is made at the user.error level. See the operating-system documentation about syslog.conf for information about configuring this.

Some typical FFST data is shown in Figure 1.

Figure 1. FFST report for IBM MQ for UNIX systems

+-----------------------------------------------------------------------------+
|                                                                             |
| WebSphere MQ First Failure Symptom Report                                   |
| =========================================                                   |
|                                                                             |
| Date/Time         :- Mon January 28 2008 21:59:06 GMT                       |
| UTC Time/Zone     :- 1201539869.892015 0 GMT                                |
| Host Name         :- mqperfh2 (HP-UX B.11.23)                               |
| PIDS              :- 5724H7202                                              |
| LVLS              :- 7.0.0.0                                                |
| Product Long Name :- IBM MQ for HP-UX                                       |
| Vendor            :- IBM                                                    |
| Probe Id          :- XC034255                                               |
| Application Name  :- MQM                                                    |
| Component         :- xcsWaitEventSem                                        |
| SCCS Info         :- lib/cs/unix/amqxerrx.c, 1.204                          |
| Line Number       :- 6262                                                   |
| Build Date        :- Jan 25 2008                                            |
| CMVC level        :- p000-L050203                                           |
| Build Type        :- IKAP - (Production)                                    |
| UserID            :- 00000106 (mqperf)                                      |
| Program Name      :- amqzmuc0                                               |
| Addressing mode   :- 64-bit                                                 |
| Process           :- 15497                                                  |
| Thread            :- 1                                                      |
| QueueManager      :- CSIM                                                   |
| ConnId(2) QM      :- 4                                                      |
| Major Errorcode   :- OK                                                     |
| Minor Errorcode   :- OK                                                     |
| Probe Type        :- INCORROUT                                              |
| Probe Severity    :- 4                                                      |
| Probe Description :- AMQ6109: An internal IBM MQ error has occurred.        |
| FDCSequenceNumber :- 0                                                      |
|                                                                             |
+-----------------------------------------------------------------------------+
 
MQM Function Stack
amqzmuc0
xcsWaitEventSem
xcsFFST
 
MQM Trace History
Data: 0x00003c87
--} xcsCheckProcess rc=OK
--{ xcsRequestMutexSem
--} xcsRequestMutexSem rc=OK

...

The Function Stack and Trace History are used by IBM to assist in problem determination. In many cases there is little that the system administrator can do when an FFST report is generated, apart from raising problems through the IBM Support Center.

However, there are some problems that the system administrator might be able to solve. If the FFST shows out of resource or out of space on device descriptions when calling one of the IPC functions (for example, semop or shmget ), it is likely that the relevant kernel parameter limit has been exceeded.

If the FFST report shows a problem with setitimer, it is likely that a change to the kernel timer parameters is needed.

To resolve these problems, increase the IPC limits, rebuild the kernel, and restart the machine.

First Failure Support Technology ( FFST ) files and UNIX and Linux clients

FFST logs are written when a severe IBM MQ error occurs. They are written to the directory /var/mqm/errors.

These are normally severe, unrecoverable errors and indicate either a configuration problem with the system or an IBM MQ internal error.

The files are named AMQnnnnn.mm.FDC, where:

nnnnn is the process ID reporting the error
mm is a sequence number, normally 0

When a process creates an FFST it also sends a record to the system log. The record contains the name of the FFST file to assist in automatic problem tracking.

The system log entry is made at the user.error level.

First Failure Support Technology is explained in detail in First Failure Support Technology ( FFST ).