Solving publish/subscribe problems

 

Overview

This section describes some problems that can occur when you develop JMS client applications that use the publish/subscribe domain. It discusses problems that are specific to the publish/subscribe domain. Refer to Handling errors and Solving problems for more general troubleshooting guidance.

 

Incomplete publish/subscribe close down

It is important that JMS client applications surrender all external resources when they terminate. To do this, call the close() method on all objects that can be closed once they are no longer required. For the publish/subscribe domain, these objects are:

  • TopicConnection
  • TopicSession
  • TopicPublisher
  • TopicSubscriber

The WebSphere MQ classes for Java Message Service implementation eases this task by using a cascading close. With this process, a call to close on a TopicConnection results in calls to close on each of the TopicSessions it created. This in turn results in calls to close on all TopicSubscribers and TopicPublishers the sessions created.

To ensure the proper release of external resources, call connection.close() for each of the connections that an application creates.

There are some circumstances where this close procedure might not complete. These include:

  • Loss of a WebSphere MQ client connection

  • Unexpected application termination

In these circumstances, the close() is not called, and external resources remain open on the terminated application's behalf. The main consequences of this are:

Broker state inconsistency
The WebSphere MQ Message Broker might contain registration information for subscribers and publishers that no longer exist. This means that the broker might continue forwarding messages to subscribers that will never receive them.

Subscriber messages and queues remain
Part of the subscriber deregistration procedure is the removal of subscriber messages. If appropriate, the underlying WebSphere MQ queue that was used to receive subscriptions is also removed. If normal closure has not occurred, these messages and queues remain. If there is broker state inconsistency, the queues continue to fill up with messages that will never be read.

Additionally, if the broker resides on a queue manager other than the application's local queue manager, correct operation of WebSphere MQ JMS depends on the communication channels between the two queue managers. If these channels fail for any reason, problems such as the above can occur until the channels restart. When diagnosing problems relating to channels, be careful not to lose WebSphere MQ JMS control messages on the transmission queue.

 

Subscriber cleanup utility

To avoid the problems associated with non-graceful closure of subscriber objects, WebSphere MQ JMS includes a subscriber cleanup utility that attempts to detect any earlier WebSphere MQ JMS publish/subscribe problems that could have resulted from other applications. This utility runs transparently in the background and should not affect other WebSphere MQ JMS operations. If a large number of problems are detected against a given queue manager, you might see some performance degradation while resources are cleaned up.

Close all subscriber objects gracefully whenever possible, to avoid a buildup of subscriber problems.

The exact behavior of the utility depends on the subscription store in use:

Subscriber cleanup with SUBSTORE(QUEUE)
When using the queue-based subscription store, cleanup runs on a queue manager when the first TopicConnection to use that physical queue manager initializes.

If all the TopicConnections on a given queue manager close, when the next TopicConnection initializes for that queue manager, the utility runs again.

The cleanup utility uses information found on the SYSTEM.JMS.ADMIN.QUEUE and SYSTEM.JMS.PS.STATUS.QUEUE to detect previously failed subscribers. If any are found, it cleans up associated resources by deregistering the subscriber from the broker, and cleaning up any unconsumed messages or temporary queues associated with the subscription.

Subscriber cleanup with SUBSTORE(BROKER)
With the broker-based subscription store, cleanup runs regularly on a background thread while there is an open TopicConnection to a particular physical queue manager. One instance of the cleanup thread is created for each physical queue manager to which a TopicConnection exists within the JVM.

The cleanup utility uses information found on the SYSTEM.JMS.REPORT.QUEUE (typically responses from the publish/subscribe broker) to remove unconsumed messages and temporary queues associated with a failed subscriber. It can be a few seconds after the subscriber fails before the cleanup routine can remove the messages and queues.

Two properties on the TopicConnectionFactory control behavior of this cleanup thread: CLEANUP and CLEANUPINT. CLEANUPINT determines how often (in milliseconds) cleanup is executed against any given queue manager. CLEANUP takes one of four possible values:

CLEANUP(SAFE)
This is the default value.

The cleanup thread tries to remove unconsumed subscription messages or temporary queues for failed subscriptions. This mode of cleanup does not interfere with the operation of other JMS applications.

CLEANUP(STRONG)
The cleanup thread performs as CLEANUP(SAFE), but also clears the SYSTEM.JMS.REPORT.QUEUE of any unrecognized messages.

This mode of cleanup can interfere with the operation of JMS applications running with later versions of WebSphere MQ JMS. If multiple JMS applications are using the same queue manager, but using different versions of WebSphere MQ JMS, only clients using the most recent version of WebSphere MQ JMS must use this option.

CLEANUP(NONE)
In this special mode, no cleanup is performed, and no cleanup thread exists. Additionally, if the application is using the single-queue approach, unconsumed messages can be left on the queue.

This option can be useful if the application is distant from the queue manager, and especially if it only publishes rather than subscribes. However, one application must clean up the queue manager to deal with any unconsumed messages. This can be a JMS application with CLEANUP(SAFE) or CLEANUP(STRONG), or the manual cleanup utility described in Manual cleanup.

CLEANUP(ASPROP)
The style of cleanup to use is determined by the system property com.ibm.mq.jms.cleanup, which is queried at JVM startup.

This property can be set on the Java command-line using the -D option, to NONE, SAFE, or STRONG. Any other value causes an exception. If not set, the property defaults to SAFE.

This allows easy JVM-wide change to the cleanup level without updating every TopicConnectionFactory used by the system. This is useful for applications or appservers that use multiple TopicConnectionFactory objects.

Where multiple TopicConnections exist within a JVM against the same queue manager, but with differing values for CLEANUP and CLEANUPINT, the following rules are used to determine behavior:

  1. A TopicConnection with CLEANUP(NONE) does not attempt to clean up immediately after its subscription has closed. However, if another TopicConnection is using SAFE or STRONG cleanup, the cleanup thread eventually cleans up after the subscription.

  2. If any TopicConnection is using STRONG Cleanup, the cleanup thread operates at STRONG level. Otherwise, if any TopicConnection uses SAFE Cleanup, the cleanup thread operates at SAFE level. Otherwise, there is no cleanup thread.

  3. The smallest value of CLEANUPINT for those TopicConnections with SAFE or STRONG Cleanup is used.

 

Manual cleanup

If you use the broker-based subscription store, you can operate cleanup manually from the command-line. The syntax for bindings attach is: 

Cleanup [-m <qmgr>] [-r <interval>]
          [SAFE | STRONG | FORCE | NONDUR] [-t]

or, for client attach: 

Cleanup -client [-m <qmgr>] -host <hostname> [-port <port>]
         [-channel <channel>] [-r <interval>]
         [SAFE | STRONG | FORCE | NONDUR] [-t]

Where:

  • qmgr, hostname, port, and channel determine connection settings for the queue manager to clean up.

  • -r sets the interval between executions of cleanup, in minutes. If not set, cleanup is performed once.

  • -t enables tracing, to the mqjms.trc file.

  • SAFE, STRONG, FORCE, and NONDUR set the cleanup level, as follows:

    • SAFE and STRONG cleanup behave like the CLEANUP(SAFE) and CLEANUP(STRONG) options discussed in Subscriber cleanup utility.

    • FORCE cleanup behaves like STRONG Cleanup. However, STRONG cleanup leaves messages that could not be processed on the SYSTEM.JMS.REPORT.QUEUE; FORCE cleanup removes all messages even if it encounters an error during processing.

      This is a dangerous option that can leave an inconsistent state between the queue manager and the broker. It cannot be run while any WebSphere MQ JMS publish/subscribe application is running against the queue manager; doing so causes the cleanup utility to abort.

    • NONDUR behaves like FORCE cleanup.

      After clearing the SYSTEM.JMS.REPORT.QUEUE, it attempts to remove any remaining unconsumed messages sent to non-durable subscribers. If the queue manager's command server is running on any queue beginning SYSTEM.JMS.ND.*, messages are cleared and the queue itself might be deleted. Otherwise, only SYSTEM.JMS.ND.SUBSCRIBER.QUEUE and SYSTEM.JMS.ND.CC.SUBSCRIBER.QUEUE are cleared of messages.

 

Cleanup from within a program

You can use a programming interface to the cleanup routines for use with SUBSTORE(BROKER), through the class com.ibm.mq.jms.Cleanup. Instances of this class have getter/setter methods for each of the connection properties; and also for the cleanup level and interval.

It exposes two methods:

cleanup()
Executes cleanup once

run()
Runs cleanup at intervals determined by the properties of the cleanup object

This class allows complete customization of publish/subscribe Cleanup, but it is intended for use by system administration programs rather than application programs.

For more details, refer to Cleanup *.

 

Handling broker reports

The WebSphere MQ JMS implementation uses report messages from the broker to confirm registration and deregistration commands. These reports are normally consumed by the WebSphere MQ classes for Java Message Service implementation, but under some error conditions, they might remain on the queue. These messages are sent to the SYSTEM.JMS.REPORT.QUEUE queue on the local queue manager.

A Java application, PSReportDump, is supplied with WebSphere MQ classes for Java Message Service, which dumps the contents of this queue in plain text format. The information can then be analyzed, either by you, or by IBM support staff. You can also use the application to clear the queue of messages after a problem is diagnosed or fixed.

The compiled form of the tool is installed in the <MQ_JAVA_INSTALL_PATH>/bin directory. To invoke the tool, change to this directory, then use the following command: 

java PSReportDump [-m queueManager] [-clear]

where:

-m queueManager
The name of the queue manager to use

-clear
Clear the queue of messages after dumping its contents

Attention: Do not use this option if you are using the broker-based subscription store. Instead, run the manual cleanup utility at FORCE level.

Output is sent to the screen, or you can redirect it to a file.

 

WebSphere is a trademark of the IBM Corporation in the United States, other countries, or both.

 

IBM is a trademark of the IBM Corporation in the United States, other countries, or both.