Messaging troubleshooting tips
These tips are to help you troubleshoot the WebSphere messaging configuration.
IBM recommends using the High Performance Extensible Logging (HPEL) log and trace infrastructure . We view HPEL log and trace information using the logViewer .
(iSeries) (Dist) To help you identify and resolve problems with messaging, we can use the WAS trace and logging facilities.
(ZOS) To help you identify and resolve problems with messaging, we can use the WAS trace and logging facilities as described in Set up component trace (CTRACE).
If we are having problems deploying or running applications that use the WAS messaging capabilities, see the following topics:
- Troubleshoot messaging
- Troubleshoot message-driven beans
- Troubleshoot service integration technologies
- Default messaging provider: Troubleshooting tips
If we see IBM MQ error messages or reason codes in WAS messages and logs, refer to the Messages section of the IBM MQ information center.
Check to see if the problem has been identified and documented, using the links in Diagnose and fix problems: Resources for learning.
Here is a set of tips to help you troubleshoot commonly-experienced problems. If we do not see a problem that resembles ours, or if the information provided does not solve our problem, contact IBM support for further assistance.
- IBM MQ resource adapter configuration is not automatically updated and requires manual maintenance
- java.lang.ClassNotFoundException exceptions occur when we install a fix pack
- Messages from IBM MQ for z/OS are not being consumed by JMS applications
- A JMS application can no longer send or receive messages
- An MDB listener fails to start
- Problems running JMS applications with security enabled
- Application server does not start when the zh_TW.EUC locale is set on Solaris
- Server memory consumption and java.lang.OutOfMemoryError exception when processing JMS messages
- TopicConnectionFactory attributes clash error
- WSEC5061E: The SOAP Body is not signed exception
- Error occurs when we use IBM MQ as an external JMS provider
- javax.jms.JMSException: MQJMS3024: unable to start MDB listener
- (ZOS) The IBM MQ messaging provider channel framework messages appear during server startup
WebSphere Q resource adapter configuration is not automatically updated and requires manual maintenance
Normally the IBM MQ resource adapter is automatically updated when applying a WAS fix pack. However, if we have manually updated the IBM MQ resource adapter on some nodes in the environment, applying a fix pack does not automatically update the resource adapter used by servers on those nodes.
To resolve this issue, see Maintaining the IBM MQ resource adapter.
java.lang.ClassNotFoundException exceptions occur when we install a fix pack
If, when installing a fix pack we see the following message, follow the instructions in Maintaining the IBM MQ resource adapter to try and resolve the problem:
J2CA0043E: An exception occurred while trying to instantiate a ResourceAdapter JavaBean instance for the installed ResourceAdapter defined by key #removed#
Messages from IBM MQ for z/OS are not being consumed by JMS applications
Messages from IBM MQ for z/OS are not being consumed by JMS applications deployed into WAS and that use connection factories or activation specifications
For more information about how to configure the Provider version property, see any of the following topics:
- Configure an activation specification for the IBM MQ messaging provider.
- Configure a unified connection factory for the IBM MQ messaging provider.
- Configure a queue connection factory for the IBM MQ messaging provider.
- Configure a topic connection factory for the IBM MQ messaging provider.
A JMS application can no longer send or receive messages
When we configure an application to use the default messaging provider, you associate it with either of the following resource sets:
- One or more message beans connected through Java Message Service (JMS) activation specifications.
- One or more enterprise beans connected through JMS connection factories and JMS destinations.
To help resolve this problem, use the following administrative console panels to inspect the configuration of the applications and JMS resources:
- For a view of the JMS resources for a given application see: Application messaging resources.
- For a view of the applications and JMS resources for a given default messaging provider destination see: Application resources for this destination.
An MDB listener fails to start
If an MDB listener deployed against a listener port fails to start, we should see the following message:
WMSG0019E: Unable to start MDB Listener {0}, JMSDestination {1} : {2}
To help resolve this problem, check the following factors:
- Check that the administrative resources have been configured correctly. For example, use the administrative console to check the listener port properties: Destination JNDI name and Connection factory JNDI name. Check that other properties of the listener port, destination, and connection factory are correct.
- Check that the queue exists and has been added to the JMS server.
- Check that the queue manager and JMS server have started.
- Check that the Remote Queue Manager Listener has started.
- (iSeries) If security is enabled, check that a component-managed authentication alias has been specified on the queue connection factory or topic connection factory used by the message-driven bean.(ZOS) If security is enabled, check that the user ID used to start the MDB listener is appropriately authorized. See Problems running JMS applications with security enabled.
Problems running JMS applications with security enabled
When we try to run a JMS application with security enabled, we can encounter authentication problems indicated by one or more of the following error messages:
WMSG0019E: Unable to start MDB Listener PSSampleMDB, JMSDestination Sample/JMS/listen : javax.jms.JMSSecurityException:This example indicates that the security credentials supplied are not valid.
To resolve this problem, check the security configuration:
- If the authentication mechanism is set to Application, the application must supply valid credentials.
- If the authentication mechanism is set to Container, configure the JMS connection factory with a container-managed authentication alias, and ensure that the associated user ID and password are valid. (ZOS) Alternatively, when running in bindings transport mode we can exploit the connector thread identity support.
MQJMS2013 invalid security authentication supplied for MQQueueManager:If we are using IBM MQ as a JMS provider, with JMS connection and using bindings transport mode, and the user specified is not the current logged-on user for the WAS process, the JMS bindings authentication by IBM MQ generates an invalid security authentication error.
To resolve this problem, check the security configuration. When we configure the IBM MQ JMS provider to use bindings transport mode, we set the property Transport type to BINDINGS on the IBM MQ queue connection factory. At this time, you also choose one of the following options:
- Use security credentials. To do this, ensure that the user specified is the currently logged-on user for the WAS process.
- Do not use security credentials. On the IBM MQ connection factory, ensure that the Component-managed Authentication Alias and the Container-managed Authentication Alias properties are not set.
For more information about messaging security, see Secure messaging.
(Dist)
Application server does not start when the zh_TW.EUC locale is set on Solaris
On Solaris, if we set the locale to zh_TW.EUC and we are using IBM MQ as a JMS provider, the application servers might not start up.
To resolve this problem, set the LANG and LC_ALL variables to zh_TW.
Server memory consumption and java.lang.OutOfMemoryError exception when processing JMS messages
When we use the default messaging provider, JMS messages are processed by a messaging engine within the application server process. This approach consumes memory from the application server JVM heap. If there is significant concurrent processing of large messages, and the amount of memory available to the JVM heap is not enough to handle this event, then a java.lang.OutOfMemoryError exception is thrown and the application server terminates.
To resolve this problem, estimate the potential number of concurrent processors or consumers of messages and the message sizes, then set the size of the application server JVM heap to handle the effect. For example:
- When we deploy a message-driven bean that processes messages concurrently, estimate the potential consumption of the application server memory by concurrent endpoints. Note that each endpoint that is concurrently processing a message request adds at least two times the message size to the server JVM heap and can add more, especially if a two-phase transaction is in place.
- Start the WAS appservers administrative console.
- Navigate to Servers > Server Types > WebSphere application servers > server > Java and Process Management > Process Definition > Java Virtual Machine, then configure the amount of memory available to the application server JVM heap by setting the Initial Heap Size and Maximum Heap Size properties.
- Navigate to Resources > JMS > JMS providers > Default messaging provider > Activation specifications > activation_specification_name, then configure the number of concurrent MDB endpoints that can process messages by setting the Maximum concurrent endpoints property of the activation specification for this message-driven bean.
TopicConnectionFactory attributes clash error
When we create a JMS topic subscriber that uses the IBM MQ messaging provider, the following error message can occur in the SystemOut.log file:
WSVR0017E: Error encountered binding the J2EE resource, TopicConnectionFactory, as <JNDI_NAME> from file:<RESOURCES_FILE> com.ibm.ws.runtime.component.binder.ResourceBindingException: invalid configuration passed to resource binding logic. REASON: Failed to create connection factory: Error raised constructing AdminObject, error code: TopicConnectionFactory attributes clash : TopicConnectionFactory attributes clash
This problem is caused by the configuration of the JMS topic connection factory used to create the subscriber, which specifies a broker version of "Basic" and a message selection value of "Broker". The "Basic" IBM MQ broker (MA0C SupportPac broker) does not support "Broker" message selection.
To resolve this problem, change the JMS topic connection factory to specify a message selection value of "Client", which is the only supported value for the IBM MQ Basic broker (MA0C SupportPac broker).
WSEC5061E: The SOAP Body is not signed exception
When we run a secured web services application that uses JMS transport under the IBM MQ messaging provider, the following error message can occur in the SystemOut.log file:
com.ibm.wsspi.wssecurity.SoapSecurityException: WSEC5061E: The SOAP Body is not signed.; null
This problem occurs under the following circumstances:
- A web service application, configured with Web Services Security, is running in an application server that has WAS security enabled.
- This web service application uses the JMS transport to send SOAP requests to a target web service.
- The JMS resource uses a remote IBM MQ server to connect to an IBM MQ queue.
- Another identical web service application, configured to use the same queue through the same IBM MQ server, is running in a different application server that does not have WAS security enabled.
The problem occurs when a request sent from the original application is processed through the same queue, but to the different application server where security is not enabled.
To resolve this problem:
- Create a unique queue manager with a unique port in the IBM MQ server.
- Reconfigure the JMS resources to use the new queue manager and port; for example, using the WAS administrative console to change the properties of the IBM MQ queue connection factory, as described in Configure a queue connection factory for the IBM MQ messaging provider.
- Rerun the application.
Error occurs when we use IBM MQ as an external JMS provider
When we use IBM MQ as an external JMS provider, and you send a message to an IBM MQ queue within a user-managed transaction, the message can arrive on the destination queue before the transaction commits. This problem occurs when the IBM MQ resource manager is not enlisted in the user-managed transaction.
To resolve this problem, use a container-managed transaction.
javax.jms.JMSException: MQJMS3024: unable to start MDB listener
This error can occur if we use an uninitialized client ID (that is, a client ID that is not associated with a durable subscription). To resolve this problem, set the client ID in one of the following three ways:
- Set the client ID as a property of the tcf, using the jmsadmin tool. For example, alter tcf(myTCF) clientid(myID).
- Set the client ID programmatically, using TopicConnection.setClientID()
- Set the client ID field administratively, using the administrative console to modify the IBM MQ messaging provider topic connection factory settings.
(ZOS) The IBM MQ messaging provider channel framework messages appear during server startup
The following message might be displayed several times in the control region adjunct process during server startup, even though the connection succeeds on subsequent retries. This message is issued because of the asynchronous way in which the z/OS TCP proxy channel starts and does not indicate any error has occurred.
Trace: 2009/06/17 08:24:41.434 01 t=9C6B58 c=UNK key=P8 (00000011) Description: Log Java Message Message: CHFW0030E: Error starting chain _InboundTCPProxyBridgeService because of exception com.ibm.wsspi.channel.framework.exception.RetryableChannelException: An exception was thrown when attempting to start the TCPProxyChannel com.ibm.ws.channel.framework.imp l.ChannelFrameworkImplThese messages might be accompanied by First Failure Data Capture (FFDC) output similar to the following example:
Exception = com.ibm.wsspi.channel.framework.exception.RetryableChannelException
Source = com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.startChainInternal
probeid = 2577
Stack Dump = com.ibm.wsspi.channel.framework.exception.RetryableChannelException:
An exception was thrown when attempting to start the TCPProxyChannel
at com.ibm.ws.tcpchannelproxy.jfap.impl.TCPProxyInboundChannel.
start(TCPProxyInboundChannel.java:153)
at com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.
startChannelInChain(ChannelFrameworkImpl.java:1410)
at com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.
startChainInternal(ChannelFrameworkImpl.java:2863)
at com.ibm.ws.channel.framework.impl.WSChannelFrameworkImpl.
startChainInternal(WSChannelFrameworkImpl.java:960)
at com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.
startChainInternal(ChannelFrameworkImpl.java:2794)
at com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.
startChain(ChannelFrameworkImpl.java:2779)
at com.ibm.ws.runtime.component.ChannelFrameworkServiceImpl.
startChain(ChannelFrameworkServiceImpl.java:666)
at com.ibm.ws.sib.jfapchannel.framework.impl.ChannelFrameworkReference$TCPProxy
BridgeServiceInboundChainStartupRunnable.run(ChannelFrameworkReference.java:1641)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1550)
Caused by: com.ibm.ws.tcpchannelproxy.jfap.NotYetInitializedException:
Server is not yet initialized
at com.ibm.ws.tcpchannelproxy.jfap.TCPProxyBridgeServicesImpl.
startListening(TCPProxyBridgeServicesImpl.java:558)
at com.ibm.ws.tcpchannelproxy.jfap.impl.TCPProxyInboundChannel.
start(TCPProxyInboundChannel.java:131)
... 8 more
Eventually the following message should be displayed indicating that the z/OS TCP proxy channel has started up correctly:
Trace: 2009/06/17 08:24:51.449 01 t=9C6B58 c=UNK key=P8 (13007002) ThreadId: 00000003 FunctionName: com.ibm.ws.channel.framework.impl.WSChannelFrameworkImpl SourceId: com.ibm.ws.channel.framework.impl.WSChannelFrameworkImpl Category: AUDIT ExtendedMessage: BBOO0222I: CHFW0019I: The Transport Channel Service has started chain _InboundTCPProxyBridgeService.
