WAS v8.5 > Troubleshoot > Troubleshooting web services
Detecting and fixing problems with WS-ReliableMessaging
The nature of WS-ReliableMessaging is that network and server failures are assumed, and therefore the target web service or message store might not be available. In these cases, message sequences cannot be completed and collections of web service messages are held awaiting transmission. We can use the SystemOut.log file, system events, and the runtime administrative panels to monitor the system and detect and fix problems with WS-ReliableMessaging. If a sequence fails, a message is written to the application server SystemOut.log file and a system event is generated. Therefore we can detect failed sequences by looking at the SystemOut.log file, or by writing an event listener (or using third party software) to monitor system events.
IBM recommends using the HPEL log and trace infrastructure. With HPEL, one views logs using the LogViewer command-line tool in PROFILE/bin.
After a sequence has been established, WS-ReliableMessaging provides retransmission of messages to a service. However if the sequence is not established (that is, if the initial CreateSequence request is refused) then the messages are not transmitted to the service. For more information, see the troubleshooting tip A sequence is not established and WS-ReliableMessaging cannot ensure messages are transmitted.
For more detailed status information at run time, and facilities to help fix problems, use the WS-ReliableMessaging dmgr console runtime panels. These panels are available at many different scopes (for example cell; application server; messaging engine). For a full list of the WS-ReliableMessaging runtime panels, and details of the scopes at which they are available, see WS-ReliableMessaging - dmgr console panels.
At all scopes, the parent panel is Reliable messaging state settings. From this panel we can investigate each of the three key runtime aspects of reliable messaging:
The following icons are displayed here and on several other reliable messaging runtime panels:
| Icon | Name | Description |
|---|---|---|
| OK | Everything here, and (if there is a link) in all runtime panels below this link, is running normally. |
| Warning | Something here, or (if there is a link) in one of the runtime panels below this link, is in an unusual state and you might have to take some action to resolve it.
For example, the system might be awaiting a response from an endpoint. In this case, either the response will be received (in which case you need take no action and the runtime information will be updated to "OK") or the reliable messaging destination has stopped acknowledging messages (in which case we have to take some action to resolve the failed sequence). |
| Error | There is a definite error that you must take some action to resolve, either here or (if there is a link) in one of the runtime panels below this link. |
To use the reliable messaging runtime panels to detect and fix problems with WS-ReliableMessaging, complete one or more of the following steps:
- Investigate problems with message stores.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name > [Additional Properties] Reliable messaging state > Runtime > Message store. The list of reliable messaging storage managers for the current scope is displayed in the Message store page form.
For the managed qualities of service, the messages are written to a messaging engine. For the unmanaged non-persistent quality of service, the messages are stored in memory. For in-memory stores the only possible value is "Running". For messages stored by a messaging engine, the possible values are "Running" or "Messaging engine not contactable", probably because the messaging engine is not running. The "OK" icon indicates the message store is running. If the messaging engine is not contactable, the "Error" icon is displayed.
For each message store in the list, the name of the associated reliable messaging application is given in the description column. If a messaging engine is not contactable, restart the message store for that application.
- Investigate problems with inbound sequences.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name > [Additional Properties] Reliable messaging state > Runtime > Inbound sequences. The runtime state of each of the inbound sequences for the current scope is displayed in the Inbound sequence page form.
We can use a filter to look at sequences that are in a particular state (for example Failed due to missing message) or that have a large number of messages awaiting dispatch to applications. If the sequence status is Error, there is a problem with the sequence and the source server hosting the other end of the sequence has terminated it. If the sequence is active and there are a large number of messages awaiting dispatch to the application, then there might be a problem with the application or, if in-order delivery is specified, delivery might be held up because the sequence has gaps in it.
We can select one or more sequences, then use the buttons provided to dispatch the messages to their associated applications, to export the messages to compressed files, to close or terminate the selected sequences, or to delete the selected sequences and all their messages.
Delete or terminate sequences only if necessary. If you delete or terminate an active sequence, the resulting messaging behavior is unpredictable and can cause loss of messages. If you are not sure whether we can safely delete or terminate a sequence, do not delete or terminate it; the system automatically deletes sequences that have been inactive for 12 hours.
To see more detailed information about a particular sequence, click the Sequence identifier field. The Inbound sequences settings form is displayed. This detailed information includes addressing information to help identify the source of the sequence, and the value (true or false) for in-order delivery for the sequence. From this panel we can also display the following forms:
- The Acknowledgement state page form. (The ranges of message sequence numbers received from the WS-ReliableMessaging source. If more than one range is displayed, this indicates a gap in the messages received. If "In-order delivery" is selected for the sequence manager, messages with a sequence number greater than the lowest gap cannot be delivered to the application until the gap is closed.)
- The Inbound message page form.
(The messages on the inbound sequence. We can use this form to delete individual messages.)
- The Message settings form. (The contents of an individual message in the sequence.)
For more guidance on diagnosing problems with inbound sequences, see Diagnose the problem when a reliable messaging source cannot deliver its messages
- Investigate problems with outbound sequences.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name > [Additional Properties] Reliable messaging state > Runtime > Outbound sequences. The runtime state of each of the outbound sequences for the current scope is displayed in the Outbound sequence page form.
We can use a filter to look at sequences that are in a particular state. For example, the state Cannot contact the remote endpoint indicates the sequence has been established but the reliable messaging destination has stopped acknowledging messages (which, coupled with a high number of messages awaiting transmission, might indicate a potential problem). If the sequence status is Error, there is a problem with the sequence and the server hosting the other end of the sequence has terminated it.
We can select one or more sequences, and use one of the buttons provided to export the messages to compressed files, to close or terminate the selected sequences, or to delete the selected sequences and all their messages. For more information about deleting sequences, see Delete a failed WS-ReliableMessaging outbound sequence.
Delete or terminate sequences only if necessary. If you delete or terminate an active sequence, the resulting messaging behavior is unpredictable and can cause loss of messages. If you are not sure whether we can safely delete or terminate a sequence, do not delete or terminate it; the system automatically deletes sequences that have been inactive for 12 hours.
To see more detailed information about a particular sequence, click the Sequence identifier field. The Outbound sequences settings form is displayed. This detailed information includes addressing information to help identify the server at which the sequence is targeted. From this panel we can also display the following forms:
- The Outbound message page form.
(The messages on the outbound sequence. We can use this form to delete individual messages.)
- The Message settings form. (The contents of an individual message in the sequence.)
For more guidance on diagnosing problems with outbound sequences, see Diagnose and recovering a WS-ReliableMessaging outbound sequence that is in retransmitting state.
- The Outbound message page form.
(The messages on the outbound sequence. We can use this form to delete individual messages.)
Subtopics
- WS-ReliableMessaging sequence reallocation
In some situations, the WS-ReliableMessaging implementation can recover from a sequence-related fault, so the application can continue without having to process the fault itself. Your application must still process the fault if the recovery fails. - Diagnose the problem when a reliable messaging source cannot deliver its messages
If you know the sequence identifier, and the target URI for the messages, we can use the runtime dmgr console panels to examine the sequence and determine why a reliable messaging source is not delivering its messages into the application server. - Diagnose and recovering a WS-ReliableMessaging outbound sequence that is in retransmitting state
You have to resolve a sequence in retransmitting state, because messages are backing up awaiting delivery to the target service. The retransmission might be due to a network failure, or a failure of the target server. The problem should resolve automatically, but you might want to investigate the cause to speed up recovery. - Delete a failed WS-ReliableMessaging outbound sequence
You have to resolve an outbound sequence in failed state, so that messages can again be transmitted to the target service. A sequence in failed state shows an unrecoverable error. The sequence can no longer be used. If messages are being delivered in order, then the failed sequence must be resolved before a new sequence can be established. - WS-ReliableMessaging troubleshooting tips
Tips for troubleshooting your WS-ReliableMessaging configuration.
Related
WS-ReliableMessaging
Add assured delivery to web services through WS-ReliableMessaging
Troubleshoot applications with HPEL
Configure WS-SecureConversation to work with WS-ReliableMessaging
Reference:
WS-ReliableMessaging - requirements for interaction with other implementations
WS-ReliableMessaging - dmgr console panels
WS-ReliableMessaging: supported specifications and standards