Restore a data store and recovering its messaging engine

When a failure occurs that cannot be dealt with by the system, we can restore the data store or data stores from a backup. Use this task to restore a backup of a data store and to recover its associated messaging engine afterward.

We should also restore the configuration files for the system, to ensure that it functions as it did at the time the backup was taken, for more information about why we should do this see Service integration backup. After we have restored the data store, we must restart the associated messaging engine.

When we restart a messaging engine after restoring a backup start it in Restart after restore mode, to minimize the effects of the messaging engine not being synchronized with any other messaging engines it was in communication with before the failure. If we restart the messaging engine in Normal mode, some of the new messages produced at this messaging engine might be discarded by the receiving messaging engine, for an indeterminate amount of time after restart. In Restart after restore mode, previously transmitted messages might be resent, potentially creating duplicates of messages that were produced before the backup was taken. However new messages are not lost or duplicated (if this is specified by the quality of service for the message).

We can restart a messaging engine in Restart after restore mode only using the wsadmin client; we cannot do it from the administrative console. We must only start a messaging engine in this mode when starting the messaging engine for the first time after restoring the backup. After the initial restart, we can undertake further restarts as usual.

Restart after restore mode is ignored if we start the server in Recovery mode. If we require both a Recovery mode start and a Restart after restore mode start:

1. Start the server in recovery mode
2. Wait for the startup to complete and for the server to stop

3. Start the messaging engine in Restart after restore mode

If we see the following message in the JVM System output file(Dist) SystemOut.log, it might indicate that we have restored from a backup and restarted the messaging engine without using the Restart after restore mode.

CWSIP0784E: Messaging engine: receivingME received a message from 
messaging engine: producingME that was not expected.

To resolve this issue, stop the messaging engine and restart it in Restart after restore mode.

This message might also appear in other situations, so we should restart the messaging engine in Restart after restore mode only if we know we have restored a backup.

For information about the JVM System output file(Dist) SystemOut.log and how to view it, see View JVM logs.

IBM recommends using the High Performance Extensible Logging (HPEL) log and trace infrastructure . We view HPEL log and trace information using the logViewer .

We can recover any number of messaging engines at the same time, by following the actions given for each messaging engine in turn.

Tasks

1. Change the initial state of the messaging engine to Stop, so that the messaging engine will not be automatically restarted by a server process:

   1. Use the administrative console to select the messaging engine by clicking Service integration -> Buses -> bus_name -> [Topology] Messaging engines -> engine_name.

   2. In the Initial state list, click Stopped.

   3. Click OK.

2. Save changes to the master configuration, ensuring that we select the Synchronize changes with Nodes check box.

3. Stop the messaging engine if it is running (see Stopping a messaging engine for instructions on how to do this). If the messaging engine does not respond, stop the server process hosting the messaging engine.

4. Restore the backup of the data store that is accessed by the messaging engine, by referring to Restore a data store.

5. Restore the backup of the configuration files using the backupConfig command (see Back up and restore administrative configuration files). This backup should have been taken at the same time as the data store backup.

6. Restart any servers that were stopped by the failure.

7. Restart the messaging engine in Restart after restore mode by performing the following steps:

   1. Start the wsadmin client.

      (iSeries) Note: (iSeries) The wsadmin scripting client is run from Qshell. (iSeries) See Configure Qshell to run WebSphere scripts .

      For more information about the wsadmin client, see wsadmin scripting tool.

   2. Invoke the start command, with the FLUSH parameter, on the MBean for the messaging engine. For example:

      wsadmin>myME=AdminControl.queryNames("type=SIBMessagingEngine,*").splitlines()[0]
      wsadmin>AdminControl.invoke(myME, "state")
      'stopped'
      wsadmin>AdminControl.invoke(myME, 'start', ["FLUSH"])
      wsadmin>AdminControl.invoke(myME, "state")
      'started'
      

   A number of messages might be output to the JVM SystemOut.log file to indicate the progress of the restart process.

8. Check the JVM SystemOut.log file for the following message that indicates that the restart was successful, in other words, no failures occurred while attempting to restart the messaging engine.

   CWSIP0783E: Messaging engine: messagingEngine started,
   flush of all delivery streams completed.
   

   If this message does not appear, a failure has occurred that has prevented the messaging engine from restarting. Resolve the cause of the failure and repeat the Restart after restore procedure until the restart is successful.

Related:

Service integration backup

Troubleshoot service integration technologies

Backing up a data store

Restore a data store

Use High Performance Extensible Logging to troubleshoot applications