Enable batch job events publishing

By using Java Messaging System (JMS), the batch server can publish job-related events to external clients.

The ability of the batch server to publish job-related events to external clients enables a monitor to see job-related events and report on failures. The batch dispatcher server can publish events for a job in the dispatching phase. The batch executor server can publish events for jobs when it moves through different phases of execution. These events are published in a topic tree in the following structures:

Structure Description
batch The default root of the topic tree.

Note: In version 17.0.0.1 and later, the root of the topic tree is configurable.

batch/jobs The topic tree for all job-related events.
   
batch/jobs/instance The topic tree for all events related to a job instance.
batch/jobs/instance/submitted A topic tree node. A message is published when the batch server creates a job instance for a new job submission.
batch/jobs/instance/jms_queued A topic tree node. A message is published when job submission is placed on the job submission queue by the batch JMS dispatcher.
batch/jobs/instance/jms_consumed A topic tree node. A message is published when the batch executor receives the job submission from the job submission queue.
batch/jobs/instance/dispatched A topic tree node. A message is published when the batch executor accepts a job instance for execution.
batch/jobs/instance/completed A topic tree node. A message is published when the job instance is completed.
batch/jobs/instance/stopped A topic tree node. A message is published when the job instance is stopped.
batch/jobs/instance/stopping A topic tree node. A message is published when the job instance is stopping.
batch/jobs/instance/failed A topic tree node. A message is published when the job instance failed.
batch/jobs/instance/purged A topic tree node. A message is published when a job instance is purged successfully.
   
batch/jobs/execution The topic tree for all events related to a job execution.
   
batch/jobs/execution/restarting A topic tree node. A message is published when the batch executor is restarting an execution.
batch/jobs/execution/starting A topic tree node. A message is published when a job execution is starting.
   
batch/jobs/execution/completed A topic tree node. A message is published when a job execution ends successfully.
batch/jobs/execution/failed A topic tree node. A message is published when a job execution ends because of failure.
batch/jobs/execution/stopped A topic tree node. A message is published when a job execution is stopped.
batch/jobs/execution/jobLogPart A topic tree node. A message is published when a new job log part is created, a job stops, or a job ends.
   
batch/jobs/execution/step/started A topic tree node. A message is published when a step execution is started.
batch/jobs/execution/step/completed A topic tree node. A message is published when a step execution is completed successfully.
batch/jobs/execution/step/failed A topic tree node. A message is published when a step execution fails.
batch/jobs/execution/step/stopped A topic tree node. A message is published when a step execution is stopped.
batch/jobs/execution/step/checkpoint A topic tree node. A message is published when a checkpoint is taken.
   
batch/jobs/execution/partition/started A topic tree node. A message is published when a partition is started.
batch/jobs/execution/partition/completed A topic tree node. A message is published when a partition is completed successfully.
batch/jobs/execution/partition/failed A topic tree node. A message is published when a partition fails.
batch/jobs/execution/partition/stopped A topic tree node. A message is published when a partition is stopped.
   
batch/jobs/execution/split-flow/started A topic tree node. A message is published when a split-flow is started.
batch/jobs/execution/split-flow/ended A topic tree node. A message is published when a split-flow is completed.

The published message for each topic is a JMS TextMessage. The contents of this message is a JSON formatted string that represents the object of the topic, such as job instance, job execution, step execution, or partition. In addition, this message also includes the following JMS message properties set:

  • com_ibm_ws_batch_internal_jobInstanceId: The job instance ID, if available.
  • com_ibm_ws_batch_internal_jobExecutionId: The job execution ID, if available.
  • com_ibm_ws_batch_internal_stepExecutionId: The job step execution ID, if available.

The batch server must be configured to enable the publishing of job-related events. The batch dispatcher and batch executor have the same configuration. The following steps enable the publication of job-related events for a batch server.

  1. Enable JMS support by adding the appropriate JMS feature to the feature manager in server.xml. If we are using the WebSphere Application Server Liberty default messaging provider, add the wasJmsClient-2.0 feature and related JMS configurations for the message engine. If we are using WebSphere MQ Messaging provider, add the wmqJmsClient-2.0 feature. If we are using WebSphere MQ Messaging provider on z/OS in BINDINGS mode, add the zosTransaction-1.0 feature.

  2. Add the batchJmsEvents element to server.xml.

    Note: If we do not specify the connectionFactoryRef attribute, the default value for connectionFactoryRef is batchConnectionFactory. We must still configure the batchConnectionFactory JMS connection factory in server.xml.

    1. To update the batch event topic root, add the topicRoot attribute to the batchJmsEvents element. 

        <batchJmsEvents connectionFactoryRef="batchConnectionFactory" topicRoot="NEW_ROOT" />

      The topicRoot attribute value replaces the default root batch, so that batch/jobs/* becomes NEW_ROOT/jobs/*.

  3. Add the corresponding JMS connection factory to the server configuration. This is not specific to batch configuration. The following example illustrates the batch events configuration and its JMS configuration using the WebSphere MQ messaging provider. 

      <!-- wmq resource adapter -->
<variable name="wmqJmsClient.rar.location" value="${server.config.dir}/wmq.wlp.rar"/>
<!-- required for BINDINGS mode -->
<wmqJmsClient nativeLibraryPath="/mqm/jms/java/lib"/>

<batchJmsEvents connectionFactoryRef="batchConnectionFactory" />

<jmsConnectionFactory id="batchConnectionFactory">
  <properties.wmqJms
    transportType="BINDINGS"
    queueManager="WMQX" />
</jmsConnectionFactory>

    The following example illustrates the batch events configuration and its JMS configuration using the Liberty default messaging provider. 

      <batchJmsEvents connectionFactoryRef="batchConnectionFactory" />
<jmsConnectionFactory id="batchConnectionFactory">
  <properties.wasJms></properties.wasJms>
</jmsConnectionFactory>


Example

The following examples illustrate sequence of events for basic execution flows.

  • Submit and run a single-step job with checkpoints. 

      batch/jobs/instance/submitted
batch/jobs/instance/jms_queued
batch/jobs/instance/jms_consumed
batch/jobs/execution/starting
batch/jobs/instance/dispatched
batch/jobs/execution/started
batch/jobs/execution/step/started
batch/jobs/execution/step/checkpoint
batch/jobs/execution/step/checkpoint
...
batch/jobs/execution/step/checkpoint
batch/jobs/execution/step/completed
batch/jobs/execution/completed
batch/jobs/instance/completed

  • Submit and run a single-step job with partition. 

      batch/jobs/instance/submitted
batch/jobs/instance/jms_queued
batch/jobs/instance/jms_consumed
batch/jobs/execution/starting
batch/jobs/instance/dispatched
batch/jobs/execution/started
batch/jobs/execution/step/started
batch/jobs/execution/partition/started
batch/jobs/execution/partition/started
batch/jobs/execution/partition/started
batch/jobs/execution/partition/completed
batch/jobs/execution/partition/completed
batch/jobs/execution/partition/completed
batch/jobs/execution/step/completed
batch/jobs/execution/completed
batch/jobs/instance/completed