Batch data streams

Batch data streams (BDS) are Java objects that provide an abstraction for the data stream processed by a batch step. A batch step can have one or more BDS objects associated with it. The grid endpoints make the BDS associated with the batch step available at run time. The grid endpoints also manages the life cycle of a BDS by invoking batch-specific callbacks.

A BDS object implements the com.ibm.websphere.batch.BatchDataStream interface. This interface is server agnostic. The implementing object can retrieve data from any type of data source, for example, files and databases. Call back methods on the BatchDataStream interface allow the grid endpoints to manage the BDS at run time. One of the key features of a BDS is its capability to convey its current position in the stream to the grid endpoints, and the capability to move itself to a given location in the data stream. This feature allows the grid endpoints to record (in the grid endpoints database) how much data a batch step has processed. This information is recorded on every checkpoint. Therefore, the grid endpoints can restart a batch job from a recorded position in the data stream if the job is canceled or fails in a recoverable manner.

The following lists the main methods that exist for the BatchDataStream interface. See the API for the BatchDataStream interface for additional information.

void open()

Called by grid endpoints to open the BDS

void close()

Called by grid endpoints to close the BDS

void initialize(String ilogicalname, String ijobstepid)

Called by grid endpoints to initialize the BDS and let it know its logical name and batch step ID

String externalizeCheckpointInformation()

Called by grid endpoints prior to a checkpoint to record the current cursor of the BDS

void internalizeCheckpointInformation(String chkpointInfo()

Called by grid endpoints to inform the BDS of the previously recorded cursor, chkpointInfo. Typically, the positionAtCurrentCheckpoint is called after this call to move the BDS to this cursor.

void positionAtCurrentCheckpoint()

Called by grid endpoints after calling internalizeCheckpointInformation to move the BDS to the cursor indicated by the chkpointInfo passed in through the internalizeCheckpointInformation call

The BatchDataStream interface does not have methods for retrieving or writing data. There are no getNextRecord and putNextRecord methods defined on the interface that a batch step calls to read or write to the BDS object. Methods for passing data between the batch step and the BDS object are the responsibility of the implementing BDS object. Review the batch Samples that this product supports to see examples of how to implement batch data streams.

Transaction environment

All methods of a BDS object are called in a global transaction. There is no guarantee that any consecutive method calls made to a BDS object happen in the same transaction because the transaction is owned by the grid endpoints, not the BDS object.

Related concepts

Batch programming model

APIs