Batch data stream framework and patterns

The batch environment provides a batch data stream (BDS) framework that includes pre-built code to work with streams such as text, byte, database, and data sets. We can implement an interface where the business logic for processing the stream is added. The pre-built code manages actions such as opening, closing, and externalizing and internalizing checkpoints.

BDS framework patterns

A BDS framework pattern is a simple Java TM interface for a particular type of data stream that a user implements to insert business logic. The BDS framework has several supporting classes for each pattern that do most of the mundane tasks related to stream management. The following table shows the patterns that the batch environment provides:

Pattern name	Description	Supporting classes
JDBCReaderPattern	Used to retrieve data from a database using a JDBC connection.	LocalJDBCReader JDBCReader CursorHoldableJDBCReader
JDBCWriterPattern	Used to write data to a database using a JDBC connection.	LocalJDBCWriter JDBCWriter
JDBCCallableStatementReaderPattern	Used to implement the business logic of processing data from a JDBC connection using a stored procedure	JDBCCallableStatementReader LocalJDBCCallableStatementReader
JDBCCallableStatementWriterPattern	Used to implement the business logic of outputting data to a JDBC connection using a stored procedure	JDBCCallableStatementWriter LocalJDBCCallableStatementWriter
ByteReaderPattern	Used to read byte data from a file.	FileByteReader
ByteWriterPattern	Used to write byte data to a file.	FileByteWriter
FileReaderPattern	Used to read a text file.	TextFileReader
FileWriterPattern	Used to write to a text file.	TextFileWriter
(ZOS) RecordOrientedDatasetReaderPattern	(ZOS) Used to read a z/OS data set.	(ZOS) ZFileStreamOrientedTextReader ZFileStreamOrientedByteReader ZFileRecordOrientedDataReader
(ZOS) RecordOrientedDataSetWriterPattern	(ZOS) Used to write to a z/OS data set.	(ZOS) ZFileStreamOrientedTextWriter ZFileStreamOrientedByteWriter ZFileRecordOrientedDataReader
JPAReaderPattern	Used to retrieve data from a database using the JPA provider configured.	JPAReader
JPAWriterPattern	Used to write data to a database using a Java Persistence API (JPA) connection.	JPAWriter

BDS framework steps

BDS framework steps minimize the amount of work to create a batch step by performing the bookkeeping tasks related to step management and delegating the business logic to a class implemented by the user.

Step	Description
Implement the generic batch step (GenericXDBatchStep)	A simple step that uses one input and one output stream.
Implement the error tolerant step (ThresholdBatchStep)	A simple step that uses one input, one output stream, and one error stream.

ThresholdPolicies

Step	Description
Declaring the record based threshold policy (RecordBasedThresholdPolicy)	This policy provides a batch implementation of the ThresholdPolicy interface.
Declaring the percentage-based threshold policy (PercentageBasedThresholdPolicy)	This policy provides a batch implementation of the ThresholdPolicy interface

Subtopics

• JDBCReaderPattern
  This pattern is used to retrieve data from a database using a Java Database Connectivity (JDBC) connection.
• JDBCWriterPattern
  The JDBCWriterPattern pattern is used to write data to a database using a JDBC connection.
• JDBCCallableStatementReaderPattern
  This pattern is used to implement the business logic of processing data from a JDBC connection using a stored procedure. The administrative tasks of opening and closing the connection are done by the batch framework.
• JDBCCallableStatementWriterPattern
  This pattern is used to implement the business logic of outputting data to a JDBC connection using a stored procedure. The administrative tasks of opening and closing the connection are done by the batch framework.
• ByteReaderPattern
  The ByteReaderPattern pattern is used to read byte data from a file.
• ByteWriterPattern
  The ByteWriterPattern pattern is used to write byte data to a file.
• FileReaderPattern
  The FileReaderPattern pattern is used to read text data from a file.
• FileWriterPattern
  The FileWriterPattern pattern is used to write text data to a file.
• RecordOrientedDatasetReaderPattern
  The RecordOrientedDatasetReaderPattern pattern is used to read data from a z/OS data set.
• RecordOrientedDataSetWriterPattern
  The RecordOrientedDataSetWriterPattern pattern is used to write data to a z/OS data set.
• JPAReaderPattern
  The JPAReaderPattern pattern is used to retrieve data from a database using OpenJPA.
• JPAWriterPattern
  The JPAWriterPattern pattern is used to write data to a database using a Java Persistence API (JPA) connection.
• PureQueryWriterPattern
  Use this pattern to write data to a database using IBM Optim™ pureQuery Runtime. The batch data stream (BDS) framework completes the administrative tasks of opening and closing connections.
• PureQueryReaderPattern
  Use this pattern is used to read data from a database using IBM Optim pureQuery Runtime. The batch data stream (BDS) framework completes the administrative tasks of opening and closing connections.

Developing batch applications

Use the batch data stream framework

Implement the generic batch step (GenericXDBatchStep)