Use the batch data stream framework

This topic shows you an example of how to use the batch data stream (BDS) framework.

Identify the correct pattern to use. Select a pattern based on what type of data stream we need. For example, to read text from a file, then select the FileReaderPattern. See Batch data stream framework and patterns for a selection of patterns.

1. Implement the pattern interface:

   <codeblock>package com.ibm.websphere.samples;
   
   import java.io.BufferedReader;
   import java.io.IOException;
   import java.util.Properties;
   
   import com.ibm.websphere.batch.devframework.configuration.BDSFWLogger;
   import com.ibm.websphere.batch.devframework.datastreams.patternadapter.FileReaderPattern;
   
   // Implement the FileReaderPattern
   public class TransactionListStream implements FileReaderPattern {
    private Properties properties;
    private BDSFWLogger logger;
   
   /**
      Save properties specified in the xJCL
   */
   
    public void initialize(Properties props) {
   // create logger
     logger = new BDSFWLogger(props);
     
     if (logger.isDebugEnabled())
      logger.debug("entering TransactionListInputStream.initialize()");
     properties = props;
   
    }
   
   
   // This method is where you should add the business logic of processing the read //string
    public Object fetchRecord(BufferedReader reader) throws IOException {
     String str = null;
     Posting posting = null;
     if (logger.isDebugEnabled())
   logger.debug("Entering TransactionListInputStream.fetchRecord");
     if(reader.ready()) {
      str = reader.readLine();
     }
     if(str != null) {
      
      posting = _generateRecord(str);
      
     }
       
     if (logger.isDebugEnabled())
      logger.debug("Exiting TransactionListInputStream.fetchRecord with " + posting);
     return posting;
     
    }
    // Helper method that parses the read string and creates an internal object for use  // by other parts of the code  private Posting _generateRecord(String str) {
     Posting post = null;
     String [] tokens = str.split(",", 3);
    
     if(tokens.length == 3) {
      
      String txTypeStr = tokens[0];
      String actNoStr = tokens[1];
      String amtStr = tokens[2];
      
      int txType = Integer.parseInt(txTypeStr);
      double amt = Double.parseDouble(amtStr);
      post = new Posting(txType,actNoStr,amt);
      
      
     } else {
      logger.error("Invalid csv string" + str);
     }
     if(logger.isDebugEnabled())
      logger.debug("Loaded posting record " + post);
     return post;
    }
    public void processHeader(BufferedReader reader) throws IOException {
     // NO OP for this sample
   
    }
   
   }
   </codeblock>

2. Reference the class created in the previous step, along with the supporting class in the xJCL.

xJCL example

<codeblock><batch-data-streams> 
  <bds>
          <logical-name>txlististream</logical-name>
          <props>
<prop name="IMPLCLASS" value= "com.ibm.websphere.samples.TransactionListStream"/>
             <prop name="FILENAME" value="/opt/inputfile.txt"/>
             <prop name="debug" value="true"/> 
          </props>
<impl-class> com.ibm.websphere.batch.devframework.datastreams.patterns.TextFileReader </impl-class>
 </bds>
</batch-data-streams>

</codeblock>

What to do next

Install the application.

Subtopics

• Batch data stream framework and patterns
  The batch environment provides a batch data stream (BDS) framework that includes pre-built code to work with streams such as text, byte, database, and data sets. We can implement an interface where the business logic for processing the stream is added. The pre-built code manages actions such as opening, closing, and externalizing and internalizing checkpoints.

• Implement the generic batch step (GenericXDBatchStep)
  A generic batch step works with one input and one output stream. This step during each iteration of the batch loop reads a single entry from the BDS Input Stream passes it to the BatchRecordProcessor for processing. The BatchRecordProcessor returns the processed data which is then passed to the BDS output stream.

• Implement the error tolerant step (ThresholdBatchStep)
  An error tolerant generic batch step works with one input, one output stream, and one error stream. This step during each iteration of the batch loop reads a single entry from the batch data stream (BDS) input stream and passes it to the BatchRecordProcessor property for processing.

• Declaring the percentage-based threshold policy (PercentageBasedThresholdPolicy)
  The PercentageBasedThresholdPolicy policy provides a batch implementation of the ThresholdPolicy interface.

• Declaring the record based threshold policy (RecordBasedThresholdPolicy)
  The RecordBasedThresholdPolicy policy provides a batch implementation of the ThresholdPolicy interface.

Related tasks

Install the batch application

JDBCReaderPattern

JDBCWriterPattern

ByteReaderPattern

ByteWriterPattern

FileReaderPattern

FileWriterPattern

JPAReaderPattern

JPAWriterPattern

PureQueryWriterPattern

PureQueryReaderPattern

(zos) RecordOrientedDatasetReaderPattern

(zos) RecordOrientedDataSetWriterPattern