Example: Splitting a text file with a regular expression delimiter and including the delimiter in the messages

Transfer a single text file to multiple messages by splitting the file at each match of a given Java regular expression and include the regular expression match in the resulting messages. To do this we use the -dqdt and -qi parameters of the fteCreateTransfer command.

About this task

Transfer a single text file to multiple messages on a queue. The file is split into variable-length sections, each of which is written to an individual message. The text file is split at each point where the text in the file matches a given regular expression. The source file is called /tmp/customers.text and has the following contents:

Customer name: John Smith
Customer contact details: john@example.net
Customer number: 314

Customer name: Jane Brown
Customer contact details: jane@example.com
Customer number: 42

Customer name: James Jones
Customer contact details: jjones@example.net
Customer number: 26

The regular expression that specifies where to split the file is Customer\snumber:\s\d+, which matches the text Customer number: followed by any number of digits. Regular expressions specified at the command line must be enclosed in double quotation marks to prevent the command shell evaluating the regular expression. The regular expression is evaluated as a Java regular expression. For more information, see Regular expressions used by MFT.

By default the number of characters that a regular expression can match is set to five. The regular expression used in this example matches strings that are longer than five characters. To enable matches that are longer than five characters edit the agent properties file to include the property maxDelimiterMatchLength.

By default, the text that matches the regular expression is not included in the messages. To include the text that matches the regular expression in the messages, as in this example, use the -qi parameter. The source file is located on the same system as the source agent AGENT_NEPTUNE, which connects to the queue manager QM_NEPTUNE. The destination queue, RECEIVING_QUEUE, is located on the queue manager QM_MERCURY. QM_MERCURY is also the queue manager used by the destination agent AGENT_MERCURY. The transfer splits the source file into sections and writes each of these sections to a message on RECEIVING_QUEUE.

Procedure

1. Stop the destination agent using the following command:

   fteStopAgent AGENT_MERCURY

2. Add the following line to the agent properties file for AGENT_MERCURY:

   maxDelimiterMatchLength=25

   Note: Increasing the value of maxDelimiterMatchLength can decrease performance.
3. Start the destination agent using the following command:

   fteStartAgent AGENT_MERCURY

4. Type the following command:

   fteCreateTransfer -sa AGENT_NEPTUNE -sm QM_NEPTUNE -da AGENT_MERCURY -dm QM_MERCURY 
    -dq RECEIVING_QUEUE
        text -dqdt "Customer\snumber:\s\d+" -qi -dqdp postfix /tmp/customers.text

   The source agent, AGENT_NEPTUNE, reads the data from the file /tmp/customers.text and transfers this data to the destination agent, AGENT_MERCURY. The destination agent, AGENT_MERCURY, writes the data to three persistent messages on the queue RECEIVING_QUEUE. These messages all have the same IBM MQ group ID and the last message in the group has the IBM MQ LAST_MSG_IN_GROUP flag set. The data in the messages is as follows.
   • First message:

     Customer name: John Smith
     Customer contact details: john@example.net
     Customer number: 314
     

   • Second message:

     Customer name: Jane Brown
     Customer contact details: jane@example.com
     Customer number: 42
     

   • Third message:

     Customer name: James Jones
     Customer contact details: jjones@example.net
     Customer number: 26
     

Parent topic: Transfer data from files to messages

Related concepts

• Transfer data from files to messages

Related tasks

• Configure an agent to perform file-to-message transfers
• Example: Transferring a single file to a single message
• Example: Splitting a single file into multiple messages by length
• Example: Splitting a text file into multiple messages using a regular expression delimiter
• Example: Setting IBM MQ message properties on a file-to-message transfer
• Example: Setting user-defined properties on a file-to-message transfer
• Starting a new file transfer

Related information

• The MFT agent.properties file
• Regular expressions used by MFT