Reference > Data load utility configuration files

Schema


Schema wc-dataload.xsd
Target Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Namespace Table http://www.ibm.com/xmlns/prod/commerce/foundation/config
http://www.w3.org/2001/XMLSchema


Directives
include wc-dataload-common.xsd

Elements
DataLoadConfiguration

Complex Types
DataLoadConfiguration
DataLoadEnvironment
DataSourceLocation
LoadItem
LoadOrder

Simple Types
DataLoadMode

Types


Complex Type DataLoadConfiguration
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram DataLoadConfiguration DataLoadEnvironment DataLoadEnvironment LoadOrder LoadOrder
Inheritance Hierarchy

 anyType {http://www.w3.org/2001/XMLSchema}
    DataLoadConfiguration {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Content Details

DataLoadEnvironment

A reference to the environment configuration file location.

An XML file that validates successfully against the XSD file wc-dataload-env.xsd.

LoadOrder

Define the data to load and the load order.

The load order is composed of one or more load items. The load items are loaded in the order that they are defined.

Attribute Type Required Default Setting Description
dataLoadMode String No Replace

The data load mode.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

Data load utility modes:

  • Insert: All data is inserted into the database. The utility generates insert SQL statements. Use this mode for initial data load.
  • Delete: All data is deleted from the database. The utility generates the delete SQL statements.
  • Replace (Default): All of the data in the database is replaced. The utility generates the insert, update or delete SQL statements depending on the data. Replace mode replace the existing data with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value, if any.

    For example:

    • If one record (line) in the CSV file represents a new object, it is inserted.
    • If the object is in the database already, it is replaced.
    • If there is a flag in the data to indicate this object should be deleted, it is deleted.

commitCount nonNegativeInteger No 1

Specifies how many lines to process before calling database commit. The default setting is 1. If the commitCount is 0, it means it will not commit until this load item finishes processing all of its input data.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

batchSize nonNegativeInteger No 1

The batch size specifies how many lines of records to process when using JDBC batch. The default value is 1, JDBC batch update is not used. If the batchSize is 0, batches all of processed input data for the entire load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

maxError nonNegativeInteger No 1

This attribute is a setting for the error tolerance during the data load process for a load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

The utility continues to load data for a load item until the error tolerance level is reached; then the utility exits.

Note that for configuration and system errors the utility ends before reaching the tolerance level.

retry boolean No false

A flag indicating that if an error occurs during the data load process, the utility retries to load the data. If configuration errors or system errors occur, the utility ends.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

Documentation

   

This configuration file controls the data load.

This file defines:

  • Which environment the data is loaded into.
  • What data and business objects to load.
  • The data load order.
  • The location of the data to load.
  • Default data load options.
  • An area to define customizable properties.

Source

 
<xsd:complexType name="DataLoadConfiguration"> 
  
<xsd:sequence>  
             
<xsd:element name="DataLoadEnvironment" type="_config:DataLoadEnvironment" minOccurs="1" maxOccurs="1">   
              
</xsd:element>  
             
<xsd:element name="LoadOrder" type="_config:LoadOrder" minOccurs="1" maxOccurs="1">   
              
</xsd:element>  
  
</xsd:sequence> 

</xsd:complexType> 


Complex Type DataLoadEnvironment
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram DataLoadEnvironment
Inheritance Hierarchy

 anyType {http://www.w3.org/2001/XMLSchema}
    DataLoadEnvironment {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Content Details

Documentation

   

The location of the environment configuration file.

An XML file that validates successfully against the XSD file wc-dataload-env.xsd.

Attribute Type Required Default Setting Description
configFile String Yes N/A

The location of the environment configuration file.

An XML file that validates successfully against the XSD file wc-dataload-env.xsd.

An absolute path, or relative path, to the file. The relative path is relative to the location of this load order configuration file.

Source

 
<xsd:complexType name="DataLoadEnvironment"> 
        
<xsd:attribute name="configFile" use="optional" form="unqualified" type="xsd:string">  
            
</xsd:attribute> 
   
</xsd:complexType> 


Complex Type DataSourceLocation
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram DataSourceLocation NameValuePair NameValuePair
Inheritance Hierarchy

 anyType {http://www.w3.org/2001/XMLSchema}
    DataSourceLocation {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Content Details

property

A section for defining custom properties in the form of name-value pairs that define custom data source information.

See the following classes for more details: com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader#getDataSourceLocation() com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource#getDataSourceLocationProperties()

Documentation

   

This node defines where the data is located.

An area to define customizable properties is provided. The properties are available for use with the configured data reader through the configuration properties.

See the following classes for more details: com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader#getDataSourceLocation() com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource#getDataSourceLocationProperties()

Attribute Type Required Default Setting Description
location String No N/A

The location of the data required by the data reader.

An absolute path, or relative path, to the file. The location relative path should be relative to the dataInputFilePath in the FilePath defined in the dataload environment configuration file. If the dataInputFilePath is not defined in the data load environment configuration file, the location relative path should be relative to the load order configuration file.

Source

 
<xsd:complexType name="DataSourceLocation"> 
        
<xsd:sequence>  
       
<xsd:element name="property" type="_config:NameValuePair" minOccurs="0" maxOccurs="unbounded">   
        
</xsd:element>  
     
</xsd:sequence> 
     
<xsd:attribute name="location" type="xsd:string" use="optional" form="unqualified">  
      
</xsd:attribute> 
   
</xsd:complexType> 


Complex Type LoadItem
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram LoadItem DataLoadMode DataLoadMode NameValuePair NameValuePair DataSourceLocation DataSourceLocation BusinessContext BusinessContext
Inheritance Hierarchy

 anyType {http://www.w3.org/2001/XMLSchema}
    LoadItem {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Content Details

property

A section for defining custom properties in the form of name-value pairs that define load item information.

DataSourceLocation

This node defines where the data is located.

An area to define customizable properties is provided. The properties are available for use with the configured data reader through the configuration properties.

See the following classes for more details: com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader com.ibm.commerce.foundation.dataload.datareader.AbstractDataReader#getDataSourceLocation() com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource com.ibm.commerce.foundation.dataload.datareader.DataReaderDataSource#getDataSourceLocationProperties()

BusinessContext

This node defines business context information specific to a load item. Note that global business context information can be specified in the environment configuration file and can be overridden at the load item.

Attribute Type Required Default Setting Description
storeIdentifier String No N/A

The store external identifier of the store. The utility resolves the store internal identifier based on the external identifier.

catalogIdentifier String No N/A

The catalog external identifier of the catalog. The utility resolves the catalog internal identifier based on the external identifier to and generate the data.

catalogIdentifier String No N/A

The catalog external identifier of the catalog. The utility resolves the catalog internal identifier based on the external identifier to and generate the data.

langId int No N/A

The language ID of the data. This attribute is the same as languageId.

languageId int No N/A

The language ID that the data should be loaded with. This attribute is the same as langId.

currency String No N/A

The currency of the data.

storeId int No N/A

The store internal identifier of the store. Do not specify if storeIdentifier is provided.

catalogId long No N/A

The catalog internal identifier of the store. Do not specify if catalogIdentifier is provided.

Documentation

   

This node defines a set of data to load. This set is referred to as a load item. This is usually a subset of the entire set of data being loaded.

A load item defines the following:

  • The name of the load item.
  • Options for how the data should be loaded.
  • The location of the data being loaded.
  • The location of the business object configuration the data should be used with.
  • Business context information that is specific to the set of data being loaded.
  • An area to define customizable properties that can be used to load the data.

Attribute Type Required Default Setting Description
name String Yes N/A

The name of the load item. This is a required attribute that captures load summary and error information.

businessObjectConfigFile String Yes N/A

The location of the business object configuration file.

An XML file that validates successfully against the XSD file wc-dataload-businessobject.xsd.

An absolute path, or relative path, to the file. The value for businessObjectConfigFile relative path should be relative to the boConfigFilePath in the FilePath defined in the dataload environment configuration file. If the boConfigFilePath is not defined in the dataload environment configuration file, businessObjectConfigFile relative path should be relative to the load order configuration file.

dataLoadMode String No Replace

This attribute the mode the data should be loaded in.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

Data load utility modes:

  • Insert: All data is inserted into the database. The utility generates insert SQL statements. Use this mode for initial data load.
  • Delete: All data is deleted from the database. The utility generates the delete SQL statements.
  • Replace (Default): All of the data in the database is replaced. Depending on the data, the utility generates insert, update or delete SQL statements. Replace mode replace the existing data with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value, if any.

    For example:

    • If one record (line) in the CSV file represents a new object, it is inserted.
    • If the object is in the database already, it is replaced.
    • If there is a flag in the data to indicate this object should be deleted, it is deleted.

commitCount nonNegativeInteger No 1

Specifies how many lines to process before calling database commit. The default setting is 1. If the commitCount is 0, it means it will not commit until this load item finishes processing all of its input data.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

batchSize nonNegativeInteger No 1

The batch size specifies how many lines of records to process when using JDBC batch. The default value is 1, JDBC batch update is not used. If the batchSize is 0, batches all of processed input data for the entire load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

maxError nonNegativeInteger No 1

This attribute is a setting for the error tolerance during the data load process for a load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

The utility continues to load data for a load item until the error tolerance level is reached; then the utility exits.

For configuration and system errors, the utility ends before it reaches the tolerance level.

retry boolean No false

A flag indicating that if an error occurs during the data load process, the utility retries to load the data. If configuration errors or system errors occur the utility ends.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

startKey long No N/A

This attribute combined with the attribute endKey defines which primary key range to use when the object does not exist in the database and it requires a new generated key.

endKey long No N/A

This attribute combined with the attribute startKey defines which primary key range to use when the object does not exist in the database and it requires a new generated key.

Source

 
<xsd:complexType name="LoadItem"> 
        
<xsd:sequence>  
    
<xsd:element name="property" type="_config:NameValuePair" minOccurs="0" maxOccurs="unbounded">   
                 
</xsd:element>  
             
<xsd:element name="DataSourceLocation" type="_config:DataSourceLocation" minOccurs="0" maxOccurs="1">   
              
</xsd:element>  
             
<xsd:element name="BusinessContext" type="_config:BusinessContext" minOccurs="0" maxOccurs="1">   
                 
</xsd:element>  
  
</xsd:sequence> 
  
<xsd:attribute name="name" type="xsd:string" use="required" form="unqualified">  
   
</xsd:attribute>         
  
<xsd:attribute name="businessObjectConfigFile" type="xsd:string" use="required" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="dataLoadMode" type="_config:DataLoadMode" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="commitCount" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="batchSize" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="maxError" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="retry" type="xsd:boolean" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="startKey" type="xsd:long" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="endKey" type="xsd:long" use="optional" form="unqualified">  
   
</xsd:attribute> 
   
</xsd:complexType> 


Complex Type LoadOrder
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram LoadOrder DataLoadMode DataLoadMode NameValuePair NameValuePair LoadItem LoadItem
Inheritance Hierarchy

 anyType {http://www.w3.org/2001/XMLSchema}
    LoadOrder {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Content Details

property

A section for defining custom properties in the form of name-value pairs that define load order information.

LoadItem

This node defines a set of data to load. This set is referred to as a load item. This is usually a subset of the entire set of data being loaded.

A load item defines the following:

  • The name of the load item.
  • Options for how the data should be loaded.
  • The location of the data being loaded.
  • The location of the business object configuration the data should be used with.
  • Business context information that is specific to the set of data being loaded.
  • An area to define customizable properties that can be used to load the data.

Documentation

   

This node defines what data is being loaded, and the order it should be loaded in.

The load order is composed of one or more load items. The load items are loaded in the order that they are defined in.

Attribute Type Required Default Setting Description
dataLoadMode String No Replace

This attribute the mode the data should be loaded in.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

Data load utility modes:

  • Insert: All data is inserted into the database. The utility generates insert SQL statements. Use this mode for initial data load.
  • Delete: All data is deleted from the database. The utility generates the delete SQL statements.
  • Replace (Default): All of the data in the database is replaced. Depending on the data, the utility generates insert, update or delete SQL statements. Replace mode replace the existing data with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value, if any.

    For example:

    • If one record (line) in the CSV file represents a new object, it is inserted.
    • If the object is in the database already, it is replaced.
    • If there is a flag in the data to indicate this object should be deleted, it is deleted.

commitCount nonNegativeInteger No 1

Specifies how many lines to process before calling database commit. The default setting is 1. If the commitCount is 0, it means it will not commit until this load item finishes processing all of its input data.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

batchSize nonNegativeInteger No 1

The batch size specifies how many lines of records to process when using JDBC batch. The default value is 1, JDBC batch update is not used. If the batchSize is 0, batches all of processed input data for the entire load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

maxError nonNegativeInteger No 1

This attribute is a setting for the error tolerance during the data load process for a load item.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

The utility continues to load data for a load item until the error tolerance level is reached; then the utility exits.

For configuration and system errors, the utility ends before it reaches the tolerance level.

retry boolean No false

A flag indicating that if an error occurs during the data load process, the utility retries to load the data. If configuration errors or system errors occur the utility ends.

If defined at the load order level, all load items inherit this value. The value can be overridden at the load item level.

Source

 
<xsd:complexType name="LoadOrder">         
        
<xsd:sequence>  
         
<xsd:element name="property" type="_config:NameValuePair" minOccurs="0" maxOccurs="unbounded">   
                 
</xsd:element>  
             
<xsd:element name="LoadItem" type="_config:LoadItem" minOccurs="1" maxOccurs="unbounded">   
              
</xsd:element>  
  
</xsd:sequence> 
  
<xsd:attribute name="dataLoadMode" use="optional" type="_config:DataLoadMode" form="unqualified">  
            
</xsd:attribute> 
  
<xsd:attribute name="commitCount" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="batchSize" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="maxError" type="xsd:nonNegativeInteger" use="optional" form="unqualified">  
   
</xsd:attribute> 
  
<xsd:attribute name="retry" use="optional" form="unqualified" type="xsd:boolean">  
            
</xsd:attribute> 
   
</xsd:complexType> 


Simple Type DataLoadMode
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram DataLoadMode
Inheritance Hierarchy

 string {http://www.w3.org/2001/XMLSchema}
    DataLoadMode {http://www.ibm.com/xmlns/prod/commerce/foundation/config}

Allowable Values

Insert All data is inserted into the database. The utility generates insert SQL statements. Use this mode for initial data load.
Replace All of the data in the database is replaced. Depending on the data, the utility generates insert, update or delete SQL statements. Replace mode replace the existing data with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value, if any.

For example:

  • If one record (line) in the CSV file represents a new object, it is inserted.
  • If the object is in the database already, it is replaced.
  • If there is a flag in the data to indicate this object should be deleted, it is deleted.

Delete All data is deleted from the database. The utility generates the delete SQL statements.

Documentation

   

This node defines the mode the data should be loaded in.

Data load utility modes:

  • Insert: All data is inserted into the database. The utility generates insert SQL statements. Use this mode for initial data load.
  • Delete: All data is deleted from the database. The utility generates the delete SQL statements.
  • Replace (Default): All of the data in the database is replaced. Depending on the data, the utility generates insert, update or delete SQL statements. Replace mode replace the existing data with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value, if any.

    For example:

    • If one record (line) in the CSV file represents a new object, it is inserted.
    • If the object is in the database already, it is replaced.
    • If there is a flag in the data to indicate this object should be deleted, it is deleted.

Source

 
<xsd:simpleType name="DataLoadMode"> 
     
<xsd:restriction base="xsd:string">  
       
<xsd:enumeration value="Insert">   
        
</xsd:enumeration>  
       
<xsd:enumeration value="Replace">   
        
</xsd:enumeration>  
       
<xsd:enumeration value="Delete">   
        
</xsd:enumeration>  
     
</xsd:restriction> 
   
</xsd:simpleType> 


Elements


Element DataLoadConfiguration
Namespace http://www.ibm.com/xmlns/prod/commerce/foundation/config
Diagram DataLoadConfiguration DataLoadEnvironment DataLoadEnvironment LoadOrder LoadOrder DataLoadConfiguration
Type DataLoadConfiguration {http://www.ibm.com/xmlns/prod/commerce/foundation/config}
Documentation

   

Data load configuration consists of the following files:

  1. The load configuration. (wc-dataload.xsd)
  2. The environment configuration. (wc-dataload-env.xsd)
  3. The business object configuration. (wc-dataload-businessobject.xsd)

Data load configuration also consists of the following non-customizable files:

  1. The default set of common definitions. (wc-dataload-common.xsd)
  2. The default data reader types. (wc-dataload-datasource-data.xsd)

Controls the data load.

This file defines:

  • Which environment the data is loaded into.
  • What data and business objects to load.
  • The data load order.
  • The location of the data to load.
  • Default data load options.
  • An area to define customizable properties.

Source

 
<xsd:element name="DataLoadConfiguration" type="_config:DataLoadConfiguration"> 
 
</xsd:element> 


Groups


Attributes


Attribute Groups


_vcc_freeselect_bottom_