Administer > Transforming, loading, and extracting data > Overview of the data load utility > Data load sample scenario

Delta load scenario

Overview

The delta load scenario is, while WCS is up and running, to insert, update, or delete catalog, inventory, and/or price data.

Recommendations

During a delta load, the data can be loaded into a staging server or a production server. However, some data such as the inventory data have to be loaded into production server directly.

Recommendations for the delta load...

• Specify the data load mode as Replace unless you are loading new data into the database, in which case you can use the Insert mode to improve the performance.

• Specify a commit count of 1 and batch size of 1 to minimize the impact to the production site.

  If you specify a large commit count and batch size, more rows are locked in the database tables, and the lock time is longer. When the rows are locked, the information might not be available online.

• Do not specify a key range with a start key and an end key unless you are sure it would not cause key conflicts.

• Specify the ID resolver cache size to 0 if your database is large.

  Loading the ID resolver cache for a large table might take a long time and also uses lots of memory. The ID resolver cache might not be loaded for a large table because of the JVM heap size limitation. So for a large database, the ID resolver cache size is set to 0. Set the ID resolver cache size to large number only to load data to some small tables.

Sample

The following code snippet is part of a sample wc-dataload.xml file to specify the Replace data load mode, a small commit count and batch size, and without a fixed key range:

<_config:LoadOrder commitCount="1" 
                   batchSize="1" 
                   dataLoadMode="Replace" >         

    <_config:LoadItem name="CatalogEntry" 
                      businessObjectConfigFile="wc-loader-catalog-entry.xml"  >              

        <_config:DataSourceLocation location="CatalogEntry.csv" />         

    </_config:LoadItem>

</_config:LoadOrder> 

The following code snippet is part of a sample wc-dataload-env.xml file to specify the ID resolver cache size to 0:

<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" 
                    cacheSize="0" />

Related tasks

Substitute attribute values with variables in data load configuration files

Related reference

Initial load scenario
Data load best practices

Related information
Data load business object configuration file

+

Search Tips   |   Advanced Search