Administer > Transforming, loading, and extracting data > Overview of the data load utility > Data load sample scenario

Initial load scenario

Overview

The initial load scenario is when you have finished creating and configuring a new WCS instance. You now want to load the initial catalog, price, and/or inventory data into the WebSphere Commerce database.

Recommendations

To improve the initial load performance:

• Specify the data load mode as Insert.

  In this mode, the insert SQL statements are generated without checking whether the data exists in the database. Therefore, the performance is better than the Replace mode because less checking is done in the database and less memory is used for the ID resolver cache.

• Specify a large commit count and batch size.

• Specify a key range with a start key and an end key.

  If you specify a fixed key range, the data load utility does not need to call the key manager to get the next available key. But ensure that the range is large enough for all the keys required for your load.

• Specify a large ID resolver cache size and large JVM heap size.

  If you specify a large ID resolver cache, the data load utility does not need to resolve the ID from the database directly. But the ID resolver cache might use lots of JVM heap memory. For 1 GB JVM heap size, you set the ID resolver cache size less than 2 million to prevent the Java out of memory error.

Sample

The following code snippet is part of the sample wc-dataload.xml file to specify...

• Insert data load mode
• Large commit count and batch size
• Fixed key range

<_config:LoadOrder commitCount="1000" 
                   batchSize="1000" 
                   dataLoadMode="Insert" >         

    <_config:LoadItem name="CatalogEntry" 
                      businessObjectConfigFile="wc-loader-catalog-entry.xml"  
                      startKey="1000001" 
                      endKey="2000000">              

        <_config:DataSourceLocation location="CatalogEntry.csv" />         

    </_config:LoadItem>

</_config:LoadOrder> 

The following code snippet is part of a sample wc-dataload-env.xml file to specify the ID resolver cache size:

<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" 
                    cacheSize="2000000" />

Related tasks

Substitute attribute values with variables in data load configuration files

Related reference

Delta load scenario

Data load best practices

Related information
Data load business object configuration file

+

Search Tips   |   Advanced Search