Scenario: Initial load

After a WebSphere Commerce instance is created and configured, we can load the initial catalog, price, and inventory data into the WebSphere Commerce database.

Configuration Recommendations

During your initial data loads, we can typically load a large amount of data if you have a large catalog. At this stage, the commerce server is not running and the data load can use all resources available to load data into the database. Therefore, it is important to configure the Data Load utility to make the initial load efficient. The following configuration considerations are recommended to help improve the initial load performance:

• Specify the data load mode as Insert in the wc-dataload.xml data load order configuration file. In this mode, the insert SQL statements are generated without checking whether any data exists in the database. Therefore, the performance is better than the Replace mode because less checking is done on the database and less memory is used for the ID resolver cache.

• Specify a large commit count and batch size.

• Specify a large ID resolver cache size and large JVM heap size. If you specify a large ID resolver cache, the Data Load utility does not need to resolve the ID from the database directly. But the ID resolver cache might use a significant amount of JVM heap memory. For 1 GB JVM heap size, set the ID resolver cache size to be less than 2 million to prevent a Java out of memory error.

Sample

The following code snippet is part of the sample wc-dataload.xml data load order configuration file. This snippet specifies the Insert data load mode, a large commit count, and batch size:

<_config:LoadOrder commitCount="1000" batchSize="1000" dataLoadMode="Insert" >
<_config:LoadItem name="CatalogEntry" businessObjectConfigFile="wc-loader-catalog-entry.xml"
<_config:DataSourceLocation location="CatalogEntry.csv" />
</_config:LoadItem>
</_config:LoadOrder> 

The following code snippet is part of a sample wc-dataload-env.xml data load environment configuration file that specifies the ID resolver cache size:

<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="2000000" />