Data Load utility

This utility loads data from a source file into a target database. Loading the file populates and updates the WebSphere Commerce database. We can also use the Data Load utility to delete data from a database.

(Developer) We can also run the data load utility in the development environment.

Command syntax

Syntax diagram for Data Load utility.

Parameter values

full_path

Full path location of the load order configuration file.

path_to_environment_file

Full path location to an environment configuration file to process. If an environment configuration file is specified in the command line, it takes precedence over the element that exists in the data load environment configuration file.

-DXmlValidation

(Boolean) Used to turn on or off XML validation. For accuracy, the defined configurations inside the XML files are validated against the Data Load utility XSDs. By default, XML validation is turned on (true). To turn off validation, enter -DXmlValidation=false.

-DLoadOrder

Specifies the order that data is loaded to the database. This parameter overrides the LoadOrder element in the wc-dataload.xml configuration file. We can specify any number of load items we want, with each item separated by a comma. Ensure that parameter values do not contain white space. If there is white space, enclose the parameters in double quotation marks. For example,

"-DLoadOrder=loadItemName1, loadItemName2"

-DpromptWarning

Specifies whether to proceed when a warning is encountered, or to prompt the user for input. By default, this parameter is set to true. If the data load is not running in an interactive mode, set the value to false.

-DConsoleHandler.level

Used to override the console logging level that is defined in the utilities_root\dataload\logging.properties file. For example, enter -DConsoleHandler.level=SEVERE to change the console logging level to SEVERE.

-DFileHandler.level

Used to override the file logging level that is defined in the utilities_root\dataload\logging.properties file. For example, enter -DFileHandler.level=WARNING to change the file logging level to WARNING.

-Dpackage_name.level

Where package_name is the name of any package. This parameter is used to override the log level for the specified package. If the package_name is blank (-D.level=level name) then the log level is changed for all packages. The level name can be: OFF, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, ALL.For example:

-Dcom.ibm.commerce.catalog.dataload.level=FINER

To help troubleshoot errors that can occur when you run the Data Load utility, override the logging levels for the following Data Load utility packages that have logging available:

-Dcom.ibm.commerce.foundation.dataload.businessobjectbuilder.level
-Dcom.ibm.commerce.foundation.dataload.businessobjectmediator.level
-Dcom.ibm.commerce.foundation.dataload.config.level
-Dcom.ibm.commerce.foundation.dataload.datareader.level
-Dcom.ibm.commerce.foundation.dataload.datawriter.level
-Dcom.ibm.commerce.foundation.dataload.idresolve.level
-Dcom.ibm.commerce.foundation.dataload.database.level
-Dcom.ibm.commerce.foundation.dataload.level

For more information about changing the logging level for a package, see Configure logging for runtime environments. For more information about logging, see Logging services.

-Dinstance

The name of the WebSphere Commerce instance with which we are working (for example, demo). Depending on the business object mediator and data that we are using, this parameter is mandatory. For example, when loading custom userData for business objects and the Data Service Layer (DSL) configuration mapping for the objects is defined in an extension directory.

By default, the Data Load utility does not search extension directories for DSL configuration mappings. For the utility to locate the DSL configuration mapping for our userData, configure the utility to search the extension directories for the configuration mappings. To configure the utility to search these directories, include the -Dinstance parameter when you run the utility. By including this parameter, the utility includes the WebSphere Commerce EAR XML directory in the class path and is able to locate all DSL configuration files. When you specify the -Dinstance parameter, ensure that you have the files for the instance that you specify in the utilities_root/instances/instance_name directory or its subdirectories. We must also have access to the files. The utility can then locate the DSL configuration mapping for our custom data. If you do not configure the utility to find this file by including this parameter, an error that the custom field is not defined in the user data occurs. We must specify this parameter when loading custom data for business objects that have a Data Service Layer configuration that is in an extension directory. For example, specify the parameter when we are loading the following types of data:

• Custom JAR files or Data Service Layer configuration files from the development environment to the WebSphere Commerce server.

• Any type of data into a workspace environment.

• Catalog data when base change history recording is enabled. When this history recording is enabled, change history is recorded whenever an object that is approved content is created, updated, or deleted with Management Center or the Data Load utility. For more information about this change history recording, see Enable the change history for approved and canceled task groups.

• Catalog entry description override data with the CatalogEntryDescriptionOverrideMediator mediator.

• SEO information for catalog entry or category data. To load SEO data with catalog entry or category data, include the loadSEO parameter with a value set to be "true". We can load SEO data with the following mediators:

  • CatalogEntryMediator

  • CatalogEntrySEOMediator

  • CatalogGroupMediator

  • CatalogGroupSEOMediator

• WebSphere Commerce search index requests. We can request search indexing to occur with the following mediators:

  • AttributeDictionaryAttributeSearchIndexMediator

  • CatalogEntrySearchIndexMediator

  • CatalogGroupRelationshipSearchIndexMediator

  • CatalogGroupSearchIndexMediator

  • Custom mediators that support indexing custom data types and extend com.ibm.commerce.foundation.dataimport.dataload.mediator.AbstractSolrInputDocumentMediator.

• Marketing image map content that includes data for custom fields. We can load marketing image map content with the following mediator:

  • MarketingContentImageMapMediator

-DlogFilePath

The full path location for where we want the Data Load utility log file to be created. By default, the wc-dataload.log log file is created in the following directory:

• (Linux) utilities_root/logs

• (Developer) WCDE_installdir\logs

-DlogFileWithTimestamp

Used to enable the Data Load utility to append a time stamp to the Data Load utility log file. By default, this parameter is turned off (false). To turn on the appending of a time stamp, enter -DlogFileWithTimestamp=true.

Running the Data Load utility

1. (Developer) On a command line, go to the WCDE_installdir\bin directory.

2. (Linux) Open a command line in the Utility server Docker container. Change the directory to utilities_root/bin directory.
   For information about entering and leaving containers, see Running utilities from the Utility server Docker container.

3. Run the following command to use Data Load utility to run the specified data load order configuration file. The file identifies the input files that include the data to load and identifies the business object configuration files that define how to load the data.

   • (Linux) ./dataload.sh ../LoadOrderFilePath/LoadOrderFile.xml

   • (Developer) dataload ..\LoadOrderFilePath\LoadOrderFile.xml

   Where

   LoadOrderFilePath

   The relative path to the data load order configuration file that identifies your input files and configuration files.

   LoadOrderFile.xml

   The data load order configuration file. Sample files are provided with WebSphere Commerce in the component-based directories in the following directories:

   • (Linux) utilities_root/samples/DataLoad

   • (Developer) WCDE_installdir\samples\DataLoad

   By default the sample load order configuration file is typically named wc-dataload.xml or wc-dataload-object.xml, where object is the name of the object that we are loading. For example, wc-dataload-catalog-entry.xml

   As an example, the following command runs the Data Load utility to load catalog entry data

   • (Linux) ./dataload.sh ../samples/DataLoad/Catalog/wc-dataload-catalog-entry.xml

   • (Developer) dataload ..\samples\DataLoad\Catalog\wc-dataload-catalog-entry.xml

   As an example, the following command runs the Data Load utility to load catalog entry data, and specifying an alternative environment configuration file to process:

   ./dataload.sh ../samples/DataLoad/Catalog/wc-dataload-catalog-entry.XML ../samples/DataLoad/Catalog/environmentfile.XML

4. If you loaded catalog filters, then you need to repopulate the EXPRESSION database table by running the PopulateExpressionsForCatalogFilter scheduler job manually or wait for the job to run. For more information about scheduler jobs, see Viewing scheduled jobs.