Extracting Commerce Composer data with the Data Extract utility

We can configure and run the Data Extract utility from a command-line utility to extract Commerce Composer data for a store into CSV output files. We can then use the Data Load utility to load the extracted data into stores in other environments, such as for testing purposes. By using these utilities to copy data between instances, business users do not need to manually create the same assets across multiple instances.

Before beginning

Ensure that you complete the following tasks:

• Verify that the store to extract data from includes Commerce Composer objects to extract. If there is no data for the object in the store, the output files that the utility generates do not include any data.

Note: If you plan to load the extracted data into another instance, the store to load data into must have the Commerce Composer tool enabled.

Task info

When we are configuring the Data Extract utility to extract Commerce Composer data, we can extract data for the following types of Commerce Composer objects:

• Layouts

• Layout templates

• Widgets

• Widget multi-lingual descriptive information

• Pages

• Page multi-lingual SEO information

Sample configuration files are provided for you to edit and use to quickly configure the utility. The sample configuration files are in the following directory:

• WCDE_installdir\samples\DataExtract\CommerceComposer

These sample files configure the Data Extract utility to extract data into CSV output files. The output files are compatible with the Data Load utility for use as input files. If needed, we can edit the other sample configuration files to change how the utility retrieves, transforms, and outputs data. For more information about the sample configuration files and the data that the files are configured to extract for each object type, see Sample: Extracting Commerce Composer data. Notes:

• The sample configuration files configure the utility to extract data for a single Commerce Composer object type in a single operation. To extract data for all Commerce Composer object types, we must run the utility repeatedly, once for each object type.If you plan to load extracted data for multiple Commerce Composer object types into another instance, the data for one object type might overwrite the data for another object type. This replacement of data can cause data for some object types to be missing after you load the data for all object types. To ensure that all data is available in your target database, extract and load the data for one object type at a time in the following sequence:

  1. Layouts

  2. Layout templates

  3. Widgets

  4. Widget multi-lingual descriptive information

  5. Pages

  6. Page multi-lingual SEO information

• When we use the utility with sample files to extract information in multiple languages, the utility generates 2 output files. 1 file includes the data in only the store language that is configured in the business context attribute of the environment configuration file. The second file includes the data for the configured language and all other languages.

• The Data Extract utility extracts data for only a single store in each extract operation. We can specify the store in the environment configuration file.

  (Enterprise) If the site uses an extended sites store model, we cannot extract data for all extended site stores at once. To extract and load asset store objects to extended sites stores, we must load the data into each extended site store individually.

• The generated file does not include any columns or values for object unique IDs. These values are specific to the store or instance that the object is being extracted from. During an extract operation, the utility can replace some primary key values such as the unique IDs with the corresponding unique external identifier value. The utility then outputs the external identifier value instead of the unique ID number. When you load the CSV file into a store with the Data Load utility, the utility resolves the unique ID for objects from the external identifier value. Since the unique ID number can be different between environments, a load operation can fail if you include unique ID values in your input file.

• The sample configuration files configure the Data Extract and Data Load utilities to extract and load all data for a Commerce Composer object type. To load specific extracted data for an object type, we can configure the Data Load utility to load only specific load items. Use the -DLoadOrder parameter when you run the Data Load utility to identify the specific load item or items to load. If we want load multiple load items, include the list of load item names in comma-separated list that is enclosed in double quotation marks. For example, to load only extracted widget restriction group data, we can extract all Commerce Composer template data and then run the Data Load utility to load the template data with the following configuration parameters to load the wireframe slot definition and widget registration data:
  -DLoadOrder="RegisterWidgetDef, WidgetSlot"

Procedure

1. Configure the Data Extract utility. When we are configuring the utility, copy the sample configuration files and directories to another local directory. Edit and use your copy of the sample configuration files. These sample files are in the following directory and subdirectories:

   • (Linux) utilities_root/samples/DataExtract/CommerceComposer

   • (Developer) WCDE_installdir\samples\DataExtract\CommerceComposer

   There is a separate data extract order configuration file (wc-dataextract-business-object.xml) for each type of Commerce Composer object:

   • For layouts, use the wc-dataextract-layout.xml configuration file.

   • For layout templates, use the wc-dataextract-template.xml configuration file.

   • For widgets, use the wc-dataextract-widget.xml configuration file.

   • For pages, use the wc-dataextract-page.xml configuration file.

2. Run the Data Extract utility to extract the data for the Commerce Composer object for the configured store.

   1. Open a command-line utility and go to the appropriate directory:

      • (Linux) utilities_root/bin

      • (Developer) WCDE_installdir\bin

   2. Enter the following command to run the sample data order file and extract the information.

      • (Linux) ./dataextract.sh ../samples/DataExtract/CommerceComposer/wc-dataextract-object.xml

      • (Developer) dataextract ..\samples\DataExtract\CommerceComposer\wc-dataextract-object.xml

      When the utility completes the extraction process, the utility generates the output files for the object within the data output location that you configured in the data extract order configuration file.

      We can configure the utility by including optional parameters in the command. For more information about the available parameters, see Data Extract utility

3. Verify that all of output files are successfully generated.

   1. Go to the data output location and open each of the generated output files for the object and review the file contents. By default, these files are generated in the following directory and subdirectories:

      • (Linux) utilities_root/samples/DataExtract/CommerceComposer/output

      • (Developer) WCDE_installdir\samples\DataExtract\CommerceComposer\output

      Within each output CSV file, the first row of the file includes the column headings. Each subsequent row includes the data for a single extracted object. The columns within the file are the same as the files that are needed to load the object data with the Data Load utility.

      Note: If we created our own configuration files, ensure that your output files are configured to include the correct case-sensitive column names and corresponding values. For instance, LanguageId cannot be languageId in your output file. If your file does not include the correct case-sensitive column names, the Data Load utility might not load successfully. For more information about the column names and data that can be in each output file, see Commerce Composer object input file definitions.

   2. Compare the contents of the file with the data in the source database tables to verify that the correct data is extracted.

What to do next

We can use the generated output files as input files with the Data Load utility to load the data into another WebSphere Commerce store or instance to quickly create or replace Commerce Composer objects in the other stores or instances. For example, to set up multiple testing environments that include the same data, such as for testing the display of new layouts or widget functionality on the storefront. We can also load the data to copy objects between stores or instances when business users need to include those objects on store pages in the target stores or instances. By extracting and loading Commerce Composer objects, developers and business users do not need to re-create the objects with the Commerce Composer tool in each store or instance. After you load your extracted data into a new environment, the objects become available in the Commerce Composer tool for the specified store.Notes:

• When you load your extracted Commerce Composer data, ensure that you load the data in the correct sequence.

• For widgets and page information that is in multiple languages, load the information in the default store language first. Then, load the data in other languages. By loading the information in the default language first, we can reduce the risk of errors during the load operation. We can specify the default language in the environment configuration file business context.

• The Data Extract utility does not extract any assets associated with Commerce Composer objects, such as images, marketing content, or multimedia. If you load extracted Commerce Composer data into another instance, copy or move any associated assets for the Commerce Composer objects into the target instance. If you do not copy or move the assets, the Commerce Composer objects that you load might not display or function correctly.

Tip: When you finish loading your extracted data, back up the file before your run another extract operation for the same type of object. You might need the older file if you need to delete or reload the data that you previously loaded. When you run an extract for the same object type, the Data Extract utility overwrites the content of the output file if the file name and directory location remain unchanged.

For samples that demonstrate how to load data for each type of Commerce Composer object, see Commerce Composer samples. When you follow these samples, edit the load order configuration files to configure the Data Load utility to load the files that were generated by the Data Extract utility.

Related concepts
Overview of the Data Extract utility