Index Load configuration files for merging indexes

Index Load requires configuration files before it can merge search indexes.

Base merge configuration file (wc-indexload-merge-base.xml)

The wc-indexload-merge-base.xml file contains business object configurations. We can find a sample version of this file as server.config.dir/resources/search/index/indexload/wc-indexload-merge-base.xml directory. Example

<_config:DataloadBusinessObjectConfiguration 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload-businessobject.xsd" 
  xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

  <_config:DataLoader className="com.ibm.commerce.foundation.server.services.indexload.loader.solr.SolrIndexLoadCSVLoader" >
   
    <_config:DataReader className="com.ibm.commerce.foundation.server.services.indexload.reader.solr.SolrIndexLoadCSVReader" 
                        firstLineIsHeader="true" useHeaderAsColumnName="true" />

     <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.internal.server.services.indexload.builder.SolrIndexLoadMapObjectBuilder">
       
      <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.internal.server.services.indexload.mediator.SolrIndexLoadBusinessObjectMediator" />
     
    </_config:BusinessObjectBuilder>
    
  </_config:DataLoader>
  
</_config:DataloadBusinessObjectConfiguration>

Merge configuration file (wc-indexload-merge.xml)

The wc-indexload-merge.xml file contains configurable performance attributes and load item configurations. For more information about the configurable performance attributes, see Tuning Index Load.

It can contain one or multiple LoadItem definitions, with every LoadItem configuration specifying the specific LoadItem configuration and coreName target. Multiple LoadItems are run in parallel, without sequence. Example

<_config:LoadItem name="PriceIndexData-1" businessObjectConfigFile="wc-indexload-merge-base.xml">
      <_config:property name="coreName" value="MC_10051_CatalogEntry_Price_generic" />
      <_config:property name="indexDataDirectory" value="C:\Shards\Shard_1" />
    </_config:LoadItem>

    <_config:LoadItem name="PriceIndexData-2" businessObjectConfigFile="wc-indexload-merge-base.xml">
      <_config:property name="coreName" value="MC_10051_CatalogEntry_Price_generic" />
      <_config:property name="indexDataDirectory" value="C:\Shards\Shard_2" />
    </_config:LoadItem>

The following configurable performance attributes apply to merge configuration files:

DataTargetLocation

The location of the index data directory of the final merged index, relative to the core's instance directory.

Default is data/index.timestamp under the core's instance directory.

DataDirectoryScanInterval

The number of seconds to wait between each checking of the index data directory. This interval ensures that all index data directories are ready before they are merged.

Default is 60.

DataDirectoryScanRetries

The number of times to check for the availability of the index data directory. This ensures that all index data directories are ready before they are merged.

Default is 10.

OptimizeAfterMerge

Indicates whether Index Load performs index optimization after they are merged.

Default is true.

ActivateAfterMerge

Indicates whether Index Load attempts to swap in the merged index data directory to be the active index of the target core upon a successful merge. Done by reloading the core using the index.properties file.

Default is true.