Index-building topologies and scenarios

WebSphere Commerce Search indexes are created separately based on a specific master catalog. Deploy the WebSphere Commerce Search index, then separately manage and rebuild each index to refresh its data.

Search indexing methods

A full indexing rebuilds the entire search index, while a delta indexing performs only incremental updates on the existing operational search index.Important: We must periodically fully rebuild the search index to ensure that the index size on the file system remains manageable. That is, when only delta indexing is performed, the index files that marked for deletion continue to accumulate. When a full indexing is performed, an optimize request is passed in that fully deletes the index files identified from previous delta indexes.

Before you deploy your index, consider the index-building scenarios, depending on your WebSphere Commerce Search environment. The following topologies are typical:

• A production environment with a dedicated index-building machine.

• A staging and clustered search server.

Search index types and subtypes

WebSphere Commerce Search contains the following search index types to suit your business and search requirements:

Catalog entry index

A search index for catalog entries in both master and sales catalog.

The catalog entry search index contains the following content:

Structured content

Structured content includes items in the product catalog and delivers search results based on items that are sold in your store.

Unstructured content

Unstructured site content includes documents that do not adhere to a specific data model, such as product attachments contained in various formats. For example, content such as user manuals and warranty information are considered unstructured content. Its elements, construction, and organization are typically unknown and can vary depending on its file type.

Site content

Site content includes HTML and other site files from WebSphere Commerce starter stores. It is fetched and crawled by the site content crawler.

Catalog group index

A search index for categories in both master and sales catalog.

WebSphere Commerce Search contains the following search index subtypes, which keeps data in a separate core for performance reasons:

Inventory

The inventory index, a separate index that contains index data, is an extension of the product index. For accurate inventory status, we can refresh the inventory index more frequently than the product index.

Price

Sets up a subindex for price data. Prices are indexed using Index Load, as it can populate a large amount of data into a separate extension index faster than the Catalog Entry index can index price data. See Index Load.

For more information about extension indexes, see WebSphere Commerce Search extension indexes.

Index-building topologies and scenario examples

The following sample topologies are available for small to large index sizes:Important: In all scenarios, IBM recommends to only perform indexing in an authoring or staging environment. That is, an environment used for quality assurance (QA) purposes, and if required, containing access to the production database. Otherwise, the runtime search performance or index integrity can be affected by indexing within a production environment.

Small or medium index size: WebSphere Commerce production environment with a dedicated index-building machine

A dedicated index-building machine, which is known as the master, is used to build the index. The index is then replicated to other search machines, which are known as targets. The runtime search performance is not affected during index-building.

Note: Building the search index can still be a timely process, and might occasionally lead to inconsistencies between the database and search index. However, the runtime search performance is not affected during index-building due to the offset of the processing load to another machine.

The following diagram illustrates a typical WebSphere Commerce Search deployment for a small or medium index size:

(Professional) (Enterprise)

The index-building processRecommended: Large index size: Staging propagation and clustered search servers

The index-building is performed on the WebSphere Commerce staging database and search staging machine. Business users can test their new data in the staging database with the updated index. After successfully completing their tests, the data is propagated from the staging database to the production database, and the index is replicated from the search staging machine to the search production machines using the master indexing server. This is the recommended approach, as this option imposes the least amount of risk to the production environment and provides a more flexible environment for making incremental changes. The following diagram illustrates a typical WebSphere Commerce Search deployment for a large index size:

See Indexing with staging propagation.

• Indexing with staging propagation
  When indexing with staging propagation, business users apply changes to a staging area, which is later propagated into the production environment by IT administrators. An index repeater is used to capture the most recent deployed index content, while also serving as a backup.

• Propagating the search index
  We can propagate the WebSphere Commerce search index by calling the indexprop REST API. Use Indexprop to request that the search index repeater replicate with the staging search index. This replication ensures that the catalog changes are populated into the WebSphere Commerce search index in production.

• Propagating the WebSphere Commerce Search index with the repeater
  We can use the indexprop utility to propagate the WebSphere Commerce search index. It sends an HTTP request to the search index repeater to start replicating with the staging search index. This process ensures that the catalog changes are populated into the WebSphere Commerce Search index in production.

• Indexing with Quick Publish for emergency fixes
  When indexing with Quick Publish, emergency fixes are applied directly to the production system by using workspaces, and bypassing staging.

• Indexing without staging propagation
  While staging propagation is recommended for WebSphere Commerce Search, a Quality Assurance (QA) server can be used for testing and previewing changes in a non-staging environment before propagating to production.

Related tasks
Setting up the search index
Propagating the WebSphere Commerce Search index with the repeater