WebSphere Commerce Search interactions

Search server architecture

A direct-access path to site navigation and the search server exists on the Solr search server. This server consists of...

• REST services
• Search runtime framework that reuses the current search programming model
• WebSphere Commerce foundation services that also provide access to the WebSphere Commerce database

Search server architecture...

Where:

• The REST service mapper is an jersey implementation of JAX-RS 2.0 that uses configurations to map a REST URL to a resource. This resource then calls the embedded search run time.

• The search run time is a pluggable programming framework that allows search expressions to be run by using properties that are defined in a configuration-based search profile. Once a search expression is composed, the runtime then forwards the request to the embedded Solr run time for processing.

• The foundation service is the WebSphere Commerce core run time that can provide basic server services, such as the JDBC Query Service to access the WebSphere Commerce database.

Example

When shoppers enter a search query in the storefront, the query is translated into a REST call. The template used to build the call is in the SearchSetup.jspf file. This template, found in the <wcf:rest/> tag, builds the call as a string.

<wcf:rest var="catalogNavigationView1" url="${searchHostNamePath}${searchContextPath}/store/${WCParam.storeId}/productview/${restType}">

The REST call is submitted to the Search server and intercepted by the request handler specified in the call (ProductViewHandler in the above scenario). The request handler checks the wc-rest-resourceconfig.xml file to confirm that the profile being passed is in the list of search profiles defined for the URI. If the profile is not in the list defined for the URI, an HTTP 400 error will be generated. If no search profile is specified in the URL (&searchProfile=X), the request handler uses the first search profile defined in wc-rest-resourceconfig.xml. For example, for a productview/bySearchTerm request, that profile is IBM_findProductsBySearchTerm.

<GetUri uri="store/{storeId}/productview/bySearchTerm/{searchTerm}" description="Get products by search term based on below search profile." searchProfile="IBM_findProductsBySearchTerm,IBM_findProductsByNameOnly,IBM_findProductsByNameAndShortDescriptionOnly,IBM_findProductsByUnstructureOnly,IBM_findProductsBySearchTerm_Summary"/>

If the call is accepted by the request handler, the Search server uses the Jersey implementation of JAX-RS 2.0 to convert the name-value pairs in the URL into a more usable form via Java annotation. For example, in the ProductViewResource:

@Path("store/{storeId}/productview") @Description("This class provides RESTful services to get the ProductView details.") @Encoded public class ProductViewResource extends AbstractSearchResourceHandler {

Where the store/{storeId}/productview/{partNumber} URL is mapped to the following method in bold:

@GET
@Path(BY_PART_NUMBER)
@Produces({ APPLICATION_JSON })
@ApiOperation(value = "Gets products by part number.")
@ApiImplicitParams({
@ApiImplicitParam(name = PARAMETER_ASSOCIATION_TYPE, value = PARAMETER_ASSOCIATION_TYPE_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_ATTRIBUTE_KEYWORD, value = PARAMETER_ATTRIBUTE_KEYWORD_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_CATALOG_ID, value = PARAMETER_CATALOG_ID_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_CONTRACT_ID, value = PARAMETER_CONTRACT_ID_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_CURRENCY, value = PARAMETER_CURRENCY_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_LANG_ID, value = PARAMETER_LANG_ID_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_CHECK_ENTITLEMENT, value = PARAMETER_ENTITLEMENT_CHECK_DESCRIPTION, required = false, dataType = DATATYPE_BOOLEAN, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_ATTACHEMENT_FILTER, value = PARAMETER_ATTACHMENT_FILTER_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY),
@ApiImplicitParam(name = PARAMETER_PROFILE_NAME, value = PARAMETER_PROFILE_NAME_DESCRIPTION, required = false, dataType = DATATYPE_STRING, paramType = PARAMETER_TYPE_QUERY)})
@ApiResponses(value = {
@ApiResponse(code = 200, message = RESPONSE_200_DESCRIPTION),
@ApiResponse(code = 400, message = RESPONSE_400_DESCRIPTION),
@ApiResponse(code = 401, message = RESPONSE_401_DESCRIPTION),
@ApiResponse(code = 403, message = RESPONSE_403_DESCRIPTION),
@ApiResponse(code = 404, message = RESPONSE_404_DESCRIPTION),
@ApiResponse(code = 500, message = RESPONSE_500_DESCRIPTION) })
public Response findProductByPartNumber(
@ApiParam(value = "The product part number.", required = true) @PathParam(PART_NUMBER) String partNumber) {

The resource then calls the embedded search runtime code, which calls the wc-search.xml file using the search profile names to look up the provider and preprocessors to call. This method is similar to a factory assembly line, where each business component can contribute its own piece of the search expression into the main query. After the search run time constructs the final Solr query, it calls Solr and processes the output by using postprocessors that are defined in the search profile. Then, the REST handler outputs the response.

The search expression is represented by the SearchCriteria Object, which contains a set of control parameters in the form of name-value parameter maps. For example, the following snippet from the trace.log file shows that the SearchCriteria is processed by the SolrSearchEDismaxQueryPreProcessor:

SolrSearchEDi > com.ibm.commerce.foundation.internal.server.services.search.query.solr.SolrSearchEDismaxQueryPreProcessor invoke(SelectionCriteria, Object) ENTRY Search profile: 'IBM_findProductsBySearchTerm', Control parameters: '_wcf.search.exclude.type=[],' 'DynamicKitReturnPrice=[true],' '_wcf.search.currency=[USD],' '_wcf.search.manufacturer=[], ' '_wcf.search.runAsId=[-1002],' '_wcf.search.contract=[10001],' '_wcf.search.catalog=[10052],''_wcf.search.catalog=[10052], ''_wcf.search.catalog=[10052],' '_wcf.search.internal.response.format=[json],'

After a search expression is composed, the run time forwards the request to the embedded Solr run time for processing.

If there are any parameters that we want to pass in from the storefront, but do not want to specify them in the findProductByPartNumber method for example, we can add it to the control parameter mapper in the wc-component.xml file. This file maps from the name-value pair into the control parameter in the SearchCriteria object.

Search programming model

The search runtime framework is driven by a programming pattern that is similar to a factory assembly line, where each business component can contribute its own piece of the search expression into the main query, which is then run against the Solr search engine.

The search runtime programming model uses POJO and raw data that is returned from the search server to perform simple name-value pair mapping: The search run time contains two main components: the search expression provider and the search expression processor:

Search Expression Provider

Depending on the nature of the request, that is, the search profile, other business components might get involved, such as Marketing for search-based merchandising rules, or contracts for entitlement. Each business component is responsible for contributing a portion of the search expression, which gets combined with the main search expression generated by the Search REST services.

As a result, the consolidated search expression is run by the search processor.

The default SolrRESTSearchExpressionProvider performs the following high-level steps in this order:

1. Validates that a search profile is given by calling SolrSearchProfileNameValidator and immediately stops if none is provided.

2. Validates the corresponding index name by calling SolrSearchIndexNameValidator and immediately stops if invalid.

3. Validates the corresponding workspace information by calling SolrSearchWorkSpaceValidator and sets the information to let the processor know.

4. Validates the search expression to ensure that the query expression is not empty and does not contain special characters by calling SolrSearchExpressionValidator.

5. Creates a list of search query expression providers that are defined in the search profile where each provider contributes a portion of the search expression.

Search Expression Processor

A search processor is the central processing unit for integrating with the search engine. Its responsibility is to run the Solr expression, based on the search profile attributes, and capture the response from the search engine.

The SolrRESTSearchExpressionProcessor is a default implementation of the expression processor that is defined in the SearchExpressionProcessorFactory.properties file on the search server.

The default SolrRESTSearchExpressionProcessor composes the final Solr expression and sets up all the required bootstrap settings for running in the Solr search run time. The result set is then reformatted into a JSON object to be returned to the caller.

The expression processor performs the following high-level steps in the following sequence:

1. Injects a debug parameter option into the SolrQuery object by calling SolrSearchDebugQueryPreprocessor.

2. Generates a list of index fields to be included in the search result set by calling SolrSearchResultFieldQueryPreprocessor.

3. Includes a relevancy score and extra tracing information into the SolrQuery object by calling SolrSearchPreviewQueryPreprocessor.

4. Enables spell correction and includes its associated parameter options into the SolrQuery object by calling SolrSearchSpellCorrectionQueryPreprocessor.

5. Enables highlighter and includes its associated parameter options into the SolrQuery object by calling SolrSearchHighlighterQueryPreprocessor.

6. Injects pagination parameter options into the SolrQuery object by calling SolrSearchPaginationQueryPreprocessor.

7. Injects sorting parameter options into the SolrQuery object by calling SolrSearchSortingQueryPreprocessor.

8. Composes a list of facet fields, along with the proper settings, to be included in the search result set by calling SolrSearchFacetQueryPreprocessor.

9. Injects the dismax query-related expression by calling SolrSearchEDismaxQueryPreProcessor as eDismax handler must occur before the main query because the boost query control parameter might need to be removed.

10. Injects the response format parameter option by calling SolrSearchResponseFormatQueryPreprocessor.

11. Injects the main search query string into the SolrQuery object by calling SolrSearchMainQueryPreprocessor.

12. Injects extra custom SolrQuery parameters into the final Solr query expression by calling SolrSearchCustomQueryPreprocessor.

13. Creates a list of search query preprocessors that are defined in the search profile to allow modifications to the SolrQuery object right before it is sent to Solr for processing.

14. Issues a request against Solr to process the SolrQuery object.

15. Creates a list of search query postprocessors that are defined in the search profile to allow modifications to the physical data object, SearchResponse, right after the QueryResponse is returned from the Solr server.

A query preprocessor is used to modify the SolrQuery object right before it is sent to the Solr server for processing.

A query postprocessor is used to modify the QueryResponse object that is returned from the Solr server.