Search and Marketing statistics in WebSphere Commerce

In a typical deployment model, business users typically do not have access to the production system. The situation is different when business users want to review live production data, such as search and marketing statistics. In this case, the data must be replicated into a staging or authoring environment to be viewed in the Management Center. The production system is not impacted by the data replication operation. It is the staging or authoring system that pulls the production data back into the staging database, without involving any production CPU cycles. The DownloadStatistics scheduler command does statistics replication from production into the staging or authoring environment. DownloadStatistics reads the search section in the wc-component.xml file from the Infrastructure component for the tables to replicate.

• Two types of statistics tables are provided. The first type is a log-style statistics table where incremental data is appended to the end of the table over time. This table requires only incremental replication. The other type of statistics table is the running-counter style where all rows are active with a running counter. In this case, a full table replication is required.

• The production system database connection is the data source connection to the production system for Quick Publish. This data pipeline is used for replicating statistics data from the production system back to the staging area.

• The DownloadStatistics scheduler command must be configured to run on a recurring basis from a staging or authoring environment, where business users review the statistic in the Management Center.

The following code is an example of the DownloadStatistics scheduler job configuration in the infrastructure component wc-component.xml file:

<_config:extendedconfiguration>
  <_config:configgrouping name="DownloadStatistics-Configurations">
    <_config:property name="SearchStatistics" value="DownloadStatistics-SRCHSTAT"/>
    <_config:property name="ElementStatistics" value="DownloadStatistics-DMELESTATS"/>
    <_config:property name="EMarketingSpotStatistics" value="DownloadStatistics-DMEMSPOTSTATS"/>
    <_config:property name="ExperimenStatistics" value="DownloadStatistics-DMEXPSTATS"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-SRCHSTAT">
    <_config:property name="TableName" value="SRCHSTAT"/>
    <_config:property name="Mode" value="move"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMELESTATS">
    <_config:property name="TableName" value="DMELESTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMEMSPOTSTATS">
    <_config:property name="TableName" value="DMEMSPOTSTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMEXPSTATS">
    <_config:property name="TableName" value="DMEXPSTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
</_config:extendedconfiguration>

Once search statistics are enabled in production, set up the replication scheduler job in the staging area. This step is necessary so that business users can view the search statistics without the need for a production connection. The following search and marketing statistics tables are configured by default:

• SRCHSTAT

• DMELESTATS

• DMEMSPOTSTATS

• DMEXPSTATS

These tables contains data that represents statistics data from the Marketing component and WebSphere Commerce Search.

Note: On the system to which data is replicated, that system cannot collect statistics from its local environment. For example, if search statistics is configured to be replicated from production to staging, then the staging environment can be used only to review production search statistics. It cannot have search statistics enabled in the staging environment. This restriction is to avoid key collision in the staging database.

We must ensure that the SRCHSTAT table in the staging and production system is regularly cleaned (dbclean) to maintain overall performance when search statistics are enabled.

Tuning search statistics

We can configure search statistics properties in the catalog component configuration file (wc-component.xml) file. See Search properties in the component configuration file (wc-component.xml).