Experiment statistics collection and calculation

Understanding how the server collects and calculates statistics for the test elements in your marketing experiments can help you evaluate the results of the experiment.

The server displays the following statistics for the test elements in the experiment:

We can view these statistics while the experiment is running and after it ends. See Viewing the statistics of an experiment.


Enable the collection of revenue statistics

To collect revenue statistics, we must enable the following statistical counters before your experiment starts:

See Enable the collection of marketing statistics in Management Center.


Frequency of statistical updates

The following table identifies how often statistics are updated in the Statistics tab while an experiment is running.

In the Views and Clicks columns The server gathers and displays these numbers using the SaveMarketingStatistics scheduled job. By default, this job runs every 30 minutes. We can change the frequency of this job; see Scheduling the SaveMarketingStatistics job.
In the View Orders, View Revenue, Click Orders and Click Revenue columns The server gathers and displays these numbers using the RaiseECEvent scheduled job. By default, this job runs every 5 minutes. We can change the frequency of this job; see Statistics considerations.


Difference between View and Click statistics

When interpreting statistical data about a web activity containing an experiment, it is important to understand the following terms:

In most cases, the click numbers are more meaningful because there is more certainty that the customer was interested in the test element if they clicked it. When interpreting the view numbers, we cannot assume that the customer actually saw the test element when the server displayed it. The view numbers, however, can give you some insight into how appealing the test element is. For example, we can calculate the click-through rate by dividing the number of times a user clicked a test element by the number of times the server displayed that element; the higher the percentage, the more successful the test element. Here is an example:

Test element Views Clicks Click-through rate
Advertisement A 500 100 20%
Advertisement B 1000 100 10%

These two advertisements were clicked the same number of times (100); however, the server displayed Advertisement B twice as many times as Advertisement A. The click-through rate indicates that Advertisement A is a more appealing advertisement.


How revenue numbers are calculated

As each customer successfully submits an order on the storefront, the server assesses the order to determine if there is any correlation between any items in the order and what the customer saw in any e-Marketing Spots used in any experiments in the current session. The server then calculates the revenue numbers using the logic described in the following table. In all cases:

If Then
The experiment displays a catalog entry The revenue numbers are updated if the customer purchases the displayed catalog entry.
The experiment displays a category The revenue numbers are updated if the customer purchases any item from the displayed category.
The experiment displays an advertisement

  • If the advertisement is associated with a catalog entry (the click action is Display catalog entry), revenue numbers are updated if the customer purchases the displayed catalog entry.

  • If the advertisement is associated with a category (the click action is Display category), the revenue numbers are updated if the customer purchases any item from the displayed category.

  • If the advertisement is not associated with any catalog entry or category, revenue numbers are updated if a customer purchases anything during the session.


Experiment statistics in a staging environment

Marketing statistics are captured on the production environment. To view experiment statistics in a staging environment for analysis, a System Administrator can transfer the statistical data from the production server to the staging environment by running the ExportStats and ImportStats commands. When running the commands, specify the DMEXPSTATS and DMELESTATS tables, which contain e-Marketing Spot statistics. See Copying marketing statistics from the production environment.