Experiments

Experimentation enables Marketing Managers to run alternative Web activities concurrently with existing activities. Experiments are similar to Web activities, in that they deliver an alternative marketing message. An experiment comprises different elements, which include the target customer segment, the e-Marketing Spot in which the marketing message is delivered, and the content of the marketing message. Marketing managers can create experiments which are based on existing activities, but change a single element:

The Web activity displayed in a particular e-Marketing Spot
The content displayed in support of an activity
The target customer segment
The e-Marketing Spot in which the activity displays

The changed element in the experiment is called the test element, while the element in the original Web activity is called the control activity. Similarly, the original Web activity is called the control activity. After the experiment is complete, Marketing Managers can compare results between the experiment and the control activity. These results help determine whether the current marketing effort is effectively maximizing revenue by targeting the right audience with the most appropriate message. If not, incorporating the test element into the control activity might improve the efficiency of the marketing message.

When creating an experiment, the Marketing Manager specifies a percentage which determines the number of customers who will see the test element. This is known as the display ratio. When customers browse the site, a random number is generated to determine whether they see the original Web activity, or the corresponding test element. When creating an e-Marketing Spot experiment, this ratio determines the percentage of times the Web activity displays in the experiment's target e-Marketing Spot as compared to the control e-Marketing Spot. Similarly, when creating a customer segment experiment, this ratio determines the percentage of requests which target the test segment compared to the control segment. Marketing Managers can also specify whether a particular customer continues to see the original result, or a new random number is generated, either for the next site visit or for the next page view. This is controlled by setting one of the following experiment result persistence values:

Lifetime of an experiment: The experiment result persists throughout the entire duration of the experiment.
Note: For this option to be applicable for guest customers, ensure that your site has enabled cookies for session management. Contact your Site Administrator.
Session: The experiment result persists for a single user session. A session is the duration of time that a customer browses your site without closing their Web browser.
Request: The experiment result is regenerated for each request. A request is submitted every time a customer views a page, refreshes a page, or returns to a page from elsewhere in the site.

Retained results are applicable only to a single customer.

To analyze and compare the experiment with the original activity, the experiments system uses a set of listeners and database tables to keep separate statistics to validate the experiment. Based on the statistical feedback available through the WebSphere Commerce Accelerator, the Marketing Manager can choose to update the original activity to incorporate the experiment's test element. The experiments generate information about the number of impressions, and the number of times the experiment was clicked. In both cases, the corresponding conversion rate is tracked, as well as the associated revenue generated. For example, if an experiment uses test advertising content, and the Marketing Manager finds that the experiment has a conversion rate that doubles the control activity's content, they can update the control activity such that it uses the content previously considered the test element.

Where e-Marketing Spots are scheduled to display both normal marketing content, and experimental content, the experimental content has precedence over normal campaign content. You can assign a priority to each experiment to define which experiment takes precedence when multiple experiments are scheduled to the same e-Marketing Spot.

Since spaces available to display marketing content in an e-Marketing Spot might be limited, apply these rules to share spot spaces between normal and experimental activities:

Add optional maximum number of result attributes to the e-Marketing Spot bean for each type of content for experimental campaign activity
By default, set the value of these attributes to the same value as normal campaign activities
The maximum number of content from normal campaign activity allowed = maximum number of result (for normal campaign activity) - number of experimental content

Experiments are created using the WebSphere Commerce Accelerator. If your store has enabled workspaces, experiments may be associated with an assigned task.

The following sections provide additional information about the four kinds of experiments.

Web activity testing

This experiment tests the effectiveness of various Web activities for a specific e-Marketing Spot. Marketing Managers select a control Web activity and an e-Marketing Spot in which to display the marketing content, and then select Web activities as test elements to display in the selected e-Marketing Spot instead of the control activity. Each activity in the experiment is assigned a display ratio, which determines the chance of displaying in the e-Marketing Spot based on a randomly generated number.

Content testing

This experiment tests the effectiveness of content for a Web activity in an e-Marketing Spot. Marketing Managers select a Web activity and an e-Marketing Spot in which to display the marketing content, and select one or more additional contents as test elements to display in the selected e-Marketing Spot instead of the control content. Each content in the experiment is assigned a display ratio, which determines the chance of displaying in the e-Marketing Spot based on a randomly generated number.

Segment testing

This experiment tests the effectiveness of the targeted customer segment of a specific Web activity in an e-Marketing Spot. Marketing Managers select a Web activity and an e-Marketing Spot in which to display the associated marketing content, and then select one or more customer segments as test elements to use as the target segment of the selected Web activity. Each customer segment in the experiment is assigned a display ratio, which determines the chance of the segment being used as the target of the Web activity based on a randomly generated number.

Spot testing

This experiment tests the effectiveness of the e-Marketing Spot to which a Web activity is assigned. Marketing Managers select a Web activity and an e-Marketing Spot in which to display the marketing content, and then select one or more additional e-Marketing Spots as test elements in which the the content associated with the control Web activity will display. Each e-Marketing Spot in the experiment is assigned a display ratio, which determines the chance of the content displaying in the test element based on a randomly generated number.

Related concepts