Management Center: Creating search rule experiments

Create search rule experiments

Create a search rule experiment to find out how effective certain search rules are at driving sales. For example, we can use two or more different search actions in a single search rule, each on a different path, and run both paths on the storefront for a time. We can then view a report that shows which path is generating more clicks, orders, and revenue from the search results page.

Before starting
Review the search rule examples and guidelines in Search rule experiments.

Task info
Search rule experiments are configured so that only the first page of search results is considered when WebSphere Commerce collects experiment statistics. For example, if a customer clicks a link on the second page of the search results, the click is not counted in the statistics. This configuration is intentional because search rules are designed to influence shoppers by promoting a relatively small set of products that would likely fit on one page. If we want more pages of search results to be considered in the statistics, ask a site administrator to change the default configuration before the experiment starts. Site administrators can change the value of the SearchStatisticsResultPagesTrackingThreshold property.

Procedure

Open the Marketing tool.
Do one of the following things:

Open an existing search rule to experiment with, and deactivate it.
Start a new search rule that contains one set of elements to test. For example, a trigger followed by an action (a target is optional). Complete the properties for the search rule, and for each element on the path, as you normally would.

From the Branching section in the palette, drag the
Experiment element onto the path to the left of the element to test. For example, to test the Specify Top Search Results action, drag the Experiment element to the left of that action. The search rule is split into two paths.
Optional: We can add more paths to the experiment by right-clicking the Experiment element, and then clicking Add Path. We can delete any paths by right-clicking the green path arrow, and then clicking Delete.
Click the Experiment element.
Enter the experiment properties:

Property Description
Name Enter a name to identify the experiment.
Branch type
Random path

This branch type creates weighted random paths. You assign each path a percentage so that, for example, 50% of customers follow Path A and 50% follow Path B.

First path for which the customer qualifies

Use this branch type only when each of your paths starts with a target. The customer can follow only one path, which is the first path that starts with a target for which the customer qualifies. The paths are evaluated from top to bottom as represented in the Search Rule Builder.

Frequency of element changes for each customer Select one of the following options:
No change - Customer always sees the same content

This option specifies that customers who perform the search multiple times always follows the same path during the life of the experiment.

Session - Customer sees the same content within a session

This option specifies that customers who perform the search multiple times always follows the same path during the current session. If they qualify for a different path, they might follow a different path during the next session.

Request - Customer might see different content on each view

This option specifies that customers who perform the search multiple times are not restricted to the same path for any length of time. If they qualify for a different path, they might follow a different path the next time they search.

Maximum number of customers This option specifies the number of customers who are to participate in the experiment. If you do not specify the maximum number of customers, we must specify an end date.
When the maximum number of customers is reached, new customers are assigned the control path (top path in the experiment) but statistics are no longer collected because the experiment is ended.
Start Date Define a start date and time for the experiment. We can either enter a date in the field, or select a date using the calendar tool. Similarly, we can enter a time directly into the field, or we can select one using the clock tool. If you do not specify a start date, the experiment starts when the activity is scheduled to start.
End Date Define an end date and time for the experiment. We can either enter a date in the field, or select a date using the calendar tool. Similarly, we can enter a time directly into the field, or we can select one using the clock tool. If you do not specify an end date, the experiment runs when the activity is scheduled to run, unless you specify a value in the Maximum number of customers field.
Session length
We can specify the amount of time in seconds from when a customer performed the search to the time of their order for the order data to be included in the revenue statistics. The default is 3600 seconds (one hour). If one hour is sufficient for our experiment, we can leave this field empty. If the Frequency of element changes for each customer is set to Session, the session length is also the amount of time during which the customer follows the same path in the experiment. After the session time expires, the customer might be assigned a different path in the experiment.
Status Select the status of the experiment. We can select either Running or Suspended.

Click the Paths tab, and enter each path's properties in the Paths table:

Property Description
Name Enter a meaningful name in the field for each path. This name identifies the path in both this tab and the Statistics tab.
Percentage The Percentage column is visible only if we selected the Random path branch type.
Enter the percentage weighting for each path. For example, if you have two paths, A and B, and each is set at 50%, then the server assigns Path A to 50% of customers and Path B to the other 50%. We can assign any percentage to the paths in the experiment, but the total percentage for all paths must be 100%. Typically, you would assign proportionate percentages to multiple paths (for example, 50/50 for two paths 25/25/25/25 for four paths), but this proportionate percentage is not required.
Sequence This column displays the path's position relative to the other paths in the experiment. This column is hidden by default; to display it, see Use table views.
Unique ID This column displays the internal database ID for this object; this ID is used for troubleshooting purposes. This column is hidden by default; to display it, see Use table views.

Set up all the paths in the experiment by adding, removing, or changing targets and actions and their properties.
Tip: We can leave one of the paths empty. Customers assigned the empty path see search results that are not affected by a search rule action. By comparing statistics from the empty path to statistics on the other paths, we can determine whether our search rules are having a positive effect.
Save the search rule experiment.
Activate the search rule.

Example
The following is an example of a search rule experiment with two equally weighted paths that change the search results in different ways:

Related concepts
Search rule experiment statistics
Search rule experiments

Property	Description
Name	Enter a name to identify the experiment.
Branch type	Random path This branch type creates weighted random paths. You assign each path a percentage so that, for example, 50% of customers follow Path A and 50% follow Path B. First path for which the customer qualifies Use this branch type only when each of your paths starts with a target. The customer can follow only one path, which is the first path that starts with a target for which the customer qualifies. The paths are evaluated from top to bottom as represented in the Search Rule Builder.
Frequency of element changes for each customer	Select one of the following options: No change - Customer always sees the same content This option specifies that customers who perform the search multiple times always follows the same path during the life of the experiment. Session - Customer sees the same content within a session This option specifies that customers who perform the search multiple times always follows the same path during the current session. If they qualify for a different path, they might follow a different path during the next session. Request - Customer might see different content on each view This option specifies that customers who perform the search multiple times are not restricted to the same path for any length of time. If they qualify for a different path, they might follow a different path the next time they search.
Maximum number of customers	This option specifies the number of customers who are to participate in the experiment. If you do not specify the maximum number of customers, we must specify an end date. When the maximum number of customers is reached, new customers are assigned the control path (top path in the experiment) but statistics are no longer collected because the experiment is ended.
Start Date	Define a start date and time for the experiment. We can either enter a date in the field, or select a date using the calendar tool. Similarly, we can enter a time directly into the field, or we can select one using the clock tool. If you do not specify a start date, the experiment starts when the activity is scheduled to start.
End Date	Define an end date and time for the experiment. We can either enter a date in the field, or select a date using the calendar tool. Similarly, we can enter a time directly into the field, or we can select one using the clock tool. If you do not specify an end date, the experiment runs when the activity is scheduled to run, unless you specify a value in the Maximum number of customers field.
Session length	We can specify the amount of time in seconds from when a customer performed the search to the time of their order for the order data to be included in the revenue statistics. The default is 3600 seconds (one hour). If one hour is sufficient for our experiment, we can leave this field empty. If the Frequency of element changes for each customer is set to Session, the session length is also the amount of time during which the customer follows the same path in the experiment. After the session time expires, the customer might be assigned a different path in the experiment.
Status	Select the status of the experiment. We can select either Running or Suspended.

Property	Description
Name	Enter a meaningful name in the field for each path. This name identifies the path in both this tab and the Statistics tab.
Percentage	The Percentage column is visible only if we selected the Random path branch type. Enter the percentage weighting for each path. For example, if you have two paths, A and B, and each is set at 50%, then the server assigns Path A to 50% of customers and Path B to the other 50%. We can assign any percentage to the paths in the experiment, but the total percentage for all paths must be 100%. Typically, you would assign proportionate percentages to multiple paths (for example, 50/50 for two paths 25/25/25/25 for four paths), but this proportionate percentage is not required.
Sequence	This column displays the path's position relative to the other paths in the experiment. This column is hidden by default; to display it, see Use table views.
Unique ID	This column displays the internal database ID for this object; this ID is used for troubleshooting purposes. This column is hidden by default; to display it, see Use table views.