We know that targeting the users at the right time and at the right situation to start an experiment is extremely important. Part of an A/B Test setup, you need to target the right area of the site. Whether it is the homepage, all product details pages or your checkout funnel, in case if your targeting is not correct, you will not get the right data against the metrics that you are tracking.

In some cases, the targeting can be more sophisticated. For example, if you are running an experiment on the mini-basket or cart drawer area, you only want to start the experiment when a user opens the mini-basket. Prior to that, the users should not be part of the experiment.

In Convert testing tool, you can use the “JS Targeting Condition”. However, one challenge with this is the time limitation – we have noticed that Convert will wait for 2 minutes for that condition to be fulfilled. Once the period has exceeded, the user will not be bucketed into the experiment even if he fulfills the condition.

Convert JS Targeting Condition Note – stating a 2 minutes cutoff

Going back to the mini-basket example, a user might open the mini-basket after 2 minutes. And as a result, the user will not be bucketed to the experiment even though the condition is fulfilled.

How to resolve this:

Luckily, there is a way and we have found a work around to resolve this issue.

Step 1: Configure Convert Global Project JS

We need to use the Global Project JS and configure the Global Javascript.

Convert – Global Project JS configuration

Step 2: Add a new mutation observer to the Global JS

Add the code below, make the necessary changes as needed:

// listen for targeted element
const targetNode = document.getElementById('element-id); // parent under which the changes occur
const config = { attributes: true, childList: true, subtree: true };
const callback = function(mutationsList, observer) {
       if (!!document.querySelector("target-element")) { // target element or trigger condition
             observer.disconnect();
             window.targetElementTouched = true;
             window._conv_q = _conv_q || [];
             window._conv_q.push(["executeExperiment","experimentId"])
      }
};
const observer = new MutationObserver(callback);
observer.observe(targetNode, config);

Step 3: Add a JS condition in the experiment Audience.

Rename targetElementTouched to the name of the element, e.g. cartDrawerElement.

window.targetElementTouched == true 

Convert JS Targeting Condition Setup

And voila – you have got the right JS targeting condition setup without having to worry about the 2 minutes rule or when the user is performing the necessary action to be bucketed into the experiment!

Summary:

In Convert, JS Condition of an experiment only runs for 2 minutes. Therefore, we cannot trigger the experiment when a user performs the action after 2 minutes of landing on the page. A MutaionObserver is added to the global project javascript which listens for the change and assigns a window variable for it. After that we can execute a particular experiment as shown above with

window._conv_q.push(["executeExperiment","experimentId"]). 

The triggers for the experiment will be checked again in which case it would fulfill the condition and user will be bucketed into the experiment.

Special thanks to Raihan, one of our Software Engineer, for his expertise and insights in shaping the content and ensuring its accuracy.

We know that targeting the users at the right time and at the right situation to start an experiment is extremely important. Part of an A/B Test setup, you need to target the right area of the site. Whether it is the homepage, all product details pages or your checkout funnel, in case if your targeting is not correct, you will not get the right data against the metrics that you are tracking.

In some cases, the targeting can be more sophisticated. For example, if you are running an experiment on the mini-basket or cart drawer area, you only want to start the experiment when a user opens the mini-basket. Prior to that, the users should not be part of the experiment.

In Convert testing tool, you can use the “JS Targeting Condition”. However, one challenge with this is the time limitation – we have noticed that Convert will wait for 2 minutes for that condition to be fulfilled. Once the period has exceeded, the user will not be bucketed into the experiment even if he fulfills the condition.

Convert JS Targeting Condition Note – stating a 2 minutes cutoff

Going back to the mini-basket example, a user might open the mini-basket after 2 minutes. And as a result, the user will not be bucketed to the experiment even though the condition is fulfilled.

How to resolve this:

Luckily, there is a way and we have found a work around to resolve this issue.

Step 1: Configure Convert Global Project JS

We need to use the Global Project JS and configure the Global Javascript.

Convert – Global Project JS configuration

Step 2: Add a new mutation observer to the Global JS

Add the code below, make the necessary changes as needed:

// listen for targeted element
const targetNode = document.getElementById('element-id); // parent under which the changes occur
const config = { attributes: true, childList: true, subtree: true };
const callback = function(mutationsList, observer) {
       if (!!document.querySelector("target-element")) { // target element or trigger condition
             observer.disconnect();
             window.targetElementTouched = true;
             window._conv_q = _conv_q || [];
             window._conv_q.push(["executeExperiment","experimentId"])
      }
};
const observer = new MutationObserver(callback);
observer.observe(targetNode, config);

Step 3: Add a JS condition in the experiment Audience.

Rename targetElementTouched to the name of the element, e.g. cartDrawerElement.

window.targetElementTouched == true 

Convert JS Targeting Condition Setup

And voila – you have got the right JS targeting condition setup without having to worry about the 2 minutes rule or when the user is performing the necessary action to be bucketed into the experiment!

Summary:

In Convert, JS Condition of an experiment only runs for 2 minutes. Therefore, we cannot trigger the experiment when a user performs the action after 2 minutes of landing on the page. A MutaionObserver is added to the global project javascript which listens for the change and assigns a window variable for it. After that we can execute a particular experiment as shown above with

window._conv_q.push(["executeExperiment","experimentId"]). 

The triggers for the experiment will be checked again in which case it would fulfill the condition and user will be bucketed into the experiment.

Special thanks to Raihan, one of our Software Engineer, for his expertise and insights in shaping the content and ensuring its accuracy.

One of the core things that create a lot of back and forth between the developers and product owners is that the information needed about a test that needs to be developed is simply not there up front. Even though you are well versed on what the hypothesis is and what you want to test, because of the missing information, the development can be stalled. We have seen some tests that are supposed to be delivered in three days’ time, took over a week as some of the key information was missing from the original test brief.

Here are the 5 most important things that the A/B testing developers need to develop and deliver the test in the most efficient way:

1. Variation designs

Obviously! And that includes what changes need to be made, any functionalities that need to be added and / or how the changes are reflected across different devices

2. Metrics

Unless you are going to track the metrics by yourself – specify the primary KPI and all other metrics you want to track. There could be custom metrics that the A/B testing developer need to implement as part of the test development across all variations (including control!)

3. Targeting pages / screens

Where is this going to be live? Homepage? Product Listing Pages? Product Details pages? What is the URL structure? If it is a sitewide test – is it truly a sitewide? What about the checkout funnel (which can be structured differently from the rest of the site)?

4. Audiences

Who will be part of this experiment? What are the audience criteria? What about visitors coming from PPC campaigns? Are there any restrictions?

5. Edge cases / Anomalies

If a product is out of stock or on sale, the product details page is displayed differently. How is that going to be reflected for the variation? It is very easy to miss the edge cases – but for the test developer, it is important to ensure that they are covered up front

The rule is simple – the clearer the information is up front, the less time spent on unnecessary back and forth!

One of the core things that create a lot of back and forth between the developers and product owners is that the information needed about a test that needs to be developed is simply not there up front. Even though you are well versed on what the hypothesis is and what you want to test, because of the missing information, the development can be stalled. We have seen some tests that are supposed to be delivered in three days’ time, took over a week as some of the key information was missing from the original test brief.

Here are the 5 most important things that the A/B testing developers need to develop and deliver the test in the most efficient way:

1. Variation designs

Obviously! And that includes what changes need to be made, any functionalities that need to be added and / or how the changes are reflected across different devices

2. Metrics

Unless you are going to track the metrics by yourself – specify the primary KPI and all other metrics you want to track. There could be custom metrics that the A/B testing developer need to implement as part of the test development across all variations (including control!)

3. Targeting pages / screens

Where is this going to be live? Homepage? Product Listing Pages? Product Details pages? What is the URL structure? If it is a sitewide test – is it truly a sitewide? What about the checkout funnel (which can be structured differently from the rest of the site)?

4. Audiences

Who will be part of this experiment? What are the audience criteria? What about visitors coming from PPC campaigns? Are there any restrictions?

5. Edge cases / Anomalies

If a product is out of stock or on sale, the product details page is displayed differently. How is that going to be reflected for the variation? It is very easy to miss the edge cases – but for the test developer, it is important to ensure that they are covered up front

The rule is simple – the clearer the information is up front, the less time spent on unnecessary back and forth!