Over the years I kept on preaching about making sure that your traffic is exposed to only one test at any given time. This is to ensure the quality of the test result data, making sure that the winners you are getting from each test are true winners without diluting the data.

This is great when you are gradually getting yourself into the habit of testing. However, when we are talking about implementing a culture of experimentation, constantly running tests to improve customer experience and conversion, this method of testing, unfortunately, fails to achieve that.

As a result, no matter how much you would like to implement a culture of experimentation, you are bound to frustrate your team due to the lack of velocity of running tests. To overcome this, we are introducing a method called Simultaneous Divisional Optimisation (SDO).

What is Simultaneous Divisional Optimisation:

An Experimentation programme that allows you to run multiple tests on any given website by breaking the site down into smaller areas and treating each area as the separate, individual website.

Each area or site section will have its own primary metric to improve via the CRO program. Metrics such as sales or revenue will be tracked to ensure that these don’t have a detrimental impact on the tests running on individual site areas. The winner will be declared based on the performance of the selected primary metric.

How does Simultaneous Divisional Optimisation work:

The best way to understand how SDO works, let’s take a fashion retail business as an example.

If we consider the user journey on any fashion retail site, it typically falls into three core categories:

1. Browse and find
2. Research and decide
3. Transact / Complete purchase

Browse and find

Users who are in this state of the journey are typically looking for the product that they need/want. Pages such as Homepage, Category Landing pages, and Category Listing Pages (or product listing pages) usually support this journey for the users.

Your aim here is to optimise the customer experience by making it easy for the users to find the right product for themselves. Whether to highlight relevant products via the homepage, entice them with the latest collections or give them an easy way to find the category of products they are looking for.

The success or the primary metric potentially at this stage of the user journey is to take the user to one or more product details pages.

Research and decide

Once the user finds a product that they are looking for, they want to find out more about the product. This includes materials, fit, reviews and pricing. The ultimate goal at this stage of the user journey is to add the product to the shopping basket, with a secondary metric of adding it to the wish list.

Optimisation ideas here are potentially to provide the right information to the user in the right manner. Additionally, for the fashion retail website, you might want to help the user by giving them alternatives or “wear it with” to potentially “shop the look”. You can also consider having social messaging such as how many visitors purchased this product, how many visitors have added this product to the basket, and how many visitors are looking at the same product at the same time to show the popularity of the product.

Primary metric is for users to satisfy their needs of finding all the necessary information and add this product to their basket.

Transact / Complete purchase

The final step of the user journey is to complete their purchase. They have done their research, they have added the products they like into their basket, and now this is where they need to go through the process of the transaction. Yes – here the primary metric is indeed the sales. Your optimisation ideas on the basket and checkout funnel are to focus on getting the user to the ‘Thank you’ page. That is the main aim and you need to continuously improve the experience to move users from the basket page to complete the transaction.

Multiple tests in different sections

Now let’s bring all three sections together – in essence, you have now got three site sections that you can optimise independently as the aim of optimising each section is now different from one another. When you are running a test on the Category Listings or Product Listings Pages, your aim to get the user to one or more product details page. Similarly, when running a test on the product details page, your aim is to make the user to add the product to their basket. And finally, when you are improving the checkout funnel, your main aim is for visitors to complete the purchase.

What about other types of websites?

You can use the same theory on pretty much any transactional site. The sections will, of course, be different – for example, if you are working with a travel site, your ‘Browse and find’ section is potentially the search results page of the holiday/flight. The primary metric for optimisation here would be to send the users to a holiday page. Once the users are on an individual holiday page, then the primary aim is to start the booking process.

In a similar manner, you can divide one website into multiple sub-sites with a clear primary objective to be achieved from those sub-sites. This way, you can simultaneously optimise the customer experience for each of the subsections in parallel.

How do you analyse the test results of SDO?

You need to make sure that your primary metric for each site section has been accurately defined. If you find a winner (or loser) based on the primary metric without negatively impacting the final business goals, you can make the decision based on the result.

For any organisation, no matter what the final business goal is, you can always break things down into smaller goals. The SDO simply provides a way to optimise the smaller goals with the aim to ultimately optimise the final business goal.

Taking things further with SDO

What if you would like to launch multiple tests at the same time within the same section? You can still do that with the following two options:

1. Mutually exclude traffic within that site section
2. Create variations – one for each hypothesis and a final one with the combination of both hypotheses together

This way, you will be able to make sure that you are getting clear results for your tests running within the same section.

In summary:

SDO provides a way to run multiple experiments at the same time with the aim to continuously improve the customer experience of individual site sections. We have applied this method in a couple of businesses where the traffic amount is over a million per month. This resulted in an increase of test velocity by 10X with a significant revenue impact to the overall business in comparison to running just one test at a time.

SDO allows the organisation to implement an experimentation culture by dividing one website into multiple sites. It engages product owners from different sections to be involved and independently improve their site areas. This method of experimentation utilises the resource more efficiently to get the best out of the CRO program.