Are you looking to experiment with different versions of your website or app? You may have heard of A/B testing, or split testing, to carry out these experiments. A/B testing delivers two different versions of your website or app to your audience to determine which is the better version, with one half of your audience receiving version A and the other half receiving version B. But let’s break away from this model and start looking at multivariate testing.

Leading experimentation platform provider Optimizely, define multivariate testing as “a technique for testing a hypothesis in which multiple variables are modified”. Think of this as A/B/n testing. Where A and B stand for version A and version B in your standard A/B test, the n stands for the number of these versions being tested.

The goal of multivariate testing

The outcome of multivariate testing is determining which is the ideal combination of all the possible combinations tested.

A visual example of multivariate testing can be seen below.

In the example above you can see the experiment is testing different combinations of the headline and image to determine which is more successful. Having more variations allows you to see how they interact with each other, giving you more information on what factors are positive or negative to your visitors and ultimate conversion goal. In fact by testing 5 or more variations, you can improve your win rate by 50-60%.

Positives

This advanced way of testing means you don’t need to carry out multiple A/B tests on the same page to reach your desired conversion goal. You can instead run a number of variations simultaneously in a reduced time frame.

Negatives

As you can imagine, the number of variations that you can test can multiply quickly! This results in more visitor traffic being needed to achieve results of statistical significance as a lower amount of traffic is being split and sent to each variation.

What to test to improve conversions

Experimenting with more variables allows you to see how they interact with each other, giving you more information on what factors will help you reach your conversion goal.

So which test should you choose?

When trying to choose if you should run a multivariate test, decide if the variations being tested are going to have a quantitative effect on the conversion goal. If you think it’s possible only one factor may contribute to the conversion goal, opt for running this as an A/B test. If you think it’s possible multiple factors may contribute to the conversion goal, and you have the necessary visitor traffic, opt for a multivariate test.