Since the first days of the World Wide Web, companies have been adopting it as a sales and user recruitment platform. As the space has been getting more competitive, companies have progressively found both more difficult and more expensive to bring visitors to their sites. As a result, companies have been increasingly using conversion optimization techniques as a way of leveraging their visitor acquisition efforts and investment, in an effort to obtain a larger amount of sales and users from an equivalent initial set of visitors.
A/B Testing has been a very commonly employed optimization technique for many years. While using A/B Testing is certainly better than doing no conversion optimization at all, it is non optimal in significant ways. Recently, continuous advances in computing power and Machine Learning have finally allowed the development of a superior approach: A/B Targeting.
This document exposes A/B Targeting advantages over A/B Testing, and how, while A/B Testing can help you increase conversion rates, A/B Targeting helps you to maximize them, thus maximizing your revenue by visitor and ultimately your company profits.
A/B Testing Use Case Overview
For the sake of comparison, let’s first explore a typical A/B Testing use case scenario.
A company sells a product and, to that end, they have set up an online advertising campaign directing to a landing page. The original landing page features an image of the product, a headline, some of the product details and a call to action.
As the company is setting up an A/B Test, an alternative design is prepared. Instead of the product image, this alternative design features a product explanation video.
The test is left to run, showing a random variation to each visitor, until a statistically significant result is achieved. The results are, let’s say, 11% and 10% conversion rates for the imagebased and videobased variations, respectively. As the imagebased variation had the best performance, it’s declared as a winner and kept. Meanwhile, the videobased variation is discarded. Barring a new test being started, future visitors will only be shown the imagebased design.
Beyond A/B Testing
It could seem that, aside from creating and testing extra variations, there is no room for further optimization, as all visitors are being shown the variation that had the higher global conversion rate. That would be true if visitors were a homogeneous group and global conversion rates for each of the variations was the only information we could get.
Let’s break down conversion rates of each variation by some visitor feature. For example, by the time of the day when the visitors arrived at the site:
While the imagebased variation conversion rates peaks around midday, the videobased variation conversion rates strongly descend around the start of typical office time, then soar again when offices close, likely because visitors are less likely to watch a video while in the office.
We have seen that conversion rates vary throughout the day. More importantly, we have seen that depending on the time of the day (and time of the day is only one example of multiple traits that visitors have) the optimal variation to show to a visitor varies. In other words, the optimal variation to show to a visitor is not a constant, but a function that has the visitor as an input.
The corollary of the A/B Test was to keep the imagebased variation, thus leaving ourselves with a level of conversion rates defined by the dark blue line. When doing A/B Targeting, we show to each visitor the variation that maximize their conversion rate, thus increasing our final conversion rate by the amount indicated by the red area in the following graph:
We have seen that visitors to a site do not form a homogeneous group. Each of them is different, behaving and responding to stimuli differently. To maximize our conversion rates, we must cater to them as specifically as possible.
For that purpose, we can categorize each visitor according to many different criteria, including the one illustrated on the example (time of the day) – but also the day of the week and the month, visitor location (city & country), visitor language, device attributes (browser, operating system, etc), browsing referrer, among others.
All of this allows us to achieve a global conversion rate higher than what any given variation can provide. Ultimately this means that conversion optimization stops chasing the silver bullet that is a site or landing page design that works optimally for all visitors. Instead of that, it prepares a set of targeted designs and let an A/B Targeting tool optimally present them to visitors.