A/B Targeting: Beyond A/B Testing

Since the first days of the World Wide Web, companies have been adopting it​ as a sales and user recruitment platform. As the space has been getting more competitive, companies have progressively found both more difficult and more expensive to bring visitors to their sites. As a result, companies have been increasingly using conversion optimization techniques as a way of leveraging their visitor acquisition efforts and investment, in an effort to obtain a larger amount of sales and users from an equivalent initial set of visitors.

A/B Testing has been a very commonly employed optimization technique for many years. While using A/B Testing is certainly better than doing no conversion optimization at all, it is non optimal in significant ways. Recently, continuous advances in computing power and Machine Learning have finally allowed the development of a superior approach: A/B Targeting.

This document exposes A/B Targeting advantages over A/B Testing, and how, ​while A/B Testing can help you increase conversion rates, A/B Targeting helps you to maximize them, thus maximizing your revenue by visitor and ultimately your company profits​.

A/B Testing Use Case Overview

For the sake of comparison, let’s first explore a typical A/B Testing use case scenario.

A company sells a product and, to that end, they have set up an online advertising campaign directing to a landing page. The original landing page features an image of the product, a headline, some of the product details and a call to action.

Image-based variation

As the company is setting up an A/B Test, an alternative design is prepared. Instead of the product image, this alternative design features a product explanation video.

Video-based variation

The test is left to run, showing a random variation to each visitor, until a statistically significant result is achieved. The results are, let’s say, 11% and 10% conversion rates for the image­based and video­based variations, respectively. As the image­based variation had the best performance, it’s declared as a winner and kept. Meanwhile, the video­based variation is discarded. Barring a new test being started, future visitors will only be shown the image­based design.

Beyond A/B Testing

It could seem that, aside from creating and testing extra variations, there is no room for further optimization, as all visitors are being shown the variation that had the higher global conversion rate. That would be true if visitors were a homogeneous group and global conversion rates for each of the variations was the only information we could get.

Let’s break down conversion rates of each variation by some visitor feature. For example, by the time of the day when the visitors arrived at the site:

Conversion by time of the day

While the image­based variation conversion rates peaks around mid­day, the video­based variation conversion rates strongly descend around the start of typical office time, then soar again when offices close, likely because visitors are less likely to watch a video while in the office.

We have seen that conversion rates vary throughout the day. More importantly, we have seen that depending on the time of the day (and time of the day is only one example of multiple traits that visitors have) the optimal variation to show to a visitor varies. In other words, ​the optimal variation to show to a visitor is not a constant, but a function that has the visitor as an input​.

The corollary of the A/B Test was to keep the image­based variation, thus leaving ourselves with a level of conversion rates defined by the dark blue line. When doing A/B Targeting, we show to each visitor the variation that maximize their conversion rate, thus increasing our final conversion rate by the amount indicated by the red area in the following graph:

Increased conversion rates

Conclusion

We have seen that visitors to a site do not form a homogeneous group. Each of them is different, behaving and responding to stimuli differently. To maximize our conversion rates, we must cater to them as specifically as possible.

For that purpose, we can categorize each visitor according to many different criteria, including the one illustrated on the example (time of the day) – but also the day of the week and the month, visitor location (city & country), visitor language, device attributes (browser, operating system, etc), browsing referrer, among others.

All of this allows us to achieve a global conversion rate higher than what any given variation can provide​. Ultimately this means that conversion optimization stops chasing the silver bullet that is a site or landing page design that works optimally for all visitors. Instead of that, it prepares a set of targeted designs and let an A/B Targeting tool optimally present them to visitors.

Insightware is an easy to use A/B Targeting tool.
Sign up for a free trial and start maximizing your conversion rates now.

Comments !