Main Principles and Benefit of A/B Testing

If somebody wants to have a successful business on the Internet, it is important to know about user actions on his/her product. This refers to conversion.

People use special SEO tools to monitor this information. As a rule, owners of big Internet resources pay much attention to this process because even a little change on a website can significantly influence the conversion.

In this case, program users are divided into two or more groups. Each of them will see a particular page or its details in different ways. One will see an original version, another one – an updated one.

So, statistical data allows us to know if any changes benefit the resource or they just do harm to it without a significant loss of customers. This method is called A/B testing.

A/B testing (bucket tests, Split URL testing) is a special kind of experiment on a web product or mobile program. The essence of this method is that product users are divided into several groups and they see a different display of the particular part of this product.

The point is to check the potential capabilities of resource optimization comparing with the original version of the site. For example, this method can be used to analyze how colors, sizes, and locations of the resource’s elements influence its conversion.

In general, A/B testing includes testing of two web components on the A and B pages in order to see where will be more clicks (CTR) in percentage terms. In most cases, such components are logos, emails, banners, CTA buttons, and other CTA for keeping the dialogue with a user.

Why Do People Use A/B Testing?

1. It Helps to Determine the Most Advantageous Design of the Site or Software

Alteration of design can be different: from a minimal redesign of the colors of buttons or some text to complete upgrading. Before you deploy one or another product version, there should be a good reason for that. You can get this reason only after A/B testing.

It will be wrong to stop testing after finishing the design changes. While the site is updating, it will be better to test other web elements. In this way, one can be sure that users see the most interesting version with the correct design.

2. It Contributes to the Detection of Visitor “Pain Points”

The concept of “pain points” means an incorrect order of actions during the payment process. It may lead to numerous cancellations of purchase in an online store or low interest in media.

Every site visitor has its own purpose: to buy something, find some information or have a consultation. Regardless of this purpose, users face common pain points: distractive titles, no button for 1-click ordering, subtle forms that are difficult to understand.

In order to minimize the effect of all these factors, we need to use data that we get after an analysis of user behavior. Mostly this is website surveys, Google Analytics, heatmaps.

Moreover, this applies to the products of any field: commercial industry, educational sphere, media, or tourism.

3. It Identifies More Useful Improvements (Based on Statistical Data)

Since A/B testing is based on data only (not on the hypothesis or a good intuition), one can easily identify the best and the worst product. Specialists come to this conclusion after analyzing the statistical data of such characteristics as time spent on the site, a number of requests and times when users leave the cart without a purchase, click rate, and so on.

4. It Decreases the Failure Rate

It is significantly important to monitor the failure rate in order to ensure the correctness of the site performance estimation.

The failure rate is a web analytical concept that shows a ratio of a number of users who left the site after visiting the home page or those who visited only one page.

There are many reasons for such failures: a wide selection of goods, misalignment between expectations and reality, etc. Since every web resource has its own purpose and is designed for different groups of customers, there is no way to accurately determine the reasons for failures.

One of the options to figure it out is A/B testing. Also, it’s possible to check several possible product versions and choose the best one. This will lead to the improvement of user experience and decrease the failure rate.

5. It Helps to Make Low-Risk Changes

With the help of A/B testing, it is possible to get an updated web page instead of redeveloping it from the very beginning. This allows reducing the risk to corrupt real statistical data of conversion.

A/B testing is a method that allows using the resources in the most effective way and making minimum changes. Hence, this improves the profitability of investments.

A great example in such a case is editing goods description. We can execute the A/B test when we need to edit or delete the description. We cannot know for sure how users react to this. But with the help of A/B testing, we will get preliminary estimates of this change.

Another example of low-risk changes is the deployment of a new function. Before you add any new functionality to the product, it’s good to run it as the A/B test on the website copy. Thus, you’ll see the results beforehand. These actions will be effective only when modifications influence over the information about the client or over the purchase funnel.

The purchase funnel is a marketing model that describes the possible customer journey on the site from the moment of his/her first visit to the accomplishing the purchase.

It will be the wrong decision to make some changes without testing. It may leave no mark or lead to losses otherwise. After the testing of some changes, there’s a bigger chance to get more qualitative results.

Benefit of АВ testing

Structure elements of А/В testing

А/В testing has 5 stages. Let’s analyze them in more detail.

Stage 1 – Collection of Statistics

Before starting to plan A/B testing, it is necessary to examine the website performance at the moment. We need to know the number of visitors, names of pages with the most traffic, reasons for conversion of different pages, and so on.

For this purpose, the best analytical tools for web resources are Google Analytics, Omniture, Mixpanel. They can help to determine the most visited pages, pages with the biggest number of failures, or the most time-consuming pages.

For example, let’s take the page with medium earnings potential or the page with the heaviest traffic. Also, heatmaps will be very useful in this case. These tools visualize user actions on the site and help to determine vulnerabilities on a website.

Moreover, consumer surveys contribute to a better understanding of web product problems. It is possible to use this testing tool with a development team or with a user directly. Also, consumer surveys can find omissions that weren’t found during general testing.

Stage 2 – Making a Hypothesis About Possible Changes

When you record test results and make an assumption about conversion increasing that is based on received data, it is significantly important to take account of business aims. Tools that you use at this stage will be useful only during the collection of data about user actions.

If you want all the information to be used correctly, you should analyze and monitor it, and after that make a resource with new data for customers. It leads to the hypothesis based on the received data.

Stage 3 – Making a Version Based on the Previous Hypothesis

The next stage in the A/B testing is the development of a version that is based on previously made hypotheses. Version is a new variant of the page with changes that need to be tested.

You can perform testing of several modifications with control in order to choose the best of them. It will be better to create a version based on a hypothesis about functioning components (according to UX).

For example, statistics say that a lot of users don’t fill the form. Is it because of a huge number of fields? Is personal data mandatory? Could it be better to use a version with few fields or a version where one can leave fields empty (if they require any personal data)?

Stage 4 – Testing

The next stage is the process of A/B testing. But there are some similar methods in marketing: A/B testing, multivariate testing, multi-page testing, and split URL testing.

Let’s analyze them in more detail.

Multivariate testing (MVT) is a method when specialists change some parts of the product and create versions for all the available sets. During this kind of testing, it is possible to execute all the test suites on one test.

Also, this test allows finding the most important component of the website. Let’s take a look at the following example. You have to test two pictures of a character and the color of the CTA button on the site.

With the help of MVT, we can create a combination for the picture of the character and a combination of the button color. In order to test all the versions, it is necessary to create versions of all the possible modifications.

In the common A/B testing, both pages are located at the same address. If we talk about split URL testing, the B version has a completely different location.

Multi-page testing is a process when specialists test the website changes on different pages. As a rule, testers take all the purchase funnels and make their new versions (that are supposed to be better). And then QA engineers test these versions. Also, this kind of testing allows checking the way how some changes (like adding or deleting some components such as reviews or safety signs) will influence the conversion.

When you perform this kind of testing, you shouldn’t forget about various factors. In order to prevent the deviation in statistics, visitors must be from the same segment without the capability to influence other ones.

You have to prevent the influence of other factors such as days of the week, the weather, advertisement, workers of the call-center, work of programmers or administration.

It is possible to solve the first issue by measuring segments simultaneously. The second one can be solved with the help of Google Analytics filters.

The most important thing in such a kind of testing is realizing that you cannot jump at conclusions here. Results change every day. Meaning, values mostly contain random data. It will take some time to identify mean value and similize them.

Stage 5 – Analysis of the Results and Deployment of a Better Version

Despite the fact, this is the last stage of determining the best version, it is significantly important to analyze the results. This stage has a main purpose since A/B testing requires constant collecting and analysis of information.

After the test run, we need to analyze the results considering the confidence level, percent increase, direct and indirect influence on other elements, and so on. After analyzing this information (and if testing was successful), we can deploy the best version. If we don’t get a definite variant, it will be better to analyze data one more time and use it in other tests.

To Sum Up

А/В testing is a method of checking and deployment of the best versions on the website. Undoubtedly, A/B tests aren’t always a good option, since it is very difficult to predict user actions and requests. But with these tests, we can check the most extraordinary hypothesis and versions without considerable losses.