A/B Testing

A/B Testing, also known as split-run testing or bucket test, is an experiment where 2 or more variants of the same thing are compared against each other to see which one performs better. The audience is randomly presented with two versions after which ab testing statistics are used to judge their performance.

Now that you know the ab testing definition, let’s understand it with an example.

Orderhive’s Experiment with A/B Testing

We decided to carry out ab testing in marketing with our e-mailers, for our then upcoming webinar. Our marketing team took upon A/B Testing in two ways. First, content-wise and second, title-wise. In the first attempt, we decided to test our email subject lines. Version A was “Only 50 seats left! Hurry and register for our webinar.” Version B said “ Our webinar is fast-filling! Haven’t you registered yet?”

The results of the test revealed that the Version A performed better with a CTR of 54% compared to the 43% CTR of Version B. We Inferred that Version A probably did better because we mentioned facts and figures that provided greater clarity to people.

The second email campaign was approached in two ways. First set of emails consisted of only text (simple email) while Version B consisted of images and graphics(HTML). Here, Version B brought greater results with a CTR of 61% compared to Version A.

Through these experiments with A/B Testing, team Orderhive was able to understand the kind of email content which resonated most with customers.

Now that you’ve read about our experiment with A/B Testing, in the subsequent sections, you can read the entire process in detail, that helped us formulate our A/B Testing strategy.

How To Find What to Test?

You can test many things through A/B Testing but the most important part is singling out which element you should put out for testing, especially something which is more likely to have an impact.

For instance, say you wish to test your website out. Elements that usually impact user behavior are-

• Headline
• Call to Action
• Graphics
• Content

While formulating an email campaign, you can do go for ab testing emails, where you might want to focus on the copy and call to action button. You can test similar ideas as well, for example, offering a discount in version A and a giveaway in version B. But for that you need to test both ideas with both groups of audiences.

How Does A/B Testing Work?

In A/B Testing, the current version(A) of the variable and a slightly modified one(B) is taken. The current version (control) is shown to part of the users and the second version( variation) is presented to the rest. The results of A/B testing are then found on the basis of audience engagement with the variable they were shown. The changes in user engagement are recorded and analyzed through statistical methods, in order to determine if the change received a positive, negative or neutral response and which version can achieve a specific conversion goal.

Let’s take an example.

You have created two versions(A and B) of your e-commerce site(Variable). Version A is the current or control version while version B is the modified one. The change in version B could be something as simple as a modified headline, button, content, call to action or page design. Now half the users will be shown A while the other half will be shown B. The responses of the audience towards both versions will be recorded and analyzed to gauge the impact of the change on user behavior and engagement.

The A/B Testing Process

• Identify Problem Areas- Data on the different elements of your website will give you the high and low performing players. You can then identify pain points, say, for instance, pages with high bounce rate, and begin strategizing on optimization.  For example, if you want to test your emails, you can choose which problem you want to focus on either your email content or headline.
• Set Conversion Goals- Goals become the criteria based on which you will understand whether the original version or modified version garners a greater response from the target audience. Goals could be getting signups, answering surveys or clicking CTAs. If you’re testing the email subject line, then your goal should be focussing on open rates and if it’s the sign up button, then it should be the click rate.
• Ideation- After deciding on your goal, you can start brainstorming ideas for creating version B. Analyse the reason for the change on the basis of factors like relevance, potential impact, difficulty and cost of implementation. In order to get a few ideas for variations you can use a few tools specifically designed for A/B Testing like Google Optimize, Kissmetrics, Unbounce and others.
• Create Variations- Select an idea and create variations for A/B testing. For more accurate modifications, you will need an A/B testing software. Alterations can be changing the color of the background, font, header placement, etc. you can create 2 variants of say, a button, with a colour change to see which one attracts more clicks.
• Run the Test- Once the variation is ready, you start A/B Testing with the variation A and B by randomly assigning them to visitors. Measure and analyze visitor interaction for each variation, analyze results to see how they perform.

Why is A/B Testing important?

A/B Testing enables businesses to understand user behavior with each change while simultaneously collecting data on the results.

The dynamic changes in user engagement patterns allow them to create a hypothesis about how and why certain elements when altered impact user reactions.

An important reason A/B Testing is useful is that it eliminates any guesswork in trying while optimizing website development and instead facilitates improvement based on hard-core statistics.

Benefits of A/B Testing

A/B Testing provides numerous advantages that significantly help to improve the performance of your website.

Higher user engagement- different elements of the website are A/B tested to see which have an impact on user engagement. When positive changes are incorporated into the final design, it increases user engagement and successfully optimizes the website for better results. A win-win for both!

Spike in Conversion Rates- when two versions are tested with visitors, you understand which attracts more interactions from the audience. When the site is designed according to the preferences of visitors, it obviously makes it easy to convert them into buyers.

Lower Bounce Rates- A/B Testing can expose the weaker areas of your website design and user response to changes. Finding the perfect combination of elements for your website can help visitors spend a greater amount of time on it, thus, drastically reducing bounce rates.

Reduced Cart Abandonment- it is a fairly common phenomenon is e-commerce when customers don’t complete the checkout process and exit your site without making a purchase. This is called cart abandonment. With the help of A/B Testing, the customer experience from the checkout page to entering delivery address can be optimized to the fullest by identifying the loopholes in the process.

Better Content Management- With A/B testing, you can test website content, email, ad copy and see whether they are generating the desired amount of engagement or not. After identifying areas that are lacking, you can tweak content in the way which accumulates captures better interactions from users.

Summing up

Every business wants more and more visitors to interact with their website and ultimately take action. The more visitors act (conversion) the greater the amount of traffic generated for the site. But understanding what makes visitors tick is not that easy, especially if your ideas are pure assumptions and not based on solid facts and figures. A/B testing removes the possibility of any guesswork and helps businesses in identifying their weaknesses and turning them into strengths backed by statistics.