The A-Z of an Effective A/B Testing: Everything You Need to Know

wordpress-split-testing1

The A-Z of an Effective A/B Testing: Everything You Need to Know

Conversion rate optimization involves capturing, convincing and converting prospects into customers.

To do this effectively, your web pages, marketing emails, designs, CTAs etc., must be well optimized to engage users at the highest level.

But, how do you know if your web tools are engaging enough? How do you know if you’re not leaving money on the table due to an element on your landing page turning visitors down? How do you know if even your email subject line is sexy enough to get you high open rate?

Certainly not with just your brains; not by being an expert in your field, per say; not even with the most carefully conducted customer survey.

Taking any of those routes in your quest to increase conversion and user engagement won’t get you exactly what you want.

Let us cut the chase …

Meet A/B testing!

That’s the core of conversion rate optimization.

Conversion methods used for improving conversion

Through A/B testing:
You can get to know what works best with your website users;
You can get to see which variation of a web tool performs better;
and what matters the most –
You can boost your conversion rate,
increase revenue;
AND achieve your business goal fast.

So, let’s get into this; let’s get to know what A/B testing is all about.

What’s A/B Testing?

In many instances, A/B testing is often used as a general term for online experimentation, however, it actually refers to the process of testing binary versions A and B of an item – mostly a web page, to see which one works better in terms of conversion.

Some campaigns may test more than two versions. When that’s the case, it’s called A/B/n testing.

Split testing is a science

We can conclude that A/B testing – also referred to as split testing, is meant to display different variations of what is being developed, to compare and evaluate which variant performs best.

This can cover things such as websites, apps, landing pages, email marketing, and even advertisement.

A good example of A/B testing is when two different versions of a landing page are displayed with the goal of getting visitors to buy a given product, where you send an equal percentage of your total number of visitors within a given period of time to each page.

Version A might have a different headline and different CTA button color from version B. Version B on the other hand may have an image, unlike version A, probably more content and so on.

Next, you track the user behavior and their responses to each page to see which page does better in getting users to take the required action.

 

Ab Testing

 

How A/B Testing Has Evolved

A/B testing has changed over the years. But, not only has it changed in itself, it has also affected the way people operate online.

A few years back, it was just a handful of companies that actually practiced testing their stuff online and they did that either through internal IT set-ups or via a handful of companies offering very limited testing options.

And back then, debates on things like partial versus full factorial testing, data waves, efficiency and iterative testing were common.

There were no much segmentation built into tools and for the most part, tools required a bit more knowledge of how to work with different groups.

A big disadvantage of online testing was cost: The cost of testing tools was high for smaller companies or private individuals to afford.

But today, you have more than enough tools to carry out testing as you want.

And the cost?

That has gone down too, significantly. Nowadays, virtually everyone can test things online. This has gone on to improve the online environment generally.

But, how do you get started?

How to Get Started with A/B Testing

Getting started with Ab testing

Pretty much anybody with a considerable online experience can get started with A/B testing. To start A/B testing, you first have to establish your goals and choose what to test (more on that later).

After setting your goals, you’d need two versions of what you want to test— a control (usually the original i.e A), and a modified version (version B).

You’d also need A/B testing tools to carry out the testing.

When to A/B Test

When to AB test

 

Generally, you can test just any time you want (it’s a free world after all), as long as you have enough visitors to split.

However, there are some specific scenarios that may require testing more.

Here are some possible ones:

1. When redesigning your website. Sad to say, website designs can reduce conversion rates to rubble. For a website designer, redesigning a site is just part of a typical workday. For you, a redesign could mean dead to your CRO and SEO. So, after redesigning, split test. Things to keep your eyes on are button color, button text, positioning of elements, text color, etc.

2. When you change a feature, service, or plug-in. Once in a while, web admins do change things on their website. When you do, remember to test. This sort of testing is specifically important when the change affects actions you want users to take.

3. When you change prices. Changing a price is sometimes seen as a risk because price change can either boost or blow your revenue. With that in mind, it’s important you A/B test the two different prices first before the implementation.

4. When the number of your website visitors increases. Maybe the last time you split tested, you had just a little over 5,000 unique monthly visitors. Now, the number has gone up to, say, 35,000+. To get the best out of the increased number of visitors, you would have to A/B test to see what works best with them.

5. When you want more. As an entrepreneur, you should strive for more—more revenue, more sales, more visitors, more conversion, you name it. When that’s the case, you really don’t have to ignore A/B testing; well, because A/B testing can really get you more.

Things You Can Split Test

Do you go on split testing just anything and everything you find on your website? No.

Should you bother selecting the things to test? Emphatically yes.

Testing without direction will get you little or no result. But knowing exactly what to test?

That can drive your conversion by 336% or more.

That said, here’s a quick guide to help you in selecting what to test:

1. Headlines. Different A/B testing case studies show that conversion flared up with different (and better) headlines.

2. Value propositions. In a previous article, we talked about the importance of testing your value propositions.

3. Email subject lines. You want as much people as possible to open your email, right? The major deciding factor of email open rate is the subject line.

Email Ab test

4. Calls to Action (CTAs). From CTA text to text color, from CTA button shape to button color, from CTA button size to placement, you’d need to test every element of the CTA to see what converts better.

5. Product pricing. How will they respond to your new prices?

6. Layout. Which email templates will users have a preference for?

7. Opt-in incentives. Which incentive will make them give you their real name and a valid email ID without you asking twice?

8. Product descriptions. So, you think long product descriptions will do better than short descriptions here, huh? Well, your site visitors may not think so. Test to see what they think.

9. Fonts. You’d be surprised at how fonts affect conversion.

How font effects conversions

10. Promotional offers. Making an offer is not enough, test different elements (ex. phrasings) in your promotional offers to find which entices users the most.

11. Product photos and their placement on a web page, especially an eCommerce product page.

12. Content length. Will visitors respond better to long-form content or are they busy folks who prefer short-length content. The content here can be on a sales page, marketing email, landing page, etc.

Shares by Content Length

 

13. Timing of upsells. When do they respond better and faster? Right after buying, or just before?

14. Social proof. On your sales page, will client testimonials work better than user reviews? Probably, but you never know until you test.

15. Forms. Don’t know about you but CAPTCHA turns me down. Plus, long forms may not be everybody’s fave. So, how many boxes are too many? Where should the form be placed? Or, is a pop-up form better?

16. Elements on a landing page. This can be your landing page image, placement of content etc.

17. Apps. Which design does better? What about the UI? And the downloading methods?

How Do You Know You’re Testing the Right Things?

Well, it all depends on what your goals are.

For instance, a B2B company might want to generate more sales for their product. With that, they need to test things like their sales pages.

BOOK-PRODUCT-IMAGE

On the other hand, an online publisher may want to increase their blog subscriber, which means they’d be testing things like their sign-up form fields, opt-in boxes etc.

So, the question really should be: What do you want to increase? What do you want people to do?

After you have the answers to those questions, test the things that make people do those things.

Split Testing a Website vs. Split Testing Apps?

Split Test , Ab Test

Among the things you can test are websites and apps. But, is there a difference between A/B testing on a website vs. A/B testing in apps?

To answer that simply. . .

. . .yes, there is.

The major difference between the two is that A/B testing an app is technically more taxing than testing a website; and the reason is that an app would need to be downloaded and run locally on a mobile device instead of being accessed via a live connection with a browser.

This makes it more difficult to track and test out new variations in apps, whereas on a website, you can do that with a breeze.

A/B Testing for Digital Publishers and Electronic Content Providers

If you publish content online, happy are you!

Online content publishers can gain a lot from A/B testing.

Ab testing for Digital Publishers and Electronic content providers

Precisely, you can use A/B testing to discover what works best with your readers and generally improve your click-through rates and readership.

You can do this by testing different headlines, content format, font, text size, images, background color and more.

As an online publisher, especially if you rely on cost-per-click based ads, you can use A/B testing to improve your conversion rate.

For instance, you can test and compare different ad units. It may surprise you but small changes in ad units can result in positive changes in click-through rates.

You can test things like ad placement, ad types (image vs. text), size, location and theme (text color, background color etc.).

A/B Testing Tools

A/B testing requires the use of web tools. There are quite a number of A/B testing tools on the interwebs, but let’s take a look at a few useful one here:

Google Analytics

 

Google Analytics logo

Google Analytics is a multi-purpose, Google-owned web tool. With the tool, you can split test different variations of a web page. Google calls this experiment.

To create an experiment, click on “Behavior” and there you’ll find “Experiments.” And then click on “Create experiment.”

Google Analytics Create

The beautiful part is that this tool is free and readily available for every website owner to use.

 

Optimizely

Optimizely split test tool

Optimizely is a tool with split, multivariate, and mobile testing options. The tool provides a WYSIWYG interface which allows users to make dynamic, real-time changes to websites on the fly. Results can be viewed in real-time and visitors can be segmented by elements like traffic source, campaign, browser, and device type (e.g. smartphone, tablet or desktop).

Optimizely’s Price: Starting at $17/month

 

Visual Web Optimizer

Visual Website Optimizer

 

Visual Website Optimizer is a testing and optimization tool for creating A/B tests and geo-behavioral targeting campaigns.

Just like Optimizely, VWO also provides with WYSIWYG editor, with which users can easily make changes to their website and create multiple variations they can A/B test.

Additionally, VWO collects analytics data for every interaction visitors initiate and generates reports from there to help webmasters understand their audience.

VWO’s Price: Starting at $49/month
TruConversion

This list of A/B testing tools wouldn’t be complete without the home team. But, TruConversion is more than just an A/B testing tool—it’s the future of CRO.

Truconversion

TruConversion is a suite of tools that give users access to all the best conversion optimization and analytics tools including Feedback polls, Heatmaps, Surveys, Conversion funnels, Form Analytics, Visitor Recordings and, yeah, A/B Testing.

This mean with TruConversion, you don’t have to go through the hassle of using multiple expensive tools, but will be able to truly understand your visitors, A/B test, and improve website conversions all on the same platform. Try it FREE here.

 

A/B Testing Examples

A/B testing work. Here are some great A/B testing that worked and how you the same reproduce the same results (or even more) for your business:

1. Meebox saw 121% increase in revenue through A/B testing

 

Meebox

Meebox offers domains and hosting services to individuals and business in Denmark

They A/B tested their Pricing page in which the Control had no discount while the Variation had discounts of 20% (40% for the highest plan) if customers signed up upfront for 2 years.

Here’s the original pricing page with information such as cost per month, space, bandwidth and domains.

Meebox 2

And here’s the variation, also with basic information:

Meebox ab test

Turned out Meebox saw:

  • 56% increase in revenue(98% chance to beat original)
  • 24% increase in Average Order Value(AOV)
  • 85% increase in conversions(customers/visitors-to-page)

Source: https://vwo.com/blog/ab-testing-price-discounts/

 

2. Emphasizing benefits on landing page led to 65% increase in revenue

A company changed their landing page copy to point up benefits of their products instead of the product features.

The final result showed they achieved increased sales by 65% with variation B.

Below are the pages tested:

Original:

landing page

Variation A:

Variation A

Variation B (Winner):

Variation B

SOURCE: https://www.conversioner.com/blog/neuromarketing-hacks

 

3. Obama raised $500+ million through A/B testing

 

Ab test

This whole talk about A/B testing won’t be complete without mentioning the Obama’s A/B testing campaign.

The Obama team used A/B testing to raise fund and gain new followers during the 2012 elections; they tested several combinations of visuals, placement and CTAs.

They tested everything from every email they sent out…

hey-5

…to several versions of their donate page. And the results they got were impressive.

Here are different versions of the pages they tested:

 

img_55aeb0ee4f563

img_55aeb03f58be8

img_55aeb146845d4

img_55adde7d4194b

 

img_55adde8973735

img_55adde9db85aa

 

img_55aeaf1e9eba6

img_55aeafc7e7b38

 

Your Turn

Ever A/B tested a thing? What was the process like, and what were the results you got? Share with us in the comments section below. What’s more, if you have any question concerning A/B testing and CRO in generally, feel free to drop them below as well and we will be glad to get back to you.

Mike is a Digital Marketing professional and currently working as a PR manager at #TruConversion.com He truly loves Online marketing and anything digital. He is very focused about who he works for. Once he starts work, he is fully committed to his duties and responsibilities at that company.

2 Comment

  1. Georgi says: Reply

    Hi Mike,

    A nice article, but why no mention of statistical design and evaluation? All the good advice you give can go to waste if the test has a poor design or uses statistical methods that are incompatible with the actual use case (like simple statistical significance tests, only suitable for fixed-sample designs).

    Any measurement has an uncertainty attached to it and not mentioning it doesn’t make it disappear 😉 For example, it would be good to have the uncertainty measures and some statistical design info on the 3 examples of successful A/B tests you provide.

    Best,
    Georgi

  2. There’s a dead link here as well.

    The link (http://contentverve.com/10-call-to-action-case-studies-examples-from-button-tests/) is going to a 404 page.

    Dead anchor: (test every element of the CTA)

    Looks like the article is no longer online. Assuming you want to fix the link, I’ve got a pretty extensive guide I’d like to submit as a replacement.

    URL: (https://www.izideo.com/blog/10-call-to-action-case-studies-examples-from-real-button-tests/)
    Title: (10 Call to Action Case Studies with Examples from Real Button Tests)

    I am also happy to re-push the link to my Twitter and email list.

    Have a nice day!

Leave a Reply to Georgi Cancel reply