what is unbounce

Become a Better Marketer. Anytime, Anywhere.

Listen and learn on the go with Unbounce’s Call to Action marketing podcast. Tune in and get inspired in the car, while you cook, or at the gym.

A Simple A/B Test that Generated a 31.54% Lift in Sign-ups [Case Study]

conversion lift
Learn how to get a conversion lift using A/B testing. (Image source)

You’re probably used to reading posts that run through a number of A/B test examples. But how often do you get the chance to read an article that gives you an in-depth look at a single case study?

Today I’m going give you a detailed look at a simple A/B test that increased sign-ups on a betting forum by 31.54%. The point is to give a deeper understanding of what goes into conducting a valuable split test that provides you with both insights and lifts.

Background information

Client: Bettingexpert.com – a free betting community where tipsters from all over the world can share betting tips, tools, data, and much more
Page: Home Page Bettingexpert.com
Optimization goal: Increase number of signups/new members

a/b testing example

Page Diagnosis

From experience, I know that it’s not always the biggest changes on the page that result in the biggest lifts. Making strategic changes to mission critical elements will often result in dramatic improvements. Instead of focusing on how much the change impacts the page itself, I’ve learned to focus on how much the change will impact the decisions of potential customers.

So when BettingExpert.com approached me and asked if I could help them increase the number of sign-ups via their home page, I immediately honed in on the most mission critical and prominent page element – the sign-up form itself.

Both the design and placement of the form work well. So at first, I didn’t see any particular need to make any changes here. The form copy however is another story, and I saw a lot of potential for improvement.

At first glance, the original from copy is pretty decent and does an okay job of clarifying the purpose of the form. The header says: “Join BettingExpert” and the button copy says: “Sign Up+”

However, the form copy doesn’t convey any value whatsoever and does nothing to answer the question: “Why should I fill out this form and give you my email?”

In other words, the form copy lacks both relevance and value – two of the main things to focus on if you want to write web copy that converts.

Form copy in A/B test

A/B Test Hypothesis and Treatment A

When I created the treatment, I focused on increasing the relevance and value communicated by the header and button copy. In other words – I focused on answering the question “Why should I fill out this form?”

The main – and most tangible – benefit of becoming a member of BettingExpert.com is that you can get free betting tips from top tipsters on a daily basis.

Based on experience from similar tests, I hypothesized that I could accelerate the decision-making process of the prospects and increase signups by focusing the form copy on the main benefit.

In my treatment the header says: “Get FREE Betting Tips” and the button copy says: “Sign Up & Get the Best Daily Tips”

New form copy for a/b test

A clear A/B test hypothesis is absolutely essential if you want to conduct clean and valuable A/B tests that provide you with both insights and lifts. To learn more about how to create a clear test hypothesis read this article.

A/B Test Setup

In order to find out whether my hypothesis would hold water and increase conversions, I set up a simple A/B test with the Control Variant (A) and my Treatment (B).

There are dozens of different tools you can use to conduct split tests, but in my opinion there is no reason to make your test setup more complicated than absolutely necessary. For experiments such as this I prefer to use tools that allow me to create variations and set up tests easily without having to mess around with code.

Check out this video to see how to set up a split test in 3 minutes with no knowledge of code.

Test Results and Validation

I ran the test for 9 days and reached a sample size of 13.560 visitors, and 291 conversions. The standard error was 0%, and the statistical confidence was 99%. From the beginning to the end, the treatment outperformed the control variant – at no point did the control take the lead. The Treatment increased signups by 31.54%.

a/b test control vs. treatment

Test validation and statistics are some of the less sexy aspects of testing – nevertheless, they are extremely important because there really is no point in testing, if you can’t rely on your tests results.

When validating your test results there are 3 basic factors that you NEED to look at:

  1. Confidence Level
  2. Conversion range
  3. Sample size

The big problem for most marketers is that they either pay no attention to these three factors, or focus only on one of them.

But you really need to be aware of all three factors in order to perform valid experiments that provide true and lasting value to your online business.

Check out this 10-minute video and get a basic introduction to test validation and determining the statistical significance of a split test:

What I learned

This test did more along the lines of confirming what I’d learned from previous experiments than actually opening my eyes to something new. But here’s a rundown of the main takeaways of the test:

The copy you use in your signup form has direct and measurable effect on your conversion rate. The tweaks described in this article may seem insignificant from a usability or design stance, nonetheless, they directly influenced the decisions of the prospects and generated a serious lift in conversions.

When you ask prospects to do something online, you will automatically start an internal dialogue in their heads. They need to assess whether the potential benefits of what you’re offering them outweigh what they have to do – or part with – in order to get it.

In connection with mission critical elements like sign-up forms, it’s important to clarify the value of your proposition so you can give your prospects a good reason to carry out the conversion goal.

Follow-up Experiments

Just because you’ve conducted one successful experiment doesn’t mean that you’ve found the best solution or that you should stop experimenting. In fact, I usually use the first test as a sort of “warm-up,” setting the scene for more experiments. Moreover, the first test will give you some initial insight and an idea of how much it takes to influence the actions of your prospects.

In this case, I followed up with an experiment where I tweaked both the headline and the form copy – as opposed to just the form copy. This simple exercise in clarifying value and adding relevance resulted in an additional 3.65% increase in sign-ups.

more a/b test experiments

I’m by no means done experimenting with this page, and I’ve conducted several other follow-up experiments that include button size, button color, and a different value prop.

All right, so this was a closer look at some of the formalities that go into conducting a valuable and clean A/B test. Now all you have to do is get started on your own website – 1, 2, 3- GO!

— Michael Aagaard

About Michael Aagaard
Michael Aagaard is the Senior Conversion Optimization Consultant at ContentVerve. When he’s not preaching the CRO gospel as a popular international speaker, he spends his time helping clients improve conversion rates in wonderful Copenhagen. Follow him on Google+, Twitter or get his new free ebook: 7 Universal Conversion Optimization Principles.
» More blog posts by


  1. Tomas

    Unfortunately I read your test manytimes before, try ty show some new exemples.

    • Oli Gardner

      I think the purpose here Tomas was to did deeper into how the whole test was run. I know I found it insightful. Do you have some good examples you can share? Would love to read them.

    • Michael Lykke Aagaard

      Hi Tomas – Thanks for your feedback.

      I’ve referred to this case study in two previous posts on ContentVerve.com. But this is the first time I’ve used it on Unbounce.com. So “many times” seems like a bit of an overstatement.

      Oli is right, the idea here was to dive deeper into how the whole test was run, instead of just showing it as an example.


      – Michael

  2. Gary Bury

    Out of interest, if you go to their website now they are not using your version of the signup form.

    I’m not doubting you results but if it produced such a large increase in signups why have they not adopted it permanently, were the sigups lower quality?

    • Peter

      Hi Gary,

      bettingexpert.com is one of our sites and I am contact person to Michael. We are very happy and excited about Michaels work. We will adopt the winners permanently when all tests are finished.

      Soon we will do a redesign and will use all the information we get from Michaels testings for making bettingexpert.com even better.


  3. Michael Lykke Aagaard

    Hi Gary – I’m still running tests on the form ;-)

    So you must have been among the visitors that are segmented to see the Control. Right now I’m testing button color.

    When I’ve found the ultimate combination of copy and design, we’ll implement it permanently on the website.

    – Michael

  4. SupportoServer

    Hello Michael, thanks for sharing your case study. I’m wondering if you know a WP plugin for implement a nice form like yours.

  5. Michael Lykke Aagaard

    As a matter of fact I do ;-) I use http://hybrid-connect.com/
    for my newsletter form. It’s excellent and now features split testing.

    – Michael

  6. Lia

    Thank you for your post Michael. I especially think your point about follow-up experiments is important. Don’t stop with that first one, keep on testing!

  7. Marketing Day: February 12, 2013

    […] A Simple A/B Test that Generated a 31.54% Lift in Sign-ups [Case Study], unbounce.com […]

  8. Will Hull, MPA

    I love all of the posts on here about landing page design. This is really helpful as a nonprofit professional seeking ways to enhance supporter conversion to donors. Thanks for putting this out there.

  9. The ABCs of Mobile A/B Testing - Instant.ly - How to Improve Your Mobile Conversation Rates | Instant.ly

    […] We’ve talked extensively about A/B testing. Just for a quick recap, A/B testing is a process where you compare two versions of a webpage, email, survey, or other piece of collateral to see which one performs or converts better. Most of the discussions that we’ve seen look at A/B testing from the perspective of traditional websites. Optimizing your site for mobile has more to do with making sure that your content renders properly across a range of devices. There’s also a big opportunity to attract more customers through this important medium. Here’s our quick guide to mobile A/B testing. […]

  10. A Complete Beginner’s Guide To A/B Testing

    […] diagnose the customer’s behaviors through both mails’ response rate. You can check out this useful A/B case study by uBounce which increased signup by […]


    […] az infógazdag slide komoly kutatásra és megalapozott érvelésre enged következtetni. Egy webdesign-os kísérletben kiderült, hogy már néhány extra szó is jelentős, esetünkben 32%-os emelkedést okozhat a […]

  12. Supercharging Email Signups for Travel Sites: The Incentive

    […] boosted signups 31.54% by offering free betting tips, as opposed to just asking visitors to […]

  13. ABCs of Copywriting: Digital Copy

    […] With no printed materials to pay for, A/B Testing becomes a lot less expensive to implement on digital copy. Be prepared to test multiple treatments of your call to action to find which ones result in more conversions. For an example, check out this article from Unbounce. […]

  14. Optimizing the Free Trial Signup - How Flow Got a 17% Lift [Case Study]

    […] one profile of converted users, you can then work to improve the lead gen process and implement A/B tests on signup forms and processes to: 1) attract and onboard more true evaluators, and 2) convert a higher number of […]

  15. Opal

    Some basic things you need to do during installation are: setup the name of your proxy, create the user
    accounts and passwords and assign the IPs to each account.
    A sales or consumption tax is more likely to support a culture of savings and not
    act as a block to individual effort. But there are some universal points to seem out for if you’re hunting for this kind of proxy servers.

  16. Why You Shouldn’t Neglect Your Form Copy · Formstack Blog

    […] giving your form copy the attention it deserves! After all, tweaking the text on your form can increase submissions by more than 31%! How? Here are 3 compelling […]

  17. A Complete Beginner’s Guide To A/B Testing - E Tips Blog

    […] diagnose the customer’s behaviors through both mails’ response rate. You can check out this useful A/B case study by uBounce which increased signup by […]

  18. 4 Indicative Situations When Your Site Needs A/B Testing - Tweak Your Biz

    […] but rather on what the customer should want. To give you an example, on BettingExperts they tried two different tests, one where they said what they wanted, ‘join BettingExperts’ while the other said what the […]

  19. online lotto results

    Hello, constantly i used to check web site posts here in the early hours in the morning, as i love to find out more and more.