6 Proven Methods for Selling A/B Testing to a Skeptical Client

By , September 12th, 2012 in A/B Testing | 4 comments

While SEO, SEM, Web Analytics, and Usability have become commonly established as cornerstones of any successful online business, A/B testing remains a discipline shrouded in mystery and misunderstanding.

The fact that few companies understand what it really is, and how they can benefit from it, leads to a high level of skepticism and anxiety that makes A/B testing a tough sell.

This article will give you 6 tested and proven methods for mitigating anxiety and selling A/B testing to a skeptical client.

1. Do your homework

The first step in building a relationship with a skeptical client is to understand who you are selling to and how your service will help them solve their specific problems.

Learn as much about the client as possible. Make sure you are familiar with their business model, products, selling points, website, conversion goals, marketing channels, and target audience.

Also, make sure to research the competition. Who are the main competitors? What are their products like? What are their main marketing channels online? What do their websites look like? What are their strengths and weaknesses?

Moreover, if you’re already in dialogue with the client, ask them to describe their main conversion related challenges, as well as the dream conversion scenario. That will help you make a more relevant case for A/B testing.

With all these facts straight, you’ll have a clear idea of who the client is, the specific challenges they are facing, and how A/B testing will help them get through these challenges.

2. Use case studies to establish credibility and expertise

In my experience, case studies that illustrate before and after results from real split tests are the most powerful way of selling A/B split testing.

Show your client case studies from your back catalog and explain step-by-step what went into the test. Describe the underlying test hypothesis, the changes you made, the challenges you faced, the results you achieved, and not least what you learned from the test.

Don’t be a show-off

With a skeptical client, the goal is to build up credibility and establish yourself as a trustworthy expert. If you only show your best and most elaborate case studies, you could come across boastful and arrogant – that is not the intention!

Show them cases where your hypotheses were off, cases where the results were surprising, as well as really simple case studies that illustrate how small changes can affect conversions.

I like to start out with a basic single-variable case study e.g. a call-to-action test where the treatment generated a relatively large lift in conversions. This makes it very easy to understand the underlying principles and value of A/B testing.

Then I move on to a “blockbuster” case study where I knocked the ball out of the park and achieved massive lifts. This is to establish the “wow effect” and showcase the dream scenario where conversions went through the roof.

After that I show a case where my hypothesis was way off, and where the treatment performed worse than the control. This is to make the point that A/B testing is not a miracle cure that magically increases conversions. Also showing a case study where I was wrong helps establish my credibility and builds trust.

Last but not least, I walk the client through case study where the initial treatment underperformed, but where we ended up with a winner by analyzing the test data and conducting follow-up experiments. This is to illustrate how even negative test results can lead to a winning variant if you understand how to work with the test data.

Displaying a wide variety of case studies, and being open about results and findings, will go a long way in establishing you as an expert and a true conversion scientist – not a show-off.

If you haven’t put together any case studies yet – get going ASAP!

3. Present solid arguments that help them understand the benefits of A/B testing

Click image for full size and check out the full collection of Conversion Insights.

In my experience the single best argument for getting into split testing is the fact that it the only way to gain certainty that your optimization efforts are working, and that they are optimizing your business.

If you aren’t testing, you’re relying on gut instinct and guess work – and that’s not the world’s safest strategy to build your business on.

Most managers understand this argument right away, but are often still skeptical.

This is where case studies become your ally. Here are a couple of great resources to find them:

  • Visual Website Optimizer: A great list of case studies.
  • Marketing Experiments: This blog has many examples of successful tests.
  • Which Test Won: Publishes a large number if A/B test case studies, and because they make a game out of picking the winner, you might be able to surprise your boss by making them test their intuition.
  • Conversion Skills: Similarly, WiderFunnel have created a new testing game where you try to guess the best performer – and then get presented with a case study at the end.

4. Be completely transparent

A view of the Unbounce A/B Test Centre as seen by a Viewer (Read-Only User) – typically your client or stakeholder.

Give the client full insight into all data and results – no matter what the outcome of the test might be.

Clients aren’t used to marketers being completely open about data and numbers.

Being completely transparent with a client builds a lot of trust and mitigates a lot of the anxiety the skeptical client is inclined to feel.

Moreover, it shows the client that you are confident enough to be held accountable for your recommendations, and that you have an extra incentive to achieve results.

5. Be prepared to answer all possible questions your client might have

A/B testing is a very complicated discipline with a million ins and outs. A skeptical client who is completely new to the subject is bound to have a ton of questions to ask you.

Common questions include:

  • “How much traffic do we need to conclude an A/B test?”
  • “How will we know when to stop the test?”
  • “How long should the test run?”
  • “Will A/B testing impact SEO?”
  • “What should we test?”

You are the expert and the better you are at answering questions, the more professional you will seem, and the more confident the client will be that you are the right partner to move forward with.

6. Make getting started as easy and painless as possible

One of the main things that keeps businesses from getting into A/B testing is the fear of having to invest in a huge elaborate technical platform, involving tons of coding and IT assistance.

Being the person who can help them get started painlessly, without involving IT too much, will dramatically increase your chances of landing the client.

If you use tools like Unbounce.com or Visualwebsiteoptimizer.com, you’ll be able to get started in no time. You will only need to involve IT to the extent that they have to implement a bit of Javascript on the website or perform a simple DNS tweak.

Start small and get your footing

Once the technical setup is in place, you still want to keep it simple. Approaching a skeptical client with a huge pervasive experiment involving major changes will probably scare them off.

Start with a simple test that is likely to give yield results. You could pick the newsletter signup form, the call-to-action on the home page, or the headline and sales copy on the main landing page.

This is a great approach for getting a footing. As soon as you show the client some real results on their own website that they can relate too, they’ll be much more inclined to let you perform a larger experiment.


Time to win over some A/B testing clients!

These tips are from my personal experience, and I have used them time and again to win over even the most skeptical clients. Try them out for yourself, and remember: don’t be the “snake oil salesman” your potential clients expect to meet. Break the mold, sell them a solution that actually works, and exceed their expectations – then you’ll have a client for life!

– Michael Aagaard


This is a guest post, all opinions are those of the author.

Michael Lykke Aagaard is a self-employed, self-confessed split test junkie and copywriting fanatic who’s obsessed with finding out what really works in online marketing. He’s Danish and hails from the fair city of Copenhagen. You can follow him on Twitter and Google+ or check out his newly launched international blog ContentVerve.com.

Comments

  1. Robert K says:

    Whenever I introduce a client to optimisation I take them through the concepts as follows:

    -What optimisation is
    -How leverage works for making more sales by enhancing the conversion rate just a tiny amount
    -How I approach optimisation (the process of understanding their business, developing ideas, testing etc)

    Once I’ve covered that, I’ll usually say to them, “Sounds easy doesn’t it?” Then I take them through a quiz asking them which case studies lifted conversion rates. They usually get a few wrong and I’ll say, “It’s fine to get them wrong – all the big guys fail sometimes. But that’s why you test.”

    In fact, I found some figures around test success rates for AMZN, MSFT and some other companies. Well worth checking out. Can also be used as a yard stick for your own testing…

    http://www.optimisationbeacon.com/conversion-optimisation/not-running-tests-is-dangerous-conversion-rate/

  2. Hi Robert – thanks for the comment!

    The quizz is also a great way of getting through to a skeptical client. I often use that one as well.

    I show two variations and ask which one they think converted best. Afterwards I ask them how big a lift in percent they think the variant they picked generated.

    That always makes for some “Aha” moments ;-)

    – Michael

  3. Chris Marsh says:

    Great post, enjoyed your resources listed in point 3 and in particular the A/B ShareThis case study from AMD (http://visualwebsiteoptimizer.com/split-testing-blog/amd-3600-social-sharing-increase/) – Bizarrely VWO haven’t implemented the strategy themselves!?

    Id love to know split test results also comparing for when the share icons remain in a static position when scrolling (see here:http://www.webseoanalytics.com/blog/8-ways-to-increase-the-relative-traffic-of-your-site/) – this makes them very prominent, but could it be too intrusive and distracting/visually displeasing, and put the readers off an article/engaging with the share?