what is unbounce
CATEGORY
DID YOU KNOW WE HAVE A PODCAST?

Become a Better Marketer. Anytime, Anywhere.

Listen and learn on the go with Unbounce’s Call to Action marketing podcast. Tune in and get inspired in the car, while you cook, or at the gym.

Beyond Optimization: Email A/B Tests That Will Improve Your Entire Business

email-tests-blog-image
Those email metrics may provide you with more insight than you thought. Image via Shutterstock.

The components of an A/B test are pretty straightforward: change some stuff, compare key metrics, deploy winner, repeat.

So when you start an A/B test on your email, this is the sort of process you fall back on. You brainstorm a couple of alternate subject lines, test them on a small segment and send the winner to everyone else. This is a great way of making sure you’re sending the better of two ideas, but does it really mean you’re sending better email?

Instead, today we’re going to focus on the benefits of A/B testing for the future. That means turning your results into actionable guidance for feature planning, branding, sales and retention strategies.

ezgif-2245072739
Maximizing is not always optimizing.

Feature planning

It can be really tough figuring out which features need the most attention, not to mention prioritizing improvements your top users would be most excited for. Email can help!

A simple email teasing upcoming improvements to X or Y feature can give you valuable insights for your next product planning session on what changes actually pique a user’s interest.

Similarly, you can test something like, “What would you like to see added to feature X” vs “…feature Y.” Even if you get little to no feedback, the comparative open rates can tell you a lot about which features people want to see updated.

This can be especially insightful for startups, because setting the wrong priorities for your development team can hamstring your growth. In cases like this where the stakes are higher, it may be more powerful to subtly present options and observe responses than to straight up ask.

image01
The problem with asking users what they want directly. Image via Frankiac.

Product branding

What if you’re getting ready to launch a new feature or plan an event, but you’re torn on what to call it. Simply run a test with a sneak peak email to your most engaged users and see what gets their attention.

This one may feel a bit weird, because branding of your product and features can feel really personal, but it’s also really important, so why leave it to your gut when you can test?

You don’t even have to build out a fancy announcement email, because you’re just looking for opens, indicating that initial spark of interest. The body can be a simple, plain text save the date or a link to a survey or something.

Subject line cheat sheet

For the rest of your emails...

Write a click-worthy subject line every time with our Subject Line Cheat Sheet.
By entering your email you'll receive weekly Unbounce Blog updates and other resources to help you become a marketing genius.

Sales materials development

Good email testing can also translate to benefits for your sales team. Imagine their eyes lighting up when you pass them a document illustrating how your highest value customers engage with different phrasings of your core features.

There are a couple of interesting ways to execute on this, but I think the most practical is to build an onboarding email with links to your features, and then test headlines for each section (bonus points if you randomize the order to satisfy the statisticians in the house).

You could also stretch this across multiple emails in your onboarding drip campaign, or send a one-off “What’s new” update.

image02
Life is about decisions. Image source.

Retention

Now that you’ve figured out which features resonate most with your high-touch users, it’s time to figure out what gets people hooked on your product or service in the first place.

There are a ton of ways to accomplish this in the traditional on-site manner, but how does email fit into the picture?

The most obvious option here is to use the information you gleaned to craft a killer onboarding campaign that introduces new users to the most beloved features first. That strategy, however, is really focused on top-of-funnel retention. Today, I want to take a look at the other end: churn prevention.

There are, of course, some users that were never a good fit to begin with and will churn regardless. But for those that just never got the hang of things, the most common move is to hit them with a “Hail Mary email” — one last-ditch effort to win them back.

A lot of times this comes in the form of a direct note from someone asking what they could have done better, but why not use that space to run some tests? Not just to squeeze out a few more opens on a low-converting email, but to see what actually gets people’s attention. Then you can take the stuff that works, and work it into your onboarding campaign to keep people from ever getting to the Hail Mary state.

image03
Saying goodbye is hard. Image source.

Conclusion

These are, of course, not the only ways that you can incorporate your learnings from tests into other aspects of your marketing, but it’s a great start if you don’t have a process like this in place.

The structure you build out to track and share the results from tests like these can be tremendously helpful for the whole team — not just in the ways I’ve outlined above, but also in just keeping everyone on the same page and in line with what your customers want to hear.

Have anything to add? I’d love to hear about it in the comments.

About Alex Mohr
Alex is a coffee roasting, Pacific coasting, jam-on-toasting marketer and email strategy consultant at Sendwithus. He’ll tell you he snowboards, but you shouldn’t believe him, he’s just trying to sound cool. In reality, he mostly just plays Rocket League and makes playlists.
» More blog posts by
  • think email based A/B testing is hard as your open rates (unless you do subject line testing) vary from segment to segment. One segment might have a much higher open rate which makes the content test bias. Helpful guide though !

  • Do not pout animated images, on article. It’s pain to read information, when it’s flashing.

  • Hi! Firstly thanks for this awesome blog. :) After that I’ll definetly try to follow your instruction. Did a great job bro. Keep writing. :)

  • Wow. Thanks for this post. :)

  • Funny memes. :D :D

  • Thanks for the tips. :)

  • Hi Alex,
    Running a successful business is a hell of tasks. Sometimes we fail to keep all the things on mind, as we are no robot. Well, you have just reminded me one vital thing and instructed to get it done as well. I appreciate your effort. Thanks

  • A timely reminder. Thanks!

  • I use Mailchimp. Each campaign will not start without the A/B test :)

  • Thank you for bringing more information to this topic for me. I’m truly grateful and really impressed. Really appreciate for your amazing article. Thank you for this valuable information.

  • A/B testing is vital when it comes to improving on what is already in place. Whether it be you’re not getting the results you would like or would like to change focus there’s always room for improvement/change. Thanks for sharing these great tips Alex!

  • Great Post, I`m using Mail Chimp, the article is very Informative & helpful