What Makes a Conversion Optimizer? 8 Experts on Separating the Pros from the Pretenders

What makes someone a genuine conversion optimizer? What should you be looking for?

She will look at the big picture of the business. Optimization comes from a process-oriented mindset (a systematic approach as opposed to a single tactics). The first thing she will do is set up a process for conducting CRO — that includes putting in place ongoing conversion research plans, setting policies for testing (prioritization, stopping rules, etc.) and so on.

Genuine optimizers are always willing to change their minds — they don’t claim to “know” what works. Instead they focus on finding out what the problems are before coming up with solutions.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

The red flag is overconfidence — an “I know what works” attitude. Focusing on little random details like colors, CTAs, etc.

They will mumble only about statistical significance when asked “What’s your stopping rule for tests?” rather than talking about sample sizes and business cycles. They don’t discuss exploration versus exploitation with you, and the need for one before the other. They promise results too good to be true.

What makes someone a genuine conversion optimizer? What should you be looking for?

The right mindset is imperative. You need the ability to cut through the clutter, define a problem and determine the best way forward. So being an adept critical thinker with a strategic approach to problem solving is a prerequisite. You also need to be bold enough to admit when you’re wrong. Stubbornly clinging to cherished notions is the bane of the CRO.

Moreover, you need a wide variety of skills. Minimum requirements include a firm grasp of statistics, web analytics, psychology, usability, design principles, the scientific approach, qualitative/quantitative research and business sense.

Real CRO is a highly complex discipline and you need a ton of skills to do it right. Unfortunately, it is usually dumbed down to a banal process of running as many A/B tests as possible. A misconception facilitated by the endless amounts of misinformed blog posts written by amateurs or theorists with zero hands-on experience practicing CRO.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Ask the candidate if they can describe their CRO process. If they hesitate, or seem confused by the question, show them to the door. A solid, proven process is what separates the pros from the amateurs. Another red flag is if they don’t mention the word “research” at all. This is an indication that they are going to base important decisions on their own opinions rather than research and real customer insight

Also, ask them if a 95% significance level is enough to guarantee valid test data. If they say yes, show them to the door. The person clearly does not understand stats, and that is a huge red flag due to the fact that stats are crucial to collecting and interpreting numerical data (a huge part of CRO).

What makes someone a genuine conversion optimizer? What should you be looking for?

Contrary to other fields in marketing (e.g., copywriting, PPC or SEO), conversion optimization requires an optimizer to have hands-on experience in every aspect of marketing: analytics, UX, psychology, copy, design, emotional targeting, business development, data and much more. It’s a position that requires the ability to understand every aspect of the business, its customers and analyze not just certain parts of a funnel but the entire online experience.

Skills and techniques can be taught, however a passionate, dedicated attitude and a desire to continuously learn, grow and drive the company forward, are rare and very hard to come by. These are the true indicators of a successful and dedicated optimizer.

Another important trait to look for is agility — someone who is able to learn, adapt and move fast. An optimizer must be able to inspire teams, be customer driven, allocate resources, constantly analyze numbers, collect feedback and change quickly as new data and research keeps coming in.

When hiring, ask for references, ask for case studies, ask to speak to other members of the team and not just the salesperson who might know the “keywords.” Consider adding milestones to your contract (i.e., specific, time-based goals) for evaluation.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Avoid…

Agencies and individuals that “do it all” (e.g., PPC, SEO, Inbound and “Oh and we’ve just added a conversion optimization department”). These types of companies or individuals are the ones that added CRO to their resume because everyone else is doing it and probably don’t have the exact expertise you’re looking for.

Companies or individuals who promise certain uplifts or huge increases in conversion rates before getting started or even looking at your site.

Those who immediately tell you what’s wrong with your funnel or landing page before looking into data.

Anyone who doesn’t have a dedicated process for analysis, observation and execution. You want to work with someone who understands the role of CRO in a business and has a set process that you can follow and learn from.

What makes someone a genuine conversion optimizer? What should you be looking for?

Great optimizers come in many flavors and need to be judged on their results.

They definitely need experience in personalization, A/B testing, behavioural design, cognitive biases, persuasive copywriting, web technology, statistics, design of experiments (DOE), project management, etc.

But, that’s not enough. You only know they are genuine if they deliver consistent business results. The challenge is reliably proving their results.

At WiderFunnel, we’ve solved this with a rigorous selection process. Each new member who completes our intensive hiring process starts as a coordinator until they prove their results delivery. Their thinking, values and temperament are tested through our weekly intensive group LIFT Zone meetings. Only the best become WiderFunnel Strategists, while others may move into other specializations that suit their skills. Using this meritocracy ensures a consistent team of A-players.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

When you’re hiring an optimization agency (or any agency, really), you’re hiring for credibility. You need questions to validate their credibility. Here are a few to start with:

• Experience: How long has the agency been fully dedicated to conversion optimization?
• Focus: Do agency revenues come primarily from conversion optimization, A/B testing and personalization services?
• Results: How many case studies of relevant work has the agency published?
• Process: How thorough is the agency’s process specific to optimization and A/B testing?
• Expertise: Do your agency contacts have stated perspectives on advanced topics like design of experiments, statistics, and behavioural personalization?
• Team depth: Do they have a cross-functional team of experts with deep individual expertise, cross-training each other to continuously learn?
• Thought-leadership: How many thought-leading articles, blogs, books, webinars and conference presentations have they produced?

What makes someone a genuine conversion optimizer? What should you be looking for?

Skepticism is critical — you should never be satisfied and should always look for problems and areas needing improvement. Your main concern should be to help the website visitor, and align their experience with the goals of the company.

You also need to be strategic in your approach and work with empowered executives within the company to get meaningful stuff done beyond just fiddling with campaigns and landing pages. This includes not just the web experience, but may also impact the marketing technology stack, business model, email communications, content creation, company organization and culture, staff training and back-end workflows. Only then will you be a true partner in growing the client’s online business.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Many fake CROs think that simply knowing how to use a split testing tool and running tests makes them an expert.

In fact, an over-reliance on testing as the only tactic to produce results shows that they are not comfortable making substantive changes to the larger business without the crutch of having every little change validated along the way.

You should also watch out for any agency claiming to have proprietary frameworks, formulas or guaranteed methods to produce results. The fact is that every business is different, and we invest enormous time and energy to understand the particular and evolving needs of each clients on an ongoing basis.

You can’t paint by numbers, and one size does not fit all.

What makes someone a genuine conversion optimizer? What should you be looking for?

The number one thing that makes a conversion rate optimizer is curiosity. Second, rigor.

Look for the CRO who collects questions and is skeptical of answers, but is always looking for data points to guide them.

They must be great with people, because a CRO is a data storyteller; we can teach them the data science, we can mentor them on turning questions into data, but we can’t make someone curious (and you certainly can’t teach someone how to gently bring others into our world of rigorous creativity).

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

If someone says they know how to make a website convert better, that’s a red flag. I’d rather have someone who says they know how to find out what makes a website convert better. Any CRO that has all the answers has not been properly humbled by the process.

Look for curiosity and empathy. If they haven’t paid for a course in the past 12 months (on any topic); if they can’t tell you the most influential books they’ve read (on any topic); if they haven’t spent time serving people in previous roles, they may not be a good CRO.

The conversion scientists who are making our clients the most money were a librarian, a wedding planner at a flower shop and a salesperson before we hired them. They are all formidably curious and good with people.

What makes someone a genuine conversion optimizer? What should you be looking for?

Look for three core capabilities:

1. Stats: The expert should know about the statistics behind online experiments (what kind of experiments to set up, giving the data available in the company).
2. Insight into psychology and behavior: The expert should know about the basis of human behavior (knowing which persuasion tactic could lead to what behavior).
3. Soft skills: The expert should have organizational soft skills (knowing when to promote what about online experiments in the company to get things done).

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Ask them the following three questions:

1. How many conversions do you test on (ask for old tests)? If you often see numbers below 1,000 conversions… don’t hire them. The other trick is to ask them if they know about Power Levels and if they calculate this up front (should be 80% or higher). This is the biggest problem with experts or agencies pretending to be awesome optimizers.
2. How do you decide what to test? If they pick a page first and then hypothesize what to optimize: dump them. You should hypothesize first (based on a proper study) and then decide where to experiment with this hypothesis.
3. How do you create the A/B-tests that were designed? If they tell you they just use the drag and drop editor from A/B-software vendors, dump then. For most good optimizations you need to write you own lines of code.

What makes someone a genuine conversion optimizer? What should you be looking for?

Curiosity is the most important trait. Great optimizers always question the status quo to improve things. They seek to understand the bigger picture, recognize patterns and don’t give up until they understand the whole thing. They are a perfect symbiosis between qualitative (“understand”) and quantitative (“measure”) skills.

You also have to differentiate between hiring a freelancer or agency and an employee. An external hire should be trusted by management to report the results and get buy in based on that data. Often the problem is that external optimizers don’t get the access to data that they need to deliver appropriate and trustworthy results.

Same applies for an internal hire, but they have mostly no trust problem. The main issue here is the missing external point of view of internal teams. Internal optimizers have to rely on feedback from outside the company — “you can’t read the label from inside the bottle” — but they suffer from a Dunning-Krüger-Effect: they don’t know what they don’t know.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Unexperienced CRO agency people spread fake uplifts and multiply them with revenue to create a fake impact that is not valid. Mostly these uplifts are too high because tests are stopped too early. So always ask for test duration, sample size and traffic sources. Unexperienced inhouse people only test small changes or elements because they did not realize yet, that they have to influence user behavior with high contrast. They also don’t know how to report the real ROI to the C-suite.