What Makes a Conversion Optimizer? 8 Experts on Separating the Pros from the Pretenders

psychologist female patient male sympathy family relationships emotions mental problems pop art retro style
Oh my! That is quite a confession! Image via Shutterstock.

I have a confession to make. It’s a hard one, so breathe deep with me and please be kind.

I am not *hard swallow* a conversion optimizer.

That’s a dangerous admission, and not just because CRO is big business.

You see, while I’m not a conversion optimizer, I’m good at faking it. Really, really good.

During 2016 alone, I wrote two number-one-ranked Google articles for the phrases “conversion rate optimization principles” and “wrong with conversion rate optimization.” According to Buzzsumo, I’ve got the number-one-most-shared article for “landing page optimization” and the number-one-most-shared article for “CRO marketing.” And just to add insult to CRO injury, I also hold the number-one Google spot for the phrase “optimize online copy.”

It’s not that those posts aren’t helpful or based on data and best practices. It’s that I’ve only tried my hand at CRO three times, and each attempt went down in conversion-rate-tanking flames. I’m a content marketer and it’s high time I stopped pretending.

Why should you care about my catharsis?

Because if I’m that good at faking it online, the CRO “expert” you’re about to hire could be too.

For many companies — especially those with deep pockets — CRO is a seductive mystery: One part tantalizing promise, one part “I don’t understand it, so I’ll just trust you.” Unfortunately, that’s a recipe for disaster.

To save you from that fate, I reached out to eight conversion rate optimization experts with the track records to back that title up and asked them two questions:

  1. What makes someone a genuine conversion optimizer?
  2. How can you spot a fake CRO expert or agency?

Peep Laja:
Principal at CXL Institute

What makes someone a genuine conversion optimizer? What should you be looking for?

She will look at the big picture of the business. Optimization comes from a process-oriented mindset (a systematic approach as opposed to a single tactics). The first thing she will do is set up a process for conducting CRO — that includes putting in place ongoing conversion research plans, setting policies for testing (prioritization, stopping rules, etc.) and so on.

Genuine optimizers are always willing to change their minds — they don’t claim to “know” what works. Instead they focus on finding out what the problems are before coming up with solutions.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

The red flag is overconfidence — an “I know what works” attitude. Focusing on little random details like colors, CTAs, etc.

They will mumble only about statistical significance when asked “What’s your stopping rule for tests?” rather than talking about sample sizes and business cycles. They don’t discuss exploration versus exploitation with you, and the need for one before the other. They promise results too good to be true.

Michael Aagaard:
Senior Conversion Optimizer at Unbounce

What makes someone a genuine conversion optimizer? What should you be looking for?

The right mindset is imperative. You need the ability to cut through the clutter, define a problem and determine the best way forward. So being an adept critical thinker with a strategic approach to problem solving is a prerequisite. You also need to be bold enough to admit when you’re wrong. Stubbornly clinging to cherished notions is the bane of the CRO.

Moreover, you need a wide variety of skills. Minimum requirements include a firm grasp of statistics, web analytics, psychology, usability, design principles, the scientific approach, qualitative/quantitative research and business sense.

Real CRO is a highly complex discipline and you need a ton of skills to do it right. Unfortunately, it is usually dumbed down to a banal process of running as many A/B tests as possible. A misconception facilitated by the endless amounts of misinformed blog posts written by amateurs or theorists with zero hands-on experience practicing CRO.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Ask the candidate if they can describe their CRO process. If they hesitate, or seem confused by the question, show them to the door. A solid, proven process is what separates the pros from the amateurs. Another red flag is if they don’t mention the word “research” at all. This is an indication that they are going to base important decisions on their own opinions rather than research and real customer insight

Also, ask them if a 95% significance level is enough to guarantee valid test data. If they say yes, show them to the door. The person clearly does not understand stats, and that is a huge red flag due to the fact that stats are crucial to collecting and interpreting numerical data (a huge part of CRO).


Talia Wolf:
Founder & Chief Optimizer at GetUplift

What makes someone a genuine conversion optimizer? What should you be looking for?

Contrary to other fields in marketing (e.g., copywriting, PPC or SEO), conversion optimization requires an optimizer to have hands-on experience in every aspect of marketing: analytics, UX, psychology, copy, design, emotional targeting, business development, data and much more. It’s a position that requires the ability to understand every aspect of the business, its customers and analyze not just certain parts of a funnel but the entire online experience.

Skills and techniques can be taught, however a passionate, dedicated attitude and a desire to continuously learn, grow and drive the company forward, are rare and very hard to come by. These are the true indicators of a successful and dedicated optimizer.

Another important trait to look for is agility — someone who is able to learn, adapt and move fast. An optimizer must be able to inspire teams, be customer driven, allocate resources, constantly analyze numbers, collect feedback and change quickly as new data and research keeps coming in.

When hiring, ask for references, ask for case studies, ask to speak to other members of the team and not just the salesperson who might know the “keywords.” Consider adding milestones to your contract (i.e., specific, time-based goals) for evaluation.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?


Agencies and individuals that “do it all” (e.g., PPC, SEO, Inbound and “Oh and we’ve just added a conversion optimization department”). These types of companies or individuals are the ones that added CRO to their resume because everyone else is doing it and probably don’t have the exact expertise you’re looking for.

Companies or individuals who promise certain uplifts or huge increases in conversion rates before getting started or even looking at your site.

Those who immediately tell you what’s wrong with your funnel or landing page before looking into data.

Anyone who doesn’t have a dedicated process for analysis, observation and execution. You want to work with someone who understands the role of CRO in a business and has a set process that you can follow and learn from.


Chris Goward:
Founder & CEO at WiderFunnel

What makes someone a genuine conversion optimizer? What should you be looking for?

Great optimizers come in many flavors and need to be judged on their results.

They definitely need experience in personalization, A/B testing, behavioural design, cognitive biases, persuasive copywriting, web technology, statistics, design of experiments (DOE), project management, etc.

But, that’s not enough. You only know they are genuine if they deliver consistent business results. The challenge is reliably proving their results.

At WiderFunnel, we’ve solved this with a rigorous selection process. Each new member who completes our intensive hiring process starts as a coordinator until they prove their results delivery. Their thinking, values and temperament are tested through our weekly intensive group LIFT Zone meetings. Only the best become WiderFunnel Strategists, while others may move into other specializations that suit their skills. Using this meritocracy ensures a consistent team of A-players.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

When you’re hiring an optimization agency (or any agency, really), you’re hiring for credibility. You need questions to validate their credibility. Here are a few to start with:

  • Experience: How long has the agency been fully dedicated to conversion optimization?
  • Focus: Do agency revenues come primarily from conversion optimization, A/B testing and personalization services?
  • Results: How many case studies of relevant work has the agency published?
  • Process: How thorough is the agency’s process specific to optimization and A/B testing?
  • Expertise: Do your agency contacts have stated perspectives on advanced topics like design of experiments, statistics, and behavioural personalization?
  • Team depth: Do they have a cross-functional team of experts with deep individual expertise, cross-training each other to continuously learn?
  • Thought-leadership: How many thought-leading articles, blogs, books, webinars and conference presentations have they produced?

Tim Ash:
CEO of SiteTuners and Chair of Conversion Conference

What makes someone a genuine conversion optimizer? What should you be looking for?

Skepticism is critical — you should never be satisfied and should always look for problems and areas needing improvement. Your main concern should be to help the website visitor, and align their experience with the goals of the company.

You also need to be strategic in your approach and work with empowered executives within the company to get meaningful stuff done beyond just fiddling with campaigns and landing pages. This includes not just the web experience, but may also impact the marketing technology stack, business model, email communications, content creation, company organization and culture, staff training and back-end workflows. Only then will you be a true partner in growing the client’s online business.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Many fake CROs think that simply knowing how to use a split testing tool and running tests makes them an expert.

In fact, an over-reliance on testing as the only tactic to produce results shows that they are not comfortable making substantive changes to the larger business without the crutch of having every little change validated along the way.

You should also watch out for any agency claiming to have proprietary frameworks, formulas or guaranteed methods to produce results. The fact is that every business is different, and we invest enormous time and energy to understand the particular and evolving needs of each clients on an ongoing basis.

You can’t paint by numbers, and one size does not fit all.


Brian Massey:
The Conversion Scientist at Conversion Sciences

What makes someone a genuine conversion optimizer? What should you be looking for?

The number one thing that makes a conversion rate optimizer is curiosity. Second, rigor.

Look for the CRO who collects questions and is skeptical of answers, but is always looking for data points to guide them.

They must be great with people, because a CRO is a data storyteller; we can teach them the data science, we can mentor them on turning questions into data, but we can’t make someone curious (and you certainly can’t teach someone how to gently bring others into our world of rigorous creativity).

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

If someone says they know how to make a website convert better, that’s a red flag. I’d rather have someone who says they know how to find out what makes a website convert better. Any CRO that has all the answers has not been properly humbled by the process.

Look for curiosity and empathy. If they haven’t paid for a course in the past 12 months (on any topic); if they can’t tell you the most influential books they’ve read (on any topic); if they haven’t spent time serving people in previous roles, they may not be a good CRO.

The conversion scientists who are making our clients the most money were a librarian, a wedding planner at a flower shop and a salesperson before we hired them. They are all formidably curious and good with people.


Ton Wesseling:
Chief Optimization Officer at Online Dialogue

What makes someone a genuine conversion optimizer? What should you be looking for?

Look for three core capabilities:

  1. Stats: The expert should know about the statistics behind online experiments (what kind of experiments to set up, giving the data available in the company).
  2. Insight into psychology and behavior: The expert should know about the basis of human behavior (knowing which persuasion tactic could lead to what behavior).
  3. Soft skills: The expert should have organizational soft skills (knowing when to promote what about online experiments in the company to get things done).

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Ask them the following three questions:

  1. How many conversions do you test on (ask for old tests)? If you often see numbers below 1,000 conversions… don’t hire them. The other trick is to ask them if they know about Power Levels and if they calculate this up front (should be 80% or higher). This is the biggest problem with experts or agencies pretending to be awesome optimizers.
  2. How do you decide what to test? If they pick a page first and then hypothesize what to optimize: dump them. You should hypothesize first (based on a proper study) and then decide where to experiment with this hypothesis.
  3. How do you create the A/B-tests that were designed? If they tell you they just use the drag and drop editor from A/B-software vendors, dump then. For most good optimizations you need to write you own lines of code.

André Morys:
CEO & Founder of konversionsKRAFT

What makes someone a genuine conversion optimizer? What should you be looking for?

Curiosity is the most important trait. Great optimizers always question the status quo to improve things. They seek to understand the bigger picture, recognize patterns and don’t give up until they understand the whole thing. They are a perfect symbiosis between qualitative (“understand”) and quantitative (“measure”) skills.

You also have to differentiate between hiring a freelancer or agency and an employee. An external hire should be trusted by management to report the results and get buy in based on that data. Often the problem is that external optimizers don’t get the access to data that they need to deliver appropriate and trustworthy results.

Same applies for an internal hire, but they have mostly no trust problem. The main issue here is the missing external point of view of internal teams. Internal optimizers have to rely on feedback from outside the company — “you can’t read the label from inside the bottle” — but they suffer from a Dunning-Krüger-Effect: they don’t know what they don’t know.

How can you spot a fake CRO expert or agency? In other words, what are the red flags?

Unexperienced CRO agency people spread fake uplifts and multiply them with revenue to create a fake impact that is not valid. Mostly these uplifts are too high because tests are stopped too early. So always ask for test duration, sample size and traffic sources. Unexperienced inhouse people only test small changes or elements because they did not realize yet, that they have to influence user behavior with high contrast. They also don’t know how to report the real ROI to the C-suite.

What hiring a real conversion optimizer comes down to

Conversion optimization is big business. And for good reason. The payoffs can be huge. Unfortunately, the costs can be just as massive.

So what does it all come down to? Hopefully you’ve picked up on the commonalities above:

  1. Process: Optimization is not A/B testing. It’s a systemic approach that includes testing, qualitative feedback through surveys and polls, quantitative data, tracking and recording digital behavior, hypothesizing, analyzing, implementing, and iterating. Lack of a comprehensive framework is the number one showstopper.
  2. Rigor: The list of disciplines and expertise CRO demands is daunting: web analytics, behavioral science, psychology, user experience, design and development, and — of course — mathematics. A well-documented track record in all these areas is a must. And anybody that can’t parse the difference between statistical significance, confidence, and correlation coefficient should be shown the door.
  3. Curiosity: More than just technical skills, optimization is about a cultivated passion for asking questions. After all, creativity comes not so much from innate genius, but wide exposure to a host of disciplines, making connections where other people see only noise, and a relentlessly inquisitive mind.

Where does all this leave me or anyone else wearing the title of “CRO”?

If you’re a copywriter, be a copywriter. If you’re a content marketer, be a content marketer. If you’re a PPC ninja, be a PPC ninja (just don’t use the word “ninja”).

By all means, learn, grow and develop. But until you’ve earned it, the last thing the online marketing world needs is more fakes.

Wanna weigh in on how to spot the pros from the pretenders? Leave a comment. Just please — if only for the sake of my mom — be kind. Catharsis over.

About Aaron Orendorff
Aaron Orendorff is the VP of Marketing at Common Thread Collective, an ecommerce growth agency helping DTC brands scale beyond $2M-$30M. Previously, he served as Editor in Chief of Shopify Plus; his work has appeared on The New York Times, Forbes, Mashable, Business Insider and more. If you’d like to connect with Aaron, reach out via social — he’s a sucker for DMs about content marketing, bunnies, and rejection.
» More blog posts by Aaron Orendorff


  1. Jen Havice

    Awesome post Aaron! Lots of good food for thought. Seeing everyone and their cousin attach the acronym to their title regardless of expertise is quickly turning CRO into snake oil. There are very few who can truly do the whole process correctly end to end. The pros you cited are some of them. I have a feeling this post might just be the catalyst we need to talk more openly about all of this.

    • Aaron Orendorff

      Thx, Jen! And I hope this moves everything in that direction, too.

      For myself, I really did get caught up “playing the part.” One of those “believe my own hype” kinda things. Until the rude awakening … but reality has a way of doing that (esp. when the data is staring you in the face).

      Plus I gotta give HUGE props to Michael Aagaard. After one of my posts, he spent like 45 minutes on the phone with me walking through exactly where I was going wrong. It was hard to hear, but incredibly kind. :-)

  2. Claire Carradice

    A truly unique article…I so admire your brutal honesty – about yourself in this article – Aaron! Wow, hats off to you! The subject of differentiating the pros from the pretenders is necessary because there is SO much information out there, and as you say, most of it from pretenders. In line with this, I am in the process of readjusting my own business brand: I am going to brand myself as an expert in marketing basics for beginner companies. Once they’ve grown and can afford the Big Boys, they can move on to a pro. This way, I am not “pretending” :-). Congrats on an awesome post :-) Again.

    • Aaron Orendorff

      SO much encouragement. You’re awesome, Claire!

      Realizing what I’m skilled at, NOT skilled at, where I want to grow, and what I love was huge for me over the last few months with so much transition.

      Love hearing that I’m not alone in how positive getting clarity truly is. :-D

  3. Daniel Davidson

    This summed up it for me: “… if they don’t mention the word “research” at all. This is an indication that they are going to base important decisions on their own opinions…”

    @aaronorendorff:disqus you did a fantastic job. I love the voice of caution you give when labeling what we are and aren’t. If I’m honest, it helps reign myself back in with regards to my own pitches.

    All 8 contributors are great, but I think Michael Aagaard won the best quote. At least in my book ;)

    • Aaron Orendorff

      Thx so much, Daniel.

      And yeah … Michael is a rockstar (and a seriously great human to boot)! Check out my reply to Jen Havice. Michael’s the one that took me aside (painful, necessary, KIND).

  4. Aaron Orendorff

    Great point … and great video. I just checked it out and Buffered it up across my social.

    Thx for jumping in on this, Bryan.

    Do needs have to do with the industry of the client or more with what they’re trying to achieve? (Or both?)

  5. Aaron Orendorff

    Wow, Paul … that’s exactly my hope too.

    And this — “Humility, Integrity, Passion, Positivity and Open-mindedness” — so many YESES!

    (PS, I tried clicking through the links, but the first two didn’t work. Can you hit me up on the old FB? Love to check ’em out.)

  6. Jason

    Great article, Aaron. Even if all that brutal honesty did hurt my feelings a bit.

  7. Lianna Patch

    Awesome post, Aaron. I agree with Jen that this might just be what we needed to open up the discussion.

    Copywriters can also do more to clarify our role in the CRO process—i.e. we do NOT manage the whole CRO process end-to-end, but what we do is integral to its success. I’ve had too many confused clients ask me if I could handle their analytics…

  8. Tom Bukevicius

    First of all great job at “faking” it, this truly shows your strength in content marketing. Thoughtful questions and excellent article.

    Few things that struck my chord:
    1. “Focus on finding out what the problems are before coming up with solutions by Peep” This is applicable not only in CRO, but in all industries. You don’t want prescription before the diagnosis.
    2. Process and rigor. Real expects have a process and execute it. They don’t look for the CRO tip of the week. I live by this motto: “Magic bullets are for losers. Execution is key.”
    3. Ninjas calling themselves ninjas. Real experts don’t call themselves experts. Their work speaks for itself.

  9. Ro Arora

    Great article! I’m neither an expert nor a pretender. Just awed by the folks at the top of the CRO game, like Peep, Talia, Brian and Chris G and others. Watching their impact on brands from afar has been inspirational. The quality of knowledge they put out is a mind-bender. Hope to learn directly from a few of the 8 above one of these days. #fanboy

  10. Momoko Price

    Just yesterday I had a former client come back to me asking if they could get a “light” version of my conversion funnel audit (which includes data analysis, heuristic analysis, add-on option for VOC research, recommended tests, and stats guidelines) in which they would show me a landing page and I would just whip up some test variant pages for them, without any research.

    I said no. Technically it would have been easy money, but without research & proper experimental design, I agree, it’s not CRO. But more importantly (and I explained this to the client), I’d be taking their money and wasting their time.

    It’s one thing if a client has a creative vision or business idea they want executed. If that’s what they want and they need someone with the skills to put it together, that’s fine. But if they want someone to *figure out ways to improve their website’s business performance* (which is ostensibly what CROs are supposed to do), then they should understand that rigorous investigation is arguably the most important part of the process.

    I think you can still be an optimizer even if you don’t run sophisticated split tests (most small biz sites don’t have the traffic to support them anyway). As long as you’re still tracking before & after performance, that’s the right mindset to have … and for low-traffic sites, the best you can do.

    But I don’t think you can be an optimizer if you’re willing to skimp on research, analysis, and proposing solutions based solely on the findings of said research & analysis.

    • Aaron Orendorff

      That’s such a powerful theme … the need for a process steeped in “research, analysis, and proposing solutions based solely on the findings of said research & analysis.”

      And good for you saying no.

      I unfortunately succumbed to the temptation more than once.

      Plus I know @makemention:disqus would 100% agree with you on the front-end research.

      • Momoko Price

        These days one of my lead qualification questions with new clients is actually “are you looking for copywriting help or CRO help?” Often times clients have the two all mixed up in their heads and asking this question gives me the opportunity to pull them apart and do a little education before moving forward (or decline the engagement and/or refer them to a dedicated direct-response copywriter, because that’s what they’re actually looking for, they just don’t realize it).

        So often the client wants to “fix the copy to get higher conversions,” but after popping the hood and shining a light on their funnel analytics, it becomes crazy-clear that that’s not actually what they need to address …

      • Momoko Price

        Also, @Jen Havice and I do indeed bitch about this very thing all the time in Slack and on Skype hahahaha :D

    • Gerry

      Not enough digital marketers follow suit here. The whole “…taking their money and wasting their time” reason. I wish more professionals would stick to this, it would make the industry as a whole look better.

      • Aaron Orendorff

        Well put, Gerry. It’s such a fundamental principle to business in general … but when you’re serving clients who don’t have expertise (like advanced marketing or CRO) the vulnerabilities are huge. Thx for your comment :)

  11. Jason Quey

    Great write up Aaron!

    Understanding and focusing on just what you are great at is a tough pill to swallow.

    There are many areas of marketing I’d love to get better at (CRO, FB ads, etc). And it’s easy to fake it too (especially with all those expert roundups too).

    But I realize I’m hurting my clients, and myself, by spreading out too thin.

    • Aaron Orendorff

      Oh for sure! It hurts everybody: clients, industry, yourself.

      On that last point, I love learning and the student mentality, but settling into what someone’s genuinely skilled at — not to get too touchy-feely — is HUGE. I experience the difference viscerally when I’m doing work I love. Plus that’s the stuff that always does best for client.

      That’s why I particularly felt Talia’s line: “a passionate, dedicated attitude and a desire to continuously learn, grow and drive the company forward, are rare and very hard to come by.”

      True across the board. But it’s magic when that’s what you pursue.

  12. Ross Simmonds

    Great post Aaron. As much as round ups get a bad rap in this industry, folks like you still make them worth reading. I loved this quote from Talia in particular:

    “When hiring for CRO, avoid agencies and individuals that ‘do it all’.”

    This is true for CRO but also for SEO, SEM, PPC, Content Marketing, Design, etc…

    It reminds me of a tweet I sent a few months back:

    I’m trying to hire a chef with expertise in Sushi, Halal, Korean, BBQ, Louisiana Creole, vegan, haute and Chinese cuisine… That’s how ridiculous you sound asking for one person/agency to do SEO, SEM, Content Marketing, Support, Code, Design, Email, PPC, Display and Growth Hacking.

    • Talia Wolf

      Love this Ross! Will definitely be quoting you on this!

    • Aaron Orendorff

      Number one, pretty much anything Talia says is GOLD!

      Number two, totally agree with the assessment. When I was starting out, I said yes to everything. You kinda have to at that point. At the same time, it was saying yes outside my expertise that led to this entire post. Good learning experience … as long as it’s a LEARNING experience and it stops me from doing that in the future.

      But it’s so good for businesses to hear that word of warning. “Sure, I can do that too!” = RED FLAG

      As for your chef … narrow it down, and then invite me over (as long as you don’t end up with the vegan one).

  13. Raphael Paulin-Daigle

    Love this. Such an awesome post Aaron, glad you wrote this. Helps legitimize the industry.

    It might be a relatively “new” practice compared to SEO or PPC, but there’s still so much BS out there. Just last week a one of the best known internet marketers was advertising his “120 proven A/B test to test right now” ebook on Facebook…

    Of course, people who don’t know better will think it’s going to revolutionize their business, but when they figure out nothing works for them, they’ll cross-off everything CRO. Quite pathetic.

    Talia’s advice on “agencies that do it all” is spot-on too. At least once a week a “360 degree” agency owner tells me they want to integrate CRO…

    Then when clients hire us because the previous agencies didn’t produce any results, I should stop being surprised to see their tests were only being run for 2 to 3 days. Agencies keep selling “testing” as upsells to their main services, yet don’t do research, and have no idea how to run a test – or what CRO is for that matter.

    There needs to be more education on the process of research, but in a way more companies get it. We’ve been asked by quite a few companies to skip the research part too – safe to say they never became clients. I see research as #1 – got a 10k word guide on this next week ;)

    Thanks for writing this Aaron!

    • Aaron Orendorff

      A thousand times YES to research! For copywriting, I pour like 80% of my energy into research (basically ripping off Jen Havices processes) … and then writing is a dream.

      Loved what you said about when clients finally head your way. I got more than a handful of direct messages and emails after this posted about CRO horror stories along those lines.

      And … THX!

  14. Talia Wolf

    Great article Aaron. I enjoyed taking part in it. I also agree with so many of the comments by Bryan, Lianna, Jen and others.

    A completing piece to this would be, how to be a CRO client and actually get your money’s worth (aka- increase conversions). So many businesses will take that step to hire an expert or an agency to optimize their online experiences and end up making many mistakes that hold them back from actually seeing those results come in. There are some important steps that need to be taken within the company when a process like this begins to ensure its success.

    • Aaron Orendorff

      That would be a GREAT topic, especially balancing accountability and trust.

      I’d be super curious about the first, especially when it comes to something like, “What are the performance metrics you should demand getting reports on at different stages as well as ongoing?” (Although, demand might need to be put more polite.)

      As for trust … I can’t remember if it was Jen Havice or Joanna Wiebe who told me once, “When a copywriting client shows up with a pencil, you know it’s not gonna go well.” Basically, a lack of trust in the expertise they’ve hired.

      Lastly, another HUGE thanks for being a part of this!

      • Talia Wolf

        That quote is everything. The #1 problem is clients arriving with that pen. Then there’s also the subject of how to get everyone on the team on board with the process for speedy implementation (so you don’t waste time), what type of reports you should be asking for and how to spread the knowledge from the tests and process to the whole company.

    • Momoko Price

      100% agreed, Talia!

  15. katy

    The truth hurts and this article cuts deep. Love it. Do you have a cliff notes version?

  16. Jenny D

    Really great article, thanks. And great embedded quotes, I Tweeted three of them they were so well said (not that I’m even in the computer industry, I am curious though).

    • Aaron Orendorff

      Thanks so much. All credit for those embedded quotes go to Ton. He went above and beyond as a contributor and made that recommendation.

      (You see … a great optimization mind never quits!)

  17. Darren DeMatas

    Definitely a ton of “fakes” out there in the Internet Marketing space. So this article addresses that topic pretty well.

    Except for one thing.

    There’s a fake on here!

    One of these people is the Donald Trump of CRO. All talk. No delivery.

    I know this because I hired one of the people on this list. I recommend one of my clients/friend work with this “CRO expert” because they are widely recognized in the industry.

    We paid $30K for a “conversion focused redesign” and the conversation rate sucked. To make it worse, the expert didnt even follow their own process they blog about. No testing, just a survey and a crappy value prop that didnt resonate.

    I know CRO is more than testing, but to redesign an entire ecommerce site with 0 testing seems strange to me.

    Granted the traffic wasnt huge, but when I asked the so called expert about it they said the site was too outdated and anything would be better. I lost a client and friend bc of this. In the end a stock shopify template performed better than the $30K redesign. The expert didnt even reach out to us after the fact. They moved on to other higher paying clients.

    Anyone providing services in this space, knows the truth. Not every client is a fit and not every project we take on is a slam dunk. But when we take money from people and deliver crap results, it would be nice to see the other person care a little and try to fix it or make it better. Nope.

    Just $30K down the drain. Which is a lot to a small ecommerce business. I lost all credibility with my client and friend after this.

  18. Tim Stewart

    Great article and I would echo the points a few others have made

    How to choose and work (professionally) with an agency (any agency not just CRO) should be a qualification/training for any company contact.

    I have lost count of the number of times I’ve been called in to fix stuff and been told “our agency/current platform vendor has not delivered” and looking into it found the agency has been undermined at every step and good advice provided, but not followed. Whether that be from lack of buy-in and support or nominal support but a lack of capable resource in-house, the effort has been made but the results have under performed.

    Effectively wasting the company money by treating professional support like an inconvenience or menial lackeys.

    And that is not even considering the opportunity cost of not getting the absolute best from the agency/tool/vendor support when your competitors are gaining ground or the market is moving quickly. It also does not account for the (not inconsiderable) cost of RFP, time spent on tender process, vendor and agency selection needed to swap to a new provider. All because the perception is that next time the grass will be greener.

    It is the most vicious of cycles and a very common issue.
    Because it does not address the root causes – not buying appropriately, not correctly using the support once you have contracted it.

    It is a skill and not one which can easily be passed to centralised procurement, but in larger companies it frequently is a separate function. The best placed people to know what is needed should be the team who will be working with the tool, consultant or agency.
    They know what they think they need and they have a clearer idea of what their resource and development reality will be in practice
    But quite often they are allowed at most a couple of bullet points on the requirements section of a procurement document

    Without that direct contact even a good consultative approach, to establish the situation, needs and potential issues … is not possible
    The pitch is designed to meet an artificial picture of capability and capacity without any chance to investigate the reality and personalities

    Which is made worse because there is little incentive to sell transaparently and consultatively. It is time consuming, even with open discussion with the right people, to match solutions to needs. Especially when the competition is grabbing market share by using a templated and easily automated and marketed pitch promising magic beans and blue skies. Magic Beans which they are well aware they can’t deliver, but don’t care because if they grow fast enough they deprive the competitors of oxygen in the market.

    Add in a bit of churn of the people in the in-house positions and you are effectively boarding a new supplier or new internal person every few months. Then someone senior will (rightly) question the unimpressive ROI against TCO and either continue to under-invest or cut funding completely

    So the spiral continues. And then someone pops up selling a magic black-box solution that promises to take the pain away with an algorithm
    Snake oil with a new shiny label. But agency or self-teaching special sauce algorithm – it still needs skills to be applied correctly.

    But it cuts both ways – the agencies should be prepared to deal with this, help their partners to avoid this, be transparent about realistic expectations and requirements from the very first sales contact. Every agency therefore needs to upskill sales and account managers to better identify and handle those situations because it is a shared responsibility. Accept that sometimes they will lose deals to a slick sales pitch with no substance. And that the clients who haven’t learned to avoid these will always be hard to win, no matter what you promise.

    Yes, it’s easier to win a deal with unrealistic promises, cookie cutter sales pitches for scale and under cutting the competition on price
    Then ‘fake it until you make it’; working out how the hell you can make that all balance after you “win” the deal.
    Much easier, that is why it is so prevalent.

    But it is nearly impossible to deliver against, very hard to maintain and grow as true partners and very often ends up being a scorched earth situation. A market where clients have been burned so often that no-one can come in and (cost-effectively) meet the standards now needed to correct this lack of confidence. The only people who prosper are the ones most skilled at selling snake oil and exiting before they are found out.

    Without a mature approach in the market it becomes a race to the bottom, a tragedy of the commons which destroys the whole market.
    It narrows the field for opportunity and effectively lets clients spiral into lower profitability and greater cynicism about the discipline.

    Pushing them further away from a top-to-bottom cultural belief in a data driven process, letting them slip back into silos and blaming “poor support from suppliers” without addressing any internal root causes.

    If you don’t establish what is needed and create that working partnership then neither you nor the client can expect the same results that you can get with a genuinely mutually beneficial symbiotic relationship.

    If either side is more parasitic the imbalance will harm the value to both sides.

    I don’t think this is something that is unique to CRO, this is common across marketing and business in general
    But it is becoming more prevalent because CRO has now matured as a market
    One where there is still enough value and growth to make short-term market share grabs profitable – it is attracting poor practitioners because there still are clients without the experience to understand what sort of partnership they need and how to identify those partners.
    And there is enough client interest (and genuine success) that client rightly believe they could and should be doing more optimisation

    So part of this is the natural supply and demand of early majority, this part of the product and hype lifecycle.

    The other irony as evidenced by the contributors and commenters – we are definitely at the stage where the Dunning Kruger principle is clearly in effect.

    The experienced optimisers who have learned over time that they have only really started to understand how much they don’t know… are being beaten in pitches and “thought leader” pieces by people who have too little experience to understand how little they actually know but are very vocal in their misplaced confidence.

    I know Paul Rouke and I have discussed imposter syndrome a lot, very easy to let that noise get into your head and exacerbate that Dunning Kruger effect – “if all these people seem to find it easy and can deliver results without the effort I have to put in… maybe I am the one who shouldn’t be here”

  19. Sina Fak

    Aaron – you’re a beast. Awesome post man.

    I’ve failed many times while wearing my “CRO” hat. But I’ve also had enough successes to know I’m in the right space.

    The biggest contribution to my CRO education has been:

    A) Meticulously evaluating my “failures”, understanding why they happened, and doing my best to re-adjust (using as much actionable data as possible of course)

    and B) Not taking on too many clients/projects as once

    I think any CRO will admit – there is no crystal ball. Consistent “wins” come from experience, practice, and continuously studying the craft.

    By working closely with a few clients, I was able to mitigate any “novice” CRO tactics, hone my craft, and actually get decent results in the end (mostly because my clients trust me 1000% and continue to work with me even when certain tests don’t bring expected results)

    The last thing you want to do when starting out is – take on too many clients, make too many promises, and multiply the number of “novice” strategies you put out.

    I think @paulrouke:disqus nailed it in the comments when he said at core of every conversion optimizer is a HIPPO – Humility, Integrity, Passion, Positivity and Open-mindedness.

    CROs need to remember they’re dealing with people’s livelihood. It’s not game to be taken lightly.

    You need to give a shit.

    • Aaron Orendorff

      Fantastic feedback!

      A = ALL of life.

      B = So true. I have a tendency to overextend. But I’m getting REALLY friendly with the word “No” these days.

      And to your last line. Ef yes you do! Crazy how genuinely rare and vital that is.

    • Paul Rouke

      Thank so for referencing the re-invented HIPPO Sina. The more time I spend in this industry, the more time I spend with my colleagues, the more time I spend speaking to people applying for a role at my agency, and the more time I spend working with in-house teams -the more I feel how important these characteristics are.

      Ironically enough having met him and spent time with him, the inventor of the original HIPPO Avinash Kaushik represents every one of the 5 characteristics of the new HIPO 👍

  20. Bryan Collins

    “Skills and techniques can be taught, however a passionate, dedicated attitude and a desire to continuously learn, grow and drive the company forward, are rare and very hard to come by. ”

    Couldn’t agree with this more. You can learn by doing.. but you can’t do if you’re not passionate about your craft. I see myself as a (copy)writer first, but I love learning from other skillsets.

    • Aaron Orendorff

      Couldn’t agree more, Bryan. (And wasn’t @taliagw:disqus’s line a standout!)

      I spoke with someone yesterday on an unrelated article about Instagram of all things … and the EXACT same principles came up. They described the core characteristics of someone worth hiring to optimize on social as (1) a high threshold for pain (meaning learning from failure) and (2) longterm commitment tied to the client or employer’s success. The word they used for both? Passion!

      BOOM! There it is all over again. (Love it when that happens.)

  21. Arlene

    I have a start-up that is being launched in early June (100% online business). What is too early to start thinking about bringing someone on to advise guide on the entire marketing g and conversion strategy.

    Admittedly, the SEO/SMO, CRO, etc is mind boggling and very confusing.. where to concentrate first and what skill set is best to do the overall strategy (what, when, who).

    I’d love comments/ recommendations.

    • Aaron Orendorff

      As a content marketer … I side with focusing hard on onsite content alongside SEO (and social promotion) to generate traffic first. But that’s my uber biased opinion.

  22. Nat

    This is the first time I read a post that soooo honest (even if it hurts someone).
    Your post is absolutely unique.
    And I got so much reference on how I should build my attitude. Thank a lot for that.

    • Aaron Orendorff

      What a FABULOUS comment, Nat. Big thx! I was a bit nervous about posting this … but that response is the money (existentially speaking).


  23. Pierre

    Great article Aaron! I’m a content marketer by trade but I’m beginning to see how much CRO can and should be tied into the process.

    @taliagw:disqus mentioned that one of the red flags to look out for is agencies/individuals that “do it all”. Now, I don’t want to toot my own horn (I work at an agency after all) but I’d argue that many of these full-service agencies are well-equipped for the CRO job thanks to their breadth of expertise across a range of disciplines (which is what CRO requires). My agency has over 60 employees so there’s plenty of room for specialization.

    Don’t get me wrong Talia, I totally agree with your point that some agencies without a core service focus tend to spread themselves too thin and sacrifice service quality. But I would add that for agencies who are properly equipped to support multiple disciplines, CRO is a natural addition. Not because everyone else is doing it, but because of how relevant it’s become to the strategic marketing process.

    Thanks to all who shared! You’ve made an aspiring CROer just a little more knowledgeable.

    • Aaron Orendorff

      Pierre … look at that. A measured and respectful blog comment presenting an opposing view! (Who knew such a thing existed?)

      That makes sense to me. But then again, we all know MY level of expertise. ;-)

      Thanks for jumping in, buddy.

  24. Sayed shahnur

    Very ilumanting article. Great read .