Excellent post about the power of a single word. It's a great reminder to think carefully about the words we choose, and being as specific as possible. I'll be using this tip for a new program.

Thanks!
Casey

The result improvement was seen month over month and week over week were a bit less, but in the ball park of awesome. The site does receive enough traffic to arrive at decisions pretty fast. As far as I know, there were no significant seasonal factors, except the start of Hockey Season :)

I’m curious what your frame of reference for time period was for your before/after observation, particularly because of the stated redesign?

I’m only poking further for the benefit of the readers and would be happy to take this offline if you were more comfortable.

I too share your enthusiasm for results like this – we all do!

And thank you Keith for the in depth discussion regarding your process – people can learn a lot about behaviour by understanding how to observe their client’s issues.

I think it’s also important to remember that as an editorial piece, the intent here was to share a story and an observed result. While potentially not 100% scientific, it’s still a compelling story that can hopefully help people to understand the importance of trying something different.

Doing this by way of digging deep into an understanding of your client and customers is an important methodology for instigating change.

I also don’t think this piece is trying to say – “it’s this easy and everyone can get results like this by doing the same” – because clearly this wasn’t an easy process and every situation is different.

At the end of the day – the message is that we should all be trying ways to improve our conversion performance and ideally it should be done using sound testing practices whenever possible – and that sometimes “if it works, it works”.

Thanks again.

I will in the future keep my posts quantitatively sound. I was excited to share, and while still confident in the change, do not wish to be lumped in with the “all hype” crowd out there. Its a good lesson for a new blogger.

To be honest, I do play it a bit “fast & loose” to achieve results faster. That is my style I guess (I do test, but not as much as I could, especially when I feel confident in the change being positive – 2 parts “sales guy”, 1 part analyst). I’m sure I could learn a lot from someone like yourself who is more focused on testing.

There is no site that has traffic and purchases reliable enough to use the before & after method.

Statistical significance is a high standard to measure, but it’s worth the extra effort to have reliable results.

This change was made after the 2nd revision of the website (the 1st revision was a total redesign). The site had been under-performing in my mind and I could not figure it out. I ordered another 5 user tests (since I had been so immersed in the site, I needed to see more external perspective). I also went on extreme Live Chat “lurking” sessions and started listening into Customer Service calls, even going on a trip with a delivery truck to see where the parts were going, and who was ordering them.

After all was said and done, all “signs” pointed to Shipping and Return Policies, Product Warranties and post purchase support. This is understandable when you are selling “junk yard parts online of course, but confused me because the site handled all those objections both overall and at the moments at points the user might be looking for that information.

While reading the product warranty, I had the epiphany that the company really did have a “money-back” warranty, and that the wording might really help. I have been making site optimizations for about 10 years now and never would have thought there would be such a big effect from such a small change. But then, I did not have a deep understanding of the customer at that time.

Here is what I understand now, after a few more months on the account, about the type of customer that site attracts and why the change was so significant (gained through the methods above, as well as site surveys, feedback, customer interviews, and live chats):

The customer is fixing their own car, a lot of the time cause they can’t afford it any other way. The one who can afford a mechanic and a new car, just love cars. In either case, they need/want a part that is going to be in their engine, take a lot of their time to install and if it is not absolutely perfect, they will have wasted all there time, and perhaps has damaged their vehicle. Auto part fitment is very complex, and auto DIY customer know it (i.e. The alternator in a car may vary by the sub model), and they want assurances.

The client is large company, but not one with brand recognition for the target retail audience (they are B2B mostly). They literally appeared out of nowhere for a lot of the traffic that was being pushed to the site. Given the savvy nature of the audience, the niche appeal of the product, and the mere fact the site sells USED auto parts from collision vehicles (think junk yard), providing a more prominent and meaningful assurance element (from the user’s perspective) was the final piece needed to achieve the “purchase synergy” the site now has.

Also remember, this change was made in the right column of every page. It was not really one change, but 35,000 or so.

While not Split tested, the change was made in complete isolation. The site and its traffic had not changed for over a month and the site was being “bench-marked” for improvement from the previous revision. I felt comfortable making a small change that I could monitor and attribute to results (I was looking at the site every day, and had been for months and was so familiar with it I didn’t feel the need to take the time to do Split testing (I was using Google Website Optimizer and would have had to wait up to 2 weeks for the developer resources to implement it).

I am confident in the cause and effect here.

To back-up the change though, there is also a lead-gen micro-site for the California market that the same change was made on. Once I saw the effect the change had, I made the same one on the micro-site and saw significant conversion increase, not 159% as on the eCommerce site, but then it was B2B lead gen on a small site, so many different factors.

I hope this helps put it all together. Like I said above, you need to really understand a business and what its customers REALLY want before you can produce REAL results. This extreme result was not from a one word change as much as it was from this deeper understanding mixed with knowledge of how to present the information.

I personally think it was a moment of brilliance, one I have bored my wife and friends with several times since. It has taught me to dig deeper, really understand the customer and not overlook small changes.

I do have the benefit of being able to really dig into a site (I have 3 clients like this one, where I act as their web manager, conversion specialist, director of eCommerce, project manager etc) which I realize is different than having a site on continuous improvement program (which I do as well).

You are completely right, the REAL way to get results is through continuous, incremental improvements. I hate to think I gave the impression otherwise. If so, I apologize.

I hope this addresses your reservations about the change described. From the sounds of it, you think the change is as phenomenal as I do, and phenomena are rarely believed till they are seen.

Keith

I would also be very surprised if this was a statistically valid test result.

I can’t speak to Keith’s testing methods, but there are several blogs promoting test results that aren’t “real” controlled tests. Be careful to ask questions about the testing method before jumping to conclusions.

The only way to have confidence in the result is to use a testing tool that performs random sampling, maintains test cell integrity and reports with statistical significance. And a test like this should be run twice to confirm, since the result seems spurious.

The bottom line: each reader should test it for yourselves before running with this result.

Chris

I know most of the Vancouver optimization guys/gals too (pretty sure you all know who I am as well) – I’d love for Chris or Raquel from WidderFunnel to chime in.

If ya’ll want to chat about this live – I’m presenting at the January 13th session of the IIMA in Vancouver on ecommerce. It would be fun!