confidence level in a/b testing landing pages

Confidence Level

Definition

The probability that the winning variation of your A/B test had more conversions for reasons other than chance. Before declaring a winner, you want a confidence level of 95% or more, as well as a sufficient number of conversions.

Conversion expert Michael Aagaard explains why confidence level is a necessary, but not sufficient factor in determining a valid test.
“Running an A/B test without thinking of statistical confidence is worse than not running a test at all.” — @peeplaja
Tweet This

Resources

  1. The Ultimate Guide to A/B Testing
  2. See How Dynamic Text on a Landing Page Helped Increase Conversions by 31.4% [A/B Test Reveal]
  3. A/B Testing Significance Calculator
  4. How to Build a Strong A/B Testing Plan That Gets Results