What is A/B testing?
A/B testing is a method of comparing two versions of a page or element by showing each to a random share of visitors and measuring which one performs better.
Is your site’s A/B testing pulling its weight?
Send me your site. I’ll look at it the way a new customer would and tell you where enquiries leak, free.
In plain English
Half your visitors see version A, half see version B. Everything else stays the same. After enough visits, you compare conversion rates and keep the winner. It removes guesswork, as long as the test runs long enough and the sample is large enough.
That last part is the catch. Reliable results need a meaningful number of conversions per version. Google retired its free Optimize tool in 2023, so most tests now run through paid tools or custom code.
Why it matters for a small business
For high-traffic pages, A/B tests settle arguments with data instead of opinion. For many small businesses, though, traffic is too low to reach a confident answer in a reasonable time. In those cases, careful before-and-after comparisons and talking to customers usually teach you more.
An example
An online jewellery boutique in Calgary with steady sales tests two product page layouts: one with the price near the top, one with it below the craftsmanship story. With enough orders each week, a few weeks of data gives a usable answer. A local accountant with a handful of enquiries a month couldn’t run the same test and trust the result.
Common mistakes
Where I see A/B testing go wrong most often.
Stopping a test the moment one version pulls ahead, before the result is reliable.
Testing tiny details like button shade when the offer or headline is the bigger question.
Running several overlapping tests on the same page so the results contaminate each other.
A/B testing, answered
It depends on your conversion rate and how big a difference you’re trying to detect. As a rule, if you get only a few conversions a week, a test will take a very long time to be trustworthy.
Not when done properly. Google’s guidance is to avoid cloaking, use temporary redirects for redirect-based tests, and end tests once you have an answer.
Keep reading
- nick@tavro.ca
- 587-439-9994
- Calgary, Alberta