A/B Testing Idea Zone
You’ve probably heard a lot about A/B testing already – so perhaps you’ve noticed the following recommendations:
- Change your Call-to-Action (CTA) Button Colour!
- Reorganize the Position of Your Contact Form!
Would you think something as simple as changing the colour of a CTA button could actually boost your business? Even strategic changes that seem minor like these, have the potential to grow your website. But based on my experience, I don’t think these changes are making any dramatic improvements…
Before you call up your neighbourhood developer and begin changing colours and contact forms, let us try and capture – or articulate – the concept of A/B testing.
Let’s look at the following:
- Things You Should Know Before Starting an A/B Test
- How to Prioritize the Items of an A/B Test
- How to Review the Performance of an A/B Test
Contents
Things You Should Know Before Starting an A/B Test
Before starting an A/B test, don’t get stuck on the small things (or the technique for that matter).
If you want to boost your business with relative ease, A/B testing may not the best solution for you. The reason is, A/B testing requires some quality time to conduct – and you’ll have to convince your boss and team members that it is worth it. Plus, external factors will always affect your test results.
It’s harder to do A/B testing if one of these situations sounds familiar to you:
- Your website’s traffic just isn’t good enough
- You can’t find a good resource to conduct A/B testing
- Your boss is a total jerk (seriously – most A/B tests will include some failure or it won’t illustrate an obvious difference, even though you may have had some great ideas)
How to Prioritize the Items of an A/B Test
Big changes come first, don’t sweat the subtle details. When you’re thinking about big changes, I’m talking overhauling the entire concept of your website type big. That would entail text changing, colours, and structure – the gamut!
The reason?
Much like a start-up, you need to figure out where your products and services should be first – otherwise, you may wind up stuck in an inescapable tunnel of trial and error. If you can figure out how your business should be laid out online, you would succeed no matter how many times you failed with subtle A/B testing. ‘Big things first,’ is ideal. In the real world, it’s hard to keep up with it all when you have a lack of resources. Being flexible and speedy is as important as getting big things done first… Wait, this is getting confusing. I’ll come back to that later.
Based on Purpose
Let’s think about the purpose of A/B testing. We need to conduct A/B testing to track growth, but A/B testing has been broken down into two types:
- Growth directly
- Getting to know your targets, which helps the company to grow in the long term
Most companies focus on growth directly – which is good sometimes – but companies need to focus more on the education factor. A/B testing can help you to figure out:
- The pain points your target has
- Your approach with your target
- Key message to your target
Even though most A/B testing includes both ‘growth directly’ and ‘growth in the long term’ simultaneously, A/B testing missing the ‘education factor’ shouldn’t be conducted unless you have absolutely nothing else to do. Knowing more about your targets through the A/B testing process, helps companies to boost their reach beyond their website.
Don’t get stuck on trivial things. It’s not marketing science, you don’t have to figure out which colour of CTA button your targets may like right this second.
How to Review the Performance of an A/B Test
The two biggest expenses of A/B testing are time and money. Keeping costs low is obviously what every company is working towards, but time and flexibility are just as important as the money. If your company runs A/B testing once a month, that’s only 12 times per year. Is that a lot? Most A/B testing is likely to fail – if we were to say, “50% of A/B tests are successful,” that’s a pretty high rate realistically.
Don’t go with precision always. What we mean is, if you want a precise figures to make sure everything is right, it takes time to work with those precise figures – they’re not all that flexible when they’re in black and white. Stay with me now, so if your website has a 5% Conversion Rate and set a 10% of Minimum Detectable Effect (10% is really precise), you need 30,000 visitors (or traffic) on the page you’re testing to make sure everything is right according to statistics acquired on Optimizely.
If your monthly traffic on the page is 6,000 visits – you’ll have to wait five months to verify the data for just one test. That seems like a long time.
So in closing, consider these your valuable takeaways:
- Bigger tests should be prioritized
- Test that include the ‘education factor’ should be prioritized
- Making a decision based on A/B test performance should be quick
Just recap the things the other websites are saying:
You’ll need to clarify the following things before running your A/B test:
- How many conversions, sessions, page views and bounce rates are required for you to feel successful?
- When do you need to finish your A/B test by?
- Where to conduct an A/B test?
- What to change?
- Why are you doing an A/B test in the first place?
And for good measure, always remember:
- Don’t change more than two things at once
- Don’t run an A/B test if you have minimal traffic
- Don’t try too many variations once
Data does NOT show us everything, but it can show the consequences of the changes you’re testing, like conversions, clicks and bounce rates. You have to keep thinking about the connection between data and change. Data doesn’t make a conclusion; rather it reveals a clue for you to figure out. You’ll have to come up with the great ideas.
Header Photo: Kaboom Pics
If you want to learn more about specific key elements of an effective marketing plan, read our 50+ page booklet on How to Write A Strategic Marketing Plan That Yields ROI – complete with explanations
