Do you (or your boss) feel like you aren’t getting enough leads? Good, that means you aren’t in the minority. Do you know what’s one of the best ways to get out of that zone and feel better about what you’re doing? Run an A/B test and do some analysis. You’ll feel so smart! What is an A/B test? We’ll answer your questions and also tell you how to A/B test.
An A/B test is an experiment where you show two different versions of content to two audiences of similar sizes and determine which version performed better. You can use A/B tests for landing pages, emails, call to actions, and many other purposes.
Still interested? Good! We’ll be running through the steps of designing, implementing, and measuring your A/B test results. Although the example in this post will be call to actions, you can use A/B testing for almost anything.
1. Decide What You’ll Test
A/B tests are great because they’re versatile. To perform an A/B test you only need two pieces of content to compare. You can do something as small as changing the color of a button or something bigger, such as an entire landing page.
One thing that you must remember is that the more pieces you add you are testing as a whole, and not the individual differences. This means that if you’re testing two versions of a landing page and you’ve changed the call to action, the length of the form, the image, and the headline, you cannot attribute the landing page’s performance to a single element. The results are attributable to all four of those elements.
For the article we will be testing our call to action by changing the color of the button.
2. Figure Out Your Goal & How to A/B Test
You’ve got to think of what you want from the A’B test. For this example, do we want to test if the color of the call to action button affects how many people click it? Fairly straight forward. However, you could test how many people click more than once.
For this example we want to use the indicator of how many clicks to measure our success.
3. Set Your Control & Treatment
Don’t get scared. The control and treatment are easy to understand. The control is “Version A” of your test. The content you normally use for your emails, landing pages, and so on. The treatment is “Version B”. Version B has the changes you want to test.
In our example the control, or Version A, is a purple color, like what you might see on our site. Version B would be something different. Perhaps we go with a bright pink.
4. Create Your A/B Test, Then Release
Once you’ve designed how your experiment will work, the only left to do is… start testing! First design and create the content for your control and your treatments. In this case the purple call to action and the pink call to action. Be sure to notice the only thing that is different is the color of the button. The text is identical, a sis the size and shape.
The copy and images used are identical. Be testing this way we can determine if a color does affect the number of clicks.
Version A
Version B
You’ll have to setup the A/B test on your marketing software. Each tool is different so we can’t give you a step by step for setup, so consult your system’s documentation for guidance.
5. Promote Your Test – But Not to Everyone
If you want your test to mean anything, it must be statistically significant. This means that you’ll have to promote your content like crazy. Send an email to a sufficiently large list, promote your landing page across social media channels, or even use some pay-per-click to get enough visitors to give you results for your A/B test.
A key “how to A/B test” element to keep in mind is that if you’re testing for a specific audience, you need to ensure your promotions are tailored for only that audience. For instance, if you want to see how Instagram users would respond, then you wouldn’t want to test anywhere other than Instagram.
In our How to A/B Test example, we’re only looking at CTA conversions. We’d promote so anyone who is interested goes to the page.
6. Gather Data Until It Becomes Significant
Now wait. Yes, it’s hard, but you’ll have to wait while you gather data. Continue promoting until your data has significance. Why have we been using the term “significance” and “significant”? It means when you have enough data to be certain that what you’re recording isn’t from luck alone. If you’re really into math you can calculate significance yourself. Otherwise, use this little tool. Once you’ve hit the significance point, you can see if your treatment is more effective than the control.
What if you never hit significance? Wait. Pretty easy, huh? Sometimes it can take up to a month to get enough data. With that being said, if it’s been over 30 days and you’re still far away from statistically significant results and a lot of traffic has been sent to your test, then it’s probably not influencing conversions. Move on to another experiment. It’s OK.
7. Marketing Funnel Investigation
Now you know if your experiment worked. Yay! But we aren’t done yet. This is where you look to see if your A/B test affected anything else in your marketing funnel. I know I told you earlier in this How to A/B Test article that you focus on one thing. This is the exception.
It might seem silly to think that changing a color will affect anything other than clicks, it could be possible. If we don’t check, we might miss something useful. If your company uses closed-loop analytics, you can check who actually became a customer following your test content.
By looking at other parts of the marketing funnel, you might see that there were affects you didn’t anticipate. If they’re good, then you might be able to use them later. Otherwise, you might want to rethink the change.
A/B tests can have larger implications than just a single metric measurement.
8. Build on Your Findings
You’ve gathered your data and checked for unintended consequences. You’re done! Yay!
You aren’t really done. It’s great that you finished your test, but you can test more. You can change the positioning, or maybe change the content.
Or, maybe you don’t trust your results. Maybe there was a holiday during your testing and you see an abnormal spike in traffic. Run the A/B test again during different days. You can always return to the this How to A/B Test article if you need a refresher.
Continue testing because you never know what will make a giant leap and at the least you might make some great strides.
Let’s Talk About Your Brand
shane@3catslabs.com | Call +65-3159-4231
Share
