The Dark Side of A/B Testing: How to Test Your Site with a Bazooka

Testing a website is the most popular passing time for digital analytics experts. However, testing the right thing is harder than ever.

Website optimisation relies heavily on A/B or multivariate testing with digital analytics tools. However, if it comes down to testing, most of the experts test only minor comparisons on a site. But testing should go forward only if you test highly tangible differences; this is where you can learn something and open new frontiers. So instead of burning through your site with a bunch of matchsticks, use a bazooka instead and start some real flames.

Test What Matters and Test Less

Real productivity comes when you leave your comfort zone

You have read this many times we guess; your life starts after you leave your comfort zone. Well, that is the same as marketing. Technology made us busier and helped to boost our productivity, but it also killed a lot in the details. Meetings, chat, endless notifications, these are the time killers, and we haven’t even mentioned social media. Being busy is nothing, being productive is something. If you get lost quickly in the little details, non-productive testing comes up in the digital analytics space, and you find yourself in the situation where you test two ad copies with minor changes between and drawing conclusions where there is none, only pure randomness.

Test less and test what matters

Now that you have stayed away from little details, you can focus on the real issues. Test what matters only, and test way-way less. By stopped paying attention to things that do not matter in the details, you can focus on testing the important stuff.

Testing, its name is an experiment. And most of the scientific experiments are dead ends for 99% of their time, producing non-accurate results. In 1%, the science team has a eureka moment, usually when they slip away from the original focus of the experiment. This is the same with digital analytics testing when you test small changes on a site; you usually end of testing random outcomes. But when you test with tremendous tangible differences, you might will learn something on your business and customers.

Be Patient with your RPG

Science is a game of patience and marketers are not scientists. There was a study with an investment firm; they concluded that their most successful investors are either dead or non-active. The reason for this is because live and active traders are impatient and burn their positions with their little actions. It’s the same with testing. So, when you set up a testing experiment, you have your goals, you made a tangible difference to the chamber and started the testing.

Don’t even look at the test until you have reached your goal. Don’t change the variables. Don’t change the tested markers. Don’t even open the testing chamber. Be patient. Draw conclusions only when the testing has finished, and you have all your figures.

Remember: testing needs a perfectly clean environment

Most experts fail in their tests because the environment they are testing in is imperfect. The site that they are testing has huge usability issues. The testing material isn’t ready yet. There is nothing to be tested. The traffic is not there so the figures are so small that drawing conclusions would be simply just guessing in the cloud.

Be ready first and sort your stuff out. Only test when you need to develop even further. Further means, you already have a pretty solid ground: a good site, with a high number of visits and only minor usability issues.

All in all, these are the simple steps you need to follow when you do testing:

• Test with an RPG: test only tangible vast differences
• Test only when you have scientist mindset: be patient and be ready to fail
• Test only in clean environment: your lab/site should be stable and established