What tests should I launch? In what order? How do I stop several tests from interfering with each other? What risks are involved and how do I minimise these? Before you begin, these are the questions to ask. Follow our advice to establish your roadmap and get results from your A/B tests.
Before you start testing, you must mark out your route. A roadmap is essential, not only for clearly defining your goals, priorities and risks, but also for setting out a timetable. With this, you can track the progress of your project, and keep all of your contributors informed.
Your roadmap should, in particular, consider the following 8 aspects:
Test names
Be sure to give your tests precise, explicit names (preferably the same as you use in your AB Tasty tool). For example, a short name such as “[HP]wordingCTA” is preferable to an entire phrase such as “Test on the wording of the CTA on the homepage”.
Test descriptions
So that all your contributors are in step with your testing procedure and can follow it as it evolves, write a short, explicit description. For example, “Replace CTA wording ‘Create an account’ with ‘Sign up now’” will allow anybody to understand the content and goals of the test.
Test priority level
It is vital to rank your tests in the order of their importance in order to decide the order in which you will launch them. It is up to you to gauge:
- The expected benefits of the test
- The technical difficulty of the test
After the results of your first few tests, you will be able to adjust your ranking to optimise the effectiveness of your tests.
Test range
It is also crucial to include in your roadmap the range of pages targeted by your test. This way, you will stop different tests from interfering with each other (when you have several tests taking place at the same time on the same page) or possible side effects. To guard against this, we advise you to cut your site into slices and to give each slice a different colour in your roadmap: for example, blue for the homepage, orange for category pages, green for product pages, yellow for the conversion funnel, etc.
Primary and secondary KPIs
For each test, you should define a primary key performance indicator (KPI) (associated with a macro conversion). This indicator, which is the main reason for creating the test, will allow you to evaluate its benefits. It might be the click-through rate from the button “Add to basket”, the number of signups to your newsletter, revenue generated, etc.
You should also define secondary KPIs (associated with microconversions), which will complement your analysis and allow you to better understand the results of your test. Examples might include time spent on the site, number of pages seen, bounce rate, and so on.
Resources required
Some tests might require:
- Technical development
- Ergonomic development
- A specific launch date
These kinds of requirements must be specified in your roadmap.
Launch date and estimated end date
This information will make your team’s reports easier to read, and make it easier to plan potential future tests. In the meantime it will allow you to plan your testing activity precisely.
Possible side effects, Who to contact, Alerts
You should include a space for “notes” for each test, which will be useful in case of any problems. Here, you can write contacts, useful information, important things to remember, and so forth. This will save you a lot of time and worry.
