TABLE OF CONTENTS



Identify your A/B Testing Goals


Every A/B test should be conducted to solve a business problem and the potential impact of solving this problem should be quantifiable in the form of a KPI such as click-through rate, conversion rate, etc. This metric (goal)

should be used to identify whether the A/B test is successful. If you aren't measuring your goals then the result of your experiment is meaningless. 


There are different types of goals:


Engagement Goals: Used to identify the Vue.ai module that works best for any given touchpoint. Identify the Vue,ai modules that you want to test and measure the click-through rate on the recommendations.


Revenue Goals: Used to identify the optimal journey that yields higher revenue. To identify the impact of Vue.ai modules on your website, run an A/B test against a no-treatment journey, and measure the uplift in the conversion rates. You can also choose Average Order Value or Revenue per Visit as your goal metric.


Construct a valid hypothesis


A prediction that you make before running a test is called a hypothesis. It is a statement that clearly states what change you want to make, why you want to do so and it's expected impact. Once a goal is identified, you can begin generating A/B testing ideas and hypotheses to create customer journey variations. A strong hypothesis is made of three components:

  • Defining a problem
  • Describing a proposed solution
  • Setting criteria for success/failure and using metrics to measure results

Some examples of the valid hypothesis are listed below:

  • Cross Product recommendations on the cart page will nudge users to buy more products and help increase the Average Order Value 
  • If the Add To Cart button is the most visible element on the product listing page, it will catch the user's attention and increase Cart Addition Rate 
  • Highlighting Similar products  recommendations on the product page will help users discover more relevant products and hence convert better, thereby improving the Conversion Rate


Identify the traffic split between A/B test groups


It is advisable to allocate the traffic split equally (50% - 50%) for optimal test conditions. However, you can run an experiment with an unequal user allocation (10% - 20% - 70%). It is important to note that higher the difference among the groups, the longer the test will take to conclude.

Read Next 


Setting up of A/B tests