TemPlaza, the website built and improved by TemPlaza., JSC. We supply templates and other qualified applications on Joomla and WordPress.

Email: support@templaza.com

A/B Testing - The Best Way To Understand Your Customers

A/B testing is one of the best way to evaluate and understand how real users experience your website or application with users’ involvement.

It is extremely vital for designers, developers, web managers and SEO purpose. Instead of spending too much time with your designs to be objective about them or wondering the reason why visitors leave your site and your products, it is more meaningful and efficient for you to collect feedback from real users.

In the scale of this article, we will discuss some useful methods used for usability testing, specifically A/B testing and multivariate testing – advanced version of A/B testing.

1. What is A/B testing?

Simply, A/B testing or split testing should be defined as a way in which you put two versions: original version A (often called the control) and a different version named version B (often called variation)into an experiment simultaneously to measure which is the most successful performance based on your specified metric and choose it for real uses. Remember that you have to split your website traffic between two versions when execute the test.


2. How to test?

These steps are applied not only for whole A/B testing process but also multivariate testing:

Step 1: Put yourself in visitors’ shoes. You should ask questions: “Why visitors leave your site?” “Why they do that?”

Step 2: Analyze your site. There are some different tools for you to analyze and understand user behaviors. Google Analytics is the most popular tool to help you.

Step 3: Build hypotheses. You can think about some hypotheses. For example: “Adding more links in footer will improve conversion rate”.

Step 4: Estimate number of visitors involving test. This will help you to calculate test duration more reasonably. You can use this tool.

Step 5: Conduct testing.

Step 5.1: Determine what to test. Your choice of what to test will obviously depend on your goals. Supposing that you have a new trial application and you want to discover the download rate, it is meaningful for you to consider effect of Download Button.Some factors related should be tested: button color, text and position.

According to Paras Chopra wrote in “A field guide to Usability testing”, there are some certain elements tested:

• Elements of the call to action (Eg: button…): text, size, color or placement.

• Product’s headline and description

• Price and promotional offers

• Form’s length and types of fields included.

• Images used: size, arrangement

• Layout and style of website

• Amount of text on pages

Step 5.2: Create variations. Based on identification of what to test or challenges, you can create different versions of tested element. Back to our previous example of new application, with existing version having grey background color and the text content that is “Download now”, you can create some other variations. For example, you can change background color of the button from grey to red.

Step 5.3: Set up and run your A/B testing. After choosing suitable tool, you are going to set up your A/B test in one of two ways.

+ Option 1: You will count on A/B tools to replace randomly original version with one of its variations before showing page to visitors.

+ Option 2: Unlike the previous one, this option should be employed when you test entire page. Selected tool will redirect some visitors to your alternate URLs uploaded.

Another important sub-step is to set up conversion goal. Conversion goal page should be defined as a page you arrive your customers to helping your A/B tools to track the success of your landing pages. For example, if your element tested is “download now” button, then after customers complete downloading process, they will be landed on the “Thank You for downloading” confirmation page. Your tools will count the number of conversions for each variation and generate results.

Step 6: Analyze results and make decision.

3. Some notes

• Test two versions simultaneously. Different testing time will lead to wrong results and then wrong decisions. Don’t run 2 A/B testing at the same time on the landing pages.

• Conclude timely. In case, conclusion is made too early, some meaningful information might be missed. You should consider statistic confidence of the result that can be calculated by your tools

• Avoid shocking regular visitors because variations may not be implemented.

• Know how long to run your test. You can refer to useful tool to calculate reasonable testing time period.

• Repeat visitors should see same variations.

• Your test should be consistent across the whole site. For example, you are testing color of “Download now” button; your variations should have same text content (“Download now”), same size and same position.

• Conduct many A/B tests and estimate the number of visitors. You use size calculator to do this.

4. Some useful tools:

Google Website Optimizer



Visual Website Optimizer

Unbounce and Performable

As mentioned early, Multivariate testing is also a more useful tool to conduct usability testing. You can find more information and comparison between these two versions in the article: A/B testing or Multivariate Testing.

Is this article useful for you? Share it! Thank you!


Comment for this post has been locked by admin.

Comments Comments