Index

Below is an index of terms associated with A/B testingAn optimization practice that involves splitting your inbound traffic between two or more variations of a webpage to find out which performs better.

This methodology allows for maximizing signups and profit on any given website without simply increasing volume of traffic.  As opposed to most A/B campaigns focusing on a single or a few elements as a means of learning how to improve conversions, Big Idea Testing involves testing very different pages that may employ different tactics to accomplish the end goal.

The risk level in doing this is higher than incremental testing but so is the payoff. Bounce Rate is the % of people that leave your site immediately after visiting.

This is the best metric to track in A/B testing if your bounce rate is high. If your visitors aren’t sticking around to check out your site they can’t possibly subscribe or buy.

Click-Through Rate is the ratio of visitors who click a link on a page to how many total visitors saw that page.

The data isn’t always linear with this number as a there can be multiple links and users can click a link multiple times as well as multiple links.

Often referred to as CTA, a call to action is .. an action you’d like your visitors to take. This usually takes place in the form of a subscribe button or a ‘buy now’ button.

In the context of A/B Testing, subscribers and buyers are vital metrics so having a good call to action can greatly benefit your conversions.  In the context of A/B Testing.. Confidence level is the likelihood that the difference in conversion rates between your variation and your control is not random.

It’s the probability that the changes you made from your control to that variation are an indication of their success or failure.

Conversion Rate is the percentage of users who take your desired action (CTA).

Fundamental to A/B Testing, your conversion rate is a central metric and indication of a tests overall success or failure.  This is your A (initially). Control page is the page you have before deciding to further optimize it with A/B Testing.

Upon successfully increasing your conversion rate, during A/B Testing, the higher performing variation (B) then becomes the Control (A). In the context of A/B Testing, Distraction is the tendency for web creators to have ‘stuff’ slowing the user from getting to their desired location on your site.

Distraction is an element you can change in the objective to get a higher conversion rate via A/B Testing. An element in the context of A/B Testing is a component of your site that you may change in hopes of increasing your conversion rate by effect of that change.

Elements one might change are: Headline, Sub-Headline, Paragraph, Text, Testimonials, Call To Action (CTA), Call To Action Button, Links, Images, Social Proof, Awards, and Badges. Eye Flow refers to the tendencies of the human eye to digest visual content in certain patterns.

For instance, google search results are typically consumed in an F pattern when speaking of Eye Flow. The Fold refers to what you can see on the screen when a page is loaded.

Common phrases are ‘above-the-fold’ and ‘below-the-fold’. It can be helpful in A/B Testing to have important information displayed above the fold so anyone who visits the page can see it and maybe be enticed to read on.  In the context of A/B Testing, forms are what you use to ask for your visitors information. How your forms look and what they say, as well as what you ask for are important modalities you may consider addressing in an A/B Test.

Here are some components to Forms you may address:

  • What you’re offering in exchange for their information
  • How much information you ask for
  • Number of fields to fill out on the form
  • Relevance of the information you’re asking for to what you’re offering
  • Design of the form
  • What is required and what’s optional
  • Position on the page
  • Header
  • Copywriting of the Call To Action button

A visual tool to aid in A/B Testing – Heatmaps map out where visitors are clicking and scrolling.

This creates a reference of how your users digest your site and what parts of your site demand the most attention. A hypothesis in the context of A/B Testing is a statement geared towards increasing your conversion rate. Essentially you take a guess at how you might be able to increase your conversion rate – then you test that guess.

Example of a hypothesis: If I make my call to action (CTA) button stand out by enlarging it then more people will want to click it and I will get more opt ins.  Incremental Testing refers to a way of testing that involves changing only one element in a given A/B Test so you know how that one element affects the overall effectiveness.

This allows you to make useful assertions about your data concerning that specific element. The opposite of this is Big Idea Testing  Key Performance Indicators are metrics crucial to the success of your business. These are things you want to be tracking before, during, and after your A/B Test.

Some KPIs for A/B Testing include:

  • opt-in rate
  • free member to paying member
  • how much your average client spends
  • how long you keep your clients

A Landing Page in the context of Marketing and A/B Testing is a webpage that serves a singular purpose. This is generally to gather emails or make sales.

Landing pages typically don’t have links to the rest of your site; the idea is to limit the visitors options so that they may be more likely to follow through your Call To Action.  In the context of A/B Testing, Lift is the percentage of increased conversion rate that your variant page experiences from the control page.

Example: My starting rate was 10% and after I changed the CTA button my conversion rate went up to 20%, giving me a LIFT of 10%. Lead generation, in the context of marketing and A/B Testing, is when a visitor sees value in your content and wants to know more about who you are and what you’re offering.

For them to get to that point, you want to offer value – whether in the form of information of free services, you want to capture their attention and establish emotional equity.  More suited for websites with very high traffic levels, multivariate testing is about identifying a few or several elements to your page that can be improved – then creating each possible combination of these altered elements and testing all.

As opposed to A/B in which the goal is simply to outperform the control page anyway you can, multivariate testing is best used to refine a page without investing too much in redesigning it.


Example: Say you want to change a Header, a button color, and a CTA position. You come up with 2 versions of each to test and create each combination; 2 headers x 2 button colors x 2 CTA positions. This creates 8 variations systematically.  In the context of Marketing and A/B Testing, Open Rate refers to how many people open your email and how many click your link

These are of course useful metrics for A/B Testing.

Often refered to ROI, Return on Investment is a measure of the gains or losses as a result of an investment. Usually a percentage, a basic ROI formula :

Net profit ÷ Total Assets = ROI

In the context of A/B Testing and Marketing in general, a funnel is a user experience that involves going from prospect to lead to buyer – a process initiated by the visitor that doesn’t involve any direct selling.  Segmentation is a way of A/B Testing when you have multiple traffic sources.

By using different landing pages for each channel you can get useful data about each respective traffic source and make more accurate judgements while testing.  Social Proof is the proof that you can provide that lets your visitors know other people like your product, service, or content.

This is the result of a phenomena called groupthink – in which people subconsciously desire conformity. Traffic is the building block of all online businesses. You don’t necessarily need a ton, but you at least need a trickle.

Traffic refers to the people who visit your site and where they come from.