A/B Testing in Terminalfour

Introduction

For example, we might have a “Hero Banner” at the top of our page that contains a “Call-to-action” inviting users to “Enquire about applications”.

Perhaps we’d like to know whether the image used in this banner has any impact on the likelihood of a user clicking on the “Call-to-action”. A/B testing allows you to test this by deploying both “variations” of the Banner and randomly showing “Variant A” to half the page visitors, and “Variant B” to the other half of page visitors. 

There are many off-the-shelf A/B Testing tools to allow you to configure, manage, and measure the success of these A/B Tests. These tools allow for advanced marketing features and additional testing criteria, but they come at an additional cost.

Using Terminalfour it’s possible to perform simple and straightforward A/B tests without the need for a third-party product.

In this article we’ll walk you through how you could create a Content Type to A/B test a scenario like the one we outlined above. We’ll be showcasing how this can be accomplished with JavaScript, but the same principles can be applied regardless of the language you choose to use (e.g. PHP). 

You can then use your existing analytics tool to show which variant results in a higher click-through on your call-to-action.

This article assumes that you are using an analytics tool (such as Google Analytics, or Fathom) that can be used to give you statistics about what links are clicked on a given page.

You can either choose to implement this as-is, or use the concepts outlined to create your own A/B Tests.

Planning

Before diving in to A/B testing it’s always a good idea to take a step back and ensure you have a clear understanding of what exactly it is you want to test. In our example we’re testing whether the image used in our banner impacts the clickthrough rate of our call-to-action.

You may want to test whether using different text in the call-to-action impact that clickthrough rate. It’s recommended to only test one variation at a time. If we were to create an alternate Banner with a different image *and* a different call-to-action text, we wouldn’t be able to clearly infer which change had an impact on the clickthrough rate.

Modifying Content Type

Let’s take a look at our Hero Banner Content Type:

The first change we’ll make is to create a new element allowing our Content Editors to select a variation of the Hero Image:

Next up we’ll modify our Content Layout to output both variants.

Here’s how that text/html layout looks without any modifications:

<section class="hero-banner" style="background-image:url(<t4 type="content" name="Hero Image" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to action" output="linkurl" modifiers="nav_sections" />" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

We’ll be making 3 changes to this layout:

1. Adding a Query Parameter to the Call to action link
2. Repeating the entire section element so that it is output a second time using the alternate image
3. Adding some JavaScript to randomly show one of the banners to users

Step 1: Add a query parameter to the Call To Action

This first change allows us to tweak the URL in the call to action so that our analytics tool can differentiate between the two variants.

We’re going to modify the <a> tag to clarify that this link comes from “variant a” by appending ?variant=a to the end of the URL inside the href attribute.

After making this change our text/html Content Layout should look like:

<section class="hero-banner" style="background-image:url(<t4 type="content" name="Hero Image" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to action" output="linkurl" modifiers="nav_sections" />?variant=a" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

Step 2: Repeat the section with the alternate image

In this next step we’re repeating the entire <section> code but in the repeated version we output the alternate image. We’ll also change the Query Parameter in the second version to reference the fact the link is coming from “variant b”.

Once we’ve made these changes our layout should look like:

<section class="hero-banner" style="background-image:url(<t4 type="content" name="Hero Image" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to action" output="linkurl" modifiers="nav_sections" />?variant=a" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

<section class="hero-banner" style="background-image:url(<t4 type="content" name="Hero Image (Alternative)" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to action" output="linkurl" modifiers="nav_sections" />?variant=b" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

If we were to preview our page now we’d see both variants of the Hero Banner displayed at the same time. Let’s make sure the alternative one is hidden, so that if a user visits our page with JavaScript disabled they won’t see both variations.

To do that we’ll add the attribute property display: none; to the style attribute in the second <section> tag.
When we’re all done and dusted our layout should look like:

<section class="hero-banner" style="background-image:url(<t4 type="content" name="Hero Image" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to action" output="linkurl" modifiers="nav_sections" />?variant=a" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

<section class="hero-banner" style="display:none;background-image:url(<t4 type="content" name="Hero Image (Alternative)" output="normal" formatter="path/*" />)">
  <h2 class="hero-banner__header">
    <t4 type="content" name="Hero Title" output="normal" modifiers="striptags,htmlentities" />
  </h2>
  <a href="<t4 type="content" name="Call to actionk" output="linkurl" modifiers="nav_sections" />?variant=b" class="hero-banner__cta">
    <t4 type="content" name="Call to action" output="linktext" modifiers="nav_sections" />
  </a>
</section>

If the “Hero Image (Alternative)” is not a required field, you may want to use “selective-output” to only output the second `<section>` element if the “Hero Image (Alternative)” element has a value selected.

Step 3: Add JavaScript to show one at random

Now we can add some JavaScript to our layout that chooses one of the two banners at random and shows one of them when the page is being loaded.

The JavaScript looks like:

<script>
  let variants = document.querySelectorAll('.hero-banner');
  variants.forEach((variant) => {
    variant.style.display = "none";    
  });
  variants[Math.floor(Math.random() * variants.length)].style.display = "block";
</script>

You may need to update the first line of this JavaScript to reference the unique class of the Content you want to be displayed. In this example we’re showing a random “hero-banner” class. This may be different if you’re modifying an existing Content Type.

We recommend instead adding a unique class to your Content Type to reference for this behaviour, such as “hero-variant” and using that in the JavaScript above.

This JavaScript does two things:

1. It finds all the “Hero Banners” on the page and sets them all to hidden
2. It selects one of them at random and displays it

This means if you want to test more than two variations of a Content Type, this same JavaScript will work. You’ll just need to repeat Step 2 above and duplicate the <section> for each variation you want to test.

Analysing results

Your Content Type is now ready to be used to perform A/B Tests. Once your Content is live, one random Hero Banner will be displayed at page load, and each call-to-action has a unique URL (thanks to the query parameters we added to the end of the URL).

You can now use your analytics tool to check whether there are more visits to the link that contains the parameter variant=a or variant=b. If one has significantly more visits than the other it’s a great indication that the Image chosen in that variant is resulting in a better performing Banner.

Conclusions and summary

We’ve walked through what’s required to perform simple A/B Testing in Terminalfour without the need for a dedicated third-party A/B Testing tool. Using this method you can very easily test simple content variations and measure the impact in those variations using your site’s analytics.

If your requirements for A/B Testing are more advanced, more nuanced, or require more in-depth analytics features you may consider dedicated A/B Testing platforms.

For most straight forward use-cases the method outlined in this article works a treat.