How To Split Test Product Pages in Shopify

Lasse Rosendahl Ravn
11 min readJan 21, 2021

--

OBS: This post has been updated over here: https://lssrvn.com/how-to-split-test-product-pages-shopify

At some point in time, you’ve probably asked yourself: is my product page good enough? I know I have, so in this post, I’ll show you how you can split test different product pages.

Some wish they could change the “Add To Cart” button color, resize the product images, or change some text on the page.

Others wish they could wipe out the entire thing and start all over with building a new page from scratch. No matter which of these two boxes you check, there’s one important thing to remember.

If you change the user experience of a significant page like your product page, this could heavily affect the purchasing behavior on your site. So do it with caution.

As an alternative, you could perform a parallel test with different variants of the product page and measure how visitors interact with it.

Then perform the changes based on the results of that test without worrying about how it might hurt your business.

And since you’re reading this post, I assume this is what you’d like to achieve. So let’s get started with split testing product pages in Shopify.

Best Ways To Split Test Shopify Product Pages

Before we dive into the technical parts of testing, I’d like to explain the different ways you can test product pages in Shopify.

Fundamentally, you can either perform a split test on what’s called “client-side” or you can do it “server-side”. If you haven’t heard of either of these, you’re probably looking for the “client-side” option.

Client-Side Testing

Client-side testing means that all the split testing logic is happening on the “client’s side”, meaning in the visitor’s browser.

The other option is server-side testing, which I won’t go into detail about in this post. Just know that the majority of the people reading this post will be using client-side testing.

Shopify Product Page Split Test

A website split test is a test where different users are shown different variants of the same page.

Imagine visiting a split tested product page. Your browser is actually downloading all the content of the page before the specific variant of the test is shown to you.

Imagine this sexy product page for this shoe:

I know, it’s world-class. You see that the customer can either choose to “quick purchase” this item or just add it to the cart.

Let’s say that after looking through my orders, I see that the vast majority of my customers simply purchase 1 item.

So if I remove the “Add to cart” button, my hypothesis is that more people will convert, as they will be led to checkout faster. So I use this template as my alternative template.

So I put up a test that includes this baseline version with the icons included and then this second variant without the icons:

Your browser initially loads the full version with the icons included. Once the page is loaded, the A/B testing tool (which we’ll get to soon) hides the icons and then shows you the page.

This normally happens so fast that you will never notice. But that’s the “science” behind it.

The clear benefit of client-side testing is the ease of creating these tests. Most A/B tools offer an intuitive drag-n-drop interface that lets you delete, edit or rearrange elements on a page.

Shopify Product Page Redirect Test

Another way to create split-test product pages in Shopify is by creating what’s called a “redirect test”.

A redirect test is a very basic method of testing different variants in a test. What it does, is simply redirect the user to a different URL for each variant.

In our example, with the product page, we’re going to use this method in one of our scenarios by using these different URLs:

  1. https://lssrvn.myshopify.com/products/those-sneakers-that-every-blogger-girl-wears
  2. https://lssrvn.myshopify.com/products/those-sneakers-that-every-blogger-girl-wears?view=simple

Notice the “?view=simple” text in the second link (also shown in my screenshots above)? This is Shopify’s ‘alternative template url parameter’. This enables us to use any template on any product, collection or page. “alt” would be replaced with whatever you would name your new product template file. Pretty cool feature.

Your mind is probably now filled with ideas on how you can test different templates. So let’s make sure you get your first test live by later today.

Server-Side Testing

Server-side testing is the more advanced option of the two main ways you can perform tests.

Here, the tests themselves are performed directly in the code (liquid code that is, when using Shopify). This is a clever use when you wish to split test core features or elements in your shop.

For example, if I wish to split-test the use of two different wishlist-apps, this is something that I can perform in a server-side split-test.

As I mentioned, I won’t further explain server-side testing in this post. That’s for a later post.

The Tools You Need To A/B Split Test Your Shopify Product Page

Before we start our first test, let’s dive into the tools needed to perform split tests of Shopify product pages.

My own personal opinion is that you need:

  • Some level of HTML & CSS knowledge
  • A proper split testing tool

The first is important since your level of knowledge will dictate how sophisticated your tests can be. Being able to add, edit, and delete elements is crucial in getting proper results from your tests.

In addition, you need a proper testing tool. I won’t go over all the different possibilities but simply recommend a free, user-friendly option: Google Optimize.

Google Optimize is a really user-friendly tool that allows you to set up tests easily and fast, which I know is something that’s very important to a lot of people working with Shopify stores (because everything else in their day-to-day is).

Best Practices For Performing Split Tests Of Shopify Product Pages

Before we set up our first test, I want to talk a bit about best practices when performing a split test.

Test few, but very different variants in your tests
If you’ve previously read articles about split testing, you’ve probably heard about the “button color” test. In summary, testing the color of your button to increase clicks on it.

Feel free to test that. Although my opinion is that it’d only show a significant difference for those coming from a very poorly designed button.

Instead, what you should focus on is testing fundamental parts of your Shopify product pages. If you only change small parts of a page, it will increase the time required to significantly tell if there’s actually a difference in user behavior.

Focus on “compound pages”
We like to think of the customer journey as a single line going from discovery to purchase. It’s never like that.

Customers have massive amounts of touchpoints with your business before making a purchase. Still, making a purchase requires a customer to go through certain steps. So my advice is, focus on the pages included in those steps. That is:

  • Homepage
  • Collection page
  • Product page
  • Cart page / Cart slide-in
  • Checkout page (for Plus merchants)

The reason I call these compound pages is that improvements of these are assumed to have a compound effect. If you increase the conversion rate of people visiting a product page to those adding a product to the cart, this affects your entire customer journey.

In other words, a 5% improvement is 5% extra revenue, which is what I personally love about split testing these compound pages.

How To Set Up A Shopify Product Page Split Test In Google Optimize

It’s time. Let’s get our hands dirty and jump right in. If you haven’t created a Google Optimize account yet, go ahead and do that before reading on.

Here’s how to create a product page split test in Shopify:

  1. Install Google Optimize Container In Shopify
  2. Create a new product template in Shopify
  3. Set up the product page split test in Optimize
  4. Analyze the outcome

Install Google Optimize Container In Shopify

There are a few different ways you can install Google Optimize in Shopify. Brillmark has written a short and useful guide on setting up Google Optimize in Shopify. Go ahead and read that and then come back here.

Create A New Product Template in Shopify

In my other blog post about how to create dynamic sections in Shopify, I’ve also mentioned how to create alternative templates. But for ease, let’s do a quick walk over again.

Follow this path to create a new template for your theme:

Online Store -> Themes -> Actions -> Edit Code -> Add a new template

Even if you’re managing all your code from a local Shopify theme development environment, you still have to go through this process every time you add a new template.

I’ve named my template “simple” but you can choose whatever name you want. Just remember that this will be visible in the url for a segment of your visitors, so choose the name wisely.

After clicking “Create template”, go ahead and locate your original product template, named something like “product.liquid”. Copy over all the code from this template to your new file.

After doing so, we’re ready to make our changes. In my very simple example, I’m simply going to comment out a few lines of code that make up the button I mentioned earlier.

Unless you’re also using the Debut theme, my screenshot here won’t be useful to you. Just know that this is the only difference between my two product templates.

Set Up Product Page Split Test in Optimize

Because you’ve done most of the hard work before this step, setting up the actual split test in Optimize is fairly easy.

If you’ve never set up an experiment, go ahead and click the “Let’s go” button and choose to create a “Redirect test” as I have here.

Then choose “Add a variant” once you’ve reached the draft page of your new experiment.

After doing so, name your variant. I like to use “Variant x: {{what’s different about this variant}}”. So “Variant 1: No Add To Cart Button” is what I found appropriate here.

Now here’s one of the more important parts. After choosing “Advanced redirect” you’ll want to choose “Add/modify query parameter”.

Query parameters are everything that comes after a “?” in the url. If you’re familiar with marketing tracking, then you probably know all about UTM parameters. Same concept.

The difference here is that Shopify allows you to access alternative template files if you pass the “view” parameter with a value that matches a template file in your theme.

So in our case, passing “?view=simple” on product pages means that the user will see our alternative template without the “Add to cart” button. Simple as that.

This technique can be utilized across all the “page types” seen in the list below:

After clicking “Finish”, you’ve now set up your new variant. As a default, qualified traffic is split evenly between the variants, but you can choose to allocate less traffic to some variants.

This is ideal if you have massive amounts of traffic or if your new variant is so far from your original that it may cause disastrous results. Better being safe than sorry.

Just know that the less traffic allocated to the variant, the longer you’ll have to wait for the results.

After doing so, you’ll choose which visitors to include in your test. This controls which page visits should trigger your test. In our case, we only want visitors on product pages to be included, so we’re utilizing regex and our knowledge of how Shopify structures its page urls to create this rule:

Lastly, we’ll choose which objective this experiment is connected to. Ideally, we’d like revenue to increase, but my first priority is to see if more people will add my blogger-shoes to their cart.

If your shop does not use an AJAX-cart (the one that slides in from the side/top when something is added to the cart), then you’ll want to choose “Page views” and then insert the link to your cart page.

But since I’m actually testing if redirecting people straight to the checkout is better than just adding it to the cart, I’ve gone ahead and inserted this custom event I have. This is triggered whenever someone visits my checkout.

Take note that I’ve chosen to only count this event once per session, as I won’t reward people going back and forth between the checkout and my product page. Quite the opposite actually.

Finally, make sure that Optimize has been correctly installed by clicking the “Check Installation” button and letting Optimize load your page.

I recommend subscribing to email updates about your experiment (you’ll be sent very few) and in my case, I’ve chosen to allocate 100% of the traffic, that meets my targeting criteria, to this test.

After reviewing everything again, we’re ready to hit the “Start” button at the top which will initiate our experiment.

Now that I’ve launched my test, it’s best to do an additional test to see if everything is working as it should.

In my case I’ve visited the product page in a private browser a couple of times until I’ve been exposed to both variants and made sure that everything worked as it should.

Analyze Your Results

Google Optimize utilises Bayesian Inference to determine when and by how much your variants are better than each other. As a rule of thumb, Optimize recommends waiting two weeks before choosing a winning variant, and the tool itself generally doesn’t tell you a winner earlier than this.

In the world of e-commerce, two weeks isn’t even that long. You’ll be able to capture any weekly cycle that may affect visitor behavior (like quiet Fridays and wild Mondays). But in countries like Denmark where people are generally paid once monthly, you won’t capture any effect that paydays may have on behavior. So sticking to at least two weeks is not a bad idea. After all, we are testing compound pages so it won’t hurt to be completely sure about the impact your test has.

I’ll be updating this post as I dig deeper into testing with Shopify and Google Optimize. If you have any questions or feedback for this post, please reach out.

--

--

Lasse Rosendahl Ravn
Lasse Rosendahl Ravn

Written by Lasse Rosendahl Ravn

Passionate about most things digital! Sharing what I know and am learning over at lssrvn.com

Responses (1)