Split Testing Tutorial for WordPress Users and Everyone Else

Split testing is one of those things that you simply must do on your site if you want to make more money. Conversion make you money, and split testing helps you optimize your site to convert traffic into sales.

It doesn’t matter if you are a vendor or an affiliate. You need to convince people to take action while on your site. Whether that be clicking the “buy now” button, or joining your mailing list, or whatever. For affiliates, often the goal is to get the visitor to take action by clicking a link to a vendor’s website.

If you use WordPress – don’t worry! I’ve got you covered in this tutorial. It’s incredibly easy to do A/B testing on any type of website these days. You don’t need to know anything about code.

What Exactly is Split Testing?

The word “split” describes what you’re doing with your traffic. You’re dividing it up, and sending each chunk of traffic to different versions of your website.

Let’s go through a quick example. Headlines are a good item to test. Say you have an idea for a new, potentially winning headline. How do you know if it is a winner?

You’d conduct a basic A/B split test. Setup a test so that half of your traffic sees headline A, and the other half sees Headline B. Those are called the “original” and “variation” pages in the experiment.

You also need to tell your split testing software how to know when you got a conversion. If you’re selling something, then a “conversion” is when the visitor lands on your thank you page, or your download page.

There are lots of ways to complicate this, but let’s NOT. Let’s keep it simple. I’ll only refer to A/B testing in this tutorial. Why simple? Because most people do nothing. They don’t split test. They are flushing money down the toilet.

I want you to at least start testing something! Start realizing the power of doing split testing on a continual basis. Then, once you’re hooked, you will have enough knowledge to go figure out the more complex tests.

Split Testing Tools: Free Split Testing Tools vs. Paid Testing Tools

There are free tools and then there are (much better) paid tools you can use for split testing. I’ll discuss both, and I’ll make sure that WordPress users walk away with a good understanding of their options.

Free split testing can be complicated in WordPress

Let’s start with a discussion of the most poplar free tool on the market. It’s called Google Website Optimizer (or GWO for short)

May 2012 UPDATE:  Google has rolled split testing into Google Analytics.  It’s called Google Content Experiments and it still works much the same as Google Website Optimizer.  See below for more info.  

Truth be told, it’s rather intimidating for a first time user. But once you understand how it works, it’s quite easy to use. It’s just that it was not designed with WordPress in mind. I’ll get to that in a moment.

To use GWO, you need to create two pages that you want to test. Say you were using WordPress and you wanted to test out a new headline. You would make a copy of your sales page, modify the headline, and then save the new page with a new name. Obviously the page would also have a new URL.

You would then go into the GWO dashboard and create a new test. Google will ask you for the URL of Page A and the URL of Page B. It will also ask you for the URL of your “conversion page”, which would be a sale confirmation page in this case.

Next, Google will spit out some code. It will tell you what code to paste onto what pages. If you are using WordPress, this can be complicated. So you either need to use a plugin to help you get the right code in the right place, or you can use a theme such as OptimizePress that has support for GWO built-in (this blog is based on OptimizePress).

Either way, split testing in WordPress isn’t completely free when you need to buy a plugin or a theme that helps you accomplish what you want to do. That said, website owners who are doing ecommerce should own a high quality theme, and OptimizePress certainly fits the bill. For the price, you really can’t beat it. If you plan to use GWO to do split tests, you should get this theme.


Google Content Experiments

Google has now integrated split testing into Google Analytics.  Since we are using Visual Website Optimizer for all of our split testing, we haven’t tried Google’s updated tool yet.  But the code installation process looks to be the SAME, so it should still work with themes that support Google Website Optimizer, such as OptimizePress.

Here’s a video showing Google’s UPDATED tool.  Note that the old Google Website Optimizer is no longer going to function after August 1st, 2012.


Paid Spit Test Tools: Visual Website Optimizer and Optimizely

There are quite a few paid tools available but I’m going to save you the trouble of looking at all of them. The only two I’d recommend right now are Visual Website Optimizer and another (newer) competitor called Optimizely. You’ll often here Visual Website Optimizer abbreviated as VWO, which I’ll use from this point forward.

Both of these players offer amazingly simple split testing the plays nice with WordPress or any other type of website.

I actually heard about VWO from Andre Chaperon, the autoresponder guru. He directly recommended it to me in a message exchange, and I am a very happy user of this service today.

I also have to admit that I love the “underdog” story surrounding VWO. The company that runs VWO is called Wingify, and it was founded by a young (under 30) software engineer in India named Paras Chopra. He build this split testing tool from the ground up. He bootstrapped the company himself, and eventually left his day job to commit to VWO full time. Within a week of launching the service, he has more than enough customers to replace his full time income.

Visual Website OptimizerVisual Website Optimizer now has customers like Microsoft, AWeber, Groupon, 99Designs and HostGator. But they absolutely cater to small website owners too. In fact, when, to write this tutorial I signed up for a free month. I ran into a few questions because my WordPress website relied on a third party sales script (DL Guard). I submitted my question and the Chief Technology Officer replied to me within 20 minutes.

I decided to test these guys out to see how good their service would be after I kept hammering them with questions. Truth be told, I didn’t need to ask all the questions I asked. Their service is incredibly easy to use. But I wanted to accurately report on my findings.

The VWO team impressed me with incredibly detailed and FAST replies every time. I never waited more than about 5 or 6 hours for a reply. Unbelievable. That’s service. That’s what won me over.

Setting up a Split Test in Visual Website Optimizer

Let me say that once you read this tutorial, you should just go implement your own split test using VWO even if you’re not ready to buy it yet. The first month is completely free and they do not ask you for any kind of payment. There really are no strings. It’s a total no-brainer.

So here’s what happens once you setup your free account. You create a new test and select the URL of the page that you want to test. This will be a page on your website. Say you select your main sales page.

Next, your actual website loads up into a window and you can visually edit any part of the page that you want to edit. You”ll be visually editing your page to create the variation that you want to test against the original.

Want to change the headline? Just click on it and edit the text. Or edit the font. Whatever you want. Feel like changing your “Add to Cart” button? Click on it and change the image. Done.

Once you finish making changes for your new test page, you’ll be asked to select a goal. If your goal is to make a sale, then you just give VWO the URL of your sale confirmation page (thank you page, download page, whatever).

Finally, VWO will give you a snipped of code to paste on your website. You only ever have to do this one time and you can stick the code right in the header section of every page on your site. This makes it VERY WordPress friendly because most themes have a place for you to paste extra tracking code or scripts. If this part still confuses you, watch the included videos, or go find someone to do this for you. It’s dead simple.

Visual Website Optimizer Pricing:

As I said before, the first month of service is 100% free. They do not ask you for any payment information at all. In fact, after your 30 days runs out you can buy more testing time for $10, which is a brilliant convenience. They’ve completely eliminated the barriers to making YOU more money with split testing. Just get on board. You’ll thank me.

Once you are ready to sign up to a plan, they start at $49 per month, letting you test up to 10,000 visitors per month. A visitor is only counted if he or she is part of your test. Say your site gets 1000 visitors per day but only 100 of these people visit your sales page per day. If your test is on the sales page, you are only “testing” 100 visitors per day, not 1000.

The math makes this a no-brainer. You pay about half a penny (.005 cents) per visitor that you test. Imagine you’re selling a $47 product. If you convert at 1%, each visitor is worth 47 cents. If it costs you only half a cent to test something that might be worth a 10% (or more) boost in sales, it’s a no brainer. You are throwing money away if you are not testing constantly. It is persistent testing that could lead you to reach a conversion rate of 2%, 3% or even more.

Put another way, say you paid $49 per month for 6 months and achieved a conversion boost from 1% to 2%, doubling your sales. If you were making $47 per day now you’re making $94 every single day. You think that might be worth paying for a high quality testing service? You bet it is.

Obviously if you’re already doing hundreds of dollars a day in sales, the math becomes so insanely in your favor that you would have to be insane not to be testing.

Optimizely: The Other Visual Split Testing Startup on the Block

Sometime after Visual Website Optimizer hit the market, two ex-Google employees decided to team up and co-found Optimizely in Silicon Valley. But these guys didn’t bootstrap the company. Instead they managed to raise a million dollars in venture funding. A million bucks is actually a very small amount of money for a startup company to raise, but for the type of company that Optimizely is, this is a fantastic start.

Co-founders Dan Siroker and Pete Koomen have done a great job building an entirely valid competitor to Visual Website Optimizer.

Full disclosure here: I have tested out their system but I am not a paying customer. I never actually got to the point of setting up a live test using Optimizely because I didn’t like their policy of requiring me to give them a credit card before going live.

Yes, Optimizely still gives you a free trial for one month. That’s great. But the evening that I was sitting at my office computer setting up a split test, I didn’t have my wallet in hand. I didn’t feel like going to get it since I had already experienced the beauty (that is VWO) who doesn’t ask for any payment details.

Instead of grabbing my wallet, I fired off an email to the Optimizely support team telling them that I thought they’d get more customers if they didn’t ask for a credit card to begin a live test. They actually replied quickly, and the sales person offered to put my test account on his personal credit card. I accepted his offer, but nothing happened. My account still sits in limbo. I’m sure if I email them to bug them about it, they’ll activate my account. But at this point I’m so darn happy with VWO, that I don’t need to. Besides, I’ve done a ton of reading and I think I have a good grasp of what differentiates Optimizely for the typical Internet Marketer customer.

What Differentiates Optimizely?

These are the few items that I think could make someone want to use Optimizely instead of VWO.

  1. Their cheapest plan costs only $19 per month. But you can only test 2000 visitors per month on this plan, which should be enough to yield one statistically valid result. It isn’t enough to test all the traffic to your sales page unless you get very little traffic.
  2. The code snipped they give you is uber-simple. It’s literally one line of JavaScript. You copy and paste it onto your site one time and you’re finished. VWO’s code snipped is several lines. It hardly makes a difference, but there is something elegant about having to only past a single line of code.
  3. Their visual editor might be a little bit better. I haven’t played with it enough to really be sure, but I think that they added more simplicity to the interface. Considering they had to compete against VWO, this makes sense. But I will tell you that since Optimizely launched, VWO has added a lot of new features. Neither of these guys are standing still.
Bottom line – you’ll be good with either Visual Website Optimizer or Optimizely. They are both incredibly powerful and financial no-brainer decisions. Just get one of them and make more money from your website.

Final Word on Split Testing: Understanding Statistics

I really want to make sure you understand how important it is to keep a test running long enough to know if you have a real winner or not.

Say you ran a test with 1000 total visitors to a test. That would be 500 visitors to the control, and 500 visitors to the variation page.

What if the results came back showing that you had 5 sales from the control and 7 sales from the test variation. Does that mean you have a winner?

Most people would think so. Hey, 7 versus 5 means a 40% improvement! That’s a winner, right? Not so fast. Any good split test calculator will tell you there is no clear winner YET.

You’d actually need about 28,000 visitors to get an accurate result based on those conversion numbers.

In fact, as crazy as it sounds, you would need to convert 14 out of 500 (2.8%) to officially “beat” the original page converting at 5 out of 500 (1%). Anything less doesn’t give you a high enough statistical confidence level.

That’s why split tests need to be run long enough so that your results are statistically valid. Google, VWO and Optimizely will all report statistics to you, showing you the percentage chance that you have a winner on your hands.

The bottom line is make sure you use a proper decision making tool to decide on a winner. Looking at the numbers and thinking to yourself, “Oh, I think this is the winner” is a fool’s way to test.

I highly recommend you grab your free 30 day trial of Visual Website Optimizer. There is no credit card required to start.

Tags: , , , , , , ,

Leave A Reply (9 comments so far)

  1. Laurent
    9 years ago

    Thanks for this. I am in process of deciding on an A/B testing tool. I was also looking at http://www.unbounce.com which has a very slick product as well. The price is higher than VWO.

    BTW I also use Optimize Press for my site. Love it.

    • Chris
      9 years ago

      Hi Laurent – thanks for your comment. Yeah, I think unbounce has been around longer but VWO has created many more features. Either way, it’s SO important to split test. I’ve now been running a few tests in parallel since publishing that tutorial. VWO has made me money, no question. I’m a big big fan.

  2. Rose
    9 years ago

    Thanks for the video. Is it as easy to use in Squarespace as in WordPress? And if so could you explain it. Thanks!

    • Chris
      9 years ago

      Hi Rose – yup, it will work perfectly with SquareSpace (nice tool also, BTW). The VWO editor just looks at the live code and doesn’t care how it is generated. So whether it be wordpress, a static page, or another system like Weebly or Squarespace … it works for them all.

  3. Chris
    9 years ago

    This post was just updated to accommodate the changes made by Google (namely, Website Optimizer is going away, and there is now something called Google Content Experiments inside of Google Analytics.

  4. Oğuz
    9 years ago

    Very Cool Thread!

    Thanks for the great info’s.. and really liked the way you teach.

    I wonder if this split testing thing work with google rankings, too..

    Most of my traffic is coming from google and I have a static home page post, (which is my sales letter and not indexed in google) and if I begin to test things on it..

    Does this effect my rankings?

    • Chris
      9 years ago

      I don’t think split testing will affect your SEO / organic rankings unless you’re making radical changes that change how Google sees the content on your page. If you’re tweaking headlines and graphics, or calls to action .. I think you’ll be fine.

  5. Kirito
    8 years ago

    Hey Chris,

    I was wondering if Prosper202 tracking is compatible with OptimizePress? I need Prosper202 to track URLs in my PPV campaign. However, I don’t know of a way that I can use Prosper202 to split test my OptimizePress landing pages. Perhaps you have some insight into that?

    Thanks, please email me back at the email I provided!

    • Chris
      8 years ago

      How does Propser202 work? I need more insight into what this software asks you to insert into your WordPress pages to help.