A/B Testing Case Studies for Volusion from Brand LabsThere is no more overused phrase in this industry than “A/B testing”.  Our previous post is testament to that.  However, that doesn’t mean that all the attention is undeserved.  A/B testing is a truly valuable tool in the hands of skilled web professionals, like Brand Labs, for instance.

We have a well-established track record as effective online marketers.  Our reputation in this arena is matched only by our reputation as skilled web designers, which is a good thing because you need both skills in order to build and maintain a strong A/B testing program and to react properly to the results.  Oh… speaking of results, it helps if you’re Google Analytics experts like we happen to be.  That way, when you see the results you’ll know what to do with them, which is good because knowledge without action is like Hall without Oates; useless.

After the jump we have some recent examples of A/B successes and some tips if you’d like to follow our example.  Hit the link below to check it out…


What you see here is a landing page for one of our oldest clients, The Shelving Store.  It offers a lot of “best practice” elements.  It has strong lifestyle photography, highly-placed SEO copy (which contributes to  TSS’s number one ranking for the search term “chrome wire shelving”) and a clear & prominent left navigation menu.  By many measures, it’s a very strong landing page, but since we aren’t wild about resting on a top Google rank, we decided to explore additional conversion possibilities for the client.

First, we examined the viewing pattern for the average user and determined that several critical elements fell outside the hot zone.  Knowing full well that you have precious few seconds to capture a shopper’s attention, we began by shifting some of the high-value assets to the hot zone.  In order to do that, we sacrificed the lifestyle photography in favor of some clear buzzword placement and more prominent placement of important products from the category.


The result of our efforts is here.  You’ll see that we’ve removed the left nav, that the entire page has been simplified and the calls to action are far more prominent.

Unfortunately, it isn’t enough to simply shuffle page elements with the hope that you’ll see improved results.  The fact is that even if you were to see an uptick in conversions, without proper A/B testing you’d have no way to know whether your changes were effective or if the increase was entirely coincidental.  That’s where our Marketing team takes the reins from the Creative department.

Using identical paid search campaigns, we sent 50% of TSS’s paid traffic to the old page and 50% to the new page.  We tracked the results in Google Analytics over a 30 day period and our findings were dramatic.  Conversions resulting from direct traffic to the new page showed an 86% increase over the same amount of traffic going to the old page.  From a straight-up statistical standpoint, that is a truly ridiculous number, but the analytics don’t lie.

The takeaway from this particular test is that best practices are an excellent starting point (even before the A/B testing, the original landing page was converting very well), but if you’re hoping to increase your conversions, you’ll have to clearly define the goals of your specific business and use that knowledge to inform your decisions about your chosen practices.  The specifics of the approach we’ve taken here, likely wouldn’t work for an apparel dealer, for example.  It’s all about testing and a deep knowledge of the business model in question.  Speaking of unique business models…

The Red Hat Society Store is unique in a couple of ways.  First, it is actually “unique” in the literal sense of the word (it is the only official store of the world’s largest social club – The Red Hat Society).  Second, it happens to be our Volusion store.  We’ve owned and operated the site since it was RedHatsAndMore.com.  In that time, we’ve guided it from its founding, through a truly saturated field of competitors to the top slot in our vertical, eventually partnering with The Red Hat Society and taking over all of their eCommerce operations.  We were able to accomplish this retail feat through a constant re-examination of the market and of the needs of our customers.

… and, testing, testing, testing.

What you see here is the original category page for “hats”.  You’ll notice that it’s a pretty straightforward design that relies on clear, vibrant product photography to instantly communicate the contents of each sub-category, and while the site was selling a helluva lot of hats, analytics told us the bounce rate from this particular page was surprisingly high.

This is where our deep knowledge of our personal vertical comes in.  While we have a full understanding that being 50+ doesn’t make you computer illiterate, it must be said that we handle a disproportionately large number of calls that have solely to do with the basic aspects of completing an online purchase.  Knowing that a good number of our clients are eCommerce virgins, it pays for us to over-explain the process whenever possible.

With all of that in mind, we speculated that the high bounce rate had to do with the possibility that an inexperienced eConsumer might land on the category page and, seeing only individual hats, they might easily believe that we only carried a single style in each color.  If they didn’t like one of the three hats on the page, they’d bounce.  Our solution was simple:  we created sub-category graphics that each showed two hats and added a “View All” call to action (indicating there was more to be seen).  In our two-month testing period, the new page yeilded a 48.7% increase in clickthroughs to the sub-categories and the average time on the site increased by 57%.

The takeaway here is that the “best practice” for your site may not exist in the giant spectrum of “best practices” that are floating around out there.  In this case, as in most, it’s about knowing your customer, tracking their onsite behaviors and creative speculation based on that information.  Now if you don’t happen to have a team of experienced eCommerce pros on your staff, the creative speculation may be a little more challenging, but that’s the beauty of A/B testing.  It allows you to make educated guesses and then determine whether or not your instincts were correct.  After that, you simply proceed with the most successful option… and then you repeat the process indefinitely for the entire time that you operate your store.

That last bit is the part that often stops eMerchants in their tracks.  No one enjoys the prospect of a never-ending process of any kind.  People like definable, achievable goals, which is why we suggest that you see A/B testing as a series of small goals as opposed to a daunting and ceaseless task.  We even have a suggestion for a simple starting point for you.

This client (who prefers to remain anonymous) was showing a high bounce rate on their cart page.  As every eMerchant knows, abandoned carts are among the most costly stumbling blocks for any business, and there are excellent ways to reclaim that revenue, but the shortest route to conversion is always the best route.  Knowing that, we decided to experiment with the over-explanation method.  We worked with the client to determine their greatest value propositions for cutomers who have already identified what they intend to buy.  We decided that their expedited shipping and liberal return policies were a clear standout in their vertical, so we simply added a graphic that called out those policies and placed it on the cart page.  In the month-long test the client saw a 12.77% increase in conversions originating from the cart page with the additional callout.

Even though every market is different, there are some things that are pretty universal.  One of those things is the fact that anyone reaching the shopping cart page of your site is probably looking for one really good reason to complete their purchase… a little push to convince them to convert.  If you can isolate your greatest value proposition and place it in the shopper’s line of sight as they make their final decision, it simply makes sense that they’re a bit more likely to finish what they started.  At the very least, it’s an excellent jumping-off point for your entry into the world of A/B testing.

Of course, we understand that a lot of the things that we covered in an offhanded fashion here aren’t always that easy to achieve on your own.  Some of the alterations we’ve referred to here involve a lot of creative and development hours, but that doesn’t mean that you shouldn’t attempt some version of the methods we’ve discussed, even if it’s a scaled-down effort or you have to enlist outside help…

hey It totally just occurred to us that we could certainly help you with your effort.  If you’d like to embark on an online marketing adventure that’s designed to increase conversions and revenue, you should give us a call.  We’re ready to add your company to our ever-growing list of success stories.