I’m a big proponent of adaptive web design and conversion rate optimization.  I love the tools that are out there such as Google’s ex Opmitizer, Visual Website Optimizer and Unbounce. But one thing I’ve learned over time using these tools is that they are subject to making inaccurate claims about a winner, giving the illusion that one concept is better than another.

Beware of Your Conversion Results

The primary issue I’ve found is that these tools are of course just software that follow rules that a programmer has given them. They can be fooled into thinking there is a winner if there is a lot of traffic and one of the designs is converting a lot faster than the other design.  In many cases, the tool you use is probably right.  But I’ve found that sometimes, if you wait long enough, those early winners are not really winners.

What happens, especially on sites that don’t change much, that have a loyal fan base, introducing something new is like bugs to a zapper. It is very attractive because it catches their eye and they wonder what it is and click through. So early on it will easily win out in this scenario and the optimization tool will declare a winner. But if you keep testing longer, if visitors don’t really care about the change, you’ll see conversions drop back down to normal levels.  Its very possible that the new design will win, but not at the initial levels, but it is also possible that it is worse than previously.’

And If You Are Testing Your E-commerce …

Another similar problem arises of when you run a test, especially for e-commerce websites. Just because something converts better during one part of the year, does not mean it will necessarily convert better at another time of the year.  For instance, the use of different colors on the site (green and red during Christmas) may work great during the rush, but wouldn’t work well at other points in the year. Its about timing of what you test and recognizing that different things work different based on time of year and geographic region.

A Simple Fix

So, the result of this is that you shouldn’t stop the testing if you get an early winner. Keep testing longer so that it normalizes and you get out all the bugs. Then you may want to try the test based on time of year or geography to help drive up those sales.  A little bit of personalization can really impact the bottom line in adaptive web design.