Should we push new collections or new colors of evergreen collections on our spill pages?
Should we offer free shipping or 10% off?
Should we use photography where the model is looking at the camera or off to the side?
Should we put the cancel button next to the OK button or below it?
Each of these questions and hundreds of others could be debated until the most persuasive (or most uncompromising) person in the room gets her way, or the questions could be modeled in an A/B test and the loser can buy drinks.
As with anything, there are the “haves” and the “have nots” when it comes to multivariate testing platforms. You might assume that the giants – the companies in the top 100 e-Commerce sites – all have this technology, but based on our experience consulting with them, I can tell you that you’d be wrong. What’s even weirder is that even when a company has a fancy analytics or multivariate testing platform they commonly don’t use it for understanding search. Why is that? In many cases, they tell us they simply don’t know what to measure or test. In this article we’ll explore a few basic ideas about what to measure and what to test in e-Commerce onsite search.