Sometimes an experiment just keeps dragging on because the difference between the winning and losing page(s) hasn’t passed Google’s criteria for significance. Generally speaking, I think it’s a good idea to keep going until Google announces a winner, but sometimes the pattern is clear, and you really want to move along to your next test, …
Continue reading ‘When it’s safe to stop an unfinished optimizer experiment’ »
Good marketers know that their opinion on a particular color or design or choice of wording doesn’t matter. What matters is how the market responds. So marketers like to test things. Which headline works best? Does a starburst announcing a risk-free trial help or hurt response? Is it better to offer a special report as …
I worked with Bob on some of this material. It’s good stuff. See Bob Brady’s 10 Google Website Optimizer Best Practices
I don’t know how Google’s website optimizer picks the winning combination on a multivariate test. It’s some complicated statistics that I don’t know and probably don’t want to know. The trouble is that the margin for error on a complicated experiment (i.e., an experiment with a lot of options) sometimes overwhelms the results. For example, …
Continue reading ‘Error bars and conversion rates in optimizer experiments’ »