When it’s safe to stop an unfinished optimizer experiment
February 26th, 2010
Sometimes an experiment just keeps dragging on because the difference between the winning and losing page(s) hasn’t passed Google’s criteria for significance.
Generally speaking, I think it’s a good idea to keep going until Google announces a winner, but sometimes the pattern is clear, and you really want to move along to your next test, so it’s worth it to call it done.
Here’s what I mean.

I don’t think anything’s going to change in this horse race.

The winning page isn’t a slam dunk, it but it’s good enough to call this race over and move on to the next idea.
Recent Comments