When it’s safe to stop an unfinished optimizer experiment

Sometimes an experiment just keeps dragging on because the difference between the winning and losing page(s) hasn’t passed Google’s criteria for significance.

Generally speaking, I think it’s a good idea to keep going until Google announces a winner, but sometimes the pattern is clear, and you really want to move along to your next test, so it’s worth it to call it done.

Here’s what I mean.

optimizer_experiment

I don’t think anything’s going to change in this horse race.

optimizer_experiment2

The winning page isn’t a slam dunk, it but it’s good enough to call this race over and move on to the next idea.

Leave a Reply

Your email address will not be published. Required fields are marked *

ten − nine =