I’ve posted a few times here about the problems with display ads and ideas about ways to measure their true effect.

Part of the problem is that web designers marginalize them — quite literally, by lining them up in the right column — and people have learned to ignore them. If you look at those funny charts that track where people actually look at websites, you’ll notice that people avoid the ads.

But ads have some effect. The problem for the advertiser is in figuring just how much.

It’s easy to measure a click, but very few people click on ads.

So the folk who sell ads have come up with a new metric. They want you to measure the effect of their ads based on views, because studies have shown that people who see an ad are more likely to type in your URL or search on your brand terms. IOW, clicks underestimate the effect of the ad. Hence the “view-through conversion.”

But there is no such thing as a “view-through conversion,” there’s only a “display then convert.” Attributing a conversion to the fact that the ad was displayed is a leap of faith. The person might have responded to some other marketing effort.

So how do you test the real effect of the ad? Clicks undervalue them and “views” overvalue them.

The first requirement is that you have a defined audience.

I’ve had countless conversations with ad salesmen who try to tell me they can test the effectiveness of their ads by comparing the people who see the ad with the people who don’t see the ad.

I often wonder if these people failed elementary school math.

The effectiveness of an ad is measured as a percentage — total sales divided by the number of people promoted. Many systems can distinguish “conversions from people who have seen my ad” from “conversions from people who have not seen my ad,” but in order to get a useful percentage they have to also tell you how many people were in both groups. And they usually can’t.

This means that the first step in measuring the effectiveness of display ads is to define the size of your test groups. There may be other ways to do this, but the way I chose was re-targeting.

In a re-targeting campaign, the universe is defined by the number of people who visit a site and get the re-targeting cookie. The sequence goes like this. A person visits your site and gets the cookie. He visits some other site in the ad network and sees your display ad. He comes back to your site and buys something, and the conversion tracking on your “thank-you” page shows a success.

To test if display ads really work, here’s what you need to have.

1. a defined universe that you can divide into subgroups — like people who visit your site.
2. a way to divide that universe into subgroups and treat the subgroups differently — i.e., display ads to one group and not the other.
3. a way to track conversions and map those conversions back to the subgroup.

Re-targeting is the perfect way to do this. First, you put a re-targeting cookie in the browser of all the visitors to your site and put conversion tracking on your “thank you” page.

Then you set up a campaign. Half of the people who visit your site go out into the wide world and see ads to bring them back. The other half of them see some irrelevant ad, like a public service announcement.

This kind of a test allows you to compare conversions in the group that saw the ad against conversions in the group that didn’t.

I recently did such a test and found that the group that saw the ads converted at three times the rate of the group that didn’t see the ad.

This tells me that I can value 66% of all the view-through conversions.

IOW, let’s say I do a re-targeting campaign where group A sees irrelevant ads, and group B sees ads designed to bring them back to my site. 100 people from group A convert, but 300 people from group B convert.

That tells me that 100 people were going to convert anyway, but that the ads brought in another 200 conversions.

So 2 out of 3 of the conversions in the “they saw an ad” group can be directly attributed to the effect of the ad.