Testing Is a Waste of Time

Testing seems to be the answer to everything advertising-related these days. When an audience member asks a tough question at a conference, a panel member inevitably says, “You have to test.”

Many of the things that we want to know or try requires that we set up a test. Tests typically need to run for a period that is twice as long as your average sales cycle (i.e., if your average sales cycle is seven days, it would be best to let that test run for a full two weeks). This allows the ads that ran on the seventh day of the test a full sales cycle, which is enough time to fairly judge the results.

Waste of Money

Testing is usually a waste of money. Tests just prove that what was tried isn’t effective. While there’s certainly value in learning something isn’t effective, the time and money that went into the test could’ve been spent on something you already know is effective.

Other times, we get lucky and the test proves to really move the needle. Unfortunately, there’s no way to know ahead of time if a test will succeed.

Because testing is a necessary practice, anything we can do to shorten the time needed to get the results of the test is a worthwhile exercise. The more time you’re out of testing mode and in production mode, the more profitable you will be.

Waste of Time

One of the most difficult ad formats to test is display advertising. The simplest method of testing display, which isn’t highly effective, is a simple A/B test.

In this type of test, you turn on display advertising and you take a look at your end results. You then turn off your display advertising, look at your end results, and compare the two. If your end results were better when display was on, then you can conclude display works.

The problem with this type of testing is it doesn’t provide any real detail. It’s highly likely that some of the display ads you ran in this test didn’t work. It’s also highly likely that some of the ad networks involved in this test didn’t work.

Why A/B Fails

If you wanted to figure out all of the beneficial elements, you would need to run a large amount of A/B tests, which would take an extraordinary amount of time, as shown in the example below:

In this chart, we’re testing two ad networks. Each network has two web sites within it, and we’re testing two creatives per site per network. The sales cycle for this company is seven days; therefore, the test cycle needs to be 14 days.

In order to isolate each element of this test to determine which elements are truly beneficial, you would need 14 tests. That would take 196 days to perform.

It’s also important to note that there are other elements you may want to consider that aren’t included in this test, such as ad sizes, demographics, and psychographics. In reality, there would be many more networks, many more sites, and a lot more creatives to test.

Attribution Management: Get Conclusive Results

So how can you test display at a granular level and greatly reduce the time needed to get conclusive results? The answer is attribution management.

In order to do attribution management, one needs to track the purchase path that the consumer navigated en route to conversion. The purchase path would show which display impressions a consumer is exposed to (view-throughs), in addition to any ad or organic link they click on along the way.

Once the paths are tracked and the conversion value (profit, revenue, orders, leads, etc.) is calculated, it would then be attributed to each step in the path in some manner.

The ideal metric for attribution is profit. By assigning profit to each step in the path, we can quickly gauge which ads and networks are contributing because any ad or network that has profit tied to it is working and any that has negative profit is not contributing.

Giving Credit Where It’s Due

In the example above, we said it would take 196 days to test all of the elements of this display campaign. By doing attribution management and tracking the purchase path, this same test can be completed in just 14 days.

Without attribution technology in place, one needs to A/B test each variable independently of the other, which takes a tremendous amount of time. With an attribution model in place, one can test variables concurrently, thus greatly reducing the amount of time needed to produce the answers to your test.

With these in place, at the end of 14 days, we’ll easily know which ads are the most and least profitable, which ad networks are most and least profitable, and we’ll also have insight into demographics, creative size, time, day, etc., and could even overlay psychographic information.

Attribution management is almost always defined simply as giving credit where credit is due. However, attribution and the tracking required to do it is beneficial in many other ways, such as the ability to greatly reduce the time one is in testing mode.

A lot of other benefits aren’t solely about giving credit where credit is due. You can learn about other benefits by viewing this recent webinar, “Attribution’s Other Actionable Benefits.”

Join us for SES Chicago 2010, the Leading Search & Social Marketing Event, taking place October 18-22! The conference offers 70+ sessions on topics including PPC management, keyword research, SEO, social media, local, mobile, link building, duplicate content, multiple site issues, video optimization, site optimization, usability and more.