Measuring Cup

[Shmuel]

We added the conversion code in April. Now it's 6 months later and we are taking a very granular look at our data, now that we have some history. Our in-house tracking was used for years before, but was only able to show new customers by adgroup, not by keyword.

So we have all this data. We played with the numbers for this 6-month period (adding in our own sales) to arrive at an acceptable CPA...the point at which we would earn profit or brake even. With Google's CPA tool, I set my max per adgroup within one of my campaigns. This was two weeks ago.

I began checking up on my changes yesterday and came across an issue which I, and then we (my director) could not come up with a good answer. How much time is a good measure? It seems like a simple question. But we all know day to day the CPA of a keyword can rise and fall dramatically. The same thing can be said week to week, two-week to two-week, etc.

So what is the right amount of time to look at? Let's say 3 months is a fair test. We have many adgroups, which in that time frame, would not receive 25 clicks, just because the keywords in that group do not have a lot of traffic. Maybe we'll get one conversion a quarter...maybe two a year. So is 25 clicks, spread out over 10 keywords enough to see if any particular keyword is one we should keep or one we get rid of?

Next issue...Let's say we have an acceptable CPA of $12.00 for one ad group. One of the keywords in that adgroup has cost $8, but has no conversions. Is it worth it to keep that keyword on, or is it just a drain on my CPA?

So what are your thoughts on this? What is an acceptable time frame to measure? Do you require a certain number of clicks? When is it time to cut a keyword off? Am I looking at things to granularly?

[abbottsys]

.....So is 25 clicks, spread out over 10 keywords enough to see if any particular keyword is one we should keep or one we get rid of?...
This is a classic question in statistical accuracy. The answer is almost certainly no, 25 clicks over 10 keywords maybe nowhere near enough clicks *but* it depends on the way the clicks are *distributed* over the 10 keywords.

Here's a rough way to calculate the error:

Suppose you had 1,000 clicks over 10 keywords, and one of the keywords got 100 clicks. That's 100/1,000 = 10% of all the clicks. So, what's the statistical error on this 10%?

Well, a rule of thumb is that the statistical error is sqrt(100)/1,000 = 10/1,000 = 1%. So your answer is 10% plus or minus 1%, and that may be accurate enough, depending on how the same calculation comes out for the other keywords.

Just play with the numbers above to get something you're happy with.

[Shmuel]

That would work if there was such a variance. Unfortunately, the clicks are distributed evenly over the keywords. Such is my predicament.

[SearchEnginePPC]

I have no idea the answer to your question, but I know that MarketingExperiments.com deals with the issue of statistical accuracy quite a bit. In fact, they've put together a certification course on the subject. I took their course on Landing Page Optimization and thought it was excellent, so you might give that a shot if you want to get in depth.

If not, they have articles and free webinars on the subject all the time. Here was one of their recent articles on the subject of validity: http://www.marketingexperiments.com/improving-website-conversion/optimization-validity-threats.html.

[abbottsys]

That would work if there was such a variance. Unfortunately, the clicks are distributed evenly over the keywords. Such is my predicament.
Well, that's my point. The method still works. So, in this case when you calculate the error for each keyword (as per my formula) you'll notice they all fall within the same range and you cannot distinguish between them. The numbers are then telling you that 25 clicks is not enough and you need far more.