Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engine Marketing Strategies > Search Engine Advertising
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 05-18-2006   #1
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
The ROI from SEM can not be measured accurately

I am in a debate where I need to prove the ROI from our SEM program is real and not just smoke and mirrors.

The problem is… I’m loosing the argument. I truly believe that cutting the budget would do much more harm than good but I can’t figure out how to prove it.

The arguments I’m up against are just too simple.

In this thread, I will play the Devils Advocate and use these simple arguments
to debate with anyone who thinks they can prove these arguments wrong. Nothing would make me happier than if someone can do just that.

First some background information to help frame the debate.

• The website is for a large brand that receives a large volume of visits and transactions

• The goal is to drive sales on the website as these direct transactions are the most profitable for the company

• SEM is only one of several dozen channels of product distribution

• There are dozens of ways to reach the website other than search or PPC

• The revenue contribution from SEM is significant but it’s only a fraction of the total volume of transactions

• Sales volume and visits can very greatly due to competitive conditions and seasonality. The typical % variance is grater than the usual contribution from SEM

• The SEM campaign is measured by a strict direct response metric for cost of advertising versus revenue generated. Branding and raising awareness is not a goal or even a consideration.

Here we go…

The ROI from SEM can not be measured accurately.

• The PPC ads for a particular keyword are either there or not.
• You can only track and measure the behavior of people who click on your ads.
• You can NOT track and measure the behavior of these people if there is no ad to click on.

If you can’t track and measure the people who were not served an ad, how do you know that these people are any les likely to make a purchase on the website?

Just because someone clicks on the ad and makes a purchase is no reason to assume that they would have not purchased the product anyway without the additional cost of the click.

Several, Many, Most or All of the people searching for a particular keyword will find the website with or without the help of PPC ads. Exactly how many of these people will reach the website and make a purchase is unknown.

If ALL of the people who click on the ad would have made a purchase on the website anyway, the ROI from the advertising expense would be Zero.

The % of people who would NOT make a purchase because the ad is NOT there to click on, is the true measure of the ROI. Because the behavior of people can not be tracked or measured without serving them an ad, the ROI can not be tracked or measured. (No control group)

If we don’t know if it’s working or not, than why are we spending millions of dollars on it?
Why not spend the money on something else or just save it?

XXXXXXXXXXXXXXXXXXXX

We also do quite a bit of Banner advertising but this is not perceived as smoke and mirrors because the ROI reporting is much more accurate and accountable than search due to the ability to measure test and control groups against each other for the same time period.

XXXXXXXXXXXXXXXXXXXX

We can run a test where we show half the audience a real banner and the other half the audience a bogus banner for a charitable organization or whatever.

Once you have been segmented into one group or the other you will continue to stay in that group. You will either see real banner ads everywhere you look or the bogus banners for the duration of the test or until you delete your cookies.

Several, Many, Most or All of the people who are only served the bogus banners will make a purchase anyway.

Let’s say this is the results…
7,000 people who only saw or clicked the bogus banners made a purchase anyway.
10,000 people who saw the real banners made a purchase.

The real banners in this example show a 3,000 lift in orders.
The ROI contribution of the banners would be attributed to 3,000 orders. (Not 10,000)

XXXXXXXXXXXXXXXXXXXX

This sort of test and control methodology is not available with search.
This is why the ROI can not be accurately measured.


Before you can accurately determine ROI from search you would need the ability to track people who search for a keyword or group of keywords and are NOT served a PPC ad. This will tell you how many of those people will make the purchase anyway or more importantly how many will not.

Google and the other search engines would need to allow advertisers to purchase a tracking pixel instead of the actual PPC ad for a % of the tested keyword queries. People in one group or the other would need to stay in that group and either see the PPC ads or not for the duration of the test or until they delete their cookies.

Until this happens, we have no way to determine if the millions of dollars spent on PPC is a wise decision or not. If we don’t know of there is any ROI or not, then why are we spending the money?
Reality Check is offline   Reply With Quote
Old 05-19-2006   #2
andrewgoodman
 
andrewgoodman's Avatar
 
Join Date: Jun 2004
Location: Toronto
Posts: 637
andrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to all
That was a pretty detailed post - I think the issues at stake can be boiled down pretty far, down to a couple of sentences.

Someone of good faith in the organization needs to believe that there is incremental revenue that can be created through a paid search program.

When some methodology is set up to attempt to prove that, they need to assess the information fairly and open-mindedly.

As you say, you can track clickers all you want, but if the argument becomes - "what's to say they wouldn't have bought anyway," you're fighting a rhetorical battle against someone who won't let you win the argument. It would take some pretty complicated math to *prove* that there is little chance that they would have bought otherwise.

Direct response metrics are the primary ones to consider, but there really is the cumulative effect of using all your promotional resources over time. When someone types "page zero" into a search engine or navigates directly to my site, I can't now prove that this lead was generated from any particular source. All I can do is be bullish on the value of various forms of promotion I'm using and measure them as well as I can. If I stop all promotion, guess what: no growth, fewer and fewer sales.

If the campaign is large enough, you could turn it on for a month, and then off for a month, and look at total revenues. Did you move the needle? Even here, a sceptic could chalk things up to seasonality if you were trying to dismiss the impact of paid search.

This argument could be made in connection with any form of advertising, though. Are the other methods being measured? Why would someone want proof that one method works, and not demand equal proof of the other methods' incremental value?

Some of the argument behind this may come from math proposed by analysts like Jim Novo, trying to help companies avoid "cannibalizing" their channels by overspending in PPC where they don't have to (because, let's say, they appear in organic etc.). Not all of Jim's premises hold with regard to today's search landscape, IMHO. It's haggling over nickels and dimes, usually with regard to very large companies who are quite wasteful in other areas of their advertising.

You're in a political type argument, it sounds like. Send them my book.
andrewgoodman is offline   Reply With Quote
Old 05-19-2006   #3
andrewgoodman
 
andrewgoodman's Avatar
 
Join Date: Jun 2004
Location: Toronto
Posts: 637
andrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to allandrewgoodman is a name known to all
Just to add a bit to this, it may be possible to start debunking some of these assumptions about the likelihood of purchase from various sources, by looking deeper into analytics.

For one thing, is there a broad-brush assumption that the company is getting a whole lot of juice from organic rankings. True or untrue? What is the actual monthly profile of organic referrals (not rankings), and could this be improved? Is there a ceiling for such improvement because Google et al control the SERP's and have a new, post-Florida philosophy that pushes commercial pages down the list more often than not, let alone the onebox type results that may show local search, news search, and other prompts at the top of the page, crowding you out of view?

If a company really wants answers, they'd probably get set up with Clicktracks Pro or something else so they understand more about user behavior post-click from both paid and unpaid search. If they think search of either type has little incremental value, well... I can't help you. A good user experience online is part and parcel of living in "today." So what if there are flyers and storefront locations, and TV ads for direct navigation, or whatever? When people go searching, the more often they get coherent representation of a brand, the more aware they are of your company as one with a solid online presence.

Many of my clients would not be on the map at all without PPC. They simply get no orders. In larger organizations, there can always be a political argument that says, we don't need this or that. We're doing great anyway, etc. My take on these no-win arguments is generally, you can lead a horse to water. I'll focus on the people who really want to proceed with a paid search program. There is only so much you can do to sell something to someone who isn't buying. My wife and the 100 vendors who sell metal doors can give me 3,000 facts about why a metal front door is better, and I'll still wind up with a wood front door even if I need to hire a separate contractor to tear down the metal door she buys and hang my big wood door.

At the very least do they not understand the market research value of 2-3 months of tracked paid search campaigns on a wide variety of keywords? Wouldn't they learn more about which landing pages convert if they ran systematic campaigns? Geez.
andrewgoodman is offline   Reply With Quote
Old 05-19-2006   #4
rustybrick
 
rustybrick's Avatar
 
Join Date: Jun 2004
Location: New York, USA
Posts: 2,810
rustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud ofrustybrick has much to be proud of
Summarizing the Issue

Let me see if I can accurately summarize the issue here Reality Check.

Big Brand X has PPC ads for the the keyword phrase of the Brand Name of Big Brand X.

That keyword phrase brings in $10,000 in sales per day but costs $5,000 per day on PPC spend.

You remove the keyword phrase from the campaign and you have the same gross sales on your site.

So in essence you are saving $5,000 per day.

But the tracking tools show you have a ROI of $5,000 per day.

Is that a good example?
rustybrick is offline   Reply With Quote
Old 05-19-2006   #5
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,153
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
Quote:
Originally Posted by Reality Check
I am in a debate where I need to prove the ROI from our SEM program is real and not just smoke and mirrors.

The problem is… I’m loosing the argument. I truly believe that cutting the budget would do much more harm than good but I can’t figure out how to prove it.

The arguments I’m up against are just too simple.

In this thread, I will play the Devils Advocate and use these simple arguments
to debate with anyone who thinks they can prove these arguments wrong. Nothing would make me happier than if someone can do just that.

First some background information to help frame the debate.

• The website is for a large brand that receives a large volume of visits and transactions

• The goal is to drive sales on the website as these direct transactions are the most profitable for the company

• SEM is only one of several dozen channels of product distribution

• There are dozens of ways to reach the website other than search or PPC

• The revenue contribution from SEM is significant but it’s only a fraction of the total volume of transactions

• Sales volume and visits can very greatly due to competitive conditions and seasonality. The typical % variance is grater than the usual contribution from SEM

• The SEM campaign is measured by a strict direct response metric for cost of advertising versus revenue generated. Branding and raising awareness is not a goal or even a consideration.

Here we go…
Okay I will see if I can add something here.

Quote:
The ROI from SEM can not be measured accurately.
This is the biggest false statement made in the post. SEM is the most accurately measured online marketing method. Any caveats - and blind tests - can just as easily be used as a counter for any marekting efforts. Your guys are just using theirs first and claiming it cannot be replicated.

Any advanced web analytics program can give you the results for paid and organic conversions. They can also capture secondary or initial marketying influences prior to a sale. You then need the CRM program to have the ability to keep the information and thus get numbers for long term purchasing practices for all marketing efforts.

Quote:
• The PPC ads for a particular keyword are either there or not.
Okay and if you don't pay your banner bill they don't appear - or if you do not buy all possible impressions your banner is rotated - this is an issue of camapign budget limits.
Quote:
• You can only track and measure the behavior of people who click on your ads.
Not true.... if you do not have an organic listing on the page.... you can pretty much measure the fact that all searchers did not go to your website from the search page.....

Quote:
• You can NOT track and measure the behavior of these people if there is no ad to click on.
Say the same thing in reverse always adds momentum against a proposition

Quote:
If you can’t track and measure the people who were not served an ad, how do you know that these people are any less likely to make a purchase on the website?

Sounds like the tree falling in the forrest routine....
The first hint to this is if they are about to buy and you have competitors who are advertising - you are going to lose potential customers - this is not a vacuum - there is no simple answer but you should consider the competition in there - if you have none then why bother advertising at all.


Quote:
Just because someone clicks on the ad and makes a purchase is no reason to assume that they would have not purchased the product anyway without the additional cost of the click.
All very true if you are the only provider - though alternatives would be in play if you were - but if you have competitors they get the sale!


Quote:
Several, Many, Most or All of the people searching for a particular keyword will find the website with or without the help of PPC ads. Exactly how many of these people will reach the website and make a purchase is unknown.
again this is all based on a no competition scenario

Quote:
If ALL of the people who click on the ad would have made a purchase on the website anyway, the ROI from the advertising expense would be Zero.
Again denies competition.
Quote:
The % of people who would NOT make a purchase because the ad is NOT there to click on, is the true measure of the ROI. Because the behavior of people can not be tracked or measured without serving them an ad, the ROI can not be tracked or measured. (No control group)

If we don’t know if it’s working or not, than why are we spending millions of dollars on it?
Why not spend the money on something else or just save it?
Are you starting to see the blind spot?
Quote:
XXXXXXXXXXXXXXXXXXXX

We also do quite a bit of Banner advertising but this is not perceived as smoke and mirrors because the ROI reporting is much more accurate and accountable than search due to the ability to measure test and control groups against each other for the same time period.
This can be done for search also.... run multiple ads with different landing pages.


Quote:
We can run a test where we show half the audience a real banner and the other half the audience a bogus banner for a charitable organization or whatever.

Once you have been segmented into one group or the other you will continue to stay in that group. You will either see real banner ads everywhere you look or the bogus banners for the duration of the test or until you delete your cookies.

Several, Many, Most or All of the people who are only served the bogus banners will make a purchase anyway.

Let’s say this is the results…
7,000 people who only saw or clicked the bogus banners made a purchase anyway.
10,000 people who saw the real banners made a purchase.

The real banners in this example show a 3,000 lift in orders.
The ROI contribution of the banners would be attributed to 3,000 orders. (Not 10,000)

XXXXXXXXXXXXXXXXXXXX

This sort of test and control methodology is not available with search.
This is why the ROI can not be accurately measured.
Again when your measurements start out as faulty then you can make any claims. Search can duplicate this... you can have someone cookied and watch future movements etc... claiming something for areas independent of search does not come with implicit correctness or accuracy




Quote:
Before you can accurately determine ROI from search you would need the ability to track people who search for a keyword or group of keywords and are NOT served a PPC ad. This will tell you how many of those people will make the purchase anyway or more importantly how many will not.

Google and the other search engines would need to allow advertisers to purchase a tracking pixel instead of the actual PPC ad for a % of the tested keyword queries. People in one group or the other would need to stay in that group and either see the PPC ads or not for the duration of the test or until they delete their cookies.

Until this happens, we have no way to determine if the millions of dollars spent on PPC is a wise decision or not. If we don’t know of there is any ROI or not, then why are we spending the money?
You can get the information on how many searches are performed for any keyword and then you can see if the person comes through organic if you have solid tracking and analytics - you can also cookie a user and follow other behavior.... you should not narrow the discussion by taking an aggressive stance. People must stay in one group or another?
AussieWebmaster is offline   Reply With Quote
Old 05-19-2006   #6
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Thanks Andrew!

Wow!

A response from the guy who wrote the book on the subject! I am ordering it today! I sincerely appreciate this level of help.


Point #1
“Someone of good faith in the organization needs to believe that there is incremental revenue that can be created through a paid search program.”

“When some methodology is set up to attempt to prove that, they need to assess the information fairly and open-mindedly.”

To paraphrase Yoda… It is, or it is not… There is no “Belief”.
The smart folks in finance and accounting aren’t familiar with this stuff. The team who is in charge of the traditional media is familiar enough to be dangerous and they have more influence than I do. This is not a faith based expense. It’s strictly a business decision. If it’s working, we’ll do more of it, if it’s not working we’ll significantly cut back on it. People with more influence than me say it’s very likely that it’s not working.

Point #2
“You're fighting a rhetorical battle against someone who won't let you win the argument.”

This is how I view the situation as well. I can incontrovertibly prove the banner advertising works but not search.

Point #3
If I stop all promotion, guess what: no growth, fewer and fewer sales.

Not necessarily… With all the money we spend on PPC we could probably purchase a Blimp. (Of course we would never do that) But… It could be argued that the message on the Blinp would be more cost effective at driving traffic and sales on the website than PPC because the ROI for PPC can’t be proven any better than messages on the Blimp.

Point #4
“If the campaign is large enough, you could turn it on for a month, and then off for a month, and look at total revenues. Did you move the needle?

Nope… “Sales volume and visits can very greatly due to competitive conditions and seasonality. The typical % variance is grater than the usual contribution from SEM”

We have a large budget but if we shut it all down for a month and revenue dropped by a Million dollars, this could easily be attributed to competitive conditions and seasonality. It wouldn’t make a large enough dent to raise any flags. Revenue might even go up during this time depending on what season it is. It wouldn’t be a statistically significant test.

Point #5
“This argument could be made in connection with any form of advertising, though. Are the other methods being measured? Why would someone want proof that one method works, and not demand equal proof of the other methods' incremental value?”

Nope… Per the example above I can perform much better tests with Banner ads than search. They want the same proof that I can deliver for other forms of advertisement. If I can’t provide this proof than why not buy the Blimp instead?

Point #6
“Some of the argument behind this may come from math proposed by analysts like Jim Novo, trying to help companies avoid "cannibalizing" their channels by overspending in PPC where they don't have to (because, let's say, they appear in organic etc.). Not all of Jim's premises hold with regard to today's search landscape, IMHO. It's haggling over nickels and dimes, usually with regard to very large companies who are quite wasteful in other areas of their advertising.”

”You're in a political type argument, it sounds like. Send them my book.”

The people here are smart and only want what’s best for the company. People in the finance and accounting departments don’t understand search as well as they do math and facts. These nickels and dimes quickly ad up to millions of dollars. The other problem is the decision makers were sold a story that implied search was more accountable and the ROI generated could be measured. Now they aren’t sure so this brings up the question… “Why are we doing it? We aren’t doing it for “Branding”… That’s much better done with television.

I disagree with the “political” argument somewhat… These people are smart who are trying to make the best decision they can for the grater good of the company. If they understand one advertising medium better than the other, than that’s the one that will likely win. Almost all offline advertising promotions are designed to drive people to the website. This is easy to understand.

Point #7
“Is there a broad-brush assumption that the company is getting a whole lot of juice from organic rankings. True or untrue?”

Not true… We don’t have very good organic rankings. (I’m working on it)
We do have several dozen online distributors all promoting us along with our competitors. I can talk all day about vertical creep, one box results, etc, but people’s eyes seem to start glazing over after a few seconds.

We have good analytics, but the signal to noise ratio is too great dismiss an argument that if we turn off one distribution channel the others will pick up the slack. If PPC was our only source of traffic and distribution we could do this easily but that’s not the case.

Point #8
“There is only so much you can do to sell something to someone who isn't buying.”

I agree… However I can show incontrovertible facts and math that shows a very cost effective and statically significant lift in revenue with banner advertising. It’s not a subjective decision. It works and I can prove it. Not using banner advertising would be an irresponsible business decision. On the other hand… The ROI from SEM can not be measured accurately so there is no business case for this decision. It becomes subjective. Do I like sexy Blimps or PPC text links?

Point #9
At the very least do they not understand the market research value of 2-3 months of tracked paid search campaigns on a wide variety of keywords? Wouldn't they learn more about which landing pages convert if they ran systematic campaigns? Geez.

Nope… As you pointed out earlier… I’m fighting a rhetorical battle against someone who won't let me win the argument. We can have all the landing pages we want and I can do A/B testing to make the copy of the PPC ad better to improve click thru rates and tweak the landing pages all I want but if these people would have purchased anyway or if a Blimp can do the same job cheaper than what’s the point?

Last edited by Reality Check : 05-19-2006 at 05:55 PM.
Reality Check is offline   Reply With Quote
Old 05-19-2006   #7
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,153
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
Don't think you can achieve anything purer on the banner side... PPC can be measured as tightly if not more so.
AussieWebmaster is offline   Reply With Quote
Old 05-19-2006   #8
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Thanks Barry

Thanks Barry,

Yes! That’s an excellent example of my problem!
"Technically”, the cookie sees the Click-Visit-Conversion due to the brand name keyword. Our analytics package and ad agency both report this as a Positive result. The $1.00 click created a $100.00 order.

“Technically” the cost to revenue ratio % and ROI for brand name PPC queries were excellent. However… This reporting is easily confused with "Reality"

This is what I call a “False Positive" result.

The ads may or may not be the driving factor in revenue that is reported, or people might click on ads only because they are there but they may or may not be responsible for producing the sales that are "Technically" attributed to the keyword at all.

Search engines like Google and Ad agencies get paid to sell advertising, not raise questions about the reporting limitations. The decision makers over here realize this and are skeptical of the ROI reports as they should be.

For example:
We were paying a small fortune for our brand name as a keyword which we ranked in the #1 spot for. About 1/3 of total visits from brand name queries were generated from PPC ads.

I analyzed our analytics data and determined that the % of brand name visits was a surprisingly consistent ratio week over week when compared to overall traffic generated by each major search engine. Traffic goes up and down significantly but the brand to generic ratio stays about the same.

Hypothesis:
If we shut off the PPC traffic for that keyword, our traffic ratio for that keyword should drop by 1/3

Result:
Nothing happened. The ratio did not drop by 1/3. It did not drop at all. It stayed exactly the same. We turned the keyword back on and still nothing. So we turned it back off for good and checked it again and still nothing.

Conclusion:
The positive ROI data for the keyword buy was a “False Positive Result”. We would have received the traffic and orders anyway. There was no business justification for the expense as measured by a direct response metric. The expense was eliminated because it may have had a “Branding” effect but the cost for this effect can be produced more cost effectively with television.

The methodology for test above works for our brand name but I wouldn’t recommend it for generic queries. Even if the traffic ratio for the generic keyword drops to zero, there is no way to prove that they would or would not be able to find us and make a purchase through the natural comparison shopping process.

Determining “Actual ROI” with search is almost impossible because there is no way to determine lift versus a control group who didn’t see the PPC ads during the same time period for the same keyword or list of keywords.
This test above saved us a boat load of cash but it also casts a shadow over the rest of my SEM efforts that I truly believe deliver an excellent value. It creates a difficult situation for me that I’m struggling to reconcile.

Last edited by Reality Check : 05-19-2006 at 05:00 PM.
Reality Check is offline   Reply With Quote
Old 05-19-2006   #9
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Thanks AussieWebmaster!

Point #1
“SEM is the most accurately measured online marketing method.”

Not exactly…
As demonstrated above…

With banner advertising, we can run a test where we show half the audience a real banner and the other half the audience a bogus banner for a charitable organization or whatever...

This sort of transparency of ad effectiveness is not likely to happen any time soon because many advertisers would suddenly realize the ROI they had assumed to be correct, is much less than previously thought. Others might realize it was better than they thought but allowing this sort of transparency could be risky.

Point #2
“Any caveats - and blind tests - can just as easily be used as a counter for any marketing efforts.”

I’m not sure what blind tests you are referring to. As far as I know… The ads are either there or not. You can’t segment the audience into two groups that either continue to see the ads or not for a period of time.

Point #3
“Any advanced web analytics program can give you the results for paid and organic conversions.”

“Technically” Yes… But not necessarily as demonstrated in the response to Barry.

Point #4
Okay and if you don't pay your banner bill they don't appear - or if you do not buy all possible impressions your banner is rotated - this is an issue of campaign budget limits.

We always pay the banner bill because we can prove they work. Basically, we can pay to show a bogus banner to half the audience and measure the difference in response from the two groups. You can’t do this with search.

Point #5
If you do not have an organic listing on the page.... you can pretty much measure the fact that all searchers did not go to your website from the search page.....

Yes… It is reasonable to assume that these people would have just as likely reached our website via some other method than search. They can directly enter the URL that they saw on the TV commercial that was financed from the money we saved from not participating in PPC.

Point #6
Me: “You can NOT track and measure the behavior of these people if there is no ad to click on.”
AussieWebmaster: Say the same thing in reverse always adds momentum against a proposition.

Not true because you CAN measure the behavior of people who do NOT see our banner ads. People who do not see our banner ads are much less likely to not make a purchase. We know this for a fact because we can measure both groups.

Point #7
“Sounds like the tree falling in the forrest routine....
The first hint to this is if they are about to buy and you have competitors who are advertising - you are going to lose potential customers - this is not a vacuum - there is no simple answer but you should consider the competition in there - if you have none then why bother advertising at all.”

“You are going to lose potential customers”
This is true… But exactly how many customers are we talking about? Is it one customer or all of them? This number can not be accurately determined without measuring the behavior of people not exposed to the ads.

If the number is small, it doesn’t justify spending millions of dollars to keep them. It might be more economical to save the money.

There is a lot of competition but it is unknown how many of the people who are not exposed to the ads will purchase from them.

We can run dozens of landing pages and do all the A/B testing we want but the ROI from search can not be effectively measured against a control group. If a TV commercial may or may not be more effective at persuading the customer to make a purchase then why not sacrifice PPC to give it a try?

Point #8
“Search can duplicate this... [Test and control situation available with other online media] you can have someone cookied and watch future movements etc... claiming something for areas independent of search does not come with implicit correctness or accuracy.”

I don’t believe this is true. I am not aware of a way that I can cookie a person searching for “Blue Widgets” unless I have an ad displayed and they click on it. If I could cookie them with out a PPC ad for them to click on, I could track them and determine how likely they would be to purchase our product without the help of an ad. Unfortunately, I can’t do this.

Point #9
“You can get the information on how many searches are performed for any keyword and then you can see if the person comes through organic if you have solid tracking and analytics - you can also cookie a user and follow other behavior.... you should not narrow the discussion by taking an aggressive stance. People must stay in one group or another?”

Again… We can not cookie them if there is no ad to click on. We have solid tracking but we can’t begin tracking them before they click the PPC ad.

I’m just playing Devils Advocate. I completely agree with you on most points but I'm having a hard time winning that argument.

Last edited by Reality Check : 05-19-2006 at 05:51 PM.
Reality Check is offline   Reply With Quote
Old 05-19-2006   #10
recoveringtechie
Newbie
 
Join Date: Mar 2005
Posts: 2
recoveringtechie is on a distinguished road
It seems what you are arguing is how much impact your entire marketing mix is influencing your overall sales. Since you have a strong brand it's always a tough question because everything you do typically increases your brand awareness. Some would infact argue that measuring branding is futile.

I can certainly see holes in the banner measurement of impressions you lay out given the amount of cookie deletion and it's impact over large time periods needed to accurately measure brand awareness. This would make me question how certain you really were about the effectiveness of the banner campaigns. I would also wonder whether you were measuring ROI with fully loaded costs or not.

My take on this is that measurement tools can give you a good idea of relative performance of your activities but one should never go sofar as to think of them as literal measurments because no matter how you look at marketing activities (online or offline) there's always things that impact your ability to get to perfect accounting numbers.

As for whether search marketing is working for you I would then look at it in two ways, (a) does the behavior suggest an increased amount of impulse buying or a general shortening of your sales cycle from click to purchase? (b) does the existence of the keyword program increase your brand awareness in other ways which are similar (and perhaps cheaper) than other brand marketing?

I would suggest that all things being equal a click on a banner might take longer to turn into a sale than a click on a keyword if the keyword is specific enough. This has a lot to do with how people progress through their sales cycle and their proximity to the sale.
recoveringtechie is offline   Reply With Quote
Old 05-20-2006   #11
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Thanks recoveringtechie

Thanks recoveringtechie,

I agree with every point. Unfortunately… The SEM campaign is measured by a strict direct response metric for cost of advertising versus revenue generated. Branding and raising awareness is not a goal or even a consideration.

I agree… Measuring branding is futile. If SEM is only good for immeasurable “Branding” purposes, I’m out of a job, so that’s the last argument I’ll be trying to make.

We are only concerned about the ROI of the click. Does it economically generate revenue or are people just as likely to make the same purchase, with or without the help of the PPC ad?

The banner test isn’t perfect, but it’s fine with me if some people are deleting their cookies because this will have a negative impact on the test results. If anything, we are underestimating the impact of the campaign. The more people delete their cookies the les conversions the banners can take credit for. I can show the positive numbers and say with confidence that “These numbers are probably even better than this because of cookie deletion.”

With search… Someone can say the numbers are likely much worse than this because a large % are just as likely to visit the website anyway. I can’t prove this to be wrong.

Even with banner advertising, we don’t care about the branding impact. If the placements per publisher don’t produce sales, we quickly cut that publisher from the banner campaign.

No matter what… The cost of the advertising placement can not exceed the profit margin on the sale or the ROI will be a negative number.

If the banner ads or SEM have a branding impact… Bonus! But… We only care about achieving a positive ROI per ad placement regardless of the online advertising medium.

We don’t expect perfect ROI numbers. I just need to show ROI numbers that can not be easily dismissed as being purely smoke and mirrors.

Quote:
“(a) does the behavior suggest an increased amount of impulse buying or a general shortening of your sales cycle from click to purchase? (b) does the existence of the keyword program increase your brand awareness in other ways which are similar (and perhaps cheaper) than other brand marketing?”

I wish those were the questions but they are not. The question is; are people just as likely to purchase with or without the PPC advertising expense. If so… By how much?

Will Some, Many, Most or All of the people still make a purchase anyway?
Without tracking a control group, I can’t answer that question in a way that will satisfy the smart folks in the finance and accounting departments.

Quote:
“I would suggest that all things being equal a click on a banner might take longer to turn into a sale than a click on a keyword if the keyword is specific enough. This has a lot to do with how people progress through their sales cycle and their proximity to the sale.”

I agree… However, if people who see the banner are just as likely to make a purchase with or without the PPC keyword, than why are we spending millions of dollars on it?

The questioning of the SEM campaign usually follows these lines…

• Show me the money! Does it work or not?
• OK then… Prove it!
• If you can’t prove that it works, then it probably doesn’t work.

It’s a bit like debating with someone who thinks the earth is flat but I don’t have the proper tools to show otherwise.
Reality Check is offline   Reply With Quote
Old 05-20-2006   #12
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Huge missing piece of the SEM puzzle

To help the conversation, I will attempt to show how the Finance and Accounting departments might see the situation.

Let’s say our product sells on our website for $100.00
This price would give us a 20% profit margin or $20.00 per unit.

The average cost per click is $1.00
Our conversion ratio is 10%
Or $10.00 PPC Cost & $10.00 Profit per order.

So in this situation…
Our ad agency and analytics package would both say…
1 Million dollars in PPC sold 100,000 units and we make 1 Million in ROI.

This would be great! The folks in Finance and Accounting would all say that with a profit margin of 10%!!! - PPC is our most profitable and efficient distribution channel. (Great ROI)

The problem is…
It’s a reasonable assumption to say that a significant % of those 1 Million visitors would have visited the website anyway and made a purchase with or without the help of PPC.

If that is the case… The % profit is inversely proportional to the % of people who would have made the purchase with or without the help of PPC.

If 25% of the people would make the purchase anyway, than this would drops the ROI per unit down to $5.00 and PPC is at best a mediocre distribution channel and certainly not our best option.

If 50% of our PPC traffic would have made the purchase with or without PPC then that leaves us with Zero ROI.

If more than 50% of the visitors would have made the purchase anyway, we are loosing money on every order. If all the people who clicked our ads would have made the purchase anyway then we just lost 1 Million dollars!

It could be argued that turning a 1 Million dollar deficit into a 3 Million dollar profit is as easy as killing the SEM campaigns.

This sounds like crazy talk to me but it’s a simple argument and I can’t figure out how to counter this argument.

If the major search engines would allow us to pay to place a "tracking pixel" in the SERPS for a set of keywords or the entire portfolio of keywords instead of the PPC ad, we would be able to accurately determine the lift in orders directly responsible for the ad placements and ROI of the expense.

In my opinion, this is a huge missing piece of the SEM puzzle.
I refuse to believe that the smart folks at the search engines can’t do it.

Until then, I believe more and more large advertisers will have their PPC budgets cut because we can’t prove if PPC working or not

Last edited by Reality Check : 05-20-2006 at 11:14 PM.
Reality Check is offline   Reply With Quote
Old 05-21-2006   #13
mcanerin
 
mcanerin's Avatar
 
Join Date: Jun 2004
Location: Calgary, Alberta, Canada
Posts: 1,564
mcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond repute
I think you may have already outlined a potential solution to your problem, Reality Check.

Quote:
I’m not sure what blind tests you are referring to. As far as I know… The ads are either there or not. You can’t segment the audience into two groups that either continue to see the ads or not for a period of time.
I think you can. Do the same thing you do with banners:

Quote:
We always pay the banner bill because we can prove they work. Basically, we can pay to show a bogus banner to half the audience and measure the difference in response from the two groups. You can’t do this with search.
Sure you can. Simply bid with PPC using same/similar ads for a different site or subsite. Heck, for a spend that size you could alternate every hour (or any other way you wanted to) for as long as it takes to get the information necessary, as long as you were using PPC software capable of making those changes on the fly. Use a second company if necessary.

Treat the PPC as you would the banner ads, just trickier to control. It sounds like all you really need to do is show the ROI (or lack thereof).

I don't doubt that there are circumstances where PPC may not help, but I've personally not run into any, unless they were very poorly optimized. There have been times where I've seen PPC sending a company rocketing to the poorhouse because they were bidding stupidly, but that's the only time. I'll allow that there may be exceptions to anything, however.

But I'd like to see the data (much like the bean counters at your company).

I think that if you put some thought and planning into it, you could use PPC to emulate the effect and testing that you see with banner ads. Since they accept the banner ad ROI measurement, that is the standard you would/should be held to, rather than some theoretical perfect scenario. I would focus on that, since it's measurable, realistic and (I believe) attainable.

Just thinking out loud,

Ian
__________________
International SEO
mcanerin is offline   Reply With Quote
Old 05-21-2006   #14
Reality Check
Member
 
Join Date: May 2006
Posts: 29
Reality Check will become famous soon enoughReality Check will become famous soon enough
Thanks Ian!

Thanks Ian!

Quote:
Simply bid with PPC using same/similar ads for a different site or subsite.

Hmmm... I’ll need to check in to this. It might work but I’ll need to research the possibilities before I can say if it will work or not in my situation.

The website would need to be relevant enough to qualify for the ad group but it shouldn’t be a competing website. It would need to be compelling enough to click on so we can cookie as many people as possible but not specifically promote our product category.

I think this might work! I’ll need to think about it for the rest of the weekend but it sounds promising.

I apologize to anyone who posted something along the same line as this earlier but somehow the concept didn’t register until now.

Thanks again. I’ll let you all know what the situation is as soon as I have some news.
Reality Check is offline   Reply With Quote
Old 05-21-2006   #15
mcanerin
 
mcanerin's Avatar
 
Join Date: Jun 2004
Location: Calgary, Alberta, Canada
Posts: 1,564
mcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond repute
I'd be interested in what the result is, Reality Check - regardless of whether it's what we are hoping for or not.

All data (even negative) is good data, and helps everyone do their job a little better.

Cheers, and good hunting!

Ian
__________________
International SEO
mcanerin is offline   Reply With Quote
Old 05-22-2006   #16
Fishcharge
Newbie
 
Join Date: May 2006
Location: Hong Kong
Posts: 2
Fishcharge is on a distinguished road
A Different Angle?

Hi there. I'm new to all this, so I apologise in advance if I'm missing some glaringly obvious thing in my comments, but you've got to learn sometime I suppose.

Having read through the previous posts, it seems to me that the problem could be approached from a different angle. Here is what I mean. Reality Check states that:

Quote:
Originally Posted by Reality Check
The problem is…
It’s a reasonable assumption to say that a significant % of those 1 Million visitors would have visited the website anyway and made a purchase with or without the help of PPC.
I'm not sure I agree that this is a reasonable assumption and this seems to come to a decision at the strategic level, not the tactical one. Why are PPC campaigns being run in areas where you're making assumptions that a significant % of traffic will arrive at your site and purchase anyway? If in reality a significant % of traffic convert all the way through to purchase - this should be reflected in your traffic analysis. Given the option between clicking on an organic link vs. a PPC ad, I think most consumers will choose the organic link. Which brings me back to the original question - why are PPC campaings being run in areas where most of your sales are being generated by organic search results?

In my mind PPC campaigns should be used to close the gaps that your SEO strategy failed to.

Secondly, it also seems to me that your analysis stops a little early. Wouldn't it help to look at the types of customer that your various strategies are creating? Which search phrases (both organic and PPC) acquire the most valuable customers (by whichever indicators you use to measure value) or which search phrases generate customers who make second and third purchases, and where did they come from (organic/PPC)? Thus you're moving away from the idea that you need to focussing on PPC ROI to acquiring profitable customers with positive ROI for your long-term business. While click ROI is a useful measure, I don't think it is necessarily the most valuable one.

Thanks,

Fishcharge.
Fishcharge is offline   Reply With Quote
Old 05-22-2006   #17
Mikkel deMib Svendsen
 
Mikkel deMib Svendsen's Avatar
 
Join Date: Jun 2004
Location: Copenhagen, Denmark
Posts: 1,576
Mikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud ofMikkel deMib Svendsen has much to be proud of
I don't think the problem is the answers you get but the questions you ask! You take a broad spectum media like search and want to meassure it with a narrow set of metrics specified for other medias. No matter how you do this you will fail. You will not come up with the answers you seek because the questions are wrong.

If you set up the "right" metrics you can prove that a car can't possible drive, flies can't fly, and TV-advertising dosn't work at all. Its all up to the questions you ask, the metrics you use and the methology you pick. But even if you prove cars can't drive my car still takes me to work every day and the flies bug me at night

If you want to meassure search marketing only by a simple set of ROI metrics set for banners you won't get what you are looking for. It's a hopeless task.

My best advise for a client like that would be: Drop search marketing and leave the market to your competitors

Last edited by Mikkel deMib Svendsen : 05-22-2006 at 04:16 AM.
Mikkel deMib Svendsen is offline   Reply With Quote
Old 05-22-2006   #18
shor
aka Lucas Ng. Aussie online marketer.
 
Join Date: Aug 2004
Posts: 161
shor is a jewel in the roughshor is a jewel in the roughshor is a jewel in the roughshor is a jewel in the rough
Interesting thread, great responses. I've been in a similar situation to RealityCheck. I work at a big corporate, millions in marketing, direct response metrics as KPIs etc. Based on my experiences I have a few recommendations.

If your company can afford to spend $millions on marketing, they could always divert money from their next brand awareness survey into a focus group:
"If I ask the research group to look for a product, how many end up finding my company's red widget unaided by PPC?". (Watch all those users click onto your competitor's PPC ads!) Wouldn't that give you measurable and accurate data that answers the question? Expensive way to do it, but it is viable...

Or subscribe to HitWise and check the share of clickstream data between your company and competitor websites when you have PPC on and off. Do you see that trend of increased search engine traffic referrals to your competitor when you have PPC off? That's the company that is taking market share from you. Those users will NOT find their own way to your website because if your competition is anyway decent, they are well on the way to converting those users into customers.

I understand that SEM's contribution to your total revenue is less than market variance but you can still create tests that answer the question. You say have several dozen distribution channels, well choose a single product and market that product through a limited and measurable set of distribution channels that includes SEM. Eg. Red Widget is only advertised through banner ads and ppc ads. I take away banner ads, what revenue do I get from PPC? I take away banner ads, what revenue do I get? I then take off both channels and I am left with revenue contributed by direct typed/bookmarked visitors.

Finally, get a SEM champion! This is the upper-management dude or dudette that tells the head of Finance to get off your back. Sell your story and argument to them, and it'll be a lot easier to keep those accounting rodents away from your cheese
Quote:
Originally Posted by Mikkel deMib Svendsen
If you want to meassure search marketing only by a simple set of ROI metrics set for banners you won't get what you are looking for. It's a hopeless task.

My best advise for a client like that would be: Drop search marketing and leave the market to your competitors
Otherwise Mikkel is correct - please leave more market share for the rest of us!
shor is offline   Reply With Quote
Old 05-22-2006   #19
mugshot
Member
 
Join Date: Sep 2004
Posts: 51
mugshot is on a distinguished road
Quote:
The problem is…
It’s a reasonable assumption to say that a significant % of those 1 Million visitors would have visited the website anyway and made a purchase with or without the help of PPC.
Assumptions are never reasonable We can then also assume that if you drop SEM, then the profit % will also decrease? Or we can assume that if those purchasing from banner ads will purchase anyways without the banner ads?

I guess in short, assumptions will only create more assumptions - at best.
mugshot is offline   Reply With Quote
Old 05-22-2006   #20
ShoeMoney
Skills To Pay The Bills
 
Join Date: Feb 2006
Location: Lincoln, NE
Posts: 18
ShoeMoney is on a distinguished road
I do not understand why people make something so simple so complex.

You spend X you make X.. what is the issue?
ShoeMoney is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off