PDA

View Full Version : Impact of Listing Position


andrewgoodman
07-14-2004, 10:39 PM
The recently-released Atlas DMT (formerly GoToast) study - http://www.clickz.com/news/article.php/3379931 - seems to show both a sharp dropoff in the clickthrough RATE the lower the ad position, and also, a smaller potential number of clicks for ad positions below the first one due to less exposure.

Although I get the general idea, I have two major quibbles. First, the data are not presented in a clear-cut way and almost seem to obfuscate what's actually going on. I understand "relative CTR" -- for example, from this sample of advertisers, ad position 6 in AdWords gets about half the CTR as ad position 1.

But I don't really follow "click potential." Why would an ad in position 2 get a lower potential number of clicks than its lower CTR would dictate? There is no major partner that displays only a single AdWords ad, is there? So wouldn't the impressions be the same in the 1 and 2 slots?

The other major drawback to the study is that it implies that ad position has an independent impact on CTR. But we cannot ascertain to what degree that impact is independent. Typically, GoToast will be used to operate high-spend campaigns that shoot for ad positions 1 through 3, and those positions will typically be occupied by brand-name advertisers who might get good CTR's based on brand recognition.

However, if brand-name advertisers or clever copywriters allowed themselves to drop down to positions 5,6,7, etc. more of the time, perhaps they would actually get decent CTR's down there. But they don't because Atlas tells them that #1 is the wisest buy!?! ;)

Although the advertiser sample is large, one questions whether the sample is still rather homogeneous and whether the campaign methodologies used to run the campaigns follow certain patterns which skew the data.

That being said, it's useful information to have, because it shows clearly how CTR and click volume decrease as you go down in ad position (I'd love to see the numbers right down to #20 or so, since believe it or not some of my clients are stuck in position 11 and 12 on some words).

On the other hand, you can get a pretty quick feel for this by just looking at a handful of campaigns and watching the impact of lower ad positions.

What did others think of this study? Was it helpful or did it fall into the "tell me something I didn't know" category? Or was it actually misleading? Any takeaways?

Finally, what about the comments made by Young-Bean Song, Atlas' director of analytics?

"There is a 40 percent drop between Google's No. 1 search ranking and its No. 2 search ranking, suggesting that the top position may be the wisest buy with Google," said Young-Bean Song, director of analytics and the Atlas Institute. Atlas DMT is an advertising technology provider and an operating unit of Seattle's aQuantive. Atlas Search is an automated search campaign management technology owned by Atlas DMT.

With Yahoo!-owned Overture, "the decline you can expect in traffic based on rank is much more smooth," Song said. "Taking the number two spot is a better strategy on Overture than it is on Google."

I'm afraid I find these difficult to digest on a couple of levels. The top position certainly isn't the wisest buy. And unless we have independent corroboration from Google and Overture, I'm not sure I'm going to buy the idea that the traffic dropoff between 1 and 2 is precipitous on Google, and smooth on Overture, especially not without some proposed explanation for this that might have been put forward in a prior hypothesis. That just seems out of whack.

seobook
07-15-2004, 01:45 AM
Finally, what about the comments made by Young-Bean Song, Atlas' director of analytics?

"There is a 40 percent drop between Google's No. 1 search ranking and its No. 2 search ranking, suggesting that the top position may be the wisest buy with Google," said Young-Bean Song, director of analytics and the Atlas Institute. Atlas DMT is an advertising technology provider and an operating unit of Seattle's aQuantive. Atlas Search is an automated search campaign management technology owned by Atlas DMT.

With Yahoo!-owned Overture, "the decline you can expect in traffic based on rank is much more smooth," Song said. "Taking the number two spot is a better strategy on Overture than it is on Google."

if you look at the numbers they show a sharper drop in Google from 1 to 2 than they do in Overture, but if you look at the 4 5 6 range you will see they show google smooths out a bit where overture continues to drop off.

andrewgoodman
07-15-2004, 04:52 PM
The Overture dropoff after fourth position is understandable because Overture largely = Yahoo and those listings are very prominent at the top of the page (1, 2, 3, 4) but much less so at the bottom (5, 6). But then again, on some queries Yahoo shows ads in the margin as well, and it appears to repeat 5, 6, before adding 7, 8. In that case, maybe 5 isn't so invisible after all.

On a query for "briefcases," the 5th-place advertiser appears at the bottom of the Yahoo SERP, AND at the top of the right-hand margin. Kind of double exposure, for far less the CPC of the top 2 spots. 4th spot also looks good to me - part of the sponsored listings area at the top, but at the very bottom of it, so it catches the eye. GoToast doesn't want you to hear that, though.

For Google to drop off like that from 1 to 2 has no rhyme or reason. I would love to see a competing study or hear any speculation at all on it from Google (which we won't get).

There are too many unanswered questions with this study. Because ad display jockeys around quite a bit, I wouldn't want jump to any major conclusions, other than the obvious -- that 6 and 10 are less visible than 1 and you'll get fewer clicks (at a lower CPC).

When it comes to GoToast's study I also wonder what the breakdown is of traffic from different parts of the network. Is content match part of it? Sure you can overpay like crazy for the top spots and get lots and lots more clicks... from terrible content match partners. Don't do it.

Lance Housley
07-16-2004, 06:22 AM
Having looked closely at the Atlas DMT report, I'm little wiser...

Previous studies (like Enquiro's Into the Mind of the Searcher) have shown that different people search in different ways, and that the likelihood of even looking at results below the first few depends on the kind of searcher one studies, whether they are on a research phase or a buying phase, and how expensive the item is that is being contemplated. Other studies show that search patterns differ according to educational background &c.

Yet more studies (e.g. Search Engine Use in North America or the Hitwise U.S. Search Engine Report May 2004) show that search engine preference is also likely to differ according to educational background and income.

I admit, these studies mostly looked at organic results rather than ads. At first glance one might also be tempted to align the results of previous studies to reach the conclusion that people who use search engine 1 tend to use strategy A while people who use search engine 2 lean towards strategy B, and this could be taken as supporting AtlasDMT's conclusion. But this would, I think, be to overlook the nuances of these earlier studies. ;)

The Atlas DMT study makes interesting reading, and it focuses on a fascinating area, but it does not appear to take on board the distinctions between searchers, strategies, stages of the research-to-purchase trail or search engine preferences which have already been identified. :eek:

Andrewgoodman's reservations seem to me to be wholly justified.

stephen
07-16-2004, 01:35 PM
Click Thru Rates do not equate to sales....

Fewer CTR's down at the bottom, might be a higher

percentage of sales for people finally ready to buy

after spending time working top to bottom and finding

pricing pretty much the same.

Too many variables.

Stephen

AussieWebmaster
07-16-2004, 02:47 PM
The number of possible variables are the biggest problem with quantatizing this.
With Google you have the rotation problem (we max the spends for many of our terms and still get juggled on position some times).
Overture has some sites that only carry 3 ads at the top while others list them differently.

Then you have the impact of the included organic listings.

No search engine is ready to give a report that tells what percentage of clicks occur for all positions which would be the only true way to make conclusions.

We have terms that we have the 1 and 2 spots for paid and the 1 and 2 spots for organic... here we find that the paid listings at Google are getting more traffic than the organic. But the listings are different - paid can be more specific while organic grabs the title and some of the description or the start of the text on the page.

With Overture it varies hugely because they go out to so many places and they organic results are not always the same as Yahoo... such as MSN.

To draw conclusions on this a lot more needs to be considered.

I agree that studies such as the education and income levels (obviously from attended survey/poll events are needed and of major interest.

This is such a new industry that a lot of testing will be developed and improved on over time.

Mikkel deMib Svendsen
07-19-2004, 06:08 AM
One thing I do not like about this report is the statement on page one:

Which Factors Determine Click Volume?
Two factors determine how many clicks you will get for a given search phrase: impressions
and click-through rate (CTR).

As most of us here know this is simply not true. Of other things that very much determine the volume of traffic you get I could mention: The copy of the other ads, distrubution partners placement and disclosure of ads, number of ads on a page and the quality of organic results - just to mention some.

If the people behind this study really believe the above statement I don't find the study very usefull. Now, if they had stated something like: "There are many factors invloved but we chose to focus on these two ..." then it would have been OK, but this way it actually looks like they do not know that there are many more factores. In my oppinion you should not leave out important facts in a study unless you specifically state what you do so, so at least you show the reader that you know about the other factors too.

dannysullivan
07-19-2004, 06:19 AM
For Google to drop off like that from 1 to 2 has no rhyme or reason. I would love to see a competing study or hear any speculation at all on it from Google (which we won't get).

I haven't looked at the study yet, but my big guess about this would be if the study didn't segregate when listings show up inline, in the old premium ads spot about editorial listings, as opposed to sidebar format.

I still see Google putting ads 1 & 2 inline occasionally. It's long been on my list to find out what the policy is behind this -- why and when it still happens. But clearly, 1 & 2 inline needs to be measured separately from when 1 and 2 happen sidebar. If they aren't, there would have been a big skew.

andrewgoodman
07-19-2004, 11:53 AM
If you can get Google to give you a precise explanation of this, then you're a better interrogator than me. :)

But as with many AdWords features, there seems to be no transparent rule, but rather it is "proprietary and algorithmic."

Back when premium CPM ads shared that space with regular AdWords advertisers, even then, the deal was that you could get promoted to that spot at Google's discretion -- probably based on some combo of CTR and bid. But why the *best* CTR/bid combo on a given search phrase (even if relatively low) would not *always* get promoted to those spots -- why they often leave them blank and show everybody in the margin -- is puzzling to me. Perhaps a way of rewarding advertisers with a high aggregate or monthly spend, I don't know. They do not need to publicize this and I don't believe they have to have equal rules for all advertisers (this is not written down anywhere), but I would certainly be interested to hear what the policy is.

I have double-checked the FAQ's, and needless to say the info is spotty.

So in connection with this study, for sure one does need some clarification of which ads showed up as bold text listings at the top and which in the margin. Either one could be considered in 1 or 2 position (the latter in cases where Google's chosen to show no sponsored listings at the top).

seobook
07-19-2004, 01:22 PM
But why the *best* CTR/bid combo on a given search phrase (even if relatively low) would not *always* get promoted to those spots -- why they often leave them blank and show everybody in the margin -- is puzzling to me.

if the top ads were there all the time people would be more inclined to realize they are ads and maybe think they are not as relevant and be more inclined to skip over them on the money searches (where they really count and are likely extremely relevant).

Google is not in it for "how much can I make off this search" they are in it for "how can I get more searches" and "how can I make more money from all the searches put together"

dannysullivan
07-20-2004, 04:32 PM
If you can get Google to give you a precise explanation of this, then you're a better interrogator than me.
Bernard just asked about why & when the inline ads appear on Google and got a reply over here:
Welcome Google AdWordsRep (http://forums.searchenginewatch.com/forum/showthread.php?t=655)

Though I must admit, I'm still confused :)

projectphp
07-21-2004, 03:57 AM
I just read the PDF report, and I dunno. What is with SEM reports and the lack of working and hard numbers? The report goes from "Click through rate" to "The data", and yet neglects to show any hard data, like total SERPs reviewed, total clicks measured etc etc. If this was a high school maths paper, it would get no marks as the working is absent. Conclusions are great, but knowing how these were derived would be nice.

Speaking of which, the "What We Have Learned" section, aka conclusions, are also somewhat dodgy. How do they know that this applies to Organic results and PI? Really, where is the proof of that?

IMO, a big problem with SEM is that a lot of the "knowledge" that exists relies on statistically unreliable, unjustified or unsubstantiated data. The author of Enquiro's "Into The Mind Of A Searcher", Gord Hotchkiss mentioned this as well: (http://www.searchengineguide.com/hotchkiss/2004/0629_gh1.html)
To be honest, I've also been surprised by how (the report has) generally been accepted without questioning of methodology or results. The one thing about research is that its accuracy is always in question.... [the report] was never intended to be complete or definitive. We expected it to be questioned. We fully intend this to be a first step, and would hope others would rise to the challenge and start to peer into the black hole as well.

A lot of what is taken as gospel in the SEM community is really innuendo masquerading as fact. Without any real data, this report, unfortunately, falls short of alleviating this situation.

While the lack of rigorous, statistically verifiable evidence is inevitable in Organic SEO, given the rate of algo change and number of factors, it is unacceptable for human / SERP interaction. This area may not be a constant, as the Internet population continually increases, but user behaviour is certainly as predictable in SEM as it is in traditional advertising mediums, with the added advantage that online has far more accurate measurement tools (like log files).

This report really just left me wondering why none of the big SE commission an independant third party to analyse this area of SEM. This would ensure more accuracy, a broader range of data, as well as more accurate insight into predict and utilising conclusions about this phenomenon. Increasing the slice of the pie available to SEM is certainly in the SEs best interests, and I bvelieve this sort of research would go a long way to breaking down resistance to SEM in traditional circles.

andrewgoodman
07-21-2004, 03:49 PM
There have been some commissioned studies -- I know Overture used to cite a couple of them prominently in the advertiser area when you log in -- but clearly, we need more. Individual SEM consultancies and tool vendors releasing studies can certainly be helpful, but at the end of the day they're intended to be marketing tools for the vendor. (I say hats off on that front -- Gord Hotchkiss can attest to the benefit of sharing stats -- and it's nice to hear that he was surprised that there wasn't more debate. Maybe that's because not a lot of people took "debating the merits of statistics 341" in school. Math is tough! Even tougher is logic and reasoning or questioning the assumptions behind studies.)

It certainly isn't only SEM that faces this problem -- the problem of "research" that conceals more than it reveals is all over the place.

Our society believes deeply in all of the medical studies that indicate "increased risk for heart disease" and so forth, without understanding the many assumptions that go into them, study methodologies, the odd definition of "increased risk," and the relationships in the real world between increased risk and actual changes in health. (Just to take an example.) I always look at my 93-year-old grandmother and her less-than-perfect diet and exercise regimen and think -- if we had put all those increased risks of bad things happening together, she'd have died fifty times over. Possibly the biggest risk associated with an occasional pat of butter is the risk of being bored to death discussing how risky it is.

And just look at what hot water so many folks got themselves into over the past decade because they had a "surface" understanding of metrics relating to their investment portfolios, believing in the solidity of certain ratios and valuations where they should have understood a bit more about how those things get created in the first place, or why they aren't always reliable.

I am delighted that this section of SEW Forums was created, because threads like this allow sceptics and scientific minds to come along and debate stats rather than just parroting them, which our industry, being so immature, has been content to do thus far.

MarketingSherpa is also working on this area (search metrics and the like) now. It will be interesting to see what they come up with.