Concerned about Validity of Overture & WordTracker Numbers

[randfish]

A website I optimize for is ranked #1 (in some cases #1 and #2) for several terms that receive the following traffic numbers according to WordTracker(WT) & Overture(OV):

term 1 - 404/day (WT) 230/day (OV) - I get an average of 31 visitors per day for this term

term 2 - 161/day (WT) 127/day (OV) - I get an average of 18 visitors per day for this term

term 3 - 46/day (WT) 22.8/day (OV) - I get an average of 0.5 visitors per day for this term

I'm very worried that this is uncommon for the industry. Have your (collective) experiences indicated that you receive about 10% of the traffic from being #1 in the SERPs at all the major engines. Are these two statistics that far off?

Thanks for your help!

[fantomaster]

In our experience, this is nothing unusual at all.
For one, both Wordtracker and Overture metrics are aggregated.
They won't specify which position received which click-thru rate.

Nor are they really comparable: whereas Overture is referencing solely
the data they collected on their own platform (and its many outlets via licensing
partners and affiliates, of course), Wordtracker monitors meta search engines -
quite a different animal overall.

Moreover, it's been our long standing contention that OV metrics are so wildly off mark,
it makes your eyes water. Unfortunately, they are less than transparent about how
and in which manner these stats are generated.

For example, what exactly constitutes a "unique click" - a click-thru from a unique IP, ok,
but within which time frame? 6 hours? 12 hours? 24 hours? A week?
And how do they handle ISPs that will assign separate dynamic IPs with every fresh
browser window opened (such as AOL, for example)?

So there's no safe, reliable way to actually tell, I'm afraid.

[randfish]

Fantomaster, thanks for answering, it's great to see you here (I hope you know you're famous in many SEO circles).

I may have been unclear in my initial post. I don't have top positions in the paid results, these are #1 positions in the organic SERPs.

I also checked the Overture stats, for which we're on the first page too, and it says that for that first term, we average 250 impressions per day... I'm really confused, because our listing is quite strong, I couldn't ask for a better description from Google or MSN (although Yahoo! has our directory listing which is less powerful). I guess I'm just at a loss - I thought that something above 70% of searchers click on the first listing, which should give us a minimum of 175 unique visitors from the Overture-served properties (MSN, Yahoo!, Ask)...

[fantomaster]

Thanks for the kudos. :)

First, let's not forget that the 70% clicks studies tend to show are, of course, only achieved
under specific test conditions.

In any case, simply applying this figure derived from one specific setup to the data from another
(OV stats) to gauge yet another, third set (your server traffic logs) seems to beg the issue of exponentially
increasing the overall margin of error if only on methodological grounds.

Moreover, not all listings are created equal. On some platforms, OV results #1 thru #3 will be
displayed, on others it may be #1 thru #5, etc. That would make your #1 position in the organic
listings an overall #4 or #6 respectively which will skew your traffic stats correlation even further.

Add to that the fact that Overture's stats are notoriously unreliable (there are those who would
even go so far as to say: downright overblown ...) in the first place, and you're in for a statistical nightmare
when matching one set of data with another.

Don't get me wrong: I'm not saying you should give it up altogether as a bad job; it's just that
more likely than not you'll always encounter this kind of issue when working from essentially incompatible data sets.
There's no easy solution I'm aware of if you're actually aiming at reliable and verifiable numbers.

I'm currently doing some research in preparation for an article on the whole issue of click fraud and the fundamentally
flawed attempts at gauging ROI on PPC bids reliably - and of course the slice of traffic taken up by organic positions
competing with sponsored listings plays a major role in this discussion, as well.
And what I've seen to date isn't very pretty, I'm afraid ...

[randfish]

Fantomaster,

Thank you for the explanation. I realize that the incompatibility of the two data sets certainly makes for inaccuracy. I never guessed it would be so bad. Orion actually mentioned something about this during the final SES session on keyword research tools (simplex optimization); it's probably important that I look into it post-haste.

It sounds like your suggesting that the only real use for the Overture & Wordtracker tools is as a reference or comparison point - a relative measure if you will, which is certainly better than no measure at all. The one thing they are pretty accurate about is noting which terms are searched for more/less than others, so I'll continue to use them for that purpose and rely more on PPC testing campaigns at Google & Overture to try to estimate real traffic numbers.

This does beg the question of why the tools are reporting such inaccurate measurements. By my estimation, Overture is off by no less than 40% on those 3 terms and I will be doing some further testing to check their overall reliability.

Thanks for your help with this issue, hope to see you around here more often :)

[St0n3y]

WT and Overture are really only representing search volume for the particular phrase. Click thru rates will be considerably different based on any number of factors such as title, description, etc. While we have found that the wordtracker numbers are considerably under-inflated for any particular search, the click thru rates are at or around 10%. Again, give or take depending on position, title, description, competitors titles and descriptions, sponsored ad titles and descriptions and the like.

[randfish]

http://www.socengine.com/seo/images/seo/gg-results.gif
http://www.socengine.com/seo/images/seo/yahoo-results.gif
http://www.socengine.com/seo/images/seo/msn-results.gif

Just wanted to post these so it's clear where the positions I'm talking about are.

St0ny - You're suggesting that WT numbers are under-inflated, but I'm getting less than 10% of the daily traffic they say types in that term despite being #1 at all engines...

In any case, this is an eye-opening experience as most of my efforts in the past have resulted in much more logical numbers in comparison to what Overture and WT give me.

[orion]

I realize that the incompatibility of the two data sets certainly makes for inaccuracy. I never guessed it would be so bad. Orion actually mentioned something about this during the final SES session on keyword research tools (simplex optimization); it's probably important that I look into it post-haste.Indeed. I have warned many in the industry not to mix and match metrics from different databases to come up with a new metric. I felt obligued to make a statement before the SES, NY audience. I also strongly disagree that this kind of approaches "is the best swag out there." (http://www.socengine.com/seo/guide/ses-nyc/advanced-search-term-research-tools.html) There are indeed better approaches. It just that the seo community has not been introduced to those. That would soon change. More on this here

"Ultimately, metrics based on search results are not just affected by relevancy scores. They can be the result of tokenization and similar procedures taking place at the level of the individual IR architectures. Overall, search engine result page metrics (SERPs) are the result of ranking algorithms, not necessarily of users' search behaviors (how often a term or phrase is queried). As a matter of facts, a ranking algorithm can care less about how frequent was a term searched this or the previous month, week or day; of course, unless we talk about paid services (e.g. pay-per click scores).

Thus, combining SERP metrics with metrics from other search engines that account, let say, for user's query behaviors (i.e., search volume from a search engine or meta engine) to come up with a new metric is a highly questionable approach. SEOs, SEMs and keyword research firms should stay away from such practices before their metrics lose credibility in the industry or with clients." http://www.miislita.com/fractals/overlapping-patterns.html

With regard to SIMPLEX Optimization, a process improvement and optimization technique (often used in chemometrics and statistical optimization methods) I use to work and research on Simplex back in 1982-1986 for my MS grad school at Univ. of PR. When I moved to ASU, AZ, I used to present seminars on the topic. This is a standard technique for multivariative optimization rather than one-variable-at-a-time approach. It was introduced back in the 60's and resurfaced in the 70's.

As I often mention at my site, Simplex has been used ever since in Economics, Chemistry, Statistics, Engineering, etc, you name it. Simplex can be applied to almost any optimization problem. Speed and convergence to the optimum is faster than traditional methods for improvind a system's state. It is just another great tool that should be part of the toolbox.

Orion

[AussieWebmaster]

The other thing is there is a big difference between impressions and clicks.

WordTracker and Overture etc. are giving overall impressions whereas you are counting the clicks you get from those numbers.

Unfortunately there is really no way to track impressions.

[St0n3y]

St0ny - You're suggesting that WT numbers are under-inflated, but I'm getting less than 10% of the daily traffic they say types in that term despite being #1 at all engines...

everything OT and Overture needs to be taken with a grain of salt. Unless they had access to search information from Google (which they don't) and other major engines their numbers will always be simply guesses from their system. In our experience they are underinflated, however I'm sure in some cases they may be over-inflated. either that, or visitors are bypassing your#1 position in favor of the #2... which I know can happen as well. It certainly does seem to be an anomoly, in any case.

[PhilC]

There are several things to bear in mind about Overture's numbers:-

(1) They rarely show plurals. They combine singular and plural versions of phrases and there is no way of knowing how many of the reported figues were singular and how many were plural. This will seriously inflate the numbers.

(2) Their figures are unreliable. I say that because I've seen many examples of listed searchterms, that had a fair number of "searches", that were so contrived and obscure that the searchterm could only possibly have been dreamed up by one person. So where did all the searches for it come from.

(3) Their figures are also unreliable because people who have PPC accounts with them, and other similar people, are responsible for many of the searches due to frequently checking their positions in the search results. That's just a guess, but it has to happen a lot. This will also inflate the numbers. In many or even most cases, it will seriously inflate them.

[ephricon]

Is it possible that this is one of those terms that is searched frequently by firms trying to optimize for this term, and thus the number of searches is artificially inflated? For example, I would expect the CTR for high rankings for a term like "search engine optimization company" to be much lower than the general average CTR, since I'm sure there are a large number of SEO companies trying to optimize for this term, and doing a good bit of manual rank checking and such to inflate those numbers.

In many cases it can be more pronounced for a lesser searched term as well. Sometimes people think a term is searched alot, they optimize for it, they monitor their ranks manually and this drives up the searches, actually causing it to show as being searched more frequently - whereas these numbers are artificial since they don't represent real prospects.

[PhilC]

If you are asking about those phrases that only one person could have dreamed up, they are really obscure - and long. Unfortunately, I can't remember one offhand but nobody would consider targeting them. It could be that people just wanted to get those phrases into the list for a bit of fun.

[hiero]

Great thread!

I've always been amazed after doing keyword research and then implementing an SEO campaign and ranking well how skewed results are. Meaning I rarely receive the clicks that I thought I would. I must say though that a lead is a lead, and something is better than nothing.

Another thing that is worth mentioning is all of the research that gets done out there adds to the overall numbers count, and if the words or phrases are popular then the research factor could over inflate counts dramatically. I check my campaigns 2 to 3 times per week so I'm adding to the false results, then add everyone else who does the same and I'm sure the numbers will be huge.

The search engines should create search API's for industry professionals where we could log into our accounts to do word and phrase research and have those numbers not be added to the daily/weekly counts.

[St0n3y]

Very good point, ephricon. Search volume is invlated exponentially on popular phrases simply because people are doing nothing more than checking their rankigns. We check our rankings for our clients on a regular schedule, but sometimes we do checks in between. Our clients are also doing searches to check thier rankings (even though we provide reports) and all their competitors are doing the same. This creates a huge search volume "markup" that overinflates the value of that term, especially when it comes to getting leads.

[randfish]

Thanks guys,

That seems like the most likely explanation to me (for this particular instance), there are certainly a lot of us who probably check those words from many computers each day.

[Nacho]

Great thread!
Yes it is! You know, you can use the Rate this Thread (http://forums.searchenginewatch.com/showthread.php?t=4580#goto_threadrating) feature down at the bottom and mark what you think with a scale of 1 - 5.

[earlpearl]

Randfish:

For the last year Month by month I ran overtures estimates for volume for my main keyword against aggregate traffic to my site. Similarly another highly ranked business/site for my industry checked his traffic volume with me. Remarkably the rise and fall of this term over the year reflected very closely our traffic volume month by month. Remarkable sinchronicity.

In January both my site and the other business had record traffic months. Meanwhile overture's reported volume for the term was about 2/3 of the highest month. It was dramatically off base with our experience. Our sites are both highly ranked(first page for all search engines) and often #1 for a variety of 2ndary terms, plus typically first for about 25 markets in the US.

Overture's January totals just didn't jive with our combined experience.

To the extent that you drive high rankings off of your position now depends on marketing rather than SEO. Your "snippet" under the title may drive people to your site. Check your description against those of other sites in the industry. Ultimately your marketing language has to be broad and compelling to drive visitors to the site.

Finally the topic is narrow and subject to widely different expectations and actions. You have to give the site time to generate leads and business from your new higher rankings. I'd check your logs for new visitors. Most users of the service aren't repeat customers, and new potential customers viewing the site will slowly grow with continued widespread exposure.

EGOL is a master at squeezing activity out of his site and his adsense ads. He tests language and words to drive traffic. He is relentless. Check his postings.

Dave

[jvracing]

The search engines should create search API's for industry professionals where we could log into our accounts to do word and phrase research and have those numbers not be added to the daily/weekly counts.
Google has already done this, haven't they? There are several tools available on the web that use Google's API to check your ranking. My personal favorite is DigitalPoint Keyword Tracker (http://www.digitalpoint.com/tools/keywords/). I have been using this for several months now, and I find it invaluable. I've built up a list of about 125 keyword phrases, and I can check them all daily with a single click of the mouse. It even keeps track of back data and can plot your rankings out on a chart. Pretty snazzy if ya ask me. :cool:

So I guess it depends if queries done through the Google API are counted in the search totals. But then again, nobody really knows what the Google search totals are! The two main tools everyone always mentions are Overture's Keyword Suggestion Tool and Wordtracker. Neither of those use Google numbers. So it seems like manually checking your ranking on Google would be a moot point as far as those tools are concerned. And doesn't Wordtracker claim to use metasearchengine data? So since virtually nobody checks their rank via metasearch, then it seems like Wordtracker would be immune. Wouldn't it?

Oh, and by the way, since this is my first post, thanks to everyone who participates in these forums. I've been lurking here for quite some time, and find it immensely informative, helpful and (usually!) interesting.

[randfish]

welcome jv! Glad to see you posting.

I too like the Digitalpoint tool, although I use different software personally. Hope we see more of you around.

[AussieWebmaster]

DigitalPoint has some great tools - that one is one I have used for about 18 months or whenever it first rolled out... gets my attention 2-3 times a week.

[hiero]

Google has already done this, haven't they? Yep they have, but I don't know that they back out the word/phrase counts from the index search totals. And while DigitalPoint has some great tools, it takes a while to do a lot of searches, it's actually quicker to use the search engines themselves when you're looking for top 30 results.

[St0n3y]

A bit off topic, but I would be interested in a tool similar to WPG to check rankings but it requires each user's Google API to use for Google checking. It would also be nice if Google lifted its 1000 daily search limit. I think Yahoo API allows 10,000.

[earlpearl]

Rand:

Yours is the 2nd site (both financial) I've looked at recently, wherein there were questions about clickthroughs even with high rankings.

Review your description vis a vis the high ranking commercial lenders that are both advertising in adsense and beneath your site in the orgagnic rankings.

I believe the other commercial sources have more compelling blurbs to attract click throughs. In some cases this issue of click throughs is a question of advertising.

I am reviewing all of our descriptions for various rankings and trying to improve them by making them more compelling.

I would run this by EGOL. He has focused on this on a regular basis and tinkers with both his natural listing information and his adsense ads to drive more traffic to his sites. He is relentless in this.

Whether overture or wordtracker's data are accurate or not, just compare the compelling nature of the "ad language" from competitors. You will see a difference.

Dave

[fantomaster]

This is an important point IMV as it shows the prevalent (mildly idiotic) view that top rankings
will automatically result in top traffic and conversions its proper place.

After all, it's about advertising, no less: compelling title tags (or ad texts) go a long way to explain
why some lower ranked sites seem to be getting all the goodies at the expense of higher ranking
competitors, whether positions have been paid for or not.

And: granted that surfers will only review positions 1 through 30 or so, as several studies indicate,
visitors who will impulsively click on whatever happens to crop up at the top of the SERPs may
arguably be too unfocused to result in good conversions in the first place.

Thus, with good-to-excellent ad copy you might perhaps get less traffic on position #6 than your
competitors on position #1 with their bland, nondescript sellbot templates but achieve better conversions
and even save tons of money in the process.

Obviously, traffic demographics will account for a lot, too: while a behemoth like Google attracts the lion
share of traffic, lots of web marketers have found that traffic generated via Yahoo! or MSN
converts a whole lot better.

Moreover, traffic that may be good for one product can prove quite useless for another. So it's not
so much about traffic as such rather than actually reaching your target group. (Think mass and niche
markets as an example.)

It's always wise to bear this in mind before setting out to crunch traffic numbers and stats spreadsheets
like crazy - if you cannot translate them into tangible qualities relevant to your business, it's a
pretty self-defeating venture.

[St0n3y]

I agree that the title and description text can be just as important as the ranking itself. I failed to mention this in my post(s) above primarily because rand stated that he was happy with the titles and descriptions of his listings. Often time its good to try and take a step back and view titles/description from a searchers perspective. Sometimes what we think is good really ain't.

[DISC]

Thus, combining SERP metrics with metrics from other search engines that account, let say, for user's query behaviors (i.e., search volume from a search engine or meta engine) to come up with a new metric is a highly questionable approach. SEOs, SEMs and keyword research firms should stay away from such practices before their metrics lose credibility in the industry or with clients." http://www.miislita.com/fractals/overlapping-patterns.html

Orion

During the last few weeks, my firm has been experimenting with using Google's and Overture's term suggestion tools to arrive at a master list that we then run through WordTracker (WT). While obviously not perfect, this process helps in arriving at acutally searched variations of core concept phrases. We then use WT only as one of several guides for final selection of terms to weave into text and code during SEO. I would be curious what you all think of this process. Below I copy a rather un-edited, internal procedures document that explains this process in more detail:

1. Collect words from client pages, from client, from competitor tags, as usual. Alphabetize. This is where creative lateral search for synonyms takes place. Serious brain work. Much of the following automated steps for gathering good phrases depends on the conceptually related breadth of terms produced in this step. Save this list as “Initial List.doc”

2. From this initial list, select a sublist of two-, maybe some three-, word phrases that cover all the relevant categories related to the client web site. Put this list below the initial list with space separating the two lists; we will call this second list the “concept list.” This concept list will be fed into thesaurus functions at Overture and Google, which will generate actually searched variations of the concept phrases fed into them.

3. http://inventory.overture.com/d/searchinventory/suggestion/ - Make a copy of the “Initial List” doc, and name it “Overture-Adwords.doc” Run each concept list phrase through this tool, cutting and pasting the table for each into this Overture-Adwords.doc, so that one table for each word is stacked below the next word’s. After each paste, delete items around 50 searches and below, though look for borderline cases that may have promise in relevance. Remember to save the number of monthly searches, so that we can refer to this doc later if needed.

4. https://adwords.google.com/select/main?cmd=KeywordSandbox Cut and past into this tool the concept list. Cut, paste the left results list below the last Overture table in the Overture-Adwords.doc. Save.

5. Copy Overture-Adwords.doc and rename the copy “initial long list.doc.” Mess with the lists in initial long list.doc to strip all formatting and have one long list of all the words. Then alphabetize the list. Then manually look through the list, deleting duplicates and irrelevant or faintly relevant items. Save. Place a space after every two pages worth of words, so that laer we can feed less than maximum 100 at a time into WT’s popularity search in step 2. This is the master list to feed into WT.

6. Import this master list into WT.

7. Skip WT’s “related Keywords” first step, and just use this imported list to do predict and KEI, as covered in out other procedures documents.

[jvracing]

After each paste, delete items around 50 searches and below, though look for borderline cases that may have promise in relevance.
Maybe it's just me, but that seems like an awfully high standard! I find that most of my conversions are from very low-traffic terms. Actually, I have very few terms that are significantly over the 50 searches per day threshold. I guess I must be really over-niche-ing... :rolleyes:

[DanThies]

You're suggesting that WT numbers are under-inflated, but I'm getting less than 10% of the daily traffic they say types in that term despite being #1 at all engines...

In any case, this is an eye-opening experience as most of my efforts in the past have resulted in much more logical numbers in comparison to what Overture and WT give me.
Wordtracker doesn't give you a traffic projection, they estimate the number of searches across all search engines. A 10% CTR for #1 listings across multiple engines is well within the range we'd expect from the data we've gathered. Considering that the engines you've cited don't add up to 100% of the search market, you may be doing very well.

[randfish]

Thank you for all of the responses. There is some excellent information in this thread that is not often covered in the SEO world.

I did note that Google is using my title tag (which I like), but is pulling the description tag from some very old text - almost 3 months old that has since been taken off the page. They spider me daily, so I assume that eventually, they will change the listing text. I have tried a re-write of some pieces to hopefully attract a little more attention when they finally do get around to updating the description.

[NuevoJefe]

Google's been experimenting with reverting to meta-descriptions and also using DMOZ titles in place of listed sites' titles.

Also, AFAIK it's common for SEO'd terms to be inflated a good bit on WT and OV.

[randfish]

NJ - Thanks a ton! I was wondering where Google was getting that lousy description, now I realize I had forgotten all about my meta tags, since I assumed GG didn't use them. Now I can go and change it rather than continuing to wait around for them to update it. I should have known better since they cache my page daily ;)

[NuevoJefe]

You're welcome! :0) Hey, a 24 hour fix... pretty rare these days!

Yea, I imagine many people are getting bizzare swings in their click-thru rate not realizing that their poorly written (stuffed or just otherwise bad) meta-descriptions are to blame. Because they've been considered by many to be worthless for quite awhile (I still always use them properly just for the sake of doing it) I bet this has some interesting results for google to analyze.

[AussieWebmaster]

I still have my DMOZ listing showing up....

[risk]

I'm very worried that this is uncommon for the industry. Have your (collective) experiences indicated that you receive about 10% of the traffic from being #1 in the SERPs at all the major engines. Are these two statistics that far off?

Thanks for your help!

just an idea, but perhaps a significant number of searchers are looking for a different type of site when they search for those terms? if the terms are what i think they are based on your previous posts, i can see myself searching for information explaining what this type of X is for, what the eligibility criteria are, perhaps some tips from SCORE or other similar organization etc.

as an example, people searching for 'liverpool football club' might want to read the latest team news, transfer rumours or perhaps download clips of King's or God's goals but not buy tickets or replica shirts.

just thought i'd throw it out there ;]

-rsk

[Mel]

A bit off topic, but I would be interested in a tool similar to WPG to check rankings but it requires each user's Google API to use for Google checking. It would also be nice if Google lifted its 1000 daily search limit. I think Yahoo API allows 10,000.

WPG 3.5 now has capability to use the Google API.

[St0n3y]

yeah, I think they released that very shortly after I posted.

See how much weight I have within the industry?!!!

:D