Are You Ready to Explode Your Keyword Lists?

[Nacho]

http://www.prweb.com/releases/2004/8/prweb152879.htm

According to Amit Singhal, principal scientist at Google, over 50 percent of the 200 million searches performed a day have never been searched before.

Wow! I'm speachless.

[I, Brian]

Interesting if there's any stats on that - but the key concern is really - how well do these terms convert, and are we already capturing them?

On-page content is the prime way to capture lots of different keyword strings of varying complexity, anyway.

In fact, one of those new keyword strings might be "complexity engine keyword search department site info list" - all of the words are on this page as content.

[seobook]

in their Haifa presentation they said that they have over 100 million unique queries each day.

[orion]

Hi, Nacho

Thanks for the two emails. I will reply to the first over the weekend. Thank you also for inviting me to stop by this thread. This is an interesting subject Gary posted back in 08/04 here http://forums.searchenginewatch.com/...=8009#post8009

What Dr. Amit Singhal, a well respected scientist, said about Web searches (quote)

http://www.research.ibm.com/haifa/Wo...pers/haifa.pdf

• Search space is HUGE
– Over 200 million queries a day
– Over 100 million are unique
– Need 2700 queries for a 5% (700 for 10%) improvement to be meaningful at 95% confidence
• Search space is varied
– Serve 90 different languages
– Can’t have a catastrophic failure in any
– Monitoring every part of the system is non-trivial
• IR style evaluation
– Incredibly expensive
– Always out of date


At one point in the presentation he admitted the following (quote)

• Clicks
– Incredibly noisy
– A click on a result does not mean a vote for it
• The destination may just be a traffic peddler
• User taken to some other site
• If anything, this (clicked) result was BAD

Another old stats can be found in his pdf presentation.

He also discusses NLP and noun phrase parsing

As I posted back in 08-05,

"After reading the pdf presentation, it occurred to me doing a search for "terrible candidate" (without quotes). One can see #2 out of 340,000 results in Google this site www.robbernard.com/archives/001099.html repeating so often the first term ("terrible") and not the other and getting away with the spamming of the first term. This makes me think whether or not Google is now checking for key phrase spamming more than for single keyword spamming. Of course I could be wrong and a single case does not amount to a pattern. I think this is worth to investigate."

A search today shows that the above page is #3 out of 302,000 results for "terrible candidate". (this example does not necessarily reflect my political opinion about the presidential candidates).

The point is that their NLP algorithm seem to work with noun phrases and phrases in general but not for unique single terms (noun-noun) in which one term is abused. I know of a SEO company is exploiting this with entry pages to its ppc ads.

Orion

[bradbyrd]

this data is an extrapolation made from a single powerpoint slide presented by Google a while back...

http://www.haifa.il.ibm.com/Workshop...pers/haifa.pdf

basically, it just says:

Search space is HUGE:
- Over 200 million queries per day
- Over 100 million are unique

to me, this usefulness of this data is questionable at best. its difficult to deduce amit's real intended meaning from this slide. i would suspect that he just means that there are 100 million different keywords/phrases searched for each day. this does not read, to me, that there are 100 million single search keywords/phrases...

in any event, if this somewhat spurious deduction is true, then it points to the importance of building intelligent keyword lists that are blends of both exact/phrase/broad match types.

has anyone seen/heard these types of numbers from sources other than the one linked to above? i think the question of one-off search queries is an interesting one, because it definitely affects SEM strategies re: planning, etc.

if anyone at Google wants to chime it, we'd love to hear more details!

[orion]

I agree. I have copy of the 3.5 Mb pdf presentation. It has old data, even data from 2001.

According to Gary, "the presentation was given in Israel on February 16th at IBM’s Second Search and Collaboration Seminar 2004." However, some stat is old. Some ideas are not that old and honestly are quite interesting.

I'm more concerned about the ideas presented rather than the stats. I found their NLP and noun phrase approach not working that well.

Orion

[Mikkel deMib Svendsen]

In the many years I've been fortunate enough to follow search patters across many sites and engines I have found exactly the same. In fact, I would say that the percentage of unique queries each day is more in the 60-70% region.

However, even though the unique phrases may be unique there are still clearly identifiable patterns. By identifying those patters you can still target most of those keywords - even if you don't know then yet.

[Nick W]

>>even if you don't know then yet.

And if you dont, install a loggable site search on your site

Nick

[shorebreak]

Hi Mikkel,

How would you describe those 'identifiable patterns'? Are the patterns related to a question being phrased, or certain categories of unique queries being discernible somehow? Would love to hear what you have to say!

Shorebreak

[Mikkel deMib Svendsen]

Sorry, I miss-placed this post - it belonged in the padded room

[AussieWebmaster]

in any event, if this somewhat spurious deduction is true, then it points to the importance of building intelligent keyword lists that are blends of both exact/phrase/broad match types.


The thing is right in there... if you cover your bases this way, the search may be unique but you have enough to catch it... to create lists of that many variations of a keyword phrase would drive you crazy... catch them with broad... or start working on a multimillion keyword list!

[orion]

Hi Mikkel

What you are describing are just combinations of query modes with the resulting results being relevant. The general form you are using is

Q = k1 + k2

where k1 = "Paul Zimmer" with quotes (EXACT mode)

"Paul Zimmer" -horse
This is an EXACT and NOT query combination

The next two are just an EXACT and FIND ALL combination where the "Paul Zimmer" can occur before or after k2.

"Paul Zimmer" +seo

or

"Paul Zimmer" +search


Orion

[Mikkel deMib Svendsen]

I think a couple of posts might have been misplaced - sorry for my mess-up. My "paul Zimmerman" posts belong in the "Whats your Google Count" thread in the padded room
http://forums.searchenginewatch.com/...ead.php?t=1368

Sorry for the confusion

[Mikkel deMib Svendsen]

Hi Mikkel,

How would you describe those 'identifiable patterns'?

This time I think I got both the question and forum right this time


I have build my own tools to identify and report to clients such patters across large sets of keywords. The keyword data I either get from WordTracker, Overture or my own local data sets - depending on the project and target language.

Next I theme all the keywords into logical groups - most often by subject or keyword. When thats done I start to look at the patters in each group: singular or plural forms, frequency of adjectives, length of queries etc.

After I've done all the analysis I output the final report in graphs and tables that really gives me a very good overview of the patterns within each group. It shows me where to put my focus to cover the majority of combinations - and how.

[pleeker]

Time out. I'm reading this entirely differently than what's suggested in the original post.

When Amit says 100 million of the daily queries are "unique", that does not mean "never been searched before". It does mean "only searched once on this date".

I mean, jeez ... human language itself isn't expanding at such a rate as to allow 100 million brand new, never-before-typed words/phrases to be queried on Google EVERY DAY. No way.

[AussieWebmaster]

I am voting for unique on any given day...

[aviener]

Doesn't it make you scratch your head when you think about the fact that 50% of searches are new, and yet Overture disallows terms based on the fact that there isn't enough traffic....

[pleeker]

Originally Posted by aviener

Doesn't it make you scratch your head when you think about the fact that 50% of searches are new, and yet Overture disallows terms based on the fact that there isn't enough traffic....

Well, no, because (again) they don't mean "never been searched before". They mean unique on that day.