Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engines & Directories > Google > Other Google Issues
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 01-26-2005   #1
I, Brian
Whitehat on...Whitehat off...Whitehat on...Whitehat off...
 
Join Date: Jun 2004
Location: Scotland
Posts: 940
I, Brian is a glorious beacon of lightI, Brian is a glorious beacon of lightI, Brian is a glorious beacon of lightI, Brian is a glorious beacon of lightI, Brian is a glorious beacon of light
Supplemental index

Just what is the supplemental index anyway?

You know:
In order to show you the most relevant results, we have omitted some entries very similar to the 2 already displayed.
If you like, you can repeat the search with the omitted results included.



I had *presumed* before that the supplemental index was effectively composed of:

1. URLs found but not crawled
2. Effective duplication of query


However, I'm not properly certain of what the supplemental index is, is for, and how pages (even sites) actually end up on it.

Comments welcome.
I, Brian is offline   Reply With Quote
Old 01-26-2005   #2
telNform
White-Hat this!
 
Join Date: Jul 2004
Location: Los Angeles
Posts: 60
telNform is on a distinguished road
Duplicated Content

From what I understand this is content that Google considers to duplicate (same sources, similar content, etc ...)
telNform is offline   Reply With Quote
Old 01-26-2005   #3
Chris_D
 
Chris_D's Avatar
 
Join Date: Jun 2004
Location: Sydney Australia
Posts: 1,099
Chris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud of
Hi Brian,

I think there are 2 separate issues here.

It is my understanding that Google generally limits the number of results from e.g. any domain. Eg in the normal serps, for a normal keyword search, you get max 2 from one domain (one indented, directly underneath).

If you watch the address line in your browser, when you hit the "repeat the search with the omitted results included" link, it appends "&filter=0".

This is effectively the Google filter which allows more results from a domain to be displayed. So 'more results' using "&filter=0" could bring up 'supplemental results' pages, or 'normal pages'.

eg try it by searching e.g. your own site for a single word eg site:www.domain.com blah (where blah is a common word on every page of the site) and then hit "repeat the search with the omitted results included" - it may bring up both 'normal' and 'supplemental results' pages.

In my testing, I've seen pages listed as 'supplemental results' where they are pages that were previously indexed - but which are no longer 'live' pages. eg if you delete an already indexed page - for a little while it will appear as a supplemental result, as Google continues to try to request it (look in the logs). It appears as a 'supplemental result' until Google finally decides that its gone for good - when it is finally removed.

That's my understanding - but I'm sure the 'supplemental results' could be used by Google for other reasons. I've even seen it where a DDOS attack on a webserver has been so prolonged that googlebot couldn't access and refresh pages (ie any request got a 404). Many pages went into 'supplemental results' for a period of a week or so - and they all came back as 'normal' results after the DDOS attack finished. But the principal is the same - its a page Google knows about - but can't access.

Best

Chris_D
Chris_D is offline   Reply With Quote
Old 01-26-2005   #4
JohnW
 
JohnW's Avatar
 
Join Date: Jun 2004
Location: Virginia Beach, VA.
Posts: 976
JohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud ofJohnW has much to be proud of
Sometimes it's easier to just manually add the &filter=0 to the url, than to find your way to the end of the results and repeat the search.

One thing for sure about the supplemental index, it's not where you want to be. As far as I can tell it's just one small step from not being in the index at all. I have always suspected that it is where pages go when they are considered totally unimportant, as in the case of penalization for duplicate content as was suggested. I have also seen hijacked pages stop there for a while on their way out of the index - which depending on perspective, may the same thing as duplicate content. I would like to know for sure what it means but we may never find out the real answer- even if they do tell us. Has anyone had any luck in recovering a page from this status?
JohnW is offline   Reply With Quote
Old 01-26-2005   #5
Chris_D
 
Chris_D's Avatar
 
Join Date: Jun 2004
Location: Sydney Australia
Posts: 1,099
Chris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud ofChris_D has much to be proud of
John asked:

Quote:
Has anyone had any luck in recovering a page from this status?
This was my experience:

Quote:
I've even seen it where a DDOS attack on a webserver has been so prolonged that googlebot couldn't access and refresh pages (ie any request got a 404). Many pages went into 'supplemental results' for a period of a week or so - and they all came back as 'normal' results after the DDOS attack finished.
I think, as you said "it's just one small step from not being in the index at all."

In a way - its 'Google purgatory'! You are in a place somewhere in between the main Google index and Google oblivion......

I think this relates to the 'supplemental result' topic:

Quote:
Your site may not have been reachable when we tried to crawl it because of network or hosting problems. When this happens, we retry multiple times, but if the site cannot be crawled, it will not be listed in our current index. If it was a transient problem, your site will likely show up in the next index, which will be completed in a few weeks.
http://www.google.com/intl/en/webmasters/2.html#B3

So there are 'innocent' ways your pages can get there e.g. if there are site related issues preventing Googlebot from spidering known indexed urls (which are recoverable); your pages can get there by deleting pages from your site (which won't be recoverable - as you deleted the pages); and there are also possibly other reasons relating to Google removing the pages.....
Chris_D is offline   Reply With Quote
Old 01-26-2005   #6
Mel
Just the facts ma'm
 
Join Date: Jun 2004
Location: Malaysia
Posts: 793
Mel is just really niceMel is just really niceMel is just really niceMel is just really nice
Google says that:

Quote:
Google augments results for difficult queries by searching a supplemental collection of more web pages. Results from this index are marked in green as "Supplemental".
I do not think this is necessarily associated with the repeat the search with the omitted results included message but is a seperate index which is used only when Google cannot find enough results in is main index to fully satisfy the query. I do not think it has anything to do server outages, but with pages that Google for one reason or another sees as being less relevant than pages in the main index.

What the critera for inclusion in the supplemental index are is not clear to me, but I do agree that its not the index of preference for webmasters.
__________________
Mel Nelson
Expert SEO Dont settle for average SEO
Singapore Search Engine Optimization and web design
Mel is offline   Reply With Quote
Old 01-27-2005   #7
MHS
Denver, Colorado WebSite Development
 
Join Date: Jan 2005
Location: Denver,Colorado
Posts: 3
MHS is on a distinguished road
Would this put your CACHED Results in 1969 LAND ALSO?
MHS is offline   Reply With Quote
Old 02-01-2005   #8
siteseo
Member
 
Join Date: Nov 2004
Posts: 24
siteseo will become famous soon enough
Here's an email I received from Google's User Support group when I inquired (via our AdWords rep) whether dupe content relegated a page to the supplemental results index (I'm 99% convinced it does, by the way):

"...supplemental sites are part of Google's auxiliary index. We're able to place fewer restraints on sites that we crawl for this auxiliary or supplemental index than sites that are crawled for our main index. For example, the number of parameters in a URL might exclude a site from being crawled for inclusion in our main index; however, it could still be crawled and added to our supplemental index.
The index in which a site is included is completely automated; there's no way you can select or change the index in which your site appears. Please be assured that the index in which a site is included does not affect its PageRank."

So being in the supp index doesn't affect your PR (whoopie) but it definitely impacts your rankings in the serps. Supp pages never rank on competitive terms.
siteseo is offline   Reply With Quote
Old 02-02-2005   #9
Ben Anderson
Member
 
Join Date: Jan 2005
Posts: 8
Ben Anderson is on a distinguished road
refreshed Pages

I had some of my old-site pages in the supp index. When I did a major refresh to this site, most of the pages came back to the main index. So, you can try also that...
Ben Anderson is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off