Google is showing URLs instead of Title in (site:mysite.com) search results

[Jorge]

Has anyone noticed many random pages are now appearing with the url instead of the title in search results when using site:yoursite.com ? It is happening to some of my pages but I noticed it is happenning to other sites as well. An example: site:linux.com.

[Marcia]

Jorge, that will happen sometimes when pages are just being picked up, but more often when they're in the process of slipping out of the regular index.

For your site, make sure there's no reason for duplicates to be detected.

[Everyman]

This phenomenon is widespread. In my estimation it has nothing to do with duplicate pages.

Here's how you can zero in on what might be the problem:

You need to find a word that's on nearly every indexed page, but is not in the URL. In the case of linux.com, the word "reserved" is used at the bottom, as in "All rights reserved."

site:www.linux.com reserved -- this will show you indexed pages
site:www.linux.com -reserved -- this will show you URL-only pages

In the case of www.linux.com, it looks to me like the ID number in the QUERY_STRING is a major problem.

However, this URL-only phenomenon is all over the place, particularly for large sites.

Try this:

site:forums.searchenginewatch.com jupitermedia
site:forums.searchenginewatch.com -jupitermedia

Now it's not so clear what the problem is.

How about this:

site:www.namebase.org "home page"
site:www.namebase.org -"home page"

There are no numbers in the filenames at all, and there are no duplicates on this site.

If you want my theory, I say that Google is broken (http://www.google-watch.org/broken.html). They've run out of docID numbers and can't index all the pages they crawl.

[Jorge]

I hope you are right and it is not that I am being kicked out. I heard something similar has happened before with google not being able to handle the work load.

[Everyman]

New essay on this topic: Google is dying: Death by a billion cuts (http://www.google-watch.org/dying.html)

[cariboo]

New essay on this topic: Google is dying: Death by a billion cuts

Oh no ! :eek:

Please, not this "4digit DocId" thing again... :D

As you pointed it out Everyman, this phenomenon is now widespread... So widespread that it raises a few questions...

And this situation has been lasting for weeks, for months now... This doesn't look like a temporary crawling problem as before. In my opinion, it reveals a change in the way Google crawls sites now.

I believe Googlebot crawls less pages than it used to... For relevancy, it's not useful to crawl all the web each day. You'd rather crawl often "important" pages, and pages that change often. I think Google has decided to crawl these kind of pages more often, and others less.

It's only a theory.

But I can't buy some of the other explanations : like Google experimenting technical problems. It lasted too long, it's too widespread. I don't think they lack bandwith, or computing power, or that Googlebot is broken...