Googles Index Increase - Truth or Fiction?

[bobmutch]

Well I found time to do an update on one of my little projects I am playing with. I have ~520 links pointed to my home page seocompany.ca/ with "bob mutch" as the anchor text and there is no no-page optimization done for the key word on that page at all.

Then I cranked out on Oct 27/2004 a file seocompany.ca/bob-mutch.html and put all the on-page optimization on that page but no off-page optimization at all other than one link so the search engines would know it existed.

While I am not going to bore you with my study I though it was interesting that after Google anounced a 5mill to 8mill increase in there index the number of page results for "bob mutch" and "mutch" increase in Google but in none of the other 2 search engines.

The results for the keyword 'Bob Mutch' in Google is 42,700 (20,000 Oct 27/2004) in Yahoo is 17,000 (17,400 Oct 27/2004) and in MSN it is 2,584 (3,159 Oct 27/2004).

The results for the keyword 'Mutch' in Google is 195,000 (96,000 Oct 27/2004) in Yahoo it is 114,000 (121,000 Oct 27/2004) and in MSN it is 21,638 (23,437 Oct 27/2004).

Google shows a 213% increase in its index for the keyword 'Mutch' and 203% increase in its index for the keyword 'Bob Mutch'. At the same time both Yahoo and Search MSN dropped for both keywords.

I am wondering if anyone else has seen the same thing? Is it possible there Google when out and found another 99,000 page with the word 'Mutch' in them or another 22,700 page with 'Bob Mutch' in them? Or did Google just "change" the way they report the pages and relize they had the pages all along and just were not reporting them?

What do you all think?

For any interested in my study page it is at seocompany.ca/bob-mutch.html

[Robert_Charlton]

Google shows a 213% increase in its index...

Wild guesses...

Perhaps a different method of counting the www and non-www pages comes close to doubling the index, and the inclusion of links to pages blocked by robots.txt adds the extra 13%.

Again, this is complete speculation... just tossing it out.

[Dave Hawley]

Googles Index Increase - Truth or Fiction?Truth.

It has been long suspected that Google was up to something prior to the formal announcement. The formal announcement just confirmed what is most likely was, i.e increasing their database.

In fact, I would say their database increases on a daily bases. This was just one HUGE increase. I suspect it not the last by any stretch.

BTW. Mutch is also a word and Bob is an extremely common name and a word. Sorry to burst your ego bubble ;) :)

[Marcia]

>>For any interested in my study page it is at seocompany.ca/bob-mutch.com

No, I went to the page in hopes of adding a live link to it and got a custom 404 page with a site map. So there goes the study, Bob - unless I'm too tired and am totally missing something here. Is the study by any chance on another page, or is there something we should know about fiddling around with custom 404 pages that we haven't caught on to yet?

Some folks are always looking for new tricks, you know. :D

[bobmutch]

Marcia: seocompany.ca/bob-mutch.html might work better : )

Dave Hawley: I think I know Bob and Mutch are words, and I am also aware that Bob is a very common name. Words can be names and names are words. More than two words are a phrase. Not sure what your point is.

[powerofeyes]

I think you are still wrong bob it is seocompany.ca/bob-mutch.html

[Nick W]

The above works for me but what bit of the page should i be looking at?

[bobmutch]

Nick W: Depends on what you are looking for. It is just my page with ALL on-page optimization and no links pointed to it other than one link so it will get indexed. Also can you kill the link please.

[Nick W]

Right, with you. thanks...

[Robert_Charlton]

Is it possible there Google when out and found another 99,000 page with the word 'Mutch' in them or another 22,700 page with 'Bob Mutch' in them?

Bob - Fess up... you've been busy building doorway pages. ;)

To try to swing this back on topic, I'm thinking that much of the change may be in the way Google is counting pages.

Way back, GoogleGuy essentially said that the 50% change in the backlink count after update Dominic happened because Google had become much better at estimating link counts. See msg #7 in:

http://www.webmasterworld.com/forum3/12801-1-15.htm

We're talking 50%... a factor of two. So, maybe the day before Microsoft launched its bigger beta, Google got better at estimating page counts also, by a factor of two... and in the other direction. Yes, page count and link count are apples and oranges, I know.

But except for links to blocked pages, I'm not seeing anything that suggests the real page count has gone up. I don't see, though, what Google sees... and, were it not for several other indexing issues that are clouding this particular announcement, and a bunch of reports like yours, I might tend to believe them.

[Dave Hawley]

Not sure what your point isThat after nearly doubling it's database it's no suprise that you find these extra pages. Especially when using at least one very common word/name.

Also, the numbers you are quoting are notoriously unreliable.

I must confess though, I'm struggling to see your point?

[Mel]

Hi Bob
When I search for "Bob Mutch" using the quotes I get only 6,190 results, so there are not all that many pages using the phrase Bob Mutch, though there might be a lot of pages using both words on the page somewhere (and its not a phrase search if you don't use the quotes). While I don't have the before comparasion for that term it is not such a large number for someone like your self that posts so prolifically as to be unbelievable.

Despite that I have seen too many reports that Google has lately been returning 2000 pages for site: searches on 1000 page sites to feel that they really have the method for estimating those numbers under total control.

Prior to Googles claimed expansion, a search for the
would return an estimate of 5 million results and now it produces an estimated 8 million results. Unless someone shows that this is in fact a Google scam, I am likely to believe that if they say there are that many pages then its true. They have too much to lose for it to be any other way.

It is interesting to me to note that the MSN beta search is claiming 5 billion pages in its index and reports around that number for a search on the.

[bobmutch]

Mel: Well I just thought it was interesting that the results number for 'mutch' and 'bob mutch' (with out the quotes) doubled at the same time that Google doubled there index. Also for a while 'the' was showing a results of 9 billion on Google.

Where do you think they came up with the extra 3 billion pages? Also that is 90TB of data if the pages were an average of 30KB each.

[Mel]

Bob No idea where they came up with the extra 3 billion pages (more like a 60% increase in size), but I would like to assume that it was a combination of new file types being included plus more spiders and more intensive spidering of sites, but somehow that does not quite square with more and more listings which are only links (not yet spidered) in google site: searches.

Do you have a before count of "bob mutch" before the announced increase in size?

[bobmutch]

Mel: No not for '"bob mutch"' only for 'bob mutch' and that was 20k.

Robert_Charlton: I read a number of the thread on the Nov 25th BL update and I didn't see the kind of jump you would expect to see in the BL numbers. Of course the BLs may just be a sample and not reflect how many you ready have. Your thoughts on this?

[Robert_Charlton]

Robert_Charlton: I read a number of the thread on the Nov 25th BL update and I didn't see the kind of jump you would expect to see in the BL numbers. Of course the BLs may just be a sample and not reflect how many you ready have. Your thoughts on this?

Bob - Basically, I haven't been paying much attention to Google backlinks since they've "randomized" their sample. I use Yahoo and MSN-Beta.

The Google backlinks are basically useless... lots of scraper sites that are running AdSense... search strings urls, etc... the kind of thing that alltheweb used to show on page 15 when you checked backlinks there.

Since you ask, though, I checked some sites that I used to monitor on Google and I see only an increase consistent with more inbounds, but that's about it.

If I understand the thrust of your question, you're perhaps saying that if my www/ non-www theory is correct to account for the doubling, then all samples on Google should roughly be doubled, including the backlinks. You're right... I don't see anything like that.

But, per my referenced GoogleGuy post above, it may be that Google had originally been counting both www and non-www backlinks and then stopped double-counting at the Dominic update, and that this new counting method persists even with whatever new methods of counting there may be now.

I hope I'm making sense. This is all speculation, of course.

I can't say that I paid that much attention to the Nov-25 BL update threads in particular. I did get some Google alerts around then that we'd gotten some more links, but they only reported the scraper sites... none of the good solid links that I know we also received.

[bobmutch]

Robert_Charlton: Well the sampling that Google shows with their link: command appears to be to be a about 20% give or take 20% of the links that Yahoo shows with their linkdomain: command. Now if that is the case in we should expect a jump of backlinks when the index doubled.

If there are 3 to 4 billion more pages I would think there would be more links some where. If there are more links and Google is 'kind of' showing a percent of the links then why wasn't there a BIG increase in links that would come with the BIG increase of pages in the index.

No of course Google could have 'fixed' the link command further so that it wouldn't show more links even with the increased links from the 'new' 3 to 4 billion pages.

I don't know, there just seems alot of inconsistencies in the whole increase. I am not saying there is not 8 billion pages but I sure am wondering about it!

[Mel]

Another possibilty is that many if not most of these new pages are not html pages at all but could be .pdf or .swf etc which would probably affect the link ratios.