Rob
02-09-2006, 06:58 PM
Hi
I'm hearing from a few clients that the Mozilla Googlebot has been acting up a lot lately. In one case it was hitting a site so hard it brought the server down and most recently it's requesting pages from a site that have never existed nor have any links to them.
The second site is a directory type site which offers accomodations listings across the US and it looks like Googlebot has realized this (through past crawls) and is now trying different combinations of page names to see if they load.
For example, on the site there are 40 Oregon cities listed yet the Mozilla Googlebot is trying to access other city pages that have never (and currently do not) exist.
All these requests are coming from:
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.144
Anyone else experiencing similar issues with Googlebot?
I'm hearing from a few clients that the Mozilla Googlebot has been acting up a lot lately. In one case it was hitting a site so hard it brought the server down and most recently it's requesting pages from a site that have never existed nor have any links to them.
The second site is a directory type site which offers accomodations listings across the US and it looks like Googlebot has realized this (through past crawls) and is now trying different combinations of page names to see if they load.
For example, on the site there are 40 Oregon cities listed yet the Mozilla Googlebot is trying to access other city pages that have never (and currently do not) exist.
All these requests are coming from:
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.144
Anyone else experiencing similar issues with Googlebot?