View Full Version : Googlebot now Mozilla compatible.....?!
Darkon
02-21-2005, 11:33 AM
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
is what my site saw.... I also saw Yahoo using this technique. Does anyone know anything about their new agent? And if more SE's use it (besides Yahoo and Google)?
There is an extremely useful extension for Firefox called User Agent Switcher. It allows you to use Firefox with those sites that refuse to work with any browser other than IE. One other useful thing you can do with this extension is to set your user agent string so that it looks like you are the Googlebot. By doing this you can see what the site shows when it thinks the Googlebot has come calling. In other words, is it cloaking.
I suspect your 'new' Googlebot might just be someone browsing with Firefox and the User Agent Switcher set to 'Googlebot'.
Chris
Darkon
02-21-2005, 01:25 PM
:D That would be sneaky! Thanks for the tip though.. I'll check out the extention for Firefox. However...
1) There are more 'rumors' on other SE watch sites....
2) It was EXACTLY at the same time when the normal googlebot was spidering our site. Coincidence....??
I agree that it would be weird if a bot would use a mozilla environment to spider sites......
seomike
02-21-2005, 01:50 PM
I could list all the of the Gbots that use a Mozilla but there are too many.
Currently known gbots = 3,506
using Mozilla UA = 738
Darkon
02-21-2005, 02:26 PM
Is there a list or more information about these bots somewhere?
(I mean.. how did you get those numbers?)
seomike
02-21-2005, 10:50 PM
Fantomaster has a complete list in their spiderspyTM file. It's updated 4 times a day with all eyes out for new bots :D
http://www.fantomaster.com/fasvsspy01.html