PDA

View Full Version : Are Commas Rather Than Slashes Slowing Our Spidering?


guarnibl
03-12-2006, 01:01 AM
Hi guys,

I help manage/run/develop a website called www.starcraft.org (part of the ORG Network). With the recent development of my content management system that we used with Starcraft.org, we've been seeing a lot more pages spidered by google for our sites.

However -- one thing concerns me: Our sister site, www.warcraft.org, has WAY more pages spidered than ours.

I don't really understand this... they have over 500,000 pages indexed... we have less than 100,000. The only differences that I really see is that our site has meta keywords, meta descriptions on each page.... and that on some cases, we do use some different characters in the URL's.

I.e., we use commas instead of more /'s to differentiate between listings of pages. For example, on some places, warcraft.org would use /maps/1 to indicate page 1, where as starcraft.org would use /maps,1. Is this affecting us at all?

This is definately confusing me and I'm just wondering if maybe someone here could give me some insight as to what's happening here, because I'm quite confused. Thanks.

TheWizardofSeCA
03-12-2006, 05:57 AM
I'm not able to load your website at all right now.

Ping/Traceroutes all time out as well - did you take it down?

guarnibl
03-12-2006, 11:53 AM
Nope, works fine for me. However, about 5 AM EST a bunch of cron jobs run and make the site inaccessable for about 5-7 minutes.

sootledir
03-12-2006, 07:48 PM
When you say sister site, what do you mean? Do the domains point to different servers?

guarnibl
03-13-2006, 03:12 AM
Both domains are hosted on completely different servers, on completely different DNS servers. It's a sister site ... meaning it's part of the same network of sites. We all help each other out on certain things :)

Anyway -- if someone could please give me some insight on this that'd be sweet... otherwise, not a problem.

guarnibl
03-22-2006, 03:51 AM
bump -- still looking for an answer to this question if possible.