Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engine Marketing Strategies > Search Engine Optimization
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 06-26-2006   #1
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Angry www1, www2, www3

Hi all,

I have a domain's www.mydomainname.com, and now i set up
www1.mydomainname.com, www2.mydomainname.com and
www3.mydomainname.com. They are all available mirrors of www.mydomainname.com.

Is this treated as spam? Will it effect SERP for mydomainname.com.

Furthermore, today i run a command for 'site:www2.domain-name.com' on Google, many pages had been indexed by it yet. Our primary site(www.mydomainname.com) will be penalized by google? Or how to use robots.txt to block spiders from my mirror sites, just the main server (www).?

Thanks in advance,
zhan is offline   Reply With Quote
Old 06-26-2006   #2
Marcia
 
Marcia's Avatar
 
Join Date: Jun 2004
Location: Los Angeles, CA
Posts: 5,476
Marcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond repute
That's duplicate content, it's a problem in the making. A page should only show up with ONE only URL - not more than one.
Marcia is offline   Reply With Quote
Old 06-26-2006   #3
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Hello Marcia,

Thanks for the quick reply.

But I'm very confused. I've noticed many companies who implement multiple subdomain and not be penalized. I see they have been successful at having www spidered, but keeping Googlebot off www2 and www3 servers:

www.cnn.com - 399,000 results
www2.cnn.com - 607 results
www3.cnn.com - 10,900 results

Can you give insight on how they're doing this? Thanks again.
zhan is offline   Reply With Quote
Old 06-27-2006   #4
seebach
Member
 
Join Date: Jun 2005
Posts: 9
seebach is on a distinguished road
You could simply put in a robots.txt file which deny's google access on the other servers.
seebach is offline   Reply With Quote
Old 06-27-2006   #5
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Could you further tell me how to use robots.txt to deny googlebot access my other servers(www1,www2,www3)? Thanks again.

Quote:
Originally Posted by seebach
You could simply put in a robots.txt file which deny's google access on the other servers.
zhan is offline   Reply With Quote
Old 06-27-2006   #6
seebach
Member
 
Join Date: Jun 2005
Posts: 9
seebach is on a distinguished road
Sure

Read this
http://www.javascriptkit.com/howto/robots.shtml

its very simple and is done by adding a sigle file to your server that is called robots.txt and contains:
User-agent: *
Disallow: /

/Seebach
seebach is offline   Reply With Quote
Old 07-01-2006   #7
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Hi Seebach,

Thanks again for your help.

I can't use those two line commands, because the multiple domains on one server(i,e www1,www2,www3 and WWW on one server). So I just don't know how to use robots.txt to block googlebot
from my mirror sites(www1,www2,www3)?


Thanks again.



Quote:
Originally Posted by seebach
Sure

Read this
http://www.javascriptkit.com/howto/robots.shtml

its very simple and is done by adding a sigle file to your server that is called robots.txt and contains:
User-agent: *
Disallow: /

/Seebach

Last edited by zhan : 07-01-2006 at 05:26 AM.
zhan is offline   Reply With Quote
Old 07-01-2006   #8
Marcia
 
Marcia's Avatar
 
Join Date: Jun 2004
Location: Los Angeles, CA
Posts: 5,476
Marcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond repute
You'll need a robots.txt for the root directory of each of those subdomains. There's a pretty simple explanation here with references:

http://modwest.com/help/kb2-197.html

It will go into the same directory as the main index pages for the subdomains.
Marcia is offline   Reply With Quote
Old 07-01-2006   #9
evilgreenmonkey
 
evilgreenmonkey's Avatar
 
Join Date: Feb 2006
Location: London, UK
Posts: 703
evilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud of
I can't believe that some companies (including a major airline) consider this to be good implementation of load balancing. Not only is it bad for SEO, it's poor/lazy technical planning. I looked at a website recently which used JavaScript to randomly choose one of four servers (www,www2,www3,www4) in an attempt to balance load. If www2 went down, the JS would still send a quarter of new visitors to the server and people with www2 in their bookmarks, history or search results would also end up visiting a broken server. Unsurprisingly the website with four times more indexed pages then actual pages didn't even rank for their own brand name.

Not a dig at you zhan, just at the people who should no better.



Rob
evilgreenmonkey is offline   Reply With Quote
Old 07-01-2006   #10
evilgreenmonkey
 
evilgreenmonkey's Avatar
 
Join Date: Feb 2006
Location: London, UK
Posts: 703
evilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud of
Quote:
Originally Posted by zhan
I can't use those two line commands, because the multiple domains on one server(i,e www1,www2,www3 and WWW on one server). So I just don't know how to use robots.txt to block googlebot
from my mirror sites(www1,www2,www3)?
Are you saying that the 4 subdomains all go to the same web folder on a server, or just hosted on the same server?

If they are pointed to different folders, you'll be able to use Marcia's advice, otherwise - can you tell us whether the server uses Apache or IIS web server and what code your pages are programmed in (HTML,PHP,ASP,CFML etc)?
evilgreenmonkey is offline   Reply With Quote
Old 07-01-2006   #11
Marcia
 
Marcia's Avatar
 
Join Date: Jun 2004
Location: Los Angeles, CA
Posts: 5,476
Marcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond repute
Each of the subdomains is equivalent to being a separate domain, and each needs its own robots.txt file

User-agent: *
Disallow: /

That will go into the root directory folder of each of the subdomains, the same folder where you've got the main index page for the subdomain. You'll find complete documentation for the protocol here at this site:

Robots Exclusion Protocol
Marcia is offline   Reply With Quote
Old 07-01-2006   #12
g1smd
Member
 
Join Date: Jun 2006
Location: UK
Posts: 253
g1smd will become famous soon enough
If the four subdomains resolve to the same physical place, then there is absolutely no need to have four subdomains in the first place.
g1smd is offline   Reply With Quote
Old 07-02-2006   #13
Marcia
 
Marcia's Avatar
 
Join Date: Jun 2004
Location: Los Angeles, CA
Posts: 5,476
Marcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond reputeMarcia has a reputation beyond repute
I haven't got a clue why there are extra subdomains and what they actually resolve to.
Marcia is offline   Reply With Quote
Old 07-02-2006   #14
evilgreenmonkey
 
evilgreenmonkey's Avatar
 
Join Date: Feb 2006
Location: London, UK
Posts: 703
evilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud of
:scratches head and backs away slowly:

evilgreenmonkey is offline   Reply With Quote
Old 07-03-2006   #15
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Many thanks for all the inputs.

The 4 subdomains all go to the same web folder on a IIs-based server, and all the pages are programmed in simple HTML code. So i can' t put robots.txt for the root directory of each of those subdomains. Any ideas?



Quote:
Originally Posted by evilgreenmonkey
Are you saying that the 4 subdomains all go to the same web folder on a server, or just hosted on the same server?

If they are pointed to different folders, you'll be able to use Marcia's advice, otherwise - can you tell us whether the server uses Apache or IIS web server and what code your pages are programmed in (HTML,PHP,ASP,CFML etc)?
zhan is offline   Reply With Quote
Old 07-03-2006   #16
zhan
Member
 
Join Date: Jun 2006
Posts: 6
zhan is on a distinguished road
Many sites have also set up many extra subdomains for load balancing, but we just use them to track sales for each sale representative.

Quote:
Originally Posted by Marcia
I haven't got a clue why there are extra subdomains and what they actually resolve to.
zhan is offline   Reply With Quote
Old 07-03-2006   #17
evilgreenmonkey
 
evilgreenmonkey's Avatar
 
Join Date: Feb 2006
Location: London, UK
Posts: 703
evilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud ofevilgreenmonkey has much to be proud of
Quote:
Originally Posted by zhan
Many sites have also set up many extra subdomains for load balancing, but we just use them to track sales for each sale representative.
If this is all you're doing it for, simply 301 each subdomain, you'll still record the hit in your web logs. It's really not a good way to do this kind of thing, if you're tracking sales, use a proper affiliate software or use folder based redirects such as http://www.domain.com/offer123



Rob
evilgreenmonkey is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off