PDA

View Full Version : Robot Exclude of a Subdirectory


sackheim
06-28-2006, 07:09 PM
I would like to have Google exclude a test site we have on our web server.

The url of the client test site is:

clientname.ourcompanyurl.com

Would this robots.txt code work?:
User-agent: *
Disallow: /clientname/

g1smd
06-28-2006, 07:52 PM
If their test site is in the root of the server, then you need:

Disallow: /


If it is inside a folder then you need:

Disallow: /clien

with enough of the beginning of the folder name to make it not match any other content that you do not want to be blocked.



Beware that Disallow: /client/ does not block a request for www.domain.com/client

Beware that a request for www.domain.com/client will serve up an index page from that folder.