PDA

View Full Version : Robots.txt vs. .htaccess


highbids
11-13-2008, 01:17 PM
I have a directory I use with some php redirect scripts for cloaking my affiliate urls
I don't want indexed by the search engines.

I put a robots.txt file in the recomends directory
I dont want the bots to index like this.

User-agent: *
Disallow: /recommends/

Is that the place to put the robots.txt file & would a .htaccess.txt be better & where should
ether one be placed at in the root or the directory I don't want index.

AussieWebmaster
11-13-2008, 04:14 PM
no you put the robots.txt file in the root directory then you write the code for blocking your script pages in the robots.txt file - I would also add noindex to the head tag of the pages containing the codes

highbids
11-13-2008, 04:38 PM
Here's the php redirect.

<?php
header('Location:http://affiliateprogram.com');
?>

Is this correct when you say add the noindex to the
header, like this.

noindex

<?php
header('Location:http://affiliateprogram.com');
?>