critter
12-17-2004, 10:49 AM
HEY All!
Client just implemented an xml file. The file is used for admin purposes and is not linked nor referenced from the homepage.
My question is I do not want the search engine spiders to visit this xml file as it has no links and no content....
Should I add a link oon the xml file back to the index page just in case? Is it safe, or can you exclude the xml file if it's in the root folder using your robots.txt file? Or do I not need to worry about the spiders visiting this xml file at all?
Thanks
Critter
Client just implemented an xml file. The file is used for admin purposes and is not linked nor referenced from the homepage.
My question is I do not want the search engine spiders to visit this xml file as it has no links and no content....
Should I add a link oon the xml file back to the index page just in case? Is it safe, or can you exclude the xml file if it's in the root folder using your robots.txt file? Or do I not need to worry about the spiders visiting this xml file at all?
Thanks
Critter