Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engine Marketing Strategies > Search Engine Optimization
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 05-21-2009   #1
maneetpuri
Member
 
Join Date: Sep 2008
Posts: 104
maneetpuri is on a distinguished road
should I block robots...........?

Hi

I keep updating my website regularly with some new changes but every time I work on the backend I block robots visits on my website to prevent wrong or incomplete indexing by them in their database.

My question is should I block robots while I am working on the scripts of my website or let them do their work while I am working on source?

Can anyone help me with the correct answer?

Thanks in advace!
maneetpuri is offline   Reply With Quote
Old 05-21-2009   #2
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,154
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
Re: should I block robots...........?

I would not bother blocking robots but if you are going all crazy maybe put it all in a folder and block that while working - just never like idea of blocking the robots on mass
__________________
Bruce Gillmer - MTV EMA
AussieWebmaster is offline   Reply With Quote
Old 05-21-2009   #3
jag
Forums Valuator, SEW
 
jag's Avatar
 
Join Date: May 2006
Location: CBE
Posts: 1,000
jag is a jewel in the roughjag is a jewel in the roughjag is a jewel in the roughjag is a jewel in the rough
Re: should I block robots...........?

I go with aussie statement

Best.
jag is offline   Reply With Quote
Old 05-24-2009   #4
ChiefLee
Promotional Products Guru
 
Join Date: May 2009
Location: Lawrence KS
Posts: 36
ChiefLee will become famous soon enough
Re: should I block robots...........?

Use your robots.txt file to tell the robots what you want them to see. Only block what you don't want them to see.
ChiefLee is offline   Reply With Quote
Old 05-24-2009   #5
AaliyahRoma
Member
 
Join Date: Apr 2009
Posts: 6
AaliyahRoma is on a distinguished road
Re: should I block robots...........?

Yes, naver or cuill like robots just make server busy.
AaliyahRoma is offline   Reply With Quote
Old 06-19-2009   #6
maneetpuri
Member
 
Join Date: Sep 2008
Posts: 104
maneetpuri is on a distinguished road
Re: should I block robots...........?

Hi

Thanks to all for your helpful suggestions I will keep them in mind while I am working on backend.

BTW AliyahRoma how can I keep my server busy while I am working on my website backend? Is there any special way to do this?
maneetpuri is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
need clarification on the differences between robots.txt, meta robots, and nofollow thwart Search Engine Optimization 16 01-10-2008 02:02 AM
Is My Robots.txt Block Right? onedodd Google Web Search 44 12-09-2005 10:46 AM
Google Adwords Robots Handling Change? financialhost Google AdWords 1 08-19-2005 12:58 PM