PDA

View Full Version : crawler code


francophilie
12-03-2006, 01:30 PM
i have crawler code which is bring the urls and links .
i'm creating file with name links.txt and i put in it by default the first url:
http://www.syntaxcms.org/

the result is good in the first time ..the code is bring soms links of that url.
At the second one ,i'm puting to the file the second url :

and the result ....the warnning message which is:

-------------------------------------------------------------
http://www.syntaxcms.org/http://blog.syntaxcms.org/

Warning: fopen(http://www.syntaxcms.org/http://blog.syntaxcms.org/ ) [function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\AppServ\www\crawlered.php on line 30

-------------------------------------------------------------
any one can help me..please.