note 78693 added to function.file-get-contents

Hi everybody! I am trying to write WebCrawler/Spider (as school project, and -of course- I am trying to be more wealthy than google ;o) So... I have big/small problem: I am using file_get_contents() (i've tryed fopen() too...). Crawler works 100% great, but sometimes it freezing. I have tryed to trace what function freezes, and i found it, it's file_get_contents()... So, i googled and found default_socket_timeout setting, i set it to 1, but sometimes its freezes and never get up again. I've done this example, so you can see, that it freezes after few iterations. I have supplyed URL, that causes freeze of my crawler (im not sure why...): #!/usr/bin/php <?php ini_set('default_socket_timeout',1); set_time_limit(0); //$url='http://ad.doubleclick.net/click'; $url='http://w.moreover.com/'; while(1) { @file_get_contents($url, false, null, 0, 10000); echo "#"; } ?> Of course, if somebody want be better than google, he have to have very good crawler. So I need very solid code, that can run and crawl a days without crash (like this one). Yeah, it's true, that this worked 1 or 2 hours before it crashes, or i stoped it, but the file_get_contents() doesn't work like i need. If you are interested to crawling, you can write me to YouSeekMe: 283//782//978 ;D And there are few statistics from my last session: +5431 URLs; 19292 Downloaded; UpTime: 21.5 mins; Buffered: 30 URLs; History: 1000 URLs; Speed: 4.22 URLs/s, 14.98 Downloads/s THX4AnyHelp ;o) ---- Server IP: 147.32.160.17 Probable Submitter: 213.220.241.158 ---- Manual Page -- http://www.php.net/manual/en/function.file-get-contents.php Edit -- https://master.php.net/note/edit/78693 Del: integrated -- https://master.php.net/note/delete/78693/integrated Del: useless -- https://master.php.net/note/delete/78693/useless Del: bad code -- https://master.php.net/note/delete/78693/bad+code Del: spam -- https://master.php.net/note/delete/78693/spam Del: non-english -- https://master.php.net/note/delete/78693/non-english Del: in docs -- https://master.php.net/note/delete/78693/in+docs Del: other reasons-- https://master.php.net/note/delete/78693 Reject -- https://master.php.net/note/reject/78693 Search -- https://master.php.net/manage/user-notes.php