Re: multi-threading within php?
| From: | bvr | Date: | Thu, 24 Jan 2002 14:32:42 +0000 |
| Subject: | Re: multi-threading within php? | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-81879@lists.php.net to get a copy of this message | ||
You may want to use 'wget' spider function, then parse the files with PHP if necessary.
http://wget.sunsite.dk/
bvr.
On Thu, 24 Jan 2002 14:52:54 +0100, Mattias Andersson wrote:
>Hello!
>
>I was making a spider for a simple searchengine and all was well until I
>started testing it on larger sites.
>The problem isn't that it doesn't work, it does, but it is very very slow,
>just handling one connection at the time.
>Basically, it just downloads a page, extract all links from it, then
>downloads those pages, extract their links, until all pages on a site has
>been downloaded and archived.
>Is there any way to do it in several threads so that it handles several
>connections simultaneously?
>If it would dynamically allocate threads that would be even better.
>
>Regards,
>Mattias Andersson
>
>Software Developer, humany AB
>Direct: 08-4540701
>Mobile: 0704-526685
>
>