Re: web spider?

Greg, > >i was wondering if it would be possible to make a web spider in PHP. > >they work by download a website, then following all the links on the > >website to other pages, and then following the links on that page... > > > >is this possible in PHP without a whole lot of work? i just want to keep > >a counter of how many links (not e-mail addresses or file downloads) it > >finds. if someone has the time, could you write up a quick recursive > >script that i could use as a basis of my project? > > Well, I love PHP, but this sounds more like a job for C/C++. The reason > being is the socket support in C/C++ is way better than that found in PHP. > Harder to code mind you, but better in the end. I learned from the manual and coded up a quick page-reader/status checker using fopen and fread - and thought I was fairly clever for something so 'quick and dirty'. However perhaps this approach would not work if something a bit more 'demanding' was required. Please would you explain the advantage/necessity for using sockets (would appreciate pointers to any reading/tutorials). Thanks, =dn