RE: PHP web spyder? (missing attachment)

From: Date: Fri, 28 Jul 2000 18:44:20 +0000
Subject: RE: PHP web spyder? (missing attachment)
References: 1  Groups: php.general 
Request: Send a blank email to php-general+get-8835@lists.php.net to get a copy of this message
> could you send the source (the attachment was missing) > Hrmmm.. Since it looked like SOMETHING was attached when I saw my message show up on the lists I just assumed the attachment sent ok.. Guess not :( (sorry for all the duplicate messages.. didn't realize attachments were disallowed) Anyhow, Here's the source -------------------------------- #www_get.php3.. a quick and dirty example of using wget <? #this script will retrieve some data from a url using wget if(!$submit) { ?> <form name=url_submit action="<? echo $PHP_SELF; ?>" method="POST"> <input type=text name=url value="http://"> <input type=text name=url_debth value="4"> <input type=submit name=submit value="Get Data!"> </form> <? } else { #attempt to grab the base url if(preg_match("/http:\/\/(.+?)\//",$url,$matches)) { $base = $matches[1]; } #the reason I wanted to grab the base separately is so I could store this in a #database and show a link to the saved page using this information. $save_location = "/some/local/directory/maybe/in/your/httpd/docroot/$base"; system("wget -P $save_location -r -nH -l $url_debth -A jpg,gif,asp,php,php3,html,htm $url"); #what these options mean: -r == recursive # -nH == No Host, eg http://www.foo.com/subdir/index.html will NOT save in $save_location/www.foo.com/subdir/index.html # -l == recursive debth.. just in case you try to use a site with lots of debth to it and don't want that much data #- A == Allow files of the following comma delimited extensions (I don't think you actually need htm and html since it is assumed) #the last argument is the URL you want to get. echo "if all went well, you should see some data in $save_location/$base which you can access at http://www.your_domain.com/$base"; } ------------------------------------------ HTH -Marc- #/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/# #Marc Swanson | \|||/ # #MSwanson Consulting | /o o\ # #Phone: (603)868-1721 |-ooo----O----ooo-# #Fax: (603)868-1730 | Solutions in: # #marcswanson@mediaone.net | 'PHP 'Perl # # | 'SQL 'C++ # # | 'HTML 'Sh/Csh# # http://www.mswanson.com | 'Javascript # #/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/#

« previous php.general (#8835) next »