slightly OT, need help with wget, was RE:php web spider

From: Date: Mon, 31 Jul 2000 23:24:59 +0000
Subject: slightly OT, need help with wget, was RE:php web spider
Groups: php.general 
Request: Send a blank email to php-general+get-9307@lists.php.net to get a copy of this message
A while ago I posted a question regarding web spiders written in PHP. I then decided on using wget to do the page retrieval to save some time. It has worked great except for one annoying case that I can't seem to fix. The problem comes when I try to pull pages that contain a tilde '~' charachter in the URL, a directory will be created on the local filesystem with a %7E instead of a ~. EG if I'm trying to download http://www.foo.com/~someguy, my directory tree will look like /some/local/dir/www.foo.com/%7Esomeguy. I've poured over the docs for wget (http://www.gnu.org/manual/wget/ ) with no aparent command line option to fix this. It is rather annoying since the links on the pages that are downloaded are set to point to /some/local/dir/www.foo.com/~someguy but that dir does not exist! I tried writing a hack to examine the file system after the wget call but when I try to do a readdir on a directory containing a %7E instead of a ~, readdir shows it as a ~?????? What gives? Any other suggestions? Thanks (and sorry for the off topic WOB) -Marc- #/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/# #Marc Swanson | \|||/ # #MSwanson Consulting | /o o\ # #Phone: (603)868-1721 |-ooo----O----ooo-# #Fax: (603)868-1730 | Solutions in: # #marcswanson@mediaone.net | 'PHP 'Perl # # | 'SQL 'C++ # # | 'HTML 'Sh/Csh# # http://www.mswanson.com | 'Javascript # #/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/#

« previous php.general (#9307) next »