Re: catching a robot or spider
| From: | Tom Woody | Date: | Wed, 20 Nov 2002 16:04:58 +0000 |
| Subject: | Re: catching a robot or spider | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-125252@lists.php.net to get a copy of this message | ||
User agent isn't very useful. Most web spiders can be set to
return what ever user agent needed. You are depending on the
writers/users of the spiders to follow commonly acceptable practices ie
looking for robots.txt and the like.
On Tue, 19 Nov 2002 14:54:49 +0100
"Wilbert Enserink" <wilbert@pdd.nl> wrote:
> Hi all,
>
>
> does anybody know a way to make a distinction between robots and
> users? should I use the user agent? Or is this not a safe method.
> If the visitor is a spider/robot I want to include some script
> containing extra URL's for the robot.
>
> regards Wilbert
>
>
> -------------------------
> Pas de Deux
> Van Mierisstraat 25
> 2526 NM Den Haag
> tel 070 4450855
> fax 070 4450852
> http://www.pdd.nl
> info@pdd.nl
> -------------------------
--
Tom Woody
Systems Administrator
NationWide Flood Research, Inc.
phone: 214-631-0400 x209
fax: 214-631-0800
Don't throw your computer out the window,
throw the Windows out of your computer!