Re: page suck attack
| From: | Michael McGlothlin | Date: | Wed, 21 May 2008 23:12:48 +0000 |
| Subject: | Re: page suck attack | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-274621@lists.php.net to get a copy of this message | ||
robert wrote:
Attachment: [application/x-pkcs7-signature] S/MIME Cryptographic Signature smime.p7s
Hi Every so often my site is "attacked" in which all URLS on my site are retrieved in the span of minutes. What is this called?? I mean what do I google for? I don't know where to begin. I'm not sure if I am going to implement such at thing but I would like to be able to research it to know my options. thank you robert Each of our websites get a page sucked about every 5 seconds from Google alone. Sometimes seems a little aggressive but I'd rather they be indexing our websites than not. Other spiders are usually more moderate in my experience. I usually make sure my robots.txt blocks downloading css, scripting, images, and files by spiders so at least all they're getting is the HTML content that is actually useful.-- Michael McGlothlin Southwest Plumbing Supply
Attachment: [application/x-pkcs7-signature] S/MIME Cryptographic Signature smime.p7s