Re: Blatantly Evil Question
| From: | Brian Dunning | Date: | Thu, 11 Aug 2005 22:46:37 +0000 |
| Subject: | Re: Blatantly Evil Question | ||
| References: | 1 2 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-220486@lists.php.net to get a copy of this message | ||
On Aug 11, 2005, at 3:44 PM, Evert | Collab wrote:
Use robots.txt 'evil' searchengines will spoof the user-agent string anywayCan you be more specific about what you mean by "use robots.txt"? I just want to cloak for Google, MSN, and Yahoo. I couldn't care less about what any other search engine (evil or not) does or sees.