Re: Blatantly Evil Question
| From: | Brian Dunning | Date: | Thu, 11 Aug 2005 23:12:49 +0000 |
| Subject: | Re: Blatantly Evil Question | ||
| References: | 1 2 3 4 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-220487@lists.php.net to get a copy of this message | ||
On Aug 11, 2005, at 4:06 PM, Evert | Collab wrote:
First hit on google: http://www.searchengineworld.com/robots/robots_tutorial.htm Search engines check for a robots.txt on your site, in the robots.txt file you can specify that certain or all search engines shouldn't index your siteI know what robots.txt is, I meant how would you use that to cloak the site. Put PHP code in robots.txt to log the IP of any requests to a db, and then use that db to cloak the rest of the site or not?