Re: some ideas for open discussion
| From: | Rasmus Lerdorf | Date: | Fri, 23 May 2003 19:47:42 +0000 |
| Subject: | Re: some ideas for open discussion | ||
| References: | 1 | Groups: | php.mirrors |
| Request: | Send a blank email to php-mirrors+get-17975@lists.php.net to get a copy of this message | ||
On Fri, 23 May 2003, Gabor Hojtsy wrote:
> Oh, so blocking bots from indexing shortcuts. That is fine with me. In
> case you can point to some good resource on how to identify bots ;)
Uh, a simple robots.txt rule should do the trick. But I am not all that
convinced we should block them. Why not let it find our nice short urls,
especially if we reduce it to a single request as you suggested.
-Rasmus