cvs: php-news-web / robots.txt
| From: | Derick Rethans | Date: | Fri, 26 Nov 2004 12:16:26 +0000 |
| Subject: | cvs: php-news-web / robots.txt | ||
| Groups: | php.mirrors | ||
| Request: | Send a blank email to php-mirrors+get-26665@lists.php.net to get a copy of this message | ||
derick Fri Nov 26 07:16:26 2004 EDT
Added files:
/php-news-web robots.txt
Log:
- Added a robots.txt file so that we stop some crawlers from bringing down
the NNTP server.
http://cvs.php.net/co.php/php-news-web/robots.txt?r=1.1&p=1
Index: php-news-web/robots.txt
+++ php-news-web/robots.txt
User-agent: *
Disallow: /