robots.txt
| From: | Andrei Zmievski | Date: | Mon, 17 Jul 2006 16:44:19 +0000 |
| Subject: | robots.txt | ||
| Groups: | php.gtk.webmaster | ||
| Request: | Send a blank email to php-gtk-webmaster+get-3264@lists.php.net to get a copy of this message | ||
Should we update our robots.txt file? I see a lot of crawling going on, both from search engine crawlers and other stuff.
-Andrei