Re: BUG: google says about news.php.net's robots.txt
| From: | Hannes Magnusson | Date: | Fri, 07 Sep 2012 13:49:26 +0000 |
| Subject: | Re: BUG: google says about news.php.net's robots.txt | ||
| References: | 1 | Groups: | php.webmaster |
| Request: | Send a blank email to php-webmaster+get-14644@lists.php.net to get a copy of this message | ||
On Fri, Sep 7, 2012 at 10:55 AM, B V Krishna Pernamitta
<pbvamsi@gmail.com> wrote:
> Hi,
>
> On Googling "php news" I can see that Google is displaying A description for
> this result is not available because of this site's robots.txt – learn more.
> as website's description.
Yes, we've had problems with the various search robots hammering the
archives over and over and over again, so couple of years ago we
decided to block most of them, except the vendors we used for our own
searches from the website.
I don't think there is a reason to change this, all our archives are
mirrored on various other servers.
-Hannes