Re: Manuals and Spiders

One can simply have the form posted to google.com with "site:php.net " prefix, and that's all... done. I saw many sites do that... On Tue, 22 Apr 2003, Gabor Hojtsy wrote: > > > looking at the huge percentage of visits that come from search robots > > > [1], I think it would be reasonable to create a robots.txt that prevents > > > those crawlers to index php.net and its mirrors. > > > > > > php.net itself features a full-text search, many mirrors do so, too. PHP > > > does not take any advantage by being searched by crawlers (at least in > > > the manuals). > > > > > > So I'd propose to create a robots.txt that at least prevents indexing > > > the manuals. > > > > > > What do you think? > > > > The quality of the current manual search function is quite bad, and I > > usually just use the url: function of alltheweb.com to search in the > > PHP manual. > > Well, Stig said that setting up search.php.net at alltheweb > would not be a huge task (but that was more than one and > a half a year ago)... He said at the intl. conf last year to > me that it is doable... But nothing happened ;(( I would really > be happy to see an alltheweb powered central search engine > for PHP.net with much more features then the current one... > > Goba > -- Best regards, Shimi ---- "Outlook is a massive flaming horrid blatant security violation, which also happens to be a mail reader." "Sure UNIX is user friendly; it's just picky about who its friends are."