Re: nl2.php.net

On Jan 02, 2003, Marc Mes (NedLinux.nl) wrote: > A search on the word "install" gives the following, alot don't you > think, almost 5 times more than php.net. Is there a possibility to get > less results and to show up the results faster like your systems guy > did? Could you ask him for the config otherwise? And why don't php.net > uses the same config as the mirrors anyway? Exclude languages you don't want to support. English and your local language should suffice. This also speeds up search. [...] > To give you an idea, the ht://Dig search database at our place is now > almost 1,5 GB and took up almost 4 hours to create it from scratch at a > very high cpu load of 80% on a AMD Athlon XP 1700+ and 512 MB of DDR RAM > which we have in our server. This is also something to take care of for > the other mirror candidates. A lesson I learned today is to definately > run the rundig script at every night ;-) (We saw the performance of our > own website http://www.nedlinux.nl with alot of PHP & MySQL > usage go > down ;-)) This was one of the first times our server really had to do > something, nice to see ;-) > > sirius:/var/lib/htdig# ls -la > total 1531692 > drwxr-xr-x 2 root root 126976 Jan 1 21:24 . > drwxr-xr-x 17 root root 4096 Dec 17 20:13 .. > -rw-r--r-- 1 root root 440336384 Jan 1 21:41 db.docdb > -rw-r--r-- 1 root root 8054784 Jan 1 21:41 db.docs.index > -rw-r--r-- 1 root root 646919892 Jan 1 21:24 db.wordlist > -rw-r--r-- 1 root root 471449600 Jan 1 21:24 db.words.db > sirius:/var/lib/htdig# du -m > 1496 . Could you give an excerpt from your htdig.conf, especially the parameter max_head_length and max_meta_description_length. As far as I understand the htdig-documentation these parameters sugnificantly influence the size of the database. With default options, and excluding all languages except english and german, I get the following: php3:/opt/www/htdig/db # ll total 263004 drwxrwxrwx 2 cgi root 8192 Jan 5 04:03 . drwxr-xr-x 6 root root 4096 Oct 22 2001 .. -rw-r--r-- 1 cgi users 37169152 Jan 5 04:03 db.docdb -rw-r--r-- 1 cgi users 1513472 Jan 5 04:03 db.docs.index -rw-r--r-- 1 cgi users 129233859 Jan 5 04:03 db.wordlist -rw-r--r-- 1 cgi users 101094400 Jan 5 04:03 db.words.db Compared with php.net the search does also return matches, but is unable to supply excerpts from the documents. I am now playing with the parameters to see which are best. Perhaps we can exchange experience... Cheers, Alex -- Alex Kiesel PGP Key: 0x09F4FA11