RE: htdig - Indexing of pages

OK found it... The problem was that I was writing to the NFS drive... I moved the indexing to a local hard disk and now it indexes about 4-5 pages/sec. I left in only EU languages as I don't believe someone would be interested to look for japanase in a Greek mirror... I also updated in httpd.conf HTSEARCH_EXCLUDE. Thanks all for your help.. BTW now during indexing cpu is at 32% idle... Spyros -----Original Message----- From: Alain Bidaud [mailto:ab@crihan.fr] Sent: Friday, February 07, 2003 10:12 AM To: Ioakim Spyros Cc: mirrors@php.net Subject: Re: htdig - Indexing of pages It took me 3 to 4 hours to create the search db with all the languages, and it had created a 1.5GB file. But then it took an very long time to have an answer on the search page I restarted the indexing process with only 5 languages activated, and it took about 30 minutes to finish resulting in 150MB database file Alain On Fri, 7 Feb 2003, Ioakim Spyros wrote: > How long does htdig take to index the pages? > I started it yesterday when I configured it with the help of Goba but it > is still indexing. > I used the default configuration which means all the languages... > So far it has created a 309MB file and still going.... > I am also noticing that it takes about 5-6 seconds to go from one page > to another in the logfile. > Is that normal or could it be that I'm creating the database on a NFS > volume? > I'm posting a couple of lines of the log file also to see the time: > 195.97.19.130 - - [07/Feb/2003:09:34:13 +0200] "GET > /manual/cs/function.ncurses-insertln.php HTTP/1.0" 200 27440 > "http://gr2.php.net/manual/cs/ref.ncurses.php" > "htdig/3.1.6 > (sioakim@ace-hellas.gr)" > 195.97.19.130 - - [07/Feb/2003:09:34:18 +0200] "GET > /manual/cs/function.ncurses-insstr.php HTTP/1.0" 200 27547 > "http://gr2.php.net/manual/cs/ref.ncurses.php" > "htdig/3.1.6 > (sioakim@ace-hellas.gr)" > 195.97.19.130 - - [07/Feb/2003:09:34:25 +0200] "GET > /manual/cs/function.ncurses-instr.php HTTP/1.0" 200 27493 > "http://gr2.php.net/manual/cs/ref.ncurses.php" > "htdig/3.1.6 > (sioakim@ace-hellas.gr)" > > Spyros > > P.S. CPU usage is at 95% idle > -- Alain Bidaud | CRIHAN E-mail : Alain.Bidaud@crihan.fr | 745, Avenue de l'Universite Phone : +33 (0)2 32 91 42 91 | 76800 Saint-Etienne du Rouvray Fax : +33 (0)2 32 91 42 92 | France