#20870 [Opn]: htdig search index is bogus

ID: 20870 Updated by: goba@php.net Reported By: philip@cornado.com Status: Open Bug Type: Website problem Operating System: all PHP Version: 4CVS-2002-12-06 (dev) New Comment: OK, so I have added a simple exclud to search.php to exluce such bogus results. It seems that these results can be triggered with .php/SOMETHING, so there is a path after the .php extension. We have no normal pages getting information through PATH_INFO, so this safely can be removed from results. This is not a fix for the original problem though, the htdig index should be fixed after all, which I am unable do... Maybe some systems@ guy will volunteer to make it update itself correctly automatically. Previous Comments: ------------------------------------------------------------------------ [2003-01-02 15:27:56] goba@php.net modifying summary to be more specific ------------------------------------------------------------------------ [2002-12-14 11:28:30] goba@php.net OK, I tried to investigate this "installation.apache2" thing a bit more.. It seems that the error can be triggered by words like "installation" and "configuration", but not by "apache2" or "install" or "configure". It's quite strange. All the results seem OK for the searches except for some innocent words like "installation" and "configuration"... As I have just revised the source of search.php, I must say that there is no code to do anything special for "installation" and "configuration"... It can also be easily proved that it does not depend on relative URLs, as it returns the same bad URL results, regardless of the starting point you initiated your search from (including subdirectories, and other mirror sites, like at.php.net for example)... So there must be some problem in the htdig database... But why for some specific words I don't know... ------------------------------------------------------------------------ [2002-12-13 14:48:29] goba@php.net The search engine part is absolutely up on Stig as far as I see. Also if you look into search.php, you'll see that the results are generated by the local search engine, and not found by PHP itself. That search script may need some development though if the central search site will not be up... ------------------------------------------------------------------------ [2002-12-08 04:03:15] goba@php.net The output of search.php contains *absolute* URIs, so a <base> won't help. The reason we do not add features to the current search page, is that a completely new powerful central search engine is in the planning stage. It will probably be provided by FastSearch (Stig ;). As Stig has the time to set it up for us, it will be used instead of local search engines, and will provide features such as filtering by langauge or location. search.php.net is reserved for this service. Well, it will still be nice to solve the problem, but as I see, it is a htdig internal problem... ------------------------------------------------------------------------ [2002-12-07 13:23:53] philip@php.net Yeah, I chose a bad summary for this, so changed it. Regarding the non htdig part, I guess this is just how relative urls work and am unsure how to fix it without problems. But, for example, going to something like: http://www.php.net/downloads.php/foo http://www.php.net/downloads.php/ Makes relative urls bogus, like a browser looks for: http://www.php.net/downloads.php/anoncvs.php Sure people shouldn't be adding / into the url but it's something to consider (especially now thet search.php creates them). Some thoughts for discussion: * Implement <base href=""> in a sane way to work with mirrors. This would solve the index.php/foo "issue" afaik. * Make search.php smarter. I've been looking at this but am giving up for now :) I have a hard time believing htdig will ever return a non-existent file like '/manual/en/index.php/cs/de/faq.installation.php' and if it does that's just horrible and search.php should band-aid it. * Default to returning results from one language making this configurable for users. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/20870 -- Edit this bug report at http://bugs.php.net/?id=20870&edit=1