DB-independent fulltext indexing
| From: | Andy Crain | Date: | Thu, 15 Apr 2004 18:33:47 +0000 |
| Subject: | DB-independent fulltext indexing | ||
| Groups: | php.pear.general | ||
| Request: | Send a blank email to pear-general+get-11870@lists.php.net to get a copy of this message | ||
Hi everyone,
My problem in a nutshell: How do I use, or at least roughly emulate,
fulltext indexing in a db-portable way? My project requires near perfect
portability, so I'm already using PEAR::DB and standard ansi sql
extensively, but I need to provide the ability to search large text fields.
I see two possible routes, but each has its drawbacks. I could:
1) Ignore the variety of proprietary implementations and roll my own
solution, something with a keyword table and a keyword_to_content mapping
table that would track occurrences of keywords in text records.
Benefits:
-entirely db-independent
-easy to add soundex, synonym lookups, controlled vocabularies or anything
else
-quick reads/selects
Problems:
-inserts/updates: every unique word in a new text record must be located in
the keyword table or else inserted there (one or two queries); then that
keyword_id and associated content_id must be inserted in the mapping table
(another query), so for a 500-word string of text, that could mean 1001
(best case) or 1501 (worst case) queries!
2) Or, since I'm already using PEAR DB, I could extend it, or each of the
drivers, to somehow modify WHERE clauses to have a particular rdbms's
fulltext syntax (e.g. "WHERE MATCH(cols) AGAINST('searchString')" for mysql,
"WHERE CONTAINS(cols, 'searchString')" for mssql, "WHERE CONTAINS(cols,
'searchString') > 0" for oracle, etc.) This seems really kludgy, though, and
it also means that, for oracle and mssql at least, there's an additional
wrinkle of having to schedule regular index refreshes/updates.
Any ideas? I realize this questions isn't totally PEAR-specific, but I ask
here in case anyone has tried something like this or extended DB like this.
Thanks,
Andy