looking for a PHP texte indexer
| From: | Mihamina Rakotomandimby | Date: | Mon, 11 Jun 2012 09:12:41 +0000 |
| Subject: | looking for a PHP texte indexer | ||
| Groups: | php.general | ||
| Request: | Send a blank email to php-general+get-318211@lists.php.net to get a copy of this message | ||
Hi all,
I have a small job ad website, where some poster tend to flood with the same ad, just in order to be on top of the recent sort.
To perturb the strict duplication detection (yes it's weak), they add one or two words that makes difference.
The result is a duplication of many ads.
I would like to search for duplicates by looking for ads with 80%-90% same words and decide they're the same, so that I can group them.
Of course, putting a limiting mecanism or even a moderation is scheduled, but I want to process existing first.
I dont want to use MySQL for indexing, I believe text indexers are best tools for this: Am I wrong?
What would you suggest me to process and lookup for duplicates in that situation?
--
RMA.