Re: searchengine in php
| From: | Joe Stump | Date: | Sat, 28 Oct 2000 20:32:34 +0000 |
| Subject: | Re: searchengine in php | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-22489@lists.php.net to get a copy of this message | ||
I don't have code laying around that I'd be willing to share with everyone, but
I will explain a simple concept that most people would enjoy hearing about.
Making a simple search engine with relevancy is actually easier than it sounds.
First let me describe some terms:
1.) "Stop Words" - words that the search engine will ignore (ie "the")
2.) "Relevancy" - an equation used to determine how "relevant" a document is
to a given search.
3.) "Index" - a table that usually holds wordID, documentID, and a relvency code
relating that word and that document.
An industry standard for simple search engines is a "btree index" or "reverse
index". What you do is take your text fiels that you want to index (ie docTitle
and docBody) and run them through and indexing script - mine usually take a
document, create a list of words in that document, strip out all punctuation,
ignore stop words, count up the number of instances of each word in that
document, and then insert the result in my index. Before this you will want to
have created a "dictionary" of words from your documents - meaning a table with
word, wordID and consisting of all the unique words in your documents.
After you have created your index you can then search for "dog" and all you have
to do to get all of those documents with dog in them is look up the wordID for
"dog" and do a join in your index table with that wordID and the documents that
have that word in it - you can order by "hits" or "instances" for simple
relevancy also.
I have done this on DMOZ (www.dmoz.org) with PHP/MySQL and I know it scales
beyond 1.7 Million documents - so it works well.
--Joe
On Sat, Oct 28, 2000 at 12:09:40PM -0700, sagar chand wrote:
> hi there,
>
> I want a search engine to be placed in my website.
> where can i get the required php script for it.
> if any one has already been familiar with this work
> and
> is having the src code. please feel free to mail me
> sagarchand@yahoo.com
>
> thanx in advance
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Messenger - Talk while you surf! It's FREE.
> http://im.yahoo.com/
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, e-mail: php-general-unsubscribe@lists.php.net
> For additional commands, e-mail: php-general-help@lists.php.net
> To contact the list administrators, e-mail: php-list-admin@lists.php.net
================================================================================
= Joe Stump joestump98@yahoo.com http://www.miester.org =
================================================================================
"Real programmers don't comment their code.
It was hard to write, it should be hard to understand."