Re: Live Search

On 9-Dec-09, at 5:37 AM, Hannes Magnusson wrote:

Considering the "original" does over 7000 stat()s per request I can't believe a sqlite search on a small database can possibly be that slow. Are you 100% sure these numbers are accurate?

It certainly wouldn't hurt to have someone else test it out. The "original" only does the stat's on the first request. Subsequent requests hit a cache. The cache is just a 300k serialized array of function/class names so it's pretty quick to read through. We could combine the two approaches. Use the database to generate the function cache. This might be a bit cleaner than scanning the directories, but the result should be the same.

We could offload some of the searching to the client itself, populating things like function/method index into json objects and cache it on the client, similar to what we do on http://people.php.net That should drastically speed up the basic search. We could look into how much data we can push to the client gzipped..

I toyed with this idea a bit. The function cache becomes 300k of (uncompressed) JSON. My initial thinking was that this would take longer to send to the client than it would for us to process server side.

Sounds good to me, but we still haven't restructured the website.. Searching may be the most fun thing to work on, but we still need all the other pieces :)

I know. Restructuring is a bit daunting. I'm not sure I can tackle that at this time. Search is nice because I can pick away at it when I get a few hours here and there. Ideally we'd have other people working on some of these other pieces at the same time. Stew