memory efficient hash table extension? like lchash ...

From: Date: Sat, 23 Jan 2010 21:11:41 +0000
Subject: memory efficient hash table extension? like lchash ...
Groups: php.general 
Request: Send a blank email to php-general+get-301386@lists.php.net to get a copy of this message
All, I'm loading millions of records into a backend PHP cli script that I need to build a hash index from to optimize key lookups for data that I'm importing into a MySQL database. The problem is that storing this data in a PHP array is not very memory efficient and my millions of records are consuming about 4-6 GB of ram. I have tried using some external key/value storage solutions like MemcacheDB, MongoDB, and straight MySQL, but none of these are fast enough for what I'm trying to do. Then I found the "lchash" extension for PHP and it looks like exactly what I want. It's a c-lib hash which is accessed from PHP. Using it would be slightly slower than using straight PHP arrays, but would be much more memory efficient since not all data needs to be stored as PHP zvals, etc. Problem is that the lchash extension can't ben installed in my PHP 5.3 build because "pecl install lchash" fails with a message about invalid checksum on the README file. Apparently this extension has been neglected and abandoned and hasn't been updated since 2005. Is there something like lchash that *is* being maintained? What would you all suggest? -- Dante

« previous php.general (#301386) next »