Re: Very Large text file parsing

Robert Cummings wrote:

On Thu, 2007-09-20 at 13:55 +0200, Paul Scott wrote:

On Thu, 2007-09-20 at 12:50 +0100, Edward Kay wrote:

In addition to Martin's good suggestions (and also assuming you're running php-cli via cron), you could use nice to stop it consuming too many resources:

This is the current approach that I am taking, was just really wondering if there was some kind of voodoo that would speed things up a bit. Thanks both for your responses, appreciate it!

Post some samples of the data you are parsing and a sample of the code you've written to parse them. If you're parsing 750 megs of data then it's quite likely you could squeeze some performance out of the parse routines themselves.

Adding to Robert's comment, try not to copy variables, but to reference them, so you will use less resources. Also free DB results after query. -- 21:50:04 up 2 days, 9:07, 0 users, load average: 0.92, 0.37, 0.18 --------------------------------------------------------- Lic. Martín Marqués | SELECT 'mmarques' || Centro de Telemática | '@' || 'unl.edu.ar'; Universidad Nacional | DBA, Programador, del Litoral | Administrador ---------------------------------------------------------