Re: Exporting large data from mysql to html using php

Jason Pruim lists@pruimphotography.com On Oct 25, 2011, at 9:58 PM, Jim Giner wrote: > "David Robley" <robleyd@aapt.net.au> wrote in message > news:49.50.34068.1B567AE4@pb1.pair.com... >> >> Consider running EXPLAIN on all your queries to see if there is something >> Mysql thinks could be done to improve performance. >> > > Why do so many responders seem to think the problem here is in the > preparation of the query? > > It is the Approach that it is the problem. It needs to re-addressed, not > simply tinkered with. I don't care if someone figures out how to spew out > 89M+ records in 5 seconds flat. What are you doing pulling up that many > detail records at one time for? Who is going to look at them? You? If so, > see my previous post on how long that is going to take you just to examine > (!) the first million alone. > > This problem needs a re-working of the data processing and user interfaces > to allow for the selection of an appropriate amount of individual records in > any result set, otherwise simply preparing a summary of some sort. It turns out the issue was actually in the pagination... I'm reworking the whole thing and stream lining it... But in the pagination that I found on the internet it used a "SELECT COUNT(*) WHERE state='{$state}'"; and the COUNT was killing the time... Once that was removed, I was displaying records faster then I could imagine... So it's off to pagination land to fix it! And possibly redo the entire thing!