RE: [PHP] huge file to insert into postgres db
| From: | Matt MacLeod | Date: | Tue, 22 Apr 2003 07:47:16 +0000 |
| Subject: | RE: [PHP] huge file to insert into postgres db | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-144785@lists.php.net to get a copy of this message | ||
The CSV came originally from an Access db. I built an ASP page to pull
records into a script that I can copy from the browser into a text
editor and save as a .php file.
I've tried pulling various numbers of records at a time but 10000 is as
high as I can go without things crashing on me.
M
-----Original Message-----
From: Mark Douglas [mailto:Mark.Douglas@Q9.com]
Sent: 22 April 2003 08:40
To: 'Matt MacLeod'
What exactly is preventing you from doing more than 10000 records at a
time?
How are you shopping the file into "manageable bits" at present?
-----Original Message-----
From: Matt MacLeod [mailto:matt@bigbarn.co.uk]
I have a large CSV file containing UK postcodes with their n/e
co-ordinates which I need to put into a postgresql database. Its become
very hard to handle as my machine cannot handle the whole CSV file so
I've been chopping it into more manageable pieces and creating PHP
insert scripts with the 'chopped bits'.
Unfortunately, my PC can only handle about 10000 records at a time and
there are 2 million+. I don't have time to do 200 scripts, so does
anybody have any further ideas.