Re: huge file to insert into postgres db

On Tuesday 22 April 2003 15:31, Matt MacLeod wrote: > I have a large CSV file containing UK postcodes with their n/e > co-ordinates which I need to put into a postgresql database. Its become > very hard to handle as my machine cannot handle the whole CSV file so > I've been chopping it into more manageable pieces and creating PHP > insert scripts with the 'chopped bits'. > > Unfortunately, my PC can only handle about 10000 records at a time and > there are 2 million+. I don't have time to do 200 scripts, so does > anybody have any further ideas. I don't understand why you have to do 200 scripts. Just write a simple script to read in your large CSV file and write out every 10K records into a smaller file, name your files sequentially. Then write a script which reads in, one at a time, the sequentially named smaller files and insert into pgsql. -- Jason Wong -> Gremlins Associates -> www.gremlins.biz Open Source Software Systems Integrators * Web Design & Hosting * Internet & Intranet Applications Development * ------------------------------------------ Search the list archives before you post http://marc.theaimsgroup.com/?l=php-general ------------------------------------------ /* omnibiblious, adj.: Indifferent to type of drink. Ex: "Oh, you can get me anything. I'm omnibiblious." */