Re: Re: Opening large file problem - fopen

Do you have long lines? If not try this: $file = "error_log.txt"; if (($fp = fopen($file,'r+') !== 0) { // Increment the 1024 sufficiently to read enough data, fgets automatically stops at \n while ($line = fgets($fp,1024)) { if ($line != "") echo($line."<br>"); } } Alternatively, do you know what the last char or the first char in each line is? (i.e. an end-of-record or start-of-record identifier) Try this: $file = "errors.txt"; $file_size = filesize($file); $byte_count = 0; $fp = fopen($file,"r+"); $last_char = ""; while ($byte_count < $file_size) { $chr = ""; $eol = ""; $line = $last_char; while ($eol == 0) { //Use this for EOR checking, replace the $EOR with your EOL/EOR char) if (($chr = fread($fp,1)) === 0 || $chr == $EOR) $eol = 1; else $line .= $chr; /** Use this for SOR checking if (($chr = fread($fp,1)) === 0 || $chr == $SOR) $eol = 1; else $line .= $chr; $last_char = $chr **/ $byte_count = $byte_count + 1; } if (trim($line) != "") { //Do Something echo($line."<br>"); } } this will need to be modified if you have a multi-byte EOR/SOR, but hopefully this will give you a good base. Matt On Tue, 2003-12-16 at 08:46, Kim Steinhaug wrote: > Attached is a little textfile on 4 lines, it will choke afte 1 line > due to the "\n\n" problem. > > I use this loop to parse it : > > $file = "error_log.txt"; > if($fp=fopen($file, "r+")) > { > while($line=trim(fgets($fp))) > { > echo $line . "<br>"; > } > > } > > -- > Kim Steinhaug > --------------------------------------------------------------- > There are 10 types of people when it comes to binary numbers: > those who understand them, and those who don't. > --------------------------------------------------------------- > > > "Kim Steinhaug" <kim@steinhaug.com> wrote in message > news:20031216132132.44272.qmail@pb1.pair.com... > > Hello, > > > > Thanks for you reply. The first approach works, I succesfully > > imported 1.903.541 entries into my mySQL database, phew! > > Problem was that as mentioned I had to "massage" the data > > beforehand, which with TextPad didnt take to much time but > > again I love my scripts to be complete and be "standalone". > > > > Back to the 2) option, I tried another approach, just to see > > how far the script woul accually go. There are ~2,1 million lines > > in the error_log so I would like to see a result for the $i in this > > range. But this script also chokes. > > > > $file = "error_log"; > > $handle = fopen ($file, "rb"); > > $contents = ""; > > do { > > $data = fread($handle, 8192); > > if (strlen($data) == 0) { > > break; > > } > > $contents = $data; > > $temp = explode("\n",$contents); > > $i = $i + count($temp); > > if($i%1000==0) // Spit out every 1000 lines count > > echo $i . "<br>"; > > } while(true); > > fclose ($handle); > > echo $count; > > exit; > > > > Do you have an example, or link to a site discussing the issue, of > > buffering the data? > > > > -- > > Kim Steinhaug > > --------------------------------------------------------------- > > There are 10 types of people when it comes to binary numbers: > > those who understand them, and those who don't. > > --------------------------------------------------------------- > > > > > > "Eugene Lee" <list-php-1@fsck.net> wrote in message > > news:20031216130100.GC1537@Dark-Age.local... > > > On Tue, Dec 16, 2003 at 01:30:14PM +0100, Kim Steinhaug wrote: > > > : > > > : I found out that what the script accually does is choke on "\n\n", > > > : empty lines. I havnt found a way to solve this with the script, > > > : > > > : What I do now is use TextPad and just replace all occurencies > > > : of "\n\n" with "\n-\n" or something and it works all nice. But > > > this > > > : is a bad way of doing it, since the script still doesnt accually > work... > > > > > > You have two options: > > > > > > 1) massage your data beforehand so that your script works properly > > > > > > 2) buffer your input lines so that you process them only after you have > > > read a complete "entry", since one entry may span multiple lines in your > > > error_log.

Attachment: [application/pgp-signature] This is a digitally signed message part signature.asc