Bug->Doc #75923 [Opn->Ana]: Body is fetched without the last line

Edit report at https://bugs.php.net/bug.php?id=75923&edit=1 ID: 75923 Updated by: cmb@php.net Reported by: jochem dot blok at fasterforward dot nl Summary: Body is fetched without the last line -Status: Open +Status: Analyzed -Type: Bug +Type: Documentation Problem Package: mailparse Operating System: Ubuntu PHP Version: 5.6.33 -Assigned To: +Assigned To: cmb Block user comment: N Private report: N New Comment: Thanks for the fine analysis, @pudge601! I have to add that RFC 5322 does not mandate that a message has to end with a newline (CRLF)[1], so the current behavior is an unfortunate limitation, but I see no reasonable way to fix this. The suggested solution (2) would break BC, since it is currently supported to incrementally parse a message even after some of it's contents have been queried. The suggested solution (1) would only cater to incremental parsing, and can easily be catered to by the userland developer by explicitly parsing a final CRLF if it is missing from the message. So changing to documentation problem. [1] <https://tools.ietf.org/html/rfc5322#section-3.5> Previous Comments: ------------------------------------------------------------------------ [2020-09-14 09:11:44] cmb@php.net Related To: Bug #78820 ------------------------------------------------------------------------ [2020-08-17 15:34:38] thomas at landauer dot at Just adding a shorter script for easier reproduction: ```php $raw = <<<EOD Date: Mon, 17 Aug 2020 18:36:05 +0100 From: <foo@example.com> foobar EOD; $mimemail = mailparse_msg_create(); mailparse_msg_parse($mimemail, $raw); var_dump(mailparse_msg_extract_part($mimemail, $raw)); ``` ------------------------------------------------------------------------ [2018-03-17 15:17:28] pudge601 at hotmail dot com I’ve tried to look into this myself and see if I can come up with a solution. Disclaimer - I’m not a C dev, and have very little knowledge of PHP internals/extensions The crux of the issue is that mailparse will only process a line when it gets to a new line character, so for an email string/file which doesn’t end with a new line, the last line will not be processed. I tried to fix this by making it “flush” when we get to EOF, but this solution would only work when parsing files. The mailparse_msg_parse function (for parsing the mail as a string) can be called multiple times to parse the mail incrementally, and so there is no way for the extension to know whether it has reached the end of the mail string. The only solutions I can think of for this (short of completely re-designing the API for parsing mail messages) would be; 1. Add a mailparse_msg_parse_flush function, which would process the remainder of contents in the buffer as if it is the last line. This would put the onus on the user to make sure they always call this function after calling mailparse_msg_parse 2. Automatically “flush” the buffer the first time any other function which works on a mime mail resource is called by the user; i.e. assume that if the user is trying to query about any aspect of the mime mail (get structure, get part data, get part), then they have finished passing in the raw data and anything left in the buffer is the end of the contents Both of these approaches would probably then need to add restrictions on calling mailparse_msg_parse after flushing the buffer. Given that these solutions aren’t particularly great, perhaps it would be better to just accept the current behaviour and document it. After all, the behaviour of only considering a line to be a line if it ends with a newline character is technically correct (as far as POSIX is concerned). The only trouble with this is that it isn’t possible to ever have a non-multipart mail whose body contents do not end in a newline, and for it to be parsed correctly by the mailparse extension (except if the contents are base64 encoded, in which case the raw newline is ignored). ------------------------------------------------------------------------ [2018-02-06 11:37:05] jochem dot blok at fasterforward dot nl Description: ------------ With a not multipart mail the body is fetched without the last line. Appending a new line to the end "solves" the problem. Test script: --------------- <?php error_reporting(E_ALL); ini_set('display_errors', 1); header('Content-type: text/plain'); $tekst = <<<EOD From: someone@example.com To: someone_else@example.com Subject: An RFC 822 formatted message This is the plain text body of the message. Note the blank line between the header information and the body of the message. EOD; $stream = fopen('php://memory', 'r+'); fwrite($stream, $tekst); fseek($stream, 0); $resource = mailparse_msg_create(); mailparse_msg_parse($resource, fread($stream, 10000)); echo mailparse_msg_extract_part($resource, $stream); Expected result: ---------------- This is the plain text body of the message. Note the blank line between the header information and the body of the message. Actual result: -------------- This is the plain text body of the message. Note the blank line 1 ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=75923&edit=1