#15630 [Fbk]: imap_utf7_decode appears to be broken

ID: 15630 Updated by: kalowsky@php.net Reported By: robert.marchand@umontreal.ca Status: Feedback Bug Type: IMAP related Operating System: SGI Irix 6.5 PHP Version: 4.2.2 New Comment: Since I have no way to test this, can anyone else confirm or deny that this patch works? I'd rather not commit blindly. Previous Comments: ------------------------------------------------------------------------ [2002-08-08 15:05:47] gamid@netilla.com Robert, The following patch fixes both imap_utf7_encode() and imap_utf7_decode() to work with UTF-16. PS: this patch is for PHP 4.2.2, the patch for CVS is posted in the php.dev Gamid Isayev --- php_imap.c Wed Aug 7 15:45:53 2002 +++ php_imap.c Thu Aug 8 14:24:16 2002 @@ -2215,14 +2215,14 @@ php_error(E_WARNING, "imap_utf7_decode: Invalid modified UTF-7 character: `%c'", *inp); RETURN_FALSE; } else if (*inp != '&') { - outlen++; + outlen += 2; } else if (inp + 1 == endp) { php_error(E_WARNING, "imap_utf7_decode: Unexpected end of string"); RETURN_FALSE; } else if (inp[1] != '-') { state = ST_DECODE0; } else { - outlen++; + outlen += 2; inp++; } } else if (*inp == '-') { @@ -2272,8 +2272,11 @@ if (*inp == '&' && inp[1] != '-') { state = ST_DECODE0; } - else if ((*outp++ = *inp) == '&') { - inp++; + else { + *outp++ = 0x00; + if ((*outp++ = *inp) == '&') { + inp++; + } } } else if (*inp == '-') { @@ -2349,29 +2352,42 @@ outlen = 0; state = ST_NORMAL; endp = (inp = in) + inlen; - while (inp < endp) { + while (inp < endp || state != ST_NORMAL) { if (state == ST_NORMAL) { - if (SPECIAL(*inp)) { + if (*inp == 0x00 && *(inp+1) < 0x80) { + /* ASCII character */ + outlen++; // for ASCII char + if (*(inp+1) == '&') + outlen++; // for '-' + inp += 2; + } else { + /* begin encoding */ state = ST_ENCODE0; - outlen++; - } else if (*inp++ == '&') { + outlen++; // for '&' + } + } else if (inp == endp || (*inp == 0x00 && *(inp+1) < 0x80)) { + /* flush overflow and terminate region */ + if (state != ST_ENCODE0) { outlen++; } - outlen++; - } else if (!SPECIAL(*inp)) { + outlen++; // for '-' state = ST_NORMAL; } else { - /* ST_ENCODE0 -> ST_ENCODE1 - two chars - * ST_ENCODE1 -> ST_ENCODE2 - one char - * ST_ENCODE2 -> ST_ENCODE0 - one char - */ - if (state == ST_ENCODE2) { - state = ST_ENCODE0; - } - else if (state++ == ST_ENCODE0) { - outlen++; + switch (state) { + case ST_ENCODE0: + outlen++; + state = ST_ENCODE1; + break; + case ST_ENCODE1: + outlen++; + state = ST_ENCODE2; + break; + case ST_ENCODE2: + outlen += 2; + state = ST_ENCODE0; + case ST_NORMAL: + break; } - outlen++; inp++; } } @@ -2388,14 +2404,17 @@ endp = (inp = in) + inlen; while (inp < endp || state != ST_NORMAL) { if (state == ST_NORMAL) { - if (SPECIAL(*inp)) { + if (*inp == 0x00 && *(inp+1) < 0x80) { + /* ASCII character */ + inp++; + if ((*outp++ = *inp++) == '&') + *outp++ = '-'; + } else { /* begin encoding */ *outp++ = '&'; state = ST_ENCODE0; - } else if ((*outp++ = *inp++) == '&') { - *outp++ = '-'; } - } else if (inp == endp || !SPECIAL(*inp)) { + } else if (inp == endp || (*inp == 0x00 && *(inp+1) < 0x80)) { /* flush overflow and terminate region */ if (state != ST_ENCODE0) { *outp++ = B64(*outp); ------------------------------------------------------------------------ [2002-08-07 11:30:22] gamid@isayev.net 1) Patched imap_utf7_decode() returns UTF-16 encoded string. So, to display string properly you should set charset to 'UTF-16' or convert UTF-16 string into preferred charset. 2) Current imap_utf7_encode() "converts 8bit data to modified UTF-7 text". Question is what mean "8bit data"? Right now I'm working on the patch for imap_utf7_encode() to add support for UTF-16 on input. It will allow do right covertion UTF-7 -> imap_utf7_decode() -> imap_utf7_encode() -> UTF-7. ------------------------------------------------------------------------ [2002-08-07 11:11:29] robert.marchand@umontreal.ca Hi, here is what I obtain with the patch applied (php-4.2.2): ---- output from Netscape 4.7: --------- folder (modified UTF-7): test&AN9ZJw- mb_convert_encoding test folder decoded: [testß?] encoded again: [test&AN9ZJw-] decoded again: [testß?] imap_utf7_decode test folder decoded: [ encoded again: [&AA-t&AA-e&AA-s&AA-t&ANE-Y$] decoded again: [ ----------------------------- Maybe there is something wrong with my installation here, but even then, I'm not sure adding zero bytes is the good solution. In my view it will break every application that now use these functions. At least this patch should be used only when --enable-mbstring is used. Thanks. ------------------------------------------------------------------------ [2002-08-07 10:52:28] kalowsky@php.net Once again, please send your updated patch to php-dev. Thank you. ------------------------------------------------------------------------ [2002-08-06 19:02:27] gamid@isayev.net JFYI: cvs -d :pserver:cvsread@cvs.php.net:/repository co php4 cd php4/ext/imap/ cvs update -r 1.112.2.1 php_imap.c patch php_imap.c _file_with_my_patch_ cvs update -A php_imap.c cvs ci php_imap.c As result: --- php_imap.c 5 Aug 2002 21:53:09 -0000 1.134 +++ php_imap.c 6 Aug 2002 23:00:31 -0000 @@ -2077,14 +2077,14 @@ php_error(E_WARNING, "%s(): Invalid modified UTF-7 character: `%c'", get_active_function_name(TSRMLS_C), *inp); RETURN_FALSE; } else if (*inp != '&') { - outlen++; + outlen += 2; } else if (inp + 1 == endp) { php_error(E_WARNING, "%s(): Unexpected end of string", get_active_function_name(TSRMLS_C)); RETURN_FALSE; } else if (inp[1] != '-') { state = ST_DECODE0; } else { - outlen++; + outlen += 2; inp++; } } else if (*inp == '-') { @@ -2134,8 +2134,11 @@ if (*inp == '&' && inp[1] != '-') { state = ST_DECODE0; } - else if ((*outp++ = *inp) == '&') { - inp++; + else { + *outp++ = 0x00; + if ((*outp++ = *inp) == '&') { + inp++; + } } } else if (*inp == '-') { ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/15630 -- Edit this bug report at http://bugs.php.net/?id=15630&edit=1