Bug #48147 [Bgs]: iconv with //IGNORE cuts the string

Edit report at https://bugs.php.net/bug.php?id=48147&edit=1 ID: 48147 Updated by: ezyang@php.net Reported by: kulakov74 at yandex dot ru Summary: iconv with //IGNORE cuts the string Status: Bogus Type: Bug Package: ICONV related Operating System: Linux PHP Version: 5.*, 6CVS (2009-05-05) Block user comment: N Private report: N New Comment: Upstream bugs: http://sources.redhat.com/bugzilla/show_bug.cgi?id=13517 http://sources.redhat.com/bugzilla/show_bug.cgi?id=13518 Previous Comments: ------------------------------------------------------------------------ [2011-12-18 19:37:53] ezyang@php.net Not broken in latest version of libiconv ezyang@javelin:~/Desktop/libiconv-1.14/src$ ./iconv_no_i18n --version iconv (GNU libiconv 1.14) Copyright (C) 2000-2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Bruno Haible. ezyang@javelin:~/Desktop/libiconv-1.14/src$ ./iconv_no_i18n -f utf-8 -t iso-8859-1//IGNORE ~/iconv.html | wc -c 15312 ezyang@javelin:~/Desktop/libiconv-1.14/src$ iconv -f utf-8 -t iso-8859-1//IGNORE ~/iconv.html | wc -c iconv: illegal input sequence at position 8168 8157 ------------------------------------------------------------------------ [2009-05-07 13:58:21] jani@php.net We still can't fix bugs in glibc iconv implementation. Try this on command line and you get same results: # iconv -f utf-8 -t iso-8859-1 iconv.html > /dev/null iconv: illegal input sequence at position 3589 # iconv -f utf-8 -t iso-8859-1//IGNORE iconv.html > /dev/null iconv: illegal input sequence at position 8168 ------------------------------------------------------------------------ [2009-05-07 07:50:52] lbarnaud@php.net Marked it as verified as I got exactly the same results: The first iconv() call (the one without //IGNORE) fails on the emphasis character "…" (value="Search…"), which can't be represented in ISO-8859-1. The second iconv() call (the one with //IGNORE) fails later (so the emphasis is ignored, which may means that the //IGNORE flag is supported), and there is no apparent reason for failing at offset 8157 (only regular ASCII chars around). ------------------------------------------------------------------------ [2009-05-06 18:36:10] jani@php.net Arnaud: Please don't reopen bogus bugs without explanation. ------------------------------------------------------------------------ [2009-05-06 18:18:07] kulakov74 at yandex dot ru No. The fact the script displays the notice "iconv(): Detected an illegal character ..." in both cases is not related to the fact whether the option is implemented: this is controlled by error_reporting(E_ALL). The option IGNORE only controls whether iconv will stop at the character or not. Also, the length of the resulting string is different (greater) with IGNORE, and while without it the string ends at exactly where the illegal character is, with IGNORE it ends at a random point where no such characters occur. Also, I did not mention - this is not the only file I converted, many others were converted correctly with the option, and their length only decreased a little. But there were 2 files which were truncated, 1 of them (the smaller) is used for the test case. Can you run the test with the latest PHP releases? Actually this is why I reported the bug. I tried it on other servers with PHP 4.3.3, 5.1.4, 5.1.6, 5.2.4 and 5.2.6 and yep! - I finally found one with 5.2.9 (built Feb 27 2009) and it displayed the same results everywhere. I repeat, the TRANSLIT option works fine, while it does the same and even more. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=48147 -- Edit this bug report at https://bugs.php.net/bug.php?id=48147&edit=1