Bug #72992 [Com]: mbstring.internal_encoding doesn't inherit default_charset

Edit report at https://bugs.php.net/bug.php?id=72992&edit=1 ID: 72992 Comment by: gerthaubrich at web dot de Reported by: zoeslam at gmail dot com Summary: mbstring.internal_encoding doesn't inherit default_charset Status: Re-Opened Type: Bug Package: mbstring related Operating System: Ubuntu 16.04 PHP Version: 7.0.10 Assigned To: yohgaki Block user comment: N Private report: N New Comment: Tested the behavior for self-compiled PHP 7.0.23, 7.1.9 and 7.2.0RC2 and it's the same problem (on Debian 8 x64). Default or set values on default_charset and internal_encoding are not handed down to mbstring.internal_encoding. It remains on its default value 'UTF-8' unless it is changed directly. On the other hand, changes are correctly reflected by iconv encodings. Example: <?php header('Content-Type: text/plain'); header('Cache-Control: no-cache, no-store, must-revalidate'); error_reporting(E_ALL); echo "mbstring internal encoding inheritance test: PHP " . PHP_VERSION . date(' , Y-m-d H:i:s') . "\n"; echo "\nBEFORE\n"; echo "default_charset : '" . ini_get('default_charset') . "'\n"; echo "input_encoding : '" . ini_get('input_encoding') . "'\n"; echo "internal_encoding : '" . ini_get('internal_encoding') . "'\n"; echo "output_encoding : '" . ini_get('output_encoding') . "'\n"; echo "mb_internal_encoding(): '" . mb_internal_encoding() . "'\n"; echo "iconv_get_encoding() :\n" . var_export(iconv_get_encoding('all'), TRUE) . "\n"; sleep(1); ini_set('default_charset', 'Windows-1252'); ini_set('output_encoding', 'ISO-8859-15'); echo "\nAFTER\n"; echo "default_charset : '" . ini_get('default_charset') . "'\n"; echo "input_encoding : '" . ini_get('input_encoding') . "'\n"; echo "internal_encoding : '" . ini_get('internal_encoding') . "'\n"; echo "output_encoding : '" . ini_get('output_encoding') . "'\n"; echo "mb_internal_encoding(): '" . mb_internal_encoding() . "'\n"; echo "iconv_get_encoding() :\n" . var_export(iconv_get_encoding('all'), TRUE) . "\n"; ?> All version have been compiled with option --enable-mbstring, whereas iconv is enabled by default. For this test, default_charset = "GB18030" has been set in php.ini and mbstring.internal_encoding and all three iconv encoding options remains empty/unset in php.ini: mbstring internal encoding inheritance test: PHP 7.2.0RC2 , 2017-09-27 08:47:18 BEFORE default_charset : 'GB18030' input_encoding : '' internal_encoding : '' output_encoding : '' mb_internal_encoding(): 'UTF-8' iconv_get_encoding() : array ( 'input_encoding' => 'GB18030', 'output_encoding' => 'GB18030', 'internal_encoding' => 'GB18030', ) AFTER default_charset : 'Windows-1252' input_encoding : '' internal_encoding : '' output_encoding : 'ISO-8859-15' mb_internal_encoding(): 'UTF-8' iconv_get_encoding() : array ( 'input_encoding' => 'Windows-1252', 'output_encoding' => 'ISO-8859-15', 'internal_encoding' => 'Windows-1252', ) It is the same result for all three PHP versions. Additionally setting internal_encoding directly in php.ini does not implicitly change mbstring.internal_encoding. Previous Comments: ------------------------------------------------------------------------ [2017-09-08 17:18:02] gerthaubrich at web dot de Hi, seems NOT to be fixed for 5.6.31, 7.0.22 and 7.1.8 from my point of view (self-compiled on Debian 8 x64). I haven't found this bug in the first place and tried to submit a new one, but luckly there is a fail safe. Here are my details: Hello, I am using Debian 8 x64 Apache/FPM and have compiled 5.6.31, 7.0.22 and 7.1.8 successfully (using php.net TAR-packages). Reviewing multi-byte functions and config options, I noticed that the value of mbstring.internal_encoding does not change in either version, if default_charset is set via ini_set() in the script or even if it is changed directly in php.ini. Also setting internal_encoding in php.ini does not change the mbstring value. The official documentation is quiet limited and the notes in the example php.ini do not seem to match the observed behaviour: ; Use of this INI entry is DEPRECATED, use global internal_encoding instead. ; internal/script encoding. ; Some encoding cannot work as internal encoding, e.g. SJIS, BIG5, ISO-2022-* ; If empty, default_charset or internal_encoding or iconv.internal_encoding is ; used. The precedence is: ; default_charset < internal_encoding < iconv.internal_encoding ;;mbstring.internal_encoding = First, "iconv.internal_encoding" seems to be a copy&paste error, I assume. I guess it should be "mbstring.internal_encoding", right? Second, the value I can read with mb_internal_encoding() or the behaviour I see, when I for example use implicit conversion via zend.multibyte=On or an explicit conversion via mb_convert_encoding() without using a source encoding as a third parameter suggests, that the value does not change unless it is directly set in php.ini or via ini_set(). Setting default_charset or internal_encoding in php.ini does NOT implicitly change mbstring's internal encoding value in the PHP versions, mentioned above (for example tested with ISO-8859-1 and ISO-8859-15). Have I missed something in the docs or is this an unintended behaviour? It is hard to believe, that this might be some system-specific issue for my self-compiled versions, but it is also strange that all three major versions behave in the same way. (Maybe I have missed something.) In contrast, iconv behaves like the comments in the php.ini suggest. With default_charset = A, all three iconv encodings implicitly change to A, if I additionally set internal_encoding = B, also iconv.internal_encoding changes to B, whereas the remaining two still use A. As far as I can see it right now, mbstring's internal encoding can currently only be changed directly. It currently uses UTF-8 for all my PHP versions. What is the source for this default value? Is it the implicit DEFAULT value of default_charset or is it the system locale, which also uses UTF-8 on my system? Nevertheless, the options usage seems to be inconsistent. -------- I would expect, that changing/setting default_charset and/or internal_encoding in php.ini would also affect mbstring's internal default encoding (as the comments suggest and as it does for iconv). Additionally it's a shame that the ini-comments provide more "insights" then the official documentation. Note 1: It would be nice, if phpinfo() would display inherited values instead of "no value" in the table (for e.g. internal_encoding, mbstring.internal_encoding, etc.). Seeing effective values would be helpful. For example "<UTF-8>" to mark, that the value has not been explicitly set in php.ini, but instead has been derived from another option in the inheritance chain. (The HTML-output also could use CSS for special formatting.) Note 2: Option default_mimetype does take its additonal charset information ("Content-Type: text/html; charset=utf-8") only from option default_charset. With option output_encoding being available, it would be more straightforward to use this one as the primary source, which implicitly defaults to default_charset (if it is not explicitly set). Best regards, Gert ------------------------------------------------------------------------ [2016-10-17 10:08:31] bwoebi@php.net Automatic comment on behalf of yohgaki Revision: http://git.php.net/?p=php-src.git;a=commit;h=8bbd0952e5bba88426bac1596dcc3bfa504dbe4e Log: Fix Bug #72992 mbstring.internal_encoding doesn't inherit default_charset ------------------------------------------------------------------------ [2016-09-12 07:38:51] yohgaki@php.net I noticed mb_internal_encoding() should be modified to reflect encoding actually used... Thank you. Anyway, current implementation does not change lower precedence INI values. Used encoding is determined like (this is iconv because iconv code is simpler) static char *get_internal_encoding(void) { if (ICONVG(internal_encoding) && ICONVG(internal_encoding)[0]) { return ICONVG(internal_encoding); } else if (PG(internal_encoding) && PG(internal_encoding)[0]) { return PG(internal_encoding); } else if (SG(default_charset)) { return SG(default_charset); } return ""; } As you can see, INI setting precedence is kept as defined in the RFC. Mbstring's encoding handling is not simple as iconv, if I'm missing something please let me know. ------------------------------------------------------------------------ [2016-09-12 06:41:57] zoeslam at gmail dot com I am sorry I still don't get what you say, but this is not important, I rely on your analysis. If: 1) "mbstring.internal_encoding" ini setting is deprecated 2) "default_charset" doesn't propagate to mb_internal_encoding() 3) and there is no fix for that At least the documentation should be updated to report that the only way to change mb_internal_encoding() is to call mb_internal_encoding() itself. May we update the manual please? ------------------------------------------------------------------------ [2016-09-12 01:53:01] yohgaki@php.net Mbstring's INI value is not significant when higher precedence INI is set when mbstring INI is not set. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=72992 -- Edit this bug report at https://bugs.php.net/bug.php?id=72992&edit=1