Req #73716 [Dup]: PHP 7.1's CHCP switching in console should be optional

Edit report at https://bugs.php.net/bug.php?id=73716&edit=1 ID: 73716 Updated by: ab@php.net Reported by: anrdaemon at freemail dot ru Summary: PHP 7.1's CHCP switching in console should be optional Status: Duplicate Type: Feature/Change Request Package: Output Control Operating System: Windows PHP Version: 7.1.0 Block user comment: N Private report: N New Comment: Mistake, dup of #72555. Thanks. Previous Comments: ------------------------------------------------------------------------ [2016-12-18 00:18:42] ab@php.net Marking as duplicate, as it is basically same fix as bug #73594 which came first. Thanks. ------------------------------------------------------------------------ [2016-12-15 20:45:42] ab@php.net Sounds good then. Thanks for the test, @anrdaemon. The console codepage is now handled separately, but is still same as internal if input_encoding and output_encoding are empty - default case. I'd still like to know more about the crashes you mentioned, as that should be fixed in first place, for more stability at least. If you're an active Windows user, please test the release candidates and give feedback, that's essential. With Far - i was actively using it in the old dark times, when moved away from DOS and Norton Commander. Really didn't touch it for a while, but I know the community is still active and the tool itself is very powerful. There is an accepted RFC, coincidentally written by Yasuo https://wiki.php.net/rfc/default_encoding :) The change for console encoding is IMO in the scope of this RFC. Though, the Windows console itself is likely behave unpredictably if only one of in or out codepage is changed, it's likely to require both. Plus, the mojibake mentioned. The most of other platforms have UTF-8 by default on console, anyway. I still recommend to use one encoding for all the things on console, even not obligatory UTF-8, as that's the most modern and stable variant. For 7.2 - yeah, there's already a portable stream_isatty() in userspace, so redirection is detectable. That also gives the base for the further console improvements on Windows, at least. I will wait for the other ticket yet, and put some docs and news then. Thanks. ------------------------------------------------------------------------ [2016-12-15 01:58:23] yohgaki@php.net > As a fallout of our discussion, I feel the issue needs a deeper revision > and a clearer definition of meaning for each of the four settings. > Ideally, an automatic conversion between encodings, where it is sensible. input_encoding/otput_encoding are for web environment now. mbstring/iconv may convert input encoding except uploaded files. mbstring/iconv may convert output encoding if output's MIME type is text/*. Under web environment, we can safely determine what's text and what's not. Unlike web environment, we cannot assume/determine what are input and output under CLI environment. e.g. pipe. We may add optional conversion feature for CLI, but I don't feel it is mandatory. It should be an option at least. However, if encoding conversion is optional, we may use other tools like "iconv", "nkf" or "lv" commands for pipes. IIRC, someone is trying to detect if STDIN/STDOUT if tty or not. I don't know the status. If STDIN/STDOUT device (and locale) can be detected with reliable manner, we may do something for STDIN/STDOUT. BTW, I didn't know the details about Windows CLI changes until now, but I think new CLI is a lot nicer for multibyte char users. I haven't tried it yet, though. > Where it is more appropriate to discuss these parts of PHP behavior? If you have concrete idea about encoding handling improvement, internals@lists.php.net would be the place. ------------------------------------------------------------------------ [2016-12-14 20:00:32] anrdaemon at freemail dot ru @yohgaki, no, the answer is way simpler. The path to PHP binaries is not only non-unicode, it is rather short and not containing spaces. I just didn't bothered to properly install extensions for the first run. Extensions relied on external DLL's crashed the party and I noticed broken console. @ab, you are not sure about Far Manager because it's a good program? :) Jokes aside, I checked 7.1.r33e96c9 and it no longer corrupts console when changing internal_encoding. There's some related behavior I'm chasing, but present issue is now resolved. As a fallout of our discussion, I feel the issue needs a deeper revision and a clearer definition of meaning for each of the four settings. Ideally, an automatic conversion between encodings, where it is sensible. Where it is more appropriate to discuss these parts of PHP behavior? ------------------------------------------------------------------------ [2016-12-14 18:09:57] ab@php.net On my side - when i switch to "Japanese (Japan)" as system codepage, so it's 932, "MS ゴシック" is the default console font. The empty boxes would indeed appear on cmd, if the system codepage is an incompatible one and the font contains no glyphs. For example, with chcp 437 it even won't let to chcp 932 or others multibyte codepages, and with 65001 the Japanese glyphs are still missing. For that case however, a custom font can be installed for the console, that contains more glyphs. Or, some more advanced console emulator can be used. Probably can be more of effort with Far, as to see, but ConEmu showed me the most of Glyphs I reqired while writing tests on the machine with 437 system codepage. There are many other good term emulators out there, but we still should be cmd.exe oriented. Thanks for checking, Yasuo. @algo13 was also supporting the development by checks and hints, that's where #72555 came from. One or another not critical issue is still present unfortunately, but the main goal is still to improve and extend the UTF-8 support, not moving back into the stone age of ANSI. Thanks. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=73716 -- Edit this bug report at https://bugs.php.net/bug.php?id=73716&edit=1