Edit report at https://bugs.php.net/bug.php?id=73716&edit=1
ID: 73716
Updated by: ab@php.net
Reported by: anrdaemon at freemail dot ru
Summary: PHP 7.1's CHCP switching in console should be
optional
-Status: Open
+Status: Duplicate
Type: Feature/Change Request
Package: Output Control
Operating System: Windows
PHP Version: 7.1.0
Block user comment: N
Private report: N
New Comment:
Marking as duplicate, as it is basically same fix as bug #73594 which came first.
Thanks.
Previous Comments:
------------------------------------------------------------------------
[2016-12-15 20:45:42] ab@php.net
Sounds good then. Thanks for the test, @anrdaemon. The console codepage is now handled separately,
but is still same as internal if input_encoding and output_encoding are empty - default case.
I'd still like to know more about the crashes you mentioned, as that should be fixed in first
place, for more stability at least. If you're an active Windows user, please test the release
candidates and give feedback, that's essential.
With Far - i was actively using it in the old dark times, when moved away from DOS and Norton
Commander. Really didn't touch it for a while, but I know the community is still active and the
tool itself is very powerful.
There is an accepted RFC, coincidentally written by Yasuo https://wiki.php.net/rfc/default_encoding :)
The change for console encoding is IMO in the scope of this RFC. Though, the Windows console itself
is likely behave unpredictably if only one of in or out codepage is changed, it's likely to
require both. Plus, the mojibake mentioned. The most of other platforms have UTF-8 by default on
console, anyway. I still recommend to use one encoding for all the things on console, even not
obligatory UTF-8, as that's the most modern and stable variant. For 7.2 - yeah, there's
already a portable stream_isatty() in userspace, so redirection is detectable. That also gives the
base for the further console improvements on Windows, at least.
I will wait for the other ticket yet, and put some docs and news then.
Thanks.
------------------------------------------------------------------------
[2016-12-15 01:58:23] yohgaki@php.net
> As a fallout of our discussion, I feel the issue needs a deeper revision
> and a clearer definition of meaning for each of the four settings.
> Ideally, an automatic conversion between encodings, where it is sensible.
input_encoding/otput_encoding are for web environment now.
mbstring/iconv may convert input encoding except uploaded files.
mbstring/iconv may convert output encoding if output's MIME type is text/*.
Under web environment, we can safely determine what's text and what's not.
Unlike web environment, we cannot assume/determine what are input and output under CLI environment.
e.g. pipe. We may add optional conversion feature for CLI, but I don't feel it is mandatory. It
should be an option at least. However, if encoding conversion is optional, we may use other tools
like "iconv", "nkf" or "lv" commands for pipes.
IIRC, someone is trying to detect if STDIN/STDOUT if tty or not. I don't know the status. If
STDIN/STDOUT device (and locale) can be detected with reliable manner, we may do something for
STDIN/STDOUT.
BTW, I didn't know the details about Windows CLI changes until now, but I think new CLI is a
lot nicer for multibyte char users. I haven't tried it yet, though.
> Where it is more appropriate to discuss these parts of PHP behavior?
If you have concrete idea about encoding handling improvement, internals@lists.php.net would be the
place.
------------------------------------------------------------------------
[2016-12-14 20:00:32] anrdaemon at freemail dot ru
@yohgaki, no, the answer is way simpler.
The path to PHP binaries is not only non-unicode, it is rather short and not containing spaces.
I just didn't bothered to properly install extensions for the first run. Extensions relied on
external DLL's crashed the party and I noticed broken console.
@ab, you are not sure about Far Manager because it's a good program? :)
Jokes aside, I checked 7.1.r33e96c9 and it no longer corrupts console when changing
internal_encoding.
There's some related behavior I'm chasing, but present issue is now resolved.
As a fallout of our discussion, I feel the issue needs a deeper revision and a clearer definition of
meaning for each of the four settings.
Ideally, an automatic conversion between encodings, where it is sensible.
Where it is more appropriate to discuss these parts of PHP behavior?
------------------------------------------------------------------------
[2016-12-14 18:09:57] ab@php.net
On my side - when i switch to "Japanese (Japan)" as system codepage, so it's 932,
"MS ã´ã·ãã¯" is the default console font. The empty boxes would indeed
appear on cmd, if the system codepage is an incompatible one and the font contains no glyphs. For
example, with chcp 437 it even won't let to chcp 932 or others multibyte codepages, and with
65001 the Japanese glyphs are still missing.
For that case however, a custom font can be installed for the console, that contains more glyphs.
Or, some more advanced console emulator can be used. Probably can be more of effort with Far, as to
see, but ConEmu showed me the most of Glyphs I reqired while writing tests on the machine with 437
system codepage. There are many other good term emulators out there, but we still should be cmd.exe
oriented.
Thanks for checking, Yasuo. @algo13 was also supporting the development by checks and hints,
that's where #72555 came from. One or another not critical issue is still present
unfortunately, but the main goal is still to improve and extend the UTF-8 support, not moving back
into the stone age of ANSI.
Thanks.
------------------------------------------------------------------------
[2016-12-14 03:43:29] yohgaki@php.net
@anatol
It is known for Japanese that console (cmd.exe and powershell) font should be "MS
ã´ã·ãã¯" (MS Gothic) to work with UTF-8/UTF-16. So I tried to set it to
"MS ã´ã·ãã¯" on my Windows 10/7, but there is no choice for "MS
ã´ã·ãã¯" only "Consolas"(TrueType), "Lucida
Console"(TrueType) and "ã©ã¹ã¿ã¼ãã©ã³ã"(Raster Font
- Non TrueType). Recent versions of Windows seems only predefined and associated fonts for the
codepage are selectable.
Raster Font doesn't work (got "The system cannot write to the specified device."
error with Japanese file name), TrueType fonts work but no Japanese font(Glyph) and got â¡ (Tofu
- glyph not found) for Japanese characters. However, it is encoded correctly so copy&pasted from
console is readable(correct, not broken text) in UTF aware editor/etc.
Even if there is font issue for console, but it seems PHP 7.1 should work well. (Much better than
older PHP at least for Japanese users)
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=73716
--
Edit this bug report at https://bugs.php.net/bug.php?id=73716&edit=1