Edit report at https://bugs.php.net/bug.php?id=72768&edit=1
ID: 72768
Updated by: ab@php.net
Reported by: mlocati at gmail dot com
Summary: Add ENABLE_VIRTUAL_TERMINAL_PROCESSING flag for
php.exe
Status: Analyzed
Type: Feature/Change Request
Package: Output Control
Operating System: Windows 10
PHP Version: Irrelevant
Block user comment: N
Private report: N
New Comment:
@mlocati, nice research so far! It's unlikely a solution with INI will be accepted to control
the terminal support. Probably one more plus to go by streams.
The difference between bash and cmd in this case is, that on older Windows the control sequences
will be output as is. So far i've seen on build 10240. As if a shell is not aware there's
VT at all, it just won't strip the control sequences. On the other hand - outputs by old
scripts might be unintentionally misinterpreted on the new systems. Win7 is going to be in use yet
good 5 years, win8 even longer. Even VT isn't supported there, at least no breakage there
should happen. To make it same as Linux, PHP might care about stripping the controls, probably no
way around.
It's already good that UTF-8 issues are not supposed to be. We need to ensure others like SJIS,
EUC-JP, GB2312, BIG5, etc. are fine. I've seen that some conflicting ranges are present in some
older IBM codepages, but that's most likely not something one needs to care about. Besides the
encoding issues, there might be font issues like mentioned before.
Other questions about what happens when redirecting output, or piping to another program, or reading
from pipe, need to be cleared out and tested as well. Again, one point could be to say - just ignore
the old stuff, or it needs to be handled in PHP. If VT is not enabled explicitly, likely some
automatic will be needed to handle BC and questionable cases.
Having control over this feature in the script space might be handier to write code compatible with
older systems. For example, I/O can be wrapped with a userspace code to disable or enable the
control sequences depending on the term/system version. That could significatly simplify the C
implementation. Otherwise, the C implementation should be smart enough to handle what is needed. But
in any case, we'd need tests to all the points for codepage, I/O, etc. that can be run on
various system configurations. If someone runs for an implementation, I were interested in this as
well and provide reviews and testing. But not right now, as the current focus is on stabilizing the
7.1 features.
Thanks.
Previous Comments:
------------------------------------------------------------------------
[2016-08-11 09:39:15] mlocati at gmail dot com
I asked to Microsoft more info about the state of the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag in
cmd.exe (see https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/15617610--re-enable-enable-virtual-terminal-processing-by
).
They confirmed that the ANSI control codes (that are described here https://msdn.microsoft.com/it-it/library/windows/desktop/mt638032(v=vs.85).aspx
) where enabled by mistake on Windows 10.0.10586, but for the later versions the applications needs
to explicitly enable them (with SetConsoleMode).
About problems with UTF-8: there isn't any problem. There's no character representation
that contains the \x27 (octal \033) byte except for the ESC character used in the ANSI control
codes.
Indeed, all the bytes whose most significant bit (MSB) is 0 (like \x27 == 00100111 in binary
notation) are not part of any multibyte character representation: in utf-8 all the multibyte
characters have bytes whose MSB is 1.
About enabling or not by default value this value, I'd prefer a behavior as near as possible to
what happens on Linux.
On a freshly installed Ubuntu, bash interprets the ANSI control codes.
For instance, if you run
php -r 'echo "\033[101;93m TEST \033[0m\n";'
- on a color-enabled terminal you'll see a colored " TEST "
- on a color-disabledterminal you'll see a not-colored " TEST " but without the
control codes (ie you won't see "?[101;93m TEST ?[0m")
A final note: IMHO it would be great to to control this flag at runtime (with ini_set()?)
------------------------------------------------------------------------
[2016-08-11 03:07:56] kalle@php.net
Anatol, that is a very valid point, and it is indeed perhaps better we encapsulate this
functionality into its own, Windows specific, extension.
Gonna leave this open for now then
------------------------------------------------------------------------
[2016-08-11 00:39:56] ab@php.net
Forget my last comment about seeing it in 14393.51, cmd started from ConEmu which seems to have used
its hooks. I might be err though, but its strange so far. Also verified with build 10240 - looks
like neither a C program nor PHP have vt100 support on pure cmd.exe. But cmd.exe is nice so far on
14393.51, can be check on pure cmd http://conemu.github.io/en/AnsiEscapeCodes.html#ANSI_and_xterm_color_maps
. AnsiColors16.ans seems to be supported good so far.
Kalle, it is a compatibility question. The UTF-8/long path topic doesn't cross with this one.
It just forced the cmd.exe codepage to mimic the default_charset. That's why i ask about some
tests with multibyte codepages and especially with 65001. There can be weird issues with some
codepages like fe #72555.
In the vt100 case, it is something quite new and not absolutely required. BC with older systems
needs to be ensured. At least with those lower win10 build 10240, from what i can say after the
short research today. Also various codepages, as mentioned, and other usage scenarios like file I/O,
pipes, etc. Integrating with streams, fe as a stream wrapper or maybe per
stream_context_set_option() is a possible safer way to go. Scripts utilizing VT100 functionality
will be a new development anyway, so that can be incapsulated good. A direct enablement will need
all the check work as well. But i don't think exposing all of SetConsoleMode is really required
in the core. This might be an idea for some extension ,however.
Thanks.
------------------------------------------------------------------------
[2016-08-10 23:11:40] kalle@php.net
@ab, well afaik we do not have the ability to define console modes on Windows, I don't see why
we could not enable it by default for those who may not have it for whatever reason on Windows 10+.
Unless you plan on exposing more of the Microsoft Console functions to userland along with the
codepage improvements?
------------------------------------------------------------------------
[2016-08-10 22:31:51] ab@php.net
@mlocati, have you checked the approach with 65001 and other multibyte codepages? Some good tests is
something we'll need. I'd suggest to integrate with the streams, and not enable by
default.
Btw i'm on 14393.51 and running your cmd line "echo ""\033[101;93m TEST
\033[0m\n"";" shows me a yellow word TEST on the red background. So it doesn't
look like something is changed in the latest version. Seems it's already enabled by default, or
it's not the vt100.
Thanks.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=72768
--
Edit this bug report at https://bugs.php.net/bug.php?id=72768&edit=1