Edit report at https://bugs.php.net/bug.php?id=72768&edit=1
ID: 72768
Updated by: ab@php.net
Reported by: mlocati at gmail dot com
Summary: Add ENABLE_VIRTUAL_TERMINAL_PROCESSING flag for
php.exe
Status: Analyzed
Type: Feature/Change Request
Package: Output Control
Operating System: Windows 10
PHP Version: Irrelevant
Block user comment: N
Private report: N
New Comment:
@mlocati, amazing work, thanks! Your code can already be used as base framework to support the PHP
integration.
The issue with a new ini is, that there are already quite a few. That's the reason it's
preferable to not to introduce a new one, especially if it's solvable another way. Regarding
streams, my thought was like piece of code below:
$fd = fopen("php://stdout", "r");
stream_context_set_option($fd, "stdio", "vt100_enabled", true);
fwrite($fd, "\033[101;93m Yellow text on red background \033[0m\n");
fclose($fd);
Same with php/stdin, etc. Any such stream is an isolated version of the original system descriptor,
it won't affect all the output. OFC in fact, it might have to be a singleton, as it's only
one I/O device at the end. But, as a stream is buffered, the decision whether to out/read the
control sequences can be deferred.It could be cool ofc, to be able doing the same with the std I/O
constants, fe like
stream_context_set_option(STDOUT, "stdio", "vt100_enabled", true);
but i'm not sure it can work exactly that way, but need to double check. Still, it could be an
additional user land stream function to deliver the terminal info or switch to the required mode.
Maybe a function were even more universal in that case, like stream_vt100_supported(STDOUT), etc.
As it looks like after thinking a bit more, stripping the control sequences might be relatively
easy. Should research more yet, but the sequences i've seen follow a particular pattern.
Especially in the case of the direct I/O, one could just scroll over them. Even with the buffered
stream, that might be not a big overhead. So probably shouldn't completely abandon that option.
Another nice thing in streams could be a stream filter, which could be used for this task.
The point with the mb encodings - I'd not write it off yet. We'll have to test and do some
adjustments under circumstances. Fe for xterm, there are several dedicated versions for Asian
encodings, the question is just why.
One point with your latest test code - to check were whether it'd fail for a non standard
terminal. There are nice alternative terminals like ConEmu, which already provide the functionality
even on lower Windows versions. I'm not sure yet, how to solve this without checking any
possible APIs. This would probably prevent the automation in some case, but is probably an overhead
ATM and can be checked at some later stage. It's ATM only about the standard cmd.exe, anyway.
It is already a very good start. If you've mood and time, you could already begin the PHP
patch. Maybe you've a better idea how to integrate it with PHP, please let me know. Writing the
tests for the prospective usage, then matching with internals to correct/improve the actual
implementation might be next step.
Thanks.
Previous Comments:
------------------------------------------------------------------------
[2016-08-19 08:27:02] mlocati at gmail dot com
Strictly related to this issue: https://bugs.php.net/bug.php?id=72896
------------------------------------------------------------------------
[2016-08-18 15:10:06] mlocati at gmail dot com
@ab
I fully understand that PHP scripts that output control codes on older systems (or when the
ENABLE_VIRTUAL_TERMINAL_PROCESSING flag is not set for the current process) is really bad.
That's why I suggested to use ini_get/ini_set.
For instance, if the key used to control if the system has color support is 'win_vt100', a
script could do something like this:
<php
ini_set('win_vt100', true); // returns true on success, false on failure
if (ini_get('win_vt100')) {
// print with ANSI control codes
} else {
// print without ANSI control codes
}
?>
About potential problems with the multibyte encodings: I think it's cmd.exe that takes care of
it. BTW, to solve any problem and to be backward compatibile, vt100 could be disabled by default,
and enabled only by the scripts that wants it.
By using the ini_get/ini_set approach, people that wants ANSI codes enabled by default could set its
value to true in their php.ini file.
PS: I created a gist with a sample C code that implements the following functions:
- check if the current console may have has colors
- determine if the current console has colors
- enable/disable color support for the current console
You can find it here: https://gist.github.com/mlocati/21a9233ac83f7d3d7837535bc109b3b7
------------------------------------------------------------------------
[2016-08-11 16:07:28] ab@php.net
@mlocati, nice research so far! It's unlikely a solution with INI will be accepted to control
the terminal support. Probably one more plus to go by streams.
The difference between bash and cmd in this case is, that on older Windows the control sequences
will be output as is. So far i've seen on build 10240. As if a shell is not aware there's
VT at all, it just won't strip the control sequences. On the other hand - outputs by old
scripts might be unintentionally misinterpreted on the new systems. Win7 is going to be in use yet
good 5 years, win8 even longer. Even VT isn't supported there, at least no breakage there
should happen. To make it same as Linux, PHP might care about stripping the controls, probably no
way around.
It's already good that UTF-8 issues are not supposed to be. We need to ensure others like SJIS,
EUC-JP, GB2312, BIG5, etc. are fine. I've seen that some conflicting ranges are present in some
older IBM codepages, but that's most likely not something one needs to care about. Besides the
encoding issues, there might be font issues like mentioned before.
Other questions about what happens when redirecting output, or piping to another program, or reading
from pipe, need to be cleared out and tested as well. Again, one point could be to say - just ignore
the old stuff, or it needs to be handled in PHP. If VT is not enabled explicitly, likely some
automatic will be needed to handle BC and questionable cases.
Having control over this feature in the script space might be handier to write code compatible with
older systems. For example, I/O can be wrapped with a userspace code to disable or enable the
control sequences depending on the term/system version. That could significatly simplify the C
implementation. Otherwise, the C implementation should be smart enough to handle what is needed. But
in any case, we'd need tests to all the points for codepage, I/O, etc. that can be run on
various system configurations. If someone runs for an implementation, I were interested in this as
well and provide reviews and testing. But not right now, as the current focus is on stabilizing the
7.1 features.
Thanks.
------------------------------------------------------------------------
[2016-08-11 09:39:15] mlocati at gmail dot com
I asked to Microsoft more info about the state of the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag in
cmd.exe (see https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/15617610--re-enable-enable-virtual-terminal-processing-by
).
They confirmed that the ANSI control codes (that are described here https://msdn.microsoft.com/it-it/library/windows/desktop/mt638032(v=vs.85).aspx
) where enabled by mistake on Windows 10.0.10586, but for the later versions the applications needs
to explicitly enable them (with SetConsoleMode).
About problems with UTF-8: there isn't any problem. There's no character representation
that contains the \x27 (octal \033) byte except for the ESC character used in the ANSI control
codes.
Indeed, all the bytes whose most significant bit (MSB) is 0 (like \x27 == 00100111 in binary
notation) are not part of any multibyte character representation: in utf-8 all the multibyte
characters have bytes whose MSB is 1.
About enabling or not by default value this value, I'd prefer a behavior as near as possible to
what happens on Linux.
On a freshly installed Ubuntu, bash interprets the ANSI control codes.
For instance, if you run
php -r 'echo "\033[101;93m TEST \033[0m\n";'
- on a color-enabled terminal you'll see a colored " TEST "
- on a color-disabledterminal you'll see a not-colored " TEST " but without the
control codes (ie you won't see "?[101;93m TEST ?[0m")
A final note: IMHO it would be great to to control this flag at runtime (with ini_set()?)
------------------------------------------------------------------------
[2016-08-11 03:07:56] kalle@php.net
Anatol, that is a very valid point, and it is indeed perhaps better we encapsulate this
functionality into its own, Windows specific, extension.
Gonna leave this open for now then
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=72768
--
Edit this bug report at https://bugs.php.net/bug.php?id=72768&edit=1