Bug #74589 [Com]: __DIR__ wrong for unicode character

Edit report at https://bugs.php.net/bug.php?id=74589&edit=1 ID: 74589 Comment by: ganlvtech at qq dot com Reported by: ganlvtech at qq dot com Summary: __DIR__ wrong for unicode character Status: Feedback Type: Bug Package: *General Issues Operating System: Windows PHP Version: 7.1.5 Block user comment: N Private report: N New Comment: in ext/standard/string.c:1647 1629 PHP_FUNCTION(dirname) ... 1646 #ifdef PHP_WIN32 1647 ZSTR_LEN(ret) = php_win32_ioutil_dirname(ZSTR_VAL(ret), str_len); 1648 #else 1649 ZSTR_LEN(ret) = zend_dirname(ZSTR_VAL(ret), str_len); 1650 #endif php_win32_ioutil_dirname is used if PHP_WIN32 defined. but in Zend/zend_compile.c:6505 6501 case T_DIR: 6502 { 6503 zend_string *filename = CG(compiled_filename); 6504 zend_string *dirname = zend_string_init(ZSTR_VAL(filename), ZSTR_LEN(filename), 0); 6505 zend_dirname(ZSTR_VAL(dirname), ZSTR_LEN(dirname)); always zend_dirname I'm not very sure about php-src's code structure. It may be a little difficult for me to produce a patch. Previous Comments: ------------------------------------------------------------------------ [2017-05-15 10:46:36] ab@php.net Thanks for this deep investigation. Yeah, zend_dirname is what my debug session leads me to. I think that is the exact point. I still couldn't repro this on a cp 437 system, so I'm getting a VM with cp 936, might take some time. If you're able to debug internals or even produce a patch, i can also evaluate/test that. Basically, there's only 7.1 with UTF-8 support and versions before. To the time of the initial patch, I explicitly left zend_dirname() as is and instead integrated the new API, fe like in the userland dirname(). The only point in the new API is, that it needs the INI to have been initialized before. So might need to check this and reevaluate, if everything is ok, then just replace zend_dirname to use the new API for Windows. Thanks. ------------------------------------------------------------------------ [2017-05-14 20:31:46] ganlvtech at qq dot com Probably reason. When php core get the filename from system, my system returns a string with cp 936. Because php5 doesn't auto convert charset, so the strlen and mb_strlen is both 22 (one chinese character is two bytes). But php7 convert charset automatically, so strlen is 27(1 char for 3 bytes in UTF-8) and mb_strlen is 17. And zend_dirname function use a macro IS_SLASH_P, and the macro call a WIN32API IsDBSCLeadByte. For cp936(GBK), the chinese character's two bytes is both larger than 0x80, IsDBSCLeadByte always return non-zero, even when testing the second byte. In php7, dirname(__FILE__) passed a converted, UTF-8 string to zend_dirname, so it works well. But there might not be a automatically conversion in the zend engine when directily using __DIR__. In php5 conversion will never automatically apply, so the two forms both don't work. Summary: Everything is caused by my system's returning bp936(GBK) encoded path. This may not be a bug of php, but it should be metioned in php docs. Thanks. ------------------------------------------------------------------------ [2017-05-14 19:33:44] ganlvtech at qq dot com <?php echo __FILE__, "\n"; echo strlen(__FILE__), "\n"; echo mb_strlen(__FILE__), "\n"; ?> (php 7.1, cp 936) D:\新建文件夹>php test.php D:\新建文件夹\test.php 27 17 (php 5.4, cp 936) D:\新建文件夹>php54 test.php D:\新建文件夹\test.php 22 22 ------------------------------------------------------------------------ [2017-05-14 18:16:11] ganlvtech at qq dot com I tested GBK and UTF-8 as default_charset and php 5.4, 5.6 and 7.1. Test Report: php: 5.4 or 5.6 default_charset: GBK or UTF-8 Results: D:\ D:\ bool(true) ----- php: 7.1 default_charset: GBK or UTF-8 Results: D:\ D:\新建文件夹 bool(false) ===== CJK charchter and even \u00a1 may cause a wrong result. Seems that, only if trailing charchter is an ASCII character, the result can be correct. ===== Hope that the tests above can help you. Thanks. ------------------------------------------------------------------------ [2017-05-14 17:52:27] ganlvtech at qq dot com php.ini is php.ini-development default_charset => UTF-8 => UTF-8 internal_encoding => no value => no value zend.multibyte => Off => Off PHP version: PHP 7.1.5 (cli) (built: May 9 2017 19:48:36) ( NTS MSVC14 (Visual C++ 2015) x64 ) System: Windows 10 Home (64bit). (x64 processor) Default language: zh-CN (There is no other system language supported in my system. My system cannot switch into English mode) Default code page: 936(GBK) I tried chcp 65001 or chcp 437, it makes no changes. ===== I have tried in Interactive shell. It seems working correctly. D:\新建文件夹>php -a Interactive shell php > echo __FILE__; php shell code php > echo __DIR__; D:\新建文件夹 ===== I have also tried php 5.4 or php 5.6 (both are use php.ini-development) "D:\新建文件夹\test.php" shows (different from php7) D:\ D:\ bool(true) So, i tried Script: <?php echo __DIR__, "\n"; echo dirname(__FILE__), "\n"; var_dump(__DIR__ === dirname(__FILE__)); echo __FILE__, "\n"; echo str_replace('\\', '/', __FILE__), "\n"; echo dirname(str_replace('\\', '/', __FILE__)), "\n"; ?> Result: (php 5.4) D:\ D:\ bool(true) D:\新建文件夹\test.php D:/新建文件夹/test.php D:/新建文件夹 ===== Anything works well on Ubuntu Server 16.04 (php7.0 and php5.5 were tested). I think it may be caused by backslash. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=74589 -- Edit this bug report at https://bugs.php.net/bug.php?id=74589&edit=1