Bug #74589 [Csd->Fbk]: __DIR__ wrong for unicode character

Edit report at https://bugs.php.net/bug.php?id=74589&edit=1 ID: 74589 Updated by: ab@php.net Reported by: ganlvtech at qq dot com Summary: __DIR__ wrong for unicode character -Status: Closed +Status: Feedback Type: Bug Package: *General Issues Operating System: Windows PHP Version: 7.1.5 Block user comment: N Private report: N New Comment: Finally got the VM, issue confirmed. I've pushed a change in this regard. Any 7.1 or master snapshot starting with ae3f975c5d58f891359a72ad3df84d845e70cdcc is suitable for a test, please fetch one from http://windows.php.net/snapshots/ Thanks. Previous Comments: ------------------------------------------------------------------------ [2017-05-15 14:38:05] ab@php.net Automatic comment on behalf of ab Revision: http://git.php.net/?p=php-src.git;a=commit;h=ae3f975c5d58f891359a72ad3df84d845e70cdcc Log: Fixed bug #74589 __DIR__ wrong for unicode character ------------------------------------------------------------------------ [2017-05-15 13:00:10] ganlvtech at qq dot com in ext/standard/string.c:1647 1629 PHP_FUNCTION(dirname) ... 1646 #ifdef PHP_WIN32 1647 ZSTR_LEN(ret) = php_win32_ioutil_dirname(ZSTR_VAL(ret), str_len); 1648 #else 1649 ZSTR_LEN(ret) = zend_dirname(ZSTR_VAL(ret), str_len); 1650 #endif php_win32_ioutil_dirname is used if PHP_WIN32 defined. but in Zend/zend_compile.c:6505 6501 case T_DIR: 6502 { 6503 zend_string *filename = CG(compiled_filename); 6504 zend_string *dirname = zend_string_init(ZSTR_VAL(filename), ZSTR_LEN(filename), 0); 6505 zend_dirname(ZSTR_VAL(dirname), ZSTR_LEN(dirname)); always zend_dirname I'm not very sure about php-src's code structure. It may be a little difficult for me to produce a patch. ------------------------------------------------------------------------ [2017-05-15 10:46:36] ab@php.net Thanks for this deep investigation. Yeah, zend_dirname is what my debug session leads me to. I think that is the exact point. I still couldn't repro this on a cp 437 system, so I'm getting a VM with cp 936, might take some time. If you're able to debug internals or even produce a patch, i can also evaluate/test that. Basically, there's only 7.1 with UTF-8 support and versions before. To the time of the initial patch, I explicitly left zend_dirname() as is and instead integrated the new API, fe like in the userland dirname(). The only point in the new API is, that it needs the INI to have been initialized before. So might need to check this and reevaluate, if everything is ok, then just replace zend_dirname to use the new API for Windows. Thanks. ------------------------------------------------------------------------ [2017-05-14 20:31:46] ganlvtech at qq dot com Probably reason. When php core get the filename from system, my system returns a string with cp 936. Because php5 doesn't auto convert charset, so the strlen and mb_strlen is both 22 (one chinese character is two bytes). But php7 convert charset automatically, so strlen is 27(1 char for 3 bytes in UTF-8) and mb_strlen is 17. And zend_dirname function use a macro IS_SLASH_P, and the macro call a WIN32API IsDBSCLeadByte. For cp936(GBK), the chinese character's two bytes is both larger than 0x80, IsDBSCLeadByte always return non-zero, even when testing the second byte. In php7, dirname(__FILE__) passed a converted, UTF-8 string to zend_dirname, so it works well. But there might not be a automatically conversion in the zend engine when directily using __DIR__. In php5 conversion will never automatically apply, so the two forms both don't work. Summary: Everything is caused by my system's returning bp936(GBK) encoded path. This may not be a bug of php, but it should be metioned in php docs. Thanks. ------------------------------------------------------------------------ [2017-05-14 19:33:44] ganlvtech at qq dot com <?php echo __FILE__, "\n"; echo strlen(__FILE__), "\n"; echo mb_strlen(__FILE__), "\n"; ?> (php 7.1, cp 936) D:\新建文件夹>php test.php D:\新建文件夹\test.php 27 17 (php 5.4, cp 936) D:\新建文件夹>php54 test.php D:\新建文件夹\test.php 22 22 ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=74589 -- Edit this bug report at https://bugs.php.net/bug.php?id=74589&edit=1