Bug #74927 [Opn->Nab]: UTF8 characters wrong processed in file system

Edit report at https://bugs.php.net/bug.php?id=74927&edit=1 ID: 74927 Updated by: ab@php.net Reported by: furun at arcor dot de Summary: UTF8 characters wrong processed in file system -Status: Open +Status: Not a bug Type: Bug Package: *Directory/Filesystem functions Operating System: Win7 and Linux PHP Version: 7.1.7 Block user comment: N Private report: N New Comment: This looks same as in bug #74934, especially on Linux the test system has most likely some environment issue. On Windows i got same results as @danack. Besides environment and process locale, the encoding of the script itself matters, too. The glob() behavior seems correct cross platform. The glob() rules are same as in a shell. For example, !3 has special meaning in bash, so it is not interpreted as a filename. Various tools also interpret files with meta chars a different way. If you're required to use such filenames, it makes likely more sense to use other iteration posibilities. Thanks. Previous Comments: ------------------------------------------------------------------------ [2017-07-15 14:15:30] spam2 at rhsoft dot net "Host x86_64-redhat-linux-gnu" - nonsense - that's the output of "Configure Command" - there is also "System" and if the hoster has patched PHP not to show you the environemnt nor tell you something sueful than ask the hoster to solve your PHP problem because nobody can help you in that case from outside if you can't follow (don't matter for hat reasons) http://www.catb.org/esr/faqs/smart-questions.html#beprecise how do you expect anybody to help? here copy&paste from a sane install and you have "System" which shows the kernel which is "4.11.10-200.fc25.x86_64" and the fc25 shows it's a Fedora 25 machine PHP Version 7.1.8-dev System Linux srv-rhsoft.rhsoft.net 4.11.10-200.fc25.x86_64 #1 SMP Wed Jul 12 19:04:52 UTC 2017 x86_64 Build Date Jul 13 2017 17:25:13 Configure Command ' ./configure' ' --host=x86_64-redhat-linux' ' --build=x86_64-redhat-linux' ' --target=x86_64-redhat-linux' ' --prefix=/usr' ' --program-prefix=' ' --libdir=/usr/lib64/php' ' --disable-all' ' --disable-dependency-tracking' ' --enable-bcmath=shared' ' --enable-calendar=shared' ' --enable-cli' ' --enable-ctype=shared' ' --enable-dom=shared' ' --enable-exif=shared' ' --enable-fileinfo=shared' ' --enable-filter' ' --enable-hash=shared' ' --enable-huge-code-pages' ' --enable-inline-optimization' ' --enable-intl=shared' ' --enable-json=shared' ' --enable-libxml' ' --enable-mbregex' ' --enable-mbstring=shared' ' --enable-mysqlnd=shared' ' --enable-opcache=shared' ' --enable-opcache-jit' ' --enable-pcntl=shared' ' --enable-pdo=shared' ' --enable-phar=shared' ' --enable-posix=shared' ' --enable-re2c-cgoto' ' --enable-session=shared' ' --enable-shared' ' --enable-simplexml=shared' ' --enable-soap=shared' ' --enable-sockets=shared' ' --enable-tokenizer=shared' ' --en! able-xml=shared' ' --enable-xmlreader=shared' ' --enable-xmlwriter=shared' ' --enable-zip=shared' ' --with-apxs2=/usr/bin/apxs' ' --with-bz2=shared, /usr' ' --with-config-file-path=/etc' ' --with-config-file-scan-dir=/etc/php.lounge.d' ' --with-curl=shared, /usr' ' --with-freetype-dir=/usr' ' --with-gd=shared, /usr' ' --with-gettext=shared, /usr' ' --with-iconv=shared' ' --with-imap-ssl=/usr' ' --with-imap=shared, /usr' ' --with-kerberos=/usr' ' --with-layout=GNU' ' --with-libdir=lib64' ' --with-libedit=shared, /usr' ' --with-libxml-dir=/usr' ' --with-libzip=/usr' ' --with-mysql-sock=/var/lib/mysql/mysql.sock' ' --with-mysqli=shared, mysqlnd' ' --with-openssl=shared, /usr' ' --with-pcre-jit' ' --with-pcre-regex=/usr' ' --with-pdo-mysql=shared, mysqlnd' ' --with-pic' ' --with-system-ciphers' ' --with-system-tzdata' ' --with-tidy=shared, /usr' ' --with-zlib=shared' ' --with-zlib-dir=/usr' ' --disable-cgi' ' --disable-dmalloc' ' --disable-dtrace' ' --disable-gcov' ' --disable-! gd-jis-conv' ' --disable-ipv6' ' --disable-mysqlnd-compression! -support' ' --disable-opcache-file' ' --disable-phpdbg' ' --disable-rpath' ' --disable-short-tags' ' --disable-static' ' --enable-gcc-global-regs' ' --disable-debug' ' build_alias=x86_64-redhat-linux' ' host_alias=x86_64-redhat-linux' ' target_alias=x86_64-redhat-linux' ------------------------------------------------------------------------ [2017-07-15 14:04:51] furun at arcor dot de the hoster says "CentOS of Red Hat Enterprise", more infos he don't give. and phpinfo shows : "Host x86_64-redhat-linux-gnu" it is a professional webhoster, lets assume they use the newest stable LTS-Version. glob() shows: File Lunix Win7 "Änderung" - - Is not in the list "Änderung.txt" "nderung.txt" "Änderung.txt" ------------------------------------------------------------------------ [2017-07-15 00:47:58] spam2 at rhsoft dot net "x86_64-redhat-linux-gnu" can be anything "cat /etc/redhat-release" and "uname -r" should you give more infos and even phpinfo() shows the kernel which has "el6", "el7" and so on as suffix which shows the RHEL/CentOS version ------------------------------------------------------------------------ [2017-07-14 22:10:10] furun at arcor dot de The linux is, Host: x86_64-redhat-linux-gnu (the sys is rented, i have only very limited control, no connection with Win??) glob() and any other file function must work correctly on any system (>=Win7(?XP)... >=linux?...). that glob() don't return a complete list, and returns incomplete filenames is the bug. i speculate the UTF8 is managed wrongly. Thanks to pretty up the code... :-) ------------------------------------------------------------------------ [2017-07-14 21:14:37] danack@php.net What "linux" system are you testing on? If it's the alleged Linux sub-system on Windows 10, you are very likely to just be seeing Windows issues come through the alleged VM. Testing on Centos with the simpler code below, doesn't show any problems, exception that glob() doesn't return any files starting with an exclamation mark. <?php define('DIR_TEMP', __DIR__ . '/test/'); @mkdir(DIR_TEMP, 0755, true); print('PHP_OS: ' . PHP_OS . '<br><br>'); $filenames = [ "!äÄ", "!Hello_I_start_with_an_exclamation", " ÿ.txt", "Āﻼ.txt", "ä!Ä.txt", "äÄ.txt", iconv('windows-1256', 'utf-8', "äÄ.tx2") ]; foreach ($filenames as $filename) { $written = file_put_contents(DIR_TEMP . $filename, 'foo'); if ($written === false) { echo "Failed to write file $filename \n"; } if (file_exists(DIR_TEMP . $filename) === false) { echo "written but doesn't exist?\n"; } } var_dump($filenames); $fileList = glob(DIR_TEMP . '*.*', 0); var_dump($fileList); PHP_OS: Linux<br><br>array(7) { [0]=> string(5) "!äÄ" [1]=> string(34) "!Hello_I_start_with_an_exclamation" [2]=> string(7) " ÿ.txt" [3]=> string(9) "Āﻼ.txt" [4]=> string(9) "ä!Ä.txt" [5]=> string(8) "äÄ.txt" [6]=> string(13) "أ¤أ„.tx2" } array(5) { [0]=> string(58) "/testing/test/ ÿ.txt" [1]=> string(60) "/testing/test/Āﻼ.txt" [2]=> string(61) "/testing/test/ä!Ä.txt" [3]=> string(60) "/testing/test/äÄ.txt" [4]=> string(67) "/testing/test/أ¤أ„.tx2" } ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=74927 -- Edit this bug report at https://bugs.php.net/bug.php?id=74927&edit=1