Bug #74927 [Fbk->Opn]: UTF8 characters wrong processed in file system
| From: | furun at arcor dot de | Date: | Sat, 15 Jul 2017 14:04:54 +0000 |
| Subject: | Bug #74927 [Fbk->Opn]: UTF8 characters wrong processed in file system | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-210067@lists.php.net to get a copy of this message | ||
Edit report at https://bugs.php.net/bug.php?id=74927&edit=1
ID: 74927
User updated by: furun at arcor dot de
Reported by: furun at arcor dot de
Summary: UTF8 characters wrong processed in file system
-Status: Feedback
+Status: Open
Type: Bug
Package: *Directory/Filesystem functions
Operating System: Win7 and Linux
PHP Version: 7.1.7
Block user comment: N
Private report: N
New Comment:
the hoster says "CentOS of Red Hat Enterprise", more infos he don't give.
and phpinfo shows : "Host x86_64-redhat-linux-gnu"
it is a professional webhoster, lets assume they use the newest stable LTS-Version.
glob() shows:
File Lunix Win7
"Ãnderung" - - Is not in the list
"Ãnderung.txt" "nderung.txt" "Ãnderung.txt"
Previous Comments:
------------------------------------------------------------------------
[2017-07-15 00:47:58] spam2 at rhsoft dot net
"x86_64-redhat-linux-gnu" can be anything
"cat /etc/redhat-release" and "uname -r" should you give more infos and even
phpinfo() shows the kernel which has "el6", "el7" and so on as suffix which
shows the RHEL/CentOS version
------------------------------------------------------------------------
[2017-07-14 22:10:10] furun at arcor dot de
The linux is,
Host: x86_64-redhat-linux-gnu
(the sys is rented, i have only very limited control, no connection with Win??)
glob() and any other file function must work correctly on any system (>=Win7(?XP)...
>=linux?...).
that glob() don't return a complete list, and returns incomplete filenames is the bug.
i speculate the UTF8 is managed wrongly.
Thanks to pretty up the code... :-)
------------------------------------------------------------------------
[2017-07-14 21:14:37] danack@php.net
What "linux" system are you testing on?
If it's the alleged Linux sub-system on Windows 10, you are very likely to just be seeing
Windows issues come through the alleged VM.
Testing on Centos with the simpler code below, doesn't show any problems, exception that glob()
doesn't return any files starting with an exclamation mark.
<?php
define('DIR_TEMP', __DIR__ . '/test/');
@mkdir(DIR_TEMP, 0755, true);
print('PHP_OS: ' . PHP_OS . '<br><br>');
$filenames = [
"!äÃ",
"!Hello_I_start_with_an_exclamation",
" ÿ.txt",
"Äﻼ.txt",
"ä!Ã.txt",
"äÃ.txt",
iconv('windows-1256', 'utf-8', "äÃ.tx2")
];
foreach ($filenames as $filename) {
$written = file_put_contents(DIR_TEMP . $filename, 'foo');
if ($written === false) {
echo "Failed to write file $filename \n";
}
if (file_exists(DIR_TEMP . $filename) === false) {
echo "written but doesn't exist?\n";
}
}
var_dump($filenames);
$fileList = glob(DIR_TEMP . '*.*', 0);
var_dump($fileList);
PHP_OS: Linux<br><br>array(7) {
[0]=>
string(5) "!äÃ"
[1]=>
string(34) "!Hello_I_start_with_an_exclamation"
[2]=>
string(7) " ÿ.txt"
[3]=>
string(9) "Äﻼ.txt"
[4]=>
string(9) "ä!Ã.txt"
[5]=>
string(8) "äÃ.txt"
[6]=>
string(13) "أ¤أâ.tx2"
}
array(5) {
[0]=>
string(58) "/testing/test/ ÿ.txt"
[1]=>
string(60) "/testing/test/Äﻼ.txt"
[2]=>
string(61) "/testing/test/ä!Ã.txt"
[3]=>
string(60) "/testing/test/äÃ.txt"
[4]=>
string(67) "/testing/test/أ¤أâ.tx2"
}
------------------------------------------------------------------------
[2017-07-14 19:52:34] furun at arcor dot de
Description:
------------
UTF8 characters wrong processed in file system
PHP 7.1.7, Win7, Xampp
PHP 7.1.5, Linux
PHP has problems to process UTF8 characters in files correctly.
there are maybe multiple buggy file-functions, but i tested: file_put_contents, file_get_contents,
glob.
file_put_contents, file_get_contents writes and reads files correctly,
but glob don't list them correctly.
(is there a .htaccess or PHP or system option to fix this, or it is a PHP-Code problem? i think it
is a PHP bug.)
(PHP7 has still a very weak support of UFT8 encoding? maybe all file functions should be tested?)
Results:
in both OS systems (Windows 7 and Linux), the files are created correctly.
but they are not read back correctly with glob(),
file_put_contents, file_get_contents (OK):
!äÃ
ä!Ã
äÃ!
ä!Ã.txt
äÃ.txt
ÿ.txt
Äﻼ.txt
أ¤أâ.tx2
glob() (BUG):
PHP_OS: WINNT
ÿ.txt 8
ä!Ã.txt 9
äÃ.txt 8
Äﻼ.txt 9
أ¤أâ.tx2 13
PHP_OS: Linux
.txt 4
!Ã.txt 7
.txt 4
.txt 4
.tx2 4
Test script:
---------------
define('DIR_BASE', realpath(dirname(__FILE__) . DIRECTORY_SEPARATOR) .
DIRECTORY_SEPARATOR);
define('DIR_TEMP', DIR_BASE . 'temp' . DIRECTORY_SEPARATOR);
print(DIR_BASE . '<br>');
print(DIR_TEMP . '<br>');
print('<br>');
print('PHP_OS: ' . PHP_OS . '<br><br>');
try {
if (! file_exists(DIR_TEMP) && ! is_dir(DIR_TEMP)) {
$check = mkdir(DIR_TEMP, 0755);
if ($check) $check = chmod(DIR_TEMP, 0755);
}
} catch (Exception $exception) {
}
print('file_put_contents, file_get_contents:<br>');
$fileName = "!äÃ";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "ä!Ã";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ!";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "ä!Ã.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = " ÿ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "Äﻼ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ.tx2";
$fileName = iconv('windows-1256', 'utf-8', $fileName);
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$data = '';
$data .= '<table class="text documentlist sortable">' . "\n";
$data .= '<tbody>' . "\n";
print('<br><br>glob:<br>');
$fileList = glob(DIR_TEMP . '*.*', 0);
foreach ($fileList as $filePath) {
$fileName = basename($filePath);
$data .= '<tr">' .
'<td>' . '<a href="temp/' . htmlentities(urlencode($fileName))
. '">' . $fileName . '</a></td>'.
'<td>' . strlen($fileName) . '</td>'.
'</tr>' . "\n"; //FIXME Einlesen
}
$data .= '</tbody>' . "\n";
$data .= '</table>' . "\n";
print($data);
Expected result:
----------------
all files listed in correct names
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=74927&edit=1