Bug #74927 [Com]: UTF8 characters wrong processed in file system
| From: | spam2 at rhsoft dot net | Date: | Sat, 15 Jul 2017 00:48:02 +0000 |
| Subject: | Bug #74927 [Com]: UTF8 characters wrong processed in file system | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-210064@lists.php.net to get a copy of this message | ||
Edit report at https://bugs.php.net/bug.php?id=74927&edit=1
ID: 74927
Comment by: spam2 at rhsoft dot net
Reported by: furun at arcor dot de
Summary: UTF8 characters wrong processed in file system
Status: Feedback
Type: Bug
Package: *Directory/Filesystem functions
Operating System: Win7 and Linux
PHP Version: 7.1.7
Block user comment: N
Private report: N
New Comment:
"x86_64-redhat-linux-gnu" can be anything
"cat /etc/redhat-release" and "uname -r" should you give more infos and even
phpinfo() shows the kernel which has "el6", "el7" and so on as suffix which
shows the RHEL/CentOS version
Previous Comments:
------------------------------------------------------------------------
[2017-07-14 22:10:10] furun at arcor dot de
The linux is,
Host: x86_64-redhat-linux-gnu
(the sys is rented, i have only very limited control, no connection with Win??)
glob() and any other file function must work correctly on any system (>=Win7(?XP)...
>=linux?...).
that glob() don't return a complete list, and returns incomplete filenames is the bug.
i speculate the UTF8 is managed wrongly.
Thanks to pretty up the code... :-)
------------------------------------------------------------------------
[2017-07-14 21:14:37] danack@php.net
What "linux" system are you testing on?
If it's the alleged Linux sub-system on Windows 10, you are very likely to just be seeing
Windows issues come through the alleged VM.
Testing on Centos with the simpler code below, doesn't show any problems, exception that glob()
doesn't return any files starting with an exclamation mark.
<?php
define('DIR_TEMP', __DIR__ . '/test/');
@mkdir(DIR_TEMP, 0755, true);
print('PHP_OS: ' . PHP_OS . '<br><br>');
$filenames = [
"!äÃ",
"!Hello_I_start_with_an_exclamation",
" ÿ.txt",
"Äﻼ.txt",
"ä!Ã.txt",
"äÃ.txt",
iconv('windows-1256', 'utf-8', "äÃ.tx2")
];
foreach ($filenames as $filename) {
$written = file_put_contents(DIR_TEMP . $filename, 'foo');
if ($written === false) {
echo "Failed to write file $filename \n";
}
if (file_exists(DIR_TEMP . $filename) === false) {
echo "written but doesn't exist?\n";
}
}
var_dump($filenames);
$fileList = glob(DIR_TEMP . '*.*', 0);
var_dump($fileList);
PHP_OS: Linux<br><br>array(7) {
[0]=>
string(5) "!äÃ"
[1]=>
string(34) "!Hello_I_start_with_an_exclamation"
[2]=>
string(7) " ÿ.txt"
[3]=>
string(9) "Äﻼ.txt"
[4]=>
string(9) "ä!Ã.txt"
[5]=>
string(8) "äÃ.txt"
[6]=>
string(13) "أ¤أâ.tx2"
}
array(5) {
[0]=>
string(58) "/testing/test/ ÿ.txt"
[1]=>
string(60) "/testing/test/Äﻼ.txt"
[2]=>
string(61) "/testing/test/ä!Ã.txt"
[3]=>
string(60) "/testing/test/äÃ.txt"
[4]=>
string(67) "/testing/test/أ¤أâ.tx2"
}
------------------------------------------------------------------------
[2017-07-14 19:52:34] furun at arcor dot de
Description:
------------
UTF8 characters wrong processed in file system
PHP 7.1.7, Win7, Xampp
PHP 7.1.5, Linux
PHP has problems to process UTF8 characters in files correctly.
there are maybe multiple buggy file-functions, but i tested: file_put_contents, file_get_contents,
glob.
file_put_contents, file_get_contents writes and reads files correctly,
but glob don't list them correctly.
(is there a .htaccess or PHP or system option to fix this, or it is a PHP-Code problem? i think it
is a PHP bug.)
(PHP7 has still a very weak support of UFT8 encoding? maybe all file functions should be tested?)
Results:
in both OS systems (Windows 7 and Linux), the files are created correctly.
but they are not read back correctly with glob(),
file_put_contents, file_get_contents (OK):
!äÃ
ä!Ã
äÃ!
ä!Ã.txt
äÃ.txt
ÿ.txt
Äﻼ.txt
أ¤أâ.tx2
glob() (BUG):
PHP_OS: WINNT
ÿ.txt 8
ä!Ã.txt 9
äÃ.txt 8
Äﻼ.txt 9
أ¤أâ.tx2 13
PHP_OS: Linux
.txt 4
!Ã.txt 7
.txt 4
.txt 4
.tx2 4
Test script:
---------------
define('DIR_BASE', realpath(dirname(__FILE__) . DIRECTORY_SEPARATOR) .
DIRECTORY_SEPARATOR);
define('DIR_TEMP', DIR_BASE . 'temp' . DIRECTORY_SEPARATOR);
print(DIR_BASE . '<br>');
print(DIR_TEMP . '<br>');
print('<br>');
print('PHP_OS: ' . PHP_OS . '<br><br>');
try {
if (! file_exists(DIR_TEMP) && ! is_dir(DIR_TEMP)) {
$check = mkdir(DIR_TEMP, 0755);
if ($check) $check = chmod(DIR_TEMP, 0755);
}
} catch (Exception $exception) {
}
print('file_put_contents, file_get_contents:<br>');
$fileName = "!äÃ";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "ä!Ã";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ!";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "ä!Ã.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = " ÿ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "Äﻼ.txt";
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$fileName = "äÃ.tx2";
$fileName = iconv('windows-1256', 'utf-8', $fileName);
file_put_contents(DIR_TEMP . $fileName, $fileName);
$data = file_get_contents(DIR_TEMP . $fileName);
print($data . '<br>');
$data = '';
$data .= '<table class="text documentlist sortable">' . "\n";
$data .= '<tbody>' . "\n";
print('<br><br>glob:<br>');
$fileList = glob(DIR_TEMP . '*.*', 0);
foreach ($fileList as $filePath) {
$fileName = basename($filePath);
$data .= '<tr">' .
'<td>' . '<a href="temp/' . htmlentities(urlencode($fileName))
. '">' . $fileName . '</a></td>'.
'<td>' . strlen($fileName) . '</td>'.
'</tr>' . "\n"; //FIXME Einlesen
}
$data .= '</tbody>' . "\n";
$data .= '</table>' . "\n";
print($data);
Expected result:
----------------
all files listed in correct names
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=74927&edit=1