Req #47096 [Opn->Csd]: move_uploaded_file not OS encoding aware

From: Date: Tue, 11 Apr 2017 15:59:26 +0000
Subject: Req #47096 [Opn->Csd]: move_uploaded_file not OS encoding aware
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-208482@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=47096&edit=1

 ID:                 47096
 Updated by:         ab@php.net
 Reported by:        nuabaranda at web dot de
 Summary:            move_uploaded_file not OS encoding aware
-Status:             Open
+Status:             Closed
 Type:               Feature/Change Request
 Package:            Filesystem function related
 Operating System:   win32 only - Windows XP
 PHP Version:        5.2.8
-Assigned To:        
+Assigned To:        ab
 Block user comment: N
 Private report:     N

 New Comment:

Fixed in 7.1, see UPGRADING.

Thanks.


Previous Comments:
------------------------------------------------------------------------
[2015-07-24 23:04:13] cmb@php.net

This is not a bug, because strings in PHP are merely an array of
bytes[1].

> Since basename() is locale aware, why not move_uploaded_file()?

To my knowledge, basename() is not locale aware. The difference to
move_uploaded_file() is that basename() doesn't interact with the
file system, but rather works on the given string[2].

> Under Unix and Linux with a properly set locale, PHP program can
> access and retrieve any file name that match the current locale;
> [...]

"Current locale" is the appropriate keyword. Unfortunately,
locales are usually maintained per process, so in a multi-threaded
environment other threads may interfere[3].

[1] <http://php.net/manual/en/language.types.string.php#language.types.string.details>
[2] <http://php.net/manual/en/function.basename.php#refsect1-function.basename-notes>
[3] <http://php.net/manual/en/function.setlocale.php#refsect1-function.setlocale-notes>

------------------------------------------------------------------------
[2015-06-13 21:34:58] vove at o2 dot pl

This problem is really annoying when you upload files received from different countries so they
usually contain non latin characters. I managed to solve it by playing with decoding and encoding:
http://stackoverflow.com/a/28143853/4029967
but I would prefer simpler solution

------------------------------------------------------------------------
[2014-05-21 08:21:52] martin dot keckeis1 at gmail dot com

Really a problem for me here too...I have directories with many files created at different
countries. (China, Russia, Europe...)

I also cannot rename them, since they are managed there (i just want to view them)

Resources i found:
http://stackoverflow.com/questions/2947941/how-to-iterate-over-non-english-file-names-in-php
http://stackoverflow.com/questions/2887909/working-with-japanese-filenames-in-php-5-3-and-windows-vista/2888039#2888039

But nothing solving my problem.

------------------------------------------------------------------------
[2012-08-23 12:32:31] nicolas dot grekas+php at gmail dot com

Well, if you really need it, there may be one possibility using a COM object:

$fs = new \COM('Scripting.FileSystemObject', null, CP_UTF8);

------------------------------------------------------------------------
[2012-04-03 15:12:07] salsi at icosaedro dot it

Just to complete my little survey of the file names encoding issue:

1. Under Windows Vista, in the control panel "Regional and Language Settings" also the
"Formats" panel must be set accordingly to the language selected in the
"Advanced" panel in order to set the LC_CTYPE property; the "Advanced" panel
only selects the translation mapping between Unicode and multi-byte encoding but does not set the
locale properties.
For example, on a western country LC_CTYPE="english_United States.1252" while in Japan it
might be LC_CTYPE="Japanese_Japan.1252".

2. Windows applies the "best fit" conversion table
(http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/) when translating from
Unicode file names to multi-byte file name
(http://msdn.microsoft.com/en-us/library/windows/desktop/dd374047%28v=vs.85%29.aspx); characters
that have not a best fit are replaced by a question mark "?".
So, for example, when the japanese locale is set (code page 932) the Latin capital letter A with
dieresis ("Ä") might map to the plain capital letter "A" and accented vouels
like "àèìòù" might be translated to the plain ASCII letters
"aeiou".
This means that from inside PHP file names retrieved from the file system via dir() or getcwd() are
only APPROXYMATIONS of the real path and there is no way to detect if they really match the actual
name.


Conclusions
===========

Under Unix and Linux with a properly set locale, PHP program can access and retrieve any file name
that match the current locale; UTF-8 is the better choice here.

Under Windows, PHP programs can generate and can access any file or file path that contains only
characters included in the current code page table; however, PHP programs cannot trust on file names
retrieved from the file system because these might be arbitrarily mangled and there is no way to
detect such artifact.

------------------------------------------------------------------------


The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at

    https://bugs.php.net/bug.php?id=47096


--
Edit this bug report at https://bugs.php.net/bug.php?id=47096&edit=1


Thread (11 messages)

« previous php.bugs (#208482) next »