Bug #77711 [Com]: CURLFile unable to handle UNICODE filenames

Edit report at https://bugs.php.net/bug.php?id=77711&edit=1 ID: 77711 Comment by: anrdaemon at freemail dot ru Reported by: anrdaemon at freemail dot ru Summary: CURLFile unable to handle UNICODE filenames Status: Assigned Type: Bug Package: cURL related Operating System: Windows PHP Version: 7.3.3 Assigned To: ab Block user comment: N Private report: N New Comment: Understood, thanks for looking into the issue. I'll bring it up with cURL, if you don't mind. Previous Comments: ------------------------------------------------------------------------ [2019-03-12 11:04:17] ab@php.net It functionality in the description has been never present in PHP in the claimed form. libcurl still uses ANSI APIs which causes reports like https://github.com/curl/curl/issues/345. A solution needs to be done on the side cURL to - always handle filenames as UTF-8, use Unicode APIs internally and do conversion, or - export additional APIs based on wide chars, or - export handlers for I/O operations, that can be overridden The first option could be the easiest, but it would be a huge BC breach. The second option might be more compatible, but it might lead to duplicating a lot of APIs. The third option would regard to fopen, stat and several other I/O APIs involving filenames. Then any consuming library could make this part of cURL compatible to its needs. Perhaps that would be the cleanest way. cURL could be also compiled with UNICODE, but i'm not sure it would really work. Fe see places like this https://github.com/curl/curl/blob/f762fec323f36fd7da7ad6eddfbbae940ec3229e/src/tool_filetime.c#L40 where ANSI function is used directly so would stay same even with UNICODE. And in any case PHP can't be compiled with UNICODE, so that's not an option. @anrdaemon what merely has changed is, that you don't pass the filename in the ANSI codepage anymore, so the commented out iconv fix brings it to the correct state. Thanks. ------------------------------------------------------------------------ [2019-03-12 00:21:33] anrdaemon at freemail dot ru If you ask me, since PHP does filenames' charset conversion by itself (Win32 API does not support UTF-8 to the best of my knowledge), the solution should likely fall to cURL ext. ------------------------------------------------------------------------ [2019-03-11 23:06:20] kalle@php.net @Anatol, can you confirm my theory? Daniel said on Twitter that he did not have the time to look into this issue, but told us to report it upstream if its an actual cURL issue ------------------------------------------------------------------------ [2019-03-11 22:55:29] kalle@php.net From a quick read, it seems like the issue is caused in cURL itself, as internally we pass the filename supplied directly to curl_formadd(). I see two ways we can fix this: 1) Upstream fix from cURL 2) Implement a file name normalization fix for Windows, however I do not know how this will affect cURL Assigning this to Daniel, perhaps he can share some insights on this. ------------------------------------------------------------------------ [2019-03-08 16:00:31] anrdaemon at freemail dot ru Description: ------------ CURLFile starting from PHP 7.1 is unable to handle local paths containing UNICODE sequences. The workaround is to iconv() the path to single-byte character set used by WinAPI non-widestring family of functions on a given OS installation. But that's limiting the usability of the language to laughable levels. $ php -i | grep -E "chars|encod" default_charset => CP866 => CP866 input_encoding => CP866 => CP866 internal_encoding => UTF-8 => UTF-8 output_encoding => CP866 => CP866 zend.script_encoding => no value => no value exif.encode_jis => no value => no value exif.encode_unicode => ISO-8859-15 => ISO-8859-15 iconv.input_encoding => no value => no value iconv.internal_encoding => no value => no value iconv.output_encoding => no value => no value mailparse.def_charset => UTF-8 => UTF-8 HTTP input encoding translation => disabled mbstring.encoding_translation => Off => Off mbstring.internal_encoding => no value => no value Test script: --------------- <?php // Cyrillic capital letter A // tempnam() returns path encoded in internal_encoding $name = tempnam(__DIR__, "\u{0410}"); //print bin2hex(basename($name)); file_put_contents($name, "Test."); $file = new \CURLFile($name, null, "file"); // $file = new \CURLFile(iconv("UTF-8", "CP1251", $name), null, "file"); // Rather arbitrary workaround $curl = curl_init($argv[1] ?: "http://example.org/post.php"); curl_setopt_array($curl, [ CURLOPT_POST => true, CURLOPT_POSTFIELDS => [ "file" => $file, ], CURLOPT_VERBOSE => true, ]); $rc = curl_exec($curl); if($rc === false) { $err = curl_errno($curl); error_log(curl_strerror($err) . " ($err): " . curl_error($curl)); die; } print "$rc"; Expected result: ---------------- Successful upload. Actual result: -------------- Failed to open/read local data from file/application (26): ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=77711&edit=1