Bug #75244 [NEW]: fgetcsv reading failure with chinese characters

From: vitor dot luis98 at gmail dot com Operating system: Windows Server 2012 - Chinese PHP version: 7.0.23 Package: Filesystem function related Bug Type: Bug Bug description:fgetcsv reading failure with chinese characters Description: ------------ When using Windows server 2012 version in Chinese language, the function fgetcsv() does not read the line correctly and some $delimiters are lost when splitting the line into Array. Situation: A CSV file with string translations, the string s are delimited by the character ";". This is an example of the File: id;en;cs;ja;de;ar;bg;bs;da;es;et;fr;hr;hu;is;it;ko;lt;lv;no;pl;pt;pt-br;ru;sk;sl;sr;tr;zh-cn;zh-tw FORBIDDEN_ACCESS;Access denied;Přístup zamítnut;アクセスが拒否されました;Zugriff verweigert;الدخول مرفوض;;Pristup nije odobren;Adgang afvist;Acceso denegado;;Accès refusé;Pristup odbijen;Hozzáférés megtagadva;;Accesso negato;액세스 거부됨;;;;Brak dostępu;Acesso negado;;В доступе отказано;Prístup zamietnutý;Dostop zavrnjen;Pristup zabranjen;Erişim Engellendi;访问被拒绝;存取被拒絕 Note: The empty columns are those who doesn't have a translation. When I open this file using fopen only in read mode and use the function fgetcsv to parse line by line, some columns are not being splitted given to me the following output: array(27) { [0]=> string(13) "Access denied" [1]=> string(19) "P艡铆stup zam铆tnut" [2]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [3]=> string(18) "Zugriff verweigert" [4]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [5]=> string(0) "" [6]=> string(20) "Pristup nije odobren" [7]=> string(13) "Adgang afvist" [8]=> string(15) "Acceso denegado" [9]=> string(0) "" [10]=> string(14) "Acc猫s refus茅" [11]=> string(15) "Pristup odbijen" [12]=> string(24) "Hozz谩f茅r茅s megtagadva" [13]=> string(0) "" [14]=> string(14) "Accesso negato" [15]=> string(20) "鞎§劯鞀?瓯半秬霅?" [16]=> string(0) "" [17]=> string(0) "" [18]=> string(13) "Brak dost臋pu" [19]=> string(13) "Acesso negado" [20]=> string(0) "" [21]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [22]=> string(20) "Pr铆stup zamietnut媒" [23]=> string(15) "Dostop zavrnjen" [24]=> string(17) "Pristup zabranjen" [25]=> string(18) "Eri艧im Engellendi" [26]=> string(32) "璁块棶琚嫆缁?瀛樺彇琚嫆绲?" } But this is only happening when the script is being ran on Windows with Chinese language, when is on English version, everything works perfectly. The point is, when I use the function fgets() and then explode(), everything works fine. Test script: --------------- <?php $file = fopen('dictionary.csv', 'r'); /** @var $langList array */ $langList = fgetcsv($file, 0, ";", "\n"); array_shift($langList); while ($line = fgetcsv($file, 0, ";", "\n")) { $id = array_shift($line); echo "Langs = " . count($langList) . ", Line Size: ". count($line) . "\n"; foreach ($langList as $i => $lang) { if (isset($line[$i]) === false) { var_dump($line); throw new Exception(sprintf('Malformed dictionary [%s] for id [lang = %s, index = %s].', $id, $lang, $i)); } } } Expected result: ---------------- array(30) { [0]=> string(16) "FORBIDDEN_ACCESS" [1]=> string(13) "Access denied" [2]=> string(19) "P艡铆stup zam铆tnut" [3]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [4]=> string(18) "Zugriff verweigert" [5]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [6]=> string(0) "" [7]=> string(20) "Pristup nije odobren" [8]=> string(13) "Adgang afvist" [9]=> string(15) "Acceso denegado" [10]=> string(0) "" [11]=> string(14) "Acc猫s refus茅" [12]=> string(15) "Pristup odbijen" [13]=> string(24) "Hozz谩f茅r茅s megtagadva" [14]=> string(0) "" [15]=> string(14) "Accesso negato" [16]=> string(19) "鞎§劯鞀?瓯半秬霅? [17]=> string(0) "" [18]=> string(0) "" [19]=> string(0) "" [20]=> string(13) "Brak dost臋pu" [21]=> string(13) "Acesso negado" [22]=> string(0) "" [23]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [24]=> string(20) "Pr铆stup zamietnut媒" [25]=> string(15) "Dostop zavrnjen" [26]=> string(17) "Pristup zabranjen" [27]=> string(18) "Eri艧im Engellendi" [28]=> string(15) "璁块棶琚嫆缁? [29]=> string(15) "瀛樺彇琚嫆绲? } Actual result: -------------- array(27) { [0]=> string(13) "Access denied" [1]=> string(19) "P艡铆stup zam铆tnut" [2]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [3]=> string(18) "Zugriff verweigert" [4]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [5]=> string(0) "" [6]=> string(20) "Pristup nije odobren" [7]=> string(13) "Adgang afvist" [8]=> string(15) "Acceso denegado" [9]=> string(0) "" [10]=> string(14) "Acc猫s refus茅" [11]=> string(15) "Pristup odbijen" [12]=> string(24) "Hozz谩f茅r茅s megtagadva" [13]=> string(0) "" [14]=> string(14) "Accesso negato" [15]=> string(20) "鞎§劯鞀?瓯半秬霅?" [16]=> string(0) "" [17]=> string(0) "" [18]=> string(13) "Brak dost臋pu" [19]=> string(13) "Acesso negado" [20]=> string(0) "" [21]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [22]=> string(20) "Pr铆stup zamietnut媒" [23]=> string(15) "Dostop zavrnjen" [24]=> string(17) "Pristup zabranjen" [25]=> string(18) "Eri艧im Engellendi" [26]=> string(32) "璁块棶琚嫆缁?瀛樺彇琚嫆绲?" } -- Edit bug report at https://bugs.php.net/bug.php?id=75244&edit=1 -- Try a snapshot (PHP 5.4): https://bugs.php.net/fix.php?id=75244&r=trysnapshot54 Try a snapshot (PHP 5.5): https://bugs.php.net/fix.php?id=75244&r=trysnapshot55 Try a snapshot (trunk): https://bugs.php.net/fix.php?id=75244&r=trysnapshottrunk Fixed in SVN: https://bugs.php.net/fix.php?id=75244&r=fixed Fixed in release: https://bugs.php.net/fix.php?id=75244&r=alreadyfixed Need backtrace: https://bugs.php.net/fix.php?id=75244&r=needtrace Need Reproduce Script: https://bugs.php.net/fix.php?id=75244&r=needscript Try newer version: https://bugs.php.net/fix.php?id=75244&r=oldversion Not developer issue: https://bugs.php.net/fix.php?id=75244&r=support Expected behavior: https://bugs.php.net/fix.php?id=75244&r=notwrong Not enough info: https://bugs.php.net/fix.php?id=75244&r=notenoughinfo Submitted twice: https://bugs.php.net/fix.php?id=75244&r=submittedtwice register_globals: https://bugs.php.net/fix.php?id=75244&r=globals PHP 4 support discontinued: https://bugs.php.net/fix.php?id=75244&r=php4 Daylight Savings: https://bugs.php.net/fix.php?id=75244&r=dst IIS Stability: https://bugs.php.net/fix.php?id=75244&r=isapi Install GNU Sed: https://bugs.php.net/fix.php?id=75244&r=gnused Floating point limitations: https://bugs.php.net/fix.php?id=75244&r=float No Zend Extensions: https://bugs.php.net/fix.php?id=75244&r=nozend MySQL Configuration Error: https://bugs.php.net/fix.php?id=75244&r=mysqlcfg