Bug #75244 [Opn->Fbk]: fgetcsv reading failure with chinese characters

Edit report at https://bugs.php.net/bug.php?id=75244&edit=1 ID: 75244 Updated by: jhdxr@php.net Reported by: vitor dot luis98 at gmail dot com Summary: fgetcsv reading failure with chinese characters -Status: Open +Status: Feedback Type: Bug Package: Filesystem function related Operating System: Windows Server 2012 - Chinese PHP Version: 7.0.23 Block user comment: N Private report: N New Comment: Not enough information was provided for us to be able to handle this bug. Please re-read the instructions at http://bugs.php.net/how-to-report.php If you can provide more information, feel free to add it to this bug and change the status back to "Open". Thank you for your interest in PHP. What's your locale setting? And what's the encoding of the csv file? Previous Comments: ------------------------------------------------------------------------ [2017-09-22 07:37:20] vitor dot luis98 at gmail dot com Description: ------------ When using Windows server 2012 version in Chinese language, the function fgetcsv() does not read the line correctly and some $delimiters are lost when splitting the line into Array. Situation: A CSV file with string translations, the string s are delimited by the character ";". This is an example of the File: id;en;cs;ja;de;ar;bg;bs;da;es;et;fr;hr;hu;is;it;ko;lt;lv;no;pl;pt;pt-br;ru;sk;sl;sr;tr;zh-cn;zh-tw FORBIDDEN_ACCESS;Access denied;Přístup zamítnut;アクセスが拒否されました;Zugriff verweigert;الدخول مرفوض;;Pristup nije odobren;Adgang afvist;Acceso denegado;;Accès refusé;Pristup odbijen;Hozzáférés megtagadva;;Accesso negato;액세스 거부됨;;;;Brak dostępu;Acesso negado;;В доступе отказано;Prístup zamietnutý;Dostop zavrnjen;Pristup zabranjen;Erişim Engellendi;访问被拒绝;存取被拒絕 Note: The empty columns are those who doesn't have a translation. When I open this file using fopen only in read mode and use the function fgetcsv to parse line by line, some columns are not being splitted given to me the following output: array(27) { [0]=> string(13) "Access denied" [1]=> string(19) "P艡铆stup zam铆tnut" [2]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [3]=> string(18) "Zugriff verweigert" [4]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [5]=> string(0) "" [6]=> string(20) "Pristup nije odobren" [7]=> string(13) "Adgang afvist" [8]=> string(15) "Acceso denegado" [9]=> string(0) "" [10]=> string(14) "Acc猫s refus茅" [11]=> string(15) "Pristup odbijen" [12]=> string(24) "Hozz谩f茅r茅s megtagadva" [13]=> string(0) "" [14]=> string(14) "Accesso negato" [15]=> string(20) "鞎§劯鞀?瓯半秬霅?" [16]=> string(0) "" [17]=> string(0) "" [18]=> string(13) "Brak dost臋pu" [19]=> string(13) "Acesso negado" [20]=> string(0) "" [21]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [22]=> string(20) "Pr铆stup zamietnut媒" [23]=> string(15) "Dostop zavrnjen" [24]=> string(17) "Pristup zabranjen" [25]=> string(18) "Eri艧im Engellendi" [26]=> string(32) "璁块棶琚嫆缁?瀛樺彇琚嫆绲?" } But this is only happening when the script is being ran on Windows with Chinese language, when is on English version, everything works perfectly. The point is, when I use the function fgets() and then explode(), everything works fine. Test script: --------------- <?php $file = fopen('dictionary.csv', 'r'); /** @var $langList array */ $langList = fgetcsv($file, 0, ";", "\n"); array_shift($langList); while ($line = fgetcsv($file, 0, ";", "\n")) { $id = array_shift($line); echo "Langs = " . count($langList) . ", Line Size: ". count($line) . "\n"; foreach ($langList as $i => $lang) { if (isset($line[$i]) === false) { var_dump($line); throw new Exception(sprintf('Malformed dictionary [%s] for id [lang = %s, index = %s].', $id, $lang, $i)); } } } Expected result: ---------------- array(30) { [0]=> string(16) "FORBIDDEN_ACCESS" [1]=> string(13) "Access denied" [2]=> string(19) "P艡铆stup zam铆tnut" [3]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [4]=> string(18) "Zugriff verweigert" [5]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [6]=> string(0) "" [7]=> string(20) "Pristup nije odobren" [8]=> string(13) "Adgang afvist" [9]=> string(15) "Acceso denegado" [10]=> string(0) "" [11]=> string(14) "Acc猫s refus茅" [12]=> string(15) "Pristup odbijen" [13]=> string(24) "Hozz谩f茅r茅s megtagadva" [14]=> string(0) "" [15]=> string(14) "Accesso negato" [16]=> string(19) "鞎§劯鞀?瓯半秬霅? [17]=> string(0) "" [18]=> string(0) "" [19]=> string(0) "" [20]=> string(13) "Brak dost臋pu" [21]=> string(13) "Acesso negado" [22]=> string(0) "" [23]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [24]=> string(20) "Pr铆stup zamietnut媒" [25]=> string(15) "Dostop zavrnjen" [26]=> string(17) "Pristup zabranjen" [27]=> string(18) "Eri艧im Engellendi" [28]=> string(15) "璁块棶琚嫆缁? [29]=> string(15) "瀛樺彇琚嫆绲? } Actual result: -------------- array(27) { [0]=> string(13) "Access denied" [1]=> string(19) "P艡铆stup zam铆tnut" [2]=> string(36) "銈偗銈汇偣銇屾嫆鍚︺仌銈屻伨銇椼仧 [3]=> string(18) "Zugriff verweigert" [4]=> string(23) "丕賱丿禺賵賱 賲乇賮賵囟" [5]=> string(0) "" [6]=> string(20) "Pristup nije odobren" [7]=> string(13) "Adgang afvist" [8]=> string(15) "Acceso denegado" [9]=> string(0) "" [10]=> string(14) "Acc猫s refus茅" [11]=> string(15) "Pristup odbijen" [12]=> string(24) "Hozz谩f茅r茅s megtagadva" [13]=> string(0) "" [14]=> string(14) "Accesso negato" [15]=> string(20) "鞎§劯鞀?瓯半秬霅?" [16]=> string(0) "" [17]=> string(0) "" [18]=> string(13) "Brak dost臋pu" [19]=> string(13) "Acesso negado" [20]=> string(0) "" [21]=> string(34) "袙 写芯褋褌褍锌械 芯褌泻邪蟹邪薪芯" [22]=> string(20) "Pr铆stup zamietnut媒" [23]=> string(15) "Dostop zavrnjen" [24]=> string(17) "Pristup zabranjen" [25]=> string(18) "Eri艧im Engellendi" [26]=> string(32) "璁块棶琚嫆缁?瀛樺彇琚嫆绲?" } ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=75244&edit=1