Bug #54379 [Com]: PDO_OCI: UTF-8 output gets truncated

Edit report at https://bugs.php.net/bug.php?id=54379&edit=1 ID: 54379 Comment by: qing0088 at hotmail dot com Reported by: sms at inbox dot ru Summary: PDO_OCI: UTF-8 output gets truncated Status: Open Type: Bug Package: PDO related PHP Version: 5.3.6 Block user comment: N Private report: N New Comment: http://www.officialbuffalobillsshop.com/trent-edwards-jersey-c-13_14.html Trent Edwards Jersey http://www.wholesaleusajerseys.us/soccer-club-c-286.html Soccer Club Jerseys http://www.wholesaleusajerseys.us/ Wholesale Jerseys Previous Comments: ------------------------------------------------------------------------ [2011-08-06 08:17:20] mitans02 at gmail dot com I have same problem with UTF8 database and UTF8 client. PDO_OCI should set string buffer length correct to handling UTF8. Check also oracle OCI documents below: http://download.oracle.com/docs/cd/B10500_01/server.920/a96529/ch6.htm#1004620 - Database settings SQL> 1 SELECT PARAMETER, VALUE 2 FROM NLS_DATABASE_PARAMETERS 3* WHERE PARAMETER IN ('NLS_CHARACTERSET', 'NLS_NCHAR_CHARACTERSET') PARAMETER VALUE ------------------------------ ------------------------------ NLS_CHARACTERSET AL32UTF8 NLS_NCHAR_CHARACTERSET AL16UTF16 - Table for test CREATE TABLE mytable (col1 NVARCHAR2(20)); - Test data INSERT INTO mytable VALUES('12345678901234567890'); /* 20 signle byte char */ INSERT INTO mytable VALUES('あいうえおかきくけこさしすせそたちつてと'); /* 20 double byte char, Japanese */ - Test Program <?php print "NLS_LANG=".getenv('NLS_LANG')."\n\n"; $db = new PDO("oci:dbname=//instance1.cf9klgqzy0gu.ap-northeast-1.rds.amazonaws.com:3306/mydb;charset=AL32UTF8", "user", "pass"); $db->setAttribute(PDO::ATTR_ERRMODE,PDO::ERRMODE_EXCEPTION); $stmt = $db->prepare("SELECT * FROM mytable"); $stmt->execute(); var_dump($stmt->fetchAll(PDO::FETCH_ASSOC)); ?> - Test Program Output # php ocitest.php NLS_LANG=Japanese_Japan.AL32UTF8 Warning: PDOStatement::fetchAll(): column 0 data was too large for buffer and was truncated to fit it in /root/ocitest.php on line 9 array(2) { [0]=> array(1) { ["COL1"]=> string(20) "12345678901234567890" } [1]=> array(1) { ["COL1"]=> string(40) "あいうえおかきくけこさしす } } - Client Environment # uname -a Linux localhost.localdomain 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:35 EDT 2010 i686 i686 i386 GNU/Linux # rpm -qa | grep oracle oracle-instantclient11.2-devel-11.2.0.2.0-1 oracle-instantclient11.2-sqlplus-11.2.0.2.0-1 oracle-instantclient11.2-basic-11.2.0.2.0-1 # php -v PHP 5.3.6 (cli) (built: Aug 5 2011 09:15:02) Copyright (c) 1997-2011 The PHP Group Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies - Suggest fix Patch to oci_statement.c # diff -u oci_statement.c oci_statement.c.new --- oci_statement.c 2011-08-06 01:07:53.000000000 -0700 +++ oci_statement.c.new 2011-08-06 01:08:03.000000000 -0700 @@ -529,7 +529,7 @@ (param, OCI_DTYPE_PARAM, &colname, &namelen, OCI_ATTR_NAME, S->err)); col->precision = scale; - col->maxlen = data_size; + col->maxlen = ( data_size + 1 ) * sizeof(utext); col->namelen = namelen; col->name = estrndup((char *)colname, namelen); ------------------------------------------------------------------------ [2011-03-25 04:39:19] sms at inbox dot ru Description: ------------ Data is stored in ANSI charset (CL8MSWIN1251) while connection uses UTF-8. PDOStatement::fetchAll() generates warning and fields containing non-english characters gets truncated. For example, PDO outputs only 53 UTF-8 russian characters for VARCHAR2(100) field. MySQL's PDOStatement::fetchAll() works fine in the same situation. Test script: --------------- $pdo = new PDO('oci:dbname=[host];charset=UTF8', '[user]', '[pass]'); $cmd = $pdo->query('SELECT * FROM user'); var_dump($cmd->fetchAll()); Expected result: ---------------- Table field(s) not truncated, no warnings Actual result: -------------- Table field(s) gets truncated, PHP warning: PDOStatement::fetchAll() [<a href='pdostatement.fetchall'>pdostatement.fetchall</a>]: column 0 data was too large for buffer and was truncated to fit it ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=54379&edit=1