Edit report at https://bugs.php.net/bug.php?id=80874&edit=1
ID: 80874
User updated by: tony at tonymarston dot net
Reported by: tony at tonymarston dot net
Summary: odbc_fetch_array() will not fetch a row containing a
UTF-8 currency code symbol
Status: Open
Type: Bug
Package: ODBC related
Operating System: Windows 10
PHP Version: 7.4.16
Block user comment: N
Private report: N
New Comment:
I have tried using the PDO ODBC driver both with and without the PDO::ODBC_ATTR_ASSUME_UTF8 option
with mixed results. I have also run tests by accessing the same data in a SQL Server database via
the PDO ODBC driver as a comparison.
I have captured the output from various tests which you can download from http://www.tonymarston.net/test-odbc-utf8.zip
TEST-1 uses the ODBC driver using data which was inserted using JDBC (the Eclipse plugin)
TEST-2 uses the ODBC driver using data which was inserted using PHP and the ODBC driver
TEST-3 uses the PDO-ODBC driver without the UTF8 attribute using data which was inserted using JDBC
TEST-4 uses the PDO-ODBC driver without the UTF8 attribute using data which was inserted using PHP
TEST-5 uses the PDO-ODBC driver with the UTF8 attribute using data which was inserted using JDBC
TEST-6 uses the PDO-ODBC driver with the UTF8 attribute using data which was inserted using PHP
These tests clearly show that the HANA ODBC driver mangles UTF8 values both when written and when
read.
The test using the PDO-ODBC driver with the UTF8 attribute turned ON produced the worst results as
it totally mangled all the non-UTF8 characters.
Previous Comments:
------------------------------------------------------------------------
[2021-03-19 13:19:31] cmb@php.net
> Byte Order Marks should *NEVER* be added to database columns, [â¦]
I suggest you tell that whoever is responsible for inserting the
BOM. It certainly is not PHP. It *might* be the ODBC driver.
Anyhow, if PDO_ODBC with PDO::ODBC_ATTR_ASSUME_UTF8 yields the
same broken results, I don't think this is a PHP issue. It might
be an issue with the driver, or some configuration issue. I
wouldn't know where to look further. Maybe some SAP HANA ODBC
driver support channel can be more helpful.
------------------------------------------------------------------------
[2021-03-18 18:27:10] tony at tonymarston dot net
> 'EFBBBF3F' in hexadecimal is a UTF-8 BOM followed by a question mark.
Byte Order Marks should *NEVER* be added to database columns, only files.
> Did you try with CHAR_AS_UTF8=FALSE?
Yes, but the results were the same.
> would this work with PDO_ODBC?
Accessing the SAP HANA database had the same result - it stopped fetching rows after the 2nd row.
Access the SQL Server database had a different result - the symbol for 'THB' was returned
as '?' on its own without the UTF-8 BOM.
------------------------------------------------------------------------
[2021-03-18 16:45:45] cmb@php.net
> [â¦] and 'EFBBBF3F' in hexadecimal. Why does it show up as '?'
Because that bytes are an UTF-8 BOM followed by a question mark.
Displaying this as ? is correct, but I wonder where that BOM comes
from.
> I have CHAR_AS_UTF8=TRUE in the additional connection properties> in the ODBC Data Source Administrator.
Did you try with CHAR_AS_UTF8=FALSE?
Also, I suggest that you check whether that would work with
PDO_ODBC. I assume that you'd get the same results as with ODBC by
default, but setting the PDO::ODBC_ATTR_ASSUME_UTF8 attribute may
yield the desired results.
------------------------------------------------------------------------
[2021-03-18 15:28:27] tony at tonymarston dot net
No notice or warning is generated on the call to odbc_fetch_array().
I have CHAR_AS_UTF8=TRUE in the additional connection properties in the ODBC Data Source
Administrator.
When I access my SQL Server database via ODBC the currency symbol returned for THB appears as
'?' in the PHP array and also when I display it in the HTML output. However, when I copy
it to my text editor it is shown as '?' in ASCII and 'EFBBBF3F' in
hexadecimal. Why does it show up as '?'
------------------------------------------------------------------------
[2021-03-18 13:03:49] cmb@php.net
>> Doesn't it also raise a warning?> How do I detect that? There is no ODBC function to retrieve> warnings.
I was referring to the general PHP error reporting mechanisms.
Anyway, the ODBC trace shows whats going on:
| General error;-10427 Conversion of parameter/column (3) from
| data type NVARCHAR to ASCII failed (-10427)
Conversion to ASCII can't succeed, but PHP does not enforce that,
but rather requests binding to SQL_C_CHAR. Is there a respective
setting to convert to UTF-8 in the driver options?
With SQL Server (ODBC Driver 17 for SQL Server) all currency
symbols are properly retrieved and displayed for me. Watch out
for font issues (call bin2hex() to see the byte values).
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=80874
--
Edit this bug report at https://bugs.php.net/bug.php?id=80874&edit=1