#43896 [Com]: htmlspecialchars returns empty string on invalid unicode sequence

From: Date: Sun, 17 Feb 2008 13:25:23 +0000
Subject: #43896 [Com]: htmlspecialchars returns empty string on invalid unicode sequence
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-122361@lists.php.net to get a copy of this message 
 ID:               43896
 Comment by:       andreas dot ravnestad at gmail dot com
 Reported By:      arnaud dot lb at gmail dot com
 Status:           Open
 Bug Type:         Strings related
 Operating System: Any
 PHP Version:      5.2.5
 New Comment:

This seems to be breaking PEAR::Text_Wiki completely when using UTF-8:
http://pear.php.net/bugs/bug.php?id=13136


Previous Comments:
------------------------------------------------------------------------

[2008-01-24 20:51:11] tallyce at gmail dot com

See also bugs 43294 and 43549 which seem to be the same thing.

This is really starting to bite now. Please can this be fixed, or
suggest how we can reliably process incoming user data in UTF8 given
this behaviour change!

------------------------------------------------------------------------

[2008-01-24 12:29:58] arnaud dot lb at gmail dot com

I made a patch for this bug:

http://s3.amazonaws.com/arnaud.lb/php_htmlentities_utf.patch

The internal get_next_char() function returns a status of FAILURE 
when it encounters a invalid or incomplete sequence, which causes 
the htmlspecialchars and htmlentities functions to return an empty 
string.

This patch modify the behavior of these functions to skip invalid 
sequences, without discarding the whole string. This involves a very 
few changes and makes the behavior of theses functions more 
consistent with previous PHP versions.

It also adds a few tests to htmlentities-utf.phpt.

------------------------------------------------------------------------

[2008-01-20 02:12:01] arnaud dot lb at gmail dot com

Description:
------------
htmlspecialchars/htmlentities returns an empty string when the input 
contains an invalid unicode sequence.

I think these functions should just skip the invalid sequences or 
encode them byte by byte (e.g. 0xE9 => é), instead of 
discarding the whole string.

Sometimes you have to display arbitrary strings of unknow encoding. 
So you make them more safe using htmlspecialchars($string, 
ENT_COMPAT, "site_encoding, utf-8 in my case"), but if there is at 
least one invalid sequence in the string, it returns an empty 
string :/

Reproduce code:
---------------
$string = "Voil\xE0"; // "Voilà", in ISO-8859-15

var_dump(htmlspecialchars($string, ENT_COMPAT, "utf-8"));


Expected result:
----------------
string(4) "Voil"

OR 

string(10) "Voilà"

Actual result:
--------------
string(0) ""


------------------------------------------------------------------------


-- 
Edit this bug report at http://bugs.php.net/?id=43896&edit=1

Thread (24 messages)

« previous php.bugs (#122361) next »