utf8_decode() and mixed character sets
| From: | James Colannino | Date: | Sun, 11 Oct 2009 03:40:59 +0000 |
| Subject: | utf8_decode() and mixed character sets | ||
| Groups: | php.general | ||
| Request: | Send a blank email to php-general+get-298831@lists.php.net to get a copy of this message | ||
Hey everyone. I'd been troubled for a while by the fact that inserting
cut-pasted special characters such as ä caused truncation when passed to
MySQL, then discovered that it was because I was cutting and pasting unicode
values into non-unicode Latin-1 strings.
Since Latin-1 also has equivalent values, I was hoping that filtering my mixed
unicode/non-unicode string through utf8_decode() would solve the problem, but
instead, where the unicode character used to be, I now get a '?', followed by a
few characters being taken out of the middle. I'm guessing that this is because
utf8_decode() assumes the whole string is unicode and therefore removes a bunch
of extra bytes from the string and corrupts it. At least, that's my guess. I
could be very wrong (I have pretty much no experience with different character
sets...)
My question is, what's a good way to translate unicode characters in a
non-unicode string to their Latin-1 equivalents? I need to be able to do this
in order to sanitize a fairly common form of input.
Thanks!
James