note 100477 deleted from function.utf8-decode by danbrown

Note Submitter: vidthekid at live dot com ---- I've extended Blackbit's example below to reject bytes and sequences that are illegal in UTF-8. They are replaced with the replacement character '�' (U+FFFD). <?PHP function charset_decode_utf_8 ($string) { /* note: apply html_entities if desired /before/ applying this function /* Only do the slow convert if there are 8-bit characters */ /* avoid using 0xA0 (\240) in ereg ranges. RH73 does not like that */ if (! ereg("[\200-\237]", $string) and ! ereg("[\241-\377]", $string)) return $string; // reject too-short sequences $string = preg_replace("/[\302-\375]([\000-\177])/", "&#65533;\\1", $string); $string = preg_replace("/[\340-\375].([\000-\177])/", "&#65533;\\1", $string); $string = preg_replace("/[\360-\375]..([\000-\177])/", "&#65533;\\1", $string); $string = preg_replace("/[\370-\375]...([\000-\177])/", "&#65533;\\1", $string); $string = preg_replace("/[\374-\375]....([\000-\177])/", "&#65533;\\1", $string); // reject illegal bytes & sequences $string = preg_replace("/[\300-\301]./", "&#65533;", $string); // 2-byte characters in ASCII range $string = preg_replace("/\364[\220-\277]../", "&#65533;", $string); // 4-byte illegal codepoints (RFC 3629) $string = preg_replace("/[\365-\367].../", "&#65533;", $string); // 4-byte illegal codepoints (RFC 3629) $string = preg_replace("/[\370-\373]..../", "&#65533;", $string); // 5-byte illegal codepoints (RFC 3629) $string = preg_replace("/[\374-\375]...../", "&#65533;", $string); // 6-byte illegal codepoints (RFC 3629) $string = preg_replace("/[\376-\377]/", "&#65533;", $string); // undefined bytes // reject consecutive start-bytes $string = preg_replace("/[\302-\364]{2,}/", "&#65533;", $string); // decode three byte unicode characters $string = preg_replace("/([\340-\357])([\200-\277])([\200-\277])/e", "'&#'.((ord('\\1')-224)*4096 + (ord('\\2')-128)*64 + (ord('\\3')-128)).';'", $string); // decode two byte unicode characters $string = preg_replace("/([\300-\337])([\200-\277])/e", "'&#'.((ord('\\1')-192)*64+(ord('\\2')-128)).';'", $string); // reject leftover continuation bytes $string = preg_replace("/[\200-\277]/", "&#65533;", $string); return $string; } ?> By rejecting illegal sequences, the chances are greatly reduced of accidentally producing special characters that weren't already clearly in the string. Warning: I haven't tested this yet.