Re: encode/decodeXMLEntities

OK guys/gals...comes down to this... YOU CANNOT transmit binary data in XML. Therefore, you must Base64 encode it....so....let's not handle that here... So...this $encodeXMLEntities should handle REMOVAL of bad characters in the string buffer... #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] This is the range...everything else must be removed... This is when I say "CRAP" but....oh, well... So...here is the $encoder so far... $encoder = array(

            chr(0) => '',chr(1) => '',chr(2) => '',chr(3) => '',chr(4) => '',chr(5) => '',chr(6) => '',chr(7) => '',chr(8) => '',
            chr(11) => '',chr(12) => '',chr(14) => '',chr(15) => '',chr(16) => '',chr(17) => '',chr(18) => '',chr(19) => '',
            chr(20) => '',chr(21) => '',chr(22) => '',chr(23) => '',chr(24) => '',chr(25) => '',chr(26) => '',chr(27) => '',chr(28) =

'',chr(29) => '',

            chr(30) => '',chr(31) => '',chr(34) => '"',chr(38) => '&',

            chr(60) => '<',chr(62) => '>',
            chr(127) => '',chr(128) => '€',chr(129) => '',
            chr(130) => '‚',chr(131) => 'ƒ',chr(132) => '„',chr(133) => '…',chr(134) => '†',chr(135) => '&#1
35;',chr(136) => 'ˆ',chr(137) => '‰',chr(138) => 'Š',chr(139) => '‹',
            chr(140) => 'Œ',chr(141) => '',chr(142) => 'Ž',chr(143) => '',chr(144) => '',chr(145) => '&#1
45;',chr(146) => '’',chr(147) => '“',chr(148) => '”',chr(149) => '•',
            chr(150) => '–',chr(151) => '—',chr(152) => '˜',chr(153) => '™',chr(154) => 'š',chr(155) => '&#1
55;',chr(156) => 'œ',chr(157) => '',chr(158) => 'ž',chr(159) => 'Ÿ',
            chr(160) => ' ',chr(161) => '¡',chr(162) => '¢',chr(163) => '£',chr(164) => '¤',chr(165) => '&#1
65;',chr(166) => '¦',chr(167) => '§',chr(168) => '¨',chr(169) => '©',
            chr(170) => 'ª',chr(171) => '«',chr(172) => '¬',chr(173) => '­',chr(174) => '®',chr(175) => '&#1
75;',chr(176) => '°',chr(177) => '±',chr(178) => '²',chr(179) => '³',
            chr(180) => '´',chr(181) => 'µ',chr(182) => '¶',chr(183) => '·',chr(184) => '¸',chr(185) => '&#1
85;',chr(186) => 'º',chr(187) => '»',chr(188) => '¼',chr(189) => '½',
            chr(190) => '¾',chr(191) => '¿',chr(192) => 'À',chr(193) => 'Á',chr(194) => 'Â',chr(195) => '&#1
95;',chr(196) => 'Ä',chr(197) => 'Å',chr(198) => 'Æ',chr(199) => 'Ç',
            chr(200) => 'È',chr(201) => 'É',chr(202) => 'Ê',chr(203) => 'Ë',chr(204) => 'Ì',chr(205) => '&#2
05;',chr(206) => 'Î',chr(207) => 'Ï',chr(208) => 'Ð',chr(209) => 'Ñ',
            chr(210) => 'Ò',chr(211) => 'Ó',chr(212) => 'Ô',chr(213) => 'Õ',chr(214) => 'Ö',chr(215) => '&#2
15;',chr(216) => 'Ø',chr(217) => 'Ù',chr(218) => 'Ú',chr(219) => 'Û',
            chr(220) => 'Ü',chr(221) => 'Ý',chr(222) => 'Þ',chr(223) => 'ß',chr(224) => 'à',chr(225) => '&#2
25;',chr(226) => 'â',chr(227) => 'ã',chr(228) => 'ä',chr(229) => 'å',
            chr(230) => 'æ',chr(231) => 'ç',chr(232) => 'è',chr(233) => 'é',chr(234) => 'ê',chr(235) => '&#2
35;',chr(236) => 'ì',chr(237) => 'í',chr(238) => 'î',chr(239) => 'ï',
            chr(240) => 'ð',chr(241) => 'ñ',chr(242) => 'ò',chr(243) => 'ó',chr(244) => 'ô',chr(245) => '&#2
45;',chr(246) => 'ö',chr(247) => '÷',chr(248) => 'ø',chr(249) => 'ù',
            chr(250) => 'ú',chr(251) => 'û',chr(252) => 'ü',chr(253) => 'ý',chr(254) => 'þ',chr(255) => '&#2

55;'); The lower ASCII characters are removed to satisfy #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] Now comes the hard part...#xC0 to #xFE These characters are invalid for Unicode and are translated to characters such as: € - e282ac � - efbfbd ‚ - e2809a ƒ - c692 „ - e2809e … - e280a6 † - e280a0 ‡ - e280a1 ˆ - cb86 ‰ - e280b0 Š - c5a0 ‹ - e280b9 Œ - c592 � - efbfbd Ž - c5bd � - efbfbd � - efbfbd ‘ - e28098 ’ - e28099 “ - e2809c ” - e2809d • - e280a2 – - e28093 — - e28094 ˜ - cb9c ™ - e284a2 š - c5a1 › - e280ba œ - c593 � - efbfbd ž - c5be Ÿ - c5b8 ¡ - c2a1 ¢ - c2a2 £ - c2a3 ¤ - c2a4 ¥ - c2a5 ¦ - c2a6 § - c2a7 ¨ - c2a8 © - c2a9 ª - c2aa « - c2ab ¬ - c2ac ­ - c2ad ® - c2ae ¯ - c2af ° - c2b0 ± - c2b1 ² - c2b2 ³ - c2b3 ´ - c2b4 µ - c2b5 ¶ - c2b6 · - c2b7 ¸ - c2b8 ¹ - c2b9 º - c2ba » - c2bb ¼ - c2bc ½ - c2bd ¾ - c2be ¿ - c2bf À - c380 Á - c381  - c382 à - c383 Ä - c384 Å - c385 Æ - c386 Ç - c387 È - c388 É - c389 Ê - c38a Ë - c38b Ì - c38c Í - c38d Î - c38e Ï - c38f Ð - c390 Ñ - c391 Ò - c392 Ó - c393 Ô - c394 Õ - c395 Ö - c396 × - c397 Ø - c398 Ù - c399 Ú - c39a Û - c39b Ü - c39c Ý - c39d Þ - c39e ß - c39f à - c3a0 á - c3a1 â - c3a2 ã - c3a3 ä - c3a4 å - c3a5 æ - c3a6 ç - c3a7 è - c3a8 é - c3a9 ê - c3aa ë - c3ab ì - c3ac í - c3ad î - c3ae ï - c3af ð - c3b0 ñ - c3b1 ò - c3b2 ó - c3b3 ô - c3b4 õ - c3b5 ö - c3b6 ÷ - c3b7 ø - c3b8 ù - c3b9 ú - c3ba û - c3bb ü - c3bc ý - c3bd We need to be able to IDENTIFY the CHARACTER to translate and escape.... HOW do you do that when the internal character encoding for PHP can be different. You must first string_recode() to UTF8 Then build the transformation and encoding table to recognize UTF-8 characters. The easiest way is to save Node.php as UTF-8 encoded file and use the characters for the encoding... Also...CANNOT use the &#xxx; format as this is an invalid character. Must use the &html; format so that the XML parsers can take the symbol and transform it to UNICODE...or whatever internal encoding it is set to use. So...this table will be evolving... James Barwick wrote:

In a fit of inspiration this morning...the following code works!!!!

    /**
    * Escape XML entities.
    *
    * @param   string  xml      Text string to escape.
    *
    * @return  string  xml
    * @access  public
    */
    function encodeXmlEntities($xml)
    {
        $encoder = array(
            chr(0) => '�',chr(1) => '',chr(2) => '',chr(3) => '',chr(4) => '',chr(5) => '',chr(6) => '',chr(7) => '',chr(8) => '',chr(9) => '	',
            chr(10) => '
',chr(11) => '',chr(12) => '',chr(13) => '
',chr(14) => '',chr(15) => '',chr(16) => '',chr(17) => '',chr(18) => '',chr(19) => '',
            chr(20) => '',chr(21) => '',chr(22) => '',chr(23) => '',chr(24) => '',chr(25) => '',chr(26) => '',chr(27) => '',chr(28) => '',chr(29) => '',
            chr(30) => '',chr(31) => '',chr(34) => '"',chr(38) => '&',
            chr(60) => '<',chr(62) => '>',
            chr(127) => '',chr(128) => '€',chr(129) => '',
            chr(130) => '‚',chr(131) => 'ƒ',chr(132) => '„',chr(133) => '…',chr(134) => '†',chr(135) => '‡',chr(136) => 'ˆ',chr(137) => '‰',chr(138) => 'Š',chr(139) => '‹',
            chr(140) => 'Œ',chr(141) => '',chr(142) => 'Ž',chr(143) => '',chr(144) => '',chr(145) => '‘',chr(146) => '’',chr(147) => '“',chr(148) => '”',chr(149) => '•',
            chr(150) => '–',chr(151) => '—',chr(152) => '˜',chr(153) => '™',chr(154) => 'š',chr(155) => '›',chr(156) => 'œ',chr(157) => '',chr(158) => 'ž',chr(159) => 'Ÿ',
            chr(160) => ' ',chr(161) => '¡',chr(162) => '¢',chr(163) => '£',chr(164) => '¤',chr(165) => '¥',chr(166) => '¦',chr(167) => '§',chr(168) => '¨',chr(169) => '©',
            chr(170) => 'ª',chr(171) => '«',chr(172) => '¬',chr(173) => '­',chr(174) => '®',chr(175) => '¯',chr(176) => '°',chr(177) => '±',chr(178) => '²',chr(179) => '³',
            chr(180) => '´',chr(181) => 'µ',chr(182) => '¶',chr(183) => '·',chr(184) => '¸',chr(185) => '¹',chr(186) => 'º',chr(187) => '»',chr(188) => '¼',chr(189) => '½',
            chr(190) => '¾',chr(191) => '¿',chr(192) => 'À',chr(193) => 'Á',chr(194) => 'Â',chr(195) => 'Ã',chr(196) => 'Ä',chr(197) => 'Å',chr(198) => 'Æ',chr(199) => 'Ç',
            chr(200) => 'È',chr(201) => 'É',chr(202) => 'Ê',chr(203) => 'Ë',chr(204) => 'Ì',chr(205) => 'Í',chr(206) => 'Î',chr(207) => 'Ï',chr(208) => 'Ð',chr(209) => 'Ñ',
            chr(210) => 'Ò',chr(211) => 'Ó',chr(212) => 'Ô',chr(213) => 'Õ',chr(214) => 'Ö',chr(215) => '×',chr(216) => 'Ø',chr(217) => 'Ù',chr(218) => 'Ú',chr(219) => 'Û',
            chr(220) => 'Ü',chr(221) => 'Ý',chr(222) => 'Þ',chr(223) => 'ß',chr(224) => 'à',chr(225) => 'á',chr(226) => 'â',chr(227) => 'ã',chr(228) => 'ä',chr(229) => 'å',
            chr(230) => 'æ',chr(231) => 'ç',chr(232) => 'è',chr(233) => 'é',chr(234) => 'ê',chr(235) => 'ë',chr(236) => 'ì',chr(237) => 'í',chr(238) => 'î',chr(239) => 'ï',
            chr(240) => 'ð',chr(241) => 'ñ',chr(242) => 'ò',chr(243) => 'ó',chr(244) => 'ô',chr(245) => 'õ',chr(246) => 'ö',chr(247) => '÷',chr(248) => 'ø',chr(249) => 'ù',
            chr(250) => 'ú',chr(251) => 'û',chr(252) => 'ü',chr(253) => 'ý',chr(254) => 'þ',chr(255) => 'ÿ');

        $newstr="";
        for ($i=0; $i<strlen($xml); $i++)
        {
            // substring is character safe with mbstring overloading...
            $c=substr($xml,$i,1);
            if (array_key_exists($c,$encoder))
               $newstr.=$encoder[$c];
            else
               $newstr.=$c;
        }
        return $newstr;
    }

This code should be implemented in XML/Tree/Node.php as soon as possible so that we can all work with UTF8!!!! Any critisism? Feedback? I hope the PEAR developers are reading this...perhaps an email directly to Christian will help.... James James Barwick wrote:

Followup - This works for me: Modification of XML/Tree/Node.php function encodeXmlEntities($xml)

    {
        $chrs      = chr(0).chr(1).chr(2).chr(3).chr(4).chr(5).chr(6).chr(7).chr(8).chr(9).

chr(10).chr(11).chr(12).chr(13).chr(14).chr(15).chr(16).chr(17).chr(18).chr(19). chr(20).chr(21).chr(22).chr(23).chr(24).chr(25).chr(26).chr(27).chr(28).chr(29).

                      chr(30).chr(31).chr(34).chr(38).
                      chr(60).chr(62).
                      chr(127).chr(128).chr(129).

chr(130).chr(131).chr(132).chr(133).chr(134).chr(135).chr(136).chr(137).chr(138).chr(139). chr(140).chr(141).chr(142).chr(143).chr(144).chr(145).chr(146).chr(147).chr(148).chr(149). chr(150).chr(151).chr(152).chr(153).chr(154).chr(155).chr(156).chr(157).chr(158).chr(159). chr(160).chr(161).chr(162).chr(163).chr(164).chr(165).chr(166).chr(167).chr(168).chr(169). chr(170).chr(171).chr(172).chr(173).chr(174).chr(175).chr(176).chr(177).chr(178).chr(179). chr(180).chr(181).chr(182).chr(183).chr(184).chr(185).chr(186).chr(187).chr(188).chr(189). chr(190).chr(191).chr(192).chr(193).chr(194).chr(195).chr(196).chr(197).chr(198).chr(199). chr(200).chr(201).chr(202).chr(203).chr(204).chr(205).chr(206).chr(207).chr(208).chr(209). chr(210).chr(211).chr(212).chr(213).chr(214).chr(215).chr(216).chr(217).chr(218).chr(219). chr(220).chr(221).chr(222).chr(223).chr(224).chr(225).chr(226).chr(227).chr(228).chr(229). chr(230).chr(231).chr(232).chr(233).chr(234).chr(235).chr(236).chr(237).chr(238).chr(239). chr(240).chr(241).chr(242).chr(243).chr(244).chr(245).chr(246).chr(247).chr(248).chr(249). chr(250).chr(251).chr(252).chr(253).chr(254).chr(255); $repchrs=array('�','','','','','','','','','	', '
','','','
','','','','','','', '','','','','','','','','','',

                       '','','"','&',
                       '<','>',
                       '','€','',

'‚','ƒ','„','…','†','‡','ˆ','‰','Š','‹', 'Œ','','Ž','','','‘','’','“','”','•', '–','—','˜','™','š','›','œ','','ž','Ÿ', ' ','¡','¢','£','¤','¥','¦','§','¨','©', 'ª','«','¬','­','®','¯','°','±','²','³', '´','µ','¶','·','¸','¹','º','»','¼','½', '¾','¿','À','Á','Â','Ã','Ä','Å','Æ','Ç', 'È','É','Ê','Ë','Ì','Í','Î','Ï','Ð','Ñ', 'Ò','Ó','Ô','Õ','Ö','×','Ø','Ù','Ú','Û', 'Ü','Ý','Þ','ß','à','á','â','ã','ä','å', 'æ','ç','è','é','ê','ë','ì','í','î','ï', 'ð','ñ','ò','ó','ô','õ','ö','÷','ø','ù', 'ú','û','ü','ý','þ','ÿ');

        $newstr="";
        for ($i=0; $i<strlen($xml); $i++)
        {
           // substring is character safe with mbstring overloading...
           $c=substr($xml,$i,1);
           $x=strpos($chrs,$c);  // this is broken if the double-byte character is a sequence in the above array
                                 // any suggestions?  can I check if only one byte long?  how?
           if ($x===false) $newstr.=$c; else $newstr.=$repchrs[$x];
        }
        return $newstr;
    }

But I need help finishing... Any way to speed this up? Seems like a slow way to do it...but...may be the only way... And...need to fix a the bug noted above... I tried to be clever doing strpos...but...think I may need to change my mind and scan an array myself.. any help?