Re: What "Unicode support" really means

On 5 Mar 2014, at 21:25, Derick Rethans <derick@php.net> wrote: > On Wed, 5 Mar 2014, Andrea Faulds wrote: > >> On 5 Mar 2014, at 21:07, Derick Rethans <derick@php.net> wrote: >>> Well, even with UTF-32 (4 bytes for every character) it is not so easy >>> to calculate the number of "characters". Characters can also be combined >>> from multiple Unicode Codepoints, so even Å (U+00C5 LATIN CAPITAL LETTER >>> A WITH RING ABOVE) can also be a combination of U+0041 LATIN CAPITAL >>> LETTER A + U+030A COMBINING RING ABOVE. So things like substr, and >>> wordwrap would have to deal with that too, making the "performance >>> benefits" of UTF-32 largely go away. >> >> Are we sure we want substr() to operate on printed characters and not >> codepoints? > > Yes, as it makes no sense to see this: A, if you really mean Å. There > are different ways to make this better though, for example by converting > to NFC before operations—but that's not always possible for every > combination. I’m unconvinced that dealing with printed characters instead of codepoints is a good idea generally. It’s necessary when doing certain types of text manipulation, but I don’t think we should do this for substr() and general string manipulation. You can’t always necessarily split a group code points into distinct characters, where a code point joins two characters together visually, for example. -- Andrea Faulds http://ajf.me/