Re: What "Unicode support" really means

On Wed, 5 Mar 2014, Helmut Tessarek wrote: > On 05.03.14 14:50 , Pierre Joye wrote: > > I do not think UTF-32 is even remotely an option. And both UTF-16 and > > UTF-32 bring us back to the initial issue we had with the previous > > unicode support attempt. UTF-16 is faster to process but the huge of > > conversions (to and from) kill any performance gains, even worst, it > > made php slower. > > UTF-8 is the only implementation that doesn't care about endianess, > since it is just interpreted as a sequence of bytes. On the other hand > a character can have 1, 2, 3 or 4 bytes, which makes length > calculations more time consuming. > > It depends on what operations are mostly done internally, but I assume > the engine itself uses rather basic input/output than substr, length > or other more complex functions, thus UTF-8 might be the most > performant approach. Well, even with UTF-32 (4 bytes for every character) it is not so easy to calculate the number of "characters". Characters can also be combined from multiple Unicode Codepoints, so even Å (U+00C5 LATIN CAPITAL LETTER A WITH RING ABOVE) can also be a combination of U+0041 LATIN CAPITAL LETTER A + U+030A COMBINING RING ABOVE. So things like substr, and wordwrap would have to deal with that too, making the "performance benefits" of UTF-32 largely go away. cheers, Derick